Specialized or experienced writers, and the original manga writers can all work the manga into a plot for an episode, but why pay them to write a new scene where nothing interesting happens.
Generally 'filler' is made by cheap writers. As mentioned in the comments, there are often conflict because the filler writers are paid less and so put less effort into background work. Unlike lead writers, filler writers wont always be provided with reference material for things to include or avoid.
Basically, if it is irrelevant to the season's story line, has no effect on character development, you have someone who works for less write it. Although, filler scenes with no dialogue and no interaction, like someone taking an uneventful walk down a straight road requires no script at all.
As for who decides, whoever decides on the main plot will probably view the filler storyboard before it gets scripted.