In Chuunibyou's case, the extraordinarily large weapons are used to emphasize the fact that this is in Rikku's imagination, and they are actually fighting with a ladle and an umbrella, which is much less extravagant.
Having said that, the 'small girl, big weapon' trope is common enough in anime. Take Mami's Trio Finale in Puella Magi Madoka Magica for example, or Black Rock Shooter's girls which all have gigantic weapons:
There are lots of instances of these types of battles, but I wouldn't restrict this trope to anime, with instances of 'Big F**king Guns' going back to games like Doom
Along with the '100T anvil' trope, These types of tropes are used to emphasize a scene. In this case, making battle scenes more explosive,action-packed and gripping.