i have my sides off right now. only because the weather is getting more humid. in the winter, i leave the sides on.
as for dust, i get a lot of dust build up, from my one exhaust fan.
however, the amount of dust depends on what your room is like. my room is pretty dusty, but yours may not be