Switching providers can help when the bottleneck is between the ISP and the Internet.
Most providers generally oversubscribe their connection to some extent. i.e. they sell 100mb of DSL access to customers (100 1mb lines or 200 512k lines, etc.) and then they run it all through a 45mb pipe or something similar to the Internet, knowing that everyone won't use their connection at the same time.
If anything happens with that pipe or if all the subscribers want to get on at once, it can affect speeds dramatically.
Switching to a different provider gives you a shot at getting someone that doesn't have their backend oversubscribed as much, though I wouldn't know how you could tell this well without switching and finding out the hard way.
If it's still hard to grasp, envision it as having a LAN party. You have 24 people over and plugged into a switch. They all have a 100mb link and can all talk to each other that fast, and can talk to your router that fast, but it only has a connection to the Internet at 256k so that's all they can get. DSL is the same sort of principal, you can connect to your provider at the full speed of your line, but if they don't have a big enough pipe (or if it's too saturated) you can only get what spare crumbs of bandwidth are available.