No, I never heard of any exercises that would stop dizziness. Exercise in general can help alleviate whatever is causing your dizziness, depending on what it is, but it's kind of ridiculous to exercise when dizzy. I can imagine what it would be like to be dizzy every day, and if this is the case you may want to try to look into other ways to manage your health issues OTHER than medication. I am talking about supplements, diet, yoga, tai-chi, etc, etc.
Insulin you can probably never stop. So that has to stay, unless you'll develop iron discipline.
Blood pressure, and cholesterol you CAN manage without meds, unless you are very, very sick, but it's a process, and until you make changes in your diet and add exercise you will have to continue with these meds. I am willing to bet it's these meds that are causing your dizziness.
Arthritis... I don't care how bad it is, but you DON'T need medication every single day. My grandma suffers from horrific pains in her joints 24/7 but she tries to avoid meds unless she feels like she is going to lose her mind from pain. If she can do it, you can too. And definitely, some arthritis meds can and do cause dizziness.
Here's a thought... My grandfather, god rest his soul, was pretty "healthy" until he developed a brain tumor after a car accident. He had diabetes and a host of other ailments, but he managed them so effectively with diet and exercise that he hardly had to worry about any of it.
He'd wake up every morning at 5am and go through his usual exercise routine. He'd do it religiously his whole life, never missing a single day! Even when he was bed bound, after his operation, he continued exercising in bed, flexing his joints, moving his arms and legs dozens of times, until exhaustion. He managed to walk again, in his mid seventies, all by himself after a stroke caused by a neurosurgeon, thanks only to his unyielding discipline.
He mostly ate tons of buckwheat, salads, soups and occasionally meat/fish. Very small portions, about 4 times a day, always at exact same times.
He got to a point where his blood sugar levels remained low, even without meds. He'd only have to take his pills when he'd allow himself to have something he shouldn't. He never suffered from high blood pressure, and never had high triglycerides or cholesterol.
My grandma on the other hand, always eats whatever she wants. So she got major problems with weight, cholesterol and blood pressure. she takes a ton of meds to manage it all, but at least she tries to avoid them when not absolutely necessary.
I know I got sidetracked with my stories and comparisons, but I say all this so that you can see that meds can be avoided or at least greatly reduced.