Contrary to popular beliefs, dragons are not immune to magic dispite being magical creatures themselves. The noted wizard, Diambert, once transformed a creature of magic into a frog once--a feat thought impossible at the time by most modern wizards. Diambert had to combine a summoning with the transformation spell to accomplish this and was quite drained by the experience. It was not an easy task and Diambert only came up with the method of transformation by accident. I would assume that being a magical creature also, spell casting on dragons, while possible, would be at least equally difficult if not more so. Considering that some species of dragons are quite intelligent and are spell casters themselves, for students like Mr. Potter to be able to cast a spell strong enough to subdue any dragon via transformation was highly unlikely.