After spending 30+ years in the business, I'm going to take a wild guess here:
Once the price was agreed to & work commenced, did you, by chance, ask for additional Electrical devices, lighting, etc., which necessitates the need for a higher rated panel?
I have never, & I mean never, done a job in which the owner did not add or change items as the job progressed (And always complain when you state that the cost will be higher!).
Just tell us, did you change anything Electrical?
If so, then you are solely responsible for any additional costs to upgrade any Electrical components required to accommodate said additions.
However, if the Contractor gave you a price & you have changed nothing Electrical, then he should be held liable for costs incurred for missing the required upgrade.