What do you plan on doing after your undergraduate degree?
Job (industry? government?)? Masters? Ph.D? MBA?
Chem E is the hardest engineering major, followed by EE. I did my undergrad in EE, then part of a masters in EE before I switched to IE (Imaginary Engineering).
Do you have more of an interest in heaviliy technical stuff and math? Or do you like engineering from the systems perspective?
You made a good choice not to go CS for sure. I've had 3 different jobs now that have been mostly programming, but I have gotten promoted to managerial roles as a result of having an EE background. Generally those hiring believe that engineers have a better understanding of the overal picture and goal than those of CS, math, and other science disciplines.
It really all depends on what you plan on doing after you graduate, and what type of role you want. Do you want to be the technical guy your whole life, or do you want to get into management as quickly as possible?
If you think you might go on to a Ph.D, you should do EE for sure since it will give you the most math out of all of them, and prepare you better.