I'm an MCSE and to network consulting/tech support for a national ISP. Here's the problems I've observed with WIndows ME:
1) It adds a new PCI->USB driver that is NOT compatible with a lot of USB hardware (especially items from HP and Canon). In order to get them to work, you may have to go back to a two year old version of the USB bus drivers, which will pretty much screw up any special power saving features, etc.
2) Drivers have to be totally rewritten for ME for some reason. Win98 drivers will work, but they may cause occasional blue screens (especially video and video capture devices). If your hardware drivers aren't specifically for ME and no new drivers are available, you may just have to live with the crashes.
3) Networking (especially with a peer-peer network) is terrible to set up. For some reason, it can take up to 15 minutes (!) for shared resources to appear in "My Network Places". On an NT network, that's normal, but that's a bit rediculous. This can eat up a whole day of troubleshooting for people that don't know this
4) It adds "shortcuts" to shared folders in My Network Places. Problem is, it never refreshes these shortcuts. Doubleclick on a shortcut that isn't available, and your machine may hang for up to 3 minutes as it tries to find the missing resource. Blech....
There are a lot of others. People like that "registry restore" feature, but there are a dozen shareware programs that do the same thing. For my money, Win98 SE is the most stable and compatible home user OS available.