More to the point, it's all but impossible in XP. IE is simply an extension of the windows shell. Don't believe me? Type a web address into a Windows Explorer window.
Removing IE simply removes the Internet Explorer executable and some other files. The core IE software -- Explorer -- will still be there because it is necessary for Windows to run properly. (Though I suppose you could theoretically replace it with something else, a la Litestep) Besides, you do need IE to access Windows Update, which is a helluva lot easier than going to Microsoft's download center and finding all the hotfixes you need manually.
Oh yeah, I've said this before and I will continue to say it: (In fact, I think I'll make it part of my sig) Firefox is okay, but it's really just a slightly dumbed-down version of Mozilla. Go with the original; it has more options, integrated email support (lets you open email links in new tabs, for example) and is the original mozilla.org product. Firefox is intended to appeal to people who are used to the blatant idiocy of IE; Mozilla is for people who have no such issues.
And yes, you can install it without the built-in email client if you want.