OK, I don't want to sound like a snob here, but you're asking about 4-500 years of music. How can we possibly make a recommendation off of that?
I guess to make some very vague, very broad suggestions, I can say this much:
The compilations idea is probably a good one. But, don't just get one with all the stuff you already know (Pachabel's Canon, the last movement of Beethoven's 5th Symphony, etc). Try to get a variety. Get some stuff from all the periods if you can, at least Baroque through Romantic. But give the Renaissance and Contemporary stuff a try to. You never know what will strike your fancy.
Many people have suggested Mendelssohn. He's probably one of the most "ear-friendly" composers, in that most of his music is based off of just your basic major and minor chords. It's not very complex and easy to digest, and very beautiful. Check out his "Songs Without Words" for piano, or the E Minor Violin Concerto. Don't forget the Hebrides Overture (one of my favorite of his pieces).
If you enjoy the Mendelssohn, also look for Franz Schubert. He's very similar to Mendelssohn in a lot of ways: both were German/Austrian, and both were on the brink of that transition from the classical to the Romantic era. Many will argue that Schubert leans slightly more toward classical than Mendelssohn, however. His music is again, very ear-friendly and beautiful, but they tend to get a bit more complex if you were to analyze their construction.
You probably will not enjoy the contemporary composers (Prokofiev, Mahler, Scriabin, Stravinsky, Glass, etc) stuff at first, because they are often abrasive and harsh. It takes a lot of intelligent listening (i.e. doing nothing else but listening and perhaps following along in the written out score) to understand what the composer was doing. It's fascinating work, but not always very "pretty" per se.
If you have any questions about particular composers or recordings or anything, just give me a PM, or ask here or something. Hope this helped, although, again, it's difficult to just recommend music from such a monstrous catalogue.