Third party is usually the crap you don't want; advertisers and trackers. They can also pull in information from embedded widgets. Let's use an embedded Soundcloud track as an example. You visit the website noise.com, and it has a Soundcloud track embedded on their site, and you listen to it. Then you go to Soundcloud proper, look up the track, and see that you've already listened to it even though you've never been to Soundcloud before. They knew you listened to the track placed on noise.com because of a third party cookie.
First party cookies are a little more useful. They change the color of links you've visited, keep login information, and generally keep track of your preferences on a site. For example, if you deleted anandtech's cookie, and closed the page, when you come back, it'll be like the first time you've looked at the site. You'll be logged out, and you can't visually tell what threads you've viewed.
On Firefox, I use a cookie manager, and delete most cookies on tab close except for a view I have whitelisted.
Facebook
Twitter