In my experience all you can do is let them go away on their own. If you try to mess with them they just get worse and look like horrible pimples.
I use Nivea aftershave balm, by the way, and it's nice. Also I recently started shaving in the shower, and I think that has helped (prevent them, not cure them).