Since we have in-order sub-2GHz duallies, now...
An old Atom should be avoided at all costs.
A new Atom (Bay Trail), or new Jaguar-based AMD (Kabini), are fine, but not ideal. Both are just shy of Athlon64 IPC. Neither can, for instance, decode HD h.264 video in software. AMD's has vastly superior IGP, while the new Atoms have slightly faster CPUs (only noticeable in benchmarks), and use much less active power (other components use enough power that idle differences are fairly small). But, you can find both in netbook-sized notebooks, for about $300-350, typically, giving you the size and battery life of an Ultrabook, but without the performance or price tag.
SB and IB Celerons and Pentiums are pretty good, and for mobile use, Haswells pretty awesome, on the cheap. But, they're only awesome in the sense that you're getting a lot for your money. An i3/i5/i7 (only Q-model mobile i7s are quads) will be quite superior. $400-450 gets you into newer Celerons and Pentiums, which blow away the Bay Trail and Kabini processors.
If you want to scour for used and refub units, you can get some of those $400 Windows notebooks, with SB, IB, and Haswell CPUs, at Chromebook prices.