Atom (
dual-core w/ HT) or Zacate would be fine, but an Atom would likely feel a bit slow, given how complex Open Atrium appears to be. Give it 2GB+ of RAM, and learn to get Drupal caching well enough (OotB, Drupal is set up for devel and debug, which is why every other CMS say's they're so much faster

), and go. Such a system should be handle 20
simultaneous users, with a little PHP and MySQL tuning (bigger MySQL buffers, APC for PHP, >128MB shared memory).
Now, I don't know whether Brazos or Atom are a better value, as I can't find any LAMP, or even straight up PHP tests w/ it, but there's no way on Earth it will not be enough. If you're just worried about it getting too bogged down, don't--there's a learning curve, but Apache, MySQL, and PHP will probably just need some tweaking, so as not to eat up too much RAM and disk. If you are also concerned about service time for each request, then I'd say Atom < Zacate < Athlon II < Phenom II, but I can't confirm Zacate's actual performance.
I think Zacate would do well
(LAMP has been a strong area for AMD since the Athlon), but I'm not sure if it's what you really want, based on current pricing. An Atom 330 or 510 barebones might be your best bang/buck, and an Athlon II box the most versatile on the cheap. If you go Athlon II, and want it to be more efficient, make sure the board has undervolting options, as not all do. The problem, cost-wise, is simply that you are paying for a nice IGP that you have no use for, with Zacate.
If electricity cost isn't much of an issue, a Biostar A880G+ and 3GHz Athlon II are only $101. You can beat that price with Atom, but you'll have far more performance w/ the Athlon II, and you can't get a Zacate board that cheap. Not sure what, if any, undervolting options that board has, but the price is right. The advantage to the Atom is that you can get a barebones, add a drive and RAM, and be done for $200-230, where the Athlon II build will be closer to $300 (case, PSU, RAM, HDD, mobo, CPU--optical drive can be borrowed for the OS install).
Maybe a lot of hd thrashing i you have 4GB or something.
4 will be plenty for that software--probably even overkill, but RAM is cheap--with that many users, as long as APC is set up, and MySQL's buffers/caches are beefed up a little. HDD thrashing, with 20 users, would mean a PEBKAC on the admin's part.
So, a few other pointers:
1. Set up Drupal's internal OotB caching.
2. Set up APC for PHP, and make the ttl for entries very long (like a day or longer). Get the monitoring script set up where you can check it, and do so once the server is in use. Most likely, you won't need to mess with it, but if it stays full, gets tons of fragmentation, etc., you might need to adjust some settings (going over 128MB requires changing a Linux kernel setting, IIRC).
3. Check MySQL's performance metrics, and make key buffers and caches larger. There are good 'net resources with more detailed, Drupal-specific, info.
All of the above will help immensely, even with small numbers of users. Drupal is big and bloated, and so caching is the answer to 90% of its performance problems.