There are lots of guides on the various parts of mail service, but there are so many possible combinations that it can be difficult to find an all-in-one roadmap. Basically, you need 3 things - 1) DNS - if you're going to have email addresses, you need an MX record pointing to your mail server. Even if you don't run your own DNS, this is rarely a problem to set up. 2) an MTA - the program that shuttles mail across the Internet and receives mail for you. This is usually Sendmail or Postfix, sometimes qmail. 3) an IMAP daemon - the program that workstations connect to in order to check mail. Often this is Courier-IMAP, though UW and Cyrus are other good candidates. Depending on the combination, you may need Procmail to get the mail from the MTA to the mailboxes in an appropriate format.
Personally, I picked Sendmail for the MTA - while it's not the easiest daemon to use, it's not as hard as people make it out to be, IMO. Also, for better or worse, it's the standard, so I wanted to learn how it works. For the IMAP daemon, I chose Courier - it's widely used, easy to set up, and does what it needs to do. Cyrus has better security, which I didn't need, at the cost of complexity. UW just didn't seem to get good reviews from anyone I talked with. Here's the basic setup.
DNS
1) Enter an MX record for the domain to mail.domain.com.
Sendmail
2) Use FEATURE(`use_cw_file'), and enter mail.domain.com and domain.com in /etc/mail/local-host-names. This tells Sendmail to receive mail for the whole domain.com domain, not just for the specific machine mail.domain.com.
3) Use MASQUERADE_AS(`domain.com'). Basically, the reverse of #2 - tell Sendmail to make all outgoing mail come from domain.com.
4) Use FEATURE(`local_procmail'). Sendmail won't deliver directly to Maildir format boxes, so this tells Sendmail to deliver through procmail, which can handle Maildir boxes just fine.
Courier-IMAP
5) Probably I edited a config file somewhere. There wasn't much to set up as far as the daemon is concerned.
6) Run maildirmake for each user. This sets up the mailbox in the appropriate format.
Procmail
7) Create an /etc/procmailrc that contains
MAILDIR=$HOME/Maildir/
DEFAULT=$MAILDIR
That will deliver messages from Sendmail in the form that Courier expects.
And that's pretty much it. There are some other things that you'll want to set up, like Sendmail access controls, procmail filtering, or SpamAssassin. Maybe Webmail, too. But that's the conceptual backbone.