But not all. Clients such as Thunderbird do offer the user the ability to set how long after first POP pull (if at all) mail is deleted from the POP server, but it would be up to POP provider--mine does let Thunderbird direct accordingly.I'm not sure what you mean by "set up a host."
If you're using Outlook to connect to a free hosted mail solution (yahoo, hotmail, gmail, whatever) then it's based on the protocol and server settings. Most POP accounts do not keep email on the server after they've been initially downloaded to a client like Outlook. If you're using an exchange-based email solution, all you need to do is put the account settings in Outlook on both computers and it does exactly what you want by default.