From the way it sounds, you can go two ways with the setup. A pre-amplifier or a distribution amp. Since you are doing a simple, residential set-up, using an antenna mounted pre-amplifier will give you a cleaner signal and probably provide enough kick to get you through the distribution box without anymore amplification, but then you risk signal overload. You need to test what kind of signal you are getting from the most simple setup possible, i.e.
antenna---->single RG6 coax---->television/receiver. If the signal is already really good, then a pre-amplifier will just mess things up for you and it will be best to get a distibution amp. If your test setup is resulting in a poor signal, then a pre-amplifier is absolutely the answer. Your inline amplifier is basically a cheap pre-amp and probably adds an excessive amount of noise to the signal, so if you go the pre-amp route, ditch the inline amplifier completely.
A distribution amp is designed to take a good signal and amplify it enough to go through splitters, distribution boxes, long cable runs, etc. and provide a clear picture on the other end. If the signal reaching the distribution amp is poor, then it just amplifies the poor signal and sends a strengthened, poor signal out to the tvs, receivers, etc.
You basically have four different options best to worst (without getting a new antenna and running new coax):
antenna---->pre-amp---->distribution box---->televisions,receivers, etc.
antenna---->distribution amp---->distribution box---->televisions,receivers, etc.
antenna---->pre-amp---->distribution amp---->distribution box---->televisions, receivers, etc.
antenna---->in-line amp---->distribution amp---->distribution box---->televisions, receivers, etc.
If you are getting a relatively good signal with your simple test setup, but the cable runs after the distribution box are long or of poor quality:
antenna---->distribution amp---->distribution box---->televisions,receivers, etc.
antenna---->in-line amp---->distribution amp---->distribution box---->televisions, receivers, etc.
antenna---->pre-amp---->distribution box---->televisions,receivers, etc.
antenna---->pre-amp---->distribution amp---->distribution box---->televisions, receivers, etc.
Try
www.dennysantennaservice.com. The site is full of info, the guy will answer your e-mails in a day and the store ships relatively fast with a money-back guarantee. Avoiding the typical Radio Shack/Wal-Mart stuff will help out immensely. Get some signal terminators to cap unused hook-ups, too. Let me know if this helps out.