Depends what you want to do with the gold, and the security you want.
If you purely want to take advantage of any changes in value, as cheaply as possible, and don't want the metal in your hands, then there are financial products that will do what you want (e.g. GLD traded on the American Stock exchange, or PHAU traded on the London stock exchange). These trade as stock, where each share represents about 0.1 oz of gold secured in a secure vault. If you're buying about $2k of gold, then with a discount stock broker, then buying and selling can cost you less than 0.5% each way - by contrast buying gold coins or bullion bars from a dealer may cost you 5% each way. However, although entry/exit fees are low, there is a daily storage fee (charged by gradually eroding the weight of gold in each share - the fee is currently about 0.39% per year) - so investing in $2k of stock, will mean you in effect pay a storage fee of about $0.02 per day.
(The safety of such a scheme has not been truely stress tested, under catastrophic financial meltdown. However, in the case of GLD, there are real gold bars in a maximum security bank vault, and the number of shares trading and bar count/serial numbers are audited daily - so such stock should be pretty safe). That said, beware of similar investment systems which don't have the real metal in stock in a vault - e.g. SLVR doesn't invest in real silver bars, but deals in financial derivatives. A whole bunch of similar funds that traded in derivatives based on metals and other commodities in London, just imploded (stock potentially worthless) as the derivatives all went through the insurance comany AIG, who are on the verge of bankgruptcy.
If you want to buy physical gold to hold, then you are best off buying bullion coins. Coins are easily recognised, easy to verify, and some are alloyed with base metals to make them hard wearing. My personal preference would be South African Krugerrands - these are instantly recognisable world wide, they are 22 carat (although each coin contains exactly 1 oz of pure gold), so they don't dent, bend or scratch like pure gold coins like American Eagles, or Candian Maples. Unlike small bullion bars, coins are often made to standard sizes, which makes them very easy to verify. If the diameter, thickness and weight and color are correct, then there really isn't any way it can be a fake.
Depending on your jurisdiction there may be some special types of coin that you may wish to invest in. E.g. in the UK, any profity made on gold is subject to capital gains tax. However, Gold Britannias are actually legal tender in the UK so any gains made on investments in these are completely tax free.
The current drop in the price of gold has led to some problems in distribution - the coin mints and refiners have sold out, and retail buyers are finding it difficult to buy coins except at huge markups. Conspiracy theories abound, but allegations that the refiners hadn't 'hedged' the price of the bulk gold that they bought at the peak, and don't want to sell for a loss seem to ring true.
Ebay has been quite good for buying and selling and gold coins - with popular coins like krugers trading at or around the wholesale gold spot price, with relatively narrow market spreads (at least as good as through a coin dealer). Again, as always with ebay, it's caveat emptor. However, some people like ebay, because it leaves less of a paper trail. You may be required to present ID when purchasing coins through a dealer, so that a record of the purchase can be sent to the IRS (or other shadow govt agency). Again, you need to check the regulations in your jurisdiction, but as gold is potentially a vehicle for money laundering, there may be significant requirements for trade reporting if buying through registered dealers.