Uh, I haven't played GT3 but if you have more than one choice - it's the transmission gearing. If you have one choice, it's the differential gearing.
You said 1st, 2nd, 3rd so I presume you are changing the ratio of each gear in the transmission. For instance, you could choose a high 1st gear ratio to get you off the line quicker...
In reality *both* make up the final drive ratio of the car, so both play a part. In real life, though - it's usually "easier" to change the gears in your rear end, so you swap out your differential gears. For instance, let's say you have a "race car" and you notice that when you have it floored your top speed is limited by your redline - you are at 6000 rpm and the car is going 160 mph. You could lower your rear end ratio from 4.11 to 3.07 - now your top speed would be 190 mph (I'm making these #s up) at 6000 rpm. The reason you wouldn't just keep lowering the ratio is that eventually your engine won't have enough *power* to make the car go that fast. You'll sometimes see top speeds listed as "rpm limited" which means just that - the car hit redline before the engine ran out of balls.
In some cases you'll see road tests where the drivers were determined to see just how fast a car could go - you could also put bigger rear tires on, which will have the same effect as changing your rear end ratio.