AGP or Advanced Graphics Port is a brown slot under your motherboard northbridge.
PCI or Peripheral Component Interconnect slots are white and are located at the bottom left of the motherboard under the AGP slot.
AGP runs at 66MHz while PCI runs at 33MHz. PCI cards share their bandwidth with all the peripherals of your system such as the HDD, Floppy, sound card, USB ports, basically anything that is connected to the motherboard. AGP runs on a separate bus which is quicker and provides some extra error checking on the data that it sends and gets to and from memory.
So, basically, an AGP bus is a quicker PCI bus with unshared bandwidth and better error checking.
AGP wasn't created because there was a bottleneck, it was created for storing textures faster and better than PCI. The need for storing textures in RAM was necessary in the chance that the game textures outweighed the video memory buffer, in which major slowdown would occur. Technically AGP isn't necessary with video cards that support large memory buffer, but it is good to have the potential or safety. As you can see
here, there was very little performance differance if any between the PCI and AGP version of the Voodoo5 5500. Now, games are bigger and might require the VPU to send bigger amounts of data over the PCI bus and AGP could be at an advantage here.
I still recommend AGP until PCI Express comes out next year.