Well it seems like you have an i845 chipset MB. There are a number of differances between it and more recent intel MB chipsets.
For instance the differance between an i845 and an i865 are:
FSB limitations... the i845 may be limited to 533MHz FSB while the i865 can go 800MHz FSB.
Dual Channel Memory... i865 has it... i845 doesn't.
AGP differances... i845 tops at 4x, i865 tops at 8x.
As the previous poster stated, you might want to double-check the differance between AGP4x and AGP8x and see if it matters... I do NOT speak from experience at saying there is a huge differance, I honestly don't know, I just know the differance between the i845 and i865 as I have both MBs in my house.
I suspect the bigger drag might just be the i845 architecture, FSB speed & Dual Channel support (or rather the lack of), and of course the differance between having a Celeron and a true P4.
Not sure what your budget is, or how much elbow grease you are willing to use, but in case you're interested the i865PE
Asus P4P800SE (refurbished) MB is $49 bucks (a new one is $96). Not saying you need a new MB, just saying if you wanted dual-channel, AGP8X and a faster FSB, it's something to consider.
Not sure if your i845 supports 800MHz FSB or not... some can with a BIOS flash, others are stuck at 533 or below. In either case, if you do decide to get a new P4, you should be looking at one with a FSB of 800MHz or above (once again, check your current MB for compatability). So which model MB do you have?
As the previous poster stated, it's the MB that limits you to AGP4x, not the CPU... but since you have a Celeron, it's not exactly a screamer at game performance either, so it probably wouldn't hurt to get a "real" Pentium 4.
Research, research, research!