I think you are painting a rather rosy picture for Intel during the early 2000s. Athlon was definitely a win for AMD as it had the clockspeed advantage over P3 (first to 1GHz), as well as slightly higher IPC IIRC. P4 Northwood mainly competed with Athlon XP, not Athlon 64. Athlon 64 competed with P4 Prescott and, whilst not winning every single benchmark, definitely had the edge overall. Athlon X2 as you said was a step above Pentium D.
So AMD was more than competitive during those 4 years, they actually had a performance lead, and more often than not, a price/performance lead as well. Being behind the curve didn't seem to impact Intels prices that much from what I remember.
What rosy picture? What part of what I said is inaccurate? P4 Northwood was better than Athlon XP, at everything. Athlon 64 was largely better than P4 Northwood and Prescott but P4 still held the lead in several benchmarks/applications. Photoshop, video encoding/transcoding, and general multi tasking. There were still very legitimate reasons why one would pick a P4 over A64 up to and including this point. Again, it wasn't until the X2 where AMD had an advantage across the board, including the Pentium D's and that across the board advantage lasted 14 months, far less than the 4 years you're giving AMD credit for. That 14 month span is the only time AMD held the type of advantage over Intel that Intel has held over AMD for the last 12 years.
I have zero reason to be bias here. I owned AMD during that entire time span including the time when P4 Northwood was kicking AMD's ass. I had a Pentium 200 MMX and everything after that up until my Q6600 was powered by AMD. Starting from a Thunderbird 1GHz to a XP 1900+, to a mobile Barton 2600+ to Athlon 64 3800+ (IIRC) and finishing off with an X2 4400+ before finally moving to a Q6600, and even that wasn't until it dropped from $1000 to $300.
I don't think either of us is disagreeing with history here, what I see happening is you think AMD's advantage over Intel was far greater than it really was.
http://www.tomshardware.com/reviews/amd,817-17.html
Pretty comprehensive list of benchmarks there. I counted 16 victories for P4 and 22 for A64. Advantage AMD to be sure, but if you don't consider that trading blows, we'll just have to agree to disagree. 3 of the 4 years of what you consider to be AMD dominance looked just like that
Moving on to the X2, which is where I feel was the only time AMD actually dominated Intel
http://www.tomshardware.com/reviews/amd,817-16.html
Athlon X2: 24
Pentium D: 6
AMD only had 14 months where the performance delta looked like that. This is the type of advantage Intel has been enjoying for 12 years.