<< Yes, I understand the physical curvature of the mirror affects the reflected image, but the question is WHY is it not so on the driver side, but only on the passenger side. >>
Because the mirror on the passanger side is farther away, so the objects are magnified. If they weren't, they'd be too small for the driver to readily identify.