If we could power a calculator with a tiny 1 inch solar panel in the 90's, you would think that you could charge a smartphone with an 8x8 inch fold out solar panel now.
The oldest calculators using solar also only had a very small & simple lcd display, that in the dark, you couldn't see. As a reference, the TI83 (not a solar calculator) had like a 6Mhz CPU.
With phones, we're talking about very large color displays (OLED) that bring their own light output, CPUs in the GHz of frequencies, bluetooth, wifi, cell modem, some few GBs of RAM, gps, gyroscopes, accelerometer, magnetometer, light sensor, pressure sensor, cameras, speakers, charge controller, temp sensors, etc.
It's a drastic difference in power requirement comparing a phone of today against a solar powered calculator of the 90s. A lot of these phones have a 4 to 4.8 amp hour battery and have considerable power save features to make the most out of that battery.
You're looking at about 15 watts per square foot of solar panel (12" x 12") assuming you have overhead sun and no clouds and efficient panels, which isn't much different than a lot of 2amp chargers can manage. If you want something half that size (12" x 6"), expect close to half the wattage.