I built a portable system for camping, though it's probably bigger than you want, and it's mostly all prebuilt parts wired together in a package with a battery. There's also a micro controller that displays the battery voltage and has a few other features like low voltage shut down. I can also serial into it with a laptop to change thresholds and calibrate the display and stuff. Why? Because I'm a nerd.
This is before I was fully done painting as I only added the top after. The bar in the middle used to be the handle but I didn't like the idea of it being fully open. It's still not watertight, but at least if a bit of drizzle got on it, it would probably be ok.
foldable 60w panel: (2 30w in parallel)
For something smaller you'd probably need something that can boost the voltage from a few cells (about 0.5v each) to a nice clean 5 volts. A typical charge controller assumes there is a battery and it will vary the voltage to charge it at a specific current rate. You don't want to do this with a phone as it has a charger built into it, it wants a nice clean 5 volts no more no less. (well there's always some tolerance).
Easiest thing to do is probably something like this:
Solar cells (let's just say 1 volt typical output) -> MPPT charge controller that can handle very low voltages and then output a decent voltage range -> 5 volt DC-DC converter that will take the controller's voltage and smooth it out to 5 volts.
You may be able to simply find a 5v DC-DC chip that also can do MPPT. If this is strictly for a phone no need for a separate battery so the only purpose the charge controller really serves is MPPT function. If you don't care about efficiency you could probably just have the cells go straight to a DC-DC boost converter but your mileage may vary.
Very simple but possibly risky is to aim for close to 5v VMP rating of cells, and connect straight to the phone. I would not personally recommend this route unless someone who knows more about electronics than me can say it's safe.