There is a submachine gun on the roof next to Sweet's house. There is body armor under the bridge supports in the canal just east of your house.
There is a betting shop just west of that big park (I forget the name) in the Baller's territory. You can save your game, then go bet the long odds on a horse. If you win, save. If you lose, then reload and retry until you win a decent pile of $$$.
Frankly, I'd just stick to the missions and advance the plot. Early on, there's not much peripheral stuff to do, since property, territory, items, and access to the other two cities are restricted.