If you scan and then have the OCR software to convert to text, then I think the PDF will be small. If you scan and are placing image files into the PDF, then it's going to be huge...
Also, having a PDF that is readable via a computer is fine, but what happens when you print those low-quality docs? I suspect they will be pretty poor quality.
If I was doing this, I'd scan with as high of quality as I could stand to wait for. Then use graphics editing software to downgrade it later until I found a size that would work for me.