I agree that this decision is a cost-benefit question that only you can answer. However, if I still had a Canon SLR and could only choose one telephoto lens (money no object,) I'd get the 70-200 F/4 IS. Many regard this as Canon's best zoom lens -- period. Better in IQ at F/4 than the 2.8 IS, with better bokeh, and with a newer IS system that is more effective than it's larger and more expensive sibling. It's also a good deal lighter, and a nice bit smaller, which is a big deal considering a lens of this size. Basically it's the superstar of Canon's telephoto lens line-up (if not their official flagship.) If I could mount one on my Nikon D700 (much like how many prefer to mount the Nikon 14-24 on their Canon 5 or 1Ds,) I would.
Of course, if you need the 2.8 (sport photography at night or in doors comes to mind as primary reason,) get the 2.8. However, if it's just the allure of having a quality L telephoto, the F/4 IS for $1K is the one to get. Note aside from the larger aperture, only the 2.8 models come with tripod collars. If you're looking to do landscape work and use a tripod, you will want to get the optional collar for the F/4 lens -- it's not a weight issue as much as a center of balance thing. It adds to the cost a bit, but it's still a better value than the 2.8 IS if you don't need the extra stop of light.