I think you're right about there not being enough good information about Vitamin D. The problem is that it's just too cheap. There is not enough money to be made from it for anyone to do good research on something like this, even though it is really important. Measuring the quantity that you are taking in is fairly difficult unless you are indoors at all times.
The problem I have with the government recommendations is that they are really just the minimums you need to avoid getting rickets. Apparently, much more than that is used by your body on a daily basis according to some articles I've seen around the web.
It seems like the only reasonably reliable method of knowing how much you need is a blood test. Even that is not all that accurate. That's why I've chosen to go with 2000 to 2800 range. It's above the government minimum rating, but should be well below toxic levels.
Much of the basic research on vitamins is actually done under Federal grants at large state land grant universities. So it isn't so much a money thing, because a lot of $$$ as it is that vitamin D does SO many things in the body and as you mentioned, nutrition research is very, very tricky.
2000 IU, or 50 mcg is the current upper intake level, in other words, the point at which the risk of overdose is high.
The NIH does note that:
[q]Several nutrition scientists recently challenged these ULs, first published in 1997 [89]. They point to newer clinical trials conducted in healthy adults and conclude that the data support a UL as high as 10,000 IU/day. Although vitamin D supplements above recommended levels given in clinical trials have not shown harm, most trials were not adequately designed to assess harm [5]. Evidence is not sufficient to determine the potential risks of excess vitamin D in infants, children, and women of reproductive age.
As noted earlier, the FNB is currently reviewing data to determine whether updates to the DRIs (including the ULs) for vitamin D are appropriate [4].[/q]
It's also worthwhile to note that much of the "long-term" effects that vitamin D might have on improving health are speculative and lack concrete evidenciary support. For example, finding a role of vitamin D in preventing cancer development does NOT necessarily mean that taking supplements will reduce risk. It is speculation on the part of the researcher. You need trials to establish that and that research might take years, if not decades when dealing with something like cancer.
The blood test will only tell you how much vitamin D is circulating in the bloodstream and will tell you if you are deficient (or in turn, if you have TOO much, which is a bad thing as well). It won't tell you anything about turnover.