What makes a study valid? Come on!

[TridenT]

well? Anyone?

According to some people out there, a study that is: Peer reviewed = bullshit. Any funding from any source ever = bullshit.

 

[Jaepheth]

It's valid when it confirms your preconceptions. Duh.

 

[waffleironhead]

there are many ways to insert bias into a study, but when it is published, the wider the audience, the more scrutiny it endures. This allows the community to weigh in on the validity and try to recreate the findings.

Just because it is peer reviewed does not mean it is 100% valid.

As far as the funding goes, you can separate the data findings from the conclusions. Look at the information and form your own.

 

[Matthiasa]

As already stated, reproducibility is an important measure of if it is valid or not.

 

[Mark R]

Appropriate study hypothesis; the study has to answer a sensible question

Appropriate study design, with design factors to avoid bias and confounding

Adequate numbers (with appropriate power analysis prior to study commencing); a small study is unlikely to to give a positive result, because the expected noise is so great

Appropriate population under study (i.e. large enough to be relevant and generalisable); a study result from a study looking at a specific group cannot be extrapolated to the general population

Appropriate analysis (i.e. appropriate use of statistics); appropriate robust methods and meaningful measures need to be calculated (e.g. the relative risk is much loved by pharma companies, but is difficult to interpret in context)

Peer review is an important part of the scientific process, it's certainly not bullshit. However, peer review often means that only one or 2 people have read the study to make sure there are no idiotic mistakes. It catches most bloopers, but certainly not all.

 

[SKORPI0]

Large number of samples and results can be replicated later on. :hmm:

 

[dullard]

Mark R has a good start. But I think he under-emphasized the study design part. Nothing can replace appropriate design of the study. Were the samples selected both randomly AND consecutively or were they cherry picked? What was the data that WASN'T published (and can people access that additional data)? Is there proper record keeping to back up the results? Is that record keeping traceable (what was done/changed and when and by whom)? Does the study logically address the hypothesis? Did the study focus on results or was it focused on opinions about the results? Etc.

No one factor proves a study is valid. But, lack of one factor can certainly indicate possible bias (intentional or not) or fraud.

 

[JulesMaximus]

well? Anyone?

According to some people out there, a study that is: Peer reviewed = bullshit. Any funding from any source ever = bullshit.

Those people are dumbfucks and should be ignored.

 

[waffleironhead]

Didnt school just start back up again?

 

[ConstipatedVigilante]

A low p-value.

 

[AyashiKaibutsu]

As already stated, reproducibility is an important measure of if it is valid or not.

This. If one group shows something happens and then a bunch of other groups get the same results it's a good indication.

 

[Cogman]

Those people are dumbfucks and should be ignored.

Andrew Wakefield's Anti-vax study was peer reviewed. There are peer reviewed anti-global warming papers etc. Heck, I've seen a peer reviewed paper on the effects of lead poisoning that had some REALLY questionable data on the effects of lead and children's behavior. (some of the values for statistical significance looked to be picked to draw a coordination where none existed, the actual data looked like buck shot.)

The peer review process is not perfect and shouldn't be treated as such. While i adds credibility, it does not add infallibility, that is achieved through repeated tests.

 

[JulesMaximus]

Andrew Wakefield's Anti-vax study was peer reviewed. There are peer reviewed anti-global warming papers etc. Heck, I've seen a peer reviewed paper on the effects of lead poisoning that had some REALLY questionable data on the effects of lead and children's behavior. (some of the values for statistical significance looked to be picked to draw a coordination where none existed, the actual data looked like buck shot.)

The peer review process is not perfect and shouldn't be treated as such. While i adds credibility, it does not add infallibility, that is achieved through repeated tests.

No, of course not. But there are people who seem to disbelieve/dispute EVERYTHING related to scientific discovery and those people are complete and utter tards.

 

[her209]

Religious texts are peer reviewed and therefore valid.

:trollface:

 

[Cogman]

No, of course not. But there are people who seem to disbelieve/dispute EVERYTHING related to scientific discovery and those people are complete and utter tards.

yeah I know the type. Every so often while surfing through radio stations I'll pick a few words from a christian radio broadcast... Man, I HATE how those guys treat science. They treat the very word as if it was a curse word "but SCIENTISTS say you can't trust this. We know better then those stupid scientists".

Being skeptic is one thing, being totally dismissive of whatever doesn't agree with you is another. Those people that are dismissive of science should stop breeding.

 

[TridenT]

I was talking about vitamins with a good friend and they were harping about the two things I mentioned. I was mentioning that vitamins usually don't do anything because studies have shown that they don't. They referred to those two things. I said, "so every study is bs then?" Conversation stopped there basically. They didn't major in the sciences.

 

[Cogman]

I was talking about vitamins with a good friend and they were harping about the two things I mentioned. I was mentioning that vitamins usually don't do anything because studies have shown that they don't. They referred to those two things. I said, "so every study is bs then?" Conversation stopped there basically. They didn't major in the sciences.

Ummm, What? Most vitamins have some REAL health benefits (and drawbacks when overdone) that are well known. It is the random herbal supplements that you have to be wary of.

 

[waffleironhead]

I was talking about vitamins with a good friend and they were harping about the two things I mentioned. I was mentioning that vitamins usually don't do anything because studies have shown that they don't. They referred to those two things. I said, "so every study is bs then?" Conversation stopped there basically. They didn't major in the sciences.

So you are both wrong. Sounds about right.

 

[TridenT]

Ummm, What? Most vitamins have some REAL health benefits (and drawbacks when overdone) that are well known. It is the random herbal supplements that you have to be wary of.

You're saying that taking a multivitamin actually does something?
http://www.wellnessuncovered.com/joomla/images/stories/centrum_multivits.jpg That that does something significant?

Last I checked, it doesn't. There are things like taking Vitamin D which apparently does something, but most don't.

 

[Cogman]

You're saying that taking a multivitamin actually does something?
http://www.wellnessuncovered.com/joomla/images/stories/centrum_multivits.jpg That that does something significant?

Last I checked, it doesn't. There are things like taking Vitamin D which apparently does something, but most don't.

Multivitamin/folic acid supplementation in early pregnancy reduces the prevalence of neural tube defects
http://jama.ama-assn.org/content/262/20/2847.short

A randomized trial of multivitamin supplements and HIV disease progression and mortality (decreases HIV)
http://www.nejm.org/doi/full/10.1056/NEJMoa040541

Small study, but rate of reported infection was decrease by multivitamin
http://www.annals.org/content/138/5/365.short

Randomised controlled trial of vitamin E in patients with coronary disease (decreases chances of heart disease)
http://www.sciencedirect.com/science/article/pii/S0140673696908661

Most (maybe not all) of the ingredients in, say, centrum you can freely google and find that, yes, they do have proven health benefits.

So, yes, I would say that taking a multivitamin DOES actually do something. Science tends to agree with me.

 

[TridenT]

Multivitamin/folic acid supplementation in early pregnancy reduces the prevalence of neural tube defects
http://jama.ama-assn.org/content/262/20/2847.short

A randomized trial of multivitamin supplements and HIV disease progression and mortality (decreases HIV)
http://www.nejm.org/doi/full/10.1056/NEJMoa040541

Small study, but rate of reported infection was decrease by multivitamin
http://www.annals.org/content/138/5/365.short

Randomised controlled trial of vitamin E in patients with coronary disease (decreases chances of heart disease)
http://www.sciencedirect.com/science/article/pii/S0140673696908661

Most (maybe not all) of the ingredients in, say, centrum you can freely google and find that, yes, they do have proven health benefits.

So, yes, I would say that taking a multivitamin DOES actually do something. Science tends to agree with me.

From what I read, in the general public it doesn't do much at all. Maybe it's different for small subsets of the population.

 

[SithSolo1]

When its not paid for by anyone who has even a remote chance of benefiting from the outcome.

 

[Cogman]

From what I read, in the general public it doesn't do much at all. Maybe it's different for small subsets of the population.

Care to actually link to "what I read"? I've already provided you with several peer reviewed papers which pretty much claim you are wrong. Do you need more?

http://scholar.google.com/scholar?hl=en&q=daily+vitamin&btnG=Search&as_sdt=0,44&as_ylo=&as_vis=0

Go nuts with ACTUAL papers. Most of them talk about benefits.

Many more can be found by breaking apart the ingredients of a daily multivitamin .

 

[SMOGZINN]

Peer review is an important part of the scientific process, but it is only one part. An important part of the question is who is doing the reviewing. There are peer reviewed magazines of homeopathic medicine.

 

[TridenT]

Care to actually link to "what I read"? I've already provided you with several peer reviewed papers which pretty much claim you are wrong. Do you need more?

http://scholar.google.com/scholar?hl=en&q=daily+vitamin&btnG=Search&as_sdt=0%2C44&as_ylo=&as_vis=0

Go nuts with ACTUAL papers. Most of them talk about benefits.

Many more can be found by breaking apart the ingredients of a daily multivitamin .

General public, again... those are subsets of people. One is people with HIV in a remote part of the world. I'm talking general public in the USA who are healthy(-ish) and regular individuals. I don't think taking multivitamins is a good substitute to eating foods with the vitamins. (Which is how a lot of people view it)