Good article (systemic suppression of skepticism in political science)

[dank69]

How did the system work?

Peer review published the article.

ONE person. ONE thought something was funny and dug into it. And at every step he was told to look the other way by peers.

So the system didn't work. The system failed.

But the members of the Church of peer review will dismiss facts that challenge them. Kind of like members of scientology.

Sort of like how you are dismissing the fact that the peer review process exposed the fraud?

 

[dawheat]

How did the system work?

Peer review published the article.

ONE person. ONE thought something was funny and dug into it. And at every step he was told to look the other way by peers.

So the system didn't work. The system failed.

But the members of the Church of peer review will dismiss facts that challenge them. Kind of like members of scientology.

So in contrast to a system that worked but isn't perfect, you want to replace it with...faith and the common man's belief? That's the way to the future.

 

[shira]

Now things have flipped: Really? Nothing? Did you read the article? It's what the entire article was about. All of Brookman's associates in the field (except his close friend) discouraged him from looking into it at every turn. Even an anonymous political science web forum suppressed his skepticism. The reasonings and motivations were laid out clearly. At several points, Brookman himself, despite his suspicions, did not want to look into it any further. The article pointed out a problem that should be fixed -- not denied or ignored.

Are you equating climate science with political science?

Something you're overlooking is that the "gold-standard in science" is not peer review followed by a methodology to expose fraudulent research. The gold standard is peer review followed by relentless attempts by other scientists in the field to reproduce research results. Because - overwhelmingly - invalid research results are the result of flaws in the studies, not fraud. Thus, whether or not LaCour's fraud was ever exposed, his results - which strongly contradicted a mountain of previous research - would have been tested by other scientist attempting to replicate his methodology and results. And those additional studies would have repudiated LaCour's results, which would then have - eventually - been rejected.

Completely consistent with what I just wrote, Broockman began to question LaCour's study in the first place because he wanted to repeat it, yet he found that the cost of performing a study that entailed obtaining 10,000 subject responses was not only much more expensive than anyone in the field could afforded, but the very firm that LaCour claimed to have used to conduct the polling said they were not equipped to carry out a study of that magnitude. And then later, when Broockman (and Kalla) were trying an unrelated study of their own, and were getting an initial response rate of only 1% (compared with LaCour's claimed response rate of 12%), they tried to contact LaCour's claimed polling firm to find out the "secret" for obtaining such a high response rate, and that's when the fraud unraveled.

I agree that it would be good for science to have a "safe path" for suspected fraudulent studies to be challenged. But my point is that exposing fraud in science is for the most part irrelevant, because fraudulent scientific results will eventually be rejected (assuming they're at odds with actual "truth"), along with all other invalid results, when scientists follow the "normal" scientific approach of attempting to replicate the results of previous studies.

 

[unokitty]

How Science is Broken

...researchers often can't validate findings when they go back and run experiments again. Just last month, a team of researchers published the findings of a project to replicate 100 of psychology's biggest experiments. They were only able to replicate 39 ...

...researchers at Amgen were unable to reproduce 89 percent of landmark cancer research findings for potential drug targets...

When a press release confused correlation with causation, 81 percent of related news articles would, too. And when press releases made unwarranted inferences about animal studies, 86 percent of the journalistic coverage did, too...

Because of these now well-known problems, it's not unusual to hear statements like those from The Lancet editor Richard Horton that "Much of the scientific literature, perhaps half, may simply be untrue." He continued: "Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness."

Long before it was mainstream to criticize science, Sheila Jasanoff, a Harvard professor, was arguing that science — and scientific facts — are socially constructed, shaped more by power, politics, and culture (the "prevailing paradigm") than by societal need or the pursuit of truth...

Just something to think about the next time a politician wants to tell you that "the science is settled."

After all, the science is clear. Everything we eat both causes and prevents cancer...

Uno

 

[shira]

How Science is Broken
Just something to think about the next time a politician wants to tell you that "the science is settled."

After all, the science is clear. Everything we eat both causes and prevents cancer...

Uno

How does this show that science is broken? It's extremely likely that the same food or substance can both prevent and cause cancers, in the same way that same behavior can both be healthy and dangerous.

For example, regular jogging is undoubtedly healthy. But there are also clearly health risks (damage to joints and connective tissue, accidents, damage to lungs caused by deeply inhaling pollutants, etc.). Would you say that science is broken if one study showed that jogging reduced cardiovascular disease while another study showed that jogging increased respiratory problems?

 

[Moonbeam]

How does this show that science is broken? It's extremely likely that the same food or substance can both prevent and cause cancers, in the same way that same behavior can both be healthy and dangerous.

For example, regular jogging is undoubtedly healthy. But there are also clearly health risks (damage to joints and connective tissue, accidents, damage to lungs caused by deeply inhaling pollutants, etc.). Would you say that science is broken if one study showed that jogging reduced cardiovascular disease while another study showed that jogging increased respiratory problems?

My bet is that he wants to think science is broken and doesn't want anything to get in his way.

 

[senseamp]

I don't see systemic suppression of skepticism in that article.
When the misgivings were brought to attention, the system did not suppress the skeptics, but rooted out the bad apples.

 

[dank69]

How Science is Broken

...researchers often can't validate findings when they go back and run experiments again. Just last month, a team of researchers published the findings of a project to replicate 100 of psychology's biggest experiments. They were only able to replicate 39 ...

...researchers at Amgen were unable to reproduce 89 percent of landmark cancer research findings for potential drug targets...

When a press release confused correlation with causation, 81 percent of related news articles would, too. And when press releases made unwarranted inferences about animal studies, 86 percent of the journalistic coverage did, too...

Because of these now well-known problems, it's not unusual to hear statements like those from The Lancet editor Richard Horton that "Much of the scientific literature, perhaps half, may simply be untrue." He continued: "Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness."

Long before it was mainstream to criticize science, Sheila Jasanoff, a Harvard professor, was arguing that science — and scientific facts — are socially constructed, shaped more by power, politics, and culture (the "prevailing paradigm") than by societal need or the pursuit of truth...

Just something to think about the next time a politician wants to tell you that "the science is settled."

After all, the science is clear. Everything we eat both causes and prevents cancer...

Uno

Giant bucket of fail. In addition to shira's comments, you are taking the word of Richard Horton, editor-in-chief of The Lancet regarding the state of science:

Autism and vaccine controversy (1998)
The Lancet was criticized after it published a paper in 1998 in which the authors suggested a link between the MMR vaccine and autism. In February 2004, The Lancet published a partial retraction of the paper. The editor-in-chief, Richard Horton, went on the record to say the paper had "fatal conflicts of interest" because the study's lead author, Andrew Wakefield, had a serious conflict of interest that he had not declared to The Lancet. The journal completely retracted the paper on 2 February 2010, after Wakefield was found to have acted unethically in conducting the research.

The Lancet '​s six editors, including the editor-in-chief, were also criticized in 2011 because they had "covered up" the "Wakefield concocted fear of MMR" with an "avalanche of denials" in 2004.

 

[OverVolt]

How does this show that science is broken? It's extremely likely that the same food or substance can both prevent and cause cancers, in the same way that same behavior can both be healthy and dangerous.

For example, regular jogging is undoubtedly healthy. But there are also clearly health risks (damage to joints and connective tissue, accidents, damage to lungs caused by deeply inhaling pollutants, etc.). Would you say that science is broken if one study showed that jogging reduced cardiovascular disease while another study showed that jogging increased respiratory problems?

Guess what, this is one giant wild ass guess that you feel entitled to spew out.

In many cases the science isn't broken but peoples interpretation of it is. For starters most food studies are fundamentally flawed because of they way they stack correlation upon correlation without further investigating enough to prove causality like they have done with smoking.

Many people also misunderstand the studies. If you classify red meat as including mcdonalds and hot dogs and compare it to the Mediterranean diet where they home cook fresh steak you get, not surprisingly, conflicting results about the health benefits or risks of red meat in your diet and is the mental clusterfuck basis for many an online news article.

The problems are

#1 people overextend the conclusions that are warranted, like you just did... just now.
#2 people skim a news article reposting their interpretation of the conclusion of a science journal behind a paywall and tons of nuances are lost in translation. If you could read the whole journal and not its click-bait misinterpreted twin you'd probably get bored.
#3 people misinterpret the impact of the results. They don't understand confidence intervals or the limits of the statistics behind data. People can respond back whatever they want in a survey. etc.

For how much you guys link "science" articles here most of them are just skimmed for the one sentence you are looking for and nobody really reads shit. Its annoying. I bet there is somewhere on the net that links 1/10th the articles and knowledgeable people actually discuss them. That place is not here. You guys are idiots. The worst kind of idiot. Idiots who know how to link poorly understood and skimmed news-based 3rd hand science articles.

 

[OverVolt]

Something you're overlooking is that the "gold-standard in science" is not peer review followed by a methodology to expose fraudulent research. The gold standard is peer review followed by relentless attempts by other scientists in the field to reproduce research results. Because - overwhelmingly - invalid research results are the result of flaws in the studies, not fraud. Thus, whether or not LaCour's fraud was ever exposed, his results - which strongly contradicted a mountain of previous research - would have been tested by other scientist attempting to replicate his methodology and results. And those additional studies would have repudiated LaCour's results, which would then have - eventually - been rejected.

Completely consistent with what I just wrote, Broockman began to question LaCour's study in the first place because he wanted to repeat it, yet he found that the cost of performing a study that entailed obtaining 10,000 subject responses was not only much more expensive than anyone in the field could afforded, but the very firm that LaCour claimed to have used to conduct the polling said they were not equipped to carry out a study of that magnitude. And then later, when Broockman (and Kalla) were trying an unrelated study of their own, and were getting an initial response rate of only 1% (compared with LaCour's claimed response rate of 12%), they tried to contact LaCour's claimed polling firm to find out the "secret" for obtaining such a high response rate, and that's when the fraud unraveled.

I agree that it would be good for science to have a "safe path" for suspected fraudulent studies to be challenged. But my point is that exposing fraud in science is for the most part irrelevant, because fraudulent scientific results will eventually be rejected (assuming they're at odds with actual "truth"), along with all other invalid results, when scientists follow the "normal" scientific approach of attempting to replicate the results of previous studies.

The term "gold standard" is mostly in pharma research and applies to double blind placebo studies... but whatever.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC557893/

In a Medline search from 1955 onwards, the first emergence of the term—albeit in a different meaning—was in 1962, in an anonymous commentary in the Lancet. Entitled “Towards a gold standard,” it pleaded to set a standard for the use of gold salts in patients with rheumatoid arthritis. It may well have been Rudd who first introduced the “gold standard” in medicine in its current sense in 1979.1 In the following years, the number of publications that employed the term grew rapidly. This was much to the dismay of one biochemist, who thought the term was “presumptuous” for a biological test, since “the subject is in perpetual evolution [and] gold standards are by definition never reached.”2 He proposed abolishing the term “because the phrase smacks of dogma... After all, the financiers gave up on the idea of a gold standard decades ago.” He failed in his mission, however: since 1995, over 10 000 publications mentioned “gold standard.”

 

[shira]

The term "gold standard" is mostly in pharma research and applies to double blind placebo studies... but whatever.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC557893/

You're wrong. "Gold standard" means anything which is considered the best of its type. The "gold standard" of phase 3 clinical drug trials is - as you say - double-blinded placebo studies. On the other hand, tPA is the current "gold standard" for the treatment of ischemic strokes. Mauna Loa is the gold standard for monitoring CO2 levels. And

One of the country's leading drug policy reformers visited Oregon Tuesday to make a pitch for legalizing pot, calling the recreational marijuana initiative headed to the November ballot "the new gold standard" of marijuana reform efforts.

But I'm sure you've seen "gold standard" used in the context of drug trials, so clearly that must be the only valid use of the term.

 

[OverVolt]

You're wrong. "Gold standard" means anything which is considered the best of its type. The "gold standard" of phase 3 clinical drug trials is - as you say - double-blinded placebo studies. On the other hand, tPA is the current "gold standard" for the treatment of ischemic strokes. Mauna Loa is the gold standard for monitoring CO2 levels. And



But I'm sure you've seen "gold standard" used in the context of drug trials, so clearly that must be the only valid use of the term.

Apparently you didn't even skim the link.

Didn't even!

 

[shira]

Apparently you didn't even skim the link.

Didn't even!

Since that link just reinforces exactly what I wrote, why on earth would you have created your post in the first place? Come to think of it, why on earth would you attempt to sidetrack a thread with a pedantic post about the meaning of "gold standard?"