Good article (systemic suppression of skepticism in political science)

[Moonbeam]

My response was a direct response to not seeing any "systemic suppression". If there was, the original study would still stand.

It's a dodgy expression, "systemic suppression" because it implies that the system is actively suppressing when in fact it is the system itself, its nature, that does the suppression passively. It's not a very good way to put it, I think.

 

[Victorian Gray]

Now things have flipped: Really? Nothing? Did you read the article? It's what the entire article was about. All of Brookman's associates in the field (except his close friend) discouraged him from looking into it at every turn. Even an anonymous political science web forum suppressed his skepticism. The reasonings and motivations were laid out clearly. At several points, Brookman himself, despite his suspicions, did not want to look into it any further. The article pointed out a problem that should be fixed -- not denied or ignored.

Are you equating climate science with political science?

Yeah, I read the article. Nothing was suppressed. And, climate science has what to do with this?

 

[Ichinisan]

Yeah, I read the article. Nothing was suppressed.

It's what the whole article was written about. Only one person cooperated with Brookman (his close friend). All his other peers and associates tried to dissuade him.

And, climate science has what to do with this?

Members in this thread jumped right into discussions of "MMGW" and conservative-bashing, so I think they were a bit confused and they thought this had something to do with climate science.

 

[ivwshane]

What's with this "MMGW" / "CBD" talk going on here? I'm genuinely confused.

Are you guys criticizing the article?

Other than a misunderstanding about when LaCour's study was published, did I say the wrong thing?

Am I being associated with conservative ideology for some reason?

We are trying to figure out what the point of this thread is since you added, essentially, zero commentary to your OP.

 

[Greenman]

When did peer review replace experimentation as the chosen method of proving an hypotheses? My tiny little bit of scientific training was a very long time ago and peer review was never mentioned.

 

[feralkid]

When did peer review replace experimentation as the chosen method of proving an hypotheses? My tiny little bit of scientific training was a very long time ago and peer review was never mentioned.

It's terminology.

"Experimentation" would be one of the tools used to "review" said hypotheses.

 

[Ichinisan]

We are trying to figure out what the point of this thread is since you added, essentially, zero commentary to your OP.

The point: "I enjoyed reading this article. Here, I will share it with you. Please enjoy."

 

[Brainonska511]

When did peer review replace experimentation as the chosen method of proving an hypotheses? My tiny little bit of scientific training was a very long time ago and peer review was never mentioned.

Way to build a straw man. Hypotheses must always be backed by data.

Peer review only works to review the data shown and see whether the conclusions in a paper are justified and supported by the data presented. When it comes to experiments and ideas that appeal to a broad audience, like those in Science and Nature, you in effect get a second round of review when people try to apply your ideas and methodology to other areas or expand on your work. One of the failings of the peer review system is it is hard to detect and fraud as you only have the data in front of you to go on. There is a degree of trust, and if you have a big name on a paper, you'd be more inclined to give that trust.*

In basic scientific education, peer review isn't brought up as much because it isn't the focus of that training. The focus is on teaching the method of hypothesis driven and data backed methods and some of the core concepts of scientific disciplines.

*A second thing to note about this particular retracted article: the co-author that requested the retraction was not the adviser of the grad student and didn't request the raw data: he trusted too much. It seems strange to me that a grad student could have an independent collaboration with a professor at another university without also involving his local adviser. You generally don't see that in biology, chemistry, or physics.

 

[Subyman]

It's what the whole article was written about. Only one person cooperated with Brookman (his close friend). All his other peers and associates tried to dissuade him.

That is not representative of the entire field. You have to place yourself in his exact situation. If he was busy building his dissertation and then went off on a tangent to disprove a very well respected researcher's paper, then I think the advice given to him was acceptable regardless of the end result. After-the-fact when it ended up being fraud, he looks like a hero, but at the time the best advice was to work on your own research and get your credential and then take on the world.

As I've pointed out before, many grants were issued to other researchers to further study the paper and the truth would have come out soon.

Brainonska makes a great point. When you make it to the grad school level and begin original research there will not be a person standing over your shoulder making sure all the data you collect is not fraudulent. The peer review process makes sure your data aligns with your written report, they do not rerun your experiments. That happens when your results are published and others use them. The incentive to not produce fraudulent data is that others will eventually find out and you would have thrown your entire career in the garbage.

 

[Brainonska511]

.

Brainonska makes a great point. When you make it to the grad school level and begin original research there will not be a person standing over your shoulder making sure all the data you collect is not fraudulent. The peer review process makes sure your data aligns with your written report, they do not rerun your experiments. That happens when your results are published and others use them. The incentive to not produce fraudulent data is that others will eventually find out and you would have thrown your entire career in the garbage.

One clarification: in the life and physical sciences, there is someone sort of standing over you: the principle investigator (your boss and adviser) overseeing the research and pulling in the grants (no way would the NIH give hundreds of thousands of dollars to a grad student to run a research project. Almost all funds flow though the PI, with the exception of individual fellowships). Sometimes, fraud still slips by, such as a student manipulating raw data or making up data. But the PI is on the papers, sees the data, guides interpretation, and helps his lab develop new experiments to test growing hypotheses.

It seems that things can be a bit different in political science.

 

[Subyman]

One clarification: in the life and physical sciences, there is someone sort of standing over you: the principle investigator (your boss and adviser) overseeing the research and pulling in the grants (no way would the NIH give hundreds of thousands of dollars to a grad student to run a research project. Almost all funds flow though the PI, with the exception of individual fellowships). Sometimes, fraud still slips by, such as a student manipulating raw data or making up data. But the PI is on the papers, sees the data, guides interpretation, and helps his lab develop new experiments to test growing hypotheses.

It seems that things can be a bit different in political science.

I agree, maybe I wasn't clear, but I mean the advisor does not typically stand over the lab work for each test (at that point he may as well run it himself.) There is an element of trust.

I do agree that it seems in some fields the graduate student has full control over the creation of the experiments and then performing them. In most physical science fields, the student helps their advisor with their current research or may branch into a side project directly relevant to the advisor's research. At least that was my experience.

 

[Darwin333]

Peer Review, Peer Review. The gold standard of all science is Peer Review.

Turns out peer review is crap, and if you are famous in a community, the review is nothing more then a glance.

Umm, peer review is what lead to the discovery of fraudulent data. Seems like the system worked exactly as it was supposed to in this case. Maybe it wasn't fast enough for your liking but this is the way science works.

 

[norseamd]

Skepticism and peer review are not the same thing even if one supports the other in certain circumstances.

Skepticism is way.

You need to find the no way.

 

[Moonbeam]

Umm, peer review is what lead to the discovery of fraudulent data. Seems like the system worked exactly as it was supposed to in this case. Maybe it wasn't fast enough for your liking but this is the way science works.

It's not the way science is supposed to work. It's the way any system of human beings who are egotistical works. You set up a system in which people compete for reputation = grants = career, and you wind up with people defending turf and afraid to rock the establishment boat for fear of reprisals.

We have a system in which the pure pursuit of knowledge and the fine qualities of curiosity that have carried mankind so far, ground down by human beings with ego and fear, justifiable fear that is a product of a competitive system. Competition creates and maintains ego. Competition creates fear and fear is hate. Competition is hate. The same fucking shit happens to whistle blowers. "Please oh please blow the whistle so the system can blow up your ass." Right, you better be a warrior at a very young age and have the moral fortitude of God.

 

[brycejones]

Umm, peer review is what lead to the discovery of fraudulent data. Seems like the system worked exactly as it was supposed to in this case. Maybe it wasn't fast enough for your liking but this is the way science works.

Your are wasting your time with that one. Rocks have more intellectual capacity.

 

[Subyman]

It's not the way science is supposed to work. It's the way any system of human beings who are egotistical works. You set up a system in which people compete for reputation = grants = career, and you wind up with people defending turf and afraid to rock the establishment boat for fear of reprisals.

We have a system in which the pure pursuit of knowledge and the fine qualities of curiosity that have carried mankind so far, ground down by human beings with ego and fear, justifiable fear that is a product of a competitive system. Competition creates and maintains ego. Competition creates fear and fear is hate. Competition is hate. The same fucking shit happens to whistle blowers. "Please oh please blow the whistle so the system can blow up your ass." Right, you better be a warrior at a very young age and have the moral fortitude of God.

We aren't machines. Obviously we have to work within the confines of our mental capacity and behavioral evolution.

 

[unokitty]

Science Fiction: Michael LaCour's Gay Rights Canvassing Hoax Shows the Limits of Peer Review

LaCour and Green’s article, “When Contact Changes Minds: An Experiment on Transmission of Support for Gay Equality,” was published in the December 2014 issue of the prestigious journal Science. It immediately caught the attention of journalists and political activists, serving as the basis for an episode of This American Life, and articles in The New York Times, The Wall Street Journal, Bloomberg Politics, and other publications. According to Ira Glass, host of This American Life, the study seemed to show that the canvassers of the LGBT Center had “invented something new, a new tool to change people’s opinions.”

Unfortunately, it increasingly looks like what was invented was not a new tool of persuasion but rather the evidence of the study itself. Challenged by subsequent researchers who have not been able to replicate the findings of the 2014 article and evidence that LaCour made false claims about funding for his research, Green has asked Science to retract the article...

While science includes gatekeeping measures to weed out inferior research, in their day-to-day collaborative activities scientists have to assume that the people they are working with are not pathological liars, that they won’t simply make up data...

The publication of so dubious an article is likely to embarrass the many parties involved; not just Green but also the LGBT Center, the journal Science, and the fellow social scientists who greenlit the article during the peer review process ...

Who'Da Thunk It?

Uno

 

[Brainonska511]

Umm, peer review is what lead to the discovery of fraudulent data. Seems like the system worked exactly as it was supposed to in this case. Maybe it wasn't fast enough for your liking but this is the way science works.

I would argue that the peer review process didn't catch it. If it had been caught, it wouldn't have been published. The peer review process's failing is that they don't really get to see all the raw data (or have the time to reanalyze everything from scratch), they only have what is in the paper to go on. The reviewers could have asked for more data, but there is still the level of trust that it isn't faked.

However, secondary informal review did catch this fraud when people tried to replicate the methods and couldn't, inviting further scrutiny and the eventual retraction. The nature of science is that it is self correcting. It just sometimes takes a lot of time for a correction to come out. The very fact that it was such a high impact publication is what led to people trying to replicate it and finding it was false so quickly.

 

[Victorian Gray]

It's what the whole article was written about. Only one person cooperated with Brookman (his close friend). All his other peers and associates tried to dissuade him.

I repeat, nothing/nobody was suppressed. The article has been withdrawn. The system worked.

Members in this thread jumped right into discussions of "MMGW" and conservative-bashing, so I think they were a bit confused and they thought this had something to do with climate science.

And this has what to do with anything *I've* posted?

 

[Greenman]

It's not the way science is supposed to work. It's the way any system of human beings who are egotistical works. You set up a system in which people compete for reputation = grants = career, and you wind up with people defending turf and afraid to rock the establishment boat for fear of reprisals.

We have a system in which the pure pursuit of knowledge and the fine qualities of curiosity that have carried mankind so far, ground down by human beings with ego and fear, justifiable fear that is a product of a competitive system. Competition creates and maintains ego. Competition creates fear and fear is hate. Competition is hate. The same fucking shit happens to whistle blowers. "Please oh please blow the whistle so the system can blow up your ass." Right, you better be a warrior at a very young age and have the moral fortitude of God.

Moonie, that is without doubt the most sensible, concise statement I've seen you write. Well done.

 

[Moonbeam]

Moonie, that is without doubt the most sensible, concise statement I've seen you write. Well done.

Thank you.

 

[Subyman]

Thank you.

Ego +1

 

[norseamd]

http://forums.anandtech.com/showthread.php?t=2431085

 

[Knowing]

The reason peer review doesn't happen, particularly when the science is cool or the scientist has a cult of personality, is that there's no money and no glory in it.

When did peer review replace experimentation as the chosen method of proving an hypotheses? My tiny little bit of scientific training was a very long time ago and peer review was never mentioned.

Formally, peer review is a completely different phase.

 

[michal1980]

Umm, peer review is what lead to the discovery of fraudulent data. Seems like the system worked exactly as it was supposed to in this case. Maybe it wasn't fast enough for your liking but this is the way science works.

How did the system work?

Peer review published the article.

ONE person. ONE thought something was funny and dug into it. And at every step he was told to look the other way by peers.

So the system didn't work. The system failed.

But the members of the Church of peer review will dismiss facts that challenge them. Kind of like members of scientology.