Peer Review

Posted: July 28, 2011 in Logic Related
Tags: , , , ,

A recent article on peer review in science:

“A common criticism is that peer review is biased towards well-established research groups and the scientific status quo. Reviewers are unwilling to reject papers from big names in their fields out of fear, and they can be hostile to ideas that challenge their own, even if the supporting data is good.”

  1. matthew2262 says:

    ‘Science … is not so much concerned with truth as it is with consensus. What counts as “truth”? is what scientists can agree to count as truth at any particular moment in time … [Scientists] are not really receptive or not really open-minded to any sorts of criticisms or any sorts of claims that actually are attacking some of the established parts of the research (traditional) paradigm — in this case neo-Darwinism — so it is very difficult for people who are pushing claims that contradict the paradigm to get a hearing. They’ll find it difficult to [get] research grants; they’ll find it hard to get their research published; they’ll, in fact, find it very hard.’

    Professor Evelleen Richards, Science Historian, University of NSW, Australia, Lateline, 9 October 1998, Australian Broadcasting Corporation.

  2. matthew2262 says:

    “One way of critiquing a piece of scientific research is to read the academic paper in detail, looking for flaws. But that may not be enough, if some sources of bias might exist outside it, in the wider system of science.

    By now you’ll be familiar with publication bias: the phenomenon where studies with boring, negative results are less likely to get written up or published.”

    -Ben Goldacre, “Researchers don’t mean to exaggerate, but a lot of things can distort findings,” The Guardian, August 12, 2011

  3. matthew2262 says:

    “Something is wrong with the peer review system when an expert considers that a manuscript is not of enough interest and it later becomes a classic in its discipline (or, even worse, when the work reported in a rejected paper earns the Nobel Prize). … Contrary to reports by the American Association for the Advancement of Science and the National Academy of Sciences, publication in a peer-reviewed journal is not necessarily the best means of identifying valid research.”

    -Juan Miguel Campanario, “On Influential Books and Journal Articles Initially Rejected Because of Negative Referees’ Evaluations,” Science Communication, Vol. 16(3):304-325 (March, 1995).

  4. matthew2262 says:

    “Peer review is a process whereby experts determine if research meets scholarly standards for publication. Ideally, the process is blind, but often that standard is impossible to attain beacuase attendance at professional meetings and conferences means that peers know who is conducting research in a given area. The process affords many opportunities for gatekeepers to suppress unpopular ideas or settle scores with actual or imagined competitors. At best, the system is flawed, but it is the only process for maintaining the integrity of research.”
    -Carol M. Swain, Ph.D.

    Swain, C.M. (2011) “Be The People,” Nashville, TN: Thomas Nelson, Inc., pp. 61.

  5. matthew2262 says:

    We’ve received some very positive feedback about my piece on problems with the peer-review publication system. Admittedly, it’s a slightly long article, so I’d like to provide a short summarized version of the arguments here:

    Point 1. Good science does not have to be published in the peer-reviewed literature.
    Groundbreaking scientific books, like Darwin’s Origin of the Species or Newton’s Principia were not published in peer-reviewed journals. There are many examples of leading journals like Nature and Science having rejected important research, including research that later won the Nobel prize. Even the U.S. Supreme Court ruled in the 1993 case Daubert v. Merrell Dow Pharmaceuticals, Inc. that “Publication (which is but one element of peer review) is not a sine qua non of admissibility; it does not necessarily correlate with reliability, and in some instances well-grounded but innovative theories will not have been published.” It’s a fallacy to claim that a scientific idea is necessarily unreliable if it hasn’t appeared in the peer-reviewed literature.

    Point 2: The peer-review system faces two common criticisms: (1) that the system wrongly rejects scientifically valid papers, and (2) that the system wrongly accepts scientifically flawed papers.
    There are many examples where journals had to retract papers because errors, or even outright fraud, went undetected by the reviewers. Studies have found that peer-review has little effect on improving the quality of articles. Peer-review publication is time-consuming and expensive and often excludes people for no good reason. But the “publish or perish” mindset keeps the system in place.

    Point 3: If you believe that scientific peer-reviewers are like perfectly objective robots, then you believe a myth.
    All scientists are humans, and none are inerrant. Political concerns, economic factors, lab-rivalry, support for one’s friends, and other normal human elements are never completely divorced from the peer-review process. Journals have huge economic interests in preserving the current flawed system, and research scientists gladly play along because peer-reviewed papers are necessary for them to maintain their positions.

    Point 4: Scientific dogmatists increasingly play the “peer-review card” to silence scientific dissent.
    Despite the deficiencies in the peer-review system, “peer-review” serves as a rhetorical weapon, enlisted for the purpose of silencing dissenting, minority scientific viewpoints. In scientific debates, we often hear sneers like “Does your criticism appear in a peer-reviewed journal?” before it will be taken seriously. It’s hypocritical when scientists push their views upon the public through non-peer reviewed venues like the media, but then try to shut down critics for responding in non-peer-reviewed venues.

    Point 5: The peer-review system is often biased against non-majority viewpoints.
    The peer-review system is largely devoted to maintaining the status quo. As a new scientific theory that challenges much conventional wisdom, intelligent design faces political opposition that has nothing to do with the evidence. In one case, pro-ID biochemist Michael Behe submitted an article for publication in a scientific journal but was told it could not be published because “your unorthodox theory would have to displace something that would be extending the current paradigm.” Denyse O’Leary puts it this way: “The overwhelming flaw in the traditional peer review system is that it listed so heavily toward consensus that it showed little tolerance for genuinely new findings and interpretations.”

    Point 6: ID proponents have published a significant body of legitimate peer-reviewed research, but it’s important to understand that being recognized in the peer-reviewed literature is not an absolute requirement to demonstrate an idea’s scientific merit.
    Despite the attempted lockout, ID proponents have published their ideas in peer-reviewed scientific journals. This shows that ID has academic legitimacy whether or not one applies the dubious “peer-review” test of good science.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s