Peer review
Overview
 
Peer review is a process of self-regulation by a profession or a process of evaluation involving qualified individuals within the relevant field. Peer review methods are employed to maintain standards, improve performance and provide credibility. In academia
Academia
Academia is the community of students and scholars engaged in higher education and research.-Etymology:The word comes from the akademeia in ancient Greece. Outside the city walls of Athens, the gymnasium was made famous by Plato as a center of learning...

 peer review is often used to determine an academic paper's suitability for publication.
Peer review can be categorized by the type of activity and by the field or profession in which the activity occurs.
Encyclopedia
Peer review is a process of self-regulation by a profession or a process of evaluation involving qualified individuals within the relevant field. Peer review methods are employed to maintain standards, improve performance and provide credibility. In academia
Academia
Academia is the community of students and scholars engaged in higher education and research.-Etymology:The word comes from the akademeia in ancient Greece. Outside the city walls of Athens, the gymnasium was made famous by Plato as a center of learning...

 peer review is often used to determine an academic paper's suitability for publication.

Terminology

Peer review can be categorized by the type of activity and by the field or profession in which the activity occurs. The following terms could be used to make these distinctions, but generally those in any given field just rely on the generic term. Even when qualifiers are applied, they may be used inconsistently. For example, Medical Peer review has been used to refer specifically to clinical peer review, to the peer evaluation of clinical teaching skills for both physicians and nurses, to scientific peer review of journal articles, and to the secondary rating of the clinical value of articles in peer-reviewed journals. Moreover, "medical peer review
Medical peer review
Medical peer review is the process by which a committee of physicians examines the work of a peer and determines whether the physician under review has met accepted standards of care in rendering medical services. Depending on the specific institution, a medical peer review may be initiated at the...

" has been used by the American Medical Association
American Medical Association
The American Medical Association , founded in 1847 and incorporated in 1897, is the largest association of medical doctors and medical students in the United States.-Scope and operations:...

 to refer not only to the process of improving quality and safety in health care organizations, but also to the process by which adverse actions involving clinical privileges or professional society membership may be pursued. Thus, the terminology has poor standardization and specificity, particularly as a database search term.

Professional peer review

Professional peer review focuses on the performance of professionals, with a view to improving quality, upholding standards, or providing certification. Professional peer review activity is widespread in the field of health care, where it is best termed Clinical peer review. Further, since peer review activity is commonly segmented by clinical discipline, there is also physician peer review
Physician peer review
Clinical Peer Review is the process by which health care professionals evaluate each other’s clinical performance. Clinical peer review is segmented by discipline. No inter-disciplinary models for clinical peer review have been described. Physician Peer Review is most common and is found in...

, nursing peer review, dentistry peer review, etc. Many other professional fields have some level of peer review process: accounting, law, engineering (e.g., software peer review
Software peer review
In software development, peer review is a type of software review in which a work product is examined by its author and one or more colleagues, in order to evaluate its technical content and quality.-Purpose:...

, technical peer review), aviation, and even forest fire management. In academia, peer review is common in decisions related to faculty advancement and tenure. Peer review is used in education to achieve certain learning objectives, particularly as a tool to reach higher order processes in the affective and cognitive domains as defined by Bloom’s Taxonomy. This may take a variety of forms, including closely mimicking the scholarly peer review processes used in science and medicine.

Scholarly peer review

Scholarly peer review (also known as refereeing) is the process of subjecting an author's scholarly
Scholarly method
Scholarly method or scholarship is the body of principles and practices used by scholars to make their claims about the world as valid and trustworthy as possible, and to make them known to the scholarly public.-Methods:...

 work, research, or idea
Idea
In the most narrow sense, an idea is just whatever is before the mind when one thinks. Very often, ideas are construed as representational images; i.e. images of some object. In other contexts, ideas are taken to be concepts, although abstract concepts do not necessarily appear as images...

s to the scrutiny of others who are experts in the same field, before a paper describing this work is published in a journal. The work may be accepted, considered acceptable with revisions, or rejected. Peer review requires a community of experts in a given (and often narrowly defined) field, who are qualified and able to perform impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish; and the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Although generally considered essential to academic quality, and used in most important scientific publications, peer review has been criticized as ineffective, slow, and misunderstood (also see anonymous peer review and open peer review
Open peer review
Open peer review describes a scientific literature concept and process, central to which is the various transparency and disclosure of the identities of those reviewing scientific publications...

). Other critiques of the current peer review process from concerned scholars has stemmed from recent controversial studies
Soon and Baliunas controversy
The Soon and Baliunas controversy involved the publication of a paper written by Willie Soon and Sallie Baliunas in the journal Climate Research, which prompted concerns about the peer review process of the paper and resulted in the resignation of several other editors and the eventual repudiation...

 published by the Harvard–Smithsonian Center for Astrophysics and NASA. These two published articles are now case studies of peer review failure
Peer review failure
Peer review failures occur when a peer-reviewed article contains obvious fundamental errors that undermines at least one of its main conclusions, when a journal publishes well-known information as a new discovery, or when important valid work is rejected out of hand. Peer review is not considered a...

. There have also recently been experiments with wiki-style, signed, peer reviews, for example in an issue of the Shakespeare Quarterly.

Pragmatically, peer review refers to the work done during the screening of submitted manuscripts and funding applications. This process encourages author
Author
An author is broadly defined as "the person who originates or gives existence to anything" and that authorship determines responsibility for what is created. Narrowly defined, an author is the originator of any written work.-Legal significance:...

s to meet the accepted standards
Norm (sociology)
Social norms are the accepted behaviors within a society or group. This sociological and social psychological term has been defined as "the rules that a group uses for appropriate and inappropriate values, beliefs, attitudes and behaviors. These rules may be explicit or implicit...

 of their discipline and prevents the dissemination of irrelevant findings, unwarranted claims, unacceptable interpretations, and personal views. Publications that have not undergone peer review are likely to be regarded with suspicion by scholars and professionals.

Justification

It is difficult for authors and researchers, whether individually or in a team, to spot every mistake or flaw in a complicated piece of work. This is not necessarily a reflection on those concerned, but because with a new and perhaps eclectic subject, an opportunity for improvement may be more obvious to someone with special expertise or who simply looks at it with a fresh eye. Therefore, showing work to others increases the probability that weaknesses will be identified and improved. For both grant-funding and publication in a scholarly journal, it is also normally a requirement that the subject is both novel and substantial.

Furthermore, the decision whether or not to publish a scholarly article, or what should be modified before publication, lies with the editor of the journal to which the manuscript has been submitted. Similarly, the decision whether or not to fund a proposed project rests with an official of the funding agency. These individuals usually refer to the opinion of one or more reviewers in making their decision. This is primarily for three reasons:
  • Workload. A small group of editors/assessors cannot devote sufficient time to each of the many articles submitted to many journals.
  • Diversity of opinion. Were the editor/assessor to judge all submitted material themselves, approved material would solely reflect their opinion.
  • Limited expertise. An editor/assessor cannot be expected to be sufficiently expert in all areas covered by a single journal or funding agency to adequately judge all submitted material.

Thus it is normal for manuscripts and grant proposals to be sent to one or more external reviewers for comment.

Reviewers are typically anonymous
Anonymity
Anonymity is derived from the Greek word ἀνωνυμία, anonymia, meaning "without a name" or "namelessness". In colloquial use, anonymity typically refers to the state of an individual's personal identity, or personally identifiable information, being publicly unknown.There are many reasons why a...

 and independent
Independence
Independence is a condition of a nation, country, or state in which its residents and population, or some portion thereof, exercise self-government, and usually sovereignty, over its territory....

, to help foster unvarnished criticism, and to discourage cronyism
Cronyism
Cronyism is partiality to long-standing friends, especially by appointing them to positions of authority, regardless of their qualifications. Hence, cronyism is contrary in practice and principle to meritocracy....

 in funding and publication decisions. However, US government guidelines governing peer review for federal regulatory agencies require that reviewer's identity be disclosed under some circumstances. Anonymity may be unilateral or reciprocal (single- or double-blinded reviewing).

Since reviewers are normally selected from experts in the fields discussed in the article, the process of peer review is considered critical to establishing a reliable body of research and knowledge. Scholars reading the published articles can only be expert in a limited area; they rely, to some degree, on the peer-review process to provide reliable and credible research that they can build upon for subsequent or related research. As a result, significant scandal ensues when an author is found to have falsified the research included in an article, as many other scholars, and the field of study itself, may have relied upon the original research (see Peer review failures below).

Procedure

In the case of proposed publications, an editor sends advance copies of an author's work or idea
Idea
In the most narrow sense, an idea is just whatever is before the mind when one thinks. Very often, ideas are construed as representational images; i.e. images of some object. In other contexts, ideas are taken to be concepts, although abstract concepts do not necessarily appear as images...

s to researchers or scholars who are experts in the field (known as "referees" or "reviewers"), nowadays normally by e-mail or through a web-based manuscript processing system. Usually, there are two or three referees for a given article.

These referees each return an evaluation of the work to the editor, noting weaknesses or problems along with suggestions for improvement. Typically, most of the referees' comments are eventually seen by the author; scientific journal
Scientific journal
In academic publishing, a scientific journal is a periodical publication intended to further the progress of science, usually by reporting new research. There are thousands of scientific journals in publication, and many more have been published at various points in the past...

s observe this convention universally. The editor, usually familiar with the field of the manuscript (although typically not in as much depth as the referees, who are specialists), then evaluates the referees' comments, her or his own opinion of the manuscript, and the context of the scope of the journal or level of the book and readership, before passing a decision back to the author(s), usually with the referees' comments.

Referees' evaluations usually include an explicit recommendation of what to do with the manuscript or proposal, often chosen from options provided by the journal or funding agency. Most recommendations are along the lines of the following:
  • to unconditionally accept the manuscript or proposal,
  • to accept it in the event that its authors improve it in certain ways,
  • to reject it, but encourage revision and invite resubmission,
  • to reject it outright.


During this process, the role of the referees is advisory, and the editor is typically under no formal obligation to accept the opinions of the referees. Furthermore, in scientific publication, the referees do not act as a group, do not communicate with each other, and typically are not aware of each others identities or evaluations. There is usually no requirement that the referees achieve consensus. Thus the group dynamics are substantially different from that of a jury
Jury
A jury is a sworn body of people convened to render an impartial verdict officially submitted to them by a court, or to set a penalty or judgment. Modern juries tend to be found in courts to ascertain the guilt, or lack thereof, in a crime. In Anglophone jurisdictions, the verdict may be guilty,...

.

In situations where the referees disagree substantially about the quality of a work, there are a number of strategies for reaching a decision. When an editor receives very positive and very negative reviews for the same manuscript, the editor often will solicit one or more additional reviews as a tie-breaker. As another strategy in the case of ties, editors may invite authors to reply to a referee's criticism
Criticism
Criticism is the judgement of the merits and faults of the work or actions of an individual or group by another . To criticize does not necessarily imply to find fault, but the word is often taken to mean the simple expression of an objection against prejudice, or a disapproval.Another meaning of...

s and permit a compelling rebuttal to break the tie. If an editor does not feel confident to weigh the persuasiveness of a rebuttal, the editor may solicit a response from the referee who made the original criticism. In rare instances, an editor will convey communications back and forth between authors and a referee, in effect allowing them to debate a point. Even in these cases, however, editors do not allow referees to confer with each other, though the reviewer may see earlier comments submitted by other reviewers. The goal of the process is explicitly not to reach consensus or to persuade anyone to change their opinions. Some medical journals, however (usually following the open access model), have begun posting on the Internet the pre-publication history of each individual article, from the original submission to reviewers' reports, authors' comments, and revised manuscripts.

Traditionally, reviewers would remain anonymous to the authors, but this standard is slowly changing. In some academic fields, most journals now offer the reviewer the option of remaining anonymous or not, or a referee may opt to sign a review, thereby relinquishing anonymity. Published papers sometimes contain, in the acknowledgments section, thanks to anonymous or named referees who helped improve the paper.

Most university presses undertake peer review of books. After positive review by two or three independent referees, a university press sends the manuscript to the press's editorial board, a committee of faculty members, for final approval. Such a review process is a requirement for full membership of the Association of American University Presses
Association of American University Presses
The Association of American University Presses is an association of mostly, but not exclusively, North American university presses...

.

In some disciplines there exist refereed venues (such as conferences
Academic conference
An academic conference or symposium is a conference for researchers to present and discuss their work. Together with academic or scientific journals, conferences provide an important channel for exchange of information between researchers.-Overview:Conferences are usually composed of various...

 and workshops). To be admitted to speak, scholars and scientists must submit papers (generally short, often 15 pages or less) in advance. These papers are reviewed by a "program committee" (the equivalent of an editorial board), which generally requests inputs from referees. The hard deadlines set by the conferences tend to limit the options to either accepting or rejecting the paper.

Recruiting referees

At a journal or book publisher, the task of picking reviewers typically falls to an editor
Editing
Editing is the process of selecting and preparing written, visual, audible, and film media used to convey information through the processes of correction, condensation, organization, and other modifications performed with an intention of producing a correct, consistent, accurate, and complete...

.
When a manuscript arrives, an editor solicits reviews from scholars or other experts who may or may not have already expressed a willingness to referee for that journal
Academic journal
An academic journal is a peer-reviewed periodical in which scholarship relating to a particular academic discipline is published. Academic journals serve as forums for the introduction and presentation for scrutiny of new research, and the critique of existing research...

 or book division. Granting agencies typically recruit a panel
Committee
A committee is a type of small deliberative assembly that is usually intended to remain subordinate to another, larger deliberative assembly—which when organized so that action on committee requires a vote by all its entitled members, is called the "Committee of the Whole"...

 or committee
Committee
A committee is a type of small deliberative assembly that is usually intended to remain subordinate to another, larger deliberative assembly—which when organized so that action on committee requires a vote by all its entitled members, is called the "Committee of the Whole"...

 of reviewers in advance of the arrival of applications.

Typically referees are not selected from among the authors' close colleagues
Collegiality
Collegiality is the relationship between colleagues.Colleagues are those explicitly united in a common purpose and respecting each other's abilities to work toward that purpose...

, students, or friends. Referees are supposed to inform the editor of any conflict of interests that might arise. Journals or individual editors often invite a manuscript's authors to name people whom they consider qualified to referee their work. Indeed, for a number of journals this is a requirement of submission. Authors are sometimes also invited to name natural candidates who should be disqualified, in which case they may be asked to provide justification (typically expressed in terms of conflict of interest). In some disciplines, scholars listed in an "acknowledgments" section are not allowed to serve as referees (hence the occasional practice of using this section to disqualify potentially negative reviewers).

Editors solicit author input in selecting referees because academic
Academia
Academia is the community of students and scholars engaged in higher education and research.-Etymology:The word comes from the akademeia in ancient Greece. Outside the city walls of Athens, the gymnasium was made famous by Plato as a center of learning...

 writing typically is very specialized. Editors often oversee many specialties, and can not be experts in all of them. But after an editor selects referees from the pool of candidates, the editor typically is obliged not to disclose the referees' identities to the authors, and in scientific journals, to each other (see Anonymous peer review). Policies on such matters differ among academic disciplines.

Recruiting referee
Referee
A referee is the person of authority, in a variety of sports, who is responsible for presiding over the game from a neutral point of view and making on the fly decisions that enforce the rules of the sport...

s is a political art, because referees, and often editors, are usually not paid, and reviewing takes time away from the referee's main activities, such as his or her own research. To the would-be recruiter's advantage, most potential referees are author
Author
An author is broadly defined as "the person who originates or gives existence to anything" and that authorship determines responsibility for what is created. Narrowly defined, an author is the originator of any written work.-Legal significance:...

s themselves, or at least readers, who know that the publication system requires that experts donate their time. Referees also have the opportunity to prevent work that does not meet the standards of the field from being published, which is a position of some responsibility. Editors are at a special advantage in recruiting a scholar when they have overseen the publication of his or her work, or if the scholar is one who hopes to submit manuscripts to that editor's publication in the future. Granting agencies, similarly, tend to seek referees among their present or former grantees. Serving as a referee can even be a condition of a grant, or professional association membership.

Another difficulty that peer review organizers face is that, with respect to some manuscripts or proposals, there may be few scholars who truly qualify as experts. Such a circumstance often frustrates the goals of reviewer anonymity and the avoidance of conflicts of interest. It also increases the chances that an organizer will not be able to recruit true experts – people who have themselves done work similar to that under review, and who can read between the lines. Low-prestige or local journals and granting agencies that award little money are especially handicapped with regard to recruiting experts.

Finally, anonymity
Anonymity
Anonymity is derived from the Greek word ἀνωνυμία, anonymia, meaning "without a name" or "namelessness". In colloquial use, anonymity typically refers to the state of an individual's personal identity, or personally identifiable information, being publicly unknown.There are many reasons why a...

 adds to the difficulty in finding reviewers in another way. In scientific circles, credentials and reputation
Reputation
Reputation of a social entity is an opinion about that entity, typically a result of social evaluation on a set of criteria...

 are important, and while being a referee for a prestigious journal is considered an honor, the anonymity restrictions make it impossible to publicly state that one was a referee for a particular article. However, credentials and reputation are principally established by publications, not by refereeing; and in some fields refereeing may not be anonymous.

Different styles of review

Peer review can be rigorous, in terms of the skill brought to bear, without being highly stringent. An agency may be flush with money to give away, for example, or a journal may have few impressive manuscripts to choose from, so there may be little incentive for selection. Conversely, when either funds or publication space is limited, peer review may be used to select an extremely small number of proposals or manuscripts.

Often the decision of what counts as "good enough" falls entirely to the editor or organizer of the review. In other cases, referees will each be asked to make the call, with only general guidance from the coordinator on what stringency to apply.

Very general journals such as Science
Science (journal)
Science is the academic journal of the American Association for the Advancement of Science and is one of the world's top scientific journals....

and Nature
Nature (journal)
Nature, first published on 4 November 1869, is ranked the world's most cited interdisciplinary scientific journal by the Science Edition of the 2010 Journal Citation Reports...

have extremely stringent standards for publication, and will reject papers that report good quality scientific work if editors feel the work is not a breakthrough in the field. Such journals generally have a two-tier reviewing system. In the first stage, members of the editorial board verify that the paper's findings—if correct—would be ground-breaking enough to warrant publication in Science or Nature. Top journals in other fields have similar policies, for instance the Journal of the ACM
Journal of the ACM
The Journal of the ACM is the flagship scientific journal of the Association for Computing Machinery . It is peer-reviewed and covers computer science in general, especially theoretical aspects. Its current editor-in-chief is Victor Vianu, from University of California, San Diego.The journal has...

. Most papers are rejected at this stage. Papers that do pass this 'pre-reviewing' are sent out for in-depth review to outside referees. Even after all reviewers recommend publication and all reviewer criticisms/suggestions for changes have been met, papers may still be returned to the authors for shortening to meet the journal's length limits. With the advent of electronic journal editions, overflow material may be stored in the journal's online Electronic Supporting Information archive.

A similar emphasis on novelty exists in general area journals such as the Journal of the American Chemical Society
Journal of the American Chemical Society
The Journal of the American Chemical Society is a weekly peer-reviewed scientific journal that was established in 1879 by the American Chemical Society. The journal has absorbed two other publications in its history, the Journal of Analytical and Applied Chemistry and the American Chemical Journal...

(JACS). However, these journals generally send out all papers (except blatantly inappropriate ones) for peer reviewing to multiple reviewers. The reviewers are specifically queried not just on the scientific quality and correctness, but also on whether the findings are of interest to the general area readership (chemists of all disciplines, in the case of JACS) or only to a specialist subgroup. In the latter case, the recommendation is usually for publication in a more specialized journal. The editor may offer to authors the option of having the manuscript and reviews forwarded to such a journal with the same publishers (perhaps, in the example given, the Journal of Organic Chemistry); if the reviewer reports warrant such a decision, the editor of such a journal may accept the forwarded manuscript without further reviewing.

Specialized scientific journals such as the aforementioned chemistry journals, Astrophysical Journal
Astrophysical Journal
The Astrophysical Journal is a peer-reviewed scientific journal covering astronomy and astrophysics. It was founded in 1895 by the American astronomers George Ellery Hale and James Edward Keeler. It publishes three 500-page issues per month....

, and the Physical Review
Physical Review
Physical Review is an American scientific journal founded in 1893 by Edward Nichols. It publishes original research and scientific and literature reviews on all aspects of physics. It is published by the American Physical Society. The journal is in its third series, and is split in several...

series use peer review primarily to filter out obvious mistakes and incompetence, as well as plagiarism, overly derivative work, and straightforward applications of known methods. Different publication rates reflect these different criteria: Nature publishes about 5 percent of received papers, while Astrophysical Journal publishes about 70 percent. Some open access journals such as Biology Direct
Biology Direct
Biology Direct is an online open access scientific journal that publishes original, peer-reviewed research papers and hypotheses in biology. The journal is published by BioMed Central....

 have the policy of making the reviewers' reports public by publishing the reports together with the manuscripts.

Screening by peers may be more or less laissez-faire
Laissez-faire
In economics, laissez-faire describes an environment in which transactions between private parties are free from state intervention, including restrictive regulations, taxes, tariffs and enforced monopolies....

 depending on the discipline. Physicists, for example, tend to think that decisions about the worthiness of an article are best left to the marketplace. Yet even within such a culture peer review serves to ensure high standards in what is published. Outright errors are detected and authors receive both edits and suggestions.

To preserve the integrity of the peer-review process, submitting authors may not be informed of who reviews their papers; sometimes, they might not even know the identity of the associate editor who is responsible for the paper. In many cases, alternatively called "masked" or "double-masked" review (or "blind" or "double-blind" review), the identity of the authors is concealed from the reviewers, lest the knowledge of authorship bias their review; in such cases, however, the associate editor responsible for the paper does know who the author is. Sometimes the scenario where the reviewers do know who the authors are is called "single-blinded" to distinguish it from the "double-blinded" process. In double-blind review, the authors are required to remove any reference that may point to them as the authors of the paper.

In many fields of study, single-blinding is the normative practice; however, in others, such as information systems
Information systems
Information Systems is an academic/professional discipline bridging the business field and the well-defined computer science field that is evolving toward a new scientific area of study...

, it is almost unheard of, and double-blinding is the norm. While the anonymity of reviewers is almost universally preserved, open peer review
Open peer review
Open peer review describes a scientific literature concept and process, central to which is the various transparency and disclosure of the identities of those reviewing scientific publications...

 is a relatively novel exception to this principle, where reviewers are revealed to the authors.

Critics of the double-blind process point out that, despite the extra editorial effort to ensure anonymity, the process often fails to do so, since certain approaches, methods, writing styles, notations, etc., may point to a certain group of people in a research stream, and even to a particular person.
Proponents of double-blind review argue that it performs at least as well as single-blind, and that it generates a better perception of fairness and equality in global scientific funding and publishing.

Proponents also argue that if the reviewers of a paper are unknown to each other, the associate editor responsible for the paper can easily verify the objectivity of the reviews. Single-blind review is thus strongly dependent upon the goodwill of the participants.

A conflict of interest
Conflict of interest
A conflict of interest occurs when an individual or organization is involved in multiple interests, one of which could possibly corrupt the motivation for an act in the other....

 arises when a reviewer and author have a disproportionate amount of respect (or disrespect) for each other. As an alternative to single-blind and double-blind review, authors and reviewers are encouraged to declare their conflicts of interest when the names of authors and sometimes reviewers are known to the other. When conflicts are reported, the conflicting reviewer is prohibited from reviewing and discussing the manuscript. The incentive for reviewers to declare their conflicts of interest is a matter of professional ethics and individual integrity. While their reviews are not public, these reviews are a matter of record and the reviewer's credibility depends upon how they represent themselves among their peers. Some software engineering journals, such as the IEEE Transactions on Software Engineering
IEEE Transactions on Software Engineering
The IEEE Transactions on Software Engineering is a bimonthly, peer-reviewed, scientific journal published by the IEEE Computer Society. It was first published in March of 1975....

, use non-blind reviews with reporting to editors of conflicts of interest by both authors and reviewers.

A more rigorous standard of accountability is known as an audit. Because reviewers are not paid, they cannot be expected to put as much time and effort into a review as an audit requires. Most journals (and grant agencies like NSF) have a policy that authors must archive
Scientific data archiving
Scientific data archiving refers to the long-term storage of scientific data and methods. The various scientific journals have differing policies regarding how much of their data and methods scientists are required to store in a public archive, and what is actually archived varies widely between...

 their data and methods in the event another researcher wishes to replicate or audit the research after publication. Unfortunately, the archiving policies are often ignored by researchers.

Anonymous peer review

Anonymous peer review, also called blind review, is a system of prepublication peer review of scientific articles or papers for journals
Scientific journal
In academic publishing, a scientific journal is a periodical publication intended to further the progress of science, usually by reporting new research. There are thousands of scientific journals in publication, and many more have been published at various points in the past...

 or academic conference
Academic conference
An academic conference or symposium is a conference for researchers to present and discuss their work. Together with academic or scientific journals, conferences provide an important channel for exchange of information between researchers.-Overview:Conferences are usually composed of various...

s by reviewers who are known to the journal editor or conference organizer but whose names are not given to the article's author. The reviewers do not know the author's identity, as any identifying information is stripped from the document before review. The system is intended to reduce or eliminate bias, although this has been challenged – for example Eugene Koonin
Eugene Koonin
Eugene V. Koonin is a Senior Investigator at the National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA...

, a senior investigator at the National Center for Biotechnology Information
National Center for Biotechnology Information
The National Center for Biotechnology Information is part of the United States National Library of Medicine , a branch of the National Institutes of Health. The NCBI is located in Bethesda, Maryland and was founded in 1988 through legislation sponsored by Senator Claude Pepper...

, asserts that the system has "well-known ills" and advocates "open peer review
Open peer review
Open peer review describes a scientific literature concept and process, central to which is the various transparency and disclosure of the identities of those reviewing scientific publications...

".

Open peer review

Open peer review describes a scientific literature
Scientific literature
Scientific literature comprises scientific publications that report original empirical and theoretical work in the natural and social sciences, and within a scientific field is often abbreviated as the literature. Academic publishing is the process of placing the results of one's research into the...

 concept and process, central to which is the various transparency and disclosure of the identities of those reviewing scientific publications. The concept thus represents a departure from, and an alternative to, the incumbent anonymous peer review process, in which non-disclosure of these identities toward the public – and toward the authors of the work under review – is default practice. The open peer review concept appears to constitute a response to modern criticisms of the incumbent system; therefore, its emergence may be partially attributed to these phenomena.

Postpublication reviews

The process of peer review does not end after a paper completes the peer review process. After being put to press, and after 'the ink is dry', the process of peer review continues as publications are read. Readers will often send letters to the editor
Letter to the editor
A letter to the editor is a letter sent to a publication about issues of concern from its readers. Usually, letters are intended for publication...

 of a journal, or correspond with the editor via an on-line journal club. In this way, all 'peers' may offer review and critique of published literature. A variation on this theme is open peer commentary
Open Peer Commentary
Open peer commentary consists of eliciting non-anonymous commentary on a peer-reviewed "target article" from a dozen or more specialists across disciplines, co-published with the author's response. It was first implemented by the anthropologist Sol Tax, who founded the journal Current...

; journals using this process solicit and publish non-anonymous commentaries on the "target paper" together with the paper, and with original authors' reply as a matter of course. The introduction of the "epub
Electronic article
Electronic articles are articles in scholarly journals or magazines that can be accessed via electronic transmission. They are a specialized form of electronic document, with a specialized content, purpose, format, metadata, and availability–they consist of individual articles from scholarly...

 ahead of print" practice in many journals has made possible the simultaneous publication of unsolicited letters to the editor together with the original paper in the print issue.

Criticism of peer review

One of the most common complaints about the peer review process is that it is slow, and that it typically takes several months or even several years in some fields for a submitted paper to appear in print.

While passing the peer review process is often considered in the scientific community
Scientific community
The scientific community consists of the total body of scientists, its relationships and interactions. It is normally divided into "sub-communities" each working on a particular field within science. Objectivity is expected to be achieved by the scientific method...

 to be a certification of validity, it is not without its problems. Drummond Rennie, deputy editor of Journal of the American Medical Association
Journal of the American Medical Association
The Journal of the American Medical Association is a weekly, peer-reviewed, medical journal, published by the American Medical Association. Beginning in July 2011, the editor in chief will be Howard C. Bauchner, vice chairman of pediatrics at Boston University’s School of Medicine, replacing ...

is an organizer of the International Congress on Peer Review and Biomedical Publication, which has been held every four years since 1986. He remarks,

There seems to be no study too fragmented, no hypothesis too trivial, no literature too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.


Richard Horton, editor of the British medical journal The Lancet
The Lancet
The Lancet is a weekly peer-reviewed general medical journal. It is one of the world's best known, oldest, and most respected general medical journals...

, has said that

The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability—not the validity—of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.

Allegations of bias and suppression

The interposition of editors and reviewers between authors and readers always raises the possibility that the intermediators may serve as gatekeepers
Gatekeepers
A gatekeeper is a person who controls access to something, for example via a city gate. In the late 20th century the term came into metaphorical use, referring to individuals who decide whether a given message will be distributed by a mass medium....

. Some sociologists of science
Science and technology studies
Science, technology and society is the study of how social, political, and cultural values affect scientific research and technological innovation, and how these, in turn, affect society, politics and culture...

 argue that peer review makes the ability to publish susceptible to control by elite
Elite
Elite refers to an exceptional or privileged group that wields considerable power within its sphere of influence...

s and to personal jealousy.
The peer review process may suppress dissent
Suppression of dissent
Suppression of dissent occurs when an individual or group which is more powerful than another tries to directly or indirectly censor, persecute or otherwise oppress the other party, rather than engage with and constructively respond to or accommodate the other party's arguments or viewpoint...

 against "mainstream
Mainstream
Mainstream is, generally, the common current thought of the majority. However, the mainstream is far from cohesive; rather the concept is often considered a cultural construct....

" theories.
Reviewers tend to be especially critical of conclusions that contradict their own views
Opinion
In general, an opinion is a subjective belief, and is the result of emotion or interpretation of facts. An opinion may be supported by an argument, although people may draw opposing opinions from the same set of facts. Opinions rarely change without new arguments being presented...

, and lenient towards those that accord with them. At the same time, established scientists are more likely than less established ones to be sought out as referees, particularly by high-prestige journals or publishers. As a result, ideas that harmonize with the established experts' are more likely to see print and to appear in premier journals than are iconoclastic or revolutionary ones, which accords with Thomas Kuhn
Thomas Kuhn
Thomas Samuel Kuhn was an American historian and philosopher of science whose controversial 1962 book The Structure of Scientific Revolutions was deeply influential in both academic and popular circles, introducing the term "paradigm shift," which has since become an English-language staple.Kuhn...

's well-known observations regarding scientific revolutions
The Structure of Scientific Revolutions
The Structure of Scientific Revolutions , by Thomas Kuhn, is an analysis of the history of science. Its publication was a landmark event in the history, philosophy, and sociology of scientific knowledge and it triggered an ongoing worldwide assessment and reaction in — and beyond — those scholarly...

.

Peer review failures

Peer review failures occur when a peer-reviewed article contains obvious fundamental errors that undermine at least one of its main conclusions. Many journals have no procedure to deal with peer review failures beyond publishing letters to the editor.

Peer review in scientific journals assumes that the article reviewed has been honestly written, and the process is not designed to detect fraud.

The reviewers usually do not have full access to the data from which the paper has been written and some elements have to be taken on trust. It is not usually practical for the reviewer to reproduce the author's work. Publication of incorrect results does not in itself indicate a peer review failure.

Dynamic and open peer review

It has been suggested that traditional anonymous peer review lacks accountability, can lead to abuse by reviewers, and may be biased and inconsistent, alongside other flaws.
In response to these criticisms, other systems of peer review with various degrees of "openness" have been suggested.

Starting in the 1990s, several scientific journals (including the high impact journal Nature
Nature (journal)
Nature, first published on 4 November 1869, is ranked the world's most cited interdisciplinary scientific journal by the Science Edition of the 2010 Journal Citation Reports...

in 2006) started experiments with hybrid peer review processes, often allowing open peer reviews in parallel to the traditional model. The initial evidence of the effect of open peer review upon the quality of reviews, the tone and the time spent on reviewing was mixed, although it does seem that under open peer review, more of those who are invited to review decline to do so.

Throughout the 2000s first academic journals based solely on the concept of open peer review were launched (see e.g. Philica
Philica
Philica is an online open access academic journal, which publishes articles on any subject.-History:Philica was founded in March 2006 by Ian Walker and Nigel Holt...

). An extension of peer review beyond the date of publication is Open Peer Commentary
Open Peer Commentary
Open peer commentary consists of eliciting non-anonymous commentary on a peer-reviewed "target article" from a dozen or more specialists across disciplines, co-published with the author's response. It was first implemented by the anthropologist Sol Tax, who founded the journal Current...

, whereby expert commentaries are solicited on published articles, and the authors are encouraged to respond.

Peer review of government policy

The technique of peer review is also used to improve government policy. In particular, the European Union
European Union
The European Union is an economic and political union of 27 independent member states which are located primarily in Europe. The EU traces its origins from the European Coal and Steel Community and the European Economic Community , formed by six countries in 1958...

 uses it as a tool in the 'Open Method of Co-ordination' of policies in the fields of employment and social inclusion.

A program of peer reviews in active labour market policy started in 1999, and was followed in 2004 by one in social inclusion. Each program sponsors about eight peer review meetings in each year, in which a 'host country' lays a given policy or initiative open to examination by half a dozen other countries and relevant European-level NGOs. These usually meet over two days and include visits to local sites where the policy can be seen in operation. The meeting is preceded by the compilation of an expert report
Expert report
An expert report is a study written by one or more experts that states findings and offers opinions.In law, expert reports are generated by expert witnesses offering their opinions on points of controversy in a legal case, and are typically sponsored by one side or the other in a litigation in...

 on which participating 'peer countries' submit comments. The results are published on the web.

History

The first recorded editorial prepublication peer-review process was at The Royal Society in 1665 by the founding editor of Philosophical Transactions of the Royal Society
Philosophical Transactions of the Royal Society
The Philosophical Transactions of the Royal Society is a scientific journal published by the Royal Society of London. It was established in 1665, making it the first journal in the world exclusively devoted to science, and it has remained in continuous publication ever since, making it the world's...

, Henry Oldenburg
Henry Oldenburg
Henry Oldenburg was a German theologian known as a diplomat and a natural philosopher. He was one of the foremost intelligencers of Europe of the seventeenth century, with a network of correspondents to rival those of Fabri de Peiresc, Marin Mersenne and Ismaël Boulliau...

. In the 20th century, peer review became common for science funding allocations. This process appears to have developed independently from the editorial peer review.

The first peer-reviewed publication may have been the Medical Essays and Observations published by the Royal Society of Edinburgh
Royal Society of Edinburgh
The Royal Society of Edinburgh is Scotland's national academy of science and letters. It is a registered charity, operating on a wholly independent and non-party-political basis and providing public benefit throughout Scotland...

 in 1731. The present-day peer-review system evolved from this 18th-century process.

A professional peer-review process is found in the Ethics of the Physician written by Ishaq bin Ali al-Rahwi (854–931) of al-Raha, Syria
Syria
Syria , officially the Syrian Arab Republic , is a country in Western Asia, bordering Lebanon and the Mediterranean Sea to the West, Turkey to the north, Iraq to the east, Jordan to the south, and Israel to the southwest....

. His work, as well as later Arabic medical manuals, state that a visiting physician must always make duplicate notes of a patient's condition on every visit. When the patient was cured or had died, the notes of the physician were examined by a local medical council of other physicians, who would review
Review
A review is an evaluation of a publication, a product or a service, such as a movie , video game, musical composition , book ; a piece of hardware like a car, home appliance, or computer; or an event or performance, such as a live music concert, a play, musical theater show or dance show...

 the practicing physician's notes to decide whether his/her performance have met the required standards of medical care. If their reviews were negative, the practicing physician could face a lawsuit
Lawsuit
A lawsuit or "suit in law" is a civil action brought in a court of law in which a plaintiff, a party who claims to have incurred loss as a result of a defendant's actions, demands a legal or equitable remedy. The defendant is required to respond to the plaintiff's complaint...

 from a maltreated patient.

Peer review has been a touchstone of modern scientific method
Scientific method
Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry must be based on gathering empirical and measurable evidence subject to specific principles of...

 only since the middle of the 20th century, the only exception being medicine
Medicine
Medicine is the science and art of healing. It encompasses a variety of health care practices evolved to maintain and restore health by the prevention and treatment of illness....

. Before then, its application was lax in other scientific fields. For example, Albert Einstein
Albert Einstein
Albert Einstein was a German-born theoretical physicist who developed the theory of general relativity, effecting a revolution in physics. For this achievement, Einstein is often regarded as the father of modern physics and one of the most prolific intellects in human history...

's revolutionary "Annus Mirabilis" papers
Annus Mirabilis Papers
The Annus Mirabilis papers are the papers of Albert Einstein published in the Annalen der Physik scientific journal in 1905. These four articles contributed substantially to the foundation of modern physics and changed views on space, time, and matter...

 in the 1905 issue of Annalen der Physik
Annalen der Physik
Annalen der Physik is one of the oldest physics journals worldwide. The journal publishes original, peer-reviewed papers in the areas of experimental, theoretical, applied and mathematical physics and related areas...

were not peer-reviewed by anyone other than the journal's editor-in-chief, Max Planck
Max Planck
Max Karl Ernst Ludwig Planck, ForMemRS, was a German physicist who actualized the quantum physics, initiating a revolution in natural science and philosophy. He is regarded as the founder of the quantum theory, for which he received the Nobel Prize in Physics in 1918.-Life and career:Planck came...

 (the father of quantum theory), and its co-editor, Wilhelm Wien
Wilhelm Wien
Wilhelm Carl Werner Otto Fritz Franz Wien was a German physicist who, in 1893, used theories about heat and electromagnetism to deduce Wien's displacement law, which calculates the emission of a blackbody at any temperature from the emission at any one reference temperature.He also formulated an...

. Although clearly peers (both won Nobel prizes in physics), a formal panel of reviewers was not sought, as is done for many scientific journals today. Established authors and editors were given more latitude in their journalistic discretion, back then. In a recent editorial in Nature, it was stated that "in journals in those days, the burden of proof was generally on the opponents rather than the proponents of new ideas."

See also

External links

  • Walter Noll
    Walter Noll
    Walter Noll is a mathematician, and Professor Emeritus at Carnegie Mellon University. He is best known for developing mathematical tools of classical mechanics and thermodynamics....

    (2009) The Future of Scientific Publication (Bibliography)
The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK