Sidebar: A Peek into Peer Review : COVID19 Demystified

This post was reviewed by Dr. Scott Covey, one of our subject matter experts.

Our ‘about us’ page very specifically states that ‘the information presented in our posts is all from peer-reviewed, published studies in reputable journals.’ In fact, we’re very strict about that. Our authors are instructed to ensure that anything they write on is peer-reviewed and to check the reputability of the journal that published it.

But what is peer review? The term is thrown around a lot but the exact inner workings are a black box for people not in research careers. What is peer review? How does it work? Why is it important?

Let’s take a look.

History of Peer Review

Well the first thing peer review is, is old. Peer review has been around since the first scientific journals. In 1731, the Royal Society of Edinburgh published a collection of peer-reviewed medical articles. The very first peer-reviewed journal, Philosophical Tractions of the Royal Society, first published a paper back in 1665. It’s believed that Philosophical Tractions was the first journal to create a formal peer review process, while under the editorship of Henry Oldenburg. It’s possible that physicians in the Arab world were reviewing each other’s procedures as the 9^th century.

As a process, peer review has walked hand-in-hand with scientific communication since scientists started to communicate. That being said, peer review only became a major part of the scientific publishing process after WWII. As the 50s and 60s came around, science started to become more specialized. Getting published suddenly became a lot more competitive. Once photocopying was invented, it was a lot easier to get copies of journals out to other people for reading… or review!

(Imagine having to type out 5 copies of your scientific article, on a typewriter, just to get it reviewed by your peers. As someone who’s used a typewriter allow me to say- it would really suck. Thanks, photocopiers!)

What is Peer Review?

Peer review is exactly what it sounds like- a review by peers. A full peer review process can take several months from beginning to end. The full peer-review process looks like this:

After months (or years) of scientific research, a group submits a paper to a journal
The paper is assessed by the editor. At this point, the editor can choose to outright reject the paper if it doesn’t meet the journal’s requirements.
- ‘Requirements’ can refer to formatting, length, structure or citation format
- Editors also check whether the research fits in with the theme and mandate of the journal. This is important because sometimes the subject of the research doesn’t quite fit with the subject of the journal.
- The editor might think the work is not novel enough, or that the research doesn’t make enough of a contribution to its field. In these cases, the editor would reject the paper for not having a wide enough appeal for the readers of the journal.

If the paper isn’t rejected by the editor for one of these reasons, it’s passed to the next step.

If the editor chooses to pass the paper on, it is sent to a minimum of two scientific experts in the field of research. These experts are called peer reviewers. Importantly, peer reviewers aren’t paid to review articles! Acting as a reviewer so is community service and is considered an important part of being a researcher.
The peer reviewers read the paper carefully and prepare a report to send back to the editor.
- In the report, the reviewers may comment on the scientific methods, analysis or interpretation. This is really important because if the methods or analysis are iffy, that could call the findings of the paper into question.
  - In cases where the methods and analysis are both alright, issues in interpretation can also lead to problems with the results. Peer reviewers point out interpretations that may be biased or conclusions that are being drawn without sufficient data (unfounded). There might also be statements that are too absolute (not accounting for exceptions) or there might be disagreement between this study and other similar studies.
- Reviewers also point out sections that are unclear or that may need more explanation. They might also suggest other research articles that should be considered and cited. Including these other articles may provide other interesting perspectives and would work to give a more fair and well-balanced overview of the other work in the field.
- Reviewers may also include comments on how interesting, new or important the research is. Publishing is very competitive, especially in big journals like Nature or Cell. These comments sometimes help the editor decide whether or not to publish the research.
- The reviewers might say that further studies are required. This is usually the major part of the review. Additional studies might be needed to further support claims that were made, to fix any holes in the data. Further studies might also help link this research to other work that’s out there. Sometimes the findings of an experiment lead to more questions, in which case there might be some ‘obvious’ studies that have to be done. These studies can extend the meaning of the research further or might work to figure out the mechanism behind the results.
The editor reads the report from the peer reviewers. Based on this report the editor can choose whether to reject the paper or accept the paper without further changes. They can also tell the authors to revisethe paper, taking into account the reviewer’s suggestions, and then resubmit.
- When the paper is resubmitted the authors will often include a rebuttal letter. In a rebuttal letter, the authors go through the reviewers’ comments one by one and explain how these comments have been addressed in the new, resubmitted draft of the paper. Rebuttal letters are a very important part of peer review since they provide authors with a voice and a way to meaningfully participate in the discussions surrounding the review process.
- Sometimes addressing a comment is as easy as adding a few lines to the paper. In other cases, the authors may have to include another supplementary figure, add in some clarifying paragraphs, cite more studies or even do more experiments. Authors explain what they’ve done to address reviewer comments in their rebuttal letter.
- Authors may also rebut reviewer comments by providing scientific evidence or further clarification.
If the authors resubmit the paper, the editor will sometimes send it right back to the same reviewers again for a re-review.
If the editors (and the reviewers, if applicable) okay the revised article, the article is ready to be published.

Types of peer review

There are several different types of peer review including open reviews, as well as single, double and triple-blind reviews.

Open: An open review is one where the authors know who’s reviewing their article and the reviewers know who wrote the article they’re reviewing. Some journals that use this type of review even publish the reviewer’s names or reports with the article. Many members of the scientific community think that open review helps to ensure that reviewers are acting ethically. That being said, others are worried that open review might allow authors to unfairly influence reviewers.

Single-blind (or closed): This is a type of peer review where the reviewers know who the authors are, but the authors don’t know who the reviewers are. Single-blind reviews help make sure that the reviewers aren’t being influenced by the authors. A potential issue, however, is that reviewers hiding behind their anonymity may be unfairly harsh.

Double-blind: In a double-blind review, both the authors and the reviewers don’t know each others’ identities. The reviewers aren’t told who wrote the article and the authors aren’t told who made the comments. By keeping the authors anonymous, we manage to eliminate bias based on gender, country of origin or academic history. This also ensures that papers submitted by very famous scientists are judged based on the science and not the authors.

Triple blind: In a triple-blind review, we add one more layer of anonymity. The reviewers don’t know who the authors are and the authors don’t know who the reviewers are. On top of that, even the editor is not aware of who the authors are.

Pre-print vs post-print

You might have noticed several different words being used to describe the new scientific articles coming across your various feeds. The two that we’re going to focus on are pre-print and post-print.

A ‘pre-print’ is an article that has been released before being peer-reviewed. This research has been released on the internet by the scientific group that conducted the study. It hasn’t been looked at by a journal editor or a peer review board yet and hasn’t been accepted into a scientific journal for publication. Pre-prints can be useful if the data contained in the study is time-sensitive or would help other groups further their research. Because peer review takes time, releasing a pre-print is a way to get the data out to other scientists more quickly. However, this quicker release is done at the expense of the stringency of the peer review process.

Many journals (such as Nature or Cell) will release ‘post-prints.’ Often, a post-print is an article that has been fully peer-reviewed and has been accepted for publication. Post-prints are released while the accepted article is being put through a final typeset for full publication. Because of this post-prints are often in a very different format than a published journal article. Despite this, post-prints are fully peer-reviewed.

It’s important to note that not all ‘post-prints’ are peer-reviewed. If you don’t know whether an article you’re reading has been put through the peer review process, it’s good to check. The document itself should have some sort of peer review information on the front page. If not, there should be information on the journal website.

Why is peer review important?

When you ask a classmate to read an essay before you submit it, or have a friend check a text before you hit send, you’re taking part in peer review. Peer review is a way to have someone else check your work for inadvertent errors, and a way to have them apply their unique viewpoint to the information you’re presenting.

When a researcher publishes a scientific article they don’t just do it to further their career. Publishing scientific results means adding to an ever-growing database of scientific knowledge. Science, at its core, depends on the reliability of that database. This means that the responsibility of scientists (and of the experts reviewing their work) is to make sure that the information being published is accurate and the product of good scientific practice. ‘Reproducible’ means that another group of scientists would be able to get the same results by following the methods outlined in the paper. If the results of a study are not reproducible, that calls the science into question.

Peer review is an external seal of approval that indicates that the research has been deemed worthy of publication by at least one set of experts. In other words, it’s a form of scientific quality assurance. Maintaining scientific quality is a core ethical responsibility of any researcher, and peer review ensures that that responsibility is shared between the authors, editors and the reviewers. Beyond that, receiving feedback from colleagues and fellow experts acts as a powerful method of training for graduate students, who will be the scientists of tomorrow.

Peer review might not be perfect, but it is important. It is one of the cornerstones of the modern scientific process. And in times like this, when science is moving forward at a blistering pace and every new day brings thousands of discoveries, peer review is more necessary than ever.