How to Review and Critically Evaluate a Scientific Research Article

South Carolina Society for Respiratory Care- Research Mentorship

“The more you read the more things you will know. The more that you learn the more places you’ll go.”
     – Dr. Seuss, “I Can Read With My Eyes Shut!”

The first step in becoming involved in clinical research is learning how to read and evaluate research articles. We all know that our profession is facing imminent changes. Technology is progressing more rapidly than ever, patient management and treatment intervention is becoming more complex, and the ability to provide the best care possible depends largely on the ability to understand and implement the information garnered from high quality scientific research.

The following is an (edited), comprehensive checklist featured in an AARC article on facilitating research, by Dean Hess. The commentary is entirely mine, so please don’t blame him for that.

Journal
Why are you looking at this journal?
Is it reputable?
Is it peer-reviewed?
What is the impact factor of this journal?

Article
Why are you looking at this particular article?
Does it relate strongly to issues you are facing in your daily work?

Title
Does it make you want to read further?

Authors
Where are the authors from?
Do you know them or have you heard of them?
What are their credentials?

Funding
Always check to see who funded the research and if there are any potential conflicts of interest. For example, if ABC Drug Company is sponsoring a huge drug trial for a potentially profitable drug, there will be a conflict of interest.

Abstract
When you start to read an article you will usually start with the abstract, and let’s be honest, sometimes that’s as far as we get. The fact we often only read the abstract is not an unknown circumstance, and it is for this reason that we should look at the abstract as if it were a “mini-paper”.

This means:

  • Is it well written?
  • Have they given you enough to want to “hook” you into reading the rest of the paper?
  • What are the null and alternative hypotheses?

Introduction/ Background Section
If the authors passed muster with the abstract (or if you are doing this for a grade), then you will move on to the intro/background section. This section should be a “mini-textbook” of the latest data on this subject. So the most important criteria here are:

  • Does it give enough information?
  • Has it included background references that are appropriate in number and quality?

Methods Section
This is where we really have to put on the thinking hats. It is not unusual for papers to be successful or unsuccessful based on the content of this section alone. So let’s get right to it:

  • This is the section that should describe what the authors actually did. Does it?
  • Does it make sense to you? Of course we will all read papers that might be a bit beyond our level of understanding, and we should continue this practice! However, it is worth noting that just because they may throw in some words you have to look up, you should still be able to get an overall sense of what the actual aim of the study was about and have an overall sense of what was done to fulfill that aim. Also, looking up information that you don’t understand is a good practice in general.
  • Make a short list (very short or you’ll be doing this all day long), of the methods.
  • Did the authors describe the study as double-blind, randomized, controlled, or cohort? What/who are the subjects/study groups? What inclusion and exclusion criteria are used?
  • Was there informed consent or animal care precautions? This is a BIG deal and we will discuss this topic alone at a later date.
  • What were the interventions?
  • Were statistics properly used? (For example: t test, analysis of variance, chi square, regression analysis, multivariate analysis). Again, we will discuss the application of statistics at a later date.
  • Was the study biased? This is another topic which we will cover later in more detail since there are many different types of biases which may only be relevant to certain types of studies.

You may not even know what counts as a description of the study type, or what the intervention was, or what constitutes a well written informed consent. And, I can promise you will often not know if the correct statistical model was used. I know very qualified people whose principal objective is to produce quality research, and they don’t always know which statistical test is appropriate. That’s what biostatisticians are for! My point is, if you don’t know all the answers, that’s OK, this is how you learn. If you don’t know find someone who does, a professor, an experienced researcher, the leader of your local journal club, or you can always email one of the research mentorship program members from the SCSRC Board of Directors (provided below). We promise to get back to you, really!

Results Section
Study the tables and figures closely.

  • Concentrate on the statistically significant differences (p-value anyone?).
  • Are those statistically significant differences also clinically important?
  • Have the authors provided enough evidence to reject the null hypothesis?
  • Was a power analysis performed before the study began? Again-ask someone!).

Discussion Section

  • How well do the authors present the importance of their study?
  • Do they explain how their results advance our collective understanding of a medical problem, question, or issue?

Summary

This is a great formula for learning to systematically review a research paper. Following this recipe, especially with the assistance of a qualified mentor, you will enhance your ability to be a good consumer of research. Being a good consumer is the platform upon which all good clinicians are built. No one just wakes up one morning and says to themselves, “Self, I think I’ll become a researcher today”, or “Self, we really need to implement protocols, I think I’ll start translating Evidence-Based Medicine into best practice for our department.” So, from now on when you see that new edition of Respiratory Care in your mailbox, don’t look at it with wistful intent! Actually open it, find an article that is of interest to you, and then use this guide to launch yourself into the next level of practice.
Again, try to find a local Journal Club to join, or email one the SCSRC board members listed below. We will be more than happy to assist you on your journey to “reading with your eyes shut”. If Dr. Seuss can do it….

Jerry Alewine, Ed.D, RRT- alewineJ@PTC.edu
Brooke Yeager, MSc, RRT- yeagerbe@musc.edu

References

1. Evidence-Based Medicine Working Group. Evidence -Based Medicine a new approach to teaching the practice of Medicine. JAMA 1992;268(17):2420-2425
2. Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L., Naylor CD, et al. Users’ guides to the medical literature: XXV. Evidence-Based Medicine Working Group. JAMA 2000;284(10);1290-1296.
3. Hess D. What Is Evidence-Based Medicine and Why Should I Care? New Horizons Symposium Papers. Respiratory Care 2004;49(7);730-741.

South Carolina Society for Respiratory Care