Skip to Main Content

Retired Guide - Systematic Review (SR) Service: STEP 4: Assessing papers retrieved

Collaborate with a MSK Research Informationist to publish your Systematic Review.

Assessing the quality of the papers retrieved

Once you have the search results you can start to reduce the number of studies by reviewing the titles/abstracts and then obtain full text for the remainder of papers.  You should definitely have more than one content expert check the eligibility of the studies based on your predetermined inclusion/exclusion criteria.   You might also want to consider developing a strategy to resolve any possible disagreements and should keep a log of excluded studies with why you have decided to exclude them. It is important at this point to appraise the quality of the studies either using simple checklist and/or quality scales.

Research Informationist’s support:

1. Can help you set-up your Covidence account; a web-based software platform that streamlines the production of systematic reviews. If you have already joined the MSK Library’s Covidence account, then you can log into Covidence with your email and password and start using this reference review management tool right away!
2. Can show you how to obtain full text papers from the MSK collection.
2. Can show you how to obtain full text papers that are not available in the MSK collection.
3. Can help you find checklists or critical appraisal tools.

NEXT:  STEP 5: Summarizing the evidence

Selected references: 
* Kellermeyer L; Harnke B; Knight S (reviewers). Covidence and Rayyan. J Med Libr Assoc. 2018 Oct; 106(4):580-583.

* Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013 Aug;19(4):722. 

* Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal Clin Epidemiol. 2009 Oct;62(10):1013-20.

* Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B, Bonas S, Booth A, Jones D. Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods.  J Health Serv Res Policy. 2007 Jan;12(1):42-7.

* Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar S, Grimmer KA. A systematic review of the content of critical appraisal tools. BMC Med Res Methodol. 2004 Sep 16;4:22.