Skip to main content

Systematic Review (SR) Service: STEP 4: Assessing papers retrieved

Collaborate with a MSK Reference Librarian to publish your Systematic Review.

Assessing the quality of the papers retrieved

Once you have the search results you can start to reduce the number of studies by reviewing the titles/abstracts and then obtain full text for the remainder of papers.  You should definitely have more than one content expert check the eligibility of the studies based on your predetermined inclusion/exclusion criteria.   You might also want to consider developing a strategy to resolve any possible disagreements and should keep a log of excluded studies with why you have decided to exclude them. It is important at this point to appraise the quality of the studies either using simple checklist and/or quality scales.

Librarian’s support:

1. Can show you how to obtain full text papers from the MSK collection
2. Can show you how to obtain full text papers that are not available in the MSK collection
3. Can help you find checklists or critical appraisal tools

NEXT:  STEP 5: Summarizing the evidence


Selected references: 
* Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B, Bonas S, Booth A, Jones D. Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods.  J Health Serv Res Policy. 2007 Jan;12(1):42-7.

* Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar S, Grimmer KA. A systematic review of the content of critical appraisal tools. BMC Med Res Methodol. 2004 Sep 16;4:22.

* Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013 Aug;19(4):722.   

* Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal Clin Epidemiol. 2009 Oct;62(10):1013-20.