Artificial intelligence (AI) is an evolving field with a lot of interest. There is the promise that it can cut down on the amount of time it takes to complete a review, and it can! But you have to know the strengths and limits of the tool you are using.
If you ever have any questions about using AI in your research or are curious about a specific AI tool, please reach out! We would be happy to explore this with you.
| Task | Utility |
|---|---|
| Define your research question | Moderate: AI tools can help brainstorm ideas and keywords, but cannot replace subject-matter expertise. |
| Write and register your protocol | Moderate: AI can assist with brainstorming and writing, but cannot replace subject matter expertise or ensure comprehensiveness or adherence to guidelines. |
| Search for evidence | Low: AI tools lack transparency and reproducibility is required for systematic search strategies. AI tools may only search one database of open-access literature and may limit the number of retrieved results. |
| Screen your results | Moderate: AI can help prioritize results, but must be paired with human review. |
| Assess the quality | Low: AI may highlight common issues, but cannot replace expert evaluation of study rigor and bias. In general, AI tools do not indicate if a publication has been retracted. |
| Collect the data | Low: AI extraction remains unreliable for complex or nuanced data. |
| Write the review | Moderate: AI can improve grammatical structure or flow, but interpretation must come from researchers. |
Before using an AI tool:
The MSK Library has experience with AI companies promising they can generate fully-automated systematic reviews for you in a short amount of time. Systematic reviews are very time and labor intensive, but if less time is taken and corners are cut, quality is lost.
At the same time, AI can be built responsibly into review software. For example, Covidence, an online software platform used for systematic and other related reviews, uses machine learning to sort records by relevance and classify randomized controlled trials. Covidence is also beta testing using team-entered inclusion and exclusion criteria to auto-remove records.
