MSK Library Guides: Artificial Intelligence: ChatGPT and Large Language Models

What is ChatGPT?

The chatbot ChatGPT is one of a growing number of generative AI tools using natural language processing, an AI domain, to respond to user queries. As it has gained increased attention, concerns about its use in academic, scientific, and other settings have grown.

The information provided by ChatGPT is not always accurate. Learn about:

Artificial hallucinations in the context of fake citations
Applications where ChatGPT can be helpful and harmful when writing and researching.
HIPAA and other privacy concerns

ChatGPT use can lead to AI-plagiarized writing. However, tools to detect AI-written content have been found to be unreliable.

On a larger scale, ChatGPT and other AI tools use unsustainable amounts of energy and fresh water.

Click "Website" at the bottom of the list to view all results in PubMed.

The Use of Generative AI

There is the promise that generative AI, like ChatGPT, can cut down on the amount of time it takes to complete a review. But you have to know the strengths and limitations of the tool you are using.

AI can be helpful for generating ideas that you vet and adapt. For example, ChatGPT might be beneficial to inform your research question and help you brainstorm inclusion and exclusion criteria and what terms you want in your search strategy.
If you ask AI a question with bias, the response will likely also contain that bias.
AI cannot build a systematic search strategy. It may recommend articles that do not exist, and may also only include information from a limited date range.
Using generative AI tools requires human skill and time to review the information you receive (information that will likely not be the same if you ask the same question on a different date). Consider whether the time you spend reviewing the AI response cancels out any time you save by using generative AI.

AI can also be built into software. For example:

Covidence, a systematic review software used at the MSK Library, uses machine learning to sort records by relevance and classify randomized controlled trials. It is also beta testing using team-entered inclusion and exclusion criteria to auto-remove records.
PubMed uses AI to sort by best match and to index records.
Databases like Scopus and tools like Science Direct are offering generative AI tools built from their content. Be aware of what is included in these tools before you use them (20 and 10 years of publications, respectively, for these two resources) and whether their algorithms for recommendations are matching your research needs.

Before using something, explore its parameters, and be skeptical about anything that seems too good to be true or removes all human intervention from a task.

Guidelines on the Use of ChatGPT and AI

A selection of journal/publishing guidelines and policies surrounding ChatGPT and similar AI tools. This is not a comprehensive list; always read the guidelines of your journals of interest.

In addition, several publishers have entered into agreements with large language models that allow the use of their content to train generative AI tools. View a list of these agreements via Ithaka S&R.

BMJ Journals - AI Use
"We expect authors or others who are creating content to disclose and describe use of AI technologies in (a) any content which is submitted to us, (b) any other work by the authors which underpins or is otherwise connected with the content submitted to us, and (c) any key sources which are cited, to the best of the authors’ knowledge. ... AI technologies will not be accepted as an author(s) of any content submitted to BMJ for publication. BMJ only recognises humans as being capable of authorship since they must be accountable for the work."
Cambridge University Press - Authorship and Contributorship
"AI use must be declared and clearly explained in publications such as research papers, just as we expect scholars to do with other software, tools and methodologies. AI does not meet the Cambridge requirements for authorship, given the need for accountability. AI and LLM tools may not be listed as an author on any scholarly work published by Cambridge Authors are accountable for the accuracy, integrity and originality of their research papers, including for any use of AI. Any use of AI must not breach Cambridge’s plagiarism policy. Scholarly works must be the author’s own, and not present others’ ideas, data, words or other material without adequate citation and transparent referencing."
Committee on Publication Ethics (COPE) Authorship and AI Tools Position Statement
"Authors who use AI tools in the writing of a manuscript, production of images or graphical elements of the paper, or in the collection and analysis of data, must be transparent in disclosing in the Materials and Methods (or similar section) of the paper how the AI tool was used and which tool was used. Authors are fully responsible for the content of their manuscript, even those parts produced by an AI tool, and are thus liable for any breach of publication ethics."
Elsevier Duties of Authors
"Where authors use generative AI and AI-assisted technologies in the writing process, these technologies should only be used to improve readability and language of the work. Applying the technology should be done with human oversight and control and authors should carefully review and edit the result, because AI can generate authoritative-sounding output that can be incorrect, incomplete or biased. The authors are ultimately responsible and accountable for the contents of the work."
International Committee of Medical Journal Editors (ICMJE) Recommendations
"[H]umans are responsible for any submitted material that included the use of AI-assisted technologies. Authors should carefully review and edit the result because AI can generate authoritative-sounding output that can be incorrect, incomplete, or biased. Authors should not list AI and AI-assisted technologies as an author or co-author, nor cite AI as an author. Authors should be able to assert that there is no plagiarism in their paper, including in text and images produced by the AI. Humans must ensure there is appropriate attribution of all quoted material, including full citations."
JAMA Network Instructions for Authors
"Nonhuman artificial intelligence, language models, machine learning, or similar technologies do not qualify for authorship. If these models or tools are used to create content or assist with writing or manuscript preparation, authors must take responsibility for the integrity of the content generated by these tools. Authors should report the use of artificial intelligence, language models, machine learning, or similar technologies to create content or assist with writing or editing of manuscripts in the Acknowledgment section or Methods section if this is part of formal research design or methods."
Nature Portfolio Editorial Policies
"Large Language Models (LLMs), such as ChatGPT, do not currently satisfy our authorship criteria. Notably an attribution of authorship carries with it accountability for the work, which cannot be effectively applied to LLMs. Use of an LLM should be properly documented in the Methods section (and if a Methods section is not available, in a suitable alternative part) of the manuscript."
NIH Peer Review Process Notice
"Reviewers should be aware that uploading or sharing content or original concepts from an NIH grant application, contract proposal, or critique to online generative AI tools violates the NIH peer review confidentiality and integrity requirements."
Proceedings of the National Academy of Sciences (PNAS) Editorial and Journal Policies
"Use of AI and generative AI software, such as Large Language Models or ChatGPT, for manuscript preparation, including drafting or editing text, must be disclosed in the Materials and Methods section (or Acknowledgments, if no Materials and Methods section is available) of the manuscript and may not be listed as an author. Authors are solely accountable for, and must thoroughly fact-check, outputs created with the help of generative AI software."
Sage Publishing Publishing Policies
"Authors are required to clearly indicate the use of language models in the manuscript, including which model was used and for what purpose. Please use the methods or acknowledgements section, as appropriate."
Science Journals Editorial Policies
"AI-assisted technologies [such as large language models (LLMs), chatbots, and image creators] do not meet the Science journals’ criteria for authorship and therefore may not be listed as authors or coauthors, nor may sources cited in Science journal content be authored or coauthored by AI tools. Authors who use AI-assisted technologies as components of their research study or as aids in the writing or presentation of the manuscript should note this in the cover letter and in the acknowledgments section of the manuscript."
Taylor & Francis - Defining Authorship in Your Research Paper
"Authors must be aware that using AI-based tools and technologies for article content generation, e.g. large language models (LLMs), generative AI, and chatbots (e.g. ChatGPT), is not in line with our authorship criteria. All authors are wholly responsible for the originality, validity and integrity of the content of their submissions. Therefore, LLMs and other similar types of tools do not meet the criteria for authorship. Where AI tools are used in content generation, they must be acknowledged and documented appropriately in the authored work."
Thieme Journal Policies
"AI tools such as ChatGPT can make scholarly contributions to papers. The use of generative AI tools should be properly documented in in the Acknowledgements or Material and Methods sections. AI tools should not be listed as authors, as they do not fulfil all criteria for authorship: they cannot take responsibility for the integrity and the content of a paper, and they cannot take on legal responsibility."
Wiley Best Practice Guidelines on Research Integrity and Publishing Ethics
"GenAI tools cannot be considered capable of initiating an original piece of research without direction by humans. Tools cannot be accountable for a published work or for research design, which is a generally held requirement of authorship (as discussed in the Authorship section in these guidelines), nor does it have legal standing or the ability to hold or assign copyright. Therefore—in accordance with COPE’s position statement on Authorship and AI tools—these tools cannot fulfil the role of, nor be listed as, an author of an article."
World Association of Medical Editors (WAME) Recommendations on ChatGPT and Chatbots in Relation to Scholarly Publications
"1) Only humans can be authors; 2) Authors should acknowledge the sources of their materials; 3) Authors must take public responsibility for their work; 4) Editors and reviewers should specify, to authors and each other, any use of chatbots in evaluation of the manuscript and generation of reviews and correspondence; and 5) Editors need appropriate digital tools to deal with the effects of chatbots on publishing."