Advice Category: Academics & Research

Effective and Responsible Use of AI in Research Guidance for Performing Graduate Research and in Writing Dissertations, Theses, and Manuscripts for Publications 

[lifted in its entirety from Georgia Tech (Georgia Institute of Technology), and revised in response to feedback from the UW Graduate School Council and key stakeholders across all three UW campuses^[1],^[2]]

The use of artificial intelligence has expanded greatly in recent years to all sectors of society. Generative AI and Large Language Models, like ChatGPT®, Microsoft Co–Pilot®, and Google Gemini® are rapidly becoming available, expanding access to this technology for use by anyone who is digitally literate.     

These tools can be used effectively in conducting research, and this guidance document is developed to assist researchers, especially graduate students, in this purpose. The guidelines presented here are recommendations only and will evolve with the rapidly changing landscape of AI.^[2]

AI tools will also impact your teaching. While some of the guidance below will be relevant to your work as a scholar, please also visit this page from the Center for Teaching and Learning to learn more about AI+Teaching

Some questions that guide the discussion are as follows: 

• What are potential equity impacts and implications for generative AI use? What are ways to reduce unintended impacts that may perpetuate equity gaps? What other ethical considerations should be considered about responsible AI use?
• How can students use AI effectively as a tool to help generate research ideas and approaches? 
• How can students use AI effectively as a writing or editing tool for publications and their thesis? 
• How accurate are the results from an AI source? 
• How confidential is the process? Is a student giving away valuable ideas or research results to an open platform (like ChatGPT) before the topic is peer reviewed and published?  Will you lose your intellectual property rights, such as patents? 
• How can students improve their skills in using AI as a tool in research?
• How might using AI affect the student’s development of personal knowledge and skills?
• Are students at risk of committing research misconduct in the form of plagiarism if they use a generative AI platform to write parts of their thesis? 
• How can students (and other researchers) transparently indicate within their written works the contribution of generative AI or other assistive technologies? 
• What restrictions do professional societies and publishers place on the use of AI in publications and in reviewing papers or proposals?

ChatGPT® was asked similar questions, and both questions and the generated responses are given in the Appendix. Note that ChatGPT® has the following disclaimer: 

“While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice.”

Strengths of Large Language Models and Generative AI

• It can sort through a large amount of information quickly to synthesize content from that information.   
• It can summarize existing information that is freely available and not behind firewalls or paywalls and is neither too old nor too recent. For example, it can summarize existing methodologies in a particular research area or positions on a topic. 
• It may help improve the communication skills of non-native speakers. 

Challenges for AI

• Current algorithms can summarize but do not necessarily generate accurate, unbiased, and dependable new or creative ideas.  
• The quality of the output depends on the algorithmic approach, the quality of the training data, and the user’s understanding of the tools’ limitations and biases as they write queries.  Though the content generated may sound very plausible, it may be inaccurate such as including non-existent publications or incorrect citations of publications. There are many companies that provide generative AI platforms, and the quality is mixed. 
• AI tools are better at summarizing information than retrieving it. Using these tools as search engines will likely result in biased results.
• People are prone to biases in their work, and AI can pick up those biases from the training data and even amplify them or introduce its own. 
• Confidentiality or security of data voluntarily input to a Large Language Model by users depends on the policies and practices of the company that owns that platform. For preciseness, these terms should be accessed outside of the AI platform, rather than prompt the AI platform for these terms. For your own security and confidentiality of your data, it is more prudent to assume that whatever is fed into a query is owned by that company.    

Use of AI in Graduate Student Research

Creating new knowledge and performing research is at the highest level of the educational experience of students.  Novice researchers must learn essential critical thinking skills needed in formulating a research idea, determining appropriate methods and approaches for the research plan, collecting data, summarizing results, and drawing conclusions. AI can be a valuable tool for assistance but is not an accountable entity for the research outcomes since the ultimate responsibility of research lies with the human.  

Your program, instructor, or advisor may have provided specific expectations around the use of AI in your research. As you consider the use highlighted below, ensure that you are honoring their expectations and any agreements/attestations you have made with them. There may be serious consequences for breaking academic agreements. UW’s Academic Misconduct Policy can be found here.

Guidance on the Use of Generative AI in Generating Research Ideas or Approaches

• Brainstorming: You can use the AI tool as a brainstorming partner, where you exchange ideas whether the AI prompts you or you prompt the AI for ideas. Brainstorming is an iterative process that can be made more effective with the way that the queries are posted. For more samples or information, post this query: “How can I use AI to help me to brainstorm an idea?” 
• Surveying Existing Approaches: Large Language Models, if trained broadly in a topic, can give an initial overview of existing approaches or existing literature on a topic.  Current research sources such as library or professional society databases are more reliable in terms of accuracy of peer-reviewed content. See the Rutgers University Library site for guidance and AI platforms for searching research databases. 
• Prompt engineering is important: Practice the prompts used for Generative AI.  The value of the response depends on the value of the prompt. If you provide a low quality or vague prompt, you will get vague results. Critical thinking skills may be enhanced by learning how to iterate on the prompts to refine results or to use the material gathered from one result to identify new avenues of inquiry to pursue.  Examples of strategies for prompt engineering are found in the OpenAI Guide and in Ivan Allen article on prompt engineering.

Advice on Usage

• Be very skeptical of the results and be sure to do your due diligence in fact-checking information. Do not trust any outputs that you do not evaluate yourself or trace back to original credible sources.  There are many stories of generative AI giving citations of articles that do not exist (see the article in the Chronicles of Higher Education by Hicks referenced below). 
• Be scientific with your prompts (or queries): Prompting is not deterministic, so the same prompt at a different time may result in a different response. Small changes in the wording of the prompt may yield very different responses. Keep records, make small changes and see how it affects the outcome, etc. 
• Don’t share any data or information that is confidential, proprietary, or have IP implications.  Your uploaded data or ideas might be incorporated into the learning model to be available for others in your research area, prior to you having a chance to publish it. If you intend to pursue commercialization or other Intellectual Property avenues for your work, putting the information into an open AI platform may be considered as disclosure. 

Guidance on the Use of Generative AI in Writing Publications or Theses

• Editorial assistance: Using an AI platform for a grammar check and for editorial improvements is similar to having a person proofread a paper or essay and generally does not require acknowledgement.  An example of an AI editorial tool is Grammarly. Depending on the AI platform, keep in mind that what is intended only as a grammar check may inevitably put your ideas into a system over which you don’t have control.
• Using AI may produce content that sounds good but would not withstand scrutiny by experts, including, for example, your thesis reading committee and peer reviewers. 
• Substantive Generative AI assistance: Using an AI to write a portion of a dissertation or paper may constitute cheating and/or plagiarism.  
  • From language on Academic Misconduct from the Office of Community Standards and Student Conduct:
    • Artificial Intelligence (AI) content generators, such as ChatGPT, present opportunities that can contribute to your learning and academic work. However, using these technologies may also violate the academic standards of the University.
  • From the Student Conduct Code 209.7.C
    • Academic Misconduct Includes: “Cheating,” which includes, but is not limited to: The use of unauthorized assistance in taking quizzes, tests, or examinations, or completing assignments;
    • Plagiarism includes, but is not limited to: a. The use, by paraphrase or direct quotation, of the published or unpublished work of another person without full and clear acknowledgment; or b. The unacknowledged use of materials prepared by another person or acquired from an entity engaging in the selling of term papers or other academic materials.
    • Academic Misconduct Includes: Unauthorized collaboration.
  • Given the mention of assistance from AI in the discussion of plagiarism section Academic Misconduct Policy, is it ever ok to use an AI source in brainstorming or otherwise to help generate research ideas?  How does this compare to brainstorming ideas with human collaborators? Consider, for example, brainstorming done among human colleagues that results in research that is published. The contribution of the collaborators, if substantial, may result in an acknowledgement in the manuscript or authorship.  AI brainstorming can be treated similarly, to an extent. An AI session that resulted in novel ideas may be recognized in the published work, but authorship is more controversial. Sample guidelines of authorship are given in the section on Professional Societies and Publishers below. Note that all of these guidelines require that an author approve the final manuscript and be held accountable for the content. Both of these requirements would be difficult for an AI agent to fulfil in order to receive authorship recognition.

More Advice on Usage

• Understand the AI policies for publications prior to submission: A professional society may have specific restrictions on the use of AI in the writing of a publication to be submitted and reviewed.  Make sure that you understand and abide by those restrictions before submission.  For example,
  • Generally, authors are responsible for the content of their submissions, regardless of whether they used generative AI or not.  So, authors would be responsible for errors in AI-generated content. 
• Some publishers require full transparency and credit for the source of AI-ideas in the manuscript.  
• Preserve confidentiality of the information: any information that you upload into a Large Language Model may become part of that model’s data and, as a result, be used to form responses to another person’s inquiries on that topic. If your paper has not yet been submitted for peer review, then you may decide to seek AI editing help only on smaller sections of the paper that don’t contain new research content.  In some cases, you may be able to opt out of your information being included in the training data.   
• Don’t short circuit the learning process: For a graduate student, an important part of their learning processes is to gain skills on analyzing, summarizing, and discussing their research results. Inputting data into a generative AI platform and asking it to write this type of content does not give the student the experience to gain those skills Putting aside the confidentiality risk mentioned above and publishers’ restrictions, hypothetically, a researcher could ask the generative AI platform to outline a discussion points in order to gain ideas as a first step in doing their own analysis and write-up.  The researcher should be aware of the limitations and possible consequences of that action. 

Act ethically

The responsible conduct of research includes a number of practices that should be considered when using AI to assist in performing research: 

• Authorship: Only include material in your writing that you or a co-author wrote or that you can cite from primary sources.  Note also that there is no guarantee that an AI-generated response does not include passages verbatim from other sources, so quoting an AI-generated response may crossover to traditional plagiarism. 
• Falsification or fabrication of data: There is no guarantee that Generative AI will produce accurate results or that it will not create new false data.   
• Responsible collection and management of data: There are ethical standards on the treatments of research subjects and their data that must be upheld. For example, you would not upload sensitive data to an open AI platform.   There’s room also to reference the question of whether use of AI generated content is ever ethical by bringing up awareness of whether authors of the training data opted in, etc.  
• Awareness of the tools you’re using: make sure you’re aware of their terms of service, where it obtained its data, and how it will use the information you put into it. 

Sample Guidance from Professional Societies, Publishers, and Funding Agencies 

Using AI in Reviewing Proposals and Papers

As a reviewer, you generally need to attest to confidentiality of the information that you are to review, especially not sharing the information with others.  Publishers and funding agencies are starting to put restrictions on the use of AI in the review process. For example,  

“NIH prohibits NIH scientific peer reviewers from using natural language processors, large language models, or other generative Artificial Intelligence (AI) technologies for analyzing and formulating peer review critiques for grant applications and R&D contract proposals.” 

From NIH NOT-OD-23-149

Authorship and AI: 

Below are several links to authorship guidelines from societies and publishers. The additional guidelines on authorship listed below have a common element: an author is responsible for the content of their work. So, it would not be acceptable for an author to excuse inaccuracies in their published results as “AI wrote it.”  

• Association for Computing Machinery
• Discussion on Journal of American Medical Association (JAMA) rules on the use of AI:
  • Guidance for Authors, Peer Reviewers, and Editors on Use of AI, Language Models, and Chatbots | Medical Journals and Publishing | JAMA | JAMA Network
• Nature: “Why Nature will not allow the use of generative AI in images and video”
• “Science journals set new authorship guidelines for AI-generated text,” Jennifer Harker, Environmental Factor, National Institute of Environmental Health Sciences, March 2023, accessed October 16, 2023
• This editorial includes the following suggestion for consideration for research publications: “Authors should specify who used the system, the time and date of the use, the prompt(s) used to generate the text, the sections(s) containing the text; and/or ideas in the paper resulting from NLP use.” From an editorial in the Journal on Accountability in Research.
• Guidelines on Authorship:  
  • “Defining the role of authors and contributors,” International Committee of Medical Journal Editors (ICMJE) (2023)
  • Editorial Policies of Nature: 
    • Authorship | Nature Portfolio
    • https://www.nature.com/nature/editorial-policies/ai
    • Adapted from McNutt et al., Proceedings of the National Academy of Sciences, Feb 2018, 201715374; DOI: 10.1073/pnas.1715374115; licensed under CC BY 4.0: 
      • Editorial Policies of Science:  https://www.science.org/content/page/science-journals-editorial-policies
      • Science Journals: Editorial Policies | Science | AAAS

Cited and Additional References and Resources 

• “New Policies Navigate Role of AI Assistants in CS Courses,” Georgia Tech, downloaded July 9, 2024
• “Prompt Engineering,” OpenAI Platform, accessed July 9, 2024 
• “Artificial Intelligence (AI) An introduction to generative AI and its use in teaching, research, and publishing,” Rutgers University Libraries
• “Using AI to write scholarly publications,” Mohammad Hosseini, Lisa M. Rasmussen & David B. Resnik (2023) Accountability in Research
• “How to use Chat GPT: Opportunities and Risks for Researchers”
• “How to Use Chat GPT for Scientific Research Paper Writing?” Mohamed Yusuf, accessed July 12, 2023
• “No ChatGPT can’t be your new research assistant,” Maggie Hicks, The Chronicle of Higher Education, August 23, 2023. Accessed October 16, 2023.
• “Artificial Intelligence and the Future of Teaching and Learning, Insights and Recommendations,” Department of Education, Office of Educational Technology, May 2023

Sample policies and guidance at other universities

• Boston University – Using Generative AI in Coursework
• Stanford – Generative AI Policy Guidance | Office of Community Standards 
• UCLA – Guidance for the Use of Generative AI – UCLA Center for the Advancement of Teaching  
• USC – CIS-Generative-AI-Guidelines

Appendix A: Questions input to ChatGPT and CoPilot and the corresponding AI generated answers.  (The UW Graduate School added sections 2,4,6 to provide an example of the variability of output received from the same question)

“How should I use ChatGPT in doing research” (dated July 12, 2023)

Response: When using ChatGPT for research, here are some general guidelines to keep in mind:

1.     Clearly define your research question: Clearly articulate the specific topic or problem you are investigating. This will help you communicate your needs more effectively to ChatGPT and get relevant responses.

Break down complex questions: If you have a complex research question, consider breaking it down into smaller sub-questions. This can make it easier to handle and analyze the information provided by ChatGPT.

3.     Provide context and background information: When asking a question, provide relevant context and any necessary background information. This will help ChatGPT better understand your query and provide more accurate responses.

4.     Evaluate and validate information: Remember that ChatGPT generates responses based on patterns it has learned from training data. While it can provide useful insights, it’s important to critically evaluate and validate the information it provides. Cross-reference the responses with reliable sources and use ChatGPT as a tool to complement your research, rather than relying solely on it.

5.     Ask for sources and citations: When requesting information from ChatGPT, explicitly ask for sources or citations whenever applicable. Although ChatGPT doesn’t have direct access to external sources, it can still provide general references or suggest search terms to help you find relevant information.

6.     Experiment with prompts and iterations: Experiment with different phrasings and prompts to refine your questions and get more precise responses. Iterate on your queries to narrow down the scope or explore different angles of your research topic.

7.     Exercise critical thinking: While ChatGPT can provide helpful suggestions and insights, it’s crucial to apply critical thinking skills to evaluate the quality and reliability of the information. Verify facts, check for bias, and consider multiple perspectives.

8.     Supplement with external research tools: Utilize external research tools and resources alongside ChatGPT. Online databases, academic journals, search engines, and domain-specific resources can provide additional information and help validate the insights gained from ChatGPT.

9.     Be aware of limitations: Remember that ChatGPT has limitations. It may generate plausible sounding but incorrect or misleading information. Be cautious when dealing with sensitive or controversial topics and verify information from reliable sources.

10.  Provide feedback: If you encounter errors or inaccuracies in ChatGPT’s responses, provide feedback to OpenAI. This helps them improve the model and provide more reliable information in the future.

Overall, ChatGPT can be a valuable tool for research, but it’s important to use it as part of a broader research process that includes critical thinking, validation, and utilization of other reliable resources.

“How should I use CoPilot in doing research” (dated November 27, 2024)

Using Copilot for research can be incredibly efficient and helpful! Here are some steps to get you started:

1. Define Your Research Topic: Clearly outline what you want to research. This helps in asking precise questions.
2. Ask Specific Questions: Break down your topic into specific questions. For example, instead of asking “Tell me about climate change,” you could ask “What are the main causes of climate change?”
3. Use Web Search: If you need the latest information or specific data, ask me to perform a web search. I can find up-to-date articles, studies, and reports.
4. Summarize Information: I can help summarize lengthy articles or papers, making it easier for you to grasp the main points quickly.
5. Organize Your Findings: I can assist in organizing your research into categories or outlines, which can be very useful for writing reports or papers.
6. Check Facts: If you need to verify information, I can cross-check facts from multiple sources.
7. Generate Ideas: If you’re stuck, I can help brainstorm ideas or provide different perspectives on your topic.

“How can I use ChatGPT in writing a dissertation?” 

Response: Using ChatGPT as a tool to assist you in writing your dissertation can be beneficial in several ways. Here are some steps you can follow to effectively incorporate ChatGPT into your dissertation writing process:

1.     Familiarize yourself with the tool: Spend some time exploring and experimenting with ChatGPT to understand its capabilities, limitations, and prompts it responds well to. This will help you maximize its utility during the dissertation writing process.

2.     Identify specific areas where ChatGPT can be helpful: Consider the sections or topics in your dissertation where ChatGPT can provide valuable insights, generate ideas, or help you refine your arguments. For example, you can use it to brainstorm ideas, ask for alternative perspectives, seek clarification on complex concepts, or request suggestions for references.

3.     Craft clear and specific prompts: When interacting with ChatGPT, provide detailed and precise prompts to get the most relevant responses. Clearly specify the information or assistance you are seeking to ensure ChatGPT understands your requirements accurately.

4.     Use ChatGPT for idea generation and expansion: If you’re experiencing writer’s block or need help generating ideas, ChatGPT can serve as a valuable resource. Ask it open-ended questions about your topic or specific aspects you’re exploring, and let it provide insights or alternative perspectives that can stimulate your thinking.

5.     Verify and fact-check responses: While ChatGPT can generate creative and thought-provoking responses, it’s essential to remember that it may also produce inaccurate or unreliable information. Always verify the responses provided by ChatGPT by cross-referencing them with credible sources and academic literature.

6.     Employ critical thinking and synthesis: As you receive responses from ChatGPT, critically evaluate and analyze the information. Assess the relevance, validity, and coherence of the generated content. Incorporate the insights provided by ChatGPT into your writing only after careful consideration and integration with your existing knowledge and research.

7.     Maintain academic integrity: While ChatGPT can be a valuable tool, it’s crucial to acknowledge its role and limitations. Clearly distinguish the content generated by ChatGPT from your original work, giving appropriate credit to the tool when necessary. Ensure that your use of ChatGPT aligns with your institution’s guidelines on academic integrity and plagiarism.

8.     Seek human feedback and guidance: While ChatGPT can provide assistance, it’s important to remember that it’s an AI language model and not a substitute for human expertise. Engage with your advisor, professors, or peers to receive feedback, guidance, and suggestions on your dissertation. They can provide valuable insights, offer constructive criticism, and help you refine your arguments.

By following these steps, you can effectively utilize ChatGPT as a supportive tool throughout your dissertation writing process. Remember to balance its assistance with your own critical thinking and scholarly efforts to produce a well-researched and original piece of academic work.

“How can I use CoPilot in writing a dissertation?  (dated November 27, 2024)

Writing a dissertation is a significant undertaking, and Copilot can be a valuable tool throughout the process. Here’s how you can leverage Copilot effectively:

1. Topic Selection:
   • Brainstorm Ideas: I can help you brainstorm potential topics based on your interests and field of study.
   • Narrow Down: Once you have a broad topic, I can assist in narrowing it down to a specific research question or hypothesis.
2. Literature Review:
   • Find Sources: I can help you locate relevant academic papers, articles, and books.
   • Summarize: I can summarize key points from these sources to help you understand the existing research landscape.
   • Organize: I can assist in organizing your literature review by themes or chronological order.
3. Research Design:
   • Methodology: I can provide information on different research methodologies and help you choose the most appropriate one for your study.
   • Data Collection: I can suggest data collection techniques and tools.
4. Writing:
   • Outline: I can help you create a detailed outline for your dissertation.
   • Drafting: I can assist in drafting sections of your dissertation, ensuring clarity and coherence.
   • Editing: I can help with editing and proofreading to improve the quality of your writing.
5. Data Analysis:
   • Statistical Analysis: I can provide guidance on statistical methods and help interpret your data.
   • Visualization: I can suggest ways to visualize your data effectively using charts and graphs.
6. Citations and References:
   • Formatting: I can help format your citations and references according to the required style guide (e.g., APA, MLA, Chicago).
   • Managing Sources: I can assist in organizing and managing your bibliography.
7. Final Review:
   • Feedback: I can help you incorporate feedback from your advisor or peers.
   • Polishing: I can assist in the final polishing of your dissertation to ensure it is ready for submission.
8. The best way to determine confidentiality of data or information that you voluntarily input into a platform is to examine the terms and conditions of the platform, accessed outside of the model.  For curiosity, the following question was entered in ChatGPT (highlight added later for emphasis):  

“If I shared data with ChatGPT during a conversation, will that data be shared with others?” 

Response: As of my knowledge cutoff in September 2021, as an AI language model, I don’t have the ability to store conversations or share information with others. I can only provide responses based on the input given to me during the current session. However, please note that while I don’t store the conversations, OpenAI may collect and store conversations for research and improvement purposes, but they are generally anonymized and stripped of personally identifiable information.

It’s always a good practice to avoid sharing sensitive or personally identifiable information while interacting with AI systems.

If you have any concerns about privacy or data security, I recommend reviewing the privacy policy or terms of service of the platform or application you are using to access ChatGPT, as they may provide more specific information on how your data is handled.

If I shared data with Co-Pilot during a conversation, will that data be shared with others? (dated November 27, 2024)

No, any data you share with me during our conversation is not shared with others. Your privacy and confidentiality are very important. The information you provide is used solely to assist you during our interaction and is not stored or shared with third parties. If you have any concerns about privacy or data security, feel free to ask! How can I assist you further?

Appendix B. An incomplete list of 20 AI tools for research and writing, generated by CoPilot;

Here’s the updated list of 20 AI tools that are particularly useful for writing and research, including Gemini:

1. Elephas: Summarizes research papers, rewrites content in various tones, and organizes your research in a central “Super Brain” for easy access.
2. ChatGPT: Summarizes news articles and answers research questions.
3. Typeset.io: Streamlines academic writing with templates and citation management.
4. Quillbot: Paraphrases text and summarizes complex materials for research.
5. Wordvice.ai: Ensures clarity, grammar, and originality in your academic writing.
6. Consensus AI: Searches vast databases and filters research papers for quality.
7. Scite.ai: Provides real citations and measures the credibility of research claims.
8. Scholarcy: Summarizes complex articles and builds a searchable research library.
9. ProofHub: Manages research projects with tasks, collaboration tools, and scheduling.
10. Research Rabbit: Builds a research library and recommends new papers.
11. Grammarly: Checks grammar, punctuation, and style, and offers suggestions for clarity and conciseness.
12. CoPilot: Assists with code writing, debugging, and documentation, making it useful for technical research.
13. Gemini: Provides insights and analytics for research data, helping to visualize and interpret results.
14. Elicit: Uses semantic search to find relevant literature and answer research questions.
15. SciSpace: Conducts literature reviews through advanced semantic search capabilities.
16. PaperPal: Streamlines academic writing and journal submission with integration into Microsoft Word.
17. ExplainPaper: Helps users understand complex language in peer-reviewed papers.
18. Julius AI: Simplifies data analysis and visualization for research.
19. Writefull: Provides language feedback and suggestions for academic writing.
20. EndNote: Manages references and citations, making it easier to organize research sources.

---

^[1] This content was gathered from experts at Georgia Tech in the areas of AI, in ethics, and in graduate education and edited by Bonnie Ferri. Specific groups of contributors include: Office of Graduate Education, Graduate Student Government Association, Office of Research, Responsible Conduct of Research Office, and Directors of Graduate Programs. 

^[2] The current version of this document is dated December 10, 2024. For suggested modifications to the content, contact the Graduate School Office of Academic Affairs at gsacad@uw.edu.