The debut of ChatGPT by OpenAI in November 2022 marked a transition into the era of artificial intelligence (AI).¹ The chatbot is a generative pretrained transformer (GPT) large language model (LLM) with an unprecedented ability to respond to complex questions or prompts in a conversational manner. However, its application to research and clinical medicine remains speculative. When asked about its use in research, ChatGPT provided a list of 10 domains (Figure 1). Several of these areas including literature reviews and synthesis, data analysis, and code generation represent potential avenues for new and experienced researchers to augment their skills and expand their research scope.

Figure 1. ChatGPT-generated list of applications of ChatGPT to research.

Help with coding is one of the more promising services of ChatGPT. Not only can it provide code snippets for specific statistical software and translate code between programs, but it can also debug and review code for troubleshooting purposes.² For example, a researcher can ask ChatGPT how to create a Kaplan-Meier curve using R (Figure 2). Additionally, ChatGPT can explain fundamental coding principles, which is particularly valuable for scientists learning to code. After obtaining an initial response, users can give feedback to ChatGPT in order to refine results. Since generative AI (GAI) is susceptible to errors, ChatGPT-generated code should be carefully reviewed and tested for accuracy.

Figure 2. Example application of ChatGPT to generate code.

Performing a literature review is one of the more tedious steps in scientific research. Asking ChatGPT to create a literature review is a simple task that can serve as an ingress for researchers to get acquainted with the interface. As with other applications of the chatbot, users obtain optimal results by providing feedback to fine-tune results. Ideally, ChatGPT could save time and avoid human errors in omitting relevant articles. However, researchers should be aware that ChatGPT-generated literature reviews may not be comprehensive and can contain inaccuracies such as incorrect or false citations.³ Suppadungsuk et al assessed the chatbot’s ability to perform nephrology-related literature reviews and found that 31% of references were fabricated.³ When they investigated components of listed citations, 54% of digital object identifiers, 14% of journal titles, and 10% of publication years were incorrect.³

While ChatGPT is unlikely to replace the creative human ability to critically evaluate research and synthesize results in a well-written paper, it can help with formatting, syntax, and drafting noncritical documents such as cover letters (Figure 3). Authors should be careful when using ChatGPT content to avoid unintentional plagiarism and sharing misinformation.^2,4

Figure 3. Example application of ChatGPT to generate cover letter draft.

Despite its infancy in the research world, many urologists are already incorporating ChatGPT into their research workflow. A recent global survey of 456 urologists found that 44.4% of participants used ChatGPT “occasionally” and 9.8% used it “frequently.”⁴ Specific academic uses of the chatbot that urologists have tested include brainstorming (45.1% of respondents), checking grammar (30.6%), developing outlines for papers (30%), coding (16.7%), and generating references (12.6%).⁴ When asked if using ChatGPT improved efficiency, respondents were evenly split between no effect and improved productivity (46.7% and 45%, respectively).⁴ Considering the effort required to verify and corroborate ChatGPT’s output, researchers will likely not gain efficacy early in the adoption phase. However, with experience, we can learn to better leverage this technology and influence future iterations of the LLM to adapt to our needs.

As ChatGPT (and other LLMs) inevitably becomes ubiquitous in research methodology and dissemination, urologists are poised to lead the medical community in incorporating it in a responsible manner. We should approach ChatGPT’s adoption in a similar fashion as other quality improvement initiatives, using systems such as the Plan-Do-Study-Act cycle.⁵ Continuous evaluation of the chatbot and other LLMs’ ability to produce reliable results is imperative as this technology continues to evolve.

Beyond considering its efficacy, the judicious implementation of ChatGPT/LLM in research warrants deliberation. Two summits on AI at the Institute for Advanced Study at the University of Amsterdam recently created “Living Guidelines for Responsible Use of Generative AI in Research” focused on the principles of accountability, transparency, and independent oversight.⁶ Proposed recommendations include specifying which GAI tools researchers have used and for what tasks, as well as scientific journals disclosing use of AI for peer review.⁶ Urologists at the University of Southern California have also established a collaboration coined the ChatGPT, Generative Artificial Intelligence, and Natural Large Language Learning Models for Accountable Reporting and Use (CANGARU) Initiative to develop guidelines to ensure “principled GAI/GPT/LLM use, disclosure, and reporting in academia.”⁷ Efforts such as these will be paramount in ensuring the ethical introduction of ChatGPT in the research world. Urologists should engage in these ventures in order to shape the future of ChatGPT so that it best serves our purposes and ultimately contributes to better outcomes for our patients.