New technology, like any transformative change, can inspire a range of emotions among users from curiosity and excitement to anxiety and resistance. The long-awaited advancements of artificial intelligence (AI) have suddenly been thrust upon the general public, urologists included, with the explosion of large language models (LLMs), machine-learning algorithms that understand, interpret, and respond to human language. Much like the early years of uncertainty with electronic medical records (EMRs), LLMs and AI are still in their infancy as medical tools. Rather than resist their use, however, we can better shape their evolution and integration into urology by embracing these technologies.

Over the past several months, several innovative LLM platforms have been launched, including ChatGPT, Bard, and Bing AI, each showcasing the remarkable capabilities of AI. The adoption of these advanced AI chatbots has been extraordinary, with Open AI’s ChatGPT amassing over 100 million users within 2 months of its public release in November 2022, making it the fastest-growing application in history.¹ Despite its young age, it has already passed the SAT,² MCAT (Medical College Admission Test),³ USMLE (United States Medical Licensing Examination) Steps,⁴ and urology (practice) board exam!⁵

LLMs have the potential to augment care and reduce work burden, not replace providers. How best to apply these tools in urological practice is still evolving, and rapidly. In this article, we discuss the potential role of machine language models for patient-facing, physician-facing, and administrative applications and review their current limitations. As a medical specialty that commonly embraces new technologies (think robotics), we aim to pique curiosity with a more comprehensive understanding of LLMs.

What Is AI and an LLM?

AI, in a broad sense, refers to computer systems capable of performing intricate tasks that once required human input. Think IBM’s Deep Blue, which defeated chess grandmaster Garry Kasparov in 1997. An LLM uses AI to perform self-supervised learning on a given set of data and subsequently performs a variety of natural language processing tasks, most commonly answering conversational questions. GPT4 (generative pretrained transformer), released in March, was trained on 170 trillion parameters from various books, websites, articles, and other publicly available sources (see Table). Compare that to 175 billion parameters for GPT3, which spurred the LLM fervor in November, and 1.5 billion parameters for GPT2 released in 2019. With more data points, the LLM output becomes more accurate and human-like.

Table. Commonly Used Current Large Language Models

After training on a data set, an LLM essentially works as a statistical model. When prompted, natural language processing reviews the prompt and generative AI replies with a word-by-word response based on patterns learned during training. User feedback is utilized to optimize future responses. Whether generating a research grant on postprostatectomy care or a Shakespearean prostatic hypertrophy soliloquy, the LLM employs the same techniques. The success of AI-powered LLMs, then, depends, ironically, on the quality of the human-input request.

Integrating LLMs into Urological Care

Broadly speaking, LLMs can be applied to urological care in 3 different applications: patient-facing, provider-facing, and administrative. Prompts can be used to enhance communication and collaboration, increase efficiency, and improve patient outcomes.^6,7 LLMs are particularly good at repetitive writing tasks. Opportunities for implementation are only limited by the creativity of our prompts (Gabrielson et al article contains excellent examples⁸). Below are potential applications of LLMs in urological practice.

• Patient-facing
  • Patient education: Provide patients with personalized and accessible information about urological conditions, treatment options, and potential side effects, empowering them to make informed decisions and promote engagement.
  • Symptom management: Answer questions about symptoms with personalized suggestions for management, such as lifestyle modifications, medications, or a health care provider evaluation. May identify early warning signs and enable intervention.
• Provider-facing
  • Clinical decision support: Assist physicians with clinical questions by referencing clinical guidelines, research, and best practices. Can improve diagnostic accuracy, facilitate evidence-based decision-making, and promote standardized care pathways.
  • EMR integration: Can provide direct access to clinical decision support as above. Also can serve as first-line response to online EMR patient inquiries, decreasing response times, improving patient satisfaction, and saving time for health care providers.⁹
  • Professional education: Serve as a platform for ongoing professional development, offering access to educational materials.
• Administrative
  • Letter generation: Write prompted letters for appointments, prior authorizations, and denial appeals, reducing burden on administrative staff.
  • Educational content creation: Create educational content about urological conditions and treatments tailored to specific audience and needs (eg, pamphlet, website, media request, social media posting).

LLM Limitations and Warnings

Despite their sophisticated architecture, machine learning models are not faultless. LLMs lack formal medical training. The quality of output content is directly tied to the quality of the training data it has been exposed to. Any biases or inaccuracies in the training data may be mirrored in the output, so they may provide unreliable health-related information.⁶ In a query of 3 AUA male sexual dysfunction guidelines (erectile dysfunction, Peyronie’s disease, and disorders of ejaculation), 30% and 36% of GPT3 responses were inaccurate or incomplete, respectively (unpublished data).

These models do not retrieve data from a preexisting database as a search engine would, instead functioning as tools that generate output by approximating an ideal response based on learned patterns and associations. Thus an LLM may generate seemingly plausible but incorrect responses. This phenomenon, known as the hallucination effect, is a prevalent issue in natural language processing models.¹⁰ LLMs have even been known to quote reference citations that do not exist.¹¹

The World Health Organization recently issued a statement warning of bias and misinformation in AI health care applications.¹² In March, numerous international figures, including several tech thought leaders, in an open letter called for an immediate pause in LLM development until safety protocols can be established.¹³ Until there is a formal body overseeing the development and use of LLMs in health care, it is imperative for end users to review the accuracy and completeness of generated content. Patients must also be cautioned of potentially misleading medical information as not all current LLMs provide this disclaimer.

In summary, LLMs offer the potential for improving patient care and reducing clinical and administrative workloads as detailed in 3 application categories—patient-facing, provider-facing, and administrative. LLM-generated output should be used as a framework and fact-checked for content given the current limitations of LLMs. Regardless, these AI tools are here to stay. As a wise senior urologist once defended health care evolution: “You’re either growing or you’re fading.” In this article, hopefully we are encouraging growth.

None of this text has been generated by AI, just the human kind.