Attention: Restrictions on use of AUA, AUAER, and UCF content in third party applications, including artificial intelligence technologies, such as large language models and generative AI.
You are prohibited from using or uploading content you accessed through this website into external applications, bots, software, or websites, including those using artificial intelligence technologies and infrastructure, including deep learning, machine learning and large language models and generative AI.

ARTIFICIAL INTELLIGENCE Connecting Patients to Clinical Trials With Artificial Intelligence

By: Qiao Jin, MD, National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, Maryland; Zifeng Wang, MS, University of Illinois Urbana-Champaign; Charalampos S. Floudas, MD, DMSc, MS, Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute National Institutes of Health, Bethesda, Maryland; Fangyuan Chen, BS, School of Medicine, University of Pittsburgh, Pennsylvania; Changlin Gong, MD, Jacob Medical Center, Albert Einstein College of Medicine, Bronx, New York; Dara Bracken-Clarke, MD, Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, Maryland; Elisabetta Xue, MD, Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute National Institutes of Health, Bethesda, Maryland; Yifan Yang, BS, National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, Maryland, School of Computer Science, University of Maryland College Park; Jimeng Sun, PhD, University of Illinois Urbana-Champaign; Zhiyong Lu, PhD, National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, Maryland | Posted on: 01 Mar 2024

In the evolving landscape of health care research, the efficient recruitment of patients for clinical trials stands as an important yet challenging task. Clinical trials play a vital role in drug development and evidence-based medicine. Traditionally, patient recruitment for clinical trials involves a painstaking review of patient histories and trial criteria. This task demands not only a deep understanding of the medical nuances but also a meticulous cross-referencing of patient data against trial requirements. The complexity and variability of medical records, coupled with the diverse and often ambiguous criteria of clinical trials, further complicate this process. As a result, the entire process of matching patients with the right trials is often a bottleneck, leading to delays in treatment initiation and potential missed opportunities for both patients and researchers.1

Recent large language models (LLMs) such as GPT-42 have demonstrated remarkable capabilities in understanding conversational contexts and generating human-like texts. They have achieved new state-of-the-art performance in a wide range of domains, including biomedicine and health care.3 For example, they can improve scientific literature search,4 summarize clinical evidence,5 and answer various biomedical questions.6,7 Therefore, we introduced TrialGPT, which utilizes LLMs to streamline matching patients with clinical trials by analyzing and understanding texts such as patient medical records and trial eligibility requirements.8

The functionality of TrialGPT is twofold. Firstly, it analyzes a patient’s medical history and evaluates each criterion of a potential trial for eligibility. This is enabled through a sophisticated understanding of natural language by LLMs, allowing TrialGPT to parse and interpret medical notes with remarkable accuracy. Secondly, TrialGPT aggregates these criterion-level assessments to generate a trial-level score, effectively ranking trials based on their suitability for the patient. In both steps, TrialGPT also generates the explanation in natural language for its predictions, providing further interpretability to potential users.

For example, Figure 1 shows a synthetic patient note used in a machine learning competition9 and Figure 2 shows a clinical trial for which the patient is annotated as eligible by the competition organizers. The predictions generated by TrialGPT are also shown in Figure 2, which include the criterion-level eligibility predictions and the natural language explanation for them. TrialGPT successfully predicts that the patient meets the inclusion criterion, with the evidence of both the age and the condition correctly explained. For the first exclusion criterion, TrialGPT infers from the patient summary that the patient does not meet this criterion. Regarding the second exclusion criterion, TrialGPT successfully uses its medical knowledge to classify the patient’s bulbar urethral stricture as an anterior urethral stricture, and thus should not be excluded by the criterion. Providing such transparent and explainable predictions to clinical trial recruiters can greatly reduce the manual reviewing efforts and facilitate the matching process.


Figure 1. An example patient summary from the Text Retrieval Conference Clinical Trials Track in 2021. BPH indicates benign prostatic hyperplasia; OR, operating room.


Figure 2. A candidate clinical trial (NCT01196572) with TrialGPT predictions for the patient summary shown in Figure 1.

We showed a case study of how TrialGPT can assist patient-to-trial matching above. More details on the systematic evaluations of TrialGPT can be found in our preprint.8 To summarize, we conducted extensive tests across 3 patient cohorts, comprising 184 individuals and over 18,000 trial eligibility annotations. TrialGPT demonstrated an expert-level accuracy in criterion-level predictions. In addition, at the trial level, TrialGPT’s scoring system was highly correlated with human eligibility judgments, outperforming existing models by significant margins. To further evaluate TrialGPT’s efficacy in real-world settings, we also conducted a pilot user study at a cancer center, and the results show that TrialGPT significantly reduces the time taken for patient trial matching by 42.6%. This substantial decrease in screening time not only signifies a leap in efficiency but also hints at a future where more patients can access potentially life-saving trials quicker than ever before. While our preliminary results with TrialGPT are promising, future investigations with larger sample sizes and a prospective study design are needed to validate its effectiveness.

To summarize, TrialGPT shows significant potential to efficiently and effectively match patients to clinical trials, standing at the forefront of a new era in clinical trial matching. Because TrialGPT can facilitate the trial matching process by nonexperts, it has the potential to decrease the disparities in clinical trial enrollment. As we continue to refine and enhance TrialGPT, its integration into clinical settings holds the promise of improving patient recruitment and ultimately accelerating clinical care.

Support: This research was supported by the NIH Intramural Research Program, National Library of Medicine.

  1. Unger JM, Cook E, Tai E, Bleyer A. The role of clinical trial participation in cancer research: barriers, evidence, and strategies. Am Soc Clin Oncol Educ Book. 2016;35(36):185-198.
  2. Achiam J, Adler S, Agarwal S, et al. GPT-4 technical report. ArXiv. 2023;arXiv:2303.08774. Published March 15, 2023.
  3. Tian S, Jin Q, Yeganova L, et al. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Brief Bioinform. 2024;25(1):bbad493.
  4. Jin Q, Leaman R, Lu Z. Retrieve, summarize, and verify: how will ChatGPT affect information seeking from the medical literature?. J Am Soc Nephrol. 2023;34(8):1302-1304.
  5. Tang L, Sun Z, Idnay B, et al. Evaluating large language models on medical evidence summarization. NPJ Digit Med. 2023;6(1):158.
  6. Jin Q, Yang Y, Chen Q, Lu Z. GeneGPT: augmenting large language models with domain tools for improved access to biomedical information. Preprint. ArXiv. 2023;arXiv:2304.09667v3. Published May 16, 2023.
  7. Singhal K, Azizi S, Tu T, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-180.
  8. Jin Q, Wang Z, Floudas C, Sun j, Lu Z. Matching patients to clinical trials with large language models. Preprint. ArXiv. 2023;arXiv:2307.15051v2. Published July 28, 2023.
  9. Roberts K, Demner-Fushman D, Voorhees EM, Bedrick S, Hersh WR. Overview of the TREC 2021 Clinical Trials Track. Proceedings of the Thirtieth Text REtrieval Conference (TREC 2021); 2021.