The Office of Research at the AUA is committed to furthering discovery and innovation with the goal of helping urological patients. This commitment manifests in many ways, including creation of the AUA Innovation Nexus, partnerships with various stakeholders, and the creation of Diversity, Equity, and Inclusion, Independent Practice, and International Research work groups.

A key part of these efforts is to critically and honestly evaluate our progress; what we are doing well, and more importantly, what we are not. However, we live in a world beyond urology and need to recognize, adapt, and change based on events around us. Over the past 5 years, and accelerated during the COVID-19 crisis, there has been a significant diminution of trust in scientific research. Some of this distrust was driven by political agendas and some by fear, but it is clear and worrisome how often medical and scientific findings which are initially met with large fanfare are later found to be nonreproducible in other settings. This is not new!

“Much of the scientific literature, perhaps half, may simply be untrue,” stated Richard Horton, editor of The Lancet, a peer-reviewed medical journal.¹ “Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.” He also posits that “something has gone fundamentally wrong with one of our greatest human creations.” There are many plausible reasons why this phenomenon is occurring, and one would argue, accelerating. As Horton points out, these include but are not limited to (1) sculpting or retrofitting of data to align with a preferred or popular theory, (2) the often statistical fairy tale of scientific “significance,” and (3) the pressure of “publish or perish.”

Given the skepticism the public now has for medical research, how should we approach our own scientific work and discovery? Moreover, given a new era of widely available artificial intelligence outlets that can be used to write journal articles and perhaps even report new findings, what strategies can we impart to the next generation of scientists and discoverers?

A prescient article written in 2005 discusses the approach to designing clinical trials then, and more so now, as fraught with landmines.² The author painstakingly reviews study designs and settings and concludes it is more likely for research findings to be false than true. He almost eerily notes that for many scientific fields, claimed research findings may simply be accurate measures of prevailing bias. Sound familiar? The author posits reasons why research bias occurs, becomes accepted, and is repeated in a wash, rinse, repeat phenomena. These include: (1) the smaller the studies conducted in a scientific field, the less likely the research findings are to be true; (2) the smaller the effect sizes in a scientific field, the less likely the research findings are to be true; (3) the greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true; and (4) the hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true. More explicitly, with many teams working within the same fields with massive amounts of experimental data, beating the competition in delivering findings is emphasized. Disseminating the most impressive positive results becomes the priority rather than the truth.

There is a term, the Proteus Phenomenon, for situations where rapidly alternating research claims occur with extremely opposite refutations. Again, this sounds like cable TV and social media during the COVID-19 crisis! Ironically, this author was criticized for questioning COVID-19 mortality data with incredible scientific overreaction. The author recommends that at minimum we should learn from other scientific experiences where biases drove the research field rather than actual scientific truths. Horton suggests that “those who have the power to act seem to think somebody else should act first.” To correct a fundamental and structural problem, there needs to be wide recognition that a problem, in fact, does exist. While to some degree the scientific community knows that this is a significant challenge, there is little motivation by stakeholders to create fundamental change. Money, politics, and science make for strange bedfellows. I do not envy the next generation of discoverers and researchers who have access to so much information but need to develop the discipline and humility to accurately analyze their findings. We can only hope to start the process of creating a better and more sustainable legacy.