Do We Still Need a Randomized Trial to Prove Anything?

By Bruce Ramshaw, M.D.  Chief Medical Information Officer, Caresyntax 

At this year’s European Hernia Society meeting, I found myself having the same conversation repeatedly. A clinical leader, scientist, or industry executive would describe a product that had been used successfully in many patients. They would cite years of favorable real-world outcomes, widespread physician adoption, and an indication or label that had remained unchanged despite growing evidence. Then they would ask a question that seemed increasingly difficult to answer: “Do we really need a randomized controlled trial to prove what our physicians already know?” 

My answer is getting simpler every year. No. 

Not because randomized trials are too expensive. Not because they take too long. And not because real-world evidence is somehow easier. The reason is more fundamental. Randomized controlled trials were designed to address a different scientific problem than the one we face in healthcare. 

For most of my career, I accepted the conventional view that randomized trials were the highest form of evidence. Like many physicians, I was trained to believe that controlling variables, reducing complexity, and isolating interventions brought us closer to scientific truth. Over time, however, I noticed a growing disconnect between what these studies told us and what I saw in real-world patient care. The patients enrolled in trials often looked very different from those on my operating table. The controlled environments in which evidence was generated bore little resemblance to the realities of healthcare delivery. The more I studied healthcare as a system, the more I realized the problem was not simply with the studies. The problem was with the underlying scientific model. 

Reductionist science has been remarkably successful in fields where systems are relatively simple and controllable. Healthcare is neither. Healthcare is a complex adaptive system composed of patients, clinicians, technologies, processes, organizations, economics, and countless interacting variables that continuously influence one another. Outcomes emerge not from isolated interventions but from interactions among all these components. Yet much of our traditional evidence-generation methodology is designed to eliminate those interactions from consideration. 

This creates a paradox. The very methods we have historically considered the most rigorous are often the least helpful in understanding what actually happens in real-world care. Randomized trials can answer a narrow question exceptionally well: under a specific set of controlled conditions, did one intervention perform differently than another? What they cannot adequately answer is how a product, procedure, or care process performs across the broad range of circumstances in which patients are treated. 

That distinction becomes increasingly important for products already in widespread clinical use. For many devices cleared through the 510(k) pathway, the question is rarely whether they function. Clinicians already know they do. The more important questions concern how they perform across diverse patient populations, practice settings, and care pathways, and where, when, and in whom they create measurable value for patients and healthcare systems. Those are fundamentally systems questions, not reductionist ones. 

This is why I believe the growing interest in real-world evidence is about much more than a new data source. What we are witnessing is the beginning of a transition from reductionist science toward systems and data science. Systems science asks different questions. Rather than attempting to eliminate variation, it seeks to understand it. Rather than focusing on isolated variables, it studies relationships and interactions. Rather than pursuing static conclusions, it enables continuous learning. Most importantly, it recognizes that outcomes, costs, resource utilization, patient experience, and long-term health are interconnected dimensions of value that must be understood together rather than separately. 

The limitations of the traditional evidence model are increasingly apparent. Registry completion rates often fall below fifty percent. Publication timelines can stretch for years. Randomized trials can take several years to complete and even longer to influence practice. By the time the evidence is published, the technology may have evolved, reimbursement conditions may have changed, competitors may have entered the market, and clinical practice may have shifted. The evidence may still be scientifically sound, but it is often disconnected from the realities clinicians, manufacturers, and healthcare leaders face. 

In contrast, modern continuous quality improvement methodologies enable learning in near real time. Data can be collected as part of routine care. Clinical outcomes can be linked directly to financial outcomes. Insights can be generated within months rather than years. More importantly, the objective is not merely to evaluate healthcare. The objective is to improve healthcare continuously. Every patient encounter becomes an opportunity to learn something that can improve care for the next patient. 

Another theme emerged repeatedly in conversations at EHS. As surgeons are presented with an expanding number of technologies and manufacturers seek to demonstrate value, there remains no widely accepted, vendor-neutral framework for evaluating products in real-world practice. Every company presents its own studies, endpoints, patient populations, and methodologies. Value analysis committees and clinicians are often forced to evaluate data that does not apply to their own local clinical environment. We call these decisions evidence-based, but in many cases they are based on fragmented evidence generated through fundamentally different approaches, often with poor quality data. 

What is needed is a framework that can measure the entire patient process using real-world clinical and financial data collected at the point of care. Rather than isolating a device from the environment in which it is used, we should measure its performance within that environment. The value of any technology ultimately depends on how it interacts with patients, clinicians, workflows, and healthcare systems. Studying it in isolation may satisfy the requirements of reductionist science, but it tells us surprisingly little about its contribution to real-world value. 

This is the philosophy behind the Continuous Quality Improvement methodology we have developed at Caresyntax. By collecting longitudinal clinical and financial data during routine patient care, we can build continuously learning systems that generate insights and improve outcomes. The methodology has already supported regulatory submissions, post-market surveillance, indication expansions, and scientific publications. More importantly, it has demonstrated that healthcare systems can learn, adapt, and improve when they are measured appropriately. 

Randomized trials will continue to be used in healthcare for the foreseeable future. But they should no longer be viewed as the sole gold standard for understanding healthcare. Healthcare is not a laboratory experiment. It is a complex adaptive system. Understanding and improving such systems requires methodologies that embrace complexity rather than attempting to eliminate it. 

After more than three decades as a surgeon, healthcare leader, and student of systems science, I no longer ask whether we can afford to generate evidence differently. The question I now ask is whether we can afford not to.