Digital health startups may not want to do randomized trials, but they need to

The general theme of digital health care is easy to grasp — maybe too easy. Somewhere, right now, a hopeful CEO is making a pitch to a potential investor about how its new digital product or service will transform health care and improve outcomes, reduce costs, and save time.

The story works. Digital health care startups are attracting billions of dollars in risk capital, with aggregate investments reaching $7 billion so far in 2018, all in the hopes of modernizing our broken health care system.

While the story is easy to tell, proving it poses a far greater challenge.

As the CEO of a company that develops products for virtual physical therapy rehabilitation, I know this from firsthand experience. About 30 months ago, my company, Reflexion Health, partnered with Duke Clinical Research Institute to conduct the first large, prospective, randomized controlled trial comparing virtual physical therapy using our FDA-cleared Virtual Exercise Rehabilitation Assistant (VERA) with traditional physical therapy.

We took this irrevocable and consequential step because we, and our investors, believe that a prospective randomized controlled trial provides the highest form of clinical and economic evidence — evidence that could help drive adoption. Without such rigor, it is easy to cherry-pick results from the clinical and economic experiences of early adopters to construct an overly optimistic view of a new innovation.

We undertook the trial with understandable trepidation. Randomized controlled trials aren’t a complete roll of the dice. But what looks good in the clinic with clinicians friendly to the technology doesn’t always stand up in an independently conducted trial.

It was a long 30 months. I can’t tell you how gratified we were to learn that, as we are announcing Monday, VERA saved nearly $3,000 per patient; the participants enjoyed the convenience of an at-home, on-demand platform that delivered outcomes that were comparable to those with face-to-face therapy and preferred recovering at home to traveling to see a physical therapist; and health care providers knew exactly how their patients were doing because they could follow them through remote monitoring technology.

But it was far from easy. Conducting a truly robust clinical trial is a huge burden for any startup that is also trying to simultaneously and judiciously allocate time, money, and resources to building a business.

Here are the challenges we encountered along the way. I suspect they apply to many companies:

High upfront cost. Carefully conducted randomized controlled trials are expensive. In our case, the direct costs alone exceeded $2 million. As any health tech entrepreneur would attest, that is a significant investment for a startup company.

Loss of control. It was entirely possible that the outcome of the trial could have been negative, potentially killing the company and maybe even the category. While “failing fast” is a startup mantra, no one actually wants to do it out loud, with an audience. A federally registered, well-publicized randomized controlled trial conducted by the world’s largest academic clinical research organization is a high-wire act.

Prolonged timelines. Recruitment for most clinical trials takes longer than expected. So does the rest of the timeline. And the cost often exceeds the initial estimate. These are huge pain points for any startup with tight budgets and aggressive timelines.

Lack of alignment between academia and startups: Academic timelines (often expressed in semesters) have a different sense of urgency than timelines for small companies managing monthly burn rates and quarterly reporting schedules. And while publishing trial findings in a highly regarded medical publication, like JAMA or the New England Journal of Medicine, can offer an imprimatur of validity, such journals have stringent requirements on how and when data are reported, with lengthy submission, review, and embargo periods.

In the oncology space, a recent analysis of the time between the press announcement of top-line data and scientific publication of complete data averaged about 300 days — that’s almost one year until the full data reached the public. That’s a lifetime for fast-paced businesses that need these data to make sales.

Competitive considerations. Although we hoped that potential buyers would firmly link our hard-won results with our proprietary physical therapy solution, it’s entirely possible that our positive result could have an unwarranted halo effect on the offering of our competitors, sort of a “rising tide lifts all boats.” That’s an important consideration when plunking down millions of dollars for a study designed to get a return on your company’s investment.

When comparing benefits and concerns, it is not difficult to see why most digital health care startups don’t pursue the rigorous validation of a well-done randomized clinical trial. But without such compelling proof, it’s easy to see why some digital health offerings experience a skeptic’s reception, a slow pace of adoption, or even commercial failure with an otherwise valuable solution.

One of the big promises of digital health is the speed with which it can transform health care delivery. The FDA has recognized that a different regulatory process is required for digital health solutions, signaling that regulation may be willing to move at the pace of innovation. But the full impact of digital health innovation is seen only with broad adoption of truly valuable solutions, and that rightfully requires evidence. Yet the best way to gather that evidence — the carefully conducted, prospective, randomized controlled trial and its subsequent publication — is currently ill-suited for the typical digital health startup.

While we are fortunate that our trial hit its primary endpoints of cost savings and clinical efficacy, this message would still be true even if it hadn’t: In digital health care, we must continue to streamline and accelerate the process for collecting definitive proof of value. Companies need to educate clinical research organizations at the start about their business needs and timelines. Those research organizations need to design new approaches to conducting clinical trials in fast-moving markets. And top journals need to offer more viable alternatives to nearly year-long timelines and prohibitive embargoes.

Digital health companies must not shrink from demanding proof of value. But we must also innovate the process of developing and disseminating such proof if we are to realize the full value of digital health.

Joseph Smith, M.D., is CEO of Digital Health Corp and Reflexion Health Inc.