GUEST POST from Arlen Meyers, M.D.

For digital health entrepreneurs, unless your intended use puts you in the FDA category of a medical device, you don’t need to show that your product is safe and effective, let alone cost-effective. In most cases, rather, you need to demonstrate to investors that it can quickly scale and make money and the sooner the better. Too bad. You would think that whether you have to or not that prudent sick care business practices would mean creating a product that does what you say it will do. That’s why only a handful of the hundreds of thousands of digital health apps are clinically valid.

However, most sick care digipreneurs avoid costly, risky trials because they are afraid of possible negative results that could be the death blow of their company, like many early stage biotech or device companies that wake up to see their valuations plummet due to a failed trial.

Digital health regulation is changing rapidly as the FDA tries to keep up with the pace of change and new products and services. On FDA regulation, the rule—for now, at least—is clear: Any device that is “intended for use in the diagnosis of disease of other conditions, or in the cure, mitigation, treatment, or prevention of disease” requires FDA approval, Curtis said. This goes for devices meant for humans and animals, as the FDA regulates both.

The emergence of digital therapeutics, the incorporation of AI and the adoption of remote sensing is challenging us how to demonstrate safety and efficacy, if not cost-efffectiveness.

For example, there’s been an explosion in the number and variety of digital apps purporting to address behavioral health issues, but a recent study published in Nature Digital Medicine casts doubts on their backing by legitimate scientific research. Based on the literature review conducted by the researchers, only 14 percent of apps described design or development that was based on real-world evidence.

One commentary to a recently published article on digital health trials noted that “The pipeline of digital health studies appears to be promising,” noted the researchers, but they also underscored that the small sample sizes in many studies “could limit their ability to yield a high level of evidence, demonstrate value, or motivate stakeholder adoption.”

Digital health technologies hold great promise to solve some of the biggest problems in our healthcare system, including achieving higher quality, lower cost, and greater access to care. a better doctor and patient experience and efficiencies in business processes. In the January 2019 issue of Health Affairs, reported that scant evidence exists demonstrating the clinical impact of twenty top-funded digital health companies. These companies tended not to study the clinical effectiveness of their products in terms of key healthcare metrics like patient outcomes, cost, and access to care.

They found 104 peer-reviewed published studies on the products or services of these companies. The majority of the studies were from three companies. Nine companies had no peer-reviewed publications. Only 28% of the studies targeted patients with high-burden, high-cost conditions or risk factors. Healthy volunteers were the most commonly studied population. Further, 15% of all studies assessed the product’s “clinical effectiveness” and only eight studies assessed clinical effectiveness in a high-cost, high-burden population. The eight clinical effectiveness studies measured impact in terms of patient outcomes, while no studies measured impact in terms of cost or access to care. There were no clinical effectiveness studies in heart disease, COPD, mental health conditions, hyperglycemia, or low back pain. Studies that did not assess clinical effectiveness may have intended to validate the product against a gold standard measurement or report feasibility of use.

This is of particular interest given the incredible amount of funding, interest, and hype in digital health. Although these companies were only a small portion of total digital health companies, they were a large portion of total private funding and had the most resources to demonstrate impact. Further, since “digital health” currently encompasses myriad technology types and approaches, these findings have broad implications.

From text messages to mobile apps, digital health devices are becoming increasingly important in clinical trials for their ability to streamline trials, lower site burden, and improve the patient experience. However, manufacturers must consider the safety, reliability, and convenience of these devices in order to effectively implement them into medical device trials. The digital components of these medical device trials must adhere to the same rigorous regulatory standards as the device itself, which can pose significant hurdles for some sponsors. Those hurdles include:

• Usability and Accuracy. Sponsors must be able to determine that a device is providing the desired endpoint values in a trial. The metric should be accurate and presented in a usable format.
• Safety. The manufacturer should be able to provide highly secure methods for transmitting data between the digital health device and the analysis site.
• Convenience. The manufacturer should be able to provide logistical support to decrease the site and subject burden. They should also be able to provide full documentation of engineering verification for the devices.
• Ease of Use. It is important to consider how the patient will interact with the device. It needs to be an appropriate size and weight, and it should allow the patient to move and behave in the same way he or she normally would.
• Reliability. To maintain data continuity, the device should have a battery life sufficient to allow it to collect data for long periods of time with minimal glitches.

In another study of digital health apps for COVID 19, apps were evaluated using the Systems Wide Analysis of mobile health-related technologies (SWAT) tool in line with the NHS Digital Assessment Questionnaire and were given a score for each category (usability, functionality, ethical values, security and privacy, user-perceived value, design, and content) by two independent assessors.

A recent review concluded that “Safety of apps is an emerging public health issue. The available evidence shows that apps pose clinical risks to consumers. Involvement of consumers, regulators, and healthcare professionals in development and testing can improve quality. Additionally, mandatory reporting of safety concerns is needed to improve outcomes.”

The concept of a “digital clinical trial” involves leveraging digital technology to improve participant access, engagement, trial-related measurements, and/or interventions, enable concealed randomized intervention allocation, and has the potential to transform clinical trials and to lower their cost. In April 2019, the US National Institutes of Health (NIH) and the National Science Foundation (NSF) held a workshop bringing together experts in clinical trials, digital technology, and digital analytics to discuss strategies to implement the use of digital technologies in clinical trials while considering potential challenges. This position paper builds on this workshop to describe the current state of the art for digital clinical trials including (1) defining and outlining the composition and elements of digital trials; (2) describing recruitment and retention using digital technology; (3) outlining data collection elements including mobile health, wearable technologies, application programming interfaces (APIs), digital transmission of data, and consideration of regulatory oversight and guidance for data security, privacy, and remotely provided informed consent; (4) elucidating digital analytics and data science approaches leveraging artificial intelligence and machine learning algorithms; and (5) setting future priorities and strategies that should be addressed to successfully harness digital methods and the myriad benefits of such technologies for clinical research.

But, when it comes to human subject pilots and trials, there are several glaring differences of digital health trials compared to drug and device trials:

1. Intellectual property concerns are usually lower in digital trials
2. Digital trials need to be done faster because the markets change so quickly and the barriers to entry are lower
3. IRBs set up to protect patients in drug and device trials are not comfortable with digital health trials
4. We are still trying to define the ethics of data science
5. Cybersafety is as important as patient safety
6. Digital health clinical trial ecosystems are primitive compared to drug and device trial ecosystems although both have recruitment and completion challenges
7. Data trials need mostly patient data, not the physical presence of patients
8. HIPAA rules get in the way
9. Lack of interoperability and getting data from non-traditional hospital based HIT systems skews the data and , consequently, the results
10. Funding sources for conducting digital health trials are scarce. Investors don’t fund research projects. They fund product development and marketing. Here are some ideas on how to convince your CFO to pilot, test and integrate your IT solution.

The FDA has stated their position when it comes to the regulation of mobile medical apps.

Digital health has evolved because:

1. Interoperability is improving
2. Sick care is turning into health care
3. The medical business model is changing
4. Costs continue to spiral out of control
5. The recognition by digipreneurs and investors that sick care is ripe for digitization
6. Patient and provider frustration with a lousy experience
7. Cheap mobile and internet technologies
8. Regulatory, IP and reimbursement changes
9. The breakdown of barriers to diffusion and implementation
10. Digital heath ecosystems

An international consortium of medical experts has introduced the first official standards for clinical trials that involve artificial intelligence. The move comes at a time when hype around medical AI is at a peak, with inflated and unverified claims about the effectiveness of certain tools threatening to undermine people’s trust in AI overall. 

Collaboratively developed guidelines for the privacy, content, security, design and operability of mobile health (mHealth) apps have been released. Compliance with the guidelines can provide a level of assurance that an app delivers value to patients, physicians and other users.

The guidelines were developed by Xcertia, a nonprofit founded by the AMA and other major health and technology organizations. They address concerns that have hindered the use of mHealth apps. Fears that an app may expose personal health information, that its content is inaccurate or that its functionality is limited have slowed adoption of mHealth digital health tools.

These advances are permeating all aspects of clinical research but the recent acceleration of decentralised and hybrid clinical trials (DCTs) illustrates how far reaching digitalisation is becoming. Patient-centricity has been driving the decentralisation of clinical trials for some time but the rapid emergence of the COVID-19 pandemic required the pharmaceutical industry to pivot operations and accelerate its DCT programmes. This response required a corresponding ramp-up in mobile technology, data management and AI.

AI can provide insight into protocol complexity and contribute to protocol designs better adapted to DCTs, including the creation of virtual control arms. The intelligent use of data to include historical data as well as the data collected during a clinical trial can optimise the number and diversity of patients needed to reach the desired endpoints and give the patients who do participate a higher value experience.

Until and unless we address ongoing digital health clinical trial issues , sick care digipreneurs will have little or no incentive to conduct digital health clinical trials using precious startup funds. Instead, they will continue to sell snake oil and lots of folks will buy it. Maybe you should sleep on it.

Image Credit: Pixabay

Sign up here to get Human-Centered Change & Innovation Weekly delivered to your inbox every week.