Some Considerations for Using AI in Your Clinical Trial

AI is (rightly) a trending topic. It’s an amazing technology with the ability to make all sorts of processes more efficient – analyzing data for insights, writing reports, matching patients to trials, finding data errors, and more. Yet, there are several issues to consider before implementing AI technology for your workflow or across your company.

Here we’ll examine a few of those issues, with a viewpoint of implementing an AI product in your clinical trial workflows.

How Much Data?

Some AI tools require training data to work optimally. In some case you would show the algorithm examples of what you want it to do, and it will learn to do them on your behalf. For instance, you might want an AI product that classifies staging for rashes for your specific skin condition in your trial (as an efficacy measure). You show it all sorts of examples of rashes and stage classifications and deploy this on an app at sites. This learning is “online” because the AI system you are using is the one you are training with your specific data.

An alternative approach is to learn “offline” (e.g. independently of a specific customer’s data) and apply the model to the customer. This is how ReadoutAI works – our models are built using our own data and approach and leveraged by our customers. We update our models ourselves. It’s like Google Translate – you didn’t give them data to translate, rather they built a translation model that you can then use.

The main differences are that an approach like ours (“offline”) is easy to integrate and can start working immediately. However, it won’t work for all use-cases: it won’t work if to identify the stages of rashes for that specific skin condition, for instance.

How Does it Integrate?

Similarly to data involvement, there is workflow involvement for an AI product. This tends to have two flavors.

The first consideration is how the AI integrates with existing technology. Tools like ReadoutAI follow a “light” integration policy – you can tie your data into our platform as lightly or deeply as you want (for instance, you can export data from your EDC to pass it in, as a light integration, or you can directly connect your EDC to ReadoutAI with custom-built piping, which is a heavy integration as now the products are intimately combined). The flexibility of light integration is important, but it also means that you often have a few more steps to use the AI than a heavier integration (e.g., someone needs to export from the EDC first).

The heaviest integration are tools that may require custom training data linked directly from one of your internal systems. Imagine slotting patients to trials with AI, where it would require hooking into an EHR, ingesting a protocol, matching patients and then contacting them. You can imagine an AI like this requiring many custom tools to connect all of this together and now that AI product is heavily linked to those products, requiring a heavy lift if you want to change products in the combined set.

However, beyond the technology integration, there is also a people and workflow integration. At ReadoutAI, we believe firmly that all integration should be light. It shouldn’t require specific hires to make technology work and workflows should be the same as they were before the AI was introduced.

In other words, the AI should run in parallel or easily slot into existing workflow, with just a bit of training, rather than requiring full, new workflows and new hires just to staff the new functionality. This is an important consideration for the AI business case. What good is getting a manual-shift sports car for yourself, if you can’t drive it without hiring a race-car driver?

How To Measure Success?

Once all the data and workflow requirements are squared away, the next consideration would be how to measure if you’re getting value from the AI! If you don’t have a target, you won’t have anything to measure against.

This really varies by the workflow the AI is chosen to support.

For ReadoutAI, we measure time savings and cost savings while considering correctness. In other words, the AI should produce the same results as the person would but do it more efficiently – saving everyone very significant time and cost. Then, you can measure that savings against the cost of the AI and produce an ROI (we are happy to share our worksheets for this, and you can start a free trial and see for yourself!).

Sometimes, the AI is doing something no human can do! For instance, maybe it is built to identify brand new protocols and sites for those protocols. In this case, it’s may be less crisp how to measure success, but it’s still doable. Maybe it’s a hard measure of success – did the protocol produce a reasonable clinical result? Or maybe it’s softer than that – would anyone have conceivably thought of that protocol or how many hours would it have taken a human to make a similar consideration? Even when it’s not as well defined, it’s still helpful to have some framework, ahead of time, to decide if the AI will even be helpful.

Are You Ready?

The first three considerations are well defined – what requirements does the AI product have for data? What workflow and personnel requirements are there? And, how do we know if the AI is really helping?

The last consideration we’ll discuss is more nebulous but no less important. Is the organization or team ready for an AI product?

This mostly hinges on communication. Has everyone discussed how the AI will fit into existing workflows? Is everyone on board with at least trying the new technology? Is the data ready (if needed)?

Since the key is communication, if the product is from a third-party, make sure you’ve had enough conversations (pitches, training, etc.) so you can start the project with confidence. If it’s an internally developed AI product, make sure the product owner or engineers are available. You might also consider regularly scheduled check-ins to discuss the measured success (with the vendor, if possible!). And finally, ask questions – make sure you understand the technology and all of your concerns are answered.

There are lots of issues to consider when implementing an AI product. We’ve outlined a few here, but there are always more! If you’d like to talk about this topic or anything else with us, we’re always happy to hear from you.