AI in Adverse Event Detection and Reporting

Good adverse event (AE) detection and reporting are crucial components of any validated, regulated, safe (and frankly, well done) clinical trial. Not only is it a requirement, it is one that is of utmost importance.

In this post, we’ll talk about how we can leverage Artificial Intelligence (AI) to automate some of the key tasks around AE detection and reporting… well, really reporting (stay tuned to see why!).

First, it’s important to note that AE capture and reporting is not only required for regulatory compliance, it’s critical for running a safe and ethical trial. For that reason, we at ReadoutAI don’t deploy models that make any important medical decisions such as the “severity” of an AE (note that AEs are graded by severity, usually according to strict metrics), the “treatment relatedness” of an AE (if an AE is believed to be caused by treatment, it must be reported), or determining whether an AE is detected (e.g., using AI to read a medical encounter and automatically pulling out the AEs mentioned). These are all tasks we leave to the medical professionals. Hence, we don’t do any AE capture.

Pictureb

That being said, once the AEs are collected, there are multiple, tedious workflows for analyzing and reporting them. Those workflows all benefit from AI, improving not only the efficiency of analysis, but also helping people in biostatistics and investigators stay informed about the safety of their trial.

AE Reporting with ReadoutAI

At ReadoutAI, we focus on workflows. That is, rather than just to a part of a task, we try to automate the whole thing – pick the data, analyze it correctly, and create the correct output. In the case of AEs, there is actually a prescribed format for how to report the Adverse Events within a Clinical Study Report (CSR). You can see the ICH E3 format here!

Fortunately, since AE reporting is standardized and the analyze is straightforward (even if tedious and data intensive), it’s a ripe opportunity for AI. So we’ve automated it! Formulating an ICE E3 compliant AE assessment/report boils down to 5 steps:

  1. Map the terms that describe the AEs to a standard vocabulary. That is, make “GERD” and “Esophagus Reflux Disease” refer to the same thing (“Gastroesophageal Reflux Disease”). The most common way to do this is to find a match in the MedDRA standard vocabulary of adverse events. This is important to make sure the counting (below) is correct.
  2. If there are multiple cohorts in your data, group the standardized AEs by cohort (e.g., group A and group B, counting the AEs for each separately)
  3. Count the standardized AEs for a Summary Report – how many were serious? How many were related to treatment?
  4. Do the analysis for the Detailed Report – for this part, you compute the relative risk of the AEs from one cohort to another. If you have more than 2 cohorts, pick a cohort as your “anchor” and always use that anchor for your relative risk, for consistency. So, if you have cohorts A, B, and C – then you have relative risk between A and B and relative risk between A and C
  5. Report the findings (write up a nice narrative, create your tables, etc.)

Here’s what all that looks like in ReadoutAI! Below you’ll see the ICH E3 recommended Safety Summary and Safety Detail report sections, as well as the automated AE coding.

Intro To Readout Latest

What’s especially cool is that this whole process (steps 1-5) takes about 5 minutes. If you (generously) assume another 30 minutes of human QC on the AI results, the whole thing is done in about 35 minutes. Compare this to the nearly 3 hours (170 minutes) for the same steps to be done, by hand (which we did, for a recent study). The time savings are about 80% – or put another way, more than 2 hours! And if you relax your QA, it’s more like saving 97% of the time!

This unlocks two huge benefits. First, it’s rapid: imagine the compounded savings across hundreds of studies. Second, because it’s so fast (and easy), you can start to do it whenever you want! If you are more generous on the QC side, and it only takes 5 minutes, why not run it at various points in the trial for the whole cohort? Or, you could keep the participation blinded since the AI is flexible enough to let you define the cohorts however you want – for instance, you could split participants by Sex or Age Grouping to see how AEs are shaping up.

Picture1

The key is that the speed of AI makes this task a much less onerous hurdle, so it can be used more frequently.

What Might be Next for AI in AE Reporting

While this workflow benefits hugely from the ReadoutAI platform, there are other areas we’d like to explore next.
One potential future direction is the bi-directional implications for the Concomitant medications (ConMeds) reporting. ReadoutAI can do ConMeds reporting. Just like AEs, our platform maps the medications to a standard vocabulary (WHODrug in this case), does the analysis, and writes up the report. However, an interesting future direction is that if we see a certain AE, the AI should double check for the existence of a corresponding entry in ConMeds! That is, certain drugs imply certain AEs occurred (and vice versa) and so those two reports can be resolved to identify missing data.

Anyway, hopefully this was a helpful introduction into the world of AI automation in Adverse Event reporting workflows. As always, please feel free to drop us a line if you’d like to talk more!