Examining statistical methods that leverage the hierarchical structure of adverse events for signal detection in randomised controlled trials

New drugs developed to treat medical conditions are evaluated through clinical trials. During a trial, we collect data on the side effects experienced by the trial participants (researchers working on trials call these "adverse events"). We analyse the data collected on side-effects to establish whether the new drug may have caused these effects. It is not possible to know in advance what side effects might happen during the trial, and so we collect a lot of different pieces of information about them, such as when they happened, how often or how severe they were. As a result, we have datasets rich in information about drug side-effects but most of the information collected does not get used in the analysis. Researchers often present the data in simple summary tables and ignore issues like multiplicity (conducting multiple hypothesis tests, which increases the chances of making wrong conclusions). New statistical approaches that make better use of this data are available: these methods group adverse events based on the body system that they relate to (e.g. the events "nausea" and "vomiting" would be grouped together in the gastrointestinal system), instead of considering each event individually. But there is a lack of evidence about which of the methods work best and therefore, none of them are being used in practice.
Improving the statistical analysis of adverse events data will help provide a clearer picture of the safety profile of a drug, which allows doctors and patients to make more informed treatment decisions. It can also help researchers identify serious side effects before a drug is offered to the public.
We want to understand how these new statistical methods work on real-world data, so we can make recommendations to statisticians working on trials. We previously conducted a review of the scientific literature and identified 18 statistical methods. We evaluated some of these methods in a simulation study, which tested the new methods on 'made-up' data that reflected different trial scenarios (e.g. on trials of different sizes) and compared their performances. We now plan to test some of these methods on real-world data from completed trials. We will focus on trials that evaluated one of two drugs: Daclizumab for the treatment of multiple sclerosis and Xeljanz for the treatment of rheumatoid arthritis. We chose these two drugs because they were both authorised for patient use after being evaluated in clinical trials, and were offered to the public before the European Medicines Agency (the body which regulates new drugs in Europe) started an investigation due to suspected links to increased risks of serious side effects, such as skin cancer or serious inflammatory brain disorders.
We aim to re-analyse the adverse events data from these trials using these new statistical methods, to investigate whether the serious side effects could have been detected earlier if we had used better suited analysis approaches. For each trial, we will compare the different methods ability to detect these known side effects, and compare our results with the results obtained in the original analyses.