An Overview of Current Methodologies in Signal Detection

Posted by

Introduction 

Pharmacovigilance is defined as “the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects or any drug-related problem”. Before entering the market following marketing approval from a regulatory authority, the medicinal products undergo preclinical studies, Phase I, III and III clinical trials to determine safety and efficacy of the product. However, the patient population exposed to the product is limited and special population such as children, pregnant females and elderly patients are usually excluded from such trials. Although, during the clinical trials, adverse event data are collected and analyzed and due to the limitation of the patient population, the exact safety profile of the product cannot be obtained. For this reason, phase IV of the clinical trials commonly called a Post-marketing surveillance phase has been designed to get more data, especially regarding the safety of the product. When the product enters the market, a wider population is exposed to the product and the manufacturers as well as regulatory authorities around the world. Pharmacovigilance is about identifying such safety issues, or commonly called as safety signals, which could not be identified during the trial phase of the product. It is imperative that pharmacovigilance activities run throughout the lifecycle of the product. This is aimed for the effective collection, review, and evaluation of safety data to better understand the safety profile as well as calculate the benefit-risk analysis of the product1.

Signal detection 

According to the World Health Organization (WHO), a Safety Signal is defined as “reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously.” A more recent definition was given by the Council for International Organizations of Medical Sciences (CIOMS)-“information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify verificatory action” 2.

Objectives of this article

  • Discussion and review of signal detection methodologies.
  • Comparison of different signal detection methodologies and impact analysis of the same.

Methods

  • Review of literature publications, regulatory databases, and websites
  • Assessment of signal detection methodologies

Signal management process

Slide1

Sources of Signals3

  • Spontaneous data
  • Literature search
  • Databases such as FAERS, Vigibase, Eudravigilance etc.
  • Regulatory authority
  • Clinical trials
  • Animal toxicology
  • In vitro experiments
  • Post Authorisation safety studies

Methodologies for Signal Detection4

Qualitative method:

Case Series review: Review of spontaneous reports and other post-marketing Adverse Events (AEs) to identify safety concerns which can be found in a single case, a cluster of cases, case trends or cases which strongly support causality, positive de-challenge / re-challenge or close temporal association between the drug and the AE. Designated Medical Events (DME) and Targeted Medical Events (TMEs) are assessed in the case and case series review.

Simple analysis of large dataset: Simple analyses of line listings, cumulative tables, analysis of periodic safety reports, Periodic Benefit-Risk Evaluation Report (PBRERs), Annual Safety Report (ASRs), Periodic Adverse Drug Experience Report (PADERs) and Investigational New Drug (IND) Safety Reports, are commonly employed for signal detection.

Quantitative method:

Frequentist Methods: The PRR (Proportional Reporting Ratio) is a measure of disproportionality of reporting used to detect Signals of disproportionate reporting (SDRs) in pharmacovigilance. The PRR involves the calculation of the rate of reporting of one specific event among all events for a given drug, the comparator being this reporting rate for all drugs present in the database (including the drug of interest). The ROR (Reporting Odds Ratio) is the ratio of the odds of reporting of one specific event versus all other events for a given drug compared to this reporting odds for all other drugs present in the database. A signal is considered when the lower limit of the 95% confidence interval (CI) of the ROR is >1.

  • Proportional Reporting Ratio (PRR) 

Advantages: Easy to compute, assess, understand and implement. More sensitive than Bayesian method

Limitations: Low specificity. Cannot be calculated for all drug-event combinations

  • Reporting Odds Ratio (ROR)

Advantages: Easy to compute, assess, understand and implement. More sensitive than Bayesian method. Logistic regression analysis with covariates possible.

Limitations: Odds ratios cannot be calculated if denominator is Zero. Lower sensitivity.

Bayesian Methods

  • Multi-item Gamma Poisson Shrinker (MGPS): MGPS is the most widely used method and it provides a singular example of large-scale Bayesian shrinkage in routine use by regulators and pharmaceutical manufacturers worldwide. The process makes use of Reporting Ratio (RR) and this method is used by FDA.

Advantages: Applicable to all Drug-event combinations. More specific than frequentist method

Disadvantages: Difficult to compute, assess, understand and implement. Lower sensitivity.

  • Bayesian Confidence Propagation Neural Network (BCPNN): The process makes use of Bayesian statistical principles to quantify apparent dependencies in the data set. This quantifies the degree to which a certain drug- ADR mixture is distinct from a background. This method is used by WHO.

Advantages: Applicable to all Drug-event combinations. More specific than frequentist method. Can be applied to pattern recognition.

Disadvantages: Difficult to compute, assess, understand and implement. Lower sensitivity.

Contingency Assessment (Computation of PRR)

Widely used method for its ease to compute, understand and implement.

Event (E) All other events Total
Medicinal Product (P) A B A+B
All other medicinal products C D C+D
Total A+C B+D A+B+C+D

 

Piccc

  • The value A indicates the number of individual cases with the suspect medicinal product P involving an adverse event E.
  • The value B indicates the number of individual cases related to the suspect medicinal product P involving any other adverse events but E.
  • The value C indicates the number of individual cases involving event E in relation to any other medicinal products but P.
  • The value D indicates the number of individual cases involving any other adverse events but E and any other medicinal products but P.

Conclusion4,5,6

  • Statistical signal detection approaches have been developed and applied in the field of drug safety surveillance-adding to the toolkit of pharmacovigilance professionals.
  • Determining a signal right approach for signal detection methodologies have always been a challenge for most MAHs. This dilemma includes whether to select Qualitative method vs Quantitative or selecting Frequentist vs Bayesian methods.
  • PRR is more sensitive than MGPS. MGPS gives better results when the number of cases reviewed is small.
  • Qualitative method of signal detection is more conservative and cost-effective: However, the major limitation of qualitative signal detection is that it can only be applied to a small data set. When there is a large data set, the quantitative signal detection method gives better results.
  • There are many factors to be considered by MAHs while deciding the method of signal detection to be used e.g. Quantitative or Qualitative. These factors are cost, workforce required, number of molecules, authorization countries, and local regulations.
  • The data derived from statistical signal detection methods should be considered with caution and guided by appropriate clinical evaluation. This clinical perspective should always be considered to support appropriate drug use, balancing drug effectiveness, safety and, above all, actual patients’ needs.
  • Although many statistical methods and algorithms are designed to aide MAHs for better signal detection, medical and scientific assessment and review done by physicians is still a cornerstone for better causality assessment and for detecting potential safety signals.
  • Further development of statistical methods and technological solutions to analyze large amounts of data (‘Big Data’) to detect signals for potential safety issues, while minimizing noise, would enhance the efficiency and effectiveness of pharmacovigilance activities.
  • The biggest challenge in signal detection is to minimize noise (false signals) and simultaneously not missing out on potential safety issues. Further refinement of signal detection methodologies can address this challenge, however, the cost for implementation of such statistical signal detection tool remains a challenge for small scale MAHs.

References

  1. Meyboom RH, Eqberts AC, Edwards IR, Hekster YA, de Koning FH, Gribnau FW. Principles of signal detection in Pharmacovigilance. Drud Saf. 1997 Jun;16(6):355-65.
  2. World Health Organization. The importance of pharmacovigilance – safety monitoring of medicinal products. World Health Organization, Geneva, 2002.
  3. CIOMS Working Group VIII. Practical aspects of signal detection in pharmacovigilance. Council for International Organizations of Medical Sciences (CIOMS), Geneva, 2010.
  4. CIOMS Working Group IV. Benefit-risk balance for marketed products: evaluating safety signals. Council for International Organizations of Medical Sciences (CIOMS), Geneva, 1999.
  5. Guideline on good pharmacovigilance practices (GVP) – Module IX (Rev 1)
  6. FDA- Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment

Authors’ Profile

Mihir.JPG Mihir Sadadia, Senior Safety Surveillance Physician – Pharmacovigilance at FMD K&L

Dr.Mihir Sadadia is a physician with an MD in Pharmacology. He has over four years of experience in Pharmacovigilance. In his previous company, he was responsible for signal detection and management, medical review of aggregate reports and ICSR and leading a team of pharmacovigilance associates. He is currently a senior safety surveillance physician in iMEDGlobal (an FMD K&L company) providing his expertise in product safety surveillance and risk management for client needs.