Brief · Use case · Pharmacovigilance

Signal Detection Methods Comparison.

LiveComparison

Compare PRR, ROR and EBGM for signal detection on what each measures, thresholds, sparse-data behavior and regulatory use.

ReaderPharmacovigilance lead choosing a method

ItemsPRR, ROR, EBGM

Criteria4

ExpertHead of Pharmacovigilance

Not askedA recommended item

LengthStandard

Signal Detection Methods Comparison is a ready-made use case for Pharmacovigilance. It ships under the Comparison template.

For pharmacovigilance leads choosing a method. You get the methods and criteria stated first, one sourced table, where the methods differ and where thresholds are not comparable. Cells are cited or marked as gaps.

Add it to your library to reuse from New → From template. Edit the saved copy in your library on the service form.

The decision

How do the proportional reporting ratio, the reporting odds ratio and the empirical Bayes geometric mean compare for safety signal detection on what each measures, thresholds in use, sensitivity to sparse data and regulatory use?

What is compared

All three flag drug and event pairs reported more often than expected. They differ in how they compute the expectation and how they treat small counts.

Method	What it measures	Where its cells come from
Proportional reporting ratio (PRR)	Compares the proportion of a drug’s reports that concern one event with the same proportion for all other drugs, in a two-by-two table. Its original evaluation used three or more cases, a PRR of at least 2 and a chi-squared of at least 4.^{1Evans, S. J. W., Waller, P. C. and Davis, S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiology and Drug Safety, 2001;10(6):483–486.}	Evans, Waller and Davis, 2001, on UK Yellow Card data.
Reporting odds ratio (ROR)	The odds of an event among a drug’s reports against the odds among all other reports. A comparison of measures on a Dutch database found the measures broadly comparable once four or more cases per combination had accumulated.^{2van Puijenbroek, E. P. et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiology and Drug Safety, 2002;11(1):3–10. DOI 10.1002/pds.668.}	van Puijenbroek and colleagues, 2002.
Empirical Bayes geometric mean (EBGM)	Shrinks each observed-to-expected ratio toward the overall pattern using an empirical Bayes model, which is intended to steady estimates in tables where most cells hold zero or one report.^{3DuMouchel, W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 1999;53(3):177–190. DOI 10.1080/00031305.1999.10474456.}	DuMouchel, 1999, in The American Statistician.

What the comparison does

Thresholds as published. The comparison states the criteria each method’s authors proposed, and says that thresholds in use are often based on a lower confidence bound and a minimum number of reports. Where a threshold is a company’s practice and not a published rule, it says so.^{1Evans, S. J. W., Waller, P. C. and Davis, S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiology and Drug Safety, 2001;10(6):483–486.,2van Puijenbroek, E. P. et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiology and Drug Safety, 2002;11(1):3–10. DOI 10.1002/pds.668.,3DuMouchel, W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 1999;53(3):177–190. DOI 10.1080/00031305.1999.10474456.}

Sparse data. The EBGM was designed for tables in which most cells hold zero or one report, and the ROR and PRR comparison found broad agreement once four or more cases had accumulated. The comparison shows what each method does when counts are small.^{2van Puijenbroek, E. P. et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiology and Drug Safety, 2002;11(1):3–10. DOI 10.1002/pds.668.,3DuMouchel, W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 1999;53(3):177–190. DOI 10.1080/00031305.1999.10474456.}

Regulatory use. The European Medicines Agency’s EudraVigilance system reports the proportional reporting ratio for signal screening, and other measures may be used by marketing authorization holders. The comparison says what the guideline and the agency’s documents state, and what they leave open.^{4European Medicines Agency. Good pharmacovigilance practices (GVP) Module IX: Signal management, and Guideline on the use of statistical signal detection methods in the EudraVigilance data analysis system.}

A flag is not a finding. Disproportionate reporting does not show causation. The caveats say so and say what a flag still needs: clinical review and further evidence.^{4European Medicines Agency. Good pharmacovigilance practices (GVP) Module IX: Signal management, and Guideline on the use of statistical signal detection methods in the EudraVigilance data analysis system.}

Recommended length

Standard

Three methods on four criteria fit one table. Standard adds where they differ most in about 1,200 words.

You can pick another length on the form. The other two stay available.

Assumptions the comparison checks

The comparison states the assumptions that make the cells comparable and says where they fail. Examples for this subject:

The method is applied to the same database and the same coding as the cited evaluation.
The thresholds compared are published values, not company practice.
The regulatory documents cited are the current versions.

Standards, guidance and sources

The methods

Evans, Waller and Davis, Use of proportional reporting ratios for signal generation from spontaneous adverse drug reaction reports, Pharmacoepidemiology and Drug Safety, 2001.^{1Evans, S. J. W., Waller, P. C. and Davis, S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiology and Drug Safety, 2001;10(6):483–486.}
van Puijenbroek et al., A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions, Pharmacoepidemiology and Drug Safety, 2002.^{2van Puijenbroek, E. P. et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiology and Drug Safety, 2002;11(1):3–10. DOI 10.1002/pds.668.}
DuMouchel, Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system, The American Statistician, 1999.^{3DuMouchel, W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 1999;53(3):177–190. DOI 10.1080/00031305.1999.10474456.}

Regulatory practice

EMA, good pharmacovigilance practices, Module IX on signal management, and the agency’s documents on statistical signal detection in EudraVigilance.^{4European Medicines Agency. Good pharmacovigilance practices (GVP) Module IX: Signal management, and Guideline on the use of statistical signal detection methods in the EudraVigilance data analysis system.}

The decision practice behind the memo itself is on the template page.

What gets flagged

Any threshold that is company practice and not a published rule, any statement about regulatory use that the agency’s own documents do not support, and any claim of better performance without a published comparison.

What arrives

What is compared, and on what criteria
The comparison
Where the difference matters
Where they do not differ
Caveats on comparability

What it will not do

It does not recommend an item. A comparison shows how the items differ and where they do not; if you need a recommendation, run a Decision Memo on the same items.

It does not run a signal analysis on your data, and it does not decide whether a signal is real.

It does not recommend an item. A comparison shows differences; a Decision Memo recommends.

It does not handle more than five items. Beyond five, the table stops being readable and a Landscape or a Scan is the better template.

It does not invent a cell. A cell without a source is a gap, and the caveats say how many there are.

References

[1] Evans, S. J. W., Waller, P. C. and Davis, S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiology and Drug Safety, 2001;10(6):483–486. Read from the abstract as summarized in search excerpts.
[2] van Puijenbroek, E. P. et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiology and Drug Safety, 2002;11(1):3–10. DOI 10.1002/pds.668. Read from the repository record and search excerpts; the full text was not read.
[3] DuMouchel, W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 1999;53(3):177–190. DOI 10.1080/00031305.1999.10474456. Abstract only.
[4] European Medicines Agency. Good pharmacovigilance practices (GVP) Module IX: Signal management, and Guideline on the use of statistical signal detection methods in the EudraVigilance data analysis system. The link is the draft guideline on EudraVigilance; the current GVP Module IX revision was not found or read. Check both on the EMA site.

Links checked 9 October 2026

These are the sources the template drew on. None of their authors or issuers has reviewed it.

About the Brief service

A Brief is a written answer from one specialist, at the depth you asked for. It arrives with its sources, and it closes by saying plainly what would change the conclusion. Nothing is hidden behind a summary, and anything the expert could not source is marked as a gap rather than filled in.

To edit a saved template, open New → Brief → From template, or the library rail on the service form.