Whether testing Yuma cabbage or assessing COVID-19 vaccines and treatments, the FDA must make safety judgements while remaining responsive to new reports.

But research published in the medical journal the BMJ suggests the evidentiary bar for making those rulings might be too low — or, at least, too opaque.

Each year, the FDA receives reports of possible problems with food or drugs via its Adverse Event Reporting System (FAERS). Report volume rose over the past decade from 780,000 in 2011 to more than 2.3 million in 2021.

One way the agency navigates the deluge is through the Sentinel Initiative, which proactively searches for potential harms by data-mining aggregated insurance claims and electronic health records.

These post-release surveillance programs provide a needed safety net for catching problems unlikely to crop up during clinical trials, such as rare allergic reactions or responses that only emerge following long use.

The FDA’s decisions affect treatments, prescriptions and insurance coverage, so making needless changes can cause harm just as not making needed changes can.

FAERS data from 2008-2019 shows the agency resolved most safety issues after three or more years (the FDA Amendments Act says the agency must publicly report potential safety reports, but sets no strict deadline) and that 80% led to FDA regulatory action — mainly product label changes.

But a closer look at a subset of data from 2014-2015 revealed less than one-third of the agency’s regulatory actions were backed by published research and none was corroborated by Sentinel Initiative assessments.

Moreover, when the FDA did use published studies from the medical literature, they tended to consist of case reports or case series, which are often small, anecdotal and hard to generalize.

The paper concludes by proposing either the FDA bases its actions on non-public evidence or the agency needs to adopt more robust and comprehensive safety assessments.

The authors acknowledged that evidence requirements can vary, that designing a study for every report is infeasible and that the FDA has access to data that may go unreported.

But they also emphasized the need for better transparency to protect consumer safety, maintain market stability and reinforce public trust.