Reporting bias undermines the integrity of the evidence base by inflating apparent drug efficacy and minimizing drug harms, thus skewing their risk-benefit ratio. This talk will review the topic of reporting bias with a focus on drugs prescribed for psychiatric conditions, especially depression, schizophrenia, bipolar disorder, and autism. Reporting bias is pervasive—although psychiatry/psychology may be the most seriously afflicted field, it occurs throughout medicine and science.
Responsibility lies with various parties—authors as well as journals, academia as well as industry—thus the motives appear to extend beyond the financial interests of drug companies. The desire for success, in combination with cognitive biases, can also influence academic authors and journals. Amid the flood of new medical information coming out each day, the attention of the news media and academic community is more likely to be captured by studies whose results are positive or newsworthy.
In the peer review system, a fundamental flaw arises from the fact that authors usually write manuscripts after they know the results. This allows hindsight and other biases to come into play, so data can be “tortured until they confess” (a detailed example is given). If a “publishable” result cannot be achieved, non-publication remains an option.
To address reporting bias, various measures have been undertaken, including registries. Drug regulatory agencies may play a more prominent role. It is suggested that journals borrow from the FDA review model. Because the significance of study results biases reviewers, results should be excluded from review until after a preliminary judgment of study scientific quality has been rendered, based on the original study protocol. Protocol publication can further enhance the credibility of the published literature.