Great progress has been made in obtaining reliable evidence on the beneficial effects of interventions, but developments in the identification, interpretation, and reporting of harmful effects is more challenging. Randomised controlled trials are the best way to evaluate small to moderate effects of healthcare interventions, and much of the evidence for benefits from treatment comes from such studies. However, they are not always suitable to evaluate harms, and this was made clear during a recent meeting jointly organised by the Cochrane Collaboration and BMJ Knowledge in London.

There are various problems with randomised controlled trials in relation to harms and some of these problems affect systematic reviews too. Firstly, trialists may know which benefits to assess but may be unaware of potential harms of the interventions they are testing. Identifying unexpected harms is difficult when the delay between the intervention and the onset of side effects is long or when a cumulative exposure is necessary to trigger the harms. Harms may be measured or grouped differently among trials, making it almost impossible to summarise, aggregate, or interpret the evidence in meaningful ways. The debate about the potentially serious cardiovascular effects of cyclooxygenase-2 (COX 2) inhibitors illustrates some of these problems. Serious cardiovascular effects associated with the use of COX 2 inhibitors have been identified recently^3,4 because they were not systematically searched for in previous trials.⁵ All this can lead to harmful drugs continuing to be used for many years before a warning is raised.

Problems exist with detection also. Rare harms may turn out to be more common than anticipated once flagged, but providing effective and balanced information to doctors and the public may be a complex and lengthy process. Even if the information is collected it might not be reported or indexed consistently well.⁶

Adverse effects can also be confused with the symptoms of the condition being treated. People taking analgesics for headache may develop analgesic induced headaches.⁷ Until this was discovered people with migraine might have thought their condition was getting worse, increased the amount of analgesics they took to compensate, and found themselves being exposed to even more of a harmful treatment.

Raising the alarm about a potential harm can also do more bad than good if the quality of the evidence or its reporting are poor. When insufficient data are available to ascertain the size of the problem for an intervention and for any alternatives, people may end up worse off. They might be deprived of an intervention that is on balance more beneficial than harmful and left with an alternative that might be worse. For example, misoprostol has been found to be an effective, cheap, and accessible way of inducing labour in the third trimester. However, controversy over its use has arisen because of potential harms of this prostaglandin analogue, including uterine rupture. Dinoprostone, the alternative prostaglandin, is more expensive, and what is more worrying is that the evidence proving that it is safer than misoprostol seems to be unavailable.⁸

Delegates at the recent meeting named a range of other sources of evidence of harms such as databases, observational studies, and specialised publications, but these have problems of their own. Databases are difficult to maintain and are not broadly accessible. Moreover, it is difficult to know how many people have been exposed. Reports from patients and case reports are susceptible to bias. Specialised publications are a very useful source of information,⁹ but share some of the limitations of databases.

Emotional and psychological barriers are also involved in the reporting of harms. People and organisations may have competing or vested interests, or come under pressure to take a lenient approach. Reporting harms may cause more trouble and discredit than the fame and glory associated with successful reporting of benefits. Our blame culture offers few incentives for reporting harms, and little gratitude is to be expected by a healthcare provider or an institution reporting that the interventions they offered were harmful. Such a declaration could lead to criticism, legal liability, withdrawal of funding, and stigmatisation.

Some of the problems with using randomised trials to assess harms can be solved, but many cannot be. The recent meeting showed that pursuing collaborative multidisciplinary initiatives involving clinical epidemiologists, pharmaco-epidemiologists, biostatisticians, and medical writers among others, are promising. The meeting delivered proposals to improve the detection and reporting of harms in randomised trials, systematic reviews, and observational studies. Other proposals aimed at improving the recording of harms in databases and tackling methodological problems during the analysis and interpretation, especially of rare harms. For example, the reporting of adverse effects might be improved through initiatives such as CONSORT for randomised trials and QUOROM for reviews.^10,11