Much attention is given to the internal validity of trials (whether the data are “true”) and less to the external validity of trial data (whether the data on drugs are generalisable to the population to whom they are to be prescribed). Our study concerned the external validity of trials of conventional NSAIDs. The basic issue we addressed was diversity—that is, the likelihood of varied effectiveness or adverse events of NSAIDs in different patients.
Pharmaceutical companies market new drugs from many angles
From our examination of the literature on diversity (including age, sex, gender, and ethnicity) we conclude that there is a complex overlap between the many different factors that might lead to social exclusion and to exclusion from either trials or equitable health care.4
For example, older people are often excluded from trials (the rationale often being that they are more vulnerable to comorbidities, adverse drug reactions, and problems with consenting), and older women are noticeably under-represented in trials of many drugs, despite their larger numbers in most developed societies.5,6
Similarly, ethnic minorities are often excluded (sometimes on the basis of language), and there is a clear overlap between ethnic minorities, economic disadvantage, and social exclusion.7
Drugs tested in one condition are often prescribed to people with another condition. We examined trials of NSAIDs in the management of osteoarthritis because that is the commonest cause of joint pain in the community and the main condition for which these drugs are marketed and used.1,8
Although we found that some aspects of the reporting of these trials was poor, the quality of the trials was generally good, and most of them were carried out on patients with well characterised osteoarthritis, in whom comorbidities were absent (by exclusion). In the population cohort that we studied the situation was different. It is likely that many of those who were prescribed an NSAID had osteoarthritis, but many will have had other causes of pain, including inflammatory forms of arthritis such as rheumatoid arthritis or systemic lupus erythematosus. Similarly, many of those patients will have had comorbidities. Unfortunately, we do not know why the drugs were prescribed to people in the cohort or the level of comorbidity and coprescribing. As a result we cannot make inferences about the mechanisms of harms; although all the patients used NSAIDs, we cannot attribute all the events to NSAIDs. What is clear is that the harms observed in practice were different from those reported in the trials, on which the use of the drugs was based, and that there was a dose relation between risk and drug use.
Pharmaceutical companies promote NSAIDs by highlighting both the benefits and the harms from their use. For example, an advertisement may contain an image of an older person enjoying pain free activity that is contrasted with the image of an ulcerated stomach, if protection of the gastrointestinal tract is the marketing angle.
Adverse event reporting
The recent literature on adverse events associated with NSAIDs has focused on gastrointestinal problems, with less attention to renal and cardiovascular toxicity (fluid retention, hypertension, impairment of renal function), despite these well known problems.9-13
Some of the early literature gave conflicting messages about the association between NSAIDs and renal disease14
; but our results are consistent with four large case-control studies from the past 15 years, in which patients, largely elderly, admitted to hospital with renal impairment were 2-4 times more likely to have used NSAIDs than control patients.15-18
Exclusion from randomised controlled trials of patients at risk from renal impairment misinforms practitioners in two important ways: by weakening their awareness of the risk of renal failure associated with NSAID and by failing to provide relevant information about the relative risks from different NSAIDs. Interest in gastrointestinal problems may be partly due to the pharmaceutical industries' endeavours to produce selective NSAIDs, or “coxibs,” that might be less toxic to the gastrointestinal tract.19-21
We have studied only the non-selective NSAIDs, but we have shown that admissions to hospital for renal failure and gastrointestinal events were increased in those using the drugs, particularly older people and those with known risk factors.
Risk benefit ratio
These findings illustrate some of the difficulties in trying to balance the potential risks of an intervention with its potential benefits. Data from trials on efficacy can provide estimates of the absolute effects of treatment by numbers needed to treat or numbers needed to harm. But these treatment effects are averages derived from highly selected, relatively small, and restricted groups of patients who are unlikely to be representative of the population that will use the drugs. A further concern is that numbers needed to treat and numbers needed to harm are often not measured on comparable scales of benefit and harm. If a drug clearly has major life saving benefits, then it is likely that these will outweigh relatively uncommon adverse effects. In the case of NSAIDs, however, comparing improvements in joint pain and stiffness with the uncommon risks of hospital admission for gastrointestinal bleeding or renal failure is not straightforward. Similarly, it is clear that assuming that the risk of harm is relatively stable whereas that of benefit depends on the severity of the disease, cannot be applied in this sort of case, due to the likely effects of disease indication, age, and comorbidity on the adverse events.
We looked specifically at trials of NSAIDs in osteoarthritis. Many trials have also been done in rheumatoid arthritis, but relatively few in other, less common rheumatic diseases such as systemic lupus erythematosus. Theoretically, renal adverse events could be greater in patients with rheumatoid arthritis, and particularly those with systemic lupus erythematosus, than in those with osteoarthritis, as these and other systemic rheumatic conditions can affect the kidneys and cardiovascular systems directly.22,23
One problem with most of the less common inflammatory rheumatic diseases, in which there are few trials, is that NSAIDs are not specifically approved for use in these conditions but are nevertheless prescribed routinely to patients with them if they have joint pain, and there is no way of knowing from the trial data whether that is damaging to the kidneys or cardiovascular system. Similarly, both gastrointestinal and renal adverse events are affected by coprescribing, including steroids and analgesics, both of which are commonly used for arthritis.24
Again, the importance of coprescribing cannot be certain from the trial data alone.
The need for record linkage
Our study was only possible because of access to the Medicine Monitoring Unit's database, in which prescribing is linked to hospital data through collaboration with NHS Information Services (ISD) in Edinburgh. The use of this sort of record linkage is particularly valuable for the examination of adverse events, but if reliable information is to be obtained, large datasets that can link prescribing data to other health records are needed. In many countries, record linkage has been made more difficult with the introduction of data protection legislation.25,26
The best way to get more reliable post-marketing data on drug toxicity in particular is by establishing more datasets like the one in Tayside, so that adverse events can be related to patient characteristics, and by performing large randomised trials using routine data for follow up.27