Identifying which drugs should be avoided for elderly patients may be a matter of controversy because evidence derived from randomized controlled trials (RCTs) is either limited or non-existent [6]. During the last decade, assessments of inappropriate prescriptions to elderly patients have commonly been based on the Beers criteria [7], [8], last updated by Fick and co-workers in 2003 [9]. These criteria concern inappropriateness of single drugs or drug dosage with or without consideration of diagnosis, and are principally formulated for the US setting. They include drugs unavailable or only rarely used in Norway, and do not include potentially harmful combinations of drugs. Thus, there is a need for criteria of relevance for Norwegian general practice and comparable settings [10].

The aim of this study was to generate a list comprising explicit criteria for pharmacological inappropriate prescriptions to elderly (≥ 70 years) patients in general practice. Furthermore, it was sought to validate the clinical relevance of the list by a panel of clinical specialists utilizing a three-round Delphi process.

In late 2006, 140 physicians were invited to participate in a Delphi consensus process regarding the clinical relevance of the suggested criteria. They included all members of the Norwegian Association for Clinical Pharmacology (n = 33), a random group of Norwegian GP specialists (n = 55), and 41 specialists in geriatrics representing about half the members of the Norwegian Geriatrics Society.

The panellists were sent a questionnaire in which they were asked to score the clinical relevance for general practice of each of the 36 criteria on a 100 mm Visual Analogue Scale (VAS), according to the statement: “In general practice, the prescription rate of this item should be as low as possible for individuals ≥70 years”, from 0 (highly irrelevant) to 100 (highly relevant). The participants were encouraged to comment on the suggested criteria and on their own ratings. In Round 2, the participants gave a new score for all criteria based on feedback from Round 1, including their own previous ratings, mean ratings for the group with standard deviations, and the comments given to each indicator. This procedure was repeated in Round 3.

The panel agreed that 36 of 37 suggested criteria for pharmacological inappropriateness were clinically relevant for patients ≥70 years in general practice. Twenty-one explicit criteria (ECs) concerned single drugs and dosages (Tables I and III), and 15 ECs concerned drug combinations to be avoided (Tables II and ​andIV).IV). Only one of the suggested recommendations, namely to avoid the combination of erythromycin or clarithromycin and digitoxin, did not meet the conditions for being included on the list (median score of 65.3 and IQR 20.0).

During the three rounds of the Delphi process, the panel held a stable opinion for 33 of the remaining 36 criteria whereas their mean rating increased significantly during the process for three (ECs 12, 31, and 32) of them (see Tables III and ​andIV).IV). From first to third round, increasing agreement was seen for 30 of the criteria (i.e. all criteria except ECs 1, 3, 14, 22, 25, and 29). The mean standard deviation, as a measure of disagreement, decreased from 22.3 in Round 1 to 15.6 in Round 2 and 14.9 in Round 3. Concurrent prescription of a non-steroid anti-inflammatory drug (NSAID) and a selective serotonin reuptake inhibitor (SSRI) (EC 29) achieved the lowest relevance rating score, while the panellists gave the highest score for concurrent prescription of three or more psychotropic drugs (EC 36). Highest agreement was seen for ECs 11 (flunitrazepam), 14 (carisoprodol), and 36 (simultaneous use of three or more psychotropic drugs). The lowest agreement concerned concurrent use of an NSAID and a glucocorticoid, and the combination of an NSAID and an SSRI (see Table IV).

Over the three Delphi process rounds, the geriatrician group showed highest internal agreement, while most disagreement was seen among the GPs. The consequences of not using the benzodiazepine hypnotic flunitrazepam (EC 11) in the elderly was judged (mean score in last round with 95% confidence interval) differently by clinical pharmacologists, 88.6 (82.4 to 94.8), and GPs 97.8 (95.3 to 100.0).

The tendency towards more agreement within the panel during the three rounds is illustrated by the increasing average relevance scores for all 36 criteria (78.0, 81.0, and 82.3, respectively), and the decrease in corresponding SDs (22.3, 15.6, and 14.9, respectively).

That 36 out of 37 suggested criteria were judged to be clinically relevant may reflect thorough preparation by the expert panel in selecting cases to be validated in the Delphi process.

The criterion that in relative terms obtained lowest agreement was the statement that GPs should avoid using an NSAID along with an SSRI due to the added risk of gastrointestinal bleeding [15]. Here, the lowest degree of agreement was found among GP specialists (SD 30.1 in Round 3), while the clinical pharmacologists tended to agree more (SD 13.1 in Round 3). The fact that the evidence for this interaction was fairly new, and maybe also because its clinical magnitude had been questioned [36], may partly explain this variable rating.

The Delphi technique is flexible and enables a large number of experts to contribute to a relatively inexpensive process without geographic limitations. The anonymity in the postal-based Delphi process prevents dominance by high-profile experts, which might represent a problem in a face-to-face setting.

The Delphi technique, like any other structured communication method, has its limitations. The method has been criticized to the extent that it forces consensus by not allowing participants to discuss the issues [37]. Following the feedback from previous rounds, some panellists changed their views, which reinforced the group's opinion. This increase in agreement may thus be the result of constructive feedback during the process, but it is also possible that the panellists conformed to the view held by the majority [35]. The numerous comments (which were subsequently forwarded to the others) suggest that the panellists participated actively in the study and that the feedback process worked satisfactory. Questions have also been raised that the selection of panellists in Delphi studies may be biased (e.g. their numbers and level of homogeneity), and that this may affect the outputs [38]. A panel that includes few participants will definitely decrease the reliability of the Delphi process [39]. We do not think that these concerns are valid for our study because our panel included more participants than commonly adopted and our panel was also constituted by three different clinical specialist groups [7].

To assess the clinical relevance of the proposed criteria, we recruited clinical specialists from the three most relevant clinical specialties: general practice, clinical pharmacology, and geriatrics. We succeeded well in obtaining balanced participation from the three specialist groups. We cannot, however, rule out the fact that the acceptance to participate in the process to some extent was influenced by an a priori positive attitude to the proposed criteria. Also contributing to the internal validity of the process is that more than four out of five who accepted, equally distributed between the three specialties, completed all three rounds. A 17% dropout rate is in line with corresponding figures seen in other Delphi studies [5], [40].