The trial took place in the West Midlands of the UK between 2006 and 2008. The protocol has been previously described in detail, including the definitions used for programming the e-Nudge algorithm (see Box 1
for group definitions).4
Box 1. Groups identified by e-Nudge software.
Patients without known cardiovascular disease or diabetes who are 50–74 years old, and whose most recent risk factor values give an estimated risk level ≥20% over 10 years.
Patients without known cardiovascular disease or diabetes, who are 50–74 years old, and whose risk profile is incomplete — more information is required to perform a risk estimate — but whose risk would be >20% over 10 years if ‘assumed’ values of the missing factors are used.a
Patients with cardiovascular disease but not diabetes, who have not had a blood glucose measurement in the past 3 years.
Patients who are not known to have cardiovascular disease or diabetes, are over 75 years old, and who have persistently elevated blood pressure based on the three most recent consecutive readings.b
aAssumed values for missing variables were derived from the Health Survey for England 2003 as the median values for the 50–75 year age group for total serum cholesterol (males 5.7 mmol/l, females 6.2 mmol/l), high-density lipoprotein cholesterol (males 1.4 mmol/l, females 1.7 mmol/l), and systolic blood pressure (males 135 mmHg, females 132 mmHg). bSystolic blood pressure ≥160mmHg or diastolic blood pressure ≥100mmHg for each of the last three measurements.
The e-Nudge software tool
The e-Nudge searches electronic health records for cardiovascular risk factor data and builds lists of patients identified in each of four groups A–D (Box 1
). Cardiovascular risk is estimated using the Framingham risk equation applied to the most recent risk factor measurements.5
The lists are updated every 24 hours to take account of new information. When the records of a patient are accessed, a screen alert message appears if the patient is currently in one of the groups. This happens during a consultation with the patient, or when notes are opened for any other clinical or administrative reason. The message explains why the patient has been identified, and, in the case of patients with insufficient information for a risk estimate, states which data are required. The user is not obliged to respond to the alert but the message continues to appear on opening the notes until the patient's record is updated in such a way that the patient is no longer in one of the groups (for example, by recording a blood pressure measurement if this was missing).
How this fits in
Cardiovascular risk factor data are recorded electronically in UK general practice, but identifying high-risk individuals requires the use of risk algorithms. A cardiovascular risk assessment tool applied to a practice database can identify people at raised risk and those most likely to benefit from further data collection. When linked to screen alert messages, this tool increased the visibility of the high-risk population and the adequacy of data to support risk estimation, but did not reduce cardiovascular event rates significantly over a 2 year period.
The e-Nudge was installed in all participating practices. An email was sent to the practice manager (or nominated administrative staff member) every 8 weeks to remind them that lists of intervention patients in the various groups were available for viewing in their practice, but no action was mandatory for the purposes of the study. For those patients randomised in the intervention arm, the software produces screen reminders as described above. For those patients in the control arm, the e-Nudge recorded if they were in any of the groups but did not generate reminders. Access to the group lists was only possible for the intervention patients. gives an example of a Group A alert message appearing on the screen.
Example of a group A message appearing during consultation.
Eligible practices were those running EMIS (Egton Medical Information Systems) LV software, as this was required to install the e-Nudge system. No other eligibility criteria were applied. EMIS supply clinical and administrative software to nearly 60% of UK practices, and 80% of their systems use the LV version. The majority of EMIS LV practices in Coventry and Warwickshire (41 out of 71 available practices) were invited to take part, and all that were willing were accepted into the trial. The 19 participant practices were based in diverse settings including rural, suburban, and inner-city environments. The practice list sizes varied from <2000 to >14 000 patients, and from single-handed practitioners to large group practices with more than six partners. Coronary heart disease indirectly standardised mortality ratios (for 2003–2005) ranged from 74 to 110 in the districts served by the practices.
The records of all individuals aged over 50 years and registered with participating practices were included in the trial. This age range was chosen to ensure a study group at relatively high cardiovascular risk.
Outcomes and sample size
The annual cardiovascular event rate was the primary outcome used for the power calculation. Assuming a cardiovascular event rate of 1260 events per 100 000 person-years (all ages) in the control arm,6
and a 10% lower event rate in the intervention arm (rate ratio of intervention to control of 0.9), a total sample of about 70 000 patients followed up for 2 years gives 80% power for a test at level 5% (two-sided) allowing for 15% withdrawal.7
This calculation was based on all-age event rates as it was not possible to find an event rate specifically for the over 50s at the time the trial was designed. The intervention was applied to the over 50s population and the outcome was measured in this age group only. A higher event rate was expected in this age group.
Randomisation and allocation concealment
Randomisation was at the level of the individual patient record. The e-Nudge software automatically randomised registered patients within each practice to intervention and control arms, depending on whether the last digit of the 10-digit NHS number was odd or even. This number is a unique identifier allocated to all individuals registered with the NHS and is generated using an algorithm that takes no account of age, socioeconomic group, or any other factor relevant to cardiovascular risk. The tenth digit is calculated according to the Modulus 11 algorithm,8
and serves as a ‘check digit’ to confirm the number's validity.
New patients registering with a practice during the study were randomised as soon as the NHS number was available in the record. Throughout the trial, users of the e-Nudge were kept unaware of the odd/even rule, but if an alert appeared on opening a record it would be evident that the patient was in the intervention arm. It was made clear to users at the outset that patients who did not trigger alerts were not necessarily at low cardiovascular risk, as they might simply be in the control arm.
The number and proportion of patients identified in each of the groups were automatically extracted by the e-Nudge and stored in a file in the practice's main computer server every 8 weeks. The mean values from the final three data captures during the 2-year period were derived for each group. The practice data were aggregated. The annual incidence of cardiovascular events was obtained from each practice's database. Following a search for relevant events (Box 2
), each individual record identified was examined using a predetermined protocol to confirm that the event had occurred within the trial period, and, where more than one event was recorded, that these were separate events.
Box 2. Definition of a cardiovascular event used in the trial.
- A new diagnosis of cardiovascular disease (that is, entry onto the Coronary Heart Disease [CHD] Register or Stroke/Transient Ischaemic Attack Register)
- A new stroke or transient ischaemic attack (whether or not already on the Stroke/Transient Ischaemic Attack Register)
- A new myocardial infarction (whether or not already on the CHD Register)
- Sudden death from cardiovascular disease
Recording of cardiovascular outcomes is prone to several sources of error, recognised in the trial protocol.4
Not all cardiovascular events result in a new coded entry into a primary care record, and sometimes a single event is recorded more than once using different codes or entry dates. When a patient dies, the need to record the final event electronically is no longer a priority for clinical care, although it is usual practice to do so. For these reasons, the study examined every electronic record identified in the outcome searches. A small substudy was also carried out in four practices to check whether any sudden cardiovascular deaths had been missed. For this substudy, extra code groups were included in the searches to increase the retrieval of cases: ‘Death administration’, ‘On examination — dead’, and all of their lower-level codes.
Changes to the trial protocol
In the original protocol, patients with existing CVD or diabetes whose blood pressure or serum cholesterol were outside the QOF target were to be identified. However, screen alert messages were introduced to all EMIS systems to support the QOF just before the start of the trial. This group was therefore withdrawn from the trial.
The e-Nudge was initially designed also to identify individuals with possible undiagnosed diabetes based on previous raised blood glucose measurements. A number of such individuals were identified among the baseline data following installation of the e-Nudge software. This led to a nationwide survey to demonstrate that such patients are identifiable across the UK,9
and the result was the introduction of a new software module to all EMIS systems nationally to support early diabetes detection.10
This group was therefore withdrawn from the e-Nudge within the first 6 months of the trial.
During practice visits, one researcher discussed practical issues surrounding the usability of the software which were noted and acted on. After approximately 9 months, the wording of the screen alerts was shortened in response to practice feedback but the information they contained remained the same.
Statistical analysis and intention to treat
Analysis was carried out using STATA 10 software. For the cardiovascular event rates, the rate ratio with 95% confidence interval (CI) and two-tailed P
-value were computed using standard likelihood inference techniques for Poisson counts.11
The group proportions were compared using χ2
-tests, and two-tailed P
-values are reported. Data were analysed from all patients whether or not their computer record had been accessed by primary care staff during the trial.
One of the practices withdrew from the study after less than 6 months, but consented to their data being included in the analysis. However, the automatically captured group data were no longer available from this practice after the software was removed, so only the cardiovascular event rate data were used as part of the final analysis. In one practice, a failure of data capture occurred at baseline and the earliest data available at this site were extracted after the intervention had been in place for 25 days.