|Home | About | Journals | Submit | Contact Us | Français|
The UK-based General Practice Research Database (GPRD) is a valuable source of longitudinal primary care records and is increasingly used for epidemiological research.
To conduct a systematic review of the literature on accuracy and completeness of diagnostic coding in the GPRD.
Six electronic databases were searched using search terms relating to the GPRD, in association with terms synonymous with validity, accuracy, concordance, and recording. A positive predictive value was calculated for each diagnosis that considered a comparison with a gold standard. Studies were also considered that compared the GPRD with other databases and national statistics.
A total of 49 papers are included in this review. Forty papers conducted validation of a clinical diagnosis in the GPRD. When assessed against a gold standard (validation using GP questionnaire, primary care medical records, or hospital correspondence), most of the diagnoses were accurately recorded in the patient electronic record. Acute conditions were not as well recorded, with positive predictive values lower than 50%. Twelve papers compared prevalence or consultation rates in the GPRD against other primary care databases or national statistics. Generally, there was good agreement between disease prevalence and consultation rates between the GPRD and other datasets; however, rates of diabetes and musculoskeletal conditions were underestimated in the GPRD.
Most of the diagnoses coded in the GPRD are well recorded. Researchers using the GPRD may want to consider how well the disease of interest is recorded before planning research, and consider how to optimise the identification of clinical events.
Information collected routinely from primary care can provide a cost-effective source of data for epidemiological research.1 One of the main benefits of using automated databases for research lies in the ability to access data from large patient populations across a wide population coverage. While these datasets represent a valuable tool for conducting research, it is important to remember that the data are collected primarily for clinical and routine use rather than specifically for research purposes. Data quality and reliability must be considered by researchers using these resources.
The General Practice Research Database (GPRD) is a widely used UK-based database of clinical primary care records, and has been extensively used in both primary care and pharmacoepidemiological research. It is the world's largest source of anonymised longitudinal data from primary care, and currently contains information on 3.6 million active patients from 450 general practices in the UK.2,3 Practices participating in the GPRD are remunerated for recording data on clinical diagnoses, test results, prescriptions, and referral data. Clinical data are captured using the Read/OXMIS (Oxford Medical Information System) coding framework, which is based on the International Classification of Diseases, (Ninth Revision, Clinical Modification: ICD-9-CM) and is widely used in British primary care. Each practice is issued with a set of GPRD recording guidelines, describing how to record all significant morbidity events in each patient's medical history.4 The raw data provided from each practice undergo extensive quality control and validity checks by a research team based at the Medicines and Healthcare products Regulatory Agency before release. These data are assessed by an ‘up to standard’ audit, confirming data recording in several key areas. Practices meeting this standard are included in the GPRD data warehouse. Patient-level data are also assessed, with patients considered ‘acceptable’ for inclusion in the GPRD if recorded details are internally consistent in four areas: age, sex, registration details, and event recording.5
While the checks conducted by the GPRD team may provide an overall evaluation of which practices are providing good-quality data meeting certain standards, they do not specifically assess the validity and completeness of individual patient records. Quality of data in disease registers can be assessed by considering two issues. First, are the data accurate; do the codes on the register represent the diagnosis under question? Second, are the data complete; what proportion of all true cases are recorded on the database?6
As the GPRD is increasingly used for academic research, it is important to consider the quality of the data available for study. The aims of this paper are to conduct a systematic review of the literature to present a description of the accuracy and completeness of recording in the GPRD. Specifically, the paper will consider both the recording of clinical diagnoses and comparisons between the GPRD and other databases or national statistics.
Six electronic databases were searched (MEDLINE, Embase, British Nursing Index, PsycINFO, Health Management Information Consortium, Social Sciences Citation Index) from inception to May 2009, using search terms relating to the GPRD in association with terms synonymous with validity, accuracy, concordance, and recording. See Appendix 1 for an example of the MEDLINE search. Two reviewers screened the titles and abstracts of all papers in the initial search, and excluded citations that did not meet the inclusion criteria. Disagreements at this stage were resolved through discussion; however, the full-text paper was ordered when either reviewer was uncertain about inclusion. One reviewer independently assessed the retrieved full-text papers for relevance and inclusion, and hand searched the reference lists of all retrieved papers. The same reviewer also conducted a hand search of the GPRD bibliography provided on the GPRD website,7 and included any potentially relevant papers in the review process.
The UK General Practice Research Database (GPRD) is increasingly used for academic primary care researchers. Data collected for automated databases such as the GPRD are subject to a number of quality checks, but the validity and accuracy of specific disease coding may vary. This paper describes a systematic review of studies that conducted validation of diagnoses in the GPRD or compared rates of disease in the GPRD against other databases. The majority of diagnoses were reliably coded; however, investigators conducting research using the GPRD should consider how information is captured in primary care and how the variation in recording practices for different diseases and prevalent conditions may affect research.
Studies that specifically considered the accuracy of recording of data in the GPRD were included, and those that looked at the accuracy of recording of data as part of a larger study. Papers written in English and non-English languages were considered for inclusion. Papers that did not look specifically at the GPRD were excluded, as well as those that were not original research and papers that did not conduct validation or comparison of information held on the GPRD. Database comparison studies were also excluded if the study compared prevalence rates over different time periods. Two investigators independently reviewed articles meeting the inclusion criteria and abstracted relevant data onto a standardised data extraction form. A third reviewer resolved any uncertainties in relation to the main outcomes.
This review considers both the accuracy and completeness of data recording in the GPRD. The accuracy of diagnoses was achieved by comparing the GPRD-coded data in the electronic patient record against the gold standard defined in each paper. Assessing the completeness of data was achieved by comparing disease prevalence and prescription rates between the GPRD and other national datasets and determining the level of under-reporting or over-reporting in the GPRD.
Positive predictive value (PPV) was defined as the proportion of GPRD-coded diagnoses validated as true cases against a gold standard (GP questionnaire, primary care medical records, or hospital letters). Where the investigators used a GP questionnaire or request for hospital notes, the percentage validated was calculated using the returned questionnaires as the denominator. Stata (version 10.0) was used to calculate binomial exact 95% confidence intervals (95% CIs) for each PPV, and to produce a forest plot of PPVs for correct coding of clinical diagnoses. A pooled PPV was not calculated, due to the variability in the included studies.
A total of 46 papers were identified through the literature search and included in this review. Three additional papers were identified through hand searching the reference lists of the final 46 papers.8–10 Therefore, a total of 49 papers entered the review (Figure 1). Results are presented in the following main groupings: (1) validation of clinical diagnoses and (2) comparisons between GPRD and other national statistics or databases.
A total of 40 papers conducted validation of a diagnosis or patient characteristic coded in the GPRD database. Most of these validation exercises involved sending a questionnaire to the patients' GPs (n = 19) and conducting independent verification of diagnoses against hospital letters or medical records in practice (n = 16). Some studies involved both sending a GP questionnaire and conducting verification against medical records (n = 5). The PPV of the accuracy of GPRD clinical codes from these validation exercises is summarised in Figure 2. The majority of papers report PPVs over 50%, and simply required patients' GPs to confirm the diagnosis on the GPRD database. Five of the seven papers reporting a PPV under 50% considered acute outcomes including drug-induced liver injury, pancreatitis, or renal failure.11–15 The studies considering acute conditions all used strict diagnostic criteria to validate cases, which included confirmation of the diagnosis via biochemical tests or specialist confirmation.
Studies in this review reported high PPVs over 90% for the recording of anorexia, bulimia,16 cataract,17 congenital heart defects (including ventricular septal defects, tetralogy of Fallot, and coarctation of the aorta),18 inflammatory bowel disease,19 cerebrovascular disease, diabetes, respiratory tract infection,19 Paget's disease,20 hip fracture,21 upper gastrointestinal bleeding,22 non-affective and non-organic psychosis,23 venous leg ulcer,24 and pressure ulcer.25
Recording of psoriasis,26 venous thromboembolism,27 schizophrenia,23 dementia, and Alzheimer's disease28 was relatively accurate, with PPVs between 80% and 90%. Other diagnoses, including cardiovascular events and thromboembolic disease,10,29 irritable bowel syndrome,30 chronic obstructive pulmonary disease,31 chronic atrial fibrillation,32 and cardiac arrhythmia,33 were not as accurately recorded, with reported PPVs lower than 80%.
Several papers validated the same diagnosis. Two papers considered the recording of autism in the GPRD.8,34 Both studies validated the diagnosis directly against hospital and patient medical records, and report that autism is well recorded in the GPRD. However, the electronic record was not detailed enough to provide sufficient differentiation between subtypes of pervasive developmental disorders. Acute myocardial infarction was also well recorded in the GPRD, with three studies reporting PPVs above 80%.9,35,36 There was good agreement in two papers that validated the recording of incident multiple sclerosis against hospital and medical records; however, both papers reported relatively low PPVs of around 60%.37,38
Two papers considering the validity of coding for ventricular arrhythmia reported markedly different PPVs.39,40 Using a GP questionnaire as the standard for validation resulted in a PPV of 93% (95% CI = 78 to 99%) for cases of sudden death or ventricular arrhythmia, whereas more stringent criteria requiring objective evidence of ventricular arrhythmia from specialist clinics and absence of recent angina or myocardial infarction resulted in a PPV of only 20.9% (95% CI = 13 to 31%). However, the investigators using a GP questionnaire to validate ventricular arrhythmia also report that only 23% (95% CI = 10 to 42%) of these diagnoses originated from outpatient events, which was their main outcome of interest. Two papers consider the validity of coding for rheumatoid arthritis; however, one study reported four validation categories (valid, invalid, possible, and unclassifiable) from which a PPV could not be derived.10,41
Two studies assessed the completeness of GP recording of diagnoses made by hospital consultants. In these studies, diagnoses were transcribed from the hospital discharge letter onto patients' electronic medical files in a high proportion (~90%) of cases.42,43
Devine et al conducted a validation study using an algorithm to identify children with neural tube defects. They reported an overall PPV of 71% (95% CI = 63 to 78%); however, the PPV varied considerably according to the specific neural tube defect diagnosis.44 While anencephaly and cephalocele were generally well recorded, spina bifida was not. The Read Code algorithm used by the authors located spina bifida in the mother and not the child in 37% of cases.
One paper considered the recording of smoking status in the GPRD.45 Although current smoking is generally well recorded, former smoking is not as well recorded. Appendectomy is also under-recorded in the GPRD; in a study of ulcerative colitis, the self-reported rate of appendectomy was 13% in the random sample of patients; however, the GPRD-coded rate of appendectomy in the same study was only 3.5%.46
Three papers reported results relating to accuracy of the date of diagnoses. There were discrepancies in date recording in 45/95 (47%) of dementia cases.28 The differences were generally small, with an interquartile range (IQR) of −7 to 0 weeks. In a study of the validity of inflammatory bowel disease recording, the median difference in the first date reported by the GP and the first inflammatory bowel disease diagnosis in the electronic record was −8 days (IQR = −81 to 0 days). However, for 33 of the 53 patients included in the study, the first recorded diagnosis of inflammatory bowel disease in the electronic record was within 30 days of the date reported by the GP.19 Recording of the date of acute myocardial infarction was also generally reliable; only 31/201 (15%) of confirmed cases had a GP-reported date that was inconsistent with the electronic record. The differences in dates were generally small; 28/31 (90%) of the GP-reported dates were within 15 days of the date in the electronic record.35
In total, 12 papers compared GPRD database prevalence or consultation rates with other primary care databases or national statistics registers. All compared the GPRD with UK data, except for one comparison with a US-based database.47
Three papers compared GPRD consultation or prevalence rates against the Morbidity Statistics from General Practice 1991–92 (MSGP4), a UK-wide survey of consultation patterns in primary care.48–50 The MSGP4 itself has been evaluated;48 96% of all consultations in the GP surgery were recorded, suggesting it contains good-quality data on consultation patterns in the UK. There was good agreement between the GPRD and MSGP4 for 11 common respiratory conditions. However, consultation rates and prevalence of diabetes and musculoskeletal conditions were underestimated in the GPRD compared with the MSGP4.49,50
Three studies compared the GPRD with the Doctors' Independent Network (DIN), a UK-based primary care database that has been collecting routine data from over 300 practices distinct from the GPRD since 1989.51–53 Generally, there was good agreement between the two databases for common childhood conditions, hay fever, ischaemic heart disease, and prescribing for skin emollients.
The six remaining papers compare the GPRD with a variety of other primary and secondary care databases. Three papers report similar rates of disease among the GPRD and other databases. The UK-based MediPlus primary care database, which covers about 150 practices across the UK, provided similar crude incidence rates to the GPRD for venous thromboembolic disease.54 Derby et al compared rates of suicide in the GPRD to a US-based database held by the Group Health Cooperative, and report that the overall rate of suicide among users of antidepressants was similar to the rate in the Group Health Cooperative.47 A comparison of the GPRD with the Hospital Episodes Statistics demonstrates comparable overall and age-specific incidence of Guillain-Barré syndrome.55
There were some differences between disease coding in the GPRD and other datasets. A comparison of the GPRD and the Living in Britain National Household Survey from 1996 suggests that current smoking rates in the GPRD are 79% of the expected rate. The rates for ex-smokers were substantially underestimated; the GPRD rate for ex-smoking was 29% of what was expected according to the National Household Survey.45 Frischer et al describe under-reporting of drug misuse recording in the West Midlands Regional Drug Misuse Database compared to the GPRD.56 The prevalence of congenital heart defects was higher in the GPRD than in the National Congenital Anomaly System.57 The same authors also validated heart defect diagnoses using a GP questionnaire, and reported an overall PPV of 0.935, suggesting that the GPRD is a good source of information on congenital heart defects.18
A systematic review of literature was carried out to validate the accuracy and completeness of the UK GPRD. The studies included in this review considered the accuracy of diagnostic codes in the GPRD and the completeness of data compared with other databases and national statistics.
Most of the diagnoses coded in the GPRD electronic record were well recorded when compared against GP questionnaire responses, medical records held at the GP practice, or hospital letters. However, it seems that acute diagnoses were not as well recorded. The studies in this review used a variety of ‘gold standard’ references to ascertain the accuracy of diagnoses, which may explain some of the differences in accuracy of diagnosis recording, especially in the validation of acute conditions.
When a questionnaire is sent to a GP to support the accuracy of a diagnosis, the GP has several options for verifying the diagnosis, including checking through the computerised medical record or free-text information, looking for supporting evidence for a diagnosis from test results or hospital discharge letters, or relying on memory alone. Investigators conducting independent validation of a diagnosis can also request copies of patient medical records, hospital discharge letters, or correspondence or biochemical test results. This extra information can be used to find key words relating to a diagnosis, or conduct expert validation of a diagnosis.
Several studies assessed the accuracy of recording against an objective standard as defined by an external body; for instance, acute liver injury defined as an increase of more than two times the upper limit of normal in alanine aminotransferase by international consensus statement, or evidence of specific behavioural or cognitive symptoms as described in the Diagnostic and Statistical Manual of Mental Disorders.15,23,34 The number of patient records validated may depend on the level of evidence required to assess the accuracy of recording.
Smoking status is an important risk factor and confounder in many epidemiological studies, and although current smoking may be recorded well enough for the purposes of epidemiological research, data on former smoking may need to be independently validated.45
Only three papers looked specifically at the differences between the date of onset of disease in the GPRD electronic medical record and the GP-reported date.19,28,35 Although there were some inconsistencies in date recording, the differences were small. Investigators who require precise dates of onset of disease may need to be aware that there could be a slight difference in the date recorded in the electronic record and, if necessary, conduct further validation.
Generally, there is good agreement in disease prevalence rates between the GPRD and other national databases and statistics; however, there were some differences identified in this review. There is no ‘gold standard’ measure against which data from one database can be compared, or to suggest which database contains the most accurate measure. Discrepancies between two data sources do not necessarily mean that one database is right and one is wrong. There may be geographical differences or disease coding system variability that will lead to systematic differences in disease prevalence or data recording.48 It is important to consider these differences when conducting research using these datasets.
There are two reasons why the GPRD may be systematically different from other datasets. First, not all consultations for chronic diseases need to be recorded in the GPRD; the GPRD recording guidelines state that the GP should make at least one entry in the medical history for each episode of illness or new occurrence of a symptom.4 The requirement to record only the first instance of disease may partially explain why consultation rates and prevalence of diabetes and chronic musculoskeletal conditions were underestimated in the GPRD compared with the MSGP4.49,50 Second, it is important to consider that practices supplying early years of data to the GPRD provided OXMIS-coded data. Other databases may use different coding systems; for instance DIN practices have always used Read Codes for recording diagnoses and prescriptions under a problem-orientated medical record, which presents each medical record as a set of intertwined but separate problems.51,52 Investigators attributed many of the differences between DIN and the GPRD to the Read and OXMIS coding systems used in the respective databases.53
This is the first study to search for studies systematically and to combine studies that consider the accuracy and completeness of the GPRD. By using broad search terms it was possible to find a wide range of literature covering a range of diagnoses. This review provides vital information to aid researchers and clinicians who are planning to conduct research using the GPRD. However, very few of the papers in this review gave results that were directly comparable. A wide range of diagnoses were considered and many of the investigators used different criteria to assess the validity of diagnoses, making it difficult to compare directly PPVs across studies even when diagnoses were the same. There was often a lack of an objective standard for comparison of data recording, and the papers in this review often used a variety of methods to judge the accuracy of clinical diagnoses in the patient electronic records. Finally, many of the studies included in this review only validated a small number of patient records, due to the expense of conducting validation of diagnoses via GP questionnaire or independent evaluation of hospital letters or medical records.
Several UK-based studies consider the quality of morbidity coding in general practice, and a systematic review of these studies shows that morbidity coding in general practice is variable. However, the investigators suggest that conditions with clear diagnostic features are better recorded than conditions with more subjective criteria.58 In their paper, Jordan et al include eight GPRD studies which are also assessed in the current review. The sensitive search strategy used in this study, which specifically considered the GPRD, made it possible to find and consolidate information from a larger number of papers validating a diagnosis in the GPRD. Thiru et al investigated the quality of data in primary care; however, their review focused more on how well GPs record the outcome of a consultation on electronic patient records.59 Their review also found that studies report consistently high PPVs, indicating that data on the patient record were valid. As noted in the present review, the authors point out that variability in the assessment of data quality made it difficult to compare results directly between studies.
Investigators conducting research using the GPRD need to consider carefully how information is recorded in primary care, and how GPs may use different Read/OXMIS codes to represent the same diagnosis. Some diagnoses may be recorded differently from others. This review suggests that researchers can be confident about case validity when using the GPRD for research into most chronic conditions. However, research into acute conditions may need additional validation. It may not be feasible to conduct validation studies of diagnoses in the GPRD for every project, as this can be expensive; current prices start at £60 per patient for a questionnaire or request for additional information from a practice.
One approach to ensure better identification of cases is to construct Read/OXMIS code diagnostic algorithms comprising several codes to identify events and diagnoses in the GPRD. Often, these diagnoses can then be internally validated using evidence within the GPRD to support the diagnosis; for instance, a Read/OXMIS code for an acute myocardial infarction may be followed by a referral to cardiology, details of a discharge letter from hospital, and relevant medication.
A study validating the recording of neural tube defects found that in some cases, the diagnosis represented a condition in the mother and not in the child. Birth defect researchers using the GPRD may wish to search the mother's medical history to determine whether the code relates to a diagnosis in the mother or the child. This supplemental information can be obtained from within the GPRD to improve the reliability of diagnostic codes.
Prescription data are well recorded in the GPRD because prescriptions for patients are generated directly from the computer, and details on drug type and dosage are digitally recorded in this automated process. Therefore, prescribing data can be used to verify clinical diagnoses, or to capture additional cases. For instance, use of inhalers was used as a proxy for asthma diagnoses.48 However, investigators should be cautious about using drug prescribing as a proxy for disease, and ensure that the prescribed drug is specific to the diagnosis of interest.
One of the future strengths of the GPRD lies in planned linkages with other national databases, including the Hospital Episodes Statistics, and Office for National Statistics databases, and the National Cancer Intelligence Network. These linkages will allow investigators to access more detailed clinical information relating to inpatient and outpatient hospital attendances and diagnoses, death registration, cause of death, and cancer diagnoses and treatment. This additional information will be a source of accurate and complete information on many of the clinical outcomes occurring outside of primary care.
|MEDLINE search: 1950 to week 3, May 2009|
|1.||“general practice research database”.ti,ab|
|3.||“Value added medical products”.ti,ab|
|6.||1 or 2 or 3 or 4 or 5||GPRD or VAMP|
|7.||valid*.ti,ab||Accuracy and completeness|
|12.||“positive predictive value*”.ti,ab|
|23.||7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 or 21 or 22||Accuracy and completeness of data|
|24.||6 and 23||Accuracy and completeness of data in the GPRD|
GPRD = General Practice Research Database. VAMP = Value Added Medical Products.
Nada Khan has undertaken this work as part of a research training fellowship funded in the UK by Macmillan Cancer Support. This work was also supported by Cancer Research UK (CR-UK) grant number C23140/A8854.
The authors have stated that there are none.
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss