This study is a cluster randomized trial, with the GP clinics as the cluster. The local GP clinics are randomly assigned either to use a referral template or to continue standard referral practice.
The 14 community GP practices in the area primarily served by University Hospital of North Norway Harstad (UNN Harstad) were included in the randomization process, with a total list size of 35,490 patients. In Norway, each individual has a regular GP. These GPs act as gatekeepers to secondary care. The health care system is relatively uniform throughout the country. In the study area, access to specialist care is practically impossible without a GP referral, whereas some access is possible in other areas of the country.
The study population will consist of patients referred to the medical department at UNN Harstad. The referrals received are, almost exclusively, electronic. Children (<18 years of age) and patients with reduced capacity to consent will be excluded from participation in the study.
The GP clinics were randomized stratified by location, to ensure adequate selection of cases and equal sociodemographic background data. Five of the centers are larger town-based centers and nine are smaller, more rural centers. The location of the center was not expected to influence the outcome variables. Initially, two centers approached declined the invitation to participate in the study, and therefore two additional GP clinics were randomly selected, as illustrated by Figure
Figure 1 Flow chart of randomization process. ‡ From the four rural clinics initially randomized to the intervention group, two clinics refused. Therefore two additional rural clinics, from the five rural clinics initially randomized to the control group (more ...)
New patients referred to the medical department within one of the four diagnostic groups described below will receive written information and a consent form together with their appointment letter. These will be sent out by a clinic nurse unaware of the status of the GP center sending the referral (intervention or control). Patients will be orally reminded at the appointment and may be given a new consent form. This process is illustrated in Figure
Recruitment process. Recruitment of patients in both the intervention and control groups follow the same procedure.
The referral templates have been developed based upon international literature [19
] and in collaboration with local specialists in the appropriate medical field. The templates have also been through a process of clinical assessment from subspecialists in other northern Norwegian hospitals. In acknowledging the problems in earlier studies with the uptake of referral interventions [17
] we have deliberately reduced the number of items in the referral templates, to ensure ease of uptake. Only information that the medical consultants thought imperative in the referral have been included as items in the templates. The study will implement referral guidelines for the following four diagnostic groups:
patients referred with dyspepsia;
patients referred with suspected colonic malignancy;
patients referred with chest pain;
patients referred with chronic obstructive pulmonary disease or suspected chronic obstructive pulmonary disease.
These diagnostic groups were chosen as they represent a substantial number of the referrals to a medical department (own data, 2008). They also represent a clear diagnostic challenge in primary care and are adept for simple referral guidelines.
The GPs at the intervention offices will use the referral template when initiating a new referral process for a patient. To ensure adequate uptake of intervention, the templates have been distributed as electronic templates as well as hard copies. The templates function as guidelines, but are not implemented as compulsory electronic checklists. The intervention referrals are sent to a separate electronic inbox at the hospital. The further evaluation and process of care has not been altered in the intervention group compared with the standard referral practice in the control group (Figure
Referral pathway. Flow of referral and process of care in the intervention (blue arrows) and control group (red arrows).
In addition, questionnaires have been developed that assess patient experiences within the care framework, with one questionnaire designed for inpatients and one for outpatients. These questionnaires have been produced by combining questions from previously validated questionnaires regarding patient experiences in general and with transitional care. They include questions from a Norwegian patient experiences questionnaire [28
]); two questions about health interaction from the Commonwealth Fund Survey 2010 [29
]; the Care Transitions Measure 3 [30
]; and demographic questions.
The questionnaires have been reviewed through an interview process with patients. This was done to ensure that the forms are acceptable to patients and to highlight possible issues that patients felt were missing from the questionnaires.
We aim to examine the impact of a referral template on the process of care, at the individual level. The primary hypothesis is that the use of a referral template in the communication between the GP and secondary care will lead to a measurable increase in the quality of health care delivered. Secondary hypotheses include that the use of a referral template in the communication between a GP and secondary care will lead to:
a • measurable improvement in referral quality;
• a change (up or down) in the number of patients defined as being in need of prioritization (as defined in national guidelines for prioritization in health care [20
• more appropriate prioritization, as measured by final diagnosis;
• an increase in the ‘appropriateness’ of the referrals (positive predictive value (PPV) of referral [31
• and better patient satisfaction, as measured by self-report questionnaires.
The primary outcome will be a quality indicator score compared between the intervention and the control group. The quality indicators have been generated from previous international quality assessment tools and national and international treatment guidelines. Some adaptation to locally accepted practice has been made, as demonstrated by others when quality indicators are used in a new context [32
]. Each patient care process will be scored according to the criteria. Scoring will be done by a panel of specialists blinded to the intervention status of the patient. We will calculate the quality score as adherence scores (number of quality criteria met divided by number of applicable criteria expressed as a percentage) as illustrated by Ashton et al
]. If a criterion is applicable, but no information can be found (applicable, but not answerable), it will be noted ‘not met’ for statistical purposes [34
]. Weighting of the criteria based upon clinical importance will not be used, as this adds complexity without adding much to the clinical findings, a finding discussed by Lyons and Payne in 1975 [35
]. The scores will be compared between the intervention and control groups.
As secondary outcomes, the quality of the referrals will be evaluated against the developed referral template, to determine if the intervention has led to improved referral quality. It is important to measure referral quality, as the premise in the study is that more information will lead to improved care. In addition, health process outcomes such as waiting time from referral to appointment, number of appointments before a diagnosis is reached, time before treatment is initiated, the application or not of appointment prioritization, and the outcome of any given referral (appointment, return information or referral rejected) will be collected and compared between the groups. Bennett et al
. used the PPV of a referral as a measure of the appropriateness of the referral [31
]. In adapting this concept from glue ear in otolaryngology to a medical department we have defined it as the proportion of the GP referrals that result in a histological diagnosis, diagnostic clarification or change in medical management. We will assess and compare the PPV of referrals in the study groups. Patient experiences will be compared between the intervention and control group. Finally, the possible outcome diagnoses have been grouped according to severity. We will correlate the continuous outcome variable ‘waiting time’ with the grouped severity, to see whether the prioritization in the intervention group is more aligned with disease severity.
Sample size was calculated based upon the primary outcome, and the initial calculation was done without regard for clustering. A change in the quality score of 10% was determined to be clinically interesting. Setting the significance level at 0.05 and using PASS 2008 (NCSS, LLC, Kaysville, UT, USA) for the calculation provides that the study would require 855 patients in the control and 855 in the intervention group, for an 80% power to detect a 10% difference between the groups.
To correct for clustering, the design effect (DE) may be calculate as:
DE = 1 + ρ(m-1)
denoted the intracluster correlation coefficient (ICC) and m
is the size of each cluster [36
]. No ICC for equivalent designs was identified from literature searches. Reviewing primary care-based interventions from the literature [37
], an expected ICC ranging from 0.001 to 0.08 does not seem improbable, giving a DE of 1.15 and 12.9, respectively, for a cluster size of 150 patients. Because only 14 GP clinics were available for randomization, further inflation of the number of clusters to achieve higher power was not possible, although this could have been advisable [40
Based upon a review of patient data at UNN Harstad from 2008, the study is expected to achieve this relatively high inclusion number by recruiting over a two-year period (personal data).
The referring GP cannot be blinded to the trial, as the intervention is actively used by the GP. The patient will not be aware of the intervention, but no active effort has been made to keep the patients blinded. The patients will be mixed with the general caseload to avoid bias in the treatment process at the hospital. For the GPs that use the electronic referral template, this usage will be visible to the hospital doctor in the presentation of the referral letter on the computer, but for the majority of the cases the hospital doctor will be blinded to the intervention status of the patient. The outcome assessors will be blinded.
Data will be extracted by both automated computer reports (for example, waiting times, number of appointments) and manual chart review (for example, PPV, group of final diagnosis). Data will be collected after the process of care that the referral initiated is completed.
We will collect the following baseline characteristics:
• patient age (mean and confidence interval) and sex (number and percentage)
• practice list size (median and interquartile range, or mean and confidence interval if normally distributed)
• referral type - electronic or paper (number and percentage)
• referred by - GP or other doctor (number and percentage).
For the primary outcome (quality score), we will calculate adherence scores as described above and compare between treatment arms. We expect substantial variation in cluster size. Because of the small number of clusters, analysis based upon the cluster level was considered [41
]. However, as there is no prior accurate estimate of variation between clusters, weighting for cluster size could not be achieved [41
]. To offer increased precision and take into account between-cluster variation, random effects linear regression will be used [42
] to generate estimates of intervention effect. It has been suggested that this can be used for studies with as few as 10 clusters [36
]. The estimated effect and confidence intervals will be reported. A P
-value <0.05 will be regarded as statistically significant. Intention to treat analysis will be employed.
The referrals will be scored using a simple scoring system related to the referral templates. Each unit of information specified in the referral template (for example, presence of weight loss specified) will provide one point in the scoring system, with no weighting applied. Scores will be compared between the groups as noted above.
For outcome severity, random effects linear regression will be used, with the severity group score as a categorical variable, and the relationship compared between the intervention and control group.
In the questionnaire, answers noted as ‘not applicable’ or no answer will be counted as missing data. The questionnaire will be scored according to a pre-set scoring system. Scores will be analyzed using the regression technique outlined above. The data will also be analyzed to determine if factors such as self-perceived health, age, gender, and education level have an impact on patient experience.
The trial will be reported according to the CONSORT standards for reporting cluster randomized trials [43
No pilot study has been carried out. To ensure acceptability of the intervention, GPs were invited to, and participated in, the development of the referral template. To ensure feasibility, the authors have collected all data specified in the protocol from the 20 patients included first. To ensure an adequate uptake of the intervention, regular reviews of all referrals received at UNN Harstad will be undertaken.
The study will follow the directions in the Helsinki Declaration, and was presented to the Regional Ethical Committee for Medical Research in northern Norway, who determined it not to be within the scope of the Health Research Act (REK NORD 2010/2259). The project has been approved by the Data Protection Official for Research. The study is registered at ClinicalTrials.gov. The trial registration number is NCT01470963. All patients must provide written informed consent.