Safety and effectiveness of nurse telephone consultation in out of hours primary care: randomised controlled trial

BMJ. 1998 October 17; 317(7165): 1054–1059.

PMCID: PMC28690

Safety and effectiveness of nurse telephone consultation in out of hours primary care: randomised controlled trial

Val Lattimer, research student,^a Steve George, director,^a Felicity Thompson, project nurse,^a Eileen Thomas, principal in public health and primary care nursing,^a Mark Mullee, senior research fellow,^c Joanne Turnbull, data manager,^c Helen Smith, senior lecturer in primary medical care,^b Michael Moore, general practitioner,^b Hugh Bond, general practitioner,^b and Alan Glasper, professor of nursing^d (the South Wiltshire Out of Hours Project (SWOOP) Group)

^aHealth Care Research Unit, Wessex Institute for Health Research and Development, ^bPrimary Medical Care, ^cMedical Statistics and Computing, ^dSchool of Nursing

Contributed by

Contributors: SG, VL, HS, ET, and EAG initiated the study and obtained funding. MM and FT facilitated the piloting of the service. FT, MM, and HB were responsible for running the service within the cooperative. Data collection was undertaken by VL, FT, and JT. Data analysis was performed by VL, JT, M Mullee, and SG. All authors participated in the discussion and interpretation of the results. VL and SG wrote the paper, with comments from all authors during the process. SG is the guarantor for the study.

Correspondence to: Dr S George, University of Southampton Health Care Research Unit, Mailpoint 805, Level B, South Academic Block, Southampton General Hospital, Southampton SO16 6YD pluto/at/soton.ac.uk

Accepted September 17, 1998.

See editorial "NHS Direct " on page 1026.

This article has been cited by other articles in PMC.

Abstract

Objective To determine the safety and effectiveness of nurse telephone consultation in out of hours primary care by investigating adverse events and the management of calls.

Design Block randomised controlled trial over a year of 156 matched pairs of days and weekends in 26 blocks. One of each matched pair was randomised to receive the intervention.

Setting One 55 member general practice cooperative serving 97

000 registered patients in Wiltshire.

Subjects All patients contacting the out of hours service or about whom contact was made during specified times over the trial year.

Intervention A nurse telephone consultation service integrated within a general practice cooperative. The out of hours period was 615

pm to 1115

pm from Monday to Friday, 1100

am to 1115

pm on Saturday, and 800 am to 1115

pm on Sunday. Experienced and specially trained nurses received, assessed, and managed calls from patients or their carers. Management options included telephone advice; referral to the general practitioner on duty (for telephone advice, an appointment at a primary care centre, or a home visit); referral to the emergency service or advice to attend accident and emergency. Calls were managed with the help of decision support software.

Main outcome measures Deaths within seven days of a contact with the out of hours service; emergency hospital admissions within 24 hours and within three days of contact; attendance at accident and emergency within three days of a contact; number and management of calls in each arm of the trial.

Results 14

492 calls were received during the specified times in the trial year (7308 in the control arm and 7184 in the intervention arm) concerning 10

134 patients (10.4% of the registered population). There were no substantial differences in the age and sex of patients in the intervention and control groups, though male patients were underrepresented overall. Reasons for calling the service were consistent with previous studies. Nurses managed 49.8% of calls during intervention periods without referral to a general practitioner. A 69% reduction in telephone advice from a general practitioner, together with a 38% reduction in patient attendance at primary care centres and a 23% reduction in home visits was observed during intervention periods. Statistical equivalence was observed in the number of deaths within seven days, in the number of emergency hospital admissions, and in the number of attendances at accident and emergency departments.

Conclusions Nurse telephone consultation produced substantial changes in call management, reducing overall workload of general practitioners by 50% while allowing callers faster access to health information and advice. It was not associated with an increase in the number of adverse events. This model of out of hours primary care is safe and effective.

Key messages

Telephone consultation is becoming an increasingly accepted approach to patient care and improves public access to medical information and advice
This study found that nurse telephone consultation halved the number of cases dealt with by general practitioners and was at least as safe as existing out of hours services
Nurse telephone consultation not only replaced telephone advice given by a doctor but led to reductions in both home visits and surgery attendances out of hours
Further testing is required of variants to the system used in this trial, including the selection and training of nurses and the decision support software used
There are clear opportunities for and potential benefits from integrating existing out of hours services with NHS Direct

Introduction

Increasing demands for out of hours care during the past two decades have placed the system of 24 hour care of patients by general practitioners under considerable strain.¹^,2 Recent developments in the delivery of primary medical care include the setting up of cooperatives of general practitioners and primary care emergency centres, which reduce the number of hours a general practitioner spends on call or facilitate arrangements for seeing patients. Other options include giving advice to patients over the telephone. Marsh reported in 1987 that 59% of all calls outside normal working hours to two general practitioners over a year could be managed by telephone advice alone, and a recent study in Denmark showed that the introduction of a dedicated telephone service run by general practitioners doubled the proportion of calls that were handled by telephone advice.³^,4 None of these interventions, however, reduces the overall workload in terms of patient consultations. The number of patients managed remains the same across the totality of general practitioners. One could wonder why the care of patients after surgery hours has remained primarily the responsibility of general practitioners when care during the daytime is covered by a primary healthcare team.

During nurse telephone consultation experienced and specially trained nurses receive, assess, and manage incoming calls to general practices after surgery hours.⁵ This term is preferred to nurse telephone triage as it indicates that the call management options include the provision of information and advice with reference to agreed guidelines, as well as referral to the general practitioner on call and direct contact with the ambulance service. In Canada, the United States, and Scandinavia a range of nurse telephone consultation services has been established.⁶^–⁸ In the United Kingdom a new advice and information service, NHS Direct, was announced in the recent white paper A New NHS.⁹ It will exceed the expectations of a service designed to manage emergency calls outlined in the chief medical officer’s report of 1997¹⁰ in providing clinical advice, general information, and referral to other NHS services. Three pilot lines for NHS Direct started in March 1998, and England is to be covered by 2000.

North American and British literature on the safety and effectiveness of telephone consultation is limited. Some studies point to the inadequacy of observed telephone encounters between health professionals and callers and highlight the potential for missed cases,¹¹^–¹⁸ while others report more favourably.¹⁹^,20

Our survey of general practitioners in 1996 showed that not all were convinced of the safety of nurse telephone consultation, although the idea was acceptable to most.²¹ The main concern, again, was the risk of “missed cases.” The effectiveness and safety of nurse telephone consultation in primary care had yet to be established in the United Kingdom. To address this issue a randomised controlled trial was required, and as a precursor to such a trial we undertook a pilot study for six weeks to establish the feasibility and acceptability of such a service to patients.²² During this study we established that most calls were to be expected during the evening. The full trial started on 23 January 1997 at 615

pm and ended on 20 January 1998 at 1115

pm. A night telephone consultation service was run for a month during the trial and is being analysed separately. We report the overall safety of nurse telephone consultation during the trial and its effects on general practitioners’ and hospitals’ workload.

Methods

Setting

We provided an out of hours telephone consultation service run by nurses for a general practice cooperative in Wiltshire of 55 general practitioners (19 practices) with a combined practice population of 97

000 patients, or roughly the optimum size for a new primary care group. The geographical area covered is about 290 km². It includes the city of Salisbury but is otherwise predominantly rural. The setting was chosen not only because of the enthusiasm of local general practitioners to take part in the trial but because its geography means that most patients attend a single accident and emergency department and are admitted as emergencies to one hospital, Odstock Hospital in Salisbury, making the monitoring of attendances and admissions comparatively straightforward. The out of hours period was defined as 615

pm to 1115

pm from Monday to Friday, 1100

am to 1115

pm on Saturday, and 800 am to 1115

pm on Sunday.

Objectives

The objective of many trials is to show that one treatment is significantly better than another, but the objective of some trials is to show that two treatments are equally effective.²³ The principal objective of this trial, and that used in determining its power, was to establish whether there was equivalence in the number of adverse events generated by a general practice cooperative augmented by nurse consultation compared with a standard cooperative service. A secondary objective was to collect data on the management of calls and on emergency hospital admissions and attendances at accident and emergency departments among those who had contacted the out of hours service.

Sample size

We had few data on adverse events arising from general practice consultations from which to estimate sample sizes. To date, the seminal study on the incidence of adverse events is the Harvard medical practice study.²⁴ In this study 30

000 randomly selected case records of inpatients admitted to acute hospitals were reviewed to develop population estimates of iatrogenic injuries according to the age and sex of the patient and the specialty of the doctor. Adverse events occurred in 3.7% of admissions. This study, however, was of hospital patients and took place in a different healthcare system. James and Pyrgos found an error rate of 3.6% when nurse practitioners in a British accident and emergency department were compared with middle grade doctors, although this was principally the result of overinvestigation.²⁵ If a rate of 3.7% were to be replicated in primary care outside normal working hours 37 calls per 1000 would result in some kind of adverse event. Anecdotally, this seems to be a high estimate, and a study based on this proportion of adverse events would likely be underpowered to establish equivalence in British primary care.

Clearly, the worst kind of adverse event is death, and we therefore used death as the basis of our calculation of sample size. For the purposes of this trial we did not try to distinguish preventable deaths from other deaths but studied total deaths among those who contacted either arm of the study during the trial year.

During one calendar year around 110 deaths per 10

000 population can be expected in England and Wales.²⁶ Hallam reports values ranging from 130 to 175 out of hours contacts per 1000 population per year.¹ Taking into account the facts that there will be a range of numbers of contacts per person from one to many and that we would be dealing only with the evening portion of the out of hours period, we estimated that the service would be contacted by around 10% of the population over a year. Applying these two figures to a population of 97

000 people, we calculated an expected number of deaths within the population contacting the out of hours service to be 107. The death rate might be increased among those contacting out of hours medical services, but the figure quoted is likely to give a more conservative estimate—that is, to produce a larger sample size. To establish exact equivalence is impossible without an infinitely large trial, so limits need to be defined within which equivalence is assumed. We used limits of equivalence from 80% to 125% of the expected number of deaths in the control arm, the usual limits applied in trials of bioequivalence (M J Campbell, personal communication). The expected number in the control arm, assuming deaths to be distributed equally, is about half the total expected, or around 54. Specifying α=0.1 (0.05 in a one sided calculation) and β=0.2, we calculated that 5455 patients would be required in each arm of the trial using the formula described by Jones et al.²⁷ A one sided calculation was used as we were interested to establish only whether the nurse intervention produced worse results (higher numbers of adverse events) than the existing service. Using Hallam’s figures again for expected numbers of calls we therefore could reasonably expect that we would achieve the desired sample size within a trial period of one year.¹

Randomisation

The trial year was divided into 26 blocks of two weeks. Within each block, one of each pair of matching out of hours periods—for example, Tuesday evenings—was randomly allocated to receive the intervention, the other being allocated to the normal service, by means of a random number generator on a Hewlett Packard 21S pocket calculator. For logistical reasons weekends (Saturdays and Sundays) were treated as single units for randomisation. The complete pattern of intervention periods was known in advance only to the lead investigators and the trial coordinator (SG, VL, and FT). Nurses providing the intervention knew their shifts only after the duty roster for general practitioners providing out of hours care had been fixed. General practitioners were therefore blind to the intervention at the point at which they were able to choose or swap duty periods. Most were not aware until the start of a period of duty whether nurses were present. The pattern of intervention and control days was not publicised and would have only become apparent to a member of the public on a particular day on calling the out of hours service and discovering whether nurse consultation was operating.

Intervention

Six experienced nurses were recruited in late 1996 and participated in a training programme in the skills required for telephone consultation for six weeks before the trial started. During intervention periods all incoming calls to the cooperative were received by a receptionist, who took patient details, and were then diverted to one of two nurses on duty. The nurse then undertook a systematic assessment of the caller’s problem and recommended an appropriate course of action, including management with nurse advice alone, contact with the general practitioner (by telephone, at the surgery, or at home), or direct contact with ambulance services. The nurse was aided by tas (telephone advice system), a computer based primary care call management system.²⁸ Confidential records were maintained on computer for each call. Calls about children under 1 year old and second calls about a patient on the same day were always referred to a doctor, unless callers had been specifically requested to call back to report progress after being given advice and their condition had improved. Patients and callers wishing to speak directly to a doctor were always able to do so. During control periods the receptionist took patient details and then passed calls on to a doctor.

Measures of process and outcome

Process measures included the age and sex of patients compared with the registered patient population; the most frequent presenting complaints; the date and time of telephone calls; the number of calls handled entirely by nurses; the number of calls handled by a general practitioner; and whether the case was managed by advice, a home visit, or attendance at a surgery or primary care emergency centre.

Outcome measures included the numbers of deaths among patients who had contacted the service or for whom the service had been contacted within the preceding seven days, the number of emergency hospital admissions within 24 hours and three days after a call, and the number of attendances at an accident and emergency department within three days after a call. A postal questionnaire was also posted to a sample of around 3000 callers across both arms of the trial. However, we were constrained to conducting a single shot survey—that is, with no second or third rounds of questionnaires—because of concerns locally that the anonymity of patients might be threatened by any system that monitored who had or who had not returned the questionnaire. Our overall response rate, therefore, was low (around 40%), and we have not reported the results in this paper.

Data and analysis

Data on workload were downloaded from the database of calls held by the cooperative and transferred into the statistical package for the social sciences (spss) for analysis.²⁹ Data on mortality from the Office for National Statistics³⁰ for the whole population of Wiltshire (residents and visitors) were initially matched with patients contacting the service using surname, date of birth, and sex. Some difficulties were encountered due to mis-spelling of names and missing dates of birth, and the computerised process was augmented by a manual search. Data on admissions and attendances were obtained from Odstock Hospital, Salisbury, and subjected to a similar matching process. A small proportion of admissions and attendances at accident and emergency departments (around 3%) were to several acute services on the periphery of the area. Data on advice to attend an accident and emergency department or on referral for admission for these admissions were collected from cooperative records, but for reasons of economy the data were not corroborated with the hospitals.

Analysis

To establish equivalence limits for data on deaths and attendance, each observed number of events in the control arm of the study was adjusted to take account of the slight difference in denominators (7308 calls in control arm v 7184 calls in intervention arm). The adjusted figure was multiplied by 0.8 and by 1.25 to give limits within which equivalence would be assumed. The upper 95% confidence interval for the corresponding figure in the intervention arm was then compared with the upper limit of equivalence for the control arm. Confidence intervals for deaths in the intervention arm were calculated using the population of patients who contacted the out of hours service during the specified hours over one year as a denominator. Confidence intervals for hospital admission and attendance at accident and emergency department were, however, calculated using the number of calls in the intervention arm as a denominator. This difference in methods takes account of the fact that a death is a once and for all event which pertains to a population, whereas a hospital admission or an attendance at an accident and emergency department can happen more than once. Two hospital admissions pertaining to the same person can thus appear in separate arms of the trial. When a series of calls about the same patient was made over a few days the last call before death or admission was used to allocate the event to either the control or the intervention arm. All confidence intervals were calculated using the confidence interval analysis program using, in each case, an exact method for a single proportion.³¹

Workload statistics are presented both as simple descriptive statistics and as a paired comparison within randomisation blocks of two weeks, with median differences between control and intervention weeks for each management option.

Results

Of 97

229 registered patients, 10

134 (10.4%) contacted the out of hours service during the specified times in the trial year on 14

492 occasions. This figure does not reflect the total number of calls received by the cooperative in all out of hours periods as it does not include calls received at night (after 11 15 pm). Table Table11 gives details of call frequency; most of the 10

134 patients called once during the year.

Table 1

Numbers of calls per registered patient in trial year

Characteristics of study population and patients in trial—

Table Table22 shows the age distribution of the study population (patients registered with general practitioners in the cooperative) and of patients in the trial. In comparison with the study population, the proportion of calls about babies under 1 year old exceeded the proportion of babies in the population by a factor of 8, and calls concerning children aged 1-4 years by a factor of 3.5. Calls about children and young people aged 5-24 years were in proportion to their numbers in the population, but calls for adults aged 25-74 were generally reduced, particularly for those aged 45-64, for whom calls were reduced by a factor of 0.5. As expected, more calls were received about patients over 75 years old than their numbers in the population would suggest. There were no substantial differences between the two trial groups. Table Table33 shows the proportions of male and female patients. In comparison with their frequency in the population, male patients were underrepresented in the trial, but no differences were found between the two arms of the trial.

Table 2

Age distribution of study population and patients in trial

Table 3

Sex distribution of study population and patients in trial. Values are numbers (percentages)

Impact of intervention on management of calls—

In all, 7308 calls were received in the control arm of the trial and 7184 in the intervention arm. Of the 7184 calls made during intervention periods, 3581 (49.8%) were managed by the nurse without referral to a doctor. There were significant reductions in workload for general practitioners in the other three categories, the largest reduction being in the amount of telephone advice given (table (table4).4). Table Table55 shows the same data analysed by randomisation week, estimating the weekly number of calls which can be handled by a service and the consequent reduction in workload of general practitioners in a cooperative of this size.

Table 4

Management outcome of calls during trial by trial group. Values are numbers (percentages) of calls

Table 5

Management outcome of calls in paired comparison within randomisation blocks of two weeks of weekly contacts

Deaths after contact with service—

A total of 125 patients died during the trial year within seven days after a contact with the out of hours service. Most were elderly (mean and median age at death 83 years (range 30-107)). Table Table66 shows how the deaths were distributed between the two arms of the trial. The deaths in the control arm are presented both as the raw figure and as a figure adjusted for the difference in denominators with limits of equivalence. Based on our method of calculating limits of equivalence, this upper limit is 83 deaths over one year. The upper 95% confidence interval for the number of deaths in the intervention arm is 75, well within the limit set.

Table 6

Adverse events during trial

Emergency admissions to Salisbury—

A total of 935 patients were admitted within three days of an out of hours contact during the trial year, constituting 6.4% of all out of hours contacts. A total of 815 patients were admitted within 24 hours. Table Table66 shows the distribution of admissions between the control and intervention arms. The upper equivalence limit for the control arm for admissions within 24 hours of a call was 541 over one year and that for admissions within three days of a call 623. The corresponding upper 95% confidence intervals for the intervention arm were 414 and 468 admissions respectively, well within the limits set.

Attendances at accident and emergency department in Salisbury—

The accident and emergency department recorded 27

771 attendances during 1997 (including both new episodes and unscheduled returns but excluding clinic appointments). A total of 810 patients attended within three days of an out of hours contact, 5.6% of all out of hours contacts and around 3% of all attendances at the accident and emergency department during the trial year. The upper equivalence limit for the control arm for attendances within three days of a call was 489 and the upper 95% confidence interval for the intervention arm 459, well within the limit set (table (table6).6). This was the only one of the three measurements of workload to show an increase in the intervention arm, but it was within statistical limits of equivalence and of no clinical importance.

Admissions and attendances outside Salisbury—

Sixty calls throughout the trial resulted in advice to attend or referral for admission to units other than Salisbury (table (table6).6). The largest number to a single unit was 48, to Princess Margaret Hospital in Swindon. As stated earlier, these data were not corroborated with the receiving units. Twenty six such referrals took place in intervention periods and 34 during control periods. Based on our method for calculating limits of equivalence, the upper limit for the control arm was 43 such events over one year. The upper 95% confidence interval for the number of events in the intervention arm was 38, well within the limit set.

Discussion

We found that nurse telephone consultation produced significant reductions in all parts of the workload of general practitioners and did not lead to any obvious adverse outcomes for patients. The results, however, apply only to the system we tested, including the selection and training of nurses and the decision support software used. Inadequate training of nurses and the use of a different software package might produce different results.

Are all contacts with general practitioners necessary?

Why did fewer patients have direct contact with a doctor in the intervention arm of this trial? We believe the explanation to be clear once it is accepted that not all patients who see a doctor need to see one, even if the contact is initiated by a doctor. With nurse telephone consultation all calls are subject to a systematic assessment with the aid of decision support software. When the nurse intervention was not operating in our trial the general practitioner on call, who was often in rural Wiltshire in a car, would receive a message from the receptionist at the switchboard that a call had been received from or about a patient, with the caller’s description of the health problem. Receptionists, while trained to operate booking systems, are not trained to undertake an assessment of patients. Under these circumstances, it may seem much more straightforward to visit the address given rather than return the call. Immediate advice from a nurse while the caller is still on the telephone can reduce the number of such contacts dramatically. Anecdotally, we are also aware that general practitioners sometimes find it difficult to refuse a patient’s request for a visit, even when they know that a visit is not indicated. The presence of a nurse acts as an effective filter under these conditions. The same explanation can be offered for the decrease in admissions to hospital observed during intervention periods.

Interpretation of results

There are some methodological difficulties in interpreting our results. For instance, the trial data do not show whether some patients received advice from a nurse when they should have been admitted to hospital, but had this occurred it did not lead to an excess of deaths. Clearly, however, many gradations of outcome exist between perfect health and death, and questions remain about differences between those who accepted nurse advice in this trial and those who experienced the usual on call system. Answering such questions is, however, difficult. When outcomes between the two trial arms are being compared, any comparison should not be restricted to those accepting nurse advice but include all those accepting advice from either a nurse or a doctor. Although this seems at first sight to be straightforward, a difficulty arises because the overall proportion of subjects accepting advice, from either a nurse or a doctor, is different in the intervention arm and the control arm of the trial. This poses considerable difficulties in terms of the validity of comparing these two groups since they have potentially different case mix characteristics. A separate point concerns the interpretation of potentially adverse events within the trial. Take the example of a repeat call after advice, which might be interpreted as a failure of advice as an intervention. However, both doctors and nurses giving advice over the telephone often ask callers to call them back to tell them whether the suggestion has worked. They also commonly say: “Please don’t hesitate to call back if you’re at all worried.” Are these bad outcomes, or good ones? As part of the further development of this work we intend now to review all deaths at a confidential audit, together with a random sample of emergency admissions from both arms of the trial, to explore the appropriateness of processes of care leading to death or admission.

Economics

Is this service affordable? It may be that the introduction of nurse telephone consultation results in a more comprehensive but more expensive service, with resulting choices for practices and, eventually, the taxpayer. There may be opportunities for economies, however. The rural nature of the trial cooperative required two general practitioners to be on duty, with two on standby to cover the area with adequate response times. However, larger urban cooperatives, with smaller areas to cover, might find it possible to replace a doctor with a telephone consultation nurse. Clearly, however, the blinded nature of this trial meant that we were not able to monitor any increase in overall demand generated by the presence of the nurse intervention service. This remains to be established by long term follow up of such services.

The way forward

Given two telephone numbers, one for a stand alone advice service and one for their doctor, many people will choose to ring the doctor. The out of hours service we studied generated 144 calls per 1000 patients per year, in comparison with existing regional health information services, which generate approximately 9 calls per 1000 population per year.³² Our study has shown, however, that a large proportion of calls to out of hours services can be handled with advice alone. There are clear arguments, therefore, in favour of centralising the process of handling calls. The success of NHS Direct, we believe, will depend on the extent to which it can integrate with primary care and social services and enable direct access to out of hours primary care services. Over the next year new primary care groups will shape integrated services for patients and will be asked to focus on the provision of “prompt, accessible, seamless care delivered to a high standard.”⁹ They, as well as managers of NHS Direct sites, should ensure that such an accessible and seamless service is made a reality.

Acknowledgments

We thank all the staff of the cooperative who helped to turn this project into reality. We thank Salisbury Healthcare NHS Trust and the staff of Salisbury and District Community Health Council for their support and help during the study. We thank BT and the South and West Regional Health Authority for funding the study, and we thank the Royal College of Nursing for its support. We thank Professor Mike Campbell, University of Sheffield, for statistical advice and Professor Jeremy Dale, University of Warwick, Mr Robert Crouch, University of Surrey, and Plain Software, the manufacturers of tas, for their technical help and advice. The opinions given here are ours alone.

Notes

Editorial by Pencheon

Footnotes

Funding: The telephone consultation service was funded by BT, which also funded the pilot stages of this work. VL was supported by a research studentship from South and West Regional Health Authority. JT was supported by a research grant from South and West Regional Health Authority.

Competing interests: BT funded the pilot stages and the service component of this trial. The research component was funded by the NHS Executive South and West Research and Development Directorate. BT agreed at the outset that the results of this trial would be published, whatever they were.

References

1. Hallam L. Primary medical care outside normal working hours: review of published work. BMJ. 1994;308:249–253. [PMC free article] [PubMed]

2. Beecham L. Home visits will fall with new GP scheme. BMJ. 1993;307:1375. [PubMed]

3. Marsh GN, Horne RA, Channing DM. A study of telephone advice in managing out-of-hours calls. J R Coll Gen Pract. 1987;37:301–304. [PMC free article] [PubMed]

4. Christensen MB, Olesen F. Out of hours service in Denmark: evaluation five years after reform. BMJ. 1998;316:1502–1505. [PMC free article] [PubMed]

5. Lattimer V, George S. Nurse telephone triage in out-of-hours primary care. Primary Care Management. 1996;6(5):3–6.

6. Wilkins VC. Paediatric hotline—meeting community needs while serving healthcare dollars. J Ontario Nurses Assoc. 1993;23(3):26–28. [PubMed]

7. Wheeler SQ, Windt JH. Telephone triage: theory, practice and protocol development. New York: Delmar; 1993.

8. Marklund B, Bengtsson C. Medical advice by telephone at Swedish health centres: who calls and what are the problems? Fam Pract. 1989;6:42–46. [PubMed]

9. Department of Health. The new NHS. London: Department of Health; 1998. (Cm3807.)

10. National Health Service Executive. Developing emergency services in the community. I and II. London: Department of Health; 1996.

11. Bradley SB. Use of the telephone by pediatric house staff: A technique for pediatric care not taught. J Pediatr. 1974;84:117–119. [PubMed]

12. Evans RJ, Mccabe M, Allen H, Rainer T, Richmond PW. Telephone advice in the accident and emergency department: a survey of current practice. Arch Emerg Med. 1993;10:216–219. [PMC free article] [PubMed]

13. Greitzer L, Stapleton B, Wright L, Wedgewood RJ. Telephone assessment of illness by practising pediatricians. J Pediatr. 1976;88:880–882. [PubMed]

14. Isaacman DJ, Verdile VP, Kohen FP, Verdile LA. Pediatric telephone advice in the emergency department: results of a mock scenario. Pediatrics. 1992;89:35–39. [PubMed]

15. Ott JE, Bellaire J, Machotka P, Moon JB. Patient management by telephone by child health associates and pediatric house officers. J Med Educ. 1974;49:596–600. [PubMed]

16. Singh G, Barton D, Bodiwala CG. Accident and emergency department’s response to patients’ enquiries by telephone. J R Soc Med. 1991;84:345–346. [PMC free article] [PubMed]

17. Soman M. Telephone management of febrile young children in a family practice setting. J Fam Pract. 1984;19:329–332. [PubMed]

18. Yanovski SZ, Yanovski JA, Malley JD, Brown RL, Balaban DJ. Telephone triage by primary care physicians. Pediatrics. 1992;89:701–706. [PubMed]

19. Perrin EC, Goodman HC. Telephone management of acute paediatric illnesses. N Engl J Med. 1978;298:130–135. [PubMed]

20. Leprohon J, Patel VL. Decision-making strategies for telephone triage in emergency medical services. Medical Decision Making. 1995;15:240–253. [PubMed]

21. Lattimer V, Smith H, Hungin P, Glasper A, George S. The future provision of out of hours medical care: a survey using two general practitioner research networks. BMJ. 1996;312:352–356. [PMC free article] [PubMed]

22. South Wiltshire Out of Hours Project (SWOOP) Group. Nurse telephone triage in out-of-hours primary care: a pilot study. BMJ. 1997;314:198–199. [PMC free article] [PubMed]

23. Pocock SJ. Clinical trials: a practical approach. Wiley: Chichester; 1983.

24. Brennan TA, Leape LL, Laird NM, Herbert L, Localio R, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. N Engl J Med. 1991;324:370–385. [PubMed]

25. James MR, Pyrgos M. Nurse practitioners in the accident and emergency department. Arch Emerg Med. 1989;6:241–246. [PMC free article] [PubMed]

26. Central Statistical Office. Social trends 25. London: HMSO; 1995.

27. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996;313:36–39. [PMC free article] [PubMed]

28. Crouch R, Dale J, Patel A. Ringing the changes: developing, piloting and evaluating a telephone advice system in accident and emergency and general practice settings. London: Department of General Practice and Primary Care, King’s College School of Medicine and Dentistry; 1996.

29. Norusis M. SPSS for windows. Base system user’s guide. Release 6.0. Chicago: SPSS; 1993.

30. Office for National Statistics. Public health mortality file 1997-1998. Fareham: ONS; 1998.

31. Gardner MJ, Altman DG. Statistics with confidence—confidence intervals and statistical guidelines. London: BMJ Publishing Group; 1989.

32. College of Health. Developing NHS Direct. London: College of Health; 1998.

Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Publishing Group