This study was part of a larger multicentre study, “The Role and Impact of Walk-In Clinics in Ontario's Health Care System,” which compared utilization, cost and quality of services in walk-in clinics with those provided in family physicians' offices and emergency departments.
We conducted our study in greater Toronto, Hamilton-Burlington and London, Ont. We approached 20 walk-in clinics (chosen by random selection), 35 family practices (32 randomly selected and 3 identified by targeted recruitment) and 13 emergency departments (randomly selected in Toronto and all departments in Hamilton-Burlington and London) from 11 geographic areas in the 3 cities. These 3 metropolitan areas were chosen because they had a relatively high concentration of walk-in clinics and were readily accessible to the research team. At least one walk-in clinic, one family practice and one emergency department were recruited from each of the 11 geographic areas to ensure that the 3 care settings drew patients from the same population pool.
Family practices were defined as settings where more than 50% of patient visits were by “regular patients,” people who used the clinic or saw a physician in the clinic for ongoing care of common medical problems and for preventive care. To be included in the study, family practices had to consist of 2 or more physicians (to increase the efficiency of patient recruitment) and remuneration had to be on a fee-for-service basis. Walk-in clinics were defined as clinics where less than 50% of visits were by “regular patients.” All walk-in clinics included in the study received fee-for-service payment.
An expert review panel was convened to select tracer conditions and to establish quality-of-care criteria. The panel consisted of 2 family physicians, 1 family/walk-in clinic physician, 1 family/emergency physician and 1 emergency/walk-in clinic pediatrician. On the basis of available data regarding frequency of diagnoses in the 3 settings10
(J. Ivan Williams, Senior Scientist, Institute for Clinical Evaluative Sciences, Toronto, personal communication 1997), 8 acute conditions commonly seen in all 3 settings were selected for study: pharyngitis, gastroenteritis, serous otitis media, acute otitis media, upper respiratory infection, acute bronchitis, urinary tract infection and low back pain. Quality-of-care criteria previously developed by the College of Family Physicians of Canada11,12
were reviewed by the panel and modified by consensus. The criteria dealt with processes of care and included performance of appropriate clinical actions and avoidance of inappropriate clinical actions. The criteria covered the domains of history-taking, physical examination, diagnostic procedures, imaging, verification of diagnosis, drugs and physical therapy, education and preventive care, and follow-up. Copies of the final criteria are available from the authors on request. The number of criteria ranged from 4 for serous otitis media to 23 for gastroenteritis.
On the basis of previous studies of patient satisfaction13
and quality of care11
and using standard methods,14
we estimated that a sample size of 150 patients per setting (total 450 patients) would be adequate to detect clinically meaningful differences in outcomes between settings. To ensure a similar patient mix in each setting, only patients being seen for an initial assessment for 1 of the 8 tracer conditions were invited to participate. Patients were approached by data collectors while they were waiting to see the primary care physician. Information from patients under 16 years of age and those not competent to respond was collected through a proxy adult respondent using questionnaires with slightly modified wording. After eligibility questions were asked and informed consent was obtained, each English-proficient patient or proxy was interviewed to ascertain demographic characteristics and whether the patient had a regular family physician. Only patients with 1 of the 8 conditions confirmed by the physician (as determined by chart audit) were eligible for the study. As well, eligibility was limited to regular patients in the family practices and “nonregular” patients in the walk-in clinics. Data were collected between Feb. 15 and Dec. 21, 1998. Data collectors recruited 600 eligible patients with a total of 625 tracer conditions (some of the 600 patients had more than one tracer condition). All 600 patients were included in the quality-of-care analyses. Of these 600 patients, 486 completed all data collection instruments, and 433 of these (174 from walk-in clinics, 122 from family practices and 137 from emergency departments), for whom we had complete information on all relevant variables, were included in the patient satisfaction analyses. For the analyses related to satisfaction with care, the diagnoses upper respiratory infection, pharyngitis and acute bronchitis were collapsed into a single category, upper respiratory illness, and the diagnoses acute otitis media and serous otitis media were collapsed into a single category, otitis media.
A tridimensional model of patients' satisfaction with the study visit was used. After seeing the physician, each patient completed a self-administered questionnaire which included 3 instruments, each measuring a separate dimension of satisfaction previously identified as important:15,16,17,18
perceptions of patient-centred communication19,20
(8 items with 4 response choices per item, averaged to a total score ranging from 1 to 4), perceptions of the physician's attitude21
(5 items with 5 response choices per item, converted to a total score ranging from 20 to 100) and delay in the waiting room21
(3 items with 5 response choices per item, converted to a total score ranging from 20 to 100). The items included in each scale are listed in Appendix 1 (see www.cmaj.ca
). The internal reliability of the 3 satisfaction scales was tested with Cronbach's α.
Although patients were clustered by provider and providers by practice, the fact that some patients were seen by more than one provider made analyses allowing for such clustering infeasible, and the subjects within each setting were considered as independent observations. Univariable comparisons of satisfaction among the 3 settings were performed by analysis of variance (ANOVA). In multivariable analyses, the different satisfaction outcomes were modelled separately with linear multiple regression analyses (backward stepwise elimination method with significance level for exclusion of p = 0.15). The independent variables considered for inclusion were setting (walk-in clinic, family practice, emergency department), sex, age, education, main activity (employment or other), income, language spoken at home, whether the patient was living with a partner, whether the patient had children, self-reported health status, perceived seriousness of condition, diagnosis (conditions of interest) and whether the patient had a regular family physician. Statistical significance was set at p < 0.05 for both univariable and multivariable analyses.
For the quality-of-care analysis, chart abstraction was performed by 12 experienced abstractors between Feb. 15 and Dec. 21, 1998. Only the record of the initial visit for the tracer condition was reviewed. The abstractors were provided with a detailed abstraction manual and participated in a 3-day training session. Intra- and inter-rater reliability and criterion validity were assessed on the basis of duplicate abstraction of 10 charts covering 7 of the tracer conditions, for which kappa statistics22
and reliability coefficients23
A quality-of-care score for each case was computed as the percentage of applicable criteria met. Therefore, each case received the same weight regardless of the number of applicable criteria. Scores for each of the 3 settings were computed as the mean of scores for all cases managed in that setting.
To adjust for potentially confounding factors, we computed mean scores for each setting, controlling for sex of the patient, age of the patient (grouped as less than 12 years and 12 years or older), city and diagnosis. In addition, preliminary investigation revealed the need to control for a significant interaction between city and setting.
We explored the potentially confounding effects of patients' health status (“In general, would you say your health is excellent, very good, good, fair or poor?”) and perceived condition severity (“On a scale of 1 to 5, with 1 being not serious at all and 5 being very serious, how would you rate the seriousness of your condition?”). Neither perceived health status nor perceived condition severity was significantly related to quality-of-care scores. Accordingly, neither of these variables was included in our analytic model.
We developed an ANOVA model from which the population marginal means of the scores for each setting were estimated. Estimated marginal means, also known as least squares means, are a function of the model parameters, without regard for the distribution of observations through the model factors.24
These means are adjusted for other factors in the model, and thus we refer to them as adjusted scores. The proportion of variability explained is given by the ratio of the between-groups sum of squares to the total sum of squares. After including setting, sex, age, city, diagnosis, and the interaction of setting and city, we checked for significant interactions between sex and age and other model factors. The only other significant interaction was between sex and diagnosis.
The study was approved by the Research Ethics Boards at The University of Western Ontario, The University of Toronto, McMaster University and participating hospitals.