For the observational study, eligible pediatricians were initially contacted by phone to assess whether they were interested in participating in a study that focused on issues of doctor–parent communication during acute care visits for children in their practice. If the physicians in the practice were interested, the research team went to the office and gave a slide presentation outlining the main objectives and field procedures for the study.
For each physician who agreed to participate, we collected data during three separate two-week periods occurring between 10/96 and 3/97. Each of these two-week data collection periods was separated by a two- to four-week time gap. This was done in an effort to more evenly distribute each physician's study visits over the entire study period.
Physicians who agreed to participate completed a postvisit survey for each study encounter that included three checklists: one for physical examination findings, one for diagnoses, and one for treatments prescribed or recommended (see Appendix 2)
. For diagnoses and treatments, physicians were also given the option to write in a diagnosis or therapy if they did not want to select one of the choices on the lists provided. Physicians in our study wrote in alternate diagnoses in 11 percent of cases. None of these cases represented either a bacterial or viral upper respiratory tract infection but rather represented diagnoses that were not eligible for study inclusion, such as cerumen impaction and acute gastroenteritis.
After the observational study was completed, the study team returned to the participating offices and presented a second slide presentation that outlined the key findings of the study and revealed our “true” objectives in conducting the study. During these poststudy presentations, physicians were asked if they had ascertained that we were interested in antibiotic prescribing during data collection.
For the nonobservational study, we abstracted medical records (see Appendix 1)
for nonobservational study patients of the physicians who participated in the observational study in order to assess the degree to which the physicians' prescribing and diagnostic patterns changed while they were under observation. All medical record abstractions were performed after the observational study was completed. For each physician who participated in the observational study, we attempted to abstract the same number of encounters as the physician completed for the observational study. For example, if a physician completed 20 encounters in the observational study, we abstracted 20 medical record encounters for that physician for the nonobservational study. For some of the physicians, 5 to 10 more encounters were abstracted for the nonobservational study than were completed in the observational study (). We abstracted approximately half of the encounters for each physician from the same time period as the observational study data collection (10/96–3/97) but only included visit dates that fell outside the dates of data collection for the observational study in that physician's office. For example, if we had collected observational study data during the last two weeks of 11/96 for a given physician, all eligible medical record encounter dates for that two-week period would be excluded for that physician. For all but one of the study physicians the other half of the abstracted medical record encounters came from visits that occurred one year after the observational study had been completed (10/97–3/98). One physician left practice in 5/97, thus all abstracted medical record encounters for this physician came from the first period (10/96–3/97). We excluded charts of patients who were involved in the observational study from the nonobservational study.
Diagnoses Assigned During the Observational Study and Nonobservational Study
For each month of the nonobservational study (10/96–3/97 or 10/97–3/98), one of two processes was used to identify medical records that contained eligible visits for data abstraction. A visit was considered eligible for abstraction if it was conducted by a study physician, the child's age fell between 2 and 10 years at the time of the encounter, the visit occurred during the appropriate time frame, and the child had one of the following diagnoses: acute otitis media, otitis media with effusion, otitis externa, asthma, bronchitis, pneumonia, mycoplasma infection, croup, streptococcal pharyngitis, viral pharyngitis, viral upper respiratory infection (URI), sinusitis, or viral syndrome. In one practice, administrative data were used to identify medical records that contained visits that were eligible for data abstraction. In the second practice, medical records were sequentially pulled and screened for study eligibility by the first author (RM-S). Medical records in this latter practice were arranged in alphabetical order. Because of administrative constraints placed on us by the participating office we were unable to randomly select charts for potential abstraction. Although selecting charts in alphabetical order may have introduced selection bias based on race/ethnicity, in our observational study we found no relationship between race/ethnicity and antibiotic prescribing patterns or the diagnoses assigned. Additionally, arriving at the final sample for this practice required working through the entire alphabet. All encounters in a selected medical record were sequentially reviewed until we identified one that met the eligibility criteria for inclusion. For each medical record abstracted, only the first eligible encounter was included. If a medical record had no encounters meeting eligibility criteria, it was not included and the next chart in the sequence was pulled and reviewed. This process was repeated until the requisite number of encounters had been abstracted for each participating physician in the second practice.
The abstraction form collected the following data for each eligible encounter: (1) The child's age, (2) date of the encounter, (3) the diagnosis, (4) whether or not a chest x-ray, sinus x-ray, rapid strep test, throat culture, or complete blood count (CBC) was performed, and if so, the results of the test, and (5) whether or not an antibiotic was prescribed, and if so, the name of the antibiotic prescribed (see Appendix 1)
. The encounter was the unit of analysis for all study outcomes.
The medical records were abstracted by the first author (RM-S) and a research assistant who was trained by the first author. Neither abstractor was blinded to the study hypotheses. A 20 percent sample of the abstractions performed by the research assistant was recoded by the first author. The kappa statistic for interrater reliability for coding of diagnoses was .92 while the kappa for coding of treatments prescribed was .75. The majority of the discrepancies in coding between the abstractors were related to difficulty in interpreting the handwriting of the participating physicians.
Calculating the Proportion of Viral Cases Where Antibiotics Were Prescribed
The denominator for this measure was the number of cases in each sample of abstracted medical record encounters where only
a viral diagnosis was made (n
= 91 for 10/96–3/97 and n
= 77 for 10/97–3/98). Viral diagnoses included all diagnoses listed under item 3 of the abstraction form that contain the word viral (see Appendix 1)
. In addition, all cases of bronchitis not otherwise specified (NOS) and croup were considered viral as well as cases of pharyngitis NOS and pneumonia NOS without laboratory values supporting a bacterial diagnosis (e.g., a positive rapid strep test, a CBC with a white blood cell [WBC] count >15,000). The numerator for this measure was the number of cases in the denominator where an antibiotic was either provided or prescribed.
Calculating the Proportion of Cases Assigned a Bacterial Diagnosis
The denominator for this measure was the entire sample of abstracted medical record encounters for each data collection period (n
= 167 for 10/96–3/97 and n
= 137 for 10/97–3/98). The numerator was all encounters for each time period with a bacterial diagnosis. Bacterial diagnoses included all diagnoses listed under item 3 of the abstraction form that contain the word bacterial (see Appendix 1)
. In addition, all cases of acute otitis media, otitis externa, otitis media with effusion, pneumonia NOS (with a supporting chest x-ray or a CBC with a WBC count >15,000), mycoplasma infection, culture or rapid antigen detection-test proven streptococcal pharyngitis, and sinusitis were considered bacterial.
Testing for Difference
Multiple logistic regression, correcting for clustering of encounters within physicians using the cluster
option of the logistic
command in STATA 6.0 (StataCorp. 1999
), and controlling for practice site, and patient age was used to examine differences in the proportion of viral cases where antibiotics were prescribed between the observational and nonobservational studies. Multiple logistic regression, cor-recting for clustering of encounters within physicians was also used to examine differences in the proportion of cases assigned a bacterial diagnosis between the observational and nonobservational studies. Differences for which p
<.05 were considered significant. All of the physicians (three) in one practice participated in the study, while in the second practice, all of the full-time clinicians (five) and two of five part-time clinicians, who work one-half day per week, participated in the study. Thus, study physicians account for 100 percent of encounters at one site and approximately 95 percent of encounters at the other site.