One of the requirements of the request for applications that funded the project was that the investigators compare practices within a geographically defined area. The setting is the Pittsburgh region, which includes the second most elderly metropolitan community in the United States and, among the elderly, a higher proportion of blacks than is their proportion nationally. Several factors determined the selection of practices, including to the extent possible, practices that: (1) served a large percentage of elderly patients; (2) were of diverse types, e.g., internal medicine, family medicine, solo practitioner, multiphysician; (3) had white and minority providers; (4) would be likely to participate; and (5) used electronic medical records. Thus, we intentionally sampled practices based on these factors and the desired racial and socioeconomic characteristics of persons in the census tracts surrounding the practice location.
In designing the practice recruitment strategy and other aspects of the study, we also sought the advice of the Community Research Advisory Board (CRAB) established by the Center for Minority Health at the University of Pittsburgh. The CRAB consists of representatives of many sectors of the minority community including religious leaders, public safety workers, community activists, health care workers, and other opinion leaders. Before submission, investigators presented the proposed study design to the CRAB who provided input into the design and made suggestions for recruiting medical practices serving primarily black patient populations.
Allegheny County and its surrounding counties were mapped using 2000 Census data for income and for percent minority population. Practice sites were identified on these maps. In selecting practices, we attempted to match a (solo or multiprovider) practice serving primarily minority patients with a similar practice serving primarily white patients in socioeconomically comparable neighborhoods. We started recruitment of practices with the University of Pittsburgh Medical Center (UPMC) Community Medicine Inc., (CMI) a large, multispecialty network composed of diverse, previously independent, nonacademic practices that had been purchased by UPMC. Approximately 50 practices/offices offer primary care and all use a common electronic billing system. A few of the practices use an electronic medical record, which includes software for tracking preventive services such as immunizations.
The 12 UPMC network practices were supplemented with four community practices, two of which are federally qualified health centers (FQHCs). UPMC practices received a letter from the Medical Director of CMI requesting their participation. All practices were recruited by direct contact from the investigators by telephone, letter, and email to the practice managers, office managers, and/or lead physician. Five practices (three UPMC and two private, solo practitioners) failed to respond to requests to consider participation or refused requests to participate.
Once a practice had agreed to participate, the project coordinator scheduled a visit with the office or practice manager. The purpose of this visit was to introduce the study team, explain all the parts of the study, answer questions, and leave consent forms, surveys, and a brochure about the study.
Clinician and Office Manager Selection
General internists, family physicians, physician assistants, and nurse practitioners with their own patient case loads, seeing elderly patients from the participating sites were eligible for inclusion. Exclusion criteria for clinicians include seeing <50% primary care patients. The office or practice manager and the lead nurse or patient care assistant from each practice were also asked to complete questionnaires. As described below, these surveys were designed to triangulate data collection.
Separate questionnaires were developed for the physicians, nurses, and the office managers. The questionnaires were designed to describe current medical practices and determine barriers and facilitators to organization change that could lead to future adoption of immunization improvement strategies. The PRECEDE-PROCEED framework (Figure ) was used to develop the surveys. Constructs from this framework included predisposing factors (physician training, Awareness-to-Adherence model); reinforcing factors (incentives, office culture); enabling factors (express vaccine clinics, provider prompts); physician behavior (recommendation to patients, own vaccination status); environmental factors (location of refrigerator relative to exam rooms, staffing ratios, qualifications). A variety of immunization strategies derived from evidence-based reviews by the Task Force on Community Preventive Services in The Guide to Community Preventive Services
and by Gyorkos et al.,7,8
were assessed, including use of patient reminders, provider prompts, standing orders, and walk-in vaccination. Mission, office organization, and staffing were determined in both the clinician and office manager surveys. Office organization included leadership, financing, length of visits, medical record management, and quality improvement processes. A published questionnaire9
was used to assess practice culture, organization, and management style using the Competing Values Framework.10,11
The questionnaires differed among physicians, nurses, and office managers, but some questions were asked of more than one type of respondent to allow comparisons among them. The surveys were developed and revised through an iterative process by a multidisciplinary team who examined them for face and content validity.12
They were pilot tested before use and revised accordingly.
Participants were offered $50.00 payment in the form of a check or gift certificate. Survey data were entered twice into an electronic data base, results were compared electronically, and discrepancies were reconciled to reduce keystroke entry error.
Two-stage stratified sampling was used to determine the impact of practice and clinician factors on immunization status. The first stage was an intentional sample of diverse practices, as discussed above, that was stratified by race. In the second stage, random sampling of patient records within the practices was conducted, leading to a clustered, random sample of patients. Because of this hierarchical structure, we used a hierarchical linear modeling sample size estimation based on Byrk and Raudenbush.13
Based on an alpha of 0.05 and power of 0.80, to detect at least a 10% difference between practices and between races, for 10 practices with three clinicians per practice, a total of 165 patient records were needed per practice. As a confirmation of the estimated sample size from the hierarchical linear modeling and for analysis of the subgroup analyses, an additional method to calculate sample size was cluster randomization using a t
test, as calculated by NCSS/PASS software based on an alpha of 0.05 and power 0.80. For five practices in each stratum, with vaccination rates ranging from 39% to 75%, the number of patients per practice required would be 88–121. Thus, a total of 10 practices with 880 to 1210 patient records would be sufficient based on cluster randomization. To be conservative, the number of practices was increased to 18, using 165 patients per practice, with a goal of 2310 records for review.
For each office, the central billing system, billing computer, or electronic medical record was used to create a list of eligible patients for sampling. Because of HIPAA regulations, record retrieval was performed by a certified honest broker.14
This individual then selected patients who were born before 1/1/1940, who were living and who had an office visit in the last 12 months, indicating that they were active patients of the practice. This list was randomized following an algorithm designed to sample at least 15 patients per physician in a practice and to proportionally distribute the patients by their most frequent primary care provider. (See Appendix
Medical Record Review
The certified honest broker scheduled visits to review medical records and signed a confidentiality agreement with those practices requesting one. Using the randomized list generated by the sampling scheme above, the honest broker first reviewed charts to determine eligibility, that is, the patient was born before 1/1/1940 and had at least one visit in 2001 and in 2005. This step was to ensure continuity of care at that practice. If eligible, the patient’s PPV vaccination status and demographic information including race and address (to determine census tract) were extracted from the complete medical record including flow sheets, paper charts, and electronic medical records (EMR). Medical records were reviewed as far back as available for documentation of PPV receipt. The data were entered into an electronic spreadsheet on a laptop computer. The honest broker continued to review medical records from the sampling list at a given practice until a sufficient number (150–175) were collected or until all eligible charts in the practice (for smaller practices) were reviewed.
Following manual record review, for UPMC practices only, an additional search for the same time period was conducted using UPMC’s electronic depository of medical records called the Medical Archival Record System (MARS) and the billing database; however, relatively few vaccines were found that were not in the office record. These two data sets were combined and the records were deidentified. A research assistant identified census tracts for each patient’s address, and the corresponding per capita income for that tract was added to the data.
Medical Practice Observations
A field observation guide used in a previous study15,16
was adapted as a tool to collect data about medical practice characteristics. This guide outlined a variety of features and aspects of medical practices that observers were to pay special attention to, such as physical condition and layout of the office, staff clothing, and quality of interpersonal interactions. It included open- and closed-ended questions and prompts for field notes to collect specific information about procedures and routines of medical practices that facilitate vaccinations, such as use of standing orders, telephone reminders, and posted information regarding vaccinations, and literacy level and cultural appropriateness of educational and reading materials in the office. They also noted practice features that might inhibit vaccination, and aspects of the physical environment and setting of each practice, including transportation routes.
The observation team consisted of four trained observers, any two of whom conducted observations at every office. Observers individually prepared and coded their field notes. These notes were then shared with the lead member of the team for verification of correct and consistent use of the codebook, with differences reconciled by consensus. All three members systematically reviewed the data to develop models of practice culture. Written descriptions of each practice were developed, using data from both the collection sheets and from field notes.
Collection of Timing Data
A timing data collection sheet was developed to facilitate collection of data related to the time patients spent in various segments of the visit from registration to check out.15,16
To collect timing data given privacy restrictions, one observer was stationed in the waiting room and another in a location where patients entering the exam room area could be observed. The waiting room observer noted the time that each person entered and left the waiting room and the description of the clothing he or she was wearing to assist with collation of time observed by the back office observer. The back office observer recorded the time that each person entered the exam room area, entered and left the exam room, and when medical staff, including the PCP, entered and left the exam room. This strategy was selected because it was compliant with HIPAA as the observers do not interact directly with patients while they are receiving medical care. Five visit times will be analyzed: length of time in the waiting room, in the back office area, (including time at a nursing station for vital signs if a separate station is used), in the examination room, time the clinician is in the exam room and the total visit.
Quantitative Data Analysis
As an initial step, descriptive statistics were generated for each of the variables and the necessary assumptions for the planned statistical tests investigated. Statistical significance was set at P
Medical record review data were used to calculate vaccination rates for PPV. A practice’s PPV vaccination rate was calculated as the number vaccinated divided by the number in the sample.
Analysis of Predictors of Patient Immunization Status
Hierarchical linear modeling (HLM), which accounts for the concomitant effects of the nested structure, will be used to model multivariable effects of variables in predicting binary outcomes (i.e., received or did not receive PPV vaccination). First, possible associations between vaccination status and selected key variables of interest will be tested; these variables include selected items from the surveys and practice description variables. Second, variables found to be related to vaccination status or to reduced variance will be entered as potential variables in these multivariable models. Race and income based on census tract will be forced into the models as level-one variables to investigate race while accounting for socioeconomic status (SES).