Validity of Parent-Reported Vaccination Status for Adolescents Aged 13–17 Years: National Immunization Survey-Teen, 2008

Public Health Rep. 2011; 126(Suppl 2): 60–69.

PMCID: PMC3113431

Validity of Parent-Reported Vaccination Status for Adolescents Aged 13–17 Years: National Immunization Survey-Teen, 2008

Christina G. Dorell, MD, MPH,^a Nidhi Jain, MD, MPH,^b and David Yankey, CPH, MS^a

^aCenters for Disease Control and Prevention, Immunization Services Division, Atlanta, GA

^bHealth Resources and Services Administration, Office of Regional Operations, San Francisco, CA

Address correspondence to: Christina G. Dorell, MD, MPH, Centers for Disease Control and Prevention, Immunization Services Division, 1600 Clifton Rd. NE, MS E-62, Atlanta, GA 30333, Phone: 404-639-5198, Fax: 404-639-3266, ; Email: cdorell/at/cdc.gov.

This article has been cited by other articles in PMC.

Abstract

Objective

The validity of parent-reported adolescent vaccination histories has not been assessed. This study evaluated the validity of parent-reported adolescent vaccination histories by a combination of immunization card and recall, and by recall only, compared with medical provider records.

Methods

We analyzed data from the 2008 National Immunization Survey-Teen. Parents of adolescents aged 13–17 years reported their child's vaccination history either by immunization card and recall (n=3,661) or by recall only (n=12,822) for the hepatitis B (Hep B), measles-mumps-rubella (MMR), varicella (VAR), tetanus-diphtheria/tetanus-diphtheria-acellular pertussis (Td/Tdap), meningococcal conjugate (MCV4), and quadrivalent human papillomavirus (HPV4) (for girls only) vaccines. We validated parental report with medical records.

Results

Among the immunization card/recall group, vaccines with >20% false-positive reports included MMR (32.3%) and Td/Tdap (36.9%); vaccines with >20% false-negative reports included VAR (35.2%), MCV4 (36.0%), and Tdap (41.9%). Net bias ranged from −25.0 to −0.1 percentage points. Kappa values ranged from 0.22 to 0.92. Among the recall-only group, vaccines with >20% false-positive reports included Hep B (33.9%), MMR (61.4%), VAR (26.2%), and Td/Tdap (60.6%); vaccines with >20% false-negative reports included Hep B (58.9%), MMR (33.7%), VAR (51.6%), Td/Tdap (25.5%), Tdap (50.3%) MCV4 (63.0%), and HPV4 (20.5%). Net bias ranged from −46.0 to 0.5 percentage points. Kappa values ranged from 0.03 to 0.76.

Conclusions

Validity of parent-reported vaccination histories varies by type of report and vaccine. For recently recommended vaccines, false-negative rates were substantial and higher than false-positive rates, resulting in net underreporting of vaccination rates by both the immunization card/recall and recall-only groups. Provider validation of parent-reported vaccinations is needed for valid surveillance of adolescent vaccination coverage.

The routine adolescent vaccination schedule has expanded since 2005, adding four additional vaccination recommendations: the tetanus-diphtheria-acellular pertussis (Tdap), meningococcal conjugate (MCV4), quadrivalent human papillomavirus (HPV4), and influenza vaccines.¹^–⁴ The advent of these recommendations creates the need for vaccination providers to accurately screen patients for vaccination and provide information for vaccination coverage surveillance to monitor uptake of these new vaccines. Accurate surveillance and provider screening are needed to decrease missed opportunities for vaccination, identify undervaccinated populations, and monitor progress in achieving coverage goals. Parent-reported vaccination status is often used in medical settings and by certain surveillance instruments, but has been found to be inaccurate among young children.⁵^–¹⁰ The validity of parent-reported adolescent vaccination histories has not yet been assessed.

Validity studies have shown that, compared with the medical record, parental report often over- or underestimates the immunization status of young children.⁷^–¹³ The validity of a parent's recall of a child's vaccinations has been found to depend upon the type of vaccine, the age of the child, and the number of shots that the parent must recall.⁹ In addition to parental recall, household-retained immunization cards have been found to be insufficient sources for estimating vaccination coverage because they are often incomplete.⁷^,¹⁰^,¹² Yet, immunization cards have better agreement with provider-reported vaccination rates compared with parental recall.⁵ For these reasons, multiple sources of information have been recommended to improve the validity of reported vaccination rates among young children.⁸^,¹¹^,¹² However, provider-reported vaccination data are often viewed as the most accurate source of information about a child's vaccinations.¹³

To our knowledge, the validity of parent-reported adolescent vaccination histories has not yet been assessed. In the absence of a medical record, an immunization card or parental report is often used when providers need to assess a child's vaccination status or when parents respond to questionnaires used for vaccination coverage surveillance. Some nationally representative surveys that collect and rely on parental report of adolescent vaccinations include the Behavioral Risk Factor Surveillance System, the National Health and Nutrition Examination Survey, the National Health Interview Survey, the National Survey of Family Growth, and the National Survey of Children's Health.¹⁴ Currently, the National Immunization Survey-Teen (NIS-Teen) is the only nationally representative survey that confirms vaccinations through provider records. We used the NIS-Teen to compare parent-reported vaccination histories with provider-reported vaccination histories to assess the accuracy of parent-held immunization cards and parental recall.

METHODS

The NIS-Teen is composed of two phases: (1) a random-digit-dialed household survey used to identify adolescents aged 13–17 years and (2) a mailed provider survey collecting provider-reported vaccination histories for adolescents. The NIS-Teen represents a stratified national probability sample of households in the United States, including all 50 states and the District of Columbia. It is built on the sampling frame of telephone numbers used by the NIS, which seeks to identify vaccination rates in children 19–35 months of age. Methods and weighting procedures for the NIS and NIS-Teen have been described previously.¹⁵^,¹⁶

Households are first screened for children aged 19–35 months and then are administered the NIS. The household is then screened for adolescents 13–17 years of age. Households in which there is not a 19- to 35-month-old child are not administered the NIS interview, but are then screened for the presence of a 13- to 17-year-old adolescent. If more than one eligible adolescent lives in the household, one of the adolescents is randomly chosen. The adult who is most knowledgeable about the teen's vaccinations is administered a computer-assisted telephone interview. With consent of the adolescent's parent or guardian, the adolescent's health-care providers are contacted by mail to request information on vaccinations from their medical records.

For brevity, we will refer to the parent or guardian interviewed as the adolescent's parent. Interviewers collect information from the parent about household sociodemographic characteristics and the adolescent's immunization history. Some of the sociodemographic information includes parental report of the adolescent's age, race/ethnicity, health insurance type, household income, mother's highest education level, and having had a preventive health-care visit at age 11–12 years. We defined poverty status by using the reported household income and the 2007 federal poverty level thresholds defined by the U.S. Census Bureau. We determined the household metropolitan statistical area by the telephone area code.

Parents reported the adolescent's immunization history by reading from an immunization card or by recall (from memory). Interviewers asked respondents if an immunization card was available for them to refer to during the household interview. If an immunization card was available, parents were first asked to look at the card to determine if a vaccine had been given to the adolescent. If parents did not see that vaccine on the card, they were asked to recall if the adolescent had ever received that vaccine without it being recorded on the immunization card. Hereafter, we will refer to this group as the “immunization card/recall group.” Respondents were instructed to report the number of doses and the dates of vaccination for each vaccine. Respondents could answer “don't know” if they were unable to give the vaccination dates. If an immunization card was not available, parents were asked to recall if the adolescent ever received each vaccine. Hereafter, we will refer to this group as the “recall-only group.” Answers included “yes,” “no,” and “don't know.” If the respondent answered “yes,” the adolescent had received the vaccination, the number of doses was asked; respondents could answer with either a specific number of doses, “all shots,” or “don't know.”

The interviewers asked respondents to give contact information for all of the vaccination providers that the adolescent had seen since birth. After obtaining parental consent, interviewers contacted the adolescent's vaccination providers by mail and asked them to complete immunization history questionnaires (IHQs) for the adolescent. The IHQs were completed by clinic staff using the adolescent's medical record and returned by mail or fax. Returned immunization histories were reviewed for completeness. If an adolescent had more than one provider that returned an IHQ, multiple provider reports were combined into a single adolescent vaccination history, or “synthesized provider immunization history,” which was used to determine if the adolescent was up-to-date for the recommended vaccines.

Statistical analysis

Data from the 2008 NIS-Teen were analyzed using SAS® version 9.2 and SUDAAN® version 10.0 to account for its complex sampling design.¹⁷^,¹⁸ Our analysis was restricted to adolescents for whom all named vaccination providers returned an IHQ to reduce underreporting of vaccinations as a result of the scattering of their immunization records.¹⁹^,²⁰ Validity of parental report of the following vaccinations was assessed: ≥1 dose of tetanus-diphtheria or Tdap (Td/Tdap), ≥1 dose of Tdap, ≥1 dose of MCV4, ≥1 dose of HPV4, ≥3 doses of HPV4, ≥2 doses of measles-mumps-rubella (MMR), ≥3 doses of hepatitis B (Hep B), and ≥2 doses of varicella (VAR). Because some surveys ask parents about the receipt of any tetanus booster and older adolescents may have received Td before the licensure of Tdap, we assessed the validity of parental report of any tetanus booster and the receipt of Tdap specifically.²¹^,²²

We performed bivariate analyses to determine the weighted percentages and 95% confidence intervals (CIs) of select sociodemographic characteristics of adolescents in the immunization card/recall group and the recall-only group. We also performed bivariate analyses to assess vaccination coverage rates by parental and provider report. We used Wald-F Chi-square tests to determine associations between the type of parental report and (1) sociodemographic characteristics and (2) provider-reported vaccination coverage rates. We considered “don't know” responses from parents regarding vaccination coverage as “no” responses.

Four validity measures were determined for each vaccination for the immunization card/recall group and the recall-only group. These included (1) percent false-positive (1-specificity), which is the percentage of parental reports of an adolescent receiving a vaccine when he/she did not receive the vaccine according to the provider record; (2) percent false-negative (1-sensitivity), which is the percentage of parental reports of an adolescent not receiving a vaccine when he/she did receive the vaccine according to the provider record; (3) net reporting bias, which is the difference between parent- and provider-reported vaccination estimates; and (4) kappa, which is the measure of agreement between parental and provider reports. Although arbitrary, we used the following commonly used benchmarks to interpret kappa results: agreement was considered poor for a kappa value <0.00, slight for 0.00–0.20, fair for 0.21–0.40, moderate for 0.41–0.60, substantial for 0.61–0.80, and almost perfect for 0.81–1.00.²³^,²⁴ The NIS-Teen was approved by the Centers for Disease Control and Prevention's Institutional Review Board.

RESULTS

Response rate

In 2008, 30,725 parents completed the NIS-Teen household survey. The Council of American Survey Research Organizations response rate was 58.7% and was the product of the resolution rate, the screening completion rate, and the interview completion rate. The resolution rate is the percentage of the total telephone numbers selected that are classifiable as nonworking, nonresidential, or residential. The screening completion rate is the percentage of known households successfully screened for the presence of age-eligible adolescents. The interview completion rate is the percentage of households with one or more age-eligible adolescents who complete the household interview. Among those who completed the household survey, 58.1% had adequate provider-reported vaccination histories.²⁵ Among adolescents with adequate provider-reported vaccination histories, 91.4% had all named vaccination providers return an IHQ. “Don't know” and “refused” responses ranged from 1.3% to 14.3% among parents in the immunization card/recall group and from 2.6% to 33.2% among parents in the recall-only group.

Demographic characteristics by report type

Among household survey respondents, 21.3% of parents responded to the NIS-Teen by immunization card/recall, while 78.7% of parents responded by recall only. Demographic characteristics for the immunization card/recall and recall-only groups are presented in Table 1.

Table 1.

Demographic characteristics of households that provided immunization histories from either parental recall only or from immunization cards and recall: National Immunization Survey-Teen, 2008

Validity measures

Vaccination coverage rates by parental and provider report are listed in Table 2. Provider-reported vaccination coverage estimates were higher among the immunization card/recall group compared with the recall-only group for all vaccines (p<0.02). The difference (net reporting bias) between parent-reported estimates given by immunization card/recall and provider-reported estimates was −7.6 percentage points for ≥3 doses of Hep B, −4.5 for ≥2 doses of MMR, −25.0 for ≥2 doses of VAR, −6.7 for ≥1 dose of Td/Tdap, −2.9 for ≥1 dose of Tdap, −12.2 for ≥1 dose of MCV4, and, among girls, −0.1 for ≥1 dose of HPV4 and –0.6 for ≥3 doses of HPV4. The difference between parent-reported estimates given by recall only and provider-reported estimates was −46.0 percentage points for ≥3 doses of Hep B, −22.1 for ≥2 doses of MMR, −9.5 for ≥2 doses of VAR, −1.6 for ≥1 dose of Td/Tdap, −4.5 for ≥1 dose of Tdap, −18.3 for ≥1 dose of MCV4, and, among girls, −1.2 for ≥1 dose of HPV4 and 0.5 for ≥3 doses of HPV4.

Table 2.

Vaccination coverage estimates provided by immunization card and recall or parental recall only: National Immunization Survey-Teen, 2008

Validity measures that compare parent- and provider-reported estimates from the immunization card/recall group are listed in Table 3. Compared with the provider report, parent-reported estimates from the immunization card/recall group had false-positive percent values ranging from 1.0% (≥3 doses of Hep B) to 36.9% (≥1 dose of Td/Tdap) and false-negative percent values ranging from 5.0% (≥2 doses of MMR) to 41.9% (≥1 dose of Tdap). Vaccines with >20% false-positive reports were MMR (32.3%) and Td/Tdap (36.9%). Vaccines with >20% false-negative reports were VAR (35.2%), Tdap (41.9%), and MCV4 (36.0%). Kappa values ranged from 0.218 to 0.920.

Table 3.

Validity of parent-reported vaccination status by immunization card and recall for adolescents aged 13–17 years at time of interview: National Immunization Survey-Teen, 2008 (n=3,661)

Validity measures that compare parent- and provider-reported estimates from the recall-only group are listed in Table 4. Compared with the provider report, parent-reported estimates from the recall-only group had false-positive percent values ranging from 4.9% (≥3 doses of HPV4) to 61.4% (≥2 doses of MMR) and false-negative percent values ranging from 16.6% (≥1 dose of HPV4) to 63.0% (≥1 dose of MCV4). Vaccines with >20% false-positive reports were Hep B (33.9%), MMR (61.4%), VAR (26.2%), Td/Tdap (60.6%), and Tdap (25.9%). Vaccines with >20% false-negative reports were Hep B (58.9%), MMR (33.7%), VAR (51.6%), Td/Tdap (25.5%), Tdap (50.3%), MCV4 (63.0%), and HPV4 (20.5%). Kappa values ranged from 0.028 to 0.761.

Table 4.

Validity of parent-reported vaccination status by recall only for adolescents aged 13–17 years at time of interview: National Immunization Survey-Teen, 2008 (n=12,822)

DISCUSSION

This is the first validation study of parent-reported vaccinations compared with provider reports from medical records for adolescents. We found that parent-reported vaccination estimates using both immunization card and recall were more accurate than parent-reported estimates using recall only, and the validity of parent-reported vaccination estimates differed by vaccine type. Validity measures differed for routinely recommended childhood-administered vaccines (Hep B and MMR) and adolescent-administered and catch-up vaccines (VAR, Tdap, MCV4, and HPV4). Adolescents who participated in this study were born from 1990 through 1996 and turned 11 years of age from 2001 through 2007; adolescent vaccinations were recommended beginning in 2005.

Parents who reported childhood-administered vaccinations by immunization card/recall had moderate accuracy of reporting for Hep B compared with the provider report, but fair accuracy for MMR. However, parents who reported childhood-administered vaccinations by recall only had slight accuracy for both Hep B and MMR, with large net underreporting biases. Compared with childhood-administered vaccinations, parental report of adolescent-administered vaccinations by immunization card/recall was better, ranging from moderate to almost perfect accuracy. However, parental report of adolescent-administered vaccinations by recall only had fair accuracy for most vaccines, with larger net underreporting biases compared with the immunization card/recall group.

These findings are similar to those made by Suarez et al., who measured the validity of parents' reports of their children's vaccination status by recall. They found that the validity of a parent's recall depended upon the vaccine, and it decreased with increasing age of the child at vaccination and with an increasing number of vaccines that the parent had to remember.⁹ Similarly, the kappa values among both reporting groups in our study were lowest for vaccines given during childhood and highest for vaccines given during adolescence. The only exception is Hep B among the immunization card/recall group, which had moderate reporting accuracy, similar to that of the adolescent-administered vaccines. Using the immunization card may have removed some of the recall bias that made recalling older vaccinations harder compared with recalling more recent vaccinations. Incomplete immunization cards still may have contributed to inaccurate reporting from some parents among the immunization card/recall group.

HPV4 had the highest kappa values compared with the other adolescent-administered vaccines for both reporting groups. Gardasil®, the quadrivalent HPV vaccine (manufactured by Merck and Co., Inc., Whitehouse Station, New Jersey) has been the subject of a large marketing campaign by both Merck and professional medical associations.²⁶^,²⁷ It has also had significant media coverage, which has increased public knowledge about HPV and its association with cervical cancer.²⁸^,²⁹ The wide coverage of HPV4 may have made this vaccine easier to recall compared with other adolescent-administered vaccines (VAR, Tdap, and MCV4) that have not received equal media coverage. This may have contributed to the high reporting accuracy among parents. Similarly, because HPV is a sexually transmitted infection, discussions with physicians about HPV4 may have been more memorable to parents. For many parents, discussing their adolescent's sexual activity may be a sensitive topic, compared with discussing vaccines for non-sexually transmitted diseases. The negligible net bias for HPV vaccination from parental report can also be partly explained by the balancing of false-positive and false-negative reports. The false-negative rate ranged from 5.0% to 20.5%, depending on the reporting group and number of HPV doses.

Furthermore, as adolescent vaccination rates increase, the measures of association for parent-reported vaccination rates may change. With increasing knowledge of adolescent-administered vaccines and higher vaccine uptake, the validity of parental report may improve. On the other hand, as these vaccinations become more routine, the validity of parental report may also decline as parents pay less attention to the administration of these specific vaccines. Because these measures of association may change over time, the validity of parental report should be reevaluated periodically as vaccination uptake and knowledge about the vaccines improve.

In our study, reporting an adolescent's vaccination status from parent-held immunization card/recall gave more accurate vaccination estimates compared with reporting by parental recall only. Additionally, adolescents of parents who reported by immunization card/recall were more likely to be vaccinated with each vaccine compared with adolescents from the recall-only group. Having an immunization card available has been associated with higher vaccination coverage among young children.⁹^,³⁰ Yet, only 21.3% of respondents to our household survey had an -immunization card available. This percentage is less than the 32.2% (unweighted) of parents who reported by immunization card/recall in the 2008 NIS for children 19–35 months of age.³¹ The availability of an immunization card among parents participating in validity studies of young children has ranged from 21% to 72% and differs by study setting.⁷^,⁹^,¹⁰^,³²^,³³ Few parents carry immunization cards with them when accessing acute or preventive health care for their young children.⁷^,¹⁰^,³³ As children age into adolescence, they have fewer preventive health-care visits, and immunizations become less frequent.³⁴ Also, as participation in immunization information systems (IISs) increases, parents may be less motivated to carry immunization cards and keep them current. Consequently, parents of adolescents may then be less likely to have an easily accessible immunization card in the home or to bring it to a health visit. However, it has been suggested that immunization cards improve communication between parents and providers and prompt discussions about vaccinations.³⁰

The recommendation to use each medical visit as an opportunity to evaluate one's vaccination status can be hindered when accurate vaccination information is not available. As seen among younger children, practitioners who receive inaccurate vaccination information may miss opportunities to vaccinate vulnerable adolescents or may even vaccinate when unnecessary.³³ To improve vaccination histories provided by parents in the absence of medical records, parents and adolescents should be encouraged to bring the adolescent's immunization card to all medical encounters. Likewise, vaccination providers should reissue new immunization cards if they are missing. Bringing the immunization card to all medical visits should improve the completeness of the card, allowing the same immunization card to be updated with each vaccination. Similarly, vaccination-provider participation in the state's IIS is a necessary component for developing comprehensive immunization registries that could be accessed when evaluating an adolescent's vaccinations. IISs allow for accurate vaccination surveillance at state and local levels, aid the identification of subgroups of unvaccinated individuals, and assist in the information exchange between different vaccination providers and facilities, which leads to improved vaccination coverage.³⁵^,³⁶

Limitations

This analysis had a few limitations. It is unclear if parents who did not respond with immunization cards did not actually possess an immunization card. Misclassification bias may have occurred if parents who had an immunization card available decided to respond by recall. A higher proportion of parents from the recall-only group responded “don't know” about their adolescent's vaccination status compared with parents from the immunization card/recall group. Because “don't know” responses were considered “no” responses, misclassification bias may have contributed to higher false-positive and false-negative rates in both reporting groups.

Another limitation was the possibility of incomplete provider-reported data. Administered vaccines that are missing from provider records may contribute to higher false-positive rates. Although weighting adjustments were made to account for nonresponse and households without landline telephones, some response bias may remain. Finally, our analysis was limited to adolescents with records from all named vaccination providers among those with adequate provider data, comprising 53% of those with completed household surveys. Potential bias resulting from limiting our analysis to this group may have led to higher reported vaccination estimates and lower false-positive rates compared with the general population, which may have more record scattering among providers.

CONCLUSIONS

Our study is the first to evaluate parental report of adolescent vaccination coverage, and our findings are similar to those of validity studies of parental report of vaccination coverage among young children. In the absence of a single and complete record of all vaccinations received by each adolescent, vaccination screening and surveillance monitoring is limited by the accuracy of parental recall, immunization cards, and medical records.

Parents and adolescents should be encouraged to bring the adolescent's immunization card to each health-care visit and vaccination site (e.g., school or workplace) to improve the completeness of the immunization card. Also, universal participation in a state's IIS is needed to provide a single, complete immunization record for each adolescent. Until there is universal participation, providers and surveillance-monitoring systems should use medical records, and IISs when possible, to assess an adolescent's immunization status. Also, the limitations of parental report and immunization cards should be understood when used to assess vaccinations. Because the validity of parent-reported vaccination status may differ by subgroup, such as those with lower vaccination rates or different racial/ethnic minority groups, or as vaccine uptake and knowledge improve, future studies should assess these differences.

REFERENCES

1. Fiore AE, Shay DK, Broder K, Iskander JK, Uyeki TM, Mootrey G, et al. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2009. MMWR Recomm Rep. 2009;58(RR-8):1–52. [PubMed]

2. Markowitz LE, Dunne EF, Saraiya M, Lawson HW, Chesson H, Unger ER. Quadrivalent human papillomavirus vaccine: recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Recomm Rep. 2007;56(RR-2):1–24. [PubMed]

3. Bilukha OO, Rosenstein N. Prevention and control of meningococcal disease. Recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Recomm Rep. 2005;54(RR-7):1–21. [PubMed]

4. Broder KR, Cortese MM, Iskander JK, Kretsinger K, Slade BA, Brown KH, et al. Preventing tetanus, diphtheria, and pertussis among adolescents: use of tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccines recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Recomm Rep. 2006;55(RR-3):1–34. [PubMed]

5. Bolton P, Holt E, Ross A, Hughart N, Guyer B. Estimating vaccination coverage using parental recall, vaccination cards, and medical records. Public Health Rep. 1998;113:521–6. [PMC free article] [PubMed]

6. Czaja C, Crossette L, Metlay JP. Accuracy of adult reported pneumococcal vaccination status of children. Ann Epidemiol. 2005;15:253–6. [PubMed]

7. Goldstein KP, Kviz FJ, Daum RS. Accuracy of immunization histories provided by adults accompanying preschool children to a pediatric emergency department. JAMA. 1993;270:2190–4. [PubMed]

8. Luman ET, Ryman TK, Sablan M. Estimating vaccination coverage: validity of household-retained vaccination cards and parental recall. Vaccine. 2009;27:2534–9. [PubMed]

9. Suarez L, Simpson DM, Smith DR. Errors and correlates in parental recall of child immunizations: effects on vaccination coverage estimates. Pediatrics. 1997;99:E3. [PubMed]

10. Williams ER, Meza YE, Salazar S, Dominici P, Fasano CJ. Immunization histories given by adult caregivers accompanying children 3–36 months to the emergency department: are their histories valid for the Haemophilus influenzae B and pneumococcal vaccines? Pediatr Emerg Care. 2007;23:285–8. [PubMed]

11. Rosenthal J, Rodewald L, McCauley M, Berman S, Irigoyen M, Sawyer M, et al. Immunization coverage levels among 19- to 35-month-old children in four diverse, medically underserved areas of the United States. Pediatrics. 2004;113:e296–302. [PubMed]

12. Rodewald L, Maes E, Stevenson J, Lyons B, Stokley S, Szilagyi P. Immunization performance measurement in a changing immunization environment. Pediatrics. 1999;103:889–97. [PubMed]

13. Kroger AT, Atkinson WL, Marcuse EK, Pickering LK. General recommendations on immunization: recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Recomm Rep. 2006;55(RR-15):1–48. [PubMed]

14. Tiro JA, Saraiya M, Jain N, Liddon N, Cokkinides V, Lai SM, et al. Human papillomavirus and cervical cancer behavioral surveillance in the US. Cancer. 2008;113(10 Suppl):3013–30. [PubMed]

15. Smith PJ, Battaglia MP, Huggins VJ, Hoaglin DC, Rodén A, Khare M, et al. Overview of the sampling design and statistical methods used in the National Immunization Survey. Am J Prev Med. 2001;20(4 Suppl):17–24. [PubMed]

16. Jain N, Singleton JA, Montgomery M, Skalland B. Determining accurate vaccination coverage rates for adolescents: the National Immunization Survey-Teen, 2006. Public Health Rep. 2009;124:642–51. [PMC free article] [PubMed]

17. SAS Institute, Inc. SAS®: Version 9.2. Cary (NC): SAS Institute, Inc; 2008.

18. Research Triangle Institute. SUDAAN®: Version 10.0. Research Triangle Park (NC): Research Triangle Institute; 2008.

19. Stokley S, Rodewald LE, Maes EF. The impact of record scattering on the measurement of immunization coverage. Pediatrics. 2001;107:91–6. [PubMed]

20. Smith PJ, Stevenson J. Racial/ethnic disparities in vaccination coverage by 19 months of age: an evaluation of the impact of missing data resulting from record scattering. Stat Med. 2008;27:4107–18. [PubMed]

21. Jain N, Stokley S, Cohn A. Receipt of tetanus-containing vaccinations among adolescents aged 13 to 17 years in the United States: National Immunization Survey-Teen 2007. Clin Ther. 2010;32:1468–78. [PubMed]

22. Centers for Disease Control and Prevention (US). Recommended immunization schedule for persons aged 7–18 years—United States, 2008. Atlanta: CDC; 2008. [cited 2010 Oct 15]. Also available from: URL: http://pediatrics.aappublications.org/cgi/data/121/1/219/DC1/1.

23. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. [PubMed]

24. Rigby AS. Statistical methods in epidemiology. V. Towards an understanding of the kappa coefficient. Disabil Rehabil. 2000;22:339–44. [PubMed]

25. Centers for Disease Control and Prevention (US). National Immunization Survey-Teen: a user's guide for the 2008 public-use data file. [cited 2010 May 26]. Available from: URL: ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NIS/NISteenPUF08_DUG.pdf.

26. Rothman SM, Rothman DJ. Marketing HPV vaccine: implications for adolescent health and medical professionalism. JAMA. 2009;302:781–6. [PubMed]

27. Herskovits B. Brand of the year. Pharm Exec. 2007;27:58–65.

28. Habel MA, Liddon N, Stryker JE. The HPV vaccine: a content analysis of online news stories. J Womens Health (Larchmt) 2009;18:401–7. [PubMed]

29. Kelly BJ, Leader AE, Mittermaier DJ, Hornik RC, Cappella JN. The HPV vaccine and the media: how has the topic been covered and what are the effects on knowledge about the virus and cervical cancer? Patient Educ Couns. 2009;77:308–13. [PubMed]

30. McElligott JT, Darden PM. Are patient-held vaccination records associated with improved vaccination coverage rates? Pediatrics. 2010;125:e467–72. [PubMed]

31. Centers for Disease Control and Prevention (US). National Immunization Survey: a codebook for the 2008 public-use data file. [cited 2010 Jan 25]. Available from: URL: ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NIS/NISPUF08_CODEBOOK.PDF.

32. AbdelSalam HH, Sokal MM. Accuracy of parental reporting of immunization. Clin Pediatr (Phila) 2004;43:83–5. [PubMed]

33. Watson MA, Feldman KW, Sugar NF, Sommer CJ, Thomas ER, Lin T. Inadequate history as a barrier to immunization. Arch Pediatr Adolesc Med. 1996;150:135–9. [PubMed]

34. Rand CM, Shone LP, Albertin C, Auinger P, Klein JD, Szilagyi PG. National health care visit patterns of adolescents: implications for delivery of new adolescent vaccines. Arch Pediatr Adolesc Med. 2007;161:252–9. [PubMed]

35. Hinman AR, Ross DA. Immunization registries can be building blocks for national health information systems. Health Aff (Millwood) 2010;29:676–82. [PubMed]

36. Schauer SL, Maerz TR, Hurie MB, Gabor GW, Flynn JM, Davis JP. The use of an immunization information system to establish baseline childhood immunization rates and measure contract objectives. J Public Health Manag Pract. 2009;15:E6–12. [PubMed]

Articles from Public Health Reports are provided here courtesy of
Association of Schools of Public Health