|Home | About | Journals | Submit | Contact Us | Français|
Despite improvements in New York City (NYC) residents' health in the past decade,1 disparities exist between economically disadvantaged and advantaged neighborhoods.2 In 2002, the NYC Department of Health and Mental Hygiene (DOHMH) created three District Public Health Offices (DPHOs) in the impoverished neighborhoods of the South Bronx, East and Central Harlem, and North and Central Brooklyn. The DPHOs aim to promote health equity in these areas. The Bronx DPHO targeted teenage pregnancy rates, which are persistently higher in the South Bronx than in the United States. In 2007, the South Bronx teen pregnancy rate was 133 per 1,000 females aged 15–19 years, compared with 72 per 1,000 in the U.S.3,4
We hypothesized that in addition to neighborhood determinants of poverty and education, the high teen pregnancy rate in the South Bronx was partly attributable to low hormonal contraceptive use. Findings from the 2005 Youth Risk Behavior Survey revealed that South Bronx students reported relatively high condom use (68%) but extremely low birth control pill use (6%) compared with U.S. teenagers (63% and 18%, respectively).5,6 A Bronx survey also found that teens with access to confidential services were three times as likely to initiate gynecologic care compared with teens who lacked such access.7
The DPHO's plan to reduce the teen pregnancy rate included implementing school-linked health care (SLHC), an evidence-based strategy for increasing access to basic sexual and reproductive health care (e.g., pregnancy testing, options counseling, contraceptive services, and sexually transmitted disease [STD]/human immunodeficiency virus [HIV] testing and treatment) by linking students to services via a trusting and caring adult.8 In Baltimore, Maryland, SLHC led to a 30% reduction in program high schools' teen pregnancy rates.9
Before work on this strategy could begin, it was necessary to conduct a needs assessment to determine which South Bronx medical practices provided services in a “teen-friendly” way. Teen-friendly clinics would be considered potential partners for SLHC, and practices that fared less well would be eligible for a series of interventions, aligned with DOHMH's Healthy Teens Initiative.10
This article presents results from a novel survey instrument used to assess the teen friendliness of all known South Bronx medical practices that see teens for sexual and reproductive health care. We then discuss the development, implementation, and evaluation of an intervention designed to address the lack of teen friendliness that the needs assessment revealed.
A structured mystery shopper telephone survey was developed to assess teen friendliness. We employed the mystery shopper methodology—wherein confederates pose as service-seekers—to assess how frontline staff members interact with clients when unaware of being observed. We chose a telephone modality as opposed to face-to-face interaction, as it requires fewer resources and allows the same shoppers to assess practices more than once. The NYC DOHMH Institutional Review Board exempted this research from human subjects oversight.
Teen friendliness was conceptualized as consisting of two components: disclosure and customer service. Information regarding frontline staff's disclosure was objectively gathered using an interactive grid through which clinics were awarded points based upon when during the call medical practice staff mentioned (1) confidentiality, (2) parental involvement, (3) insurance, and (4) whether the teenager could be seen regardless of her ability to pay. For example, if staff mentioned confidentiality after the scripted introduction during which the caller stated her age and reason for calling, the response was awarded three points. If confidentiality was mentioned after a scripted, purposefully broad prompt, two points were earned. To earn one point, staff had to discuss confidentiality after the caller inquired about it more directly. No points were awarded if confidentiality was never mentioned. Scores on all four variables were summed to create the disclosure scale score. The script was developed in collaboration with teenage members of the DPHO's Youth Advisory Board (YAB) to assure that the prompts the caller used were realistic.
Customer service scores were based upon Likert scale ratings of how knowledgeable, friendly, and comforting front desk staff presented themselves. Ratings on these three variables were summed to create the customer service scale. Finally, callers indicated whether they would seek care from the practice they had called.
During the summer of 2006, two rounds of calls were made to the 93 known South Bronx medical practices providing adolescent health care—one round by a college intern and another by two YAB members. Callers posed as 15-year-old girls seeking oral contraceptives. To assure reliability of caller ratings, training consisted of verbal instruction, practice, and observed telephone calls. Calls were considered complete if callers spoke with clinic staff, were instructed to leave a message on an automated system, or allowed the telephone to ring 15 times.
A total of 173 calls were completed. Eighty practices received two calls and 13 received one call. All calls were given equal weight in the analyses. There was no answer during business hours 6% of the time. The average call lasted 3.6 minutes.
Potential disclosure scale values ranged from 0 (no disclosure of important information) to 12 (unprompted disclosure on all four variables). Scores above the midpoint of 6 were considered teen friendly. The mean scale score was 2.6, and 79 calls (46%) resulted in scores of 0. Only 12 calls (7%) resulted in disclosure scores above the scale midpoint. No practice assessed twice earned teen-friendly scores on both calls, and no practice assessed once earned a score higher than 6. Scale scores did not significantly differ between the two rounds of calls.
Customer service scale scores ranged from 0 (very negative demeanor) to 6 (very positive demeanor). The scale had excellent internal consistency (Chronbach's a = 0.83). Overall, 75 of the staff (43%) were judged positively (scores greater than the midpoint of 3), whereas 62 (36%) were rated negatively. The college intern rated the staff more positively (mean score = 3.9) than did the YAB callers (mean score = 2.8, p<0.01).
More than half (51%) of callers responded that they would not seek care from the medical practice based upon their interaction with the frontline staff. About 18% said they would seek care and 31% were uncertain. The YAB members were significantly less willing to seek care than the college intern was (p<0.01).
Because no practice was deemed teen friendly, we developed a two-hour training called HELP Teens: The Importance of Our Frontline Medical Practice Staff, which could be delivered to all practices. Based on adult learning theory, consultation with local experts, and a four-site pilot, HELP Teens covers minors' rights to confidential health care under New York State law and basic customer service principles. It features a referral guide, role-play activities, and a desktop prompt for communicating with teens seeking sexual and reproductive health care that is customized to each practice's policies. Because clinic policies vary, the purpose of the front desk prompt was to facilitate full disclosure of the site-specific confidentiality, parental involvement, and payment policies to every teen who calls.
In the spring of 2008, a team consisting of six public health educators (PHEs) was trained to deliver HELP Teens. PHEs visited each site and spoke with the medical director or lead site administrator to gauge interest. Of the 105 known practices operating in 2008, 75 agreed to the training. By March 2009, all trainings were completed.
After the training, participants evaluated the trainers and material covered. Responses were overwhelmingly positive: on a Likert scale of 1 (strongly disagree) to 4 (strongly agree), participants strongly agreed (mean rating ≥3.5) that the materials were useful; the trainer was prepared, organized, and knowledgeable; and they felt confident in their abilities to refer teens to services and discuss minors' rights to sexual and reproductive health care. Qualitative data similarly revealed that trainees enjoyed HELP Teens and found the material important for their work.
Once a training date for a site was set, three mystery shopper calls were made during the two weeks prior. To assess changes in frontline behavior resulting from the training, three follow-up mystery shopper calls were made one month post-intervention, and another three calls were made six months post-intervention. For these evaluation calls, caller age (14–16 years) and method of contraception sought (pills, Depo-Provera®, Nuva-ring®, or patch) were randomly assigned and adjusted for in the analyses.
At all three time intervals (pre-training, one-month follow-up, and six-month follow-up), 6% to 7% of calls went unanswered. There was no improvement in either disclosure or customer service scale scores from pre-to post-intervention (Table).
Our needs assessment revealed that although there are ample medical practices in the South Bronx providing adolescent health care, existing services were unwelcoming toward teenagers. Almost no frontline staff disclosed information that encouraged teens to seek care, and fewer than half presented themselves as friendly, knowledgeable, and comforting. Unfortunately, despite a well-designed and received intervention based on the needs assessment, the training we developed to minimize these barriers was ineffective.
In terms of identifying potential SLHC partners, results of the assessment forced us to reconsider our selection criteria. We initially wanted teen friendliness as a prerequisite; however, our assessment showed that we could not maintain that requirement, as no practice qualified as teen-friendly. We have since linked with four partner clinics that have all made significant changes to their clinic practices in terms of teen accessibility, including more intense staff training and implementing special teen office hours and events. Evaluation data from those linkages are promising.
Medical practice behavior is less amenable to one-time trainings than we thought. While it is possible that the HELP Teens training itself was inadequate or that the mystery shopper instrument was not a sensitive enough evaluation tool, it seems more likely that rapid frontline staff turnover, the need to maintain front desk coverage during training, and lack of buy-in from those who determine medical practice procedures made a two-hour training insufficient. From working with our four SLHC partners, we have learned that continuous dialogue and check-in with clinic staff are essential to promoting and maintaining change in a medical setting.
This study had some limitations. Phone calls may not be reflective of the reception teenagers would receive in person or from other clinic staff. Additionally, despite training on the instrument, inter-rater reliability was not formally assessed via calculation of a kappa statistic. However, there were no differences between the college intern and the YAB members in terms of their disclosure scale ratings of the same medical practices, indicating consistent use of the scoring system. And though the customer service scores were subjective, the decision to seek care is itself subjective, and the variables assessed were uncomplicated and easily defined.
While the DPHO still aims to improve sexual and reproductive health-care accessibility for teens and to increase hormonal contraceptive use in this population, our office is not currently pursuing efforts to change medical practice behavior via brief interventions. Instead, as we scale up our SLHC program, we will work more intensely with willing clinic partners to change their frontline staff procedures in terms of both disclosing important information to teens and general customer service. While our ultimate goal is to decrease the teen pregnancy rate in the South Bronx by increasing the dual use of condoms along with another effective form of birth control, improving clinic teen friendliness and access to sexual and reproductive health care are important intermediate outcomes. Though our large-scale intervention proved ineffective, we believe more focused efforts with enthusiastic medical practices hold promise.
The authors acknowledge the contributions of the New York City Department of Health and Mental Hygiene (NYC DOHMH) staff and interns, and local and national experts who provided guidance on plans, including the NYC DOHMH Bronx District Public Health Office Youth Advisory Board.