|Home | About | Journals | Submit | Contact Us | Français|
The coarse pitch information in cochlear implants might hinder the development of singing in prelingually-deafened pediatric users. In the present study, seven prelingually-deafened children with cochlear implants (5.4 to 12.3 years old) sang one song that was the most familiar to him or her. The control group consisted of 14 normal-hearing children (4.1 to 8.0 years old). The fundamental frequencies (F0) of each note in the recorded songs were extracted. The following five metrics were computed based on the reference music scores: (1) F0 contour direction of the adjacent notes, (2) F0 compression ratio of the entire song, (3) mean deviation of the normalized F0 across the notes, (4) mean deviation of the pitch intervals, and (5) standard deviation of the note duration differences. Children with cochlear implants showed significantly poorer performance in the pitch-based assessments than the normal-hearing children. No significant differences were seen between the two groups in the rhythm-based measure. Prelingually-deafened children with cochlear implants have significant deficits in singing due to their inability to manipulate pitch in the correct directions and to produce accurate pitch height. Future studies with a large sample size are warranted in order to account for the large variability in singing performance.
Singing is universal across regions and cultures. It conveys important communicative, social, and religious functions. Singing is also where language and music intertwine, both of which define us as being human (Patel, 2008). The time of the onset of spontaneous singing in typically-developing children is a matter of debate. Dowling (1999) suggested that spontaneous singing starts to occur around nine months to one year of age. Around 18 months of age, children start to discover the ability to imitate sounds of others and produce tunes that are repeatable and sometimes self-made (Ostwald, 1973). It is not until the age of 4 to 5 years, however, that children develop consistently accurate pitch matching and interval reproduction (Flowers and Dunn-Sousa, 1990).
Speech perception performance by cochlear implant recipients has shown great improvement due to tremendous advances in multichannel cochlear implant technology in the past two decades [see Zeng (2004) for a review]. Prelingually-deafened children with cochlear implants have demonstrated excellent intelligibility in their speech production (Tobey et al., 2003). However, most cochlear implant recipients still demonstrate poor music perception in general [see McDermott (2004) for a review]. Studies on music perception in cochlear implant recipients typically show deficient performance in pitch-related tasks such as pitch ranking, pitch perception, melody contour identification, and familiar melody recognition. On the other hand, cochlear implant recipients perform as well as normal-hearing listeners in rhythm-based tasks, such as rhythm discrimination and rhythm test (e.g., Cooper et al., 2008; Fujita and Ito, 1999; Galvin et al., 2007; Gfeller et al., 1998, 2002; Kong et al., 2004; Nimmons et al., 2008; Pijl et al., 1997). Those results were consistent with our understanding about pitch perception and electrical hearing. Current cochlear implant technology provides precise rhythmic information, but fails to encode adequate pitch information in the electrical stimulations [see Moore (2003) for a review].
Although the studies discussed thus far have focused on postlingually-deafened adults with cochlear implants, a few recent studies on melody or song recognition in children with cochlear implants also demonstrated significant deficits (Stordahl, 2002; Vongpaisal et al., 2006; Mitani et al., 2007; Hsiao et al., 2008). Children with cochlear implants can identify songs at better than chance levels with either the original vocal or instrumental versions (Vongpaisal et al., 2006; Mitani et al., 2007; Hsiao et al., 2008). It is important to note the distinction between postlingually-deafened adults and prelingually-deafened children cochlear implant users. The postlingually-deafened adults know what music sounds like and remember many of the songs that they had heard before becoming profoundly deaf, whereas the prelingually-deafened children have little or no acoustic hearing and grow up hearing through a cochlear implant.
Poor pitch perception might result in poor vocal pitch production in prelingually-deafened children with cochlear implants. Previous studies have documented deficits in lexical tone production in tone-language-speaking children with cochlear implants (Xu et al., 2004; Peng et al., 2004; Han et al., 2007; Zhou and Xu, 2008). Averaged tone production intelligibility from a group of 14 prelingually-deafened children with cochlear implants was only 48% correct. The tone production intelligibility of 14 age-matched normal-hearing children was 78% correct (Han et al., 2007). In particular, the prelingually-deafened children with cochlear implants produced the flat tone (tone 1 of Mandarin Chinese) with relatively high accuracy (71.8% correct), the falling tone (tone 4 of Mandarin Chinese) with moderate accuracy (55.2% correct), and the rising tone (tone 2 of Mandarin Chinese) with poor accuracy (21.6% correct) (Zhou and Xu, 2008). Consistent with these findings, Peng et al. (2007) also revealed deficits in the production of rising intonation in native English-speaking pediatric cochlear implant users. Great individual differences in tone production performance in children with cochlear implants also exist, a part of which could be accounted for by their age at implantation (Han et al., 2007).
Recently, Nakata et al. (2006) studied vocal singing in 12 congenitally-deafened children (4.9 to 10.3 years of age) who had received cochlear implants and six normal-hearing children (6.7 to 9.9 years of age), who served as controls. The report indicated that children with cochlear implants were capable of singing familiar songs from memory, although their vocal pitch patterns were largely unrelated to the direction of pitch patterns in the target songs (Nakata et al., 2006). The children with cochlear implants, however, had age-appropriate rhythm reproduction (i.e., relative timing). Notably, all 12 deaf children had hearing aid experience prior to cochlear implantation. Nine of the 12 deaf children wore hearing aids on the contralateral side of the implants after cochlear implantation. Thus, it is not clear whether the use of hearing aids or cochlear implants has contributed to the ability of singing, or the lack thereof, in deaf children. Limited data on music perception in cochlear implant users with residual hearing suggest that the use of low-frequency acoustic hearing provides benefit in perception of real world musical stimuli (Gfeller et al., 2006).
The present study was designed to replicate and expand on the work of Nataka et al. (2006). We recruited a group of prelingually-deafened children with cochlear implants and attempted to determine the proficiency of their singing using a quantitative approach based on acoustical analyses of the sung notes. We hypothesized that children with cochlear implants would show impairment in pitch-based measures, but not in rhythm-based measures. We further hypothesized that children with cochlear implants would make more errors in ascending intervals than in descending intervals. Given that there is a lack of reliable evaluation tools for music perception in children with cochlear implants, objective evaluations of singing proficiency, as demonstrated in the present study, can provide a glimpse of the music ability of these children.
Seven prelingually-deafened children who received Advanced Bionics CII or 90K cochlear implants at Beijing Tongren Hospital participated in the present study. The only inclusion criterion was that the child or the parents claimed that the child could sing. The seven subjects were between the ages of 5.4 and 12.3 years (Mean±SD, 7.70±2.87). The ages at implantation ranged from 1.7 to 9.5 years old (Mean±SD, 4.26±3.0). The subjects were all fit with HiResolution or HiResolution 120 sound-processing strategy (Firszt, 2004; Koch et al., 2007; Firszt et al., 2009; Han et al. 2009), and the durations of implant use ranged from 1.7 to 5.3 years (Mean±SD, 3.44±1.23). Subjects CI 2 and CI 4 used bilateral hearing aids for 4 years and 1 year before cochlear implantation, respectively. The other five subjects had little or no history of hearing aid use before cochlear implantation. None of the children continued to use hearing aids after cochlear implantation. Details of the demographic information are provided in Table 1. As controls, 14 normal-hearing, typically-developing children between the ages of 4.1 and 8.0 years (Mean±SD, 5.88±1.2) were also recruited to participate in the present study. The individual ages of the normal-hearing subjects, NH 1 through NH 14, were 7.5, 6.9, 8.0, 4.8, 4.2, 5.5, 5.1, 6.2, 6.5, 7.2, 5.7, 5.9, 4.1, and 4.7 years, respectively. The normal-hearing status was based on parent report. The use of human subjects was approved by the Institutional Review Boards of Ohio University and Beijing Tongren Hospital.
Vocal singing samples from both normal-hearing children and children with cochlear implants were recorded in a quiet office. Each child was asked to sing a song that he or she could sing the best. An Electro Voice omnidirectional microphone (Model RE50B) connected to an external sound card was used for the recording. The distance between the lips and the microphone was kept at approximately 10 cm. The output of the microphone was sent to a laptop computer at a sampling rate of 44.1 kHz with a 16-bit resolution. Allowing the subject to select a song of his or her own choice, as used in Nataka et al. (2006), was “child friendly” and ensured familiarity of the child with the material. There are some limitations to asking each child to sing a song of his or her choice. Although the most frequent song choice was the Chinese version of Frère Jacques, which 10 out of the 21 children preferred, the remaining children’s selection of songs varied. Due to the assortment of songs chosen, the music structures (number of notes, pitch range, level of difficulties, etc.) were inconsistent. This may have influenced the accuracy in the evaluation of the children’s singing proficiency. One way to compensate for the lack of control of song choice is to administer a standardized task, such as repeating back a short melody within a chosen pitch range (e.g., Ramsey, 1983) or teaching the child a new, unfamiliar song and then assessing the singing performance of the child (e.g., Flowers and Dunne-Sousa, 1990).
The recorded vocal singing samples were subject to a number of acoustical analyses. The sung notes were isolated using a sound processing program [CoolEdit 2000 (Syntrillium Software, Scottsdale, AZ)]. The fundamental frequencies (F0) of each note were computed from the steady-state portions of the produced vowels using an autocorrelation method realized in a MATLAB 7.0 (MathWorks, Natick, MA) environment. The accuracy of F0 extraction was then manually checked (see also Xu et al., 2004; Han et al., 2007; Zhou and Xu, 2008). For each note, the mode value of the F0s was taken as the F0 for that note. The duration of each note was also measured. Based on the literature of objective assessment of singing (Nakata et al., 2006; Dalla Bella et al., 2007), we developed the following metrics to quantify the children’s singing performance using the music scores of the target songs as reference: (1) percent correct of F0 contour direction of the adjacent notes, (2) F0 compression ratio of the entire song, (3) mean deviation of the normalized F0 across the notes, (4) mean deviation of the pitch intervals, and (5) standard deviation of the note duration difference.
Children with cochlear implants produced a variety of songs with lengths ranging from 10 to 80 notes. The extracted pitch contours from six randomly chosen, normal-hearing children are plotted in Figure 1. The target scores are represented by black lines with filled symbols, and the sung notes from each child are represented by gray lines with open symbols. The pitch contours of the sung notes and the music scores were aligned with each other by subtracting the mean in semitones from the respective contours. This normalization minimized the differences between the key in which the child sang and key of the target song. The length of the notes is not represented in the figure.
Figure 2 plots the pitch contours produced by each of the seven children with cochlear implants using the same normalization procedure as in Fig. 1. Variability in pitch contour accuracy among the children with cochlear implants was evident. For example, subject CI 4 sang only 10 notes of a song and the direction of the notes appeared to be unrelated to the target scores. In contrast, subject CI 7 produced a rather complicated song and the pitch contour followed the target scores fairly closely.
Figure 3 summarizes the results of the four pitch-based metrics (Panels A through D) and one rhythm-based metric (Panel E) that we derived to assess the singing proficiency of all 14 normal-hearing children and the seven children with cochlear implants. Fig. 3A shows the accuracy of F0 contour direction of any two adjacent notes. This metric compared the direction of any adjacent notes produced by each child with that of the target intervals. Adjacent notes that were identical in pitch in the target scores were ignored. In other words, only adjacent notes that gave rise to ascending or descending intervals were counted. If the adjacent sung notes went to the same direction as the target scores, irrespective of the produced interval size, it was counted as correct. The median value of this measure for the children with cochlear implants was 52.3% correct (chance level = 50% correct), in comparison to 94.4% correct from the normal-hearing group. The differences between the two groups were statistically significant (Wilcoxon rank sum test, z = 3.25, p = 0.001).
We then evaluated whether the direction of the next note in the target scores (i.e., either ascending or descending) had any effect on the accuracy of the produced F0 contour direction. The children with cochlear implants produced the ascending intervals with a median accuracy of 50% correct, and produced the descending intervals with a median accuracy of 56% correct. The difference was not statistically significant as revealed by a repeated-measure t test (t = 1.79, p = 0.12). The accuracy of producing ascending intervals by the normal-hearing children (92% correct) was almost identical to that of producing the descending intervals (93% correct) (t = 0.01, p = 0.99). Thus, for both groups, pitch direction errors were made irrespective of the target intervals being ascending or descending.
Fig. 3B shows the compression ratio of the song. This metric measures the difference between the highest and lowest note produced by each child and divides it by the expected pitch range of the song. For example, if a subject sang a song with a range of 9 semitones and the target scores had a range of 12 semitones, the compression ratio was 0.75 (i.e., 9/12). Thus, the smaller the compression ratio, the greater the amount of pitch compression was present in the child’s singing. A compression ratio greater than 1 indicates expansion of the pitch range. The children with cochlear implants showed a median compression ratio of 0.47. In other words, these children sang within a pitch range that was less than half of the expected pitch range. When compared to the normal-hearing children, who had a median compression ratio of 0.76, the amount of pitch compression observed from the implanted children was not statistically significantly different from that observed in the normal-hearing controls (Wilcoxon rank sum test, z = 1.6, p = 0.1). This might be due to the small sample size (N = 7) in the present study.
Fig. 3C shows the mean deviation of the normalized F0 across notes. This metric was obtained by calculating the absolute differences between the two normalized pitch contours, note by note. For each subject, the absolute differences were averaged across all notes. The children with cochlear implants showed a significantly larger mean deviation of the normalized F0s (2.32 to 3.44 semitones, median = 2.32 semitones) than the normal-hearing children (0.84 to 2.72 semitones, median = 1.60 semitones) (Wilcoxon rank sum test, z =3.17, p = 0.002).
Fig. 3D shows the mean deviation of the pitch intervals. Pitch interval sizes were measured for the sung notes and the target scores. The mean of the absolute differences in the pitch interval sizes between the sung notes and the target scores was obtained. Measuring interval size deviation, rather than note deviation (Fig. 3C), may be advantageous because the former is independent of how the pitch contours were normalized. The children with cochlear implants showed a significantly larger value of this metric (2.19 to 3.98 semitones, median = 2.86 semitones) than the normal-hearing children (0.91 to 2.03 semitones, median = 1.51 semitones) (Wilcoxon rank sum test, z = 3.62, p = 0.0003).
The rhythm-based metric is the standard deviation of the note duration differences. The duration of a quarter-note was arbitrarily set as 500 ms long. First, the duration differences between the sung notes and the target notes were computed. For each subject, the standard deviation of duration differences for all the notes was calculated to reflect the variation of the rhythm deviation. Therefore, this metric was independent of the absolute speed of the singing. A particularly fast or slow singer could produce large absolute rhythm deviations from the arbitrarily set rhythm standard. As long as the singer keeps a consistent relationship with the target rhythm, the standard deviation of the measured duration differences should still be small. Fig. 3E shows the results of this rhythm-based measure. The two groups did not differ significantly on this measure (Wilcoxon rank sum test, z = 0.783, p = 0.43).
A correlational analysis using data from the cochlear implant group was performed to examine whether any pairs of measures of singing accuracy were correlated with each other. Contour direction was positively correlated with the compression ratio (r = 0.90, z test, p < 0.01) and negatively correlated with the mean interval deviation (r = −0.88, z test, p < 0.01). All other pairs of measures were not significantly correlated (p > 0.05). None of the five measures were found to be significantly correlated with the chronological age, age at implantation, or duration of cochlear implant use of the children (all p > 0.05). In addition, the measures of singing accuracy in the normal-hearing children were not significantly correlated with their age (all p > 0.05).
Results from the present study indicate that some prelingually-deafened children can develop certain forms of vocal singing using current cochlear implant technology. This development of the ability to sing is not trivial given the difficulties the implanted children may encounter with limited pitch information encoded in the current multichannel cochlear implant systems [see Moore (2003) for a review]. We did find that the children with cochlear implants showed significantly poorer performance than the normal-hearing children on almost all pitch-based assessments of singing. The rhythm-based measure, however, revealed no differences between the two groups. In a previous report, Nakata et al. (2006) showed that children with cochlear implants had a tendency to perform poorly with respect to pitch accuracy. The contour direction of the children with cochlear implants was at chance level, consistent with our findings (Fig. 3A). Their normal-hearing children produced near-perfect accuracy in contour direction (96% correct), again similar to the results of our normal-hearing children (94.4% correct). Note that their normal-hearing children were of older ages. The mean age of their normal-hearing group was 8.18 years old, 2.3 years older than the mean age of our normal-hearing group. Moreover, five of the six normal-hearing children were taking music lessons outside of school whereas none of our normal-hearing children had extracurricular musical training. Thus, our findings were consistent with the notion that basic singing abilities emerge spontaneously and precociously during development in normal-hearing children (see Dalla Bella et al., 2007).
Nakata et al. (2006) and the present study also showed similar results on the measure of mean deviation of pitch intervals. They reported an average of 2.43 semitones of deviation in pitch intervals produced by the children with cochlear implants. The median of the mean deviation of pitch intervals was 2.86 in the present study (Fig. 3D). It appears that this singing deficit is a consequence of relatively poor pitch discrimination ability in the implant users.
In a pitch discrimination test, Nimmons et al. (2008) found that the majority of their adult patients with cochlear implants had a difference limen of 2 to 4 semitones in a pitch direction discrimination test. Other studies suggested that the difference limen for pitch direction discrimination in cochlear implant users was much higher. Fujita and Ito (1999) found that adult patients with cochlear implants needed 4 semitones to 2 octaves to judge which of the two consecutive piano notes was higher in pitch. Similarly, Gfeller et al. (2002) reported a mean threshold of 7.56 semitones (range of 1–24 semitones) in a complex-tone pitch discrimination test in adult implant users.
A direct assessment of pitch discrimination in young children with cochlear implants, albeit difficult, would provide valuable information on how they perceive pitch differences in melody. One study reported that the difference limen of pitch discrimination in a same-or-different paradigm (as opposed to the pitch direction discrimination task in the reports cited above) was <1 semitone in a group of 10 older children with cochlear implants (8 – 19 years of age) (Vongpaisal et al., 2006). It should also be noted that the pitch discrimination paradigm might have revealed a difference limen of percepts other than pitch, such as tone quality. In any case, we caution the direct comparison of pitch perception ability with the ability to sing. It is worth noting that the amount of deviation of pitch interval in singing is likely to be constrained by the physiology of the articulator. Our normal-hearing children showed a median of 1.51 semitones in deviation of pitch intervals (Fig. 3D), which apparently is not related to their perceptual capabilities. Stalinski et al. (2008) recently demonstrated that the difference limen for the 5- and 6-year-old normal-hearing children to identify the direction of pitch changes is as small as 0.3 semitone. In cochlear implant users, particularly in prelingually-deafened pediatric users, the perception of differences in two notes or a sequence of notes might not be readily translated into differences in pitch height.
Poor pitch perception has also been implicated in poor lexical tone production (Xu et al., 2004; Peng et al., 2004; Han et al., 2007, 2009; Zhou and Xu, 2008). We found that the rising tones were particularly difficult to produce for the prelingually-deafened, native Mandarin-Chinese-speaking children with cochlear implants. This was consistent with the findings of poor production of rising speech intonation in native English-speaking children with cochlear implants (Peng et al., 2007). To increase the voice F0, the vocal fold tension, or stiffness, has to be effortfully modulated by the contractile forces of the vocal fold musculature. On the other hand, to decrease the voice F0, lowering the subglottal pressure is sufficient (Lieberman and Blumstein, 1988). Contrary to our hypothesis that is based on our observations with lexical tone production, we found no differences in the accuracy of producing ascending versus descending intervals in the children with cochlear implants. Therefore, the production process for singing and lexical tones might likely involve different mechanisms. We have studied tone production of isolated words only. It will be informative to study tone production in connected speech by children with cochlear implants and then to compare those results with the data of vocal singing.
The rhythm-based measure showed similar results between the implanted children and the normal-hearing children groups (Fig. 3E). Nakata et al. (2006) also reported that relative timing in singing was similar in children with cochlear implants and normal-hearing children. There were some differences in the way that the rhythm accuracy was quantified in the two studies, though. Nakata et al. (2006) first normalized the duration measures individually to each child's tempo and then calculated the mean of absolute rhythm deviation. In this study, we used the standard deviation of note duration difference as the measure of rhythm accuracy. Thus, it is not sensitive to the absolute speed of singing, ridding the need to normalize the duration measures to each singer’s tempo. Nakata et al. (2006) acknowledged that the timing deviations were larger, even in their musically-trained, normal-hearing children, than one would expect from older children or adults. It is known that occasional singers in the general population tend to sing at a faster rhythm relative to professional singers (Dalla Bella et al., 2007). Current cochlear implant systems can faithfully deliver rhythmic information. Patients with cochlear implants have been reported to perform similarly in rhythmic perception tasks (e.g., Gfeller et al., 1997; Kong et al., 2004; see McDermott, 2004 for a review). It seems that, as a result of the preserved rhythmic information, the rhythmic aspect of singing in implanted children was not significantly affected.
Our data also showed large individual differences in all five metrics in both normal-hearing children and children with cochlear implants (Fig. 3). We do not know what factors might have contributed to this variability. Large variability appears to persist in the normal-hearing, general population (Dalla Bella et al., 2007). The overall performance of the children with cochlear implants in the pitch-based metrics was poorer than that of the normal-hearing children. Performance of a small number of children with cochlear implants, however, did fall within the normal range. It is thus important to consider what factors may have contributed to the individual differences in the children with cochlear implants. Age of implantation was found to be important for lexical tone production in prelingually-deafened children with cochlear implants (Han et al., 2007). The group in Nakata et al. (2006) had an older median age at implantation, and the authors emphasized the importance of the long-term memory of music their subjects had for their ability to sing. Although our subjects lacked prior experiences of singing, the median age at implantation in the present study was younger [3.72 years of age (Table 1)]. Whether the relative younger age at implantation might have contributed to the development of vocal singing in our group of prelingually-deafened children still remains to be tested. Other factors that one might consider, include chronological age, duration of implant use, nonverbal intelligence, gender, implant characteristics, and educational settings. Our data revealed no significant correlation between the chronological age, age at implantation, or duration of implant use to any of the singing measures. The sample size of the present study is limited, which prevents us from determining which factor or factors may be the major contributors. An ongoing, large-scale study by our research team is aiming at elucidating the contributing factors for singing proficiency in children with cochlear implants, as well as determining the relationship between vocal singing and lexical tone production. Such a study will have significant implications for aural rehabilitation in children with cochlear implants.
The authors express appreciation to Natalie Bevilacqua, Jiong Hu, and Cyndi Welch for their technique assistance. Three anonymous reviewers provided useful comments to an early version of the manuscript. This work was supported by NIH NIDCD Grant Nos. R03 DC006161 and R15 DC009504 and Advanced Bionics Corporation.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.