We conducted 39 cognitive interviews between April and October 2007. shows the characteristics of the participants. The study population reflected a range of cancer types and stages. The greater number of women in the study was a result of the repeated cognitive interviews that were needed to clarify items in one of the women-specific item categories. In the sections that follow, we review our findings about the relevance of the items, recall period, wording changes to improve sensitivity, appropriateness, and clarity, and item ordering. We also discuss the assessment of sexual functioning without reference to specific activities and the challenges of addressing and measuring sexual functioning in the context of sexual experiences that include the use of sexual aids.
Relevance of items
Although we attempted to create items that could be answered by anyone—both men and women, regardless of sexual orientation or marital/partnered status—many participants expressed that certain types of questions did not apply to them. For example, many men said that items about breast tenderness or enlargement and hot flashes applied only to women, though these can be side effects of cancer and its treatments for both men and women. Because these questions seemed to confuse some male participants, we elected to ask the questions of women only. Similarly, some participants assumed that items about rectal pain applied only to persons who engaged in anal sex, though patients with colorectal cancer may experience rectal discomfort during other sexual activities. These items made many participants uncomfortable, so for the purposes of later PROMIS item testing, we asked the questions about rectal pain only of participants who reported having engaged in anal sex during the previous 30 days. These findings highlight the need for PROMIS to provide explicit guidance about which subsets of items should be used in particular populations when the final measure is released.
We assessed the recall period in one of two ways. First, when we asked participants to describe how they arrived at their answers, we recorded whether they made reference to a time frame and, if so, what time frame the participant said they considered. Second, for some items we asked specifically about what time frame the participants used when recalling their answers.
Most participants said that, when they were answering the questions, they were thinking of the past 30 days, the past 4 weeks, or the past month. Three participants reported they were considering the time since the beginning or end of their treatment (ie, a longer time frame than 1 month). In responding to the items, some participants wanted to rate even sexual experiences that had occurred before the 1-month recall period. For example, 2 participants stated they had answered the question in reference to their last sexual encounter, which had occurred months earlier. Three participants suggested that we consider extending the time period for rating the items. For example, 1 participant said, “Some patients have treatments that last 30 days and they come in 2 or 3 times a week. They may be so tired or sick they are unable to have sex…. [Thirty] days might be too short, because they might not have had a sex life in the past 30 days because of their treatment.” These responses suggest that some patients believe the priority in asking these questions should be on obtaining a personal judgment about an important life experience, rather than ensuring that the judgment is restricted to a particular recall period.
Self-report data from cognitive interviews is helpful in identifying cases in which participants said they constructed their answers in a way that was inconsistent with the intended time frame. However, even if participants said they used a 1-month recall period, the cognitive interview data cannot inform the accuracy of the participant's recall for a particular event. For this type of evaluation, a comparison with daily diary data would be more informative [19
]. The PROMIS Sexual Function Domain Committee is currently conducting this study. Meanwhile, we will continue to use a 30-day reference period for the items.
Wording changes to improve sensitivity, appropriateness, and clarity
Participants identified problems with the wording of some items that prompted important revisions. It is instructive that participants identified these problems after many rounds of review by investigators and survey methodology specialists. We provide several examples below. provides an example of how we evaluated the results of the interviews and modified the items based on participant feedback.
Example of Item Testing and Modification Using Participant Feedback
Implied judgments in item wording
One of the original items read, “When having sex with a partner, how often have you needed fantasies to help you stay interested?” Two participants thought the word “needed” implied a negative judgment toward the use of sexual fantasies. We reworded the item to read, “When having sex with a partner, how often have you used fantasies to help you stay interested?” During retesting, no participant commented that the revised item implied a negative judgment.
Awkward descriptors of sexual experience
In an effort to develop a single item that assessed vaginal discomfort during sexual activity, we asked, “How would you rate the overall comfort of your vagina during sex?” Some participants had difficulty with the term overall comfort. One participant said, “I think of my sofa as being comfortable, not really my vagina during sex.” One participant said that the response options excellent to poor were “like a report card for your vagina.” We changed the response options to range from “very comfortable” to “very uncomfortable.” The item now reads, “How would you describe the comfort of your vagina during sexual activity?” When we retested the items, participants did not identify the wording as awkward or inappropriate.
Participants with low literacy (n = 2) had difficulty with the term distracting thoughts in the item “How often have you lost your arousal (been turned off) because of distracting thoughts?” We retested the item as “How often have you lost interest once a sexual activity begins?” with 7 participants, and all participants who had low literacy (n = 3) said they understood and preferred the revised item.
Problems with item ordering
Cognitive interviews also identified confusion created by the order of some items. For example, we originally presented the item “Are you married or in a relationship that could involve sexual activity?” before the item “Over the past 30 days, have you had any type of sexual activity with another person?” Five out of 20 participants thought that the question was asking about sexual activity with someone other than their partner. In a subsequent round, we modified the second question to read, “Over the past 30 days, have you had any type of sexual activity with another person, including your partner?” and reversed the order of the 2 items. After making this change, the proportion of participants who had difficulty with the item was halved. We plan to evaluate the consistency of this item among other questions about sexual activity using data from a large sample from item testing.
Assessing sexual function and satisfaction without reference to specific activities
In an ideal measure of sexual functioning, any person who could claim to have a sexual component to his or her life would be eligible to receive a score on as many subdomains as possible. However, many existing instruments measure function and satisfaction with items that place the assessment in the context of specific sexual activities, which were often specific to either men or women. To ensure that our measure was broadly applicable, we first created a subdomain that asked about the frequency of performing various activities (eg, holding hands, kissing, sexual intercourse). Then, for the remaining items in the measure, we attempted to remove all references to specific activities. To accomplish the second goal, we needed to find a generic term to broadly describe sexual experience. We used verbal probes with the cognitive interview participants to identify inclusive ways to describe sexual intimacy. Participants identified sex life as the most encompassing term. For most of the participants, sex life included a range of sexual activities, including affectionate behavior, foreplay, and masturbation. Some participants mentioned that the term also included the emotions they experience in addition to physical activities. The term sexual activity was thought to be the second broadest term and, like sex life, was associated with a range of sexual activities, including masturbation, with less emphasis on the emotions experienced. Participants identified having sex as the most exclusive term, pertaining solely to sexual intercourse. Participants noted that the broad terms sex life and sexual activity made the items applicable to both men and women.
On a related note, participants repeatedly reported a preference for items that referred to emotional intimacy. Many participants liked the item “Over the past 30 days, how satisfied have you been with your ability to share warmth and intimacy with another person?” because it captured elements of an intimate relationship like eye contact, feeling close to someone, snuggling, and holding hands. All of the participants noted that the item “Over the past 30 days, how often have you and another person spent time holding or hugging each other romantically?” was an effective way to inquire about nonsexual aspects of relationships and intimacy. These results were consistent with findings from focus groups, in which participants identified emotional intimacy and affectionate behavior as a key part of their sex life [13
]. These reactions also suggest that the inclusion of items on intimacy and affectionate behavior make important contributions to the face validity of the measure from the respondent's perspective.
Adjusted and unadjusted functioning: the case of sexual aids
Focus groups with men and women revealed the importance of sexual aids (eg, lubricants for women and erectile dysfunction drugs for men) to the sexual functioning of cancer survivors [14
]. In developing the measure, we were interested in evaluating participants' sexual functioning as it occurred in their everyday lives. For example, if participants typically used sexual aids during sexual intercourse, we wanted them to respond to the items considering the times they were using these aids. Such an assessment is a measure of “adjusted functioning.” Several instruments have been developed to assess sexual functioning with therapeutic aids in men (eg, International Index of Erectile Function [20
], University of California, Los Angeles, Prostate Cancer Index [21
]). However, there are no validated instruments that address adjusted functioning in women.
We tested the erectile function and vaginal function items with instructions about sexual aids in a variety of formats. The instructions asked participants who regularly used therapeutic aids to describe their sexual experiences when using the aids. In the first round of interviews, we included the instructions before the set of items (). In the second round of interviews, we tested the items with a parenthetical phrase after each relevant item (). In both rounds of interviews, we found that, when answering items about erectile function, men consistently considered the use of therapeutic aids if they normally used them. However, when answering items about vaginal dryness and pain, women had more difficulty deciding whether to consider the times when they had used a personal lubricant during sexual activity. For example, some women thought they were to answer the question while considering the times when they used a lubricant, even if they had not used a lubricant in the past 30 days.
Instructions for Responding to Items Regarding the Use of Therapeutic Aids During Sexual Activity: Placement Before the Item Set
Instructions for Responding to Items Regarding the Use of Therapeutic Aids During Sexual Activity: Placement After Each Item
Since our goal was to achieve uniform interpretation of the items, we conducted a third round of cognitive interviews with women only, screening potential participants depending on how frequently they used vaginal lubricants. We classified 9 participants into 1 of 3 groups: “always” (n = 4), “sometimes”(n = 3), and “never”(n = 4). During the cognitive interview, the interviewer showed the participant the vaginal function items without reference to lubricants. After the participant had seen all of the items, the interviewer asked whether the participant had considered their use of lubricants during sexual activity when answering each question.
Although the participants had no difficulty understanding the questions, we found that women in the “sometimes” group differed from women in the “always” group with regard to whether they considered their use of lubricants during sexual activity when they responded to the questions. Furthermore, even within these groups, the participants differed in how they responded to the items. Among women who sometimes used lubricants, there were inconsistencies for two thirds of the 12 items, compared to inconsistencies for one third of the items among women who always used lubricants during sexual activity. In addition, participants were not consistent about whether they considered their use of sexual aids across different items. Some women indicated that they did not think about their use of lubricants when responding to items about arousal or foreplay, but they considered their use of lubricants when responding to items about sexual intercourse.
At the end of these interviews, we asked for feedback comparing the instructions from the first round of interviews () and the parenthetical phrases used in the second round (). Participants did not consistently favor either approach, and only 1 participant said the instructions would have been helpful in answering the questions.
To our knowledge, no data have been published regarding how women interpret and respond to these types of questions. Thus, we have identified an important challenge that will likely be faced by any measure purporting to assess sexual functioning in women. The PROMIS team is initiating a diary study to examine sexual functioning and the use of personal lubricants on an event-by-event basis to determine (1) how women's general responses to the measure correspond to detailed reports of specific sexual activities and (2) whether a daily diary approach would be superior to a 30-day recall approach in measuring adjusted functioning. In the meantime, item testing in a large sample of patients with cancer will use the items without special instructions, similar to all other existing measures of sexual functioning in women.
Limitations of cognitive interviews for measure development
We have discussed many of the strengths and contributions of the cognitive interviews; however, there are some limitations inherent in the cognitive interview process. First, as with almost all empirical studies, the participants might not be representative of all potential respondents in terms of how they processed and responded to the items under study. Although there is no way to eliminate this limitation, its impact can be reduced by recruiting a sample that reflects variability in education and cultural background. Future research on measure development in sexual functioning and intimacy should consider qualitative feedback from additional populations who may have been underrepresented in this work, such as adolescents and patients and survivors who identify as homosexual or bisexual. Second, the results of the interviews highlight a problem but do not always lead directly to a solution, as with the items on vaginal function and the use of personal lubricants. In this case, further research is required to find the best way to properly address the issue.