|Home | About | Journals | Submit | Contact Us | Français|
Multilevel intervention research holds the promise of more accurately representing real-life situations and, thus, with proper research design and measurement approaches, facilitating effective and efficient resolution of health-care system challenges. However, taking a multilevel approach to cancer care interventions creates both measurement challenges and opportunities.
One-thousand seventy two cancer care articles from 2005 to 2010 were reviewed to examine the state of measurement in the multilevel intervention cancer care literature. Ultimately, 234 multilevel articles, 40 involving cancer care interventions, were identified. Additionally, literature from health services, social psychology, and organizational behavior was reviewed to identify measures that might be useful in multilevel intervention research.
The vast majority of measures used in multilevel cancer intervention studies were individual level measures. Group-, organization-, and community-level measures were rarely used. Discussion of the independence, validity, and reliability of measures was scant.
Measurement issues may be especially complex when conducting multilevel intervention research. Measurement considerations that are associated with multilevel intervention research include those related to independence, reliability, validity, sample size, and power. Furthermore, multilevel intervention research requires identification of key constructs and measures by level and consideration of interactions within and across levels. Thus, multilevel intervention research benefits from thoughtful theory-driven planning and design, an interdisciplinary approach, and mixed methods measurement and analysis.
Multilevel intervention research holds the promise of more accurately representing real-life situations than single-level intervention research and, thus, with the proper research design and measures, being more likely to facilitate the effective and efficient resolution of complex health-care systems problems (1–3). Taking a multilevel approach to cancer care interventions, however, creates measurement challenges and opportunities. Some measurement issues are similar to those arising in single-level interventions, whereas some are unique to multilevel interventions (4,5). The purpose of this article is to examine the state of measurement in multilevel cancer intervention research and to suggest how opportunities in the field may be addressed by drawing on the measurement literature in other fields.
One of the unique aspects of measurement in multilevel intervention research is that consideration of within and between level effects is needed. For multilevel interventions, it is not sufficient to measure effects at the different levels of intervention (eg, patient and health-care team), cross-level effects must also be taken into consideration (eg, the effect of patients on the team and the team on the patients). Furthermore, some measurement issues may be more complex when conducting multilevel intervention research, and some measurement rules of thumb may be uniquely associated with specific levels of analysis (eg, group or organization levels of analysis) (6,7). Because multilevel interventions are based on systems theory [whether general systems (8–10), social ecological systems (11,12), or complex adaptive systems (13) models], they require identification of key constructs and measures by level and consideration of interactions within and across levels (6). Thus, multilevel intervention research benefits from thoughtful planning and design, an interdisciplinary approach, mixed methods (eg, a combination of objective factors, surveys, interviews, observations) measurement and analysis, and careful attention to the congruence of theory, constructs, and measures.
Through a systematic review of the literature, this article examines the state of measurement in multilevel cancer care intervention research, identifies measurement opportunities in the field, and draws upon other fields to provide direction about measures and measurement to multilevel cancer care intervention researchers.
We conducted a systematic literature review focused on cancer care intervention research and measures published in the past 5 years. This contrasts with Stange et al. in this monograph (14), which focuses on the broad spectrum of health care and prevention research over a 10-year period (2000–2010). Thus, our review provides a comprehensive overview of a more recent set of cancer-specific intervention studies.
Multiple bibliographic databases were searched to identify articles related to cancer care intervention research and measurement. Almost 1800 articles were identified, of which 1072 were sufficiently related to the topic to warrant further review. Ultimately, 234 multilevel articles were identified, 40 of which involved cancer care interventions. A level was defined by the degree or type of social aggregation associated with an intervention target. For example, a single-level study targets a single type of social aggregation, for example, only individuals or only groups. Approaches to measurement and measures in these 40 studies were systematically analyzed. Supplementary Appendices A and B (available online) provide details about the search methodology and the article review and classification process.
Of 1072 articles reviewed, 78% (838 articles) concerned single-level studies. The distribution of studies across the cancer care continuum differed by level. Most of the single-level studies focused on treatment or survivorship or both, whereas the multilevel studies addressed mostly detection/screening, treatment, or survivorship.
Of the 234 multilevel articles, 40 (17%) involved interventions (randomized controlled trials or empirically evaluated nonrandomized interventions). These 40 articles were further classified by number and level of intervention targets and units of analysis. Supplementary Appendix C (available online) provides summaries of each article, including a description of the interventions, intervention targets, and measures. All of the 40 articles had multiple intervention targets and/or multiple measurement targets. However, for the majority of the studies, these targets were at a single level. The most common interventions (13 studies, 33%) were interventions focused on the individual patient and/or physician and involving multiple intervention targets at a single level and multiple measurements at a single level. Interventions targeting groups, organizations, and communities were underrepresented.
The 40 multilevel intervention articles represented 37 unique research studies. For each of the 37 studies, related articles were examined to identify the best source of information regarding each study’s measures and measurement approaches. This section summarizes findings regarding the types of measures used, their properties, and measurement approaches in the 37 studies.
The vast majority of measures used in the 37 studies were individual-level measures. Group-, organization-, and community-level measures were minimally present. Supplementary Appendix D (available online) provides a summary of measures and their properties by study. Objective measures of individual outcomes, such as completing a screening test (14–23,24), and measures of individual perceptions, knowledge, attitudes, or behavioral intentions (19,25–33) were the most common types of measures. Of the 37 studies, almost half (43%, 16 studies) relied upon reviews of patient charts or medical records to collect key outcome data, such as screening completion.
Two studies used group-level measures; one used a group consensus process in palliative care delivery teams to assess group satisfaction with elements of the program (34). Another used quality circle discussions to identify barriers to screening (34). Two studies aggregated data to conduct analyses at the community level (32,36). Two studies analyzed care processes as part of their investigations (37,38).
The main outcome measures for randomized controlled trials were notably different from measures for other types of empirical intervention studies. All of the randomized controlled trials used an individual-level objective measure (eg, patient-completed screening test) as the primary outcome measure, whereas only one-quarter of the other empirical intervention studies did. These types of studies were more likely to use changes in knowledge, perceptions, or intentions or changes in prescribing rates or abnormality detection as their primary outcome measure.
Of the 37 studies, 32% (12 studies) used some type of established measurement scale; 14% (five studies) mentioned creating a measurement scale. Although 29 different established measurement scales were used in the 12 studies, the most frequently used scales were health-related quality-of-life instruments, such as the European Organization for Research and Treatment of Cancer or Functional Assessment of Cancer Therapy (35,38–41); depression instruments, such as the Hospital Anxiety and Depression Scale (38,40,41) or Center for Epidemiological Studies Depression Scale (37,40,42); and fatigue scales, such as Brief Fatigue Inventory or Multidimensional Fatigue Inventory (21,28). Supplementary Appendix E (available online) has a complete list of these measures. However, many studies simply referred to pre- or post-questionnaires, and the general topics respondents were asked about, providing no details about the measures or instruments used (16,19,26,31,32,43–47). Others noted that they developed measures or adapted measures from another instrument but provided no details regarding the specifics of the adapted measures.
Discussion of measurement validity and reliability was scant in the 37 studies. Only 14% (five studies) discussed validity at all, 11% (four studies) made specific reference to the reliability of at least one measure, and another 11% (four studies) did not provide an assessment of measure reliability but noted that such information was available elsewhere.
In three of the 37 studies, a measure of an individual-level phenomenon was analyzed at another level (25,32,36). For example, in the study by Ganz et al. (24), individual patient screening data were aggregated to assess the organizational rate of cancer screening.
Furthermore, measures were used both as interventions and as outcomes (38,39,48,49). For example, in the study by Hilarius et al. (39), patients completed a health-related quality-of-life survey by computer, which generated a graphic summary for the patient and the nurse (the intervention); health-related quality-of-life data also were used to assess outcomes related to the intervention. In other examples (48,49), performance metric reports (eg, detection rate of abnormalities, the number of surgical procedures to standard) were used as part of the interventions (ie, feedback to people targeted) and also as intervention outcome measures. This approach was used in interventions designed to achieve quality improvements, process improvements, or organizational changes.
Two other approaches to measurement were assessments of process (37,38) and economic/cost–benefit analyses (17,21,34,37,39). For example, Tamminga et al. (37) asked patients and nurses to periodically evaluate the care process, and Velikova et al. (38) analyzed charts to assess the process of care. Shankaran et al. (17) calculated an incremental cost-effectiveness ratio.
Studies that focused on quality improvements tended to have multiple intervention targets, multiple approaches to measurement, and multiple measures. Furthermore, they often used a model or checklist to guide design and evaluation of the intervention (eg, Deming's continuous quality improvement practices, Put Prevention Into Practice [PPIP], force field analysis, PRECEDE, Agency for Healthcare Research and Quality [AHRQ] checklist for improving delivery of preventive services). Ten (27%) of the studies fell into this category (18,19,21,22,24–27,35,37,48). For example, Ganz et al. (24) delivered a quality improvement intervention to organizations over 2 years. Interventions targeted patients, physicians, provider organizations, staff, and medical directors using an iterative change management process and multiple approaches to data collection (chart reviews, patient interviews, staff meetings, etc.). Fitzgibbon et al. (18,24) used a similar continuous quality improvement approach. Wei et al. (22) and Ling et al. (19) used the PPIP model to design process improvement interventions with multiple targets (eg, patient, physician, office manager) and multiple measures and measurement approaches (eg, medical records, patient surveys, organizational informants).
A number of themes emerged in this review of measures in multilevel intervention studies in cancer care. First, the majority of “multilevel” intervention studies are actually multitarget interventions that focus on a single level—the individual level—for both interventions and measurement. The patient–physician dyad is the primary intervention and measurement target. This occurs despite the fact that relatively few processes across the cancer care continuum involve interaction with only a physician. The most overlooked, unstudied levels from both intervention and measurement perspectives are the group, organization, and community levels.
Second, many studies lack an explicit consideration of the levels involved in an intervention, leading to failure to consider the measurement implications of the levels of the intervention. A number of studies intervened with groups or organizations. However, they did not identify groups or organization as an intervention target nor did they measure any group or organization-level phenomena (18,19,24,29,30,32,40,49,50); they merely measured patient outcomes, sometimes aggregated to the group or organization level.
Third, studies that did attempt to intervene at multiple levels in addition to the patient, more often used a theory, conceptual model, or checklist to guide their intervention. The most frequently cited conceptual approach to multilevel intervention design was quality improvement (18,24,25,35,48). The most common models or checklists used for multilevel intervention design were PPIP (19,22), PRECEDE (51), the AHRQ checklist of practice items to improve delivery of care (26); strengths, weaknesses, opportunities, and threats analysis (26); and force field analysis (35).
Fourth, scant attention was paid to measurement issues such as independence, reliability, or validity. This may not be of concern when studies use single-item objective measures (such as screening completion ascertained through review of a medical record) or measurement instruments with well-known well-established psychometric properties; however, when multi-item scales are created for a study (see examples noted above), these are very significant concerns.
This review indicates a need to think about measurement and the measure selection process in a comprehensive and thoughtful manner when conducting multilevel intervention research. The nature of the interventions, the intervention targets, and the levels of the targets must be explicitly considered in research design and when developing the measurement and analysis plan. Standard measurement issues such as the independence, reliability, and validity of measures, sample size, and power require special attention in multilevel intervention research. Key constructs and measures should be defined by level, so that within-level and cross-level interactions may be assessed while careful attention can be paid to the congruence of theory, constructs, and measures. Interdisciplinary and mixed methods research approaches may help identify expected and unexpected effects and interactions of multilevel interventions.
This section discusses some of the challenges and opportunities associated with designing and implementing multilevel interventions highlighted in the literature review, and best practices and/or exemplar approaches for addressing them. Specific techniques that have evolved to deal with multilevel research issues, such as the lack of independence, need for congruence, aggregation, reliability, validity, and research design, are discussed.
In single-level intervention research, researchers need to confirm the independence of measures, assess the reliability and validity of measures, and address sample size and power requirements. Multilevel intervention research also must address these issues and may require a different or more complex approach than single-level research. For example, in a single-level study, the independence of measures is typically established using statistical tests to justify the use of standard analytical procedures. However, a multilevel study may have to accommodate the expected lack of independence among measures through the use of special analytical or aggregation procedures (eg, hierarchical linear modeling, structural equation modeling, or aggregating respondent data to the group or organization level).
A key concept in multilevel measurement is that theories, constructs, and measures must be congruent. The introduction of interventions at multiple levels requires that the level of theory, level of measurement, and level/type of statistical analysis be consistent (6,7). For example, collecting a measure at an organization or group level and claiming that it represents an individual-level concept would violate this principle of congruency and lack internal consistency and intellectual rigor. Measurements are conceptually distinguished by the level at which they are collected and analyzed (52). For example, a physician education intervention may be conceptualized and measured in a number of ways. Conceptualized as an individual-level intervention, one might ask a series of questions about individual knowledge, opinions, or actions. Conceptualized as a group-level intervention, one might ask a series of questions about group knowledge, group opinions, or group actions. Conceptualized as a multilevel intervention, one might ask a series of questions about both individual and group knowledge, opinions, or actions. To justify the aggregation of individual responses to the group level and make cross-group comparisons, one would have to demonstrate adequate agreement among group members and adequate between-group variance. Typically, this is accomplished by calculating rwg and Intraclass Correlation Coefficients (ICC) (6,7). The individual-level data also might be aggregated to the organization-level to assess organizational impact, typically without requiring any special justification. If the purpose was to compare aggregated results across organizations, within-organization agreement and across-organization variance generally needs to be established. To understand cross-level interactions, more sophisticated statistical modeling techniques are required.
The rationale for aggregating variables is typically conceptualized using one of three models—the consensus, additive, and dispersion models—and statistically justified with measures of reliability, agreement, and predictive validity (53,54). Consensus models conceptualize aggregate means of individual ratings as indications of a common construct perceived by the group, organization, or community. For example, patients may rate reactions to an intervention, and consensus among respondents supports the hypothesis that ratings represent a shared phenomenon. Ratings may be conceptualized with an individual-, group-, organization-, or community-level referent (eg, I am more likely to get the test, My team is more likely to use the procedure, My organization is supportive of the test) (53). In the example of a physician education intervention, responses to questions about individual beliefs vs responses to questions about group beliefs are examples of individual-level and group-level referents, respectively. When using a consensus model, it is important to conceptualize and frame the measure at the appropriate level and test whether empirically there is agreement among respondents. This is typically accomplished by computing the rwg statistic (54). If individual-level constructs and measures are used, aggregation is not theoretically justified, even if it may be statistically supported.
Additive models do not require agreement statistics, but aggregation should be theoretically well grounded and statistically justified. For example, it may not be necessary for all members of an organization to agree that coworkers cooperate, but statistically there needs to be more variation between organizations than within organizations to have the power to detect relationships across organizations. ICCs are used to provide this statistical evidence (54).
Finally, dispersion models conceptualize the variation among ratings as an important variable in understanding group-, organization-, or community-level constructs. For example, the variation in perceived openness to change within an organization may be an important consideration when assessing intervention effectiveness.
Just as in single-level studies, multilevel studies must also be concerned with common method variance. Because many measurement approaches use surveys to collect data from individuals, researchers must be careful regarding common methods variance. This problem occurs when both independent and dependent variables are collected with the same instrument. For example, a survey may ask about a group process, such as coordination, and an outcome, such as satisfaction. The correlation between these measures often is inflated because it is affected by the tendency of individual respondents to score responses similarly no matter the content of the questions. The strongest studies use different data collection methods to collect data on the independent and dependent variables (eg, a survey of individual team members to collect independent variable data and interviews with team managers to collect dependent variable data).
For measurements requiring aggregation of individual responses, particular attention should be paid to sample size and power issues. A sample size that might seem quite adequate for an individual-level analysis (eg, 250 respondents) may be insufficient for a group- or organization-level analysis (eg, 250 respondents may account for only 20 groups or one organization). This also has implications for the number of measures that may be included in a study (eg, in the previous example, if the group is the level of analysis, the appropriate number of measures will be much lower than if the individual is the level of analysis). Generally, the higher the level of analysis, the larger the number of individual respondents needed to achieve an equivalent power and the more parsimonious the inclusion of measures in the study should be.
Compared with single-level intervention studies, multilevel intervention studies also may require a more complex approach to establishing measurement reliability. Reliability may have to be assessed by measure by level. See Wageman et al. (55) for a discussion of this and examples. Validity also may be approached somewhat differently in multilevel intervention studies; external validity may be emphasized over internal validity. See Shapiro et al. (40) for an example where the focus was on external validity in a study of intervention services as routinely delivered within a community.
Because multilevel interventions are based on systems theory (6), they require conceptualization of key constructs and measures by level and consideration of within- and cross-level effects. Thus, multilevel intervention research benefits from thoughtful theory-based planning and design, interdisciplinary approaches, and mixed methods measurement.
One of the key challenges in designing multilevel interventions and the subsequent measurement plan is deciding on the intervention—what levels should be targeted for intervention and what factors should be measured at the various levels to understand the effect of the chosen intervention(s). The use of models, frameworks, and/or theories may help guide the planning and design of multilevel interventions and ensure that potential influences on an outcome are systematically identified and assessed.
The implementation science literature has examples of multilevel models and associated measures that may be useful when designing multilevel interventions. One model is the organization transformation model (56), which identifies five drivers of change to improve quality of care. Central to this model are improvement initiatives conducted by teams (group-level). Other constructs in the model that span levels are leadership from the executive level to frontline supervisors; alignment of strategy, rewards, evaluation, resource allocation, and other systems; and integrating mechanisms across departments and projects. The model suggests that an impetus, from either an internal source (organization-level) or outside source (community-level) is needed to motivate change. The drivers of change operate on an organization's culture, mission, strategy, and work processes (organization-level), and on individuals in its workforce to develop needed skills and to engage in the change effort. Thus, the organization transformation model addresses interventions and measures at four levels (community, organization, group, and individual). Measures of the constructs in the model are obtained through quantitative and qualitative methods.
The Promoting Action on Research Implementation in Health Services (57) model, another implementation model, addresses three concepts at two levels: quality of the evidence being implemented, organizational readiness for change, and facilitation, which includes group process. These concepts can be measured with the Organizational Readiness to Change Assessment instrument (58) administered to organization members. Many investigators also assess individual attitudes about the particular practice being implemented and evidence-based practice in general, as well as provider demographic characteristics.
The complex effects of multilevel interventions are probably best understood using interdisciplinary and mixed methods approaches. Interdisciplinary approaches can capture knowledge and insights from different fields. When that knowledge is effectively integrated and focused on solving problems, it may yield significant innovation. Designs and measures that assess the full complexity of factors operating within and across levels are more likely to effectively advance our understanding. Thus, design and measurement approaches that facilitate data collection in complex environments, such as sophisticated statistical designs (1–3), quasi-experimental designs, and mixed methods approaches that include both quantitative and qualitative data, are likely to be useful in multilevel intervention research.
Based upon the literature review conducted for this chapter, the least developed domains in multilevel cancer care interventions are the group, organization, and community levels. To address this issue and provide readers with a starting point for further exploration, this section draws upon literature from the health services, social psychology, and organizational behavior fields to present commonly used measures at the group, organization, and community levels and discuss how measurement is approached at each of these levels, highlighting differences across levels. Examples of key constructs and measures by level are summarized in Tables 1–3. These tables are not an exhaustive list of all constructs and measures by level, but they do summarize important constructs that may need to be taken into consideration when designing multilevel interventions. In addition, it provides examples or references for the measures. Tables are organized into sections for group (Table 1), organization (Table 2), and community (Table 3).
Many models of team dynamics (112,113) and a rich body of research about conditions that influence group outcomes exist (114,115). According to this literature, certain aspects of groups—the environment, characteristics, context, processes, and emergent states—are related to group outcomes. For each of these categories of conditions, key constructs and references for the associated established measures are summarized in Table 1. For an overview of the field, see Stewart (116,117) and Mathieu et al. (118). For an overview of the field from a health-care perspective, see Lemieux-Charles and McGuire (119), West et al. (96), and Poole and Real (120). See Wageman et al. (55), Kirkman et al. (63) and the articles by Campion et al. (61,62) for discussion of a wide variety of group measures and their psychometric properties.
The most common approach to measurement at the group level is to use a group-level construct and an associated group-level measure to collect information through a survey of the individual members of the group(s). Then various statistical tests (ie, rwg, ICC) are conducted to justify aggregation of individual responses to the group level, address the lack of independence of these answers, and establish variance between groups. This approach is warranted only if group-level constructs and group-level measures are used; if individual-level constructs and measures are used, aggregation is not theoretically justified, even if it may be statistically supported.
Another approach to measurement at the group level, although not used as frequently, is to use group constructs and related group measures to collect data directly from the group(s) through consensus (eg, the group together fills out a survey). This approach may be conducted with or without group facilitation. The advantage of this approach is that there is no need to deal with aggregation issues, as the group provides only one response. Disadvantages of this approach include more complicated data collection logistics and the possibility that group dynamics may bias the responses. For an example of this approach, see how Grady and Travers (34) used the consensus technique to generate group-level measures of satisfaction with a home hospice program.
Both of these approaches to group-level measurement have been found to be reliable and valid and to yield comparable results. To avoid common method bias when conducting group-level research, data are usually collected not only from group members but also from third parties such as patients or other medical personnel who interact with the group. For example, patients may be asked to provide feedback on the performance of a caregiver group (eg, if the desired outcome was achieved).
Although organization-level measures are rare in the cancer care intervention literature, the use of aggregated individual-level data, informants, metrics/scorecards, records, and other organizational documents/data is common in studies of organizations. Researchers often use the American Hospital Association database of hospitals and delivery systems for objective information such as size. Aspects of organization structure, such as degree of decentralization, are typically obtained through key informants in an organization. For example, see Wei et al. (22), who assessed practice characteristics at baseline and post intervention, as reported by physician informants. Measures of organization culture are typically obtained through surveys. Culture is a good example of a measure in which dispersion of responses has meaning. High agreement among organization members is a measure of a “strong culture,” whereas low agreement indicates a “weak culture.”
A broad array of constructs and measures characterizes the organizational literature (Table 2), and in determining what constructs are most relevant, the researcher should be guided by theory and logic. For example, if an intervention requires coordination among providers, it would be wise to include measures of structures and processes that facilitate or hinder coordination [see (90,95)] as well as measures of the processes of coordination [see (94)]. As a further example, the organization literature argues that structuring an oncology practice by multidisciplinary service lines facilitates coordination across disciplines, but this does not ensure such coordination. Thus, it is important also to measure actual coordination among these providers.
Community-level measures include the context, resources, and norms of populations. Measures are often aggregated at varying levels including the social network, neighborhood, catchment area, region, state, or country. Table 3 summarizes the types of constructs under each of these domains that are used most frequently in the literature. We have restricted our summary to just a few illustrative example measures.
Community-level measures often are derived by aggregating individual-level measures (eg, socioeconomic status) from population-based surveys and administrative datasets to measure different constructs (eg, socioeconomic deprivation) at the population level. As noted previously, to distinguish the independent effects of community-level constructs, hierarchical modeling techniques that control for individual-level variation while estimating community-level effects are needed when using these measures. Numerous prior studies have documented that these aggregated measures of community context have effects on outcomes that operate independently from the individual-level measures from which they are derived [see (121)].
The most common contextual measures used in community-level research are probably those that describe the socioeconomic conditions of a community defined by geographic boundaries. Most typically derived from regionally aggregated census data, these measures may summarize the income or education distribution of a population or combine several indicators of socioeconomic status into a single summary index (102). In multilevel intervention research, these measures would most likely be used as control or stratifying measures but also may be conceptualized as potential moderating factors for interventions directed toward individuals or communities.
Exposure to information or media is a commonly used cancer intervention strategy to enhance population knowledge and awareness, but it also may be an important moderator or control measure in multilevel cancer intervention research. For example, Han et al. (103) found that exposures to mass media (television, radio, Internet, health news) were associated with perceived ambiguity about cancer prevention recommendations.
Economic measures of market behavior and economic capacity are another type of contextual measure that may be used in multilevel intervention research. One type of measure characterizes the economics of markets in ways that enable inferences of spillover effects of competitive organizational or physician behavior on clinical outcomes. Typical measures include concentration of specific delivery systems in a market (eg, penetration of managed care enrollment in a community), or more direct measurement of the competitive character of the health market place (eg, the Herfindahl–Hirschman index) (105). For example, Keating et al. (104) used a measure of managed care penetration to draw inferences about the quality of cancer care.
Health policy also is a contextual factor that can be important to include in multilevel intervention research, either as an intervention or as a mediating or moderating influence on cancer interventions. Cancer intervention research has used state policy measures in studies of tobacco cessation research (eg, state policy adoption of system strategies), cancer screening (eg, state insurance coverage mandates), and—to a lesser extent—treatment (eg, insurance coverage mandates and minority population access to clinical trials). Often this policy domain is measured by the presence or absence of a policy, but more complex measures have been used to characterize the leniency or restrictiveness of existing policies. Government policies are not the only policies that have been measured in cancer intervention research. Professional standards and guidelines are other important areas of policy that are suited to measurement, usually in process-oriented clinical guidelines that are measured at the individual level and then aggregated to the population level.
Measures of community-level resources that may be important to consider in multilevel cancer intervention research include health services delivery capacity, community resources, and social support. Measures of health service delivery capacity could include community-level measures of health-related infrastructure, workforce, and technology. A final community-level resource construct to consider in multilevel cancer intervention research is social support. Cancer support groups have been widely used as a strategy for enhancing coping skills among cancer survivors [see (110)], and a recent study of colorectal cancer screening behavior examined the effects of tangible and emotional sources of social support on adherence to screening guidelines (122).
Professional and social norms also are important to consider as community-level constructs, as they affect individual behavior. Professional norms include procedure utilization, prescribing patterns, and attitudes related to use of guidelines. Social norms include those related to health behaviors, end-of-life care, and use of narcotic medications.
A number of important conclusions emerge from our review of the literature on multilevel measures and interventions in cancer. First, explicit consideration or mention of levels is generally lacking in intervention studies. Second, few studies consider the group, organization, or community levels. Third, many studies are not guided by conceptual frameworks. Fourth, insufficient attention has been paid to measurement issues, such as scale construction, reliability, validity, sample size, or power. The special issues related to measurement of multilevel interventions also are underdeveloped (eg, dealing with lack of independence, aggregation, the need for alignment of theories, constructs, and measures, and the complexity of analyzing cross-level interactions). Finally, multilevel measurement needs to be considered in a comprehensive and complex manner, with attention given to effects within and across levels.
This material is based upon work supported in part by the Health Services Research and Development Service, Office of Research and Development, Veterans Health Administration, Department of Veterans Affairs. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government.