|Home | About | Journals | Submit | Contact Us | Français|
In this study, we assess the potential of three subscales of the Children's Self‐Perceptions of Adequacy in and Predilection for Physical Activity (CSAPPA), a measure of generalised self‐efficacy, as possible screens for developmental coordination disorder (DCD).
We used the Bruininks‐Oseretsky Test of Motor Proficiency short form (BOTMP‐SF) to identify probable cases of DCD. We administered the BOTMP‐SF and the CSAPPA to 590 children in grades 4–8 from four schools in the Niagara region of Ontario, Canada. We used receiver operator characteristic (ROC) analysis to assess and compare the performance of the subscales and the full instrument.
The area under the receiving operating characteristic curve (AUC), a measure of the overall performance of the test against a diagnostic standard, was good for the full CSAPPA (AUC=0.81, 95% CI 0.75 to 0.87). The adequacy (AUC=0.79, 95% CI 0.73 to 0.85) and predilection (AUC=0.80, 95% CI 0.74 to 0.87) subscales had performance statistically equivalent to the full scale. Since the adequacy subscale is shorter and has good content validity with respect to DCD, we ran additional analyses on this measure. A cut‐point of 24 on this subscale gives a sensitivity of 0.86 (95% CI 0.76 to 0.97) and a specificity of 0.47 (95% CI 0.43 to 0.51).
The adequacy subscale of the CSAPPA appears to be equivalent to the full measure for the purposes of screening for DCD. Further research should explore the possibility of adding further criteria to improve the CSAPPA's modest specificity in this role.
Developmental coordination disorder (DCD) is characterised by poor motor proficiency that results in a significant impairment in social and academic functioning and is not the result of another psychiatric, neurological or other medical condition.1 DCD is common, with prevalence estimated at 5–6%.1,2 The specific manifestations of the disorder are varied and pervasive, and include gross and/or fine motor skill impairment. These problems make day‐to‐day activities such as tying shoelaces, writing and participating in activities such as skipping or basketball extremely difficult. It is not surprising, therefore, that children with DCD tend to participate less in social activities than other children, as social activities in childhood often involve physical activity.3
Despite its relatively high prevalence, most children with DCD are never diagnosed.3 Rather, teachers typically describe these children as clumsy, awkward or lazy.4 However, DCD is strongly associated with behavioural and emotional problems,5,6 low self‐worth,7,8,9 poor perceived competence,9 anxiety,9,10 depression,11,12 bullying8 and obesity.13 Cairney et al13 recently demonstrated that children with DCD tend not to participate in physical activities, increasing the likelihood of overweight/obesity and poor cardiorespiratory fitness.14
If identified early, the physical health and academic and emotional needs of affected children can be addressed and negative experiences prevented.15,16 The potential for improved quality of life justifies efforts to screen for and identify children with DCD in non‐clinical settings.17 However, existing screening measures are based either on parent18 or teacher19 reporting of motor coordination difficulties. To date, child self‐report measures have not been available.
Previous work has examined the possibility that the Children's Self‐Perceptions of Adequacy in and Predilection for Physical Activity scale (CSAPPA) may be useful as a screening instrument for DCD in children aged 9–14.3,17 When compared with a standardised motor assessment, the Bruininks‐Oseretsky Test of Motor Proficiency short form (BOTMP‐SF),20 sensitivity and specificity for boys (0.90 and 0.89, respectively) and girls (0.88 and 0.75, respectively) on the CSAPPA were moderate to high. The advantages of the CSAPPA over motor testing are: i) it can be administered to children in groups in 15–20 min (unlike motor testing, which is administered individually); ii) it is easy to score; and iii) it can be administered by teachers or research personnel. However, there are instances where the 19‐item CSAPPA measure is too demanding and a shorter screening instrument is required. In clinical settings, brief and effective screening instruments are preferable. Moreover, in population‐based studies, where multiple measures are being administered in a single survey, a premium is placed on shorter instruments that require little time to complete.
The 19‐item CSAPPA is composed of three subscales: i) perceived adequacy (seven items); ii) predilection toward physical activity (nine items); and iii) enjoyment of physical education class (three items).21 The purpose of this study was to compare the CSAPPA with a standardised measure of motor proficiency which is often used to identify children with DCD and to evaluate the three CSAPPA subscales as possible short‐form screens for DCD.
The study involved a cross‐sectional investigation of students in grades 4–8 from five elementary schools in the Niagara region of Ontario, Canada. Data collection within the school board was not randomised, but attention was given to the selection of schools to ensure that the participants represented the socioeconomic, ethnic and urban/rural mix that occurs in the general Canadian population.22 Eighteen children with pre‐existing physical limitations and eight children with learning disorders were excluded from all analyses. From a potential 929 students, 590 children (322 males, 268 females; 63.6% response) provided informed consent and participated in the study. Following a listwise deletion of cases with missing values, the total sample size was 546. Both the Brock University Research Ethics Board and the District School Board of Niagara Research Ethics Committee approved the protocol for this study. Informed consent was obtained from the parents of each child.
Children's motor proficiency was evaluated using the BOTMP‐SF.20 This test examines the full scope of motor proficiency using selected items from the full scale. The short form takes 30 min to complete, as opposed to 2 h for the full version, and has been validated against the full scale, with correlations of between 0.90 and 0.91 for children in the 8–14‐year age range.20 While not providing an in‐depth analysis of each aspect of motor proficiency, it provides an assessment of general motor functioning.23,24 In this study, the BOTMP‐SF was individually administered by a trained research assistant to each consenting child behind a curtained barrier in the school's gymnasium. The BOTMP‐SF examiner was blind to the results of the CSAPPA scale. A BOTMP‐SF standard score (age adjusted) below 38, which is at or below the 10th percentile rank on the BOTMP‐SF, was used to identify cases of “probable DCD”. We use the term probable DCD as no formal diagnosis was established by a physician. Table 11 outlines the complete DSM‐IV case identification criteria for DCD and the assessment protocol used in this study. This assessment protocol for probable DCD (pDCD) has been used in previous studies.13,23,25,26,27,28
The CSAPPA is a 19‐item scale designed to measure children's perceptions of their adequacy in performing, and their likelihood of selecting, physically active games or sports.21 The scale employs a structured alternative choice format. Children are asked to choose the option that best describes them from pairs of statements such as “some kids are among the last to be chosen for active games” and “other kids are usually picked to play first” by indicating whether the selected sentence was either “sort of true for me” or “really true for me”. Hay21,29 designed the CSAPPA scale for 9–16‐year‐old children; it has demonstrated a high test–retest reliability (r=0.84–0.90) as well as strong predictive and construct validity.17,21,29 With regard to construct validity, the CSAPPA is significantly correlated with aerobic fitness, physical activity, anthropometry and motor proficiency.21,29,30 The internal consistencies for each factor are: adequacy, α=0.83; predilection, α=0.83; and enjoyment, α=0.86. The CSAPPA and its component scales are scored such that higher scores reflect greater self‐efficacy (range 19–76).
Receiver operator characteristic (ROC) curve techniques were used to assess performance of the CSAPPA and its three subscales (fig 11).). In a medical context, ROC curves provide a graphical and numerical means of assessing the ability of a test to discriminate between people with and without a condition of interest. The larger the area under the ROC curve (AUC), the more potent the diagnostic ability of the test. The AUC is equal to the probability that a randomly selected case will have a higher score than a randomly selected non‐case. The accuracy of tests with AUCs of between 0.50 and 0.70 is considered low; between 0.70 and 0.90, moderate; and over 0.90, high.31
In the first stage of the analysis, we plotted ROC curves for the full CSAPPA and each of its three subscales, and calculated AUCs for each. Given previous evidence of gender differences in DCD,17,26 we then tested for differential performance in male and females subsamples by comparing AUCs calculated for each group separately. To determine whether the performance of each subscale differed from that of the full CSAPPA, we then performed pairwise tests comparing the AUC of each subscale to that of the full instrument. On the evidence of the preceding analyses, we selected the adequacy subscale as the best candidate for a short‐form DCD screen and calculated sensitivity, specificity, and positive and negative predictive values for each of its potential cut‐points. Because some confidence intervals would otherwise have overlapped 0 or 1, these were calculated using the Wilson score method.32 We then identified a cut‐point suitable for screening purposes; as the first aim of a screen should be to retain true positives for further assessment, we gave most weight to sensitivity in making this selection. All analyses were performed using the roctab, roccomp, rocfit and rocgold procedures in Stata 8.0.33
A total of 590 students participated in this study (table 22).). The proportion of students scoring below our cut‐point for probable DCD was 7.5% (95% CI 5.4% to 9.6%). Although somewhat more common among females (9.4%, 95% CI 5.9% to 13.0%) than males (5.9%, 95% CI 3.3% to 8.5%) in this sample, this difference was not significant (χ2 2.6, df=1, p=0.11). Age did not differ significantly between students with and without probable DCD (t=0.09, df=584, p=0.93).
AUC values (table 33)) were in the moderate range for the full CSAPPA and for the predilection and adequacy subscales, while that for the enjoyment factor was considerably lower. Formal tests for differences between the AUCs revealed that this difference was significant, while performance of the other three scales was equivalent.
Since the use of the empirical method of ROC analysis may bias the performance of short instruments downwards, we also fit an ROC curve for the enjoyment subscale (which includes only three items) using the binormal method.34 This approach resulted in an estimated AUC of 0.71, which, while marginally higher than the original estimate, is still well below those of the other subscales and the full instrument.
To be consistent with previous work,17 we also tested each of the four scales for evidence of differential performance on male and female subsamples by comparing AUCs calculated for each group separately. Our examination of gender differences in performance revealed no statistically significant differences for any of the four scales.
As the performance of the predilection and adequacy factors was very similar to that of the full scale, we selected the adequacy subscale for further analyses. This decision was based on its brevity (the adequacy scale includes seven items, predilection includes nine) and on theoretical considerations, self‐reported adequacy in physical activity seeming more closely related to possible movement disorder than predilection.
Sensitivity was reasonable (>0.80) for adequacy scores from 22 to 28. In this part of the scale, specificity ranges from 0.10 to 0.64 (table 44).). Children scoring below 24 might be considered for further testing, as this would eliminate 47% (specificity; 95% CI 0.43 to 0.51) of those without DCD while retaining 86% (sensitivity; 95% CI 0.76 to 0.97) of those with it. Due to the modest specificity and low prevalence, however, the positive predictive value at this point of the scale is only 0.12 (95% CI 0.09 to 0.16).
Since there may be circumstances where a different trade‐off between sensitivity and specificity is preferable, the full range of values is provided (table 44).). For the full scale, a cut‐off of 63 corresponds to a sensitivity of 0.91 (95% CI 0.78 to 0.96) and a specificity of 0.53 (95% CI 0.49 to 0.57).
The ability of the CSAPPA and its subscales to detect cases of DCD is moderate31 when caseness criteria are set to include children with mild to moderate impairment. While a higher level of agreement would be desirable, DCD is difficult to detect, in part due to its heterogeneous nature.35 Until the clinical criteria and nature of this condition are more clearly defined, the development of highly accurate screening tools with excellent sensitivity and specificity will be difficult. In light of these considerations, the instrument's ease of administration and its usefulness for other purposes, the CSAPPA performs well. Its ability to identify cases of DCD in a community setting is comparable to that of the Developmental Coordination Disorder Questionnaire, a dedicated parent report instrument.16 Results in the present study, however, are not wholly in agreement with those previously reported17; in particular, our estimates of sensitivity and specificity, and the optimal cut‐point on the scale, are different. This is surprising, since both samples were derived from schools in the same geographical area and the measures are identical to those used in the original study. The likeliest explanation is that the relatively small sample size in the previous study is responsible. After stratification by gender, there appear to have been no more than 11 cases of DCD in each group, which suggests large confidence intervals for sensitivity. Thus, the difference between the original estimate and the one in this study is likely to be within the bounds of statistical error.
The CSAPPA adequacy and predilection subscales appear to be good candidates for short screening instruments. There is no significant degradation of performance when either of these are used instead of the full CSAPPA. However, on all scales, specificity is modest at high levels of sensitivity. High sensitivity and low or moderate specificity means that true cases will be accompanied by a relatively high number of false positives. This is typical for screening tools that are intended for use in a two‐stage detection process; their function is simply to exclude a large proportion of the true negatives before the administration of further assessment. As they are brief instruments with reasonable sensitivity, subscales of the CSAPPA could be useful in such a process. The second stage could involve a brief clinical interview of parents, or follow‐up with another instrument, such as the DCD‐Q,16,23 to determine the impact of motor coordination problems on everyday activities, followed by a standardised motor assessment such as the Movement Assessment Battery for Children18 or the BOTMP.20 The high proportion of false positives does, however, limit the usefulness of the CSAPPA when used alone. At the cut‐off we suggest only 12% of those screening positive will have DCD. While this represents a considerable saving over testing all children, it also indicates that a low CSAPPA or adequacy subscale score must be supplemented by other evidence of movement disorder. In a screening context, this is supplied by the second stage of case ascertainment. In a clinical setting, however, the CSAPPA should be considered as one piece of evidence among others; when combined with a known prior probability in this way, it will be considerably more informative.
A strength of the CSAPPA is that it has uses beyond screening for DCD. A child's low self‐assessment of adequacy in physical activity is problematic in itself, and may be useful in identifying other difficulties or risk factors for inactivity and obesity. The CSAPPA as a whole can help identify children who dislike physical activity, or think they are poor at it, and may suggest interventions by providing information on specific areas of difficulty.
Finally, it is important to note that, unlike false positives on instruments intended to diagnose serious disorders such as cancer or HIV, the burden associated with a false positive screen for DCD, if reported appropriately, is likely to be minor, and is unlikely to concern children or parents in the absence of other evidence for movement problems. In contrast, missing cases of DCD may have serious consequences. While the disorder is not well understood, the symptoms are noticeable to those around the child, and the consequences significant. Most children with DCD, and their parents, are aware of their motor problems.36 In the absence of a diagnosis, parents are left with little leverage in advocating for their child.36 Lacking a diagnosis, children and their parents may believe that the condition can be outgrown, or alleviated by behavioural change (if the child just tries harder or pays more attention to what he/she is doing), views that are clearly erroneous.36 Recognition of the daily struggles of these children is critical to prevention of deterioration in self‐worth.9
In clinics that handle referrals for paediatric developmental problems, the full CSAPPA may actually be more useful than the results presented here suggest, since the prevalence of DCD in this population will be considerably higher than in the general population. Positive predictive values are dependent on the background prevalence of the disorder; when the disorder is rare, most positive results are likely to be false.37,38,39 In populations where a larger number of positive cases are to be expected, a larger proportion of children screening positive will actually have the disorder. Alternate cut‐offs may be required for this purpose, however, since children in a clinical sample would likely score lower on the scale than children from a general population sample.
Further work in this area should also explore triangulation methodologies, where more than one screening instrument is used to maximise sensitivity and specificity. The DCD‐Q, for example, is a parent‐completed questionnaire designed to screen for DCD cases.16,23 Using both the CSAPPA scales and the DCD‐Q for parents may significantly improve case detection in a field setting. It would also be useful to compare the multi‐method approach across different samples (child versus teacher‐reported versus parent reported motor proficiency).
Finally, as with any study, there are limitations that need to be considered when interpreting the results. Most notably, as mentioned earlier, our cases of DCD are “probable” because we did not use a clinical assessment using the full DSM‐IV criteria1 to assess caseness. In the absence of a “gold standard” instrument for diagnosing DCD,19,23,40 however, we believe the BOTMP to be a reasonable choice, as it is widely used to detect motor coordination problems in children.23,41,42,43 Increasingly, however, clinicians and researchers are using the Movement‐ABC18 as the preferred instrument for identifying cases of DCD. Work showing that the CSAPPA performs similarly with other measures of motor impairment (such as the Movement‐ABC) would be reassuring.
To conclude, we have confirmed that the CSAPPA, a brief scale intended to measure self‐efficacy towards physical activity in children, is a useful screen for motor difficulties in children. Furthermore, the adequacy and predilection subscales perform as well as the full instrument. Given its brevity (seven items) and content (perceived adequacy toward physical activity), the adequacy subscale appears to be a sufficient substitute for the full CSAPPA, especially in situations where shorter instruments are desired. The CSAPPA is an instrument that has the possibility to be useful as both a clinical and research tool to screen for children with DCD.
AUC - area under the curve
BOTMP‐SF - Bruininks‐Oseretsky Test of Motor Proficiency short form
CSAPPA - Children's Self‐Perceptions of Adequacy in and Predilection for Physical Activity
DCD - developmental coordination disorder
ROC - receiver operator characteristic
Competing interests: None.