The left-hand column of Table shows the ICC calculations (with 95% confidence intervals) from the first data download in September 2008, involving 1,572 children from 15 polyclinics. The ICCs for standing height, weight, mid-upper arm circumference, hip circumference, head circumference, triceps and subscapular skinfold thickness and glucose were all relatively low (< 0.10), indicating modest clustering within polyclinics. The ICC of 0.005 for impedance indicated that this measurement was particularly consistent across polyclinics--perhaps not surprising, as impedance is measured using an automated scale, and is therefore less subject to differences in measurement technique.
Intraclass correlation coefficient (ICC) calculations (with 95% confidence intervals) for continuous outcome measurements at data downloads between September 2008 and May 2010
The ICC of 0.92 for sitting height, however, was higher than expected based on a previous follow-up of PROBIT at age 6.5 years (when the ICC was 0.09) [10
]. To identify the reason for such a high degree of within-center clustering, we constructed side-by-side box-plots of sitting height at each polyclinic (Figure ). The plot revealed that the mean sitting height measurements of polyclinics B and AB were approximately 50 cm higher than the other clinics.
Figure 1 Side-by-side box-plot of sitting height (cm) measurements, by polyclinic. (Missing polyclinics in each box-plot reflect polyclinics that had either not commenced recruitment, or had no records entered into the database at the date of the download). 1A: (more ...)
We suspected that these two clinics had been using an incorrect ruler to obtain the sitting height measurement. At the start of the study, we provided each pediatrician with a standard wall-mounted stadiometer containing two integrated rulers: one for measuring standing height and one for sitting height. The sitting height ruler started at 50 cm above the floor, to account for the standard 50 cm study stool upon which the child was seated while measured. Pediatricians at these two polyclinics confirmed that they had been using the wrong ruler, and we adjusted the measurements by 50 cm.
Results from a subsequent download six months later (4,865 records from 27 polyclinics) revealed that the ICC for sitting height had dropped markedly (from 0.92 to 0.28); however, this value still suggested some degree of clustering. Another box-plot graph was helpful in identifying the problems (Figure ). This plot revealed several extreme measurements, particularly at clinics B and N, which had been taken with the incorrect ruler before we corrected the pediatricians' technique but entered into the database later. Additionally, sitting height at clinic AB was now noticeably lower than the others. When questioned, the pediatrician from clinic AB explained that she had been using a sitting stool that was 40 cm high rather than the 50 cm study stool. When we added 10 cm to the sitting height measurements from site AB, the mean was consistent with the other clinics.
Further monitoring of the ICC calculations indicated improvements in data quality. In a 10/2009 download of 8,949 records, the sitting height ICC dropped from 0.28 to 0.03. A box-plot of sitting height by clinic from this download confirmed that measurements were more consistent across clinics (not shown). Even after we corrected these errors in measurement and reporting and achieved low ICC's, we found that it was important to account for clustering by center in our analyses. For example, in a preliminary analysis unadjusted for clustering by polyclinic, sitting height was significantly higher (0.49 cm, 95% CI: 0.35, 0.64) in intervention compared with control clinics. However, once we accounted for clustering using the PROC MIXED procedure within SAS, this difference disappeared (-0.09 cm, 95% CI: -0.86, 0.68).
Monitoring ICC calculations was helpful in identifying more subtle procedural inconsistencies as well. The waist circumference ICC from the initial download was 0.10, which was higher than the ICCs for other measurements made using a measuring tape, such as hip (0.05) or head (0.03) circumference. A box-plot of waist circumference by clinic showed some variation across clinics, but the differences were less obvious than the sitting height measurements (Figure ). We consulted the study monitors, who explained that the pediatricians were not measuring waist circumference consistently. Because the Russian translation of the manual of procedures was not entirely clear, some pediatricians were measuring waist circumference overlying the iliac crest, and others were measuring just above it. Some pediatricians also held the tape at a gentler tension than others while taking the measurement.
Side-by-side box-plot of waist circumference (cm) measurements, by polyclinic.
The ICC for systolic blood pressure (0.16) also suggested more variability across clinics than expected. A box-plot graph (not shown) of systolic blood pressure suggested that a handful of polyclinics had notably higher mean blood pressure measurements. Discussions with the pediatricians and monitors revealed that some pediatricians were not instructing the child to rest for five minutes prior to measuring blood pressure.
We also noticed an increase in the ICC for head circumference, which was relatively low until the data download in October, 2009, at which point it jumped from 0.05 to 0.13 (Table ). Discussions with the pediatricians revealed that two clinics had been placing the measuring tape just below, as opposed to over, the occipital prominence. As a result, their measurements were lower than the other clinics. This discrepancy was not apparent until their records had been entered into the study database. Lastly, we noticed that the ICC for triceps skinfold thickness increased slightly from 0.07 to 0.11 from March to October 2009 (Table ), but after review of the box plots and discussion with the field staff we were unable to identify any cause for this increase.