Measures were limited in some cases for participants with more severe vision loss using the standard letter charts due to the restricted measuring range of the ETDRS (≥20/1600 at 50 cm) chart compared with the GAT (20/32 to 20/6400). The range of visual acuities measured were comparable across the two tests for the RP group (ETDRS 0.322 to 2.003 logMAR and GAT 0.398 to 2.204 log-MAR), whereas the ranges obtained for the OR group were narrower for the GAT (ETDRS 0.98 to 1.903 logMAR and GAT 0.398 to 1.35 logMAR). The CS ranges were wider when measured with the GCS test than the PR for both the RP (converted GCS 0.02 to 1.07 logCS, PR −0.10 to 0.60 logCS) and OR groups (converted GCS 0.31 to 1.8 log CS, PR −0.05 to 1.20 logCS).
demonstrates the relationship between the within-session mean GAT and ETDRS acuities obtained at each visit for each eye for all subjects. The better eye is designated by open symbols and the worse eye with filled symbols. In the RP group, the relationship between the GAT and ETDRS establishes equivalence of the GAT test (), whereas in the OR group, the relationship is poorer for patients with more severe pathology (). To compare whether the GAT was in good agreement with the ETDRS, we plotted the difference between measures obtained with the two methods in a given session against the mean of both measures in . The slopes (m) of the regression lines for these Bland-Altman25
plots for RP () and OR () were not statistically significantly different from zero, when using multilevel, mixed effects models to account for correlations within subjects and eyes (RP: m = 0.11; 95% CI: −0.06, 0.29; p = 0.21; OR: m = −0.07; 95% CI: −0.33, 0.20; p = 0.62).
FIGURE 2 Scatter plot demonstrating the relationship between VA with the ETDRS charts and the GAT for the RP (A) and OR (B) groups. Each subject’s ETDRS acuity (y-axis) was plotted against his/her within-session mean GAT acuity (x-axis) at each visit for (more ...)
Bland-Altman plots of the difference between the two VA tests (grating test minus letter chart) vs. the mean of both tests for the RP (A) and OR (B) groups.
demonstrates the relationship between the GCS and PR measures for each eye at each visit. We dichotomized the measures into two subgroups according to the difference between their ETDRS VA and the PR letter size (1.54 logMAR or 20/700 Snellen). Subjects with poorer acuity (VA ≤20/350; i.e., not more than 0.3 logMAR better than the PR letter size) are presented with open symbols, whereas subjects with better acuity (VA >20/350, i.e., more than 0.3 logMAR better than the PR letter size) are presented with filled symbols in .
FIGURE 4 (A, B) Scatter plot demonstrating the relationship between CS obtained with the PR charts and the GCS test. Each subject’s PR acuity was plotted against his/her within-session mean GCS at each visit for each eye and each group. The GCS values (more ...)
The RP subgroup in the upper acuity range fared better with a stronger relationship between methods (m = 0.59, r = 0.45, p = 0.02, n = 26, excluding measures at floor from two subjects with the poorest acuity levels). However, there were more measures (n = 31) at floor in the RP group with poorer acuity than the OR group. The OR subgroup with better acuity (>20/350) yielded a better relationship between measures (m = 0.53, r = 0.39, p = 0.06, n = 24) than the poorer acuity subgroup (m = 0.27, r = 0.31, p = 0.08, n = 34); however, both were marginally significant. On average, higher logCS levels were obtained using the GCS than the PR in both groups. Some subjects with VA close to or worse than 20/700 demonstrated very poor CS when measured with the PR chart but were able to obtain much better CS levels with the GCS. This result was likely obtained because the resolution of the GCS test was set at 0.6 logMAR above their GAT threshold, while the resolution of the PR chart is limited and fixed.
The linear regression slope (m) for the Bland-Altman analysis for differences between the two CS measures was not statistically significantly different from zero for the RP group (m = 0.07; 95% CI: −0.09, 0.22; p = 0.39), when using multilevel, mixed effects models to account for correlations within subjects and eyes. This slope was statistically significantly different from zero for the OR group for subjects with acuity ≤20/350, however (m = 0.41; 95% CI: 0.12, 0.70; p = 0.005).
Between- and within-session reliability for the VA and CS tests are shown in the box plots in . The between-visit median CR.95 was 0.11 to 0.16 log units for the ETDRS charts and GAT in both groups (). The between-visit median CR.95 for the converted GCS test in the RP group was 0.15 logMAR and 0.14 for the PR chart. For the most part, measures were consistent across all eyes/subjects except for two extreme values in the CS scores. The OR group showed more variability for the CS tests: 0.24 median logCS for PR and 0.34 and 0.41 median logCS for the worse and better acuity subgroup, respectively (, right). The within-session median CR.95 was 0.11 and 0.17 logMAR for the OR and RP GAT, respectively (). The median converted GCS was 0.13 logCS for the RP Group and 0.13 and 0.15 for the worse and better acuity subgroups, respectively (shown in , right). GAT measures on average were more consistent than the GCS measures for within- and between-session measurements for all subjects. The OR subgroup that was limited by poorer acuity (≤20/350) on their PR scores showed more within-session variability in their GCS scores.
FIGURE 5 Box plots of the 95% coefficients of repeatability (CR.95) between test sessions (A) and within test sessions (B) for each test in each group. The bottom and top of the box are the 25th and 75th percentile (the lower and upper quartiles, respectively), (more ...)
The increased variability observed in the GCS between-group CR.95 may be due to the inclusion of participants with severe vision loss who experienced floor effects when tested with the PR. To assess this potential effect, paired samples t-tests between CR.95 values were conducted for tests with and without participants with values at floor. No significant differences were found for the RP group when including participants with values at floor (9 eyes) on the PR [t(15) = −1.37, p = 0.19] compared with those without [t(6) = 2.35, p = 0.06]. Although the observed difference between GCS and PR was greater in the OR group, no significant differences were found when including the one subject with values at floor [t(15) = 1.75, p = 0.10] vs. without this subject’s data [t(14) = 1.86, p = 0.09]. There was a slight difference in CR.95 values observed between the ETDRS and GAT measures for the RP group. A paired samples t-test indicated a significant difference [t(15) = 2.26, p = 0.04] when including those eyes with floor measures on the ETDRS (three eyes) relative to the between test comparisons without floor values [t(14) = 0.22, p = 0.83]. No participants were excluded from the VA measures in the OR group. At least for the RP group, it appears that the increased variability in GAT measures may have been influenced in part by including subjects with floor values in the ETDRS.
The between-visit CR.95 did not depend on the magnitude of the VA or CS results obtained with the ETDRS, PR, or GCS tests. The multilevel linear regression slopes for the between-visit CR.95 as a function of mean VA or CS were not statistically significantly different from zero for RP ETDRS (m = −0.003, p = 0.93, n = 13), as shown in , left; RP PR (m = −0.17, p = 0.40, n = 7) and RP GCS tests (m = −0.13, p = 0.06, n = 16) in , left; OR ETDRS (m = 0.07, p = 0.34, n = 15), as shown in , right; and OR PR (m = 0.04, p = 0.74, n = 15) and OR GCS tests (m = 0.10, p = 0.51, n = 16) in , right. , right subdivides the data by acuity limit according to . Filled symbols represent data in the lower acuity range for both tests and open symbols represent the upper acuity range. Graphically, it appears that variations in VA do not contribute to the observed CR.95 values. The multilevel linear regression slope was statistically significantly different from zero for the GAT and is shown in the scatterplot of the between-visit CR.95 as a function of mean VA for RP GAT (m = 0.10, p = 0.001, n = 13) (, left). Therefore, it appears that there may be a slight increase in variability for the GAT for RP subjects with severely reduced mean VA. A statistically significant slope was also found for the GAT in the OR group (m = −0.25, p = 0.01, n = 15) (, right). However, we also performed a sensitivity analysis in which we excluded the OR subject with the most variability who also had the best VA in the group and found that the slope was no longer statistically significant (m = −0.19, 95% CI: 0.004, −0.38, p = 0.055). Because it is difficult to assess the variability according to the level of acuity loss in the OR group due to the limited range of the GAT results we obtained for that group and because the linear regression was no longer statistically significant after removing the single most variable subject, the regression line is not shown in , right.
FIGURE 6 Scatter plots indicating the relationship between the 95% coefficient of repeatability (CR.95) and mean VA measured with the ETDRS charts and GAT for each eye tested for each group (A), including a regression line for the GAT in the RP subjects indicating (more ...)