Table presents per indicator and per questionnaire the mean scores, the 5th and 95th percentile, and the 90% central range. At t0 the experiences of respondents on all indicators were positive for the interview questionnaire, as were the experiences of representatives and assisted-living clients on the mail questionnaire: 3.23 (2.82-3.55), 3.19 (2.90-3.41), and 3.33 (3.16-3.46), respectively. For the interview questionnaire the 90% range was 0.73, and for the mail questionnaires (representatives and assisted-living clients) it was 0.51 and 0.30, respectively.
The largest difference between the 5th and 95th percentile of the interview questionnaire was for the indicators 3_2 'Autonomy' (1.08), 6_1 'Care plan and evaluation' (1.06), and 6_2 'Shared decision making' (1.05). Housing and privacy (indicator 2_3) is an indicator of the representatives' questionnaire that showed considerable variation between organizational units (1.25). No indicators of the third questionnaire diverged as much as the other questionnaires. Indicator 6_5 'Coherence in care' showed the most variation (0.54).
Figure shows the change scores per indicator for all three questionnaires. We expected change scores to be limited by the 90% observed range on the first measurement; this range varied between indicators and questionnaires (Table ). Theoretically, an indicator score can change 3 points (from 1 to 4, and vice versa). The change scores for the interview questionnaires were all positive (0.06-0.37). For the interviews with residents and the mail questionnaire for representatives, the three indicators that improved the most were the indicators 6_2 'Shared decision making' (0.37; 0.23), 6_1 'Care plan and evaluation' (0.36; 0.15), and 6_3 'Information' (0.26; 0.10). Indicator 2_3 'Housing and privacy' also changed by 0.10 for the representatives questionnaire. Scores of the questionnaire for assisted-living clients improved the most with respect to indicators 6_2 'Shared decision making' (0.25), 6_5 'Coherence in care' (0.09), and 6_4 'Telephone accessibility'(0.08). The surveys of the representatives showed a decline in indicator scores 4_1 'Mental well-being' (-0.04), 6_6 'Availability staff' (-0.02), and 2_2 'Atmosphere' (-0.01). In the assisted-living clients questionnaire, three indicators showed a decline: indicators 6_1 'Care plan and evaluation' (-0.08), 3_1 'Daily activities' (-0.05), and 6_6 'Availability staff' (-0.05). The change score of the assisted-living clients (range -0.08 to 0.25) did not diverge to the same extent as the change scores of the other questionnaires.
Change scores of all indicators and the 95% confidence interval.
To illustrate the differences in scores between questionnaires, Figure shows the mean change scores over all indicators between the two measurement points (t0 and t1). As expected, there was a negative relationship between prior performance and change (Figure ). At t1 the organizational units of performance groups 1, 2 and 3 showed an average improvement of 0.34 (SD = 0.29), 0.27 (0.23), and 0.11 (0.20), respectively. At t1 the organizational units of performance groups 4 and 5 showed an average improvement of 0.06 (0.12) and 0.02 (0.12), respectively.
Mean difference in average score on all indicators and the 95% confidence interval.
Change scores of the indicators based on the interview questionnaire changed to a greater extent than those of the other two questionnaires: all scores between the performance groups showed a significant difference (p < 0.001) and the effect was large (ω2 = 0.16). Regarding the questionnaire for representatives there was a significant difference between the performance groups (p < 0.001), which was a medium effect (ω2 = 0.05). Performance groups 1 and 2 differed from all the other performance groups. Performance groups 3, 4 and 5 did not differ from each other (p > 0.05). For the questionnaire for assisted-living clients there was a significant difference between all four performance groups (p < 0.001), which was a large effect (ω2 = 0.15). Performance group 5 was excluded from the analysis, because there was only one indicator in group 5 at t0.
Combining change scores with the scores at the first measurement (t0
) provides insight into the improvement potential per indicator. To illustrate this, Figure shows the scores at t0
together with the change scores (t1
) for the indicators based on the interview questionnaire. The same information for the indicators can be constructed for the other two questionnaires (presented in Additional file 1
), but the main findings of both figures are described. The three highest indicator scores at the second measurement (t1
) were for indicators 2_4 'Experience of safety' (3.78), 2_3 'Housing and privacy' (3.76) and 5_1 'Professionalism and safety care giving' (3.55). The three highest scores of the questionnaire for representatives were for the indicators 2_3 'Housing and privacy' (3.58), 6_1 'Care plan and evaluating' (3.56), and 7_12 'Physical restrains' (3.51). Finally, the three highest scores for the assisted-living clients were for the indicators 5_3 'Reliability caregivers' (3.69), 5_2 'Respectful treatment' (3.61), and 6_1 'Care plan and evaluation' (3.57).
Per indicator, the average score of the first measurement (t0) and the change score (t1 - t0) for the face-to-face interview questionnaire for residents.
Although indicator 6_2 'Shared decision making' changed the most between the t0 and t1, the score at t1 remained the lowest (2.98) of the interview questionnaire. Indicators 6_3 'Information', 1_2 'Meals', and 6_6 'Availability staff' are ones that can make the most improvement in a future measurement (2.99, 3.04, 3.07, respectively), also because their 5th and 95th percentile range shows differences between the care homes (> 0.75 range score).
Indicator 2_4 ('Experience of safety') based on questionnaires for representatives/family members barely changed between t0 and t1. The score on indicator 6_2 'Shared decision' increased considerably, but is still one of the lowest indicator scores. Indicators that scored high at t0 and remained high at t1 were indicators 2_3 'Housing and privacy', 6_1 'Care plan and evaluation', 7_11 'Physical restrains', and 1_2 'Meals'.
Overall, the indicator scores of the questionnaire for assisted-living clients showed only minor changes (0.01-0.09). An exception to this was the indicator 6_2 ('Shared decision making'), which changed the most (0.25) but still had one of the lowest scores (2.94). Indicators scoring low at t0 and remaining low were the indicators 3_1 'Daily activities and participation' (2.84) and 6_5 'Coherence in care' (3.08).