The results of this study strongly indicate that the hypothesised relationships between structure, process, and outcome exist in the context of quality systems.
Structure characteristics, such as available time and staff with quality improvement competence, seem to be strongly related to other aspects of quality systems. Likewise do the existence of a current quality manual with documented routines and task responsibilities, or highly available administrative support, such as secretaries.
Structure seems to be related to process characteristics, such as support from colleagues, in the forms of acceptance towards quality improvement or through active participation in projects.
Structure also seems to be related to outcome characteristics, such as clear and unambiguous goals for the quality system, periodical evaluations of the goals, documentation of the results of the evaluations and feedback of the results to the staff.
Given structure, process also related with outcome. This could indicate that even though structure aspects such as resources and administration are important, work to improve process aspects could further improve outcome. For instance, work to increase support from colleagues could increase the probability that quality efforts get systematically evaluated.
The model suggests that, for instance, if there is enough time to work with quality improvement (structure), there is more support from colleagues (process), and improvements are also evaluated to higher degree (outcome).
A systematic and evidence-based approach to quality improvement may increase the chance of effective and efficient use of resources [26
For instance, resources and administration (structure) could be improved by implementing guidelines for quality improvement. Studies show that clinical guidelines, based on evidence rather than opinion, have the potential to promote interventions of proved benefit and discourage ineffective practices [29
]. Guidelines for quality improvement might have similar effects.
Moreover, culture and co-operation (process) could be enhanced by rewarding good examples and by refraining from punishment when mistakes are reported[33
]. Co-operative ability could be increased by teamwork training.
Last, evaluation of goal achievement and development of competence (outcome) could benefit from rapid feedback on which measures are effective and which are not[35
The methods used in this paper could also demonstrate how theoretical models can be analysed quantitatively to complement the numerous qualitative studies that exist within the field of health care quality research. For instance, it could be used to analyse the consistency of questions in quality referentials.
However, this study did not show, and could not show due to its design, any links between quality systems and better health outcomes, since no health outcomes were measured.
In further studies, it would be interesting to investigate the implementation of quality systems and relate these processes to departments' history of quality management and overall organisational structure of hospitals. It could also be interesting to evaluate the achievement of specific quality goals because some goals are probably more easily achieved than others.
Although the questionnaire questions may seem general, they were directly developed from the interview study. The CFA showed that most of them clearly reflected their intended factors as indicated by significant factor loadings (sensitivity). The factors could also be clearly separated from each other as shown by factor correlations significantly below 1 (specificity). Each factor was represented by at least three variables, which is considered sufficient to effectively represent most factors[25
]. Finally, the good model fit indices also indicated that the questions included in the model were adequate.
Some of the questions were about potentially sensitive matters, such as B3 which was about attitudes to incident reporting. However, these questions were removed from the analyses since the CFA indicated that they were not adequate reflections of their factors. Thus, the potential subjectivity or sensitivity of these questions could not have affected the results of the SEM.
A minimum sample size of 200 is recommended for doing CFA or SEM if the proposed model is not overly complex and if a maximum likelihood type of estimation is used[24
] Thus, the sample size of 386 was deemed adequate since it was 1.9 times the minimum recommendation and since the analysed models were not complex. Moreover, the particular type of estimation chosen for the analyses (RMLE) is distribution independent and thus does not require data multivariate normality[25
Only the early responder and heads-of-department groups had the recommended minimum of 200 cases. Thus, it was decided not to analyse differences between groups, for instance, between departments in hospitals of different sizes. However, large differences between groups of cases would have lead to the rejection of the tested model[25
Although the response rate was high (75%), bias created by non-responders cannot be ignored as a possibility. The quality systems of non-responding clinics may have differed from the quality systems of responding clinics in such ways as to affect the probability of responding to the questionnaire. Moreover, non-responders might have had different attitudes towards the questionnaire or towards quality systems in general than responders had.
The data missing from non-responding departments could also potentially affect the robustness of the SEM estimates and the model. However, the SEM estimates and the model fit indices passed the tests with good margins according to recommended statistical standards[24
]. Furthermore, the SEM estimates and model fit indices of the early responder group did not differ from those of the whole sample. Moreover, there were no significant differences in composition between responders and non-responders with regard to hospital size or department speciality. Thus, the results appear stable and independent of the time of response.
The estimated parameters obtained by the RMLE and the WLSE did not differ significantly and did not change the conclusions of this study. Thus, the results appear stable and independent of the choice of method of estimation.
It is possible that heads of departments would answer the questions differently compared with employees without managerial positions. However, how heads of departments and quality coordinators responded did not differ significantly.
An advantage of using CFA and SEM is that these methods analyse both the significance of the relationships and the model fit. Thus, it can be justified to include even small significant relationships, such as the one between process and outcome, as long as model fit improves and they have reasonable interpretations.
In theory, the structure of quality systems affects process and outcome. Since this is a cross-sectional study it is important to be careful when discussing causal relationships. However, structure is strongly related to the other two aspects, which may suggest that it is more important.
This is to our knowledge the first large quantitative study that applies Donabedian's model to quality systems.
Formal ethical approval of this study was not needed. However, steps were naturally taken to assure compliance with general ethical principles for conducting research. Respondents were informed of the purpose of the study and that their responses would be kept confidential. Responding to the questionnaire was voluntary and no gifts or other rewards were promised responders. The research project plan was approved by the Faculty of Medicine at Uppsala University (MedFarm 2003/1313-C4:2).