Part one: delivering the intervention
Identified hospitals were randomly allocated [1
] to two groups of four hospitals at the start of the study. Identical baseline surveys evaluating hospital care within the classical Donabedian framework of structure, process, and outcome [9
] were then conducted between 9 July and 19 August 2006 [2
]. During these baseline surveys, training was arranged with the administrators of both intervention and control hospitals. We have previously described in detail the training (ETAT+) provided to intervention hospitals [10
]. In brief, however, a five and one-half day course was provided incorporating one and one-half days of lecture material combined with three days of small-group, interactive, practical sessions based largely on clinical scenarios and including skills training provided by at least four trained facilitators/instructors. The course also included reflective exercises – a walkabout review of current practice and audit – and end of course, individual testing of participants. Use of standard paediatric admission records (PARs) and CPGs was an integral part of this practical training. We were able train 32 staff from each hospital, of all cadres, hoping to work with the hospital to concentrate on those staff providing services where sick children or newborns are commonly encountered (see Table ).
Summary of training provided to study hospitals at the start of the intervention and, for intervention hospitals, during the 18 months intervention period.
In control hospitals, only the lectures were provided in the form of a one and one-half day seminar aimed at an audience of 40 to 45 health workers providing paediatric services in the hospital. After the training in both intervention and control sites, hospitals were given copies of the Ministry of Health's CPG booklet http://www.health.go.ke
, copies of wall charts containing the same material, and four copies of three basic reference texts [11
] for paediatric areas in the hospital. At the conclusion of the training seminar, a 60-minute presentation and discussion of the results of the baseline survey were given, and detailed, printed reports of the survey findings were provided to each senior administrator and department head. The hospitals' administration, all seminar participants, and all staff providing data during the baseline survey were aware that follow-up surveys were planned approximately every six months for 18 months. All training was conducted between 16 September and 2 November 2006, with participation summarized in Table .
Ongoing training using elements of the same ETAT+ materials
In addition to the initial training, the implementing team (ME, GI and SN) provided intermittent training while conducting supervisory visits (Tables and ). These were largely conducted as forms of continuous medical education (CME) aimed, if possible, at times when clinical interns rotated. These very occasionally took the form of short local seminars lasting a maximum of one and one-half days and requiring at most two trained instructors. However, in most instances ongoing training was conducted in sessions lasting one to three hours. Within hospitals, staff were also encouraged to organize, by themselves, ongoing CME sessions of approximately 30 to 60 minutes using original ETAT+ training materials given to the hospital at the end of the course.
Summary of major activities undertaken by the supervisory team with time measured in weeks from the onset of the first intervention hospital training. Control site surveys were undertaken in parallel with those illustrated for the intervention sites
Supervision and feedback
Each intervention hospital was linked to lead researchers (H1 and H3, SN and ME: H2 and H4, GI and ME). The aim was for these researchers to try and play a role approximating that of a regional supervisor tasked with implementing government guidelines and improving paediatric hospital care (for timing of these visits, see Table ). Control hospitals did not receive this supervision and only received written feedback after surveys. As well as the ongoing training aspects outlined above, this role relied on two to three monthly personal visits and involved:
Intermittent face-to-face discussions with the hospital administration
These focused on the progress in implementation of guidelines and improving care and local strategies for solving problems in the provision of effective care. These aspects were particularly addressed when providing feedback that often involved a small group discussion with senior hospital staff during the survey to promote immediate problem solving; this was followed six to eight weeks later by a more formal presentation, open to a wider group of senior and other hospital staff, at which written reports (n = 20) were distributed within the site.
An intermittent but visible presence in the hospital demonstrated that an interest was being taken in the hospital's progress. This involved personal visits to each department, informal discussions with staff members on duty, bedside clinical case discussions where the use of the guidelines could be promoted, and observation and discussion of practice and organization of care.
At the start of the project, the hospitals were asked to select from among their own staff a facilitator who was either a nurse (three hospitals) or a CO (one hospital). To ensure that this person was available, the hospitals were supported to release their nominee from full-time duties in return for 18 months of locum funding to cover their routine duties. As part of their preparation, the facilitators received three days of training, together with the research team, aimed at building their skills in: characterizing and defining problems; defining barriers to good practice; achievable goal setting; communication skills; negotiation skills; building partnerships; and managing groups and small meetings. Facilitators also received ETAT+ training outside their hospital before the start of the intervention and a second time with their hospital colleagues so that they were completely familiar with the guidelines and job aides, and able to provide support to hospital staff who had not received formal training. To support the facilitator, one of the supervisors (GI, ME and SN) contacted the facilitator every one to two weeks by telephone to provide encouragement and advice and help identify goals, priorities, and strategies for their work. The facilitators received no financial incentives and remained Ministry of Health employees. The major roles undertaken by facilitators, identified from the major themes in telephone follow-up logs, were remarkably consistent across the four intervention sites and are outlined in Appendix 1.
Part two: Health workers' perceptions on the nature of feedback and supervision provided during the intervention
Preferences for and response to feedback
In total, 84 health workers across the eight hospitals contributed data (see Table ). A number of mechanisms for providing feedback were tried over 18 months in the intervention hospitals by the implementation team. It appeared that staff preferred, in order: power point presentations to an open meeting for all staff; feedback incorporated into CME; written reports; summary sheets; and finally, local performance charts. Power point presentations and CME were favored, according to the health workers, because they were more interactive, less personalized, and provided a forum where all types of health worker and all the pediatric departments met. Additionally, these interactive sessions, which included the hospital administration, increased their involvement in guideline implementation. Written reports were said only to be available to the senior staff of the hospital, and although summary sheets and performance 'run' charts produced by the facilitator were available in all pediatric departments, these were reported to raise little interest among staff, some of whom also found interpreting them difficult:
'I think it [feedback] is good because when you present to people as a multivariate group of people, you do not present to individuals, it's the hospital. So it's not personalized, I think it's a good way of showing us the weaknesses, the good points because we are a mixed lot. Now if you were giving an individualized thing, someone would feel really intimidated (laughs).'
'The performance charts on the walls done by [Facilitator] are a good way of presenting information but I wonder whether everybody in our ward know what they are reflecting, or what they mean, there is a day I tried studying one but ... and [Facilitator] does these charts in the Paeds ward, the MCH, and the OPD, and he does it so well, and when they come out he replaces them, but you find that us, the people he puts them up for, never read them.'
There was a general consensus that the feedback information was accurate, with health workers describing the first feedback after the baseline survey as the only predominantly negative feedback delivered by the study team. There was a subtle preference for receiving feedback from the external study team rather than the local hospital staff or the facilitator, with reports of better turnout and greater credibility with the study team, although some doubted that feedback would achieve anything:
'At first when they came [study team feedback], the figures were a bit low and we were demotivated that we were not doing well, and we knew we had to work and improve things and we gained so much from the training to improve things.'
' [Feedback is] very good and very eye opening. Actually, these feedbacks have helped us identify gaps which without KEMRI [Kenya Medical Research Institute] we would not have been able to identify. So we have been using this feedback and I hope we will continue to use them to address positively these gaps that have been identified and continue to work with the KEMRI team.'
Q: 'Do you think the feedback that KEMRI has been given here has had any impact on the health workers here?'
A: 'I tend to think that it is halfway known. They take very little interest and they tend to think that these are things concerning the administration and [the facilitator] will implement after all, so what is commented on that feedback, very few will come back to check what went wrong – very few.'
Recognition and encouragement of good performance were reported during feedback meetings to be most critical to the health worker, as well as associated improvements in provision of resources and equipment by the hospital administration. Thus, health workers positively associated feedback information with improved pediatric practice attributed to improved motivation to do the correct thing, the provision of reminders, and increasing positive outcome expectancy. Interestingly, in one intervention hospital, locally generated feedback on progress was incorporated into regular hospital management team meetings, and in another initiated in-house client exit surveys:
'It [feedback]' has been very much useful ... when they come and then they check the emergency tray, and then maybe there are some drugs missing like let's say Phenobarb [a drug used for treating convulsions], they will then push the pharmacy to buy the drug because they have come for the supervisory visit. So, the administration will be told that you have such and such drugs missing because you know you may be missing something and you are not aware. Like we were missing a sucker in MCH the last time they came and they brought it up in the feedback then we chased for one and we got it. So these visits are really useful, because they push the administration to provide things that are not there, and we are very happy.'
Experience of supervision
Health workers' descriptions of their experience of supportive supervision from the study team could be characterized as guided, experiential learning with provision of open, evaluative information on how to improve care provided to children through the use of guidelines. However, the impact of supervision and feedback was felt to be strongly dependent on individual health workers' appetite for and willingness to change. Direct clinical supervision of patient care by the study team was received with mixed feelings, however, with interns and new staff welcoming the learning opportunity while some health workers felt that the team came to scrutinize mistakes. Interestingly, health workers preferred the study team to help perform some of their clinical duties as a show of support and a better acknowledgement of their responsibilities:
'They were just giving what they found on the ground, and as I said, they were supportive and facilitative, they give the feedback the way they found on the ground and support the team. Where the team was doing well, they would praise them and encourage them on the parts that were missing, and where things were done poorly, they were brain-storming together with the team. They would find out why such a thing was happening and what action should be taken, and normally it was the team that was suggesting how to solve the problem, they were never telling the team what to do, they would just suggest what to do, so they were like counselors.'
'I don't know .... if in your [supervisory] team you have nurse and doctors, then they should be coming and working with us, not just ... so that they know how we are doing. If there is a nurse, let her come with us, we do that midwifery, we deliver, we resuscitate that baby, we see how it goes. But the way you come, it's like looking for mistakes ... to be in our shoes, to know how things are. ... But if you helped, we will not feel like you were wasting our time, but that you were with us and then may be in the end you can even make ... you will have seen how I was working. Like yesterday I heard the doctor saying 'they are always coming here, wasting our time' yet he is busy wanting to do something.'
In control hospitals, health workers continued to report the lack of local supervision and feedback well over a year after the implementation of the guidelines. Where hospital supervision was reported in control hospitals or intervention hospitals prior to the intervention it was characterized as infrequent, haphazard, and in the form of vague departmental visits by the senior staff and the department in-charges. There was no real attempt at internal performance evaluation and feedback.
Health workers' perceptions about the role and practice of local facilitation in intervention hospitals
Generally, health workers regarded the facilitators positively and their observations of the facilitator's role were closely associated with those identified by the implementing team (Appendix 1).
(Facilitator): 'my roles are like ... drawing those graphs, giving them feedback reports, CMEs, helping them with some procedures, like doing intra-osseous, then when there are no resources, colluding with the office, the stores, the pharmacist, then see what to do like negotiating with them to do the purchasing.'
The facilitators managed to be guiding and supportive without provoking negative emotions amongst colleagues in all but a few situations that were slowly resolved. Health workers described facilitators as role models, peer educators, a reminder to use the guidelines, in some cases as friendly supervisors and as a link between the health workers and the hospital administration:
'Hey, he [facilitator] is very helpful. You know, he is a link between us and the administration in case there are shortages in terms of supplies; he makes sure we get them or any other problems we are facing. Again, he is always there on the forefront sensitizing people when it comes to ETAT even when you see that people are not willing, and then he is also there to arrange for CME's.'
' [Facilitator] is ... a tank of support and he ... was my conscience when I was working in pediatrics ... because may be there were times when I would be tired ..., maybe I [had] just finished a ward round and I just want to run away ... but then he would remind me.'
However, some clinicians expressed their dissatisfaction that a nurse as a facilitator might influence clinical management decisions, illustrating the somewhat rigid thinking about the hierarchy of roles seen in Kenyan hospital care. Interestingly, although they were regarded as leaders in the implementation of the programme, there was also a prevalent perception that their main work was as data collectors for the study team. Linked to this there was a misplaced perception that the four facilitators must have been receiving a financial incentive that explained their enthusiasm for their role.
'Well, I guess he's actually doing what he ... what he's supposed to do or what he can actually do within his jurisdiction, but I think it would have been more effective if it was a clinician rather than a nursing staff ... you get ... so that you're part and parcel of the ward round and you're part and parcel of making the decisions...'
(Facilitator): '...in fact there is someone who was saying, ' [facilitator] is getting 60,000 from KEMRI per month, on top of his salary, wacha akuje afanye kazi (let him come and work).' Imagine that situation where people do not even want to see you.'
The facilitators, in describing their experience in the implementation of guidelines, characterized it as: emotionally taxing, hectic, and requiring considerable patience and persistence both with the administration and the staff:
(Facilitator): 'But at the same time, its hectic, there is a lot of headache as a facilitator. At times, you might tell someone that this one is supposed to be done this way, then you find that person repeating the same mistake you corrected, you have to swallow your anger and start afresh. So, that process of training and reminding people on the same things everyday, and at times some people are just slow, you just have to adjust and accept them the way they are. So at times you want to get annoyed but you have to cover that annoyance and you don't want to show anyone that you are annoyed, sometimes you wonder whether may be you are the one who is not handling them the right way.'
The most challenging experiences, the facilitators reported, were in the OPD that predominantly serves adults while providing services to sick children at nights and weekends, and with the COs. These departments and individuals were reported to embrace change the least well while the pediatric wards were felt to have shown the best improvement.
(Facilitator): 'For me, I think people believe that children should be seen separately from the adults so the children landing in OPD during odd hours are not getting the proper care, it's just negligence, because sometimes a clinician will say, 'me, I don't want to see children'.'
Success stories described by the facilitators that illustrate their role to promote change included: having enabled networking within hospitals; developing a role as team builders and team players; building collaborative relationships with the administration; and, more importantly, a sense that they were contributing to a reduction in child mortality and morbidity in their hospitals.
(Facilitator): '(sighs) it has come with a lot of things. One thing, it has taught me how to network with people, that one is for sure. This programme has made me be a team builder. Before, I just used to make sure that everything that I do, I do it right; but when I became a facilitator, it dawned on me that I have to make the other person do it perfectly. So it has made me be a team player to ensure that other people do it right. So I came from being an individual to interacting with the other people to talking to the clinicians, talking to the other nurses, getting very close to the administration especially, getting things done.'
Among all the facilitators, there was a general consensus that facilitation will have to be maintained permanently for sustainable implementation in the different hospitals.
(Facilitator): 'Sustainability really depends on who is on the ground. I think, as for me it is still my responsibility to maintain ETAT.'