Survey teams visited all 33 sampled health facilities as scheduled. In these facilities, 100 health workers performed outpatient consultations; 93 (93.0%) were interviewed, and seven left before interviews could be administered. No health worker refused to participate. Of the 93 workers interviewed, 64 treated at least one included patient; and of the seven workers missed, six treated at least one included patient. Thus, patient-level indicators of case-management quality reflected the practices of 70 health workers (64 interviewed, six missed).
Altogether, 778 patients came for initial consultations; 389 of whom were selected for inclusion. Of these 389 patients, 177 were included, 170 were missed (i.e., no surveyor was available to begin the "follow the patient" procedure), 38 refused, three withdrew after enrollment, and one patient left the facility before completing all survey steps. Although many selected patients were not surveyed (i.e., 170 missed + 38 refusals + three withdrawals + one lost, or 212 patients), most (170/212, or 80.2%) were missed patients. These missed patients were unlikely to have been much different from included patients, as they were essentially missing at random. If missed patients are excluded, the "participation rate" (i.e., the proportion of patients approached for consent who were included) was 80.8% (177 included/[177 + 38 refusals + three withdrawals + one lost]).
Health facility characteristics
Of the 33 facilities, five (15.2%) were hospitals and 28 (84.8%) were health centers. A very large number of health workers (3564) was estimated in the 57 eligible facilities (all health workers assigned to the facilities, including all departments and shifts). A median of five nurses per facility had AL training. All facilities had at least one nurse trained to use AL and RDTs.
Among the 33 sampled facilities, 1,103 patients sought care on the day of the survey visit: 778 (70.5%) initial consultations and 325 (29.5%) follow-up visits. The median numbers of total and initial consultations/facility/day were 26 (range: 12-119) and 18 (range: 5-77), respectively.
Nearly all facilities had thermometers and scales; but only half had AL algorithms (Table ). All performed malaria testing. All facilities had AL in stock, and most (72.7%) had all four AL blister packs. However, two-thirds of facilities had AL stock-outs in the preceding three months, and most lacked quinine.
Availability of equipment, staff, and medicines needed to manage malaria and other febrile illnesses in outpatient health facilities, Huambo Province, Angola
Health worker characteristics
Among the 93 interviewed health workers, the median age was 36 years (range: 21-70), and 51.6% (48/93) were female. Nearly all (91/93, or 97.8%) were nurses; two (2.2%) were physicians. Nurses had 2-5 years of pre-service medical training. Sixty percent of interviewed health workers had formal AL training, and two-thirds had informal AL training (Table ). Most formal training courses (89.3%) covered RDT use, training duration was usually three days (range: 1-15), and most (75.0%) occurred in 2007.
Health worker training and supervision in outpatient health facilities, Huambo Province, Angola
Three-quarters of health workers were supervised in the preceding six months (median = 1 visit, range: 0-5), but only one-third reported that supervision included observation and feedback on a consultation. Half of workers reported supervision on AL use; and of these, almost half (20/47, or 42.6%) reported never having received supervision with observation and feedback on a consultation. These results reflected the plan to have AL-related supervision in the first year of AL scale-up focus on pharmaceutical management.
Health worker caseloads (all consultation types combined) ranged from 1-44 patients/day (median = 13), and 16.3% (15/92 [1 missing]) had high caseloads of ≥ 25 patients/day. A knowledge assessment revealed that no health worker knew the complete description of which patients should be tested for malaria--even though workers were told they could consult reference or training materials during the interview. However, two-thirds (59/90, or 65.6%) of health workers knew fever was a criterion for testing.
Responses to case scenarios of hypothetical patients revealed several patterns. First, in three scenarios describing adults with fever and a negative test (RDT or microscopy), most health workers (72.0-81.7%) seemed to ignore the test and gave an incorrect diagnosis of malaria or suspected malaria. For nearly all (96-100%) case scenario patients that health workers diagnosed with malaria or suspected malaria, workers said they would treat with an anti-malarial. Second, in a scenario of an adult with fever, convulsions, and a positive blood smear, only two-thirds of workers (63/93, or 67.7%) correctly diagnosed severe malaria; but correctly diagnosed cases were usually (53/63, or 84.1%) treated with an injectable anti-malarial, and nearly all (87/93, or 93.5%) were referred for hospitalization. Third, in three scenarios of patients with symptoms suggestive of malaria, most workers (76.3-87.1%) correctly responded that they would test the patient. Finally, summarizing knowledge across all seven scenarios, the median percentage of questions correctly answered per health worker was 56.3% (range: 31.3-87.5%).
Patient characteristics: demographics, consultation attributes, and illnesses
Patient ages ranged from 0-80 years (median = 8), and 72 (45.0%) were under-5s. Ninety-nine (55.9%) patients were female, and five (2.2%) reported being pregnant. Half (53.9%) of patients were seen by health workers with formal AL training, and 75.3% of patients were seen by workers with any AL training (formal or informal). Only 30.8% of patients sought care on the day of illness onset or the next day.
The chief complaint of half (49.0%) of patients was fever or malaria, and 119 (70.5%) had a febrile illness (fever by history or temperature > 37.5°C). Among these 119 patients, only 69.5% gave a chief complaint of fever or malaria (85.8% for under-5s and 54.9% for patients ≥ 5 years old). These results show why health workers must ask about fever and measure temperatures for all patients; solely relying on chief complaints could lead to many missed cases of febrile illness.
As defined in Huambo, 136 (77.8%) patients had suspected malaria (fever or three non-fever symptoms), half (48.0%) of whom had signs of a non-malaria illness--most commonly, respiratory infections (Table ). Of 62 under-5s with suspected malaria, 58 (93.5% [unweighted]) would have been detected by fever history or measured temperature > 37.5°C alone. That is, if suspected malaria had been defined as "fever history or temperature > 37.5°C", the inclusion of "three non-fever symptoms" added little benefit (only four more patients, an additional 6.9% [4/58, unweighted]). For older patients, the corresponding benefit was somewhat larger: an additional 21.3% (13/61, unweighted).
Patient characteristics in outpatient health facilities, Huambo Province, Angola
According to the analysis algorithm, one patient (0.8%) had complicated malaria, 58 (35.0%) had uncomplicated malaria, and 118 (64.2%) had no malaria (Table ). Proportions were similar for under-5s and older patients. Of the 59 malaria cases, 17 had a positive malaria test (based on testing performed by the observed health facility staff--not by surveyors); and the other 42 had suspected malaria, were not tested, and had no other (non-malaria) cause of fever identified (see Figure , footnote 7).
Patient characteristics: quality of clinical assessment and use of diagnostic testing
Clinical assessments were evaluated by estimating the proportion of patients for whom health workers had determined if a given sign or symptom was present. "Determined" meant the worker was exposed to the information by any means (e.g., spontaneously offered by the patient, provided by the patient in response to a question, or obviously evident). This approach avoids penalizing health workers who do not ask for symptoms when it is not necessary. Fever history was determined for 87.6% of patients. However, temperatures were measured in only 25.9% of consultations, and assessment quality was poor for all other symptoms needed to identify suspected malaria. Symptom-specific proportions ranged from 1.8% (fatigue) to 39.2% (headache). A sub-analysis for patients without fever showed similarly poor quality (details available on-line) (see Additional file 1
The pre-September 2007 policy recommended testing all patients with suspected malaria with microscopy or RDTs. Thus, 136 (77.8%) patients needed testing (Table ). Only 30.7% of patients needing testing were tested, 79.2% of patients not needing testing were not tested (i.e., little over-testing), and overall adherence to the policy was 41.5%. Results did not vary by age.
Univariate statistical modeling identified several factors that were positively associated with malaria testing among patients needing testing: increasing supervision on AL use, lower caseload (< 25 versus ≥ 25 patients), higher patient temperature, and facility type (health centers versus hospitals) (details available on-line) (see Additional file 1
). Two factors not associated with testing had p-values low enough (p < 0.15) to retain in the multivariable model: any AL training (formal or informal) and health worker age. The following were not associated with testing and not retained in the model: formal AL training, days of AL training, health worker knowledge score and sex, chief complaint of fever or malaria, and patient age and sex.
Multivariable modeling identified two factors significantly associated with testing (Table ). First, the odds of testing among patients (needing testing) seen by health workers with caseloads < 25 patients/day were 18-fold greater than for those seen by workers with higher caseloads. Based on unadjusted results, the proportion of patients tested by health workers with lower and higher caseloads were 49.0% and 7.5%, respectively--a large difference of 41.5 percentage points. Second, the odds of testing increased by 2.5-fold for each increase in patient temperature by 1°C. Based on predicted probabilities from the reduced model, for each 1°C increase in temperature (in the 37-39°C range), the proportion of patients tested increased by about 13-22 percentage points. The multivariable model also revealed that the association between testing and any AL training (formal or informal) was of borderline statistical significance (odds ratio = 5.4; p = 0.072). Based on unadjusted results, the proportion of patients tested by health workers with any AL training and no AL training were 38.1% and 17.2%, respectively--a moderate difference of 20.9 percentage points.
Predictorsa of correct malaria testing among patients with suspected malariab in outpatient health facilities, Huambo Province, Angola
Patient characteristics: results of malaria testing and quality of diagnosis
Among the 69 non-pregnant patients ≥ 5 years old with suspected malaria tested by surveyors with microscopy, 2 (3.4%; 95% CI: 0-8.5) were parasitemic with P. falciparum. Health workers tested 64 patients, 62 of whom had results available the same day and were included in subsequent analyses. Of these 62 patients, 50 were tested only with RDTs, 9 with smears only, and 3 with both an RDT and smear. Seventeen (27.4%) of these 62 patients tested positive, and results were similar for under-5s (8/30, or 26.7%) and patients ≥ 5 years old (9/32, or 28.1%).
Among the 27 patients tested by surveyors with microscopy (the gold standard for evaluating diagnostics) and by health workers (with RDT or microscopy), the sensitivity of health worker testing was 2/2, and the specificity was 19/25 (76.0%, unweighted). Results for health worker RDTs only were similar (sensitivity = 2/2, specificity= 17/22, or 77.3%, unweighted).
With the analysis algorithm as the standard, 66.1% of health workers' malaria-related diagnoses were correct, 20.1% were minor errors, and 13.9% were major (potentially life-threatening) errors (Table ). With survey microscopy as the standard: a) for the one microscopy-positive case of uncomplicated malaria, the health worker's diagnosis was uncomplicated malaria (correct); b) for the one microscopy-positive case of complicated malaria, the worker's diagnosis was uncomplicated malaria (major error); and c) for the 70 microscopy-negative cases, workers correctly diagnosed no malaria in 34 cases (48.6%, unweighted) and incorrectly over-diagnosed malaria in 36 cases (51.4%, unweighted).
Quality of malaria diagnosis in outpatient health facilities, Huambo Province, Angola
Twenty-two staff from a convenience sample of 20 facilities were assessed as they performed RDTs. All the correct steps (one drop of blood and six drops of buffer solution) were followed in only half (11/22) of observations. The most common error was using too little buffer solution (two to five drops). The amount of blood was usually correct. Most staff knew that one should wait 15 minutes or a little longer, although one-third might have thought that waiting times were < 15 minutes or were unsure. Additionally, in some facilities, test results were reported in batches instead of reporting individual results as they were ready. This practice caused unnecessarily long waiting times (≥ 1 hour) for some patients (RDT results should be available in < 30 minutes).
Patient characteristics: quality of treatment and counseling
Among all 177 patients, 61.4% of prescribed malaria-related treatments were correct, 22.3% were minor errors, and 16.3% were major (potentially life-threatening) errors (Table ). The most common errors were prescribing no anti-malarials for patients with uncomplicated malaria and prescribing AL for patients without malaria. Errors such as under-dosing AL and using ineffective anti-malarials were uncommon.
Quality of malariaa treatment in outpatient health facilities, Huambo Province, Angola
Among the 59 patients with malaria, the quality of prescribed treatments was lower: only 49.0% were correct, 5.4% were minor errors, and 45.6% were major errors (Table ). In an analysis of treatment quality from the patient's perspective (i.e., patient left the facility with the anti-malarial in hand and demonstrated knowledge on administering it at home), only 27.1% of patients received recommended care, 5.4% received adequate (but not recommended) care, and 67.5% received inadequate care.
An analysis of 62 patients prescribed AL (whether or not it was indicated) revealed that health workers almost always dosed AL correctly, but the quality of counseling was mixed (Table ). Nearly all patients received the correct dose (95.1%) and complete dosing instructions (88.2%). However, only 60.9% of patients could repeat all the dosing instructions. Furthermore, few patients were given the first dose during the consultation (10.7%) or advised to take the medicine with food (31.3%).
Use of AL: frequency of prescription, and appropriateness of dosing and counseling (whether or not AL was indicated, according to guidelines) in outpatient health facilities, Huambo Province, Angola
Graphical pathway analysis
A graphical pathway analysis was performed to link results of individual steps of the case-management process and identify strengths and weaknesses of health worker practices. For simplicity, percentages are unweighted. Among the 40 patients without febrile illness/suspected malaria (and thus no malaria), we found that a large majority of patients (35/40, or 87.5%) were not tested for malaria, nearly all (34/35, or 97.1%) of these untested patients were not diagnosed with malaria; and of the 34 patients without a malaria diagnosis, none were treated with an anti-malarial (Figure , steps along the bottom of the figure). In other words, most patients were managed correctly at all points in the case-management process.
Graphical pathway analysis of the case-management process for 40 patients without suspected malaria. AL = artemether-lumefantrine; HW = health worker.
Among the 78 patients with febrile illness/suspected malaria but no gold standard malaria diagnosis, many (36/78, or 46.2%) were not tested, even though they should have been (Figure ). Among the 41 patients with a negative test result, over half (24/41, or 58.5%) were diagnosed with malaria. Nearly all (33/36, or 91.7%) patients diagnosed with malaria were prescribed an effective anti-malarial; and of the 42 patients without a malaria diagnosis, none were treated with anti-malarials.
Graphical pathway analysis of the case-management process for 78 patients with suspected malaria but no gold standard malaria diagnosis. AL = artemether-lumefantrine; HW = health worker.
Among the 59 patients with febrile illness/suspected malaria and a gold standard malaria diagnosis, most (42/58, or 72.4%) were not tested, even though they should have been (Figure ). Of 26 patients who did not receive anti-malarials (major error), none had been tested. All 17 patients with a positive malaria test were diagnosed with malaria. A large majority of patients (31/35, or 88.6%) diagnosed with malaria were prescribed effective anti-malarials; and of the 24 patients without a malaria diagnosis, none were treated with anti-malarials.
Figure 6 Graphical pathway analysis of the case-management process for 59 patients with a gold standard malaria diagnosis. AL = artemether-lumefantrine; HW = health worker. a None of these 26 patients had been tested for malaria, although all were seen at health (more ...)
In summary, many patients who should have been tested were not, which led to many incorrect diagnoses. Health workers did not trust their negative test results: over half of test-negative patients were diagnosed with malaria (26/45, or 57.8%) and treated with anti-malarials (27/45, or 60.0%). Results were similar for microscopy and RDTs, but distrust was greater when the patient was ≥ 5 years old. Malaria was diagnosed in 73.9% (17/23) of test-negative patients ≥ 5 years old (versus 40.9% [9/22] for under-5s), and anti-malarials were prescribed for 73.9% (17/23) of test-negative patients ≥ 5 years old (versus 45.5% [10/22] for under-5s). Health workers did trust positive test results, as all test-positive patients were diagnosed with malaria. Prescribed treatments closely matched health worker diagnoses: among 73 patients whom workers diagnosed with uncomplicated malaria, a large majority (83.6%) received AL (usually correctly dosed); and among 102 patients without a diagnosis of malaria, nearly all (97.3%) received no anti-malarial treatment.
Design effects and intraclass correlation coefficients
Design effects and intraclass correlation coefficients (ρ) were examined for 24 key patient-level indicators of case-management quality related to patient assessment, diagnosis, treatment, and counseling. Weighted design effects ranged from about 1.0 (no correlation) to 3.4 (moderate correlation); the median was 1.7. Weighted values of ρ ranged from just under zero to about 1.0; the median was 0.3. Design effects for the two indicators of correct malaria treatment were close to one (i.e., 1.1 and 1.6), which reflects relatively little correlation.
As heterogeneous analysis weights tend to increase design effects and decrease precision [19
], an unweighted analysis was performed to quantify the effect of heterogeneous weights (final weights ranged from 1.7-40.6). As expected, unweighted design effects and ρ
were usually lower than values from the weighted analysis. An examination of the ratio of weighted design effects to unweighted design effects revealed a median ratio of 1.48. In other words, heterogeneous weights typically increased design effects by 48%, which widened CIs by 22% (i.e., square-root [1.48] = 1.22) (details available on-line) (see Additional file 1