Step 1. Define performance measures
The variation between scenarios was reflected in the variation of the cost per screen detected child which was calculated for each scenario. A societal perspective was used for the analysis which means that costs for all potential payers are considered.
Costs for diagnosis, referral, treatment of dysplasia of the hip, and time costs for the parents were included. The relevant clinical outcome was defined as a child detected and treated with treatment occurring within a year after diagnosis (true positives of screening). In essence, we summed and compared all the costs incurred for the screening and treatment according to the various scenarios and divided these by additional number of children detected and treated. We also included the cost of the missed cases and false positives.
Step 2 and 3. Analyze the performance of the existing workflow: 'Current and new strategy for detecting DDH'
In the Netherlands the current strategy for detecting children with DDH consists of history taking and physical examination during routine consultation at the infant health care centers (IHC) organized by youth health care organizations. Parents visit the IHC at regular intervals and consultation either the nurse or the physician. At the age of 1, 3, 6, 9 and 14 months the consultation with the physician comprises identification of risk factors and a physical examination of the hips. In case of a positive test result, the child is referred to a general practioner (GP). The GP will repeat the tests performed by the IHC physician and will generally refer the child to the radiology department or an orthopedic surgeon. In an outpatient setting an ultrasound (US) and/or X-ray and an orthopedic examination are performed to determine whether DDH is present. In almost 84% of the children referred the presence of DDH is refuted based on these specialist exams[7
The new screening strategy requires that US examinations of the infants' hips are performed on the hips at the age of three months. According to the principles of Graf hips can be classified in six classes (type 1, 2b, 2c, D, 3 and 4) where type 1 is normal and type 4 complete luxation of the hip[13
]. IHC-nurses, IHC physicians and radiographic technicians (RT) who work at the IHC may potentially qualify as screeners. These disciplines were trained to perform the examinations. However, the selection of disciplines performing the US examination may influence the cost-effectiveness in two ways. If physicians should appear better at performing the examinations this implies increased costs in terms of wages but on the other hand this might imply higher effectiveness. Also, at an organizational level the cost-effectiveness attained may be influenced by the available capacity of screening personnel. This also applies to actual rooms and equipment required to perform the US examinations, training capacity, planning etc.
Regardless of the screening strategy there will be children who have a higher risk of DDH. In the current screening, all children whose parents during history taking state that one of the known risk factors is present are sent to their GP and/or an orthopedic surgeon regardless of the results of the physical examination. In the situation where US screening is implemented, we presumed that only those with a positive screening result would be referred to a medical specialist, i.e., instead of acting on possible risk factors being present. In table the values in terms of missed cases, false positives and current screening and US-screening are given.
Performance measures of the current screening and the US-screening at the age of three months.
Step 4 and 5. Brainstorm about improvement, define alternatives
By exploring changes in cost-effectiveness ratios we could actually explore whether alternatives values of the experimental variables (input) might be defined in such a way that these remained plausible. The next step was defining alternative implementation scenarios. We used the ideas and comments which had been stated in focus group interviews among stakeholders such as policymakers, managers in the IHC and screeners (physicians, nurses and radiographic technicians). The choices for these groups were based on the evaluation of the Soundchec 2 project team. For each of these groups of professionals separate interviews took place. In each group four to eight persons participated. The purpose for the interviews was to discuss possible limitations and opportunities for the implementation of ultrasound screening. These interviews provided important insight regarding the definition of the experimental variables, and also for performance measures. The complete article describing the focus group interviews will be published by the project team at a later phase.
Important performance measures defined were the attendance rate and the quality of screening A high attendance is critical for effective screening[14
]. This performance measure will provide important information for the policy makers and the IHCs. Attendance in rural areas may be different from that in urban areas[15
]. In rural areas people need to travel more because of the spread of IHCs, which likely influences attendance. On the other hand, in urban areas more people who have different cultural background reside, which may again influence the attendance rate. Another important aspect of screening which can influence the attendance rate is the organizer of the screening. The interviewees suggested that screening should be planned and conducted by an easy access organization which is familiar to parents. The quality of the screening needs to be compared with the current screening strategy. In a previous study (Soundchec 1) the rates of missed cases, treated and false positives were established[7
]. Quality of screening is an important outcome measure since missed cases and false positives will influence the cost-effectiveness.
Ideas that were brought forward for defining the experimental variables were:
- availability of US-machines at every IHC
- implementation of the screening outside the IHC- organizations
- consultation in evening hours
- implementation of the screening combined with routine three month consultation, this would imply at every IHC an US machine
- guaranteed high quality screening
- premise to call it screening is a high attendance, which may be achieved through organization of the screening close to the residence of parents
- influence of travel distance
- no purchase of US machines, but make use of available US machines in other settings, e.g. in obstetric centers.
From these global ideas four experimental variables were derived which were considered to be pivotal for the organization of ultrasound screening at IHCs. These variables were subsequently used for experimentation (table ).
Based on these experimental variables, 72 possible scenarios can be drawn up. With the following assumptions, all scenarios were run in a simulation model:
1. If the screeners are employees currently working in the IHC and/or the location of the screening is in buildings owned by the IHC, the screening is organized by the IHC.
2. An integrated consultation implies a regular three month consultation comprising an US examination. When the screener is a radiographic technician or a medical specialist, the infant health care physicians have to be present at the location of screening to perform the regular IHC consultation.
3. An integrated consultation with an infant health care nurse as screener, means that infant health care physicians will delegate their tasks in the regular three month consultation to the infant health care nurse. Delegation (substitution) of tasks under specified conditions is increasingly applied in IHC [16
Step 6. Experimentation with alternatives
The simulation model was build in ARENA[11
]. Each scenario was evaluated using sequences where patients go to different stations defined by the sequence.
To model each of the 72 scenarios the levels for each of the experimental variables were set. The levels were set in accordance with expert opinion and data gathered from IHC organizations. Besides the levels of the experimental variables, the attendance rate and the quality of screening examinations were quantified. This was achieved using data available from IHC-organizations (attendance rate and cost price) and expert opinion (quality of screener). Other information needed to run the model was based on the reports of the Soundchec 1 study [7
], Dutch guideline for cost-effectiveness studies [18
], IHC-organization reports [15
]and expert opinion.
The population simulated repeatedly comprised 2300 children expected to visit the IHC at regular time intervals over a period of 18 months. This figure corresponds with our ongoing implementation study. We excluded the children who were referred and treated in their first three months of life.
Quality and attendance rate
The quality of the screener is defined in terms of cases missed (false negatives) and false positives, both as a proportion of the total number of screened children. Since only the proportions were known for radiographic technicians (Soundchec 1 study), we set the proportions for IHC physicians, IHC nurses and medical specialists by expert opinion (Table ).
Quality of screening result for different screener type.
Attendance rate was based on three participating organizations in the screening program. One organization in a rural area and two organizations in an urban/suburban area. For the rural area we used an attendance rate of 90% and for the urban area a rate of 85%. The re-attendance rate (after a reminder) for the urban area was estimated to be 80% and 90% for the rural area.
With regard to the relevant estimates of cost items travel time is based on the Dutch guideline for CE analyses [18
]: 0.20 per kilometer by car or public transportation. The travel distance to the different locations are 4-7 kilometers for IHC and ten for external locations[15
]. Estimates of treatment costs were based on the Soundchec 1 study [7
]. Treatment costs were included for each Graf type. Also the treatment cost of current screening after one and two months were based on this study. For this period, treatment costs for Graf type 2a were included since this type is only given to infants up to two months of age. The cost of false positives in current screening consisted of one hospital visit including the cost of absence of work of the parents (€61 + €36). In the model, the salaries were adjusted when screening was done in evening hours (35% extra costs) and when an extra consultation took place (double the salary costs and travel time).
For cost of training we assumed that also medical specialists would need training since many of them do not know how to perform US-screen for DDH and the setting is different. We further assumed two day training sessions lasting eight hours for the calculation. We used the average salary of the four screener types plus (16 hours * €73) an additional amount for salary of the instructors and material (€ 130). These numbers were justified by expert opinions who give training to radiographic technicians.
For the costs of depreciation of the machine we calculated the annuity for the total period of five years with an interest rate of 5% (€ 7560)[18
]. Each machine costs € 32725, the insurance per year costs € 1000. We also included maintenance costs of 8% of € 32725 (€ 2618)[18
]. The total costs per year are therefore € 11178, which resulted in an average of € 5 per child (2300 children). In table all items are presented together with the point estimates and sources. Prices were adjusted using the price index rate (statline.cbs.nl) for the year 2006.
Cost items, source and values
Explanation quantification levels of experimental variables
A. Number of Ultrasound machines
The scenarios for the levels pertaining to the number of US machines are defined in accordance with the following assumptions:
- Many machines: average number of IHC assumed to be efficient is seven based on literature and expert opinion [15
- Limited: one US machine, which means one salary compartment and one overhead for each cycle.
- None: buildings will be used where US machines are already available, like in an obstetrics center or in a multilevel health care facility center.
For one consultation at the IHC parents incur one hour productivity loss. For an extra consultation this would mean in total two hours of productivity loss for the diagnosis and detection of DDH. Extra travel time cost was also included for an extra consultation.
For each screener type the salary was calculated for ten minutes. For each screener, the variability of the quality of screening was included with the use of expert opinion.
For each location attendance rate, travel time and costs were included. Extra cost for the evening hours was 35% of the average salary[18
]. For an external location rent needed to be paid.
Step 7. Select best alternatives
For each of the 72 scenarios we calculated the cost per screen detected child. For each scenario we divided the total cost by the number of true positives which gave us the cost effectiveness-ratio (CE-ratio). We divided the results in terms of iCERS in quartiles of cost-effectiveness (CE_A to CE_D), thus each group included 18 scenario's (see table ). For each of the four groups we estimated the frequency of the experimental variables.
For experimental variable A we saw that the frequency for level 1 (many machines are bought) in the least cost-effective scenario's (group CE_ D) was high. For level 2 (limited machines are bought) we saw a slightly higher frequency in group CE_B and for level 3 (no machines are bought) a higher frequency in group CE_A.
For Experimental variable B consultation the frequencies were mostly located in the most cost effective quartiles for level one (integrated consultation). Level 2 (extra consultation) was mostly located in the least cost effective quartiles.
Figure 1 Experimental variable A Number of ultrasound machines. Quartiles of cost-effectiveness (CE_A to CE_D) ranging from most cost-effective (group CE_A) to least cost-effective (group CE_D). Levels used: 1. As many as necessary US machines, 2. Limited number (more ...)
Figure 2 Experimental variable B Consultation. Quartiles of cost-effectiveness (CE_A to CE_D) ranging from most cost-effective (group CE_A) to least cost-effective (group CE_D). Levels used 1. Integrated consultation with regular consultation at the age of 3 months (more ...)
For experimental variable C a less clear pattern could be revealed. The frequencies for level 1 (infant health care physicians) were high in group CE_C For level 2 (nurses) we saw a high frequency in group CE_A and for level 4 (medical specialist) a high frequency in group CE_B.
Figure 3 Experimental variable C Screener. Quartiles of cost-effectiveness (CE_A to CE_D) ranging from most cost-effective (group CE_A) to least cost-effective (group CE_D). Levels used 1. Infant health care physician, 2. Infant health care nurse, 3. Radiographic (more ...)
For level 1 (daytime in IHC centers) of experimental variable D we found that the frequency was highest in group CE_B and for level 2 (rented buildings) in group CE_C. Level 3 (evening sessions) was almost equally distributed among the four groups, but had the highest frequency in group CE_D.
Figure 4 Experimental variable D Location. Quartiles of cost-effectiveness (CE_A to CE_D) ranging from most cost-effective (group CE_A0 to least cost-effective (group CE_D). Levels used 1. Daytime at IHC, 2. Organized in buildings not owned by IHC (external) and (more ...)
Table Least and most cost-effective scenarios
The five most cost-effective scenarios (47, 29, 55, 49 and 64) and the least cost-effective scenarios (61, 15, 21, 69 and 9) are presented in table . The difference in cost per screen detected child between the least and the most cost-effective scenarios is approximately 2000 euro. This difference is due to the fact that in most cost-effective scenarios no US machines were bought (level 3 for experimental variable A) and screening took place in buildings currently owned by the IHC (level 1 experimental variable D). The screening is done by one of the four screener types and in three cases the screening takes is done by a nurse. All five cost-effective scenarios take place in an integrated consultation. For the least cost-effective scenarios we noted that there were many machines bought, the screening took place in a building that had to be rented sometimes or in the evening with additional salary costs and an extra consultation occurred.
Five Least en most cost-effective scenarios