The determination of a suitable estimate of effect size is notoriously difficult [26
]. Our mailed survey of all Canadian obstetricians attempted to estimate the clinically important treatment effect required to change practice in the prevention of preterm birth, in order to increase the likelihood that our research would be relevant, with the potential to influence clinical practice. Our study found that responding clinicians were willing to identify an increase in duration of pregnancy that they believed would change their practice. The small amount of missing data indicated that respondents found the clinical scenarios plausible.
Our study found that minimal clinically important treatment effect was associated with the invasiveness of the treatment, and current clinical practice of the respondents. A one or two week prolongation of pregnancy was the minimum effect size required by around 70% of obstetricians to introduce new progesterone treatments into their practice for treating women at risk of preterm birth. Cerclage, a more invasive treatment, required a larger minimum effect size: a three week increase was needed to introduce this treatment. Availability of the treatment also influenced the minimum important treatment effect: clinicians who already had access to cerclage were willing to accept a smaller effect size to utilize this treatment.
The impact of the choice of effect size may best be illustrated using the parameters we employed when calculating the sample size for our progesterone trial [27
]. For that study, we used local data from a two year period to estimate the usual (untreated) mean and standard deviation gestational age of delivery for multiple pregnancies. Sample size calculations using a power of 80% and two-sided significance level of 0.05, would estimate sample sizes as follows: to detect a three week difference, a sample of 36 (18 per randomised group) would be required; for two weeks, the sample size would be 78 (39 per group); for one week, the sample size would be 156 (78 per group). If the three week effect size was chosen, and therefore the smaller sample size was used, a real difference between treated and control groups of one or two weeks would not be identified as a significant difference. These examples clearly show that the choice of clinically important treatment effect has a practical effect on the design, conduct and clinical relevance of the study. Applying these examples to our hypothetical scenarios, it is clear that a trial of cerclage versus no cerclage would be smaller in size than a trial of a progesterone treatment versus placebo.
The principal goal of prophylactic treatment for women at risk of preterm labour is to improve the outcome for the neonate by reducing morbidity and mortality. The consequences of even late preterm birth are considerable, with worse developmental outcomes and academic difficulties up to seven years of age compared to term infants [28
]. In our questionnaire study, the majority of respondents (72.2%) clearly recognised this by stating that decreased fetal morbidity was the most important outcome to justify changing clinical practice. Despite this, in our hypothetical scenarios where clinicians were asked to consider changing their clinical practice on the basis of prolongation of pregnancy, all but 4% were willing to make a choice based on that outcome. It therefore appears that clinicians would be willing to accept a surrogate outcome on which to base a change in clinical practice. This is an important point: a trial that measures impact of a treatment on fetal or neonatal morbidity, even in a trial examining preterm birth where adverse neonatal events would be expected, will need to be far larger than one that measures impact on pregnancy duration, where every pregnancy will have an outcome [29
]. In addition, the definition of a 'neonatal morbidity' composite outcome is fraught with difficulty, and is open to misinterpretation by clinicians and patients [29
]. Thus the choice of effect size and choice of outcome will impact on the size and expense of a clinical trial. Both of these choices will also impact on the adoption of trial findings into clinical practice.
We were interested about the feasibility of trials of treatments to prevent preterm birth. Concern for the patient and increasing invasiveness of the intervention impacted on respondents' decisions about taking part in a trial. The majority of respondents would be willing to take part in both hypothetical progesterone trials. The most frequently cited reason for declining to take part in the multiple trial was feasibility (lack of suitable cases), while concern about the intervention (for example patient discomfort) was most commonly cited for the intramuscular progesterone trial. Concern about cerclage itself was a common reason for not wishing to take part in a trial, although 13.2% believed that there was already sufficient evidence to use prophylactic cerclage in women with a short cervix. We did not lead clinicians in their reason for not wishing to take part in each trial, rather leaving them to provide text answers to open-ended questions. Therefore we believe the responses were those that mattered most to the clinicians.
The design of our questionnaire could be criticised for a number of reasons. Firstly, the questionnaire described three hypothetical scenarios, including different risk factors for preterm birth. Although designed to represent clinical experience, we do not know whether the responses would reflect actual practice. Secondly, in the clinical scenarios, we used a forced choice method that asked clinicians to choose a one, two or three week prolongation of pregnancy as the outcome for the hypothetical treatments. It is possible that clinicians would be willing to accept a shorter increase or would prefer a longer increase in gestation before changing their practice. We set the earliest gestational age for treatment at 16 weeks, and the longest duration of pregnancy as 33 weeks and 4 days [30
], for each hypothetical prolongation of pregnancy. Thus it is possible that the choice of clinically important increase in duration of pregnancy may have been affected by our descriptions. These scenarios and choices were necessarily complex, reflecting situations commonly encountered by obstetricians who manage high risk pregnancies.
Much research relevant to the prevention of preterm birth has been published since our survey was carried out, reflecting the ongoing interest and importance of this topic. Recent publications conclude that, in appropriately selected women with high-risk singleton pregnancies, progesterone can reduce preterm birth before 33 weeks' gestation [32
], and cerclage can reduce preterm birth and improve neonatal outcome [33
]. Trials of progesterone in multiple pregnancies have failed to prevent preterm birth [34
], most likely because of a difference in aetiology, perhaps involving larger fetal and placental mass, and greater stretching of uterine muscle [35
]. Nonetheless, preterm birth remains the most concerning perinatal problem for obstetricians, and therefore new progesterone trials continue to be undertaken [36
], and the search for new effective treatments continues.
Our survey of all obstetricians was challenging and would not be feasible for all trials, partly because such surveys are costly and time consuming, and partly because clinicians would be plagued by many similar questionnaires, leading to declining response rates and possible bias in the results. In the future, we believe that a more limited survey of a random sample of practicing clinicians would be more appropriate. In our study, however, obstetricians were willing to respond to our fairly complex questionnaire: we achieved a response rate of 42.1% after only two reminders, similar to response rates from other surveys of clinical practice [25
]. Our respondents came from a range of clinical settings and had a range of experience. Our large number of responses (n = 544) leads us to believe that our findings are more representative of clinician beliefs than the more usual practice of identifying the minimum clinically important treatment effect by discussion with trial collaborators [25
] or experts [23