|Home | About | Journals | Submit | Contact Us | Français|
To determine whether UK obstetric trainees transitioning from directly to indirectly-supervised practice have a higher likelihood of adverse patient outcomes from operative deliveries compared to other indirectly supervised trainees and to examine whether performing more procedures under direct supervision is associated with fewer adverse outcomes in initial indirect practice.
We examined all deliveries (13,861) conducted by obstetricians at a single centre over 5 years (2008-2013). Mixed-effects logistic regression models were used to compare estimated blood loss, maternal trauma, umbilical arterial pH, delayed neonatal respiration, failed instrumental delivery, and critical incidents for trainees in their first indirectly-supervised year with trainees in all other years of indirect practice. Outcomes for trainees in their first indirectly-supervised 3 months were compared to their outcomes for the remainder of the year. Linear regression was used to examine the relationship between number of procedures performed under direct supervision and initial outcomes under indirect supervision.
Trainees in their first indirectly-supervised year had a higher likelihood of >2 litres estimated blood loss at any delivery (OR 1.32;CI(1.01-1.64) p<0.05) and of failed instrumental delivery (OR 2.33;CI(1.37-3.29) p<0.05) compared with other indirectly-supervised trainees. Other measured outcomes showed no significant differences. Within the first three months of indirect supervision, the likelihood of operative vaginal deliveries with >1litre estimated blood loss (OR 2.54;CI(1.88-3.20) p<0.05) was higher compared to the remainder of the first year. Performing more deliveries under direct supervision prior to beginning indirectly-supervised training was associated with decreased risk of >1litre estimated blood loss (p<0.05).
Obstetric trainees in their first year of indirectly-supervised practice have a higher likelihood of immediate adverse delivery outcomes, which are primarily maternal rather than neonatal. Undertaking more directly supervised procedures prior to transitioning to indirectly-supervised practice may reduce adverse outcomes, suggesting that experience is a key consideration in obstetric training programme design.
The aim of obstetric training programs worldwide is to produce obstetricians who achieve good fetal and maternal outcomes in independent practice. One of the key steps in achieving independent practice in any operative skill is the transition between performing a procedure under direct supervision to performing it with indirect supervision, i.e. with help immediately available if required but without a senior clinician present at all times (1). Determining the right time for trainees to embark upon this transition involves a difficult balance between providing optimal educational opportunities to advance surgical skills and ensuring patient safety (2, 3).
Transitioning to independent practice is a practical and psychological milestone and according to transitional psychology trainees should move through the key stages of preparation, encounter, adjustment and stabilization (4). Moreover, the principles of proximal development and constructive friction suggest that the most effective learning will occur when there is a gap between a trainee's current unaided capabilities and the skill level required for full independence (5, 6). In theory, an incremental model in which trainees progress through an individualized program of stages, each involving various degrees of supervision, might provide an optimal balance between skill development and safety. Yet as the traditional apprenticeship model of medical education has been increasingly replaced by a more standardized structure (7), there are fewer opportunities for supervisors to make repeated observations of an individual trainee to gauge skill level and readiness (8). As a result, standardized clinical training programmes are typically designed according to one of three basic models: time-based training, competency-based training or experience-based training (9, 10).
In the United Kingdom (UK), obstetric training is organized following competency-based principles. Assessment of readiness for the transition from the final year of directly supervised training to the first year of indirectly supervised training is based upon completion of a certain number of workplace based assessments for operative skills. Senior obstetricians assess skills such as Caesarean section and operative instrumental vaginal delivery, aiming for the situation where ‘the majority of cases are managed with no direct supervision or assistance…’ (11). Only once these assessments are successfully completed can trainees transition to independent practice.
There is little evidence, however, that assessment of basic competence is sufficient either to optimally meet trainees' learning needs (12) or to prevent increases in operative obstetric complications. Although under a competency-based model it might be expected that adverse procedure outcomes will decrease over time with increasing experience––and thus that trainees in the initial period of indirect supervision will likely have higher rates of complications compared with other indirectly supervised trainees––gaining a greater degree of experience before making the transition to indirectly supervised practice might limit patient harm further. Ideally, although trainees will learn to perform procedures more quickly and efficiently over time as they proceed through indirectly supervised training (13), there should be no substantial differences in adverse patient outcomes over time or across training groups. Moreover, gaining more supervised experience may confer further educational benefits: reducing supervision before the trainee feels ready can have a negative psychological impact on subsequent learning (12) and the experience of making a significant medical error can have a devastating impact on trainees' self-confidence (14).
The objectives of this study are firstly to determine whether trainees in the initial stages of indirectly supervised practice have a significantly higher likelihood of adverse operative outcomes compared to trainees at later stages of indirect training and secondly to examine whether performing a higher number of cases in the directly supervised period is associated with an improvement in patient outcomes in the initial transition to indirectly supervised practice. We hypothesize that deliveries by trainees in the initial phase of indirectly supervised practice will be associated with more adverse events, and that such events could be reduced through greater experience during directly supervised training.
We identified 13,861 deliveries performed by trainee and senior obstetricians over a 5-year period (January 2008 - December 2013) in a single tertiary obstetrics centre in the UK. In this centre, most spontaneous vaginal deliveries are performed by midwives, and hence all cases in the sample are elective or emergency Caesarean sections, instrumented operative vaginal deliveries using forceps or vacuum assistance, or complex spontaneous deliveries, for example in the breech position or pre-term.
Following medical school graduation in the UK, new doctors first undertake two years of general medical and surgical practice. Once this training is complete, they become eligible for a seven-year specialty-training program in obstetrics and gynecology, during which they change hospitals yearly. In our analysis, trainee obstetricians are those in years one to five of specialty training, while senior obstetricians are those with greater than five years of specialty training. Our sample (n=100) is comprised of 11 senior obstetricians permanently employed at the study centre, 25 senior obstetricians who worked at the centre for 1-2 years during the study period, and 64 specialty trainees who worked at the centre during some or all of the study period. Fifty-three obstetricians who had completed a year at the study centre early in their training returned to work there in later training years (n=18) or as senior obstetricians (n=27) or both (n=8). Doctors with fewer than 10 deliveries during the study period were excluded from the analysis as they were most likely locums or visiting fellows who may not have trained according to the UK standardized national curriculum. Frequencies of each type of delivery carried out by trainees in each year of specialty training pooled over the five-year study period are shown in Table 1.
Trainees in their first two years of specialty training perform obstetric procedures under direct supervision. The final year of directly supervised practice thus corresponds to the second year of specialty training. Trainees in years two to five of specialty training typically perform procedures under indirect supervision. The first year of indirectly supervised practice thus corresponds to the third year of specialty training. Direct supervision is defined as having a senior clinician present in the operating room who can guide as necessary all the steps performed during the procedure. The trainee does not have to recognize a problem and ask for help – the onus will be on the supervisor to identify issues and to take over as appropriate. Complications are thus attributed to the supervising clinician. Indirect supervision is defined as having a senior clinician responsible for attending in person at the trainee's request immediately available within the hospital. The trainee carries the responsibility of asking for assistance and thus complications are attributed to the trainee. All of the trainees in our study were assessed during their training according to standard competency-based principles by yearly review. All trainees passed these assessments and progressed to subsequent training years.
On-duty midwives recorded data on each delivery undertaken during the study period in the hospital's electronic system. The obstetrics center regularly performs audits to check data accuracy by comparison to original patient notes. This study involved a secondary analysis of de-identified data and no medical records were accessed. The Institutional Review Board at our institution determined that the study was exempt from full review. For each delivery, maternal and fetal complications were retrieved from the electronic database. Delay in neonatal respiration was recorded where spontaneous respiration was not achieved within 1 minute of delivery. Umbilical cord blood was obtained immediately following delivery, and the arterial pH recorded. Umbilical arterial pH was categorized as ≥7.1 or <7.1 (15). A critical-incident form was generated at delivery in the case of any obstetric or neonatal emergency associated with delivery, including neonatal resuscitation, shoulder dystocia, maternal visceral injury or any other event triggering an obstetric emergency call. Maternal blood loss was measured immediately after delivery, using suction blood collection and weighing of swabs and other pads. Blood loss was categorized as <1 litre, 1-2 litres and >2 litres. Severe maternal perineal trauma was defined as any disruption to the anal sphincter complex.
We also obtained characteristics of the maternal-fetal dyad, including maternal age (at time of delivery), BMI (at first trimester prenatal booking), parity (prior to delivery), and the birth-weight of the infant from the electronic database. Birth-weight was recorded to the nearest gram. Gestational age was determined from first trimester ultrasound and recorded to the nearest week. Mode of delivery was identified as elective Caesarean section, emergency Caesarean section, instrumental vaginal delivery (sub-classified as forceps or ventouse, and hereafter referred to as “instrumental delivery”) and unassisted vaginal delivery (sub-classified as either breech or cephalic). All trainees undergo training in both non-rotational forceps and ventouse delivery.
To examine whether deliveries conducted during the study period by trainees in their first year of indirectly supervised training (third year of specialty training) have a higher likelihood of maternal and fetal complications compared to those conducted by other indirectly supervised trainees, we used a series of binary mixed-effects logistic regression models with trainee-level random-effects. These models allow us to account for individual differences between trainees and to produce standard errors that are robust to the clustering of deliveries within trainees. Birth-weight, gestational age, maternal age and maternal BMI were included as fixed-effects. We did not compare rates of complications for trainees in their final year of direct practice versus their first year of indirect practice because complications arising from cases undertaken during direct practice are typically attributed to the supervising senior doctor.
To further reduce inter-operator variability and investigate the relationship between increasing trainee experience and procedure outcomes, we used a series of binary mixed-effects logistic regression models with trainee-level random effects to compare outcomes for deliveries performed within the first 3 months of indirectly supervised practice versus deliveries performed in the subsequent 9 months of the same year by the same group of trainees. Birth-weight, gestational age, maternal age and maternal BMI were included as fixed effects.
We then assessed the association between performing a greater number of procedures in the period of directly supervised practice and trainee complication rates in the initial period of indirectly supervised practice. Only fifteen individual trainees met the criteria of both spending their last year of directly supervised practice and their first year of indirectly supervised practice within the same obstetrics unit and within the 5-year study timeframe. For each individual, we plotted the percentage of adverse outcomes from deliveries in the first indirectly supervised year against the number of deliveries completed in the final directly supervised year. Adverse outcomes were those demonstrated by the analyses described above to be significantly higher among trainees in the first indirectly supervised year. We then used linear regression models to find the line of best fit and the sum of least squares to determine the goodness-of-fit. The small sample size prohibited multivariable modeling at the individual level, but to check for systematic differences in the difficulty of deliveries performed between trainees, we constructed an index variable using maternal age, maternal BMI, birth-weight and gestation for all deliveries performed by individual trainees during the first indirectly supervised year. Such differences were negligible and thus no adjustment was made.
Finally, we investigated whether trainees' experience could be accurately assessed using the number of workplace-based assessments they had completed. The procedure described above was used to examine the correlation between the number of work-placed based assessments obtained and the total number of directly supervised procedures performed within the last year of directly supervised training.
All analyses were conducted using the R statistical software package version 2.14.1. Findings were considered statistically significant at an alpha level of 0.05.
Analysis of data pooled across all five years of the study period shows that trainees in their final year of directly supervised practice performed 8.1% (1,119) of the total deliveries compared to 20.5% (2,841) performed by trainees in their first year of indirectly supervised practice (Table 1). Comparing those in their final year of directly supervised training with those in their first year of indirectly supervised training, the average number of emergency Caesarean section, forceps, and ventouse deliveries per trainee increased 2.3, 3.8 and 2.6-fold respectively,.
Table 2 shows the comparison of delivery outcomes performed by trainees in their first year of indirectly supervised practice versus trainees in their second or higher years of indirectly supervised practice. Trainees in their first year of indirectly supervised practice have a higher likelihood of estimated blood loss >2 litres at any delivery (OR=1.32, p<0.05) and of estimated blood loss >1 litre at instrumental delivery (OR =1.79, p<0.05). Estimated blood loss >1 litre at Caesarean section (either elective or emergency) is not significantly different between the groups. Trainees in their first indirectly supervised year also have a higher likelihood of failed instrumental deliveries (OR=2.33, p<0.05). The likelihood of severe maternal perineal trauma is also higher, although this association is not statistically significant (OR=1.28, p=0.09). There is no difference between the groups with respect to umbilical artery pH <7.1, delay to neonatal respiration, or critical incidents occurring at delivery.
Trainees have a higher likelihood of estimated blood loss >2 litres at any delivery (OR=1.54, p<0.01) and of estimated blood loss >1 litre at instrumental delivery (OR=2.54, p<0.01) during the first 3 months of indirectly supervised practice compared to the following 9 months of the first year (Table 3). The likelihood of severe maternal perineal trauma is also higher, although this association is not statistically significant (OR=1.71, p=0.07). There are no significant differences in the likelihood of other adverse maternal or fetal outcomes.
Among the fifteen trainees who completed both the final year of direct and the first year of indirect training at the study centre, we observe a significant negative correlation between the total number of deliveries performed in the final year of direct supervision and the percentage of deliveries with an estimated blood loss of >1 litre in the first year of indirectly supervised practice (R2= 0.31, p<0.05) (Figure 1A). We also note a trend towards a lower incidence of severe maternal perineal trauma at instrumental delivery in the first year of indirect supervision, although this association was not statistically significant (R2 =0.26, p=0.09) (Figure 1C). A negative correlation exists between the number of instrumental deliveries performed under direct supervision and both rates of estimated blood loss >1 litre at instrumental delivery and failed instrumental delivery, but neither association is statistically significant (Figures 1B and D).
There is no correlation between the number of Caesarean sections performed by trainees and the number of assessments completed (Figure 2A), but a significant positive correlation (p<0.05) is demonstrated for operative vaginal deliveries (Figure 2B).
Our findings strongly suggest that trainees' operative skills improve as they gain more experience over time during indirectly supervised training. We show that obstetric trainees in their first year of indirectly supervised practice have higher rates of maternal haemorrhage and failed instrumental delivery compared to trainees in subsequent indirectly supervised years. Additionally, within their first 3 months of indirectly supervised practice, trainees have higher rates of maternal haemorrhage compared to the remainder of the first year. For other obstetric outcomes, including compromised neonatal respiration, severe perineal trauma, and critical incidents, we found no significant differences among trainees in their first indirectly supervised year compared to their more experienced colleagues. While these findings suggest that the competency-based training program at our center prepares trainees well for some aspects of independent practice, they also raise the important question of whether providing additional experience prior to the transition to indirectly supervised practice could help reduce all types of adverse patient outcomes.
For the small number of trainees who undertook both directly and indirectly supervised training at our centre, we show that having gained more experience prior to transitioning to indirectly supervised practice is associated with lower rates of maternal haemorrhage. In addition, there was a 3-fold increase in the number of emergency operative deliveries performed by trainees in their first year of indirect supervision compared to trainees in their final year of direct supervision. While a major limitation is that we did not have a sufficiently large sample size (n=15) to robustly assess these correlations, our findings (especially when taken together with the improvements in outcomes seen over time during the indirectly supervised phase) suggest possible benefits to trainees of gaining experience rather than simply preparing for assessments of competency. While we were able to assess only immediate delivery outcomes, maternal obstetric haemorrhage and failed instrumental delivery are both strongly associated with maternal morbidity in the puerperium (16, 17). Failed instrumental delivery is also associated with neonatal morbidity (18), although we did not find significant differences in other immediate neonatal outcomes by training year.
Our findings raise intriguing questions about what factors account for the observed differences in degree of trainee experience during the directly supervised training period––the variability in the number of procedures undertaken by individual trainees is striking. One possibility is that greater opportunities are given to trainees who are perceived by their educators to show more surgical aptitude, enthusiasm or dedication to training. If so, then experience per se might not be directly related to better outcomes, but rather might serve as a surrogate for other aspects of trainee performance or trainee characteristics. Conversely, those with the poorest skills may be perceived by their trainers to require most training and may be pushed to perform extra cases. Alternatively, there may be other individual attributes that determine the number of cases a trainee will be perform. A better understanding of such influences could shed further light on training program design and is a prime target for future research.
Previous studies across a number of surgical disciplines have also identified the need for increased operative experience in terms of total case numbers (19-21). Within obstetrics, many trainees do not achieve adequate volumes of instrumental deliveries during training overall, even within teaching hospitals (20) and despite the influence of obstetrician experience on instrumental delivery success (17). Focusing specifically on the transition from directly to indirectly supervised training, we have identified a potential opportunity to modify the structure of the training programme to integrate needed experience. The current competency-based model does not appear to provide adequate training to prevent an increase in all adverse patient outcomes in the initial phase of indirectly supervised practice, perhaps because assessments are often based upon straightforward cases. On the basis of our findings, consideration of a more experience-based model might not only ensure competence to perform a straightforward procedure but also to allow trainees the opportunity to develop the skills to avoid complications as they arise. These opportunities could include the use of simulation-based training (22) or the requirement of a certain quota of procedures and a test of peer-level competence before transitioning to the next training stage. Workplace-based assessment numbers provide a faithful estimation of the total numbers of procedures performed for operative vaginal delivery (but not for Caesarean section) and may therefore be a useful tool in assessing training from an experience point-of-view. Future research should examine not only the total number of assessments completed but also the relationship between how well trainees performed on these assessments and their subsequent clinical outcomes in independent practice. These findings could illuminate whether it is possible to prospectively identify trainees who are at risk of poor performance.
The use of patient outcomes to evaluate training programmes in obstetrics allows a practical and patient-centred approach (23), which has also been used in other surgical disciplines (24). We have focused primarily on operative outcomes but we also recognize that increasing seniority demands simultaneous development of other important skills (including communication and organizational skills) for which trainees undergoing important career transitions may feel under-prepared (25). Quality of surgical supervision and learning experience is also a key factor in determining how valuable trainees consider their programs (26, 27), and this may correlate better with surgical outcomes than the absolute number of cases performed. Continuity of training within the same institution and surgical team may also be important in developing surgical skills more efficiently, as communication and relationship with supervising senior clinicians have often been identified as important to surgical education (26, 28).
The main strengths of our study are the inclusion of trainees who had completed both directly and indirectly supervised practice in the same setting, thus minimizing the variability in training opportunities available during the study period and inter-operator differences. Our study also has several limitations. Focusing on trainees following the national RCOG curriculum for obstetrics training in a single centre means that our findings may not be generalizable to other settings. Additionally, while we were able to control for characteristics of the maternal-fetal dyad and for variability between individual trainees, we were unable to control for factors such as the urgency of each procedure and the management of the patient during labor and prior to delivery. In particular, a potential source of bias is that more junior trainees may have undertaken systematically less complicated deliveries involving lower-risk women and infants, while more senior trainees may have undertaken higher-risk deliveries. We cannot fully control for this possible selection bias in our models.
Transitioning to indirect supervision for operative procedures is a significant milestone in the training pathway. All the trainees in our study passed competency-based training. However, analysis of their operative outcomes suggests a need for further operating experience to limit patient harm, particularly with respect to maternal outcomes. Calculation of the number of directly supervised procedures required to produce no difference in patient outcomes in the initial independent phase is a potential target for future research. The need for increased opportunities to build experience is relevant not only to obstetric training but could apply across other surgical and procedural disciplines.
Funding: ARA is supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) grant for Infrastructure for Population Research at Princeton University (Grant R24HD047879). During the initial preparation of this manuscript she was also supported by an NICHD Individual Predoctoral Fellowship (Grant F31HD079182) and an NICHD infrastructure grant awarded to the Population Research Center at The University of Texas at Austin (Grant R24HD042849).
Disclosures of Interest: The authors have no conflicts of interest to declare.
Author Contributions: CA, AA and JB conceived of and designed the study. CA, AA, HP and AP collected and analysed the data. CA, AA, JB, HP and AP wrote and edited the manuscript. All authors have approved the final version.
Ethics Approval: No patient-identifiable data were accessed in the course of this research, which was performed as part of a provision-of-service study for the obstetrics centre. Individual medical records were not accessed at any stage. The Institutional Review Board confirmed that the work was exempt from expedited or full board review. The work was carried out in accordance with the Declaration of Helsinki, including, but not limited to there being no potential harm to participants and the anonymity of participants is guaranteed.