PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Educ. Author manuscript; available in PMC 2016 July 1.
Published in final edited form as:
PMCID: PMC4731334
NIHMSID: NIHMS752664

Factors associated with adverse clinical outcomes among obstetric trainees

Catherine E. Aiken PhD, PhD, MRCOG,1,* Abigail Aiken, PhD,2 Hannah Park, MB/BChir,1 Jeremy C. Brockelsby, PhD, MRCOG,1 and Andrew Prentice, FRCOG1

Abstract

Objective

To determine whether UK obstetric trainees transitioning from directly to indirectly-supervised practice have a higher likelihood of adverse patient outcomes from operative deliveries compared to other indirectly supervised trainees and to examine whether performing more procedures under direct supervision is associated with fewer adverse outcomes in initial indirect practice.

Methods

We examined all deliveries (13,861) conducted by obstetricians at a single centre over 5 years (2008-2013). Mixed-effects logistic regression models were used to compare estimated blood loss, maternal trauma, umbilical arterial pH, delayed neonatal respiration, failed instrumental delivery, and critical incidents for trainees in their first indirectly-supervised year with trainees in all other years of indirect practice. Outcomes for trainees in their first indirectly-supervised 3 months were compared to their outcomes for the remainder of the year. Linear regression was used to examine the relationship between number of procedures performed under direct supervision and initial outcomes under indirect supervision.

Results

Trainees in their first indirectly-supervised year had a higher likelihood of >2 litres estimated blood loss at any delivery (OR 1.32;CI(1.01-1.64) p<0.05) and of failed instrumental delivery (OR 2.33;CI(1.37-3.29) p<0.05) compared with other indirectly-supervised trainees. Other measured outcomes showed no significant differences. Within the first three months of indirect supervision, the likelihood of operative vaginal deliveries with >1litre estimated blood loss (OR 2.54;CI(1.88-3.20) p<0.05) was higher compared to the remainder of the first year. Performing more deliveries under direct supervision prior to beginning indirectly-supervised training was associated with decreased risk of >1litre estimated blood loss (p<0.05).

Conclusions

Obstetric trainees in their first year of indirectly-supervised practice have a higher likelihood of immediate adverse delivery outcomes, which are primarily maternal rather than neonatal. Undertaking more directly supervised procedures prior to transitioning to indirectly-supervised practice may reduce adverse outcomes, suggesting that experience is a key consideration in obstetric training programme design.

Keywords: obstetrics, training, supervision, delivery outcomes, education

Introduction

The aim of obstetric training programs worldwide is to produce obstetricians who achieve good fetal and maternal outcomes in independent practice. One of the key steps in achieving independent practice in any operative skill is the transition between performing a procedure under direct supervision to performing it with indirect supervision, i.e. with help immediately available if required but without a senior clinician present at all times (1). Determining the right time for trainees to embark upon this transition involves a difficult balance between providing optimal educational opportunities to advance surgical skills and ensuring patient safety (2, 3).

Transitioning to independent practice is a practical and psychological milestone and according to transitional psychology trainees should move through the key stages of preparation, encounter, adjustment and stabilization (4). Moreover, the principles of proximal development and constructive friction suggest that the most effective learning will occur when there is a gap between a trainee's current unaided capabilities and the skill level required for full independence (5, 6). In theory, an incremental model in which trainees progress through an individualized program of stages, each involving various degrees of supervision, might provide an optimal balance between skill development and safety. Yet as the traditional apprenticeship model of medical education has been increasingly replaced by a more standardized structure (7), there are fewer opportunities for supervisors to make repeated observations of an individual trainee to gauge skill level and readiness (8). As a result, standardized clinical training programmes are typically designed according to one of three basic models: time-based training, competency-based training or experience-based training (9, 10).

In the United Kingdom (UK), obstetric training is organized following competency-based principles. Assessment of readiness for the transition from the final year of directly supervised training to the first year of indirectly supervised training is based upon completion of a certain number of workplace based assessments for operative skills. Senior obstetricians assess skills such as Caesarean section and operative instrumental vaginal delivery, aiming for the situation where ‘the majority of cases are managed with no direct supervision or assistance…’ (11). Only once these assessments are successfully completed can trainees transition to independent practice.

There is little evidence, however, that assessment of basic competence is sufficient either to optimally meet trainees' learning needs (12) or to prevent increases in operative obstetric complications. Although under a competency-based model it might be expected that adverse procedure outcomes will decrease over time with increasing experience––and thus that trainees in the initial period of indirect supervision will likely have higher rates of complications compared with other indirectly supervised trainees––gaining a greater degree of experience before making the transition to indirectly supervised practice might limit patient harm further. Ideally, although trainees will learn to perform procedures more quickly and efficiently over time as they proceed through indirectly supervised training (13), there should be no substantial differences in adverse patient outcomes over time or across training groups. Moreover, gaining more supervised experience may confer further educational benefits: reducing supervision before the trainee feels ready can have a negative psychological impact on subsequent learning (12) and the experience of making a significant medical error can have a devastating impact on trainees' self-confidence (14).

The objectives of this study are firstly to determine whether trainees in the initial stages of indirectly supervised practice have a significantly higher likelihood of adverse operative outcomes compared to trainees at later stages of indirect training and secondly to examine whether performing a higher number of cases in the directly supervised period is associated with an improvement in patient outcomes in the initial transition to indirectly supervised practice. We hypothesize that deliveries by trainees in the initial phase of indirectly supervised practice will be associated with more adverse events, and that such events could be reduced through greater experience during directly supervised training.

Methods

We identified 13,861 deliveries performed by trainee and senior obstetricians over a 5-year period (January 2008 - December 2013) in a single tertiary obstetrics centre in the UK. In this centre, most spontaneous vaginal deliveries are performed by midwives, and hence all cases in the sample are elective or emergency Caesarean sections, instrumented operative vaginal deliveries using forceps or vacuum assistance, or complex spontaneous deliveries, for example in the breech position or pre-term.

Following medical school graduation in the UK, new doctors first undertake two years of general medical and surgical practice. Once this training is complete, they become eligible for a seven-year specialty-training program in obstetrics and gynecology, during which they change hospitals yearly. In our analysis, trainee obstetricians are those in years one to five of specialty training, while senior obstetricians are those with greater than five years of specialty training. Our sample (n=100) is comprised of 11 senior obstetricians permanently employed at the study centre, 25 senior obstetricians who worked at the centre for 1-2 years during the study period, and 64 specialty trainees who worked at the centre during some or all of the study period. Fifty-three obstetricians who had completed a year at the study centre early in their training returned to work there in later training years (n=18) or as senior obstetricians (n=27) or both (n=8). Doctors with fewer than 10 deliveries during the study period were excluded from the analysis as they were most likely locums or visiting fellows who may not have trained according to the UK standardized national curriculum. Frequencies of each type of delivery carried out by trainees in each year of specialty training pooled over the five-year study period are shown in Table 1.

Table 1
Number of deliveries and type of deliveries performed by doctors between Jan 2008 and December 2013. Data are displayed as n (percentage). ‘Total’ refers to the number of deliveries performed by all the doctors at each level during the ...

Trainees in their first two years of specialty training perform obstetric procedures under direct supervision. The final year of directly supervised practice thus corresponds to the second year of specialty training. Trainees in years two to five of specialty training typically perform procedures under indirect supervision. The first year of indirectly supervised practice thus corresponds to the third year of specialty training. Direct supervision is defined as having a senior clinician present in the operating room who can guide as necessary all the steps performed during the procedure. The trainee does not have to recognize a problem and ask for help – the onus will be on the supervisor to identify issues and to take over as appropriate. Complications are thus attributed to the supervising clinician. Indirect supervision is defined as having a senior clinician responsible for attending in person at the trainee's request immediately available within the hospital. The trainee carries the responsibility of asking for assistance and thus complications are attributed to the trainee. All of the trainees in our study were assessed during their training according to standard competency-based principles by yearly review. All trainees passed these assessments and progressed to subsequent training years.

On-duty midwives recorded data on each delivery undertaken during the study period in the hospital's electronic system. The obstetrics center regularly performs audits to check data accuracy by comparison to original patient notes. This study involved a secondary analysis of de-identified data and no medical records were accessed. The Institutional Review Board at our institution determined that the study was exempt from full review. For each delivery, maternal and fetal complications were retrieved from the electronic database. Delay in neonatal respiration was recorded where spontaneous respiration was not achieved within 1 minute of delivery. Umbilical cord blood was obtained immediately following delivery, and the arterial pH recorded. Umbilical arterial pH was categorized as ≥7.1 or <7.1 (15). A critical-incident form was generated at delivery in the case of any obstetric or neonatal emergency associated with delivery, including neonatal resuscitation, shoulder dystocia, maternal visceral injury or any other event triggering an obstetric emergency call. Maternal blood loss was measured immediately after delivery, using suction blood collection and weighing of swabs and other pads. Blood loss was categorized as <1 litre, 1-2 litres and >2 litres. Severe maternal perineal trauma was defined as any disruption to the anal sphincter complex.

We also obtained characteristics of the maternal-fetal dyad, including maternal age (at time of delivery), BMI (at first trimester prenatal booking), parity (prior to delivery), and the birth-weight of the infant from the electronic database. Birth-weight was recorded to the nearest gram. Gestational age was determined from first trimester ultrasound and recorded to the nearest week. Mode of delivery was identified as elective Caesarean section, emergency Caesarean section, instrumental vaginal delivery (sub-classified as forceps or ventouse, and hereafter referred to as “instrumental delivery”) and unassisted vaginal delivery (sub-classified as either breech or cephalic). All trainees undergo training in both non-rotational forceps and ventouse delivery.

To examine whether deliveries conducted during the study period by trainees in their first year of indirectly supervised training (third year of specialty training) have a higher likelihood of maternal and fetal complications compared to those conducted by other indirectly supervised trainees, we used a series of binary mixed-effects logistic regression models with trainee-level random-effects. These models allow us to account for individual differences between trainees and to produce standard errors that are robust to the clustering of deliveries within trainees. Birth-weight, gestational age, maternal age and maternal BMI were included as fixed-effects. We did not compare rates of complications for trainees in their final year of direct practice versus their first year of indirect practice because complications arising from cases undertaken during direct practice are typically attributed to the supervising senior doctor.

To further reduce inter-operator variability and investigate the relationship between increasing trainee experience and procedure outcomes, we used a series of binary mixed-effects logistic regression models with trainee-level random effects to compare outcomes for deliveries performed within the first 3 months of indirectly supervised practice versus deliveries performed in the subsequent 9 months of the same year by the same group of trainees. Birth-weight, gestational age, maternal age and maternal BMI were included as fixed effects.

We then assessed the association between performing a greater number of procedures in the period of directly supervised practice and trainee complication rates in the initial period of indirectly supervised practice. Only fifteen individual trainees met the criteria of both spending their last year of directly supervised practice and their first year of indirectly supervised practice within the same obstetrics unit and within the 5-year study timeframe. For each individual, we plotted the percentage of adverse outcomes from deliveries in the first indirectly supervised year against the number of deliveries completed in the final directly supervised year. Adverse outcomes were those demonstrated by the analyses described above to be significantly higher among trainees in the first indirectly supervised year. We then used linear regression models to find the line of best fit and the sum of least squares to determine the goodness-of-fit. The small sample size prohibited multivariable modeling at the individual level, but to check for systematic differences in the difficulty of deliveries performed between trainees, we constructed an index variable using maternal age, maternal BMI, birth-weight and gestation for all deliveries performed by individual trainees during the first indirectly supervised year. Such differences were negligible and thus no adjustment was made.

Finally, we investigated whether trainees' experience could be accurately assessed using the number of workplace-based assessments they had completed. The procedure described above was used to examine the correlation between the number of work-placed based assessments obtained and the total number of directly supervised procedures performed within the last year of directly supervised training.

All analyses were conducted using the R statistical software package version 2.14.1. Findings were considered statistically significant at an alpha level of 0.05.

Results

Analysis of data pooled across all five years of the study period shows that trainees in their final year of directly supervised practice performed 8.1% (1,119) of the total deliveries compared to 20.5% (2,841) performed by trainees in their first year of indirectly supervised practice (Table 1). Comparing those in their final year of directly supervised training with those in their first year of indirectly supervised training, the average number of emergency Caesarean section, forceps, and ventouse deliveries per trainee increased 2.3, 3.8 and 2.6-fold respectively,.

Table 2 shows the comparison of delivery outcomes performed by trainees in their first year of indirectly supervised practice versus trainees in their second or higher years of indirectly supervised practice. Trainees in their first year of indirectly supervised practice have a higher likelihood of estimated blood loss >2 litres at any delivery (OR=1.32, p<0.05) and of estimated blood loss >1 litre at instrumental delivery (OR =1.79, p<0.05). Estimated blood loss >1 litre at Caesarean section (either elective or emergency) is not significantly different between the groups. Trainees in their first indirectly supervised year also have a higher likelihood of failed instrumental deliveries (OR=2.33, p<0.05). The likelihood of severe maternal perineal trauma is also higher, although this association is not statistically significant (OR=1.28, p=0.09). There is no difference between the groups with respect to umbilical artery pH <7.1, delay to neonatal respiration, or critical incidents occurring at delivery.

Table 2
Delivery outcomes for trainees in their first year of indirect supervision compared to other indirectly supervised trainees. Delivery outcomes are adjusted for birth-weight, gestational age, maternal age and maternal BMI as fixed effects, and trainee ...

Trainees have a higher likelihood of estimated blood loss >2 litres at any delivery (OR=1.54, p<0.01) and of estimated blood loss >1 litre at instrumental delivery (OR=2.54, p<0.01) during the first 3 months of indirectly supervised practice compared to the following 9 months of the first year (Table 3). The likelihood of severe maternal perineal trauma is also higher, although this association is not statistically significant (OR=1.71, p=0.07). There are no significant differences in the likelihood of other adverse maternal or fetal outcomes.

Table 3
Delivery outcomes for indirectly supervised trainees in their first 3 months compared to the same group in the subsequent 9 months. Delivery outcomes are adjusted for birth-weight, gestational age, maternal age and maternal BMI, as fixed effects and trainee ...

Among the fifteen trainees who completed both the final year of direct and the first year of indirect training at the study centre, we observe a significant negative correlation between the total number of deliveries performed in the final year of direct supervision and the percentage of deliveries with an estimated blood loss of >1 litre in the first year of indirectly supervised practice (R2= 0.31, p<0.05) (Figure 1A). We also note a trend towards a lower incidence of severe maternal perineal trauma at instrumental delivery in the first year of indirect supervision, although this association was not statistically significant (R2 =0.26, p=0.09) (Figure 1C). A negative correlation exists between the number of instrumental deliveries performed under direct supervision and both rates of estimated blood loss >1 litre at instrumental delivery and failed instrumental delivery, but neither association is statistically significant (Figures 1B and D).

Figure 1
Delivery outcomes during the third year of training plotted against number of deliveries performed in second year of training. Each closed circle represents a single trainee. A) Total number of deliveries of any type in second training year (year 2) v. ...

There is no correlation between the number of Caesarean sections performed by trainees and the number of assessments completed (Figure 2A), but a significant positive correlation (p<0.05) is demonstrated for operative vaginal deliveries (Figure 2B).

Figure 2
Number of work-placed based assessments (WPBA) confirming competence obtained during the second year of training plotted against number of deliveries performed in second year of training. Each closed circle represents a single trainee

Discussion

Our findings strongly suggest that trainees' operative skills improve as they gain more experience over time during indirectly supervised training. We show that obstetric trainees in their first year of indirectly supervised practice have higher rates of maternal haemorrhage and failed instrumental delivery compared to trainees in subsequent indirectly supervised years. Additionally, within their first 3 months of indirectly supervised practice, trainees have higher rates of maternal haemorrhage compared to the remainder of the first year. For other obstetric outcomes, including compromised neonatal respiration, severe perineal trauma, and critical incidents, we found no significant differences among trainees in their first indirectly supervised year compared to their more experienced colleagues. While these findings suggest that the competency-based training program at our center prepares trainees well for some aspects of independent practice, they also raise the important question of whether providing additional experience prior to the transition to indirectly supervised practice could help reduce all types of adverse patient outcomes.

For the small number of trainees who undertook both directly and indirectly supervised training at our centre, we show that having gained more experience prior to transitioning to indirectly supervised practice is associated with lower rates of maternal haemorrhage. In addition, there was a 3-fold increase in the number of emergency operative deliveries performed by trainees in their first year of indirect supervision compared to trainees in their final year of direct supervision. While a major limitation is that we did not have a sufficiently large sample size (n=15) to robustly assess these correlations, our findings (especially when taken together with the improvements in outcomes seen over time during the indirectly supervised phase) suggest possible benefits to trainees of gaining experience rather than simply preparing for assessments of competency. While we were able to assess only immediate delivery outcomes, maternal obstetric haemorrhage and failed instrumental delivery are both strongly associated with maternal morbidity in the puerperium (16, 17). Failed instrumental delivery is also associated with neonatal morbidity (18), although we did not find significant differences in other immediate neonatal outcomes by training year.

Our findings raise intriguing questions about what factors account for the observed differences in degree of trainee experience during the directly supervised training period––the variability in the number of procedures undertaken by individual trainees is striking. One possibility is that greater opportunities are given to trainees who are perceived by their educators to show more surgical aptitude, enthusiasm or dedication to training. If so, then experience per se might not be directly related to better outcomes, but rather might serve as a surrogate for other aspects of trainee performance or trainee characteristics. Conversely, those with the poorest skills may be perceived by their trainers to require most training and may be pushed to perform extra cases. Alternatively, there may be other individual attributes that determine the number of cases a trainee will be perform. A better understanding of such influences could shed further light on training program design and is a prime target for future research.

Previous studies across a number of surgical disciplines have also identified the need for increased operative experience in terms of total case numbers (19-21). Within obstetrics, many trainees do not achieve adequate volumes of instrumental deliveries during training overall, even within teaching hospitals (20) and despite the influence of obstetrician experience on instrumental delivery success (17). Focusing specifically on the transition from directly to indirectly supervised training, we have identified a potential opportunity to modify the structure of the training programme to integrate needed experience. The current competency-based model does not appear to provide adequate training to prevent an increase in all adverse patient outcomes in the initial phase of indirectly supervised practice, perhaps because assessments are often based upon straightforward cases. On the basis of our findings, consideration of a more experience-based model might not only ensure competence to perform a straightforward procedure but also to allow trainees the opportunity to develop the skills to avoid complications as they arise. These opportunities could include the use of simulation-based training (22) or the requirement of a certain quota of procedures and a test of peer-level competence before transitioning to the next training stage. Workplace-based assessment numbers provide a faithful estimation of the total numbers of procedures performed for operative vaginal delivery (but not for Caesarean section) and may therefore be a useful tool in assessing training from an experience point-of-view. Future research should examine not only the total number of assessments completed but also the relationship between how well trainees performed on these assessments and their subsequent clinical outcomes in independent practice. These findings could illuminate whether it is possible to prospectively identify trainees who are at risk of poor performance.

The use of patient outcomes to evaluate training programmes in obstetrics allows a practical and patient-centred approach (23), which has also been used in other surgical disciplines (24). We have focused primarily on operative outcomes but we also recognize that increasing seniority demands simultaneous development of other important skills (including communication and organizational skills) for which trainees undergoing important career transitions may feel under-prepared (25). Quality of surgical supervision and learning experience is also a key factor in determining how valuable trainees consider their programs (26, 27), and this may correlate better with surgical outcomes than the absolute number of cases performed. Continuity of training within the same institution and surgical team may also be important in developing surgical skills more efficiently, as communication and relationship with supervising senior clinicians have often been identified as important to surgical education (26, 28).

The main strengths of our study are the inclusion of trainees who had completed both directly and indirectly supervised practice in the same setting, thus minimizing the variability in training opportunities available during the study period and inter-operator differences. Our study also has several limitations. Focusing on trainees following the national RCOG curriculum for obstetrics training in a single centre means that our findings may not be generalizable to other settings. Additionally, while we were able to control for characteristics of the maternal-fetal dyad and for variability between individual trainees, we were unable to control for factors such as the urgency of each procedure and the management of the patient during labor and prior to delivery. In particular, a potential source of bias is that more junior trainees may have undertaken systematically less complicated deliveries involving lower-risk women and infants, while more senior trainees may have undertaken higher-risk deliveries. We cannot fully control for this possible selection bias in our models.

Conclusion

Transitioning to indirect supervision for operative procedures is a significant milestone in the training pathway. All the trainees in our study passed competency-based training. However, analysis of their operative outcomes suggests a need for further operating experience to limit patient harm, particularly with respect to maternal outcomes. Calculation of the number of directly supervised procedures required to produce no difference in patient outcomes in the initial independent phase is a potential target for future research. The need for increased opportunities to build experience is relevant not only to obstetric training but could apply across other surgical and procedural disciplines.

Acknowledgments

Funding: ARA is supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) grant for Infrastructure for Population Research at Princeton University (Grant R24HD047879). During the initial preparation of this manuscript she was also supported by an NICHD Individual Predoctoral Fellowship (Grant F31HD079182) and an NICHD infrastructure grant awarded to the Population Research Center at The University of Texas at Austin (Grant R24HD042849).

Footnotes

Disclosures of Interest: The authors have no conflicts of interest to declare.

Author Contributions: CA, AA and JB conceived of and designed the study. CA, AA, HP and AP collected and analysed the data. CA, AA, JB, HP and AP wrote and edited the manuscript. All authors have approved the final version

.

Ethics Approval: No patient-identifiable data were accessed in the course of this research, which was performed as part of a provision-of-service study for the obstetrics centre. Individual medical records were not accessed at any stage. The Institutional Review Board confirmed that the work was exempt from expedited or full board review. The work was carried out in accordance with the Declaration of Helsinki, including, but not limited to there being no potential harm to participants and the anonymity of participants is guaranteed.

References

1. ACGME. New Supervision Standards: Discussion And Justification. 2011
2. Teunissen PW, Westerman M. Opportunity or threat: the ambiguity of the consequences of transitions in medical education. Med Educ. 2011 Jan;45(1):51. [PubMed]
3. Kennedy TJ. Towards a tighter link between supervision and trainee ability. Med Educ. 2009 Dec;43(12):1126–8. [PubMed]
4. Nicholson N. A theory of work role transitions. Adm Sci Q. 1984;29:172.
5. Chaiklin S. The zone of proximal development in Vygotsky's analysis of learning and instruction. In: Kozulin A, Gindis B, Ageyev V, S M, editors. Vygotsky's educational theory in cultural context. Cambridge: Cambridge University Press; 2003. pp. 39–64.
6. Vermunt JD, Verloop N. Congruence and friction between learning and teaching. Learning and Instruction. 1999;9:257–80.
7. Teunissen PW, Scheele F, Scherpbier AJ, van der Vleuten CP, Boor K, van Luijk SJ, et al. How residents learn: qualitative evidence for the pivotal role of clinical activities. Med Educ. 2007 Aug;41(8):763–70. [PubMed]
8. Kennedy TJ, Regehr G, Baker GR, Lingard LA. Progressive independence in clinical training: a tradition worth defending? Acad Med. 2005 Oct;80(10 Suppl):S106–11. [PubMed]
9. Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010 Dec 4;376(9756):1923–58. [PubMed]
10. Gruppen LD, Mangrulkar RS, Kolars JC. The promise of competency-based education in the health professions for improving global health. Hum Resour Health. 2012;10(1):43. [PMC free article] [PubMed]
11. Annual Expectation of Educational Progression. Royal College of Obstetricians and Gynaecologists. 2013
12. Dijksterhuis MG, Voorhuis M, Teunissen PW, Schuwirth LW, ten Cate OT, Braat DD, et al. Assessment of competence and progressive independence in postgraduate clinical training. Med Educ. 2009 Dec;43(12):1156–65. [PubMed]
13. Naiditch JA, Lautz TB, Raval MV, Madonna MB, Barsness KA. Effect of resident postgraduate year on outcomes after laparoscopic appendectomy for appendicitis in children. J Laparoendosc Adv Surg Tech A. 2012 Sep;22(7):715–9. [PubMed]
14. Engel KG, Rosenthal M, Sutcliffe KM. Residents' responses to medical error: coping, learning, and change. Acad Med. 2006 Jan;81(1):86–93. [PubMed]
15. Yeh P, Emary K, Impey L. The relationship between umbilical cord arterial pH and serious adverse neonatal outcome: analysis of 51,519 consecutive validated samples. BJOG. 2012 Jun;119(7):824–31. [PubMed]
16. Al-Kadri H, Sabr Y, Al-Saif S, Abulaimoun B, Ba'Aqeel H, Saleh A. Failed individual and sequential instrumental vaginal delivery: contributing risk factors and maternal-neonatal complications. Acta Obstet Gynecol Scand. 2003 Jul;82(7):642–8. [PubMed]
17. Aiken CE, Aiken AR, Brocklesby JC, Scott JG. Factors Influencing the Likelihood of Instrumental Delivery Success. Obstet Gynecol. 2014;123(4):796–803. [PMC free article] [PubMed]
18. Alexander JM, Leveno KJ, Hauth JC, Landon MB, Gilbert S, Spong CY, et al. Failed operative vaginal delivery. Obstet Gynecol. 2009 Nov;114(5):1017–22. [PMC free article] [PubMed]
19. Rowe MI, Courcoulas A, Reblock K. An analysis of the operative experience of North American pediatric surgical training programs and residents. J Pediatr Surg. 1997 Feb;32(2):184–91. [PubMed]
20. Kyser KL, Lu X, Santillan D, Santillan M, Caughey AB, Wilson MC, et al. Forceps delivery volumes in teaching and nonteaching hospitals: are volumes sufficient for physicians to acquire and maintain competence? Acad Med. 2014 Jan;89(1):71–6. [PMC free article] [PubMed]
21. Bell RH, Jr, Biester TW, Tabuenca A, Rhodes RS, Cofer JB, Britt LD, et al. Operative experience of residents in US general surgery programs: a gap between expectation and experience. Ann Surg. 2009 May;249(5):719–24. [PubMed]
22. Smith A, Siassakos D, Crofts J, Draycott T. Simulation: improving patient outcomes. Semin Perinatol. 2013 Jun;37(3):151–6. [PubMed]
23. Asch DA, Nicholson S, Srinivas SK, Herrin J, Epstein AJ. How do you deliver a good obstetrician? Outcome-based evaluation of medical education. Acad Med. 2014 Jan;89(1):24–6. [PubMed]
24. Bancroft GN, Basu CB, Leong M, Mateo C, Hollier LH, Jr, Stal S. Outcome-based residency education: teaching and evaluating the core competencies in plastic surgery. Plast Reconstr Surg. 2008 Jun;121(6):441e–8e. [PubMed]
25. Calman KC, Donaldson M. The pre-registration house officer year: a critical incident study. Med Educ. 1991 Jan;25(1):51–9. [PubMed]
26. Kanashiro J, McAleer S, Roff S. Assessing the educational environment in the operating room-a measure of resident perception at one Canadian institution. Surgery. 2006 Feb;139(2):150–8. [PubMed]
27. Iwaszkiewicz M, Darosa DA, Risucci DA. Efforts to enhance operating room teaching. J Surg Educ. 2008 Nov-Dec;65(6):436–40. [PubMed]
28. Singh P, Aggarwal R, Pucher PH, Duisberg AL, Arora S, Darzi A. Defining quality in surgical training: perceptions of the profession. Am J Surg. 2014 Apr;207(4):628–36. [PubMed]