PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Plast Reconstr Surg. Author manuscript; available in PMC 2010 September 1.
Published in final edited form as:
PMCID: PMC2746659
NIHMSID: NIHMS139759

Archie Cochrane and his vision for evidence-based medicine

Hriday M. Shah, BS, Medical Student1 and Kevin C. Chung, MD, MS, Professor of Surgery2

Abstract

Archibald (Archie) Cochrane's most influential mark on healthcare was his 1971 publication, “Effectiveness and Efficiency.” This book strongly criticized the lack of reliable evidence behind many of the commonly accepted healthcare interventions at the time. His criticisms spurred rigorous evaluations of healthcare interventions and highlighted the need for evidence in medicine. His call for a collection of systematic reviews led to the creation of The Cochrane Collaboration. Archie Cochrane was a visionary person who helped lay down much of the foundation for evidence-based medicine. This paper will introduce evidence-based medicine to Plastic Surgery by tracing its history to the seminal efforts by Archie Cochrane.

Keywords: Archie Cochrane, Cochrane Collaboration, Evidence-based medicine, Archibald Cochrane

The American healthcare system is at a crossroads. Healthcare costs are weighing heavily on the American people with costs totaling $2.1 trillion (or $7,026 per person) in 2006, and consuming approximating 16% of GDP.1 It is particularly disturbing that healthcare costs are predicted to comprise 19.5% of American GDP by 2017.2 In spite of these staggering expenditures, 15-20% of Americans remain uninsured and emergency rooms across the country are strained (Fig. 1).3-5 It comes as no surprise that many Americans consider health care reform an issue of utmost priority.6 As a result, the political arena has been dominated by discussions regarding health care reform, as legislators work with leading experts in the healthcare field to increase coverage and reduce health care costs. Recently, presidential hopefuls have proposed using evidence-based medicine (EBM) as a method to curb expenditure and improve health care delivery.7,8 This movement towards using current evidence to guide medical decisions can be referred to as the “fourth revolution” of health care (the first three being the introduction of health insurance, the backlash of payers to reduce costs, and outcomes-based research).9,10 To understand how EBM gained such a strong foothold in the American mind frame, it is important to understand its origins and to become familiar with the Cochrane Collaboration, a testament to the EBM revolution in today's society.

Figure 1
Historical and projected national healthcare expenditures. Data obtained from the National Health Expenditure Accounts.5

What is EBM?

Sir William Osler once noted, “The practice of medicine is an art, based on science. Medicine is a science of uncertainty and an art of probability.”11 The uncertainties of medicine remain prevalent today in spite of great advances in research and technology. Various studies have found significant variation of care amongst practitioners.12-14 This is because many practitioners follow a “logical” treatment plan based on their understanding of the pathophysiological process.15 However, what may seem like obvious logic is often flawed and causes unnecessary patient morbidities. This was at the root of the recent controversy surrounding rofecoxib (Vioxx). By selectively inhibiting cyclooxygenase-2, it was thought that this drug would reduce the adverse gastrointestinal effects of inhibiting cyclooxygenase-1. Unexpectedly, outcomes studies determined that rofecoxib significantly increased the risk of cardiovascular complications and eventually led to the withdrawal of the drug.16 The goal of EBM is to solidify the scientific foundation of medicine and to reduce uncertainties in medical decision-making. It is the sensible use of the best available external evidence from systematic research assimilated with individual clinical expertise to evaluate the best course of care for a patient (Fig. 2).17 There are several subtle, key points to note from this definition. First, clinical evidence and clinical experience are not mutually exclusive in this definition. In other words, EBM is not meant to diminish the importance of the “art of medicine.” Second, in order to use the best available external evidence, the physician must remain educated on current relevant research. Finally, it is important to remember that EBM seeks to discover “the best course of care for a patient,” a principle integral to medicine long before the dawn of EBM. Therefore, it is important for the physician to vigilantly assess the course and effects of the intervention and use clinical acumen to make the necessary adjustments.

Figure 2
Three quintessential components of EBM.

The search for quality evidence can be difficult and time-consuming for physicians, whose lives are often already hectic. The search is further complicated by the remarkable increase in the number of published papers and the vast amount of resources available to physicians since the advent of the internet.18 Often, multiple studies suggest contradictory interventions and it can be very difficult to sort through the evidence and synthesize a clear-cut plan of treatment. The existence of publication bias with regards to the direction and strength of study findings makes matters worse, even for the most meticulous physician.19 These problems can be addressed by resorting to systematic reviews, which unlike traditional reviews, adhere to reproducible methods and recommended guidelines.20 The Cochrane Database of Systematic Reviews is regularly updated and maintained by the Cochrane Collaboration, an organization at the forefront of the EBM revolution.

The Cochrane Collaboration

The Cochrane Collaboration was founded in 1993 by Ian Chalmers and a group of 70 other international colleagues.21-23 The goal of the Cochrane Collaboration is to create and disseminate up-to-date review of RCTs of healthcare interventions in order to help health care professionals make informed decisions. In the last fifteen years, the organization has enjoyed tremendous growth and has transformed into an international conglomeration of 11,500 people in over 90 countries.24 This makes it the world's largest organization committed to preparing systematic reviews to facilitate medical decision making.25 The increase in organizational size has paralleled the increase in healthcare professionals' increasing need to utilize evidence-based medicine.

The Cochrane Collaboration is most known for its publication of the The Cochrane Library. A key component of the library is Cochrane Database on Systematic Reviews, a collection of reviews and protocols maintained by review groups. Each review is designed to answer a specific question such as “For distal radius fractures, how effective is external fixation as a surgical intervention compared with a conservative intervention such as casting?” The database has increased from less than a hundred reviews in 1995 to over 6,300 reviews currently on topics ranging from fertility regulation to lung cancer. The reviews take into account unpublished and negative results to reduce publication bias. The review groups specify strict inclusion criteria that give priority to Level I clinical evidence (randomized control trials), giving the review higher credibility. The final review is published in a standardized structured format that makes it simple for the physician to understand the evidence regarding the healthcare intervention and apply it to daily practice. In addition, the library also provides (1) a summary on reviews not created by the Cochrane Collaboration, (2) bibliographic data on reports of controlled trials, (3) bibliographic data on reports of methods used when conducting controlled trials, (4) health technology assessments, and (5) cost-benefit information on healthcare interventions. In retrospect, the Cochrane Library owes much of its success to its electronic mode of publication.

It was realized early that a printed database would be outdated almost as soon as it was printed. Thus, the decision was made to publish electronically.23 Publishing electronically has proven to be a great benefit for the Cochrane Collaboration. Today, The Cochrane Database of Systematic Reviews is published quarterly with the latest Cochrane Reviews on a CD-ROM and the internet. Although the abstracts are available to everyone for free on the internet, full text articles require a subscription. However, due to the increasing need for EBM, there has been a growing trend for countries to have a national subscription to the database, allowing everyone in those countries full access.24 Studies have shown that Cochrane reviews are consistently of equal quality or better than those published in print journals. They have the added advantage of being updated more frequently, thus making them optimal for EBM.26

Furthermore, electronic publishing allows for contribution from more 10,000 members across the world. The library is arranged similar to a wiki in that reviewers can make additions based on their enthusiasm and the reviewers are not compensated financially for their services. It has been a point of criticism that reviews are added based on reviewer preference and not population health needs. However, it is a credit to the reviewers that the top 10 causes of disability in both the developed and developing countries have pertinent reviews available in The Cochrane Library.27 The success and productivity of the collaboration is matched by that of its namesake, Archie Cochrane.

Archie Cochrane

Archie Cochrane (Fig. 3) is perhaps best known today for the publication of his 1971 monograph “Effectiveness and Efficiency” in which he wrote a scathing criticism on the state of medicine in Britain. While this was certainly his most famous accomplishment, it was but a small part of Cochrane's life. He was born on January 12th, 1909 in Galashiels, Scotland to a wealthy family. He was born with porphyria, a condition that played a significant role in the development of his professional views. After attending a preparatory school, Cochrane earned a scholarship to Uppingham School and later to King's College at Cambridge. Cochrane's life was a meandering one, which included psychoanalysis received in Germany for a sexual condition, service in two wars, being a prisoner of war (POW), and even studying the epidemiology of tuberculosis in Philadelphia, PA, USA.

Figure 3
Archie Cochrane in 1980s.

Archie Cochrane was a staunch supporter of randomized control trials (RCTs) and spent much of his career promoting their use in research. Someone once asked Archie Cochrane how far he was prepared to take this randomizing game. He famously replied, “You should randomize till it hurts.” 28 To understand the essence of Cochrane's statement, it is necessary to understand what RCTs are. An RCT, at its most elementary level, involves assigning patients to either the experimental group or the control group by using some method independent of human influence. Examples of such a method include using a computerized random number generator or even rolling a dice. By assigning subjects arbitrarily to groups, the subject characteristics are randomized between the two groups, and it is possible to experimentally determine whether the experimental group benefited more from the treatment than the control group. The Centre for Evidence-Based Medicine (CEBM) rates RCTs as level I clinical evidence, the highest level.29 RCTs, when available, form the backbone of EBM because they provide the best evidence. Archie Cochrane championed using RCTs throughout most of his professional career and his support for their use culminated in the publication of his 1971 book, “Effectiveness and Efficiency.” However, the ideas Cochrane articulates in his well-known monograph originated some forty years after he graduated from King's College at Cambridge.

After graduation in 1931, he became a research student with Dr. N. Wilmerat, Strangeways Laboratory, Cambridge, working on tissue culture studies. He soon became disenchanted with work and faced a personal problem with anejaculation, which he attributed later in his life to porphyria.30 It has been suggested recently that Cochrane's sexual problems did not have porphyria as the cause but rather tuberculosis that he suffered in his childhood.31 He left for Germany and studied psychoanalysis under the guidance of Theodor Reik, while at the same time receiving psychoanalytic treatment from Reik. Reik, one of the first students of Sigmund Freud and of Jewish decent, had to leave Berlin due to the political atmosphere of the time and move first to Vienna, and later to Holland. Cochrane followed and began his medical studies during this time. The psychoanalysis proved to be of little help, and Cochrane decided to return to London after becoming convinced that his psychoanalytic treatment lacked a scientific basis. Later in his life, Cochrane condemned the entire field of psychiatry for “using a large number of therapies whose effectiveness has not been proven” and for being “basically inefficient.”28

In London, Cochrane resumed his medical studies at the University College Hospital. The next twenty years were the formative ones in his life. His studies were interrupted by WWII, during which time he enlisted and was captured as a prisoner of war by the Germans. During his time with Theodor Reik, Cochrane had picked up some German and as a result of his linguistic abilities, he was assigned the dual role of a medical officer for the POWs and a negotiator. To his credit, Cochrane remained vigilant about helping the other prisoners in spite of being served only a 600 calories/day diet for extended periods of time. It was during this time that Cochrane performed his first trial. He conducted a non-random trial involving 20 subjects to convince the Germans that a yeast supplement to the prisoner diet would cure the widespread edema amongst them. In spite of the lack of scientific rigor, Cochrane succeeded in convincing the Germans. He later described the venture as his “first, worst, and most successful trial.”32 It was also during this time that Cochrane realized the extent to which unsubstantiated claims plagued medicine. As Cochrane reveals, “I remember at the time reading one of those propaganda pamphlet, considerable suitable for POW medical officers about ‘clinical freedom and democracy.’ I found it impossible to understand. I had considerable clinical choice of therapy: my trouble was that I did not know which to use and when.”28 This frustration planted the seeds for a lifelong journey of trying to distinguish between scientifically valid medical therapies and the invalid ones.

After returning from the war, Cochrane trained as an epidemiologist. His training involved spending a year in Philadelphia and receiving tutelage from Sir Bradford Hill, who pioneered the RCT and was the first to demonstrate a connection between the cigarette smoke and lung cancer. He started his career by studying whether pneumoconiosis progresses to Progressive Massive Fibrosis (PMF) as a result of tuberculosis in coal miners. Although the experiment did not produce conclusive results because the prevalence of tuberculosis fell spontaneously in the control population, Cochrane did manage to achieve response rates close to 100%, and his study was the first to use a whole population in a controlled experiment.30

It was the high response rates that earned Cochrane the reputation of being an excellent epidemiologist. In the sixties, he left the pneumoconiosis research unit to work as a David Davies professor of tuberculosis and chest diseases at the Welsh National School of Medicine. During this time, he also served as the honorary director of the Medical Research Council Epidemiology Unit. Cochrane maintained his expectations of a high response rate during this time. In fact, within his unit, Cochrane used the index “Cochrane units” with a response rate of one Cochrane being 91%, a response rate of two Cochranes being 95%, etc. A response rate of below 90% (negative Cochrane units) was simply unacceptable.30 His reputation enabled him to become professor of tuberculosis at Cardiff in 1960. After nine years here, he left to lead a new Medical Research Council Epidemiology Unit. It was during this time that he was offered a Rock Carling Fellowship by the Nuffield Provincial Hospitals Trust to write a book about evaluating the British National Health Service (NHS).

Effectiveness and Efficiency

In 1971, “Effectiveness and Efficiency”, a monograph less than a hundred pages long, hit the shelves and shook the field of medicine. Cochrane hurriedly wrote most of the first draft for the book within a span of just three hours to meet a publishing deadline.32 He was, therefore, pleasantly surprised at the enormous success of the book. It was written with the target audience being medical students and non-medical intellectuals. In addition to receiving highly favorable reviews, the book was translated into Spanish, French, Italian and Polish.

In a manner appropriate for someone writing on the topic of the biases and inconsistencies in medicine, Cochrane began the book by first revealing his own biases. He used the terms effectiveness, efficiency and equality as his yardstick for evaluating the NHS. Effectiveness was used as a measure of how much a medical activity changes the natural course of a disease in a RCT. Efficiency was used to refer to how well the health care system utilized resources such as doctors, nurses, medical equipment, lodging, etc. to implement an effective medical intervention. Whereas effectiveness and efficiency were used to assess the cure aspect of medical interventions, equality was used to assess care and the variation of care amongst different hospitals.

Cochrane was certainly not the first to question the effectiveness of medical therapies. This prompts the question of what made his book so popular. Cochrane's book was unique because it used an amalgamation of studies to show that the problem of evidence plagued not just one aspect of medicine but was pervasive. He used a variety of medical conditions such as diabetes and ischemic heart disease to illustrate that many therapies used regularly by the medical profession had not been fully tested using a RCT. For example, at the time of the book's publication, tonsillectomies were among the commonest cause of admission of children to hospital. Cochrane suggested that the operation was too often performed without any indication. At the same time, Cochrane showed that in contrast to the vast amount of tonsillectomies performed by doctors, hearing problems of the elderly were largely ignored by doctors. The presentation of one example after another effectively drove home the point that many medical therapies were not based on fundamentally sound evidence.

Cochrane believed that the ultimate duty of medical doctors was to make decisions between alternative therapies based on cost/benefit comparisons. According to him, “these can really only be obtained by an adequately costed RCT.”28 However, even he realized that RCTs did not always provide an unequivocal answer. One can easily imagine a scenario in which different RCTs provide contradictory results. This was the case with RCTs performed to evaluate the benefit of tonsillectomies. Cochrane found that while one RCT found tonsillectomy to be a beneficial procedure with regards to otitis media, another RCT found no associated benefit. Cochrane realized that clinicians would not find a litany of RCTs helpful and would not necessarily know which trial was the best. This led Cochrane to reach the conclusion that “It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized control trials.”33 His call for an organized database of RCTs served as an impetus for the formation of the Cochrane Collaboration and ultimately for the creation of EBM.

Evidence-Based Medicine and Surgery

Even though the role of EBM has become increasingly important in healthcare, its relationship to surgery has been tenuous. It was estimated that 30-50% of general medicine decisions are based on RCTs, only 10-20% of surgical decisions are based on RCTs.34 The process of translating evidence into clinical practice is fraught with difficulties and poses special problems for the field of Surgery. Much of this is because of a lack of sound evidence. There is a scarcity of Level I clinical evidence because RCTs are quite difficult to conduct to test surgical interventions.35,36 In fact, RCTs, even when conducted properly, are not infallible. For example, the inclusion criteria for RCTs dramatically change the practicality of the study. RCTs with stringent inclusion criteria have limited population-wide applicability, and RCTs with liberal inclusion criteria lose sensitivity. Moreover, it is not possible to test many surgical interventions through the use of RCTs. An examination of surgical literature revealed that only 40% of surgical interventions could possibly be studied with RCTs.37 Often, as a result of these restrictions, physicians resort to conducting studies less stringent than RCTs such as cohort studies, case-controlled studies, etc. The data from these studies have a tendency to produce artifacts, which lead physicians to “overconclude” and prescribe an inappropriate intervention.38,39 In spite of these shortcomings, EBM has benefitted surgery in many ways.40,41 If nothing else, even the sternest critic will agree that it has raised awareness of the importance of evidence and highlighted the shortcomings of many commonly performed surgical interventions. Furthermore, it allows the surgeon of today to avoid unnecessary guessing. For example, a surgeon considering whether or not pre-operative hair removal prevents surgical site infections can simply reference the Cochrane Library to discover what the latest research shows.

Like other surgical fields, the field of Plastic Surgery has also suffered from a dearth of level I clinical evidence. This was recently illustrated in a survey of articles published in Plastic and Reconstructive Surgery in the years 1983, 1993, and 2003. A large portion of published studies (86.9%) remained uncontrolled in 2003. However, there was a demonstrated increase in controlled studies (7.2% of studies in 1983 versus 13.7% of studies in 2003), and the number of RCTs increased from 0 in 1983 to 7 in 2003.42 Due to the inherent difficulty associated with conducting RCTs in Surgery, it is unlikely that Plastic Surgery will ever enjoy having many level I clinical evidence found in other fields. As health care costs continue to rise, there will be an increasing trend towards the incorporation of EBM into practice to justify what are often costly medical interventions. Organizations such as the American Society of Plastic Surgeons (ASPS) have presented innovative publications to communicate up-to-date guidelines to physicians and to spur evidence-based practice.9

It is difficult to determine exactly the extent to which Cochrane's legacy will affect Surgery in the coming decades. Certainly, an interplay of medical, economic and political factors will play a large role in determining EBM's future. Cochrane fought ardently for his ideals, regardless of whether they were about politics or healthcare. His self-written obituary, published after he passed away in 1988 ended as follows: “He was a man with severe porphyria who smoked too much and was without consolation of a wife, a religious belief, or a merit award—but he didn't do so badly.”

Acknowledgments

Supported in part by grants from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01 AR047328), a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120), and an Exploratory/Developmental Research Grant Award (R21AG030526) (To Dr. Kevin C. Chung)

Footnotes

Disclosure: None of the authors has a financial interest in any of the products, drugs or devices mentioned in this manuscript.

Reference List

1. Catlin A, Cowan C, Hartman M, et al. National health spending in 2006: a year of change for prescription drugs. Health Aff (Millwood) 2008;27:14–29. [PubMed]
2. Keehan S, Sisko A, Truffer C, et al. Health spending projections through 2017: the baby-boom generation is coming to medicare. Health Aff (Millwood) 2008;27:w145–55. [PubMed]
3. The Number of Uninsured Americans is at an All-Time High. Center on Budget and Policy Priorities. [June 02, 2008]. Available at: http://www.cbpp.org/8-29-06health.htm.
4. Kellermann AL. Crisis in the emergency department. N Engl J Med. 2006;355:1300–3. [PubMed]
5. National Health Expenditure Data Historical. Centers for Medicare & Medicaid Services. [June 16, 2008]. Available at: http://www.cms.hhs.gov/NationalHealthExpendData/03_NationalHealthAccountsProjected.asp.
6. The Henry J. Kaiser Family Foundation; [June 02, 2008]. Kaiser Health Tracking Poll: Election 2008 -- April 2008. Available at: http://www.kff.org/kaiserpolls/h08_pomr042908pkg.cfm.
7. Healthcare. Obama '08. [June 02, 2008]. Available at: http://www.barackobama.com/issues/healthcare/
8. American Health Choices Plan. Hillary for President. [June 02, 2008]. Available at: http://www.hillaryclinton.com/issues/healthcare/americanhealthchoicesplan.pdf.
9. Chung KC, Ram AN. Evidence-based medicine, the fourth revolution in American medicine? Plast Reconstr Surg. In Press. [PubMed]
10. Relman AS. Assessment and accountability: the third revolution in medical care. N Engl J Med. 1988;319:1220–2. [PubMed]
11. Osler W, Silverman ME, Murray TJ, et al. The Quotable Osler. Philadelphia: American College of Physicians--American Society of Internal Medicine; 2003.
12. Asch SM, Kerr EA, Keesey J, et al. Who is at greatest risk for receiving poor-quality health care? N Engl J Med. 2006;354:1147–56. [PubMed]
13. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–45. [PubMed]
14. Steinberg EP. Improving the quality of care--can we practice what we preach? N Engl J Med. 2003;348:2681–3. [PubMed]
15. Sackett DL, Rosenberg WM. The need for evidence-based medicine. J R Soc Med. 1995;88:620–4. [PMC free article] [PubMed]
16. Waxman HA. The lessons of Vioxx--drug safety and sales. N Engl J Med. 2005;352:2576–8. [PubMed]
17. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312:71–2. [PMC free article] [PubMed]
18. Davis Sears E, Burns PB, Chung KC. The outcomes of outcome studies in plastic surgery: a systematic review of 17 years of plastic surgery research. Plast Reconstr Surg. 2007;120:2059–65. [PubMed]
19. Johnson RT, Dickersin K. Publication bias against negative results from clinical trials: three of the seven deadly sins. Nat Clin Pract Neurol. 2007;3:590–1. [PubMed]
20. Margaliot Z, Chung KC. Systematic reviews: a primer for plastic surgery research. Plast Reconstr Surg. 2007;120:1834–41. [PubMed]
21. The Cochrane Collaboration. Chronology of the Cochrane Collaboration. [June 04, 2008]. Available at: http://www.cochrane.org/docs/cchronol.htm.
22. Starr M, Chalmers I. The Evolution of the Cochrane Library, 1988-2003. Update Software. [June 04, 2008]. Available at: http://www.update-software.com/history/clibhist.htm.
23. Levin A. The Cochrane Collaboration. Ann Intern Med. 2001;135:309–12. [PubMed]
24. New Comer's Guide. The Cochrane Collaboration. [June 04, 2008]. Available at: http://www.cochrane.org/docs/newcomersguide.htm.
25. Clarke M. The Cochrane Collaboration and the Cochrane Library. Otolaryngol Head Neck Surg. 2007;137:S52–4. [PubMed]
26. Jadad AR, Cook DJ, Jones A, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998;280:278–80. [PubMed]
27. Grimshaw J. So what has the Cochrane Collaboration ever done for us? A report card on the first 10 years. CMAJ. 2004;171:747–9. [PMC free article] [PubMed]
28. Cochrane AL. Effectiveness and efficiency: random reflections on health services. London: Nuffield Provincial Hospitals Trust; 1973.
29. Phillips B, Ball C, Sackett D, et al. Levels of evidence and grades of recommendation. Center for Evidence-Based Medicine. [June 13, 2008]. Available at: http://www.cebm.net.
30. Cochrane AL, Maynard A, Chalmers I. Non-random reflections on health services research: on the 25th anniversary of Archie Cochrane's Effectiveness and Efficiency. London: BMJ Publishing Group; 1997.
31. Macleod S. Cochrane's problem: psychoanalysis and anejaculation. Australas Psychiatry. 2007;15:144–7. [PubMed]
32. Cochrane A. One Man's Medicine. London: BMJ Publishing Group; 1989.
33. Cochrane A. 1931–1971: a critical review, with particular reference to the medical profession. Medicines for the Year. 2000;1979:1–11.
34. Sauerland S, Lefering R, Neugebauer EA. The pros and cons of evidence-based surgery. Langenbecks Arch Surg. 1999;384:423–31. [PubMed]
35. Maier RV. What the surgeon of tomorrow needs to know about evidence-based surgery. Arch Surg. 2006;141:317–23. [PubMed]
36. Offer GJ, Perks AG. In search of evidence-based plastic surgery: the problems faced by the specialty. Br J Plast Surg. 2000;53:427–33. [PubMed]
37. Solomon MJ, McLeod RS. Should we be performing more randomized controlled trials evaluating surgical operations? Surgery. 1995;118:459–67. [PubMed]
38. Naylor CD. Grey zones of clinical practice: some limits to evidence-based medicine. Lancet. 1995;345:840–2. [PubMed]
39. Feinstein AR, Horwitz RI. Problems in the “evidence” of “evidence-based medicine” Am J Med. 1997;103:529–35. [PubMed]
40. Chung KC, Rohrich RJ. Measuring Quality of Surgical Care: Is It Attainable? Plast Reconstr Surg. In Press. [PubMed]
41. Chung KC, Shauver MJ. Measuring quality in healthcare and its implications for pay-for-performance initiatives. Hand Clinics. In Press. [PMC free article] [PubMed]
42. Loiselle F, Mahabir RC, Harrop AR. Levels of evidence in plastic surgery research over 20 years. Plast Reconstr Surg. 2008;121:207e–11e. [PubMed]