Ted was fortunate to arrive at Stanford at a time when Joshua Lederberg, Edward Feigenbaum, Bruce Buchanan, and others were exploring the development of artificial intelligence approaches to medicine and molecular biology. He came to Stanford as an M.D.-Ph.D. student, one of a very select group of young scientists admitted into that program. His research advisor was Stan Cohen, probably because Stan was one of the few clinical faculty in the Medical School working with computers. Although Stan went on to invent gene splicing, his work at the time involved developing a computer program that could reference a large data base of drugs and identify the possibly deleterious effects of administering combinations of them. 1
Gio Wiederhold was working with Stan on database development and Bruce Buchanan was involved on the artificial intelligence side.
Ted’s undergraduate experience working with MUMPS in the MGH LCS instilled in him good program design principles and considerable programming skills. At Stanford, he enrolled in a computer science course, Introduction to Artificial Intelligence (AI), taught at the time by Cordell Green and Jerome Feldman. Ted started spending time in the Dendral project lab (the Heuristic Programming Project), run by Feigenbaum, Lederberg, and Buchanan, to find out more about AI.
At the lab, Ted noticed there was no regular journal club, so he organized one, with a focus on journal articles about innovative uses of computers in medicine. Patterned after the ACM’s special interest groups–the SIGs–the lunch-time journal club was called “SIGDoc.” Years later the scope widened beyond medicine (so the name changed to “SIGLunch”) and it became the longest running informal colloquium devoted to AI topics—an early indication of Ted’s perceiving a need and finding a way to fill it.
The first research project Ted and Stan Cohen sketched out was to develop a computer program that could help clinical practice review boards decide when a clinician had made gross errors in prescribing drugs. Quickly, however, they saw the wisdom of offering advice to clinicians before they made errors. A program that only criticized physicians after the fact was sure to be itself the subject of much criticism. But in order for a program to know that a drug was inappropriate, it first had to decide which one(s) would be appropriate. This was the origin of his Ph.D. research problem, for which Bruce Buchanan became Ted’s unofficial advisor.
Cordell Green suggested that Ted do his programming in Lisp, and got him set up with an account at Stanford Research Institute (SRI) to work over the ARPANet, the forerunner of the Internet, so he could work with the new, interactive version of Lisp (which became InterLisp) developed at Bolt, Beranek, and Newman, Inc. Cordell’s class had also introduced Ted to Carbonell’s program SCHOLAR, which answered questions about the geography of South America based on facts stored in a large semantic network. So the first prototype of the program that was to become MYCIN was designed as a semantic net, question-answering machine. He ran experiments on that early program and concluded that spreading activation through a semantic net was not a precise enough reasoning mechanism for clinical decision making. Thus, the design of MYCIN was revised to build on the production rule approach used by Dendral.
Production rules were known to be useful in compiler design, and were being used at Carnegie Tech (now Carnegie Mellon University) to build psychological simulations of human reasoning. The Stanford group believed that systems of production rules could encode the knowledge of experts and drive a simple reasoning system for reasoning about complex problems. A part of the Dendral project had demonstrated its feasibility. Ted was eager to try using production rules to encode Cohen’s knowledge about drug use in clinical medicine, using a cleaner design than Dendral’s. Ted and Stan wisely decided that antimicrobial therapy for bacteremia was sufficiently complex.
“It was clear from the beginning that Ted had a remarkable insight into both what was needed in biomedical computing and for what was needed in clinical medicine. MYCIN at that time was an especially remarkable accomplishment for a young graduate student, also a medical student, spending his time in medical studies and also spending much of his time writing the algorithms and having the discussions necessary in order to bring MYCIN to fruition.”
— Stanley Cohen, Professor, Stanford University School of Medicine
“Ted was the brilliant kid from their yard, the Stanford medical school, who came to play in our yard. He demonstrated to us with his Ph.D. thesis on MYCIN how to do expert systems another way. So thanks Ted for everything you were able to do for us at Stanford over all of those years…it was really wonderful.”
— Edward Feigenbaum, Chair, Computer Science at Stanford, 1970s
“My years at Stanford were kind of magical…I was in a wonderful supportive environment. My Ph.D. advisor was Stan Cohen, who’s best known for gene splicing and his work as a geneticist…and I think he instilled in me a real belief that biomedical informatics and what I wanted to do was as rigorous scientifically and as important as anything I might do with test tubes in a laboratory. I’ve tried to maintain that philosophy not only in my own work but in the way I’ve trained students ever since.”
— Ted Shortliffe
At the start, Ted and colleagues at Stanford believed that all the relevant medical knowledge about bacteremia and antibiotics could be encoded in production rules. Friends at MIT took exception to this, arguing in many spirited discussions that frame-based representations were necessary (and sufficient). The truth turned out to be that both approaches could work, and MYCIN encoded a substantial amount of its knowledge about clinical medicine in class-instance hierarchies and property lists. Overall, however, the reasoning in MYCIN was simple backward chaining through the production rules, starting with the goal rule, which was (more or less): “If the patient’s infection is significant and the cause of the infection is known, then prescribe the drugs effective against the cause of the infection.” 2
One problem that consumed more time than any other in the design of MYCIN was how to determine the strength of belief when the evidence for a belief is not known with certainty and the inference rules themselves are less than certain. Bayes’ Theorem was known to the group, as was the work by Gorry and Barnett 3
that showed how Bayes’ Theorem could be used sequentially through a chain of inferences. But Bayesian reasoning required numbers for both prior and conditional probabilities that the project’s clinicians could not provide. There were too many widely variable estimates for the probabilities, and the clinicians didn’t know all of them in any case. What they could give, however, was a single number for each rule: If you knew the facts mentioned in the antecedent with certainty, how much more would you believe the fact(s) mentioned in the conclusion (on a 1–10 scale).
MYCIN’s introduction of certainty factors (CFs) and a CF-combining calculus initiated a new set of concerns about reasoning under uncertainty, 4
which has become a substantial focus of AI research. Although Heckerman and Horwitz later showed 5
that there was a probabilistic interpretation of this number—as a probability update—the strength of belief (called a CF) was conceived as a degree of confirmation and not a probability.
Ted had a self-imposed deadline of two years for his Ph.D. research and dissertation. 6
Because MYCIN required solving new problems in clinical decision making, such as managing uncertainty, he finished in two years and a summer—just before classes began in the fall. He could not take the summer off, as he had planned.
Although it was never deployed outside of the experimental setting, MYCIN was shown in a 1979 evaluation to equal or to outperform members of the Stanford infectious disease faculty on sample cases (see ). The MYCIN work has been one of the most heavily cited advances in our field.
Sample MYCIN consultation session.
One year following his 1975 Ph.D. degree, in 1976, Ted obtained his M.D (see ), and received the Grace Murray Hopper award of the Association of Computing Machinery (ACM). The ACM gives the Hoppper award to a distinguished scientist under age 30 for important contributions to the field.
Edward H. Shortliffe, MD, PhD, at Stanford graduation, 1976.
Ted’s work on MYCIN clearly demonstrated the potential of using production systems for encoding knowledge of experts and bringing it to bear on new problems. Randall Davis and William van Melle, two computer science students at Stanford, drove home the point that MYCIN-style systems could be used in domains outside of medicine. Van Melle’s dissertation research removed all remaining references to medicine in the code, leaving a “shell” system called EMYCIN. This was the origin of the concept of expert system shells in which domain-specific knowledge is cleanly separated from domain-independent representation and reasoning mechanisms. Davis expanded the explanation system in MYCIN and showed the power of having the system explain its own reasoning—both for understanding the rationale behind a recommended action and for determining the sources of errors in the knowledge base.
In the decade after Ted finished his dissertation, at least five other Ph.D. dissertations at Stanford directly built upon his work, and numerous other projects were spawned from it. A number of Stanford colleagues, including Ted, started the first expert systems company. Rule-based expert systems based on MYCIN and EMYCIN are widely used today in a broad spectrum of domains and applications areas. †
Because of the wide interest that MYCIN had attracted, Ted suggested (while he returned to Boston and MGH to serve as a medical intern) writing a book based on the published papers of the students and visiting researchers who had worked on MYCIN. 6
What seemed like a simple task stretched on for several years. It included twenty-four published papers and twelve new chapters summarizing the experimental work and lessons learned from MYCIN.
In the Foreword to the MYCIN book, Allan Newell wrote that the emergence of expert systems transformed the field of AI and that MYCIN is “the granddaddy of them all—the one that launched the field.”
“They separated the program from the knowledge base, they separated the reasoning from the output, they required the program to be able to justify their conclusion. All of this was very transparent. And that approach that he brought as a medical guy to the computer science department really contributed tremendously to the whole founding of the field of medical informatics. … The first time I met him I was introduced to him by Ed Feigenbaum who said, ‘Don I want you to meet Ted Shortliffe; he’s really brilliant’. I thought of Ed Feigenbaum in those terms and he didn’t use that terminology very often for anybody else, but he was right in this case…and I’ve enjoyed knowing Ted every day since those early days.”
— Donald A.B. Lindberg, Director, National Library of Medicine
Ted’s career at Stanford was interrupted only by his one-year internship back at MGH before returning for a residency in Internal Medicine at Stanford. There, he joined the faculty in the Departments of Medicine and Computer Science in 1979. He worked closely with Bruce Buchanan (see ) and other members of the Stanford Computer Science Department, and initiated the NLM-sponsored Stanford Training Program in Medical Informatics. In 1982, Ted established the Section on Medical Informatics (SMI) in the Stanford University School of Medicine’s Department of Medicine, and would continue to call Stanford home for the next two decades. Under Ted’s leadership the SMI grew to become one of the premier medical informatics laboratories in the world (See ).
Ted Shortliffe and Bruce Buchanan as young Stanford faculty members, circa 1979.
At Stanford in the 1980s, the MYCIN program led to the ONCOCIN Project, which was supported in part by major grants from the National Library of Medicine. The ONCOCIN project—deliberately designed to be challenging as well as clinically useful—was Ted’s first major project after joining the Stanford faculty and was an active effort from 1979 to 1987. 8, 9
“The domain we chose was the field of clinical oncology, in particular the management of patients enrolled in chemotherapy protocols.”
“One of the key moments in the development of expert systems here at Stanford … was a presentation, in 1986, by Ted to the Xerox Palo Alto Research Center. It was one of the first demonstrations of medical expert systems using graphical user interfaces, and even though the people at Xerox had developed the graphical user interface, they had never seen the kind of interfaces that we were showing. It was really a key milestone in the development of the ONCOCIN system, which followed on Ted’s early work in MYCIN.”
—Larry Fagan, Faculty Member, Stanford
The ONCOCIN problem was, indeed, challenging. The clinical goal was to tailor oncology therapy to individual patients within the guidelines of experimental oncology protocols, thus keeping them on protocol as long as possible. The AI and informatics goals came to be understood, inter alia, as representing the time-course, including loops, of clinically significant events in the protocols, and representing the variability and uncertainty implicit in the protocols. The oncology protocols themselves were complex and were essentially paper documents. The discipline of representing them for ONCOCIN occasionally revealed inconsistencies and holes. ONCOCIN, like MYCIN, spawned several other Ph.D. dissertations.
The SUMEX-AIM computer resource was established in 1973, when Ted was still a student. Funded by the Division of Research Resources of the NIH (which later became the National Center for Research Resources), SUMEX-AIM provided computational resources to investigators at Stanford. From the outset, the resource also extended these capabilities to the external investigator community via early Internet access. Nobel Laureate Joshua Lederberg was the first SUMEX-AIM principal investigator (PI), and when he left for Rockefeller University, Ed Feigenbaum took over as PI. 10
When Ted joined the faculty, Ed made Ted the PI, and this arrangement continued through the last nine years of the project (out of 18). Ted then initiated, as PI, the NLM-funded CAMIS grant, which was the successor to SUMEX and continued for another five years.
Ted’s initial research on MYCIN (and the continued work on DENDRAL) were part of the justification for the initial SUMEX proposal. The SUMEX computer was a DEC-10 (later a DEC-20) running LISP, which became the “home machine” for Ted’s work for the next 18 years as well as the shared resource of several other biomedical computing groups around the country. In the 1980s it was the pioneering SUMEX systems staff, under Tom Rindfleisch’s direction, that helped SMI to bring in individual LISP machines and graphical workstations. It’s important to note that SUMEX was the first non-DOD machine to be connected to the ARPAnet.
“Part of his skill is being articulate, but also it involves building an environment where smart people want to come, where new ideas happen and are valued and pursued. Ted has been extremely effective and generous of his time in helping to enunciate and shape informatics policy at local, national, and international levels.”
— Tom Rindfleisch, Senior Associate Dean for Research emeritus, Stanford University
“Speaking personally, my collaboration with Ted was always intense and interesting. Our few disagreements engendered lively discussion and not once left either of us feeling anything less than renewed respect. There were many priceless years of collaborative work on MYCIN while he was a student and then again after Ted returned to Stanford as a faculty member. In his student years we were merely predicting that he would be a star; but as we saw his leadership and organizational skills at work nationally and internationally, we could see our predictions come true. I continue to value his opinions on every matter, and value his friendship even more.”
— Bruce Buchanan