We developed and evaluated the knowledge base and SMART app at a large, multi-specialty, ambulatory academic practice that provides medical care for adults, adolescents, and children throughout the Houston community. Providers utilize Allscripts Enterprise Electronic Health Record (v11.1.7; Chicago, IL) to maintain patient notes and problem lists, order and view results of laboratory tests, and prescribe medications. Providers can manually link medications to a diagnosis within the patient’s clinical problem list, though this feature is seldom utilized. We randomly selected 5000 de-identified patients with an outpatient encounter between July 1, 2010 and December 31, 2010 and at least one active, coded clinical problem and medication. Because the knowledge base depends on RxNorm CUIs and SNOMED CT codes, we included only medications and problems that could be mapped to the respective vocabulary. This study was approved by The University of Texas Health Science Center at Houston’s Committee for the Protection of Human Subjects (HSC-SHIS-10-0238).
The primary design and performance goals for this project aimed to:
- Extract and evaluate a prototype clinical knowledge base for linking medications with clinical problems.
- Utilize the knowledge base to automatically associate a patient’s medications with the clinical problem(s) for which they were prescribed.
- Generate the list of patient problems and linked medications in less than 1 second.
- Employ the SMART architecture to develop a patient summary view generalizable to various EHRs.
Knowledge Base Generation
To facilitate organization of patient medications by clinical problems, we generated a knowledge base from concepts within the UMLS Metathesaurus. depicts the process by which we generated the knowledge. We first extracted medication indication information from NDF-RT using the “may_treat” relationship and mapped terms to SNOMED CT and RxNorm. To account for varying specificity in recording of problems and medications within the EHR, we allowed linking of medications and problems to child concepts within NDF-RT. For example, a “may_treat” relationship exists between “Insulin” and “Diabetes mellitus type 1”, but not between “Insulin” and “Insulin-dependent diabetes mellitus type IA”, a child concept of “Diabetes mellitus type 1.” We inferred relationships using the “isa” relationship between SNOMED CT problems, such that children of problems in a problem-medication pair also linked to the medication. We traversed child concepts for each problem until no new relationships were inferred, which occurred after twelve iterations of problem inferences. Our resulting knowledge base contained RxCUI and SNOMED CT Code pairs, which may be mapped to medications and problems within any ONC-approved EHR (15
Generation of knowledge base for linking patient medications with clinical problems using RxNorm, NDF-RT, and SNOMED CT. Dashed line indicates inferred relationship.
SMART App Development
Knowledge Base and SMART App Evaluation
To evaluate the coverage of the knowledge base, we first counted the number of unique medications within RxNorm for which we were able to identify one or more clinical problems within SNOMED CT using NDF-RT relationships. We also calculated the total number of problem-medication links identified within each level of the SNOMED CT hierarchies.
To simulate use of the newly developed clinical knowledge base within an EHR, we evaluated the problem-medication linking system against the 5000 de-identified Allscripts patients using a locally developed display tool and the SMART test patients using the SMART app within the SMART Reference EMR. For each, we calculated the percentage of medications linking to a clinical problem within the patient’s record. We measured the execution speed of the linking system on problem-medication links generated for a subset of 1000 randomly selected patients with at least three clinical problems and five medications, calculating the mean time to process each medication on the patient’s list.
To assess the accuracy of the problem-medication linking knowledge base, we first calculated the number of automatically linked problems and medications that matched links manually entered by ordering providers within Allscripts. We then randomly selected a subset of 25 Allscripts patients with at least three clinical problems and five medications documented in the EHR for expert review. For each problem-medication pair, a board-certified internist (AL) determined whether the problem was an appropriate indication for the medication. Based on these results, we calculated the estimated sensitivity and specificity of the problem-medication linking system.