Several key findings should be discussed in relation to our project and results to date. First, our work on ELRs demonstrates that by using novel data aggregation techniques, it is possible to create value out of existing accessible data. The value of these enhanced views of information depends on a well-developed health information exchange in order to deliver relevant information to providers at the right time. Most importantly, this information is specifically targeted and relevant to providers who do not have EMRs and who are simply signed up to receive their results through the DOCS4DOCS clinical messaging program.
As we developed and implemented the ELR processor we discovered challenges and complexities along the way. In the mapping and coding steps of the ELR Test and Drug sets into the Regenstrief concept dictionary, it became apparent that certain ostensibly important terms did not already exist in the dictionary. For example, when we initially conceived of the associated child elements for the CREATININE Drug Set, we included Biguanides, Sulfonylureas, Alpha-glucosidase inhibitors, and Thiazolidinediones. When these terms were being coded in, however, we discovered that the concept dictionary only had one existing less granular grouping for these types of medications: “Oral Hypoglycemics.” Other culprits included medications that involved multiple classes of drugs in combination such as combination antihypertensives (ie, Thiazide/β-blocker), oral hypoglycemics (ie, Sulfonylurea/Biguanide), and cholesterol medications (ie, HMG-coenzyme A Reductase Inhibitors/Niacin). The hierarchical organization of dictionary terms is a complex issue. This gets to how dictionaries evolve over time—we know that dictionaries need to evolve gracefully,13
but this rarely happens in the real world. The complexities of maintaining dictionaries become apparent when they are utilized for use cases such as ELRs. Efforts are under way to further characterize the current granularity and structure of dictionary terms.
Naturally, more tests will need to be enhanced in addition to the 10 we have started with. Previous work has examined the frequency of tests ordered through all institutions in the INPC: 784 tests accounted for 99% of the volume from all institutions; 244–517 observation codes representing 99% of the volume at each institution also captured all results for more than 99% of the patients at those institutions.12
This work provides a viable and realistic strategy for deciding on which laboratory studies to enhance. If we, for example, targeted those same 244–517 tests to enhance, we would be covering all tests for more than 99% of all patients in the INPC. We naturally will be cognizant of the fact that not all reports would necessarily benefit from enhancement.
Some limitations of this work deserve mention. This paper focuses on the development of the ELR and has a minimal evaluation component. The work is thus preliminary, and results after implementation, especially on quality of care, are needed. The number of enhanced reports so far is small, and the number of providers surveyed is also small. ELRs depend heavily on the well-developed health information exchange and clinical messaging capability, and this might limit the generalizability of our approach.
Looking ahead, we plan to carry out further phases of this project by evaluating a larger cohort of live-implementation ELR users, and increasing the robustness and intelligence of the G-CARE rules into later versions of ELRs. These will likely take the form of rules which incorporate more data elements into the recommendation trees. In this phase, we will attempt to measure impacts to the clinical care process as a result of ELR implementation, specifically looking at time saved during medical decision-making and effect on provider actions when results are delivered to them in an ELR format as opposed to the traditional report with no enhancements.