With the advent of the World Wide Web, numerous online health information resources have become available. ⁹ These resources have demonstrated great potential to resolve many of the physicians' information needs, especially those related to knowledge gaps. ^10,11 Yet, a number of barriers, notably lack of time, lack of seamless access to resources, resource deficiencies, and clinicians' suspicions that an answer does not exist or cannot be easily found, hinder more frequent and efficient attempts to fulfill information needs that arise at the point of care. ^5,8,9

A subset of the approaches to reducing those barriers and increasing the use of information resources during the course of patient care rely on the theory that the context of a particular problem dictates workers' information needs. ¹² It would follow that information needs can be predicted in the context of an interaction between a clinician and a computer. “Infobuttons” are information retrieval tools that use the context of this interaction to predict the most likely information needs and to offer a list of links to resources and content topics that may fulfill them. ^13–15 Several healthcare organizations have implemented infobuttons in their electronic medical record (EMR) systems. Assessments of these implementations have shown that infobutton users were able to quickly answer their questions and that these answers positively affected patient care decisions. ^16–18

Infobuttons are typically implemented using a software component called an “Infobutton Manager.” ¹⁵ Infobutton Managers have a knowledge base that represents both the types of information needs that are most likely to arise in a particular context, and the available information resources that could fulfill them. ^14,16 When an infobutton is clicked, context attributes are sent to the Infobutton Manager. Examples of context attributes include the clinical data element that is associated with the infobutton (e.g., a medication, a problem list item), the task or action that a user is performing in the EMR (e.g., medication order entry, laboratory results review), and data about the infobutton user (e.g., discipline) and the patient (e.g., age, gender). The Infobutton Manager responds with a web page that contains links to information resources that may be relevant in the particular context (). These links send a request containing a set of context parameters that are used by the target information resource to retrieve potentially relevant content. These links may also indicate the specific question or content topic in which the user is interested (e.g., dose of drug X as opposed to just drug X). In this report, we refer to the latter links as “topic links” and to those that do not specify a topic as “nonspecific links.”

By using “topic links”, Infobutton Managers may further constrain the retrieved content to more closely match the clinician's implied question. As suggested by authors of previous studies, links pointing to specific topics may be more effective than nonspecific links. ^18,19 However, the differences between topic links and nonspecific links have not been assessed and quantified.

The comparison between these types of links is important, especially because topic links are more difficult to implement than nonspecific links. To implement topic links, Infobutton Managers need to “wisely choose” from a potentially long list of topics those few instances that are most likely to be relevant in a given context. The knowledge that drives this decision typically relies on labor intensive studies of information needs in the context of EMR use. ^6,20,21 Also, information resources need to be structured and indexed in a way to identify content relevant to the topic specified in the infobutton request. This requirement is difficult to implement and most currently available information resources do not meet it.

The study subjects conducted 4,101 sessions in the study period. Out of these, 3,729 (90%) met the session duration eligibility range. Of the 104 enrolled subjects, 90 (87%) used the infobuttons at least once (half from each group). Further analysis showed that 12 (86%) of the 14 users who did not use infobuttons in the study period never accessed the medication order entry module of the HELP2 EMR during the study period and hence were not exposed to the medication infobuttons. Of the study subjects who accessed the medication order entry module, 98% (102 out of 104) also accessed infobuttons.

Since the analysis of session duration and number of sessions was based on paired subjects, only those pairs where both subjects conducted at least one session during the study period were included in the analysis. There were 40 pairs (77% of the total number of pairs) where both subjects had at least one session. These pairs conducted 3,260 sessions during the study period (1,792 control vs. 1,468 intervention).

The study presents evidence to support the study alternate hypothesis that session length was shorter in intervention group. In fact, users spent 17% less time on their information seeking efforts when provided with links that attempt to lead them to content that more specifically addressed their questions. On one hand, this difference may be deemed an important finding, given that lack of time and seamless access are among the main barriers to the use of online health information resources at the point of care ^1,5 and one of the key motivations of infobuttons is to reduce these barriers. ^14,17,18 On the other hand, it may be argued that the absolute session length difference between the control and intervention groups was not large enough to produce an impact that is clinically significant.

In theory, the idea behind topic links is that these links lead clinicians to content subsections that are likely to be more closely related to the clinician's question, reducing exposure to unnecessary information and minimizing cognitive effort. Cognitive psychology theories support the notion that information systems should minimize cognitive overload, providing the minimal amount of information to support decisions. This view has been well elaborated in a study by Weir et al., who stated that “any computerized information system needs to be designed to support strategies that allow for ‘fast and frugal’ decision-making. These strategies are often used by decision-makers in the real world. They include multiple methods for rapidly identifying the minimally sufficient data to support a decision, attempts to efficiently balance accuracy versus speed, and creating personal information displays that support recognition-primed actions.” ²⁹ Therefore, topic links may have reduced clinicians' cognitive workload when compared to unspecific links. Yet, the present study has not aimed at proving such an impact. Future studies, with a stronger qualitative focus, may be necessary to address this question more appropriately.

A more pronounced difference may have been observed if, instead of fixing the default topic to “adult dose,” a more accurate prediction method were employed to determine the topic that is most likely to match the clinician's information need in a given context. In the intervention group, users frequently had to override the default topic, a task that demands additional cognitive effort and time from users. This overhead can be minimized if the default topic is accurately predicted. In a previous study, we have demonstrated that accurate prediction models can be derived from data of previous infobutton sessions using machine learning methods, suggesting this to be a sound alternative for further improvement of infobuttons. ^24,25

No statistical difference was found regarding infobutton usage. According to the technology acceptance model, perceived usefulness and ease of use of a technology are the main determinants of actual use. ²⁶ Therefore, it is possible that users may have not perceived the relative advantage of topic links regarding usefulness and ease of use when compared to nonspecific links. Another potential explanation is that the study subjects, who were all frequent users, already used infobuttons whenever a need was perceived. Finally, a significant difference might have been detected if the study were conducted for a longer period of time, giving users sufficient experience with the improved functionality to affect their perceived usefulness and ease of use. The fact that almost half of the subjects were enrolled during the course of the study and therefore had even less time to interact with topic links would have contributed to this.

There was no significant difference in terms of information seeking success rate and impact. This may be explained by the fact that the baseline success rate and impact were already very high, leaving little room for improvement, at least within the context of medication order entry. Other infobutton studies have also shown high session success rates, supporting this hypothesis. ^17,18

The analysis of unsuccessful sessions raised two important points. First, users tended to quit their search effort very quickly without looking at different topics and information resources that were offered. The results were more pronounced than those of general purpose search engines, where the majority of users perform up to two queries and view up to two web sites when searching for content. ²⁷ The difference may be explained by the busier nature of patient care settings and highlights the importance of information resources providing clinicians with content that rapidly addresses their questions. ⁵

Second, a high percentage of the reported unsuccessful sessions was due to issues related to the use of inappropriate drug codes in infobutton requests, reflecting incomplete content indexing. Once identified, these issues were all fixed, so that, in the future, similar requests will successfully retrieve content. Therefore, these issues underscore the importance of continuous monitoring of infobutton sessions as a good knowledge management practice. ²⁸ Continuous monitoring allows problems with code mapping, content indexing, lack of content, and changes to the information resource APIs to be identified and fixed in the Infobutton Manager infrastructure or by the content provider. For example, reports of unsuccessful content requests could be shared with content providers so they can optimize content indexing or decide how to allocate content development resources.

A number of studies have assessed the effectiveness of information resources at the point of care. ³⁰ However, differences in methodology, study population, clinical setting, goals of the information retrieval tool, and types of information needs make a comparison among these and the present study difficult and some times inappropriate. Therefore, comparisons should be considered with caution, considering the lack of a uniform methodology.

In our study, clinicians reported that they were able to find answers to their questions in 88.5% of the infobutton sessions and improved their decisions in 36.3% of these sessions. In a study conducted by Mabrabi et al., physicians reported that their questions were completely or partially answered in 73% of the information seeking sessions conducted via Quick Clinical, an online evidence system. ³¹ Yet, physicians in the Quick Clinical study used the tool for a variety of clinical questions including types of questions that were probably more complex. Conversely, the present study focused solely on medication-related queries.

A mixed method study conducted by Pluye et al. revealed that residents perceived that information accessed via a tool called InfoRetriever was relevant to their search objectives in 85.9% of the searches. ³² However, the concept of “relevancy” in the Pluye et al. study, did not have the same meaning as “success” in the present study, since the former does not necessarily indicate that a question has actually been answered.

A systematic review of studies that evaluated the impact of clinical information retrieval technology on physicians found that 20 to 82% of searches produced any type of positive impact on physicians. Yet, according to the authors of the review, the higher estimates were subject to recall bias, which tend to overestimate impact. Therefore, the most plausible estimates ranged from 20 to 39%. ³⁰ In the present study, a positive impact was found in 63.7% of the infobutton sessions. Since the post-session survey was prompted immediately after the conclusion of infobutton sessions, recall bias was not likely to be an important factor in our results.

Not many studies evaluated applications that enable context-sensitive access to information resources within EMR systems. ³³ In a study conducted by Maviglia et al., users were able to meet their medication-related information needs in 84% and enhanced their decisions in 15% of the sessions. ¹⁸ However, users in the present study were likely to be more experienced with infobuttons than those in the former study. While the former study included all users of a medication order entry system, our study selected a subpopulation of frequent infobutton users. In addition, the present study was performed five years after the initial release of infobuttons, while the former study was initiated once infobuttons were firstly released at their institution.

The present study found an overall median session duration of 39.3 seconds and 35.5 seconds in the intervention group. Users in the study conducted by Maviglia et al. spent 25 seconds looking for information. ¹⁸ However, the session time measurement method was different in these two studies. Session duration in the present study was measured from the infobutton click event to the content window close event, therefore the entire session was captured. Maviglia et al., on the other hand, stopped the timer when the last page viewed in a session was opened. The authors of that study estimated that users may have spent approximately 21 seconds on this last page. Therefore, their total session time can be estimated to be roughly 45 seconds, which is very similar to the median session length in our control group. Interestingly, the infobuttons in the Maviglia et al., study did not provide a topic links functionality, otherwise it is conceivable that they would have observed session lengths similar to the ones observed in the intervention group of the present study.

Cimino conducted a survey to assess the overall user satisfaction and effectiveness of infobuttons at Columbia University. Among the respondents, 74% indicated that infobuttons produced a positive effect on patient care decisions and 20% reported positive impact on patient care. ¹⁷ Differences in methodology preclude a more direct comparison between the present and Cimino's results. However, both studies provide evidence that supports the usefulness of infobuttons.

In a randomized trial, Rosembloom et al. compared two user interfaces that enabled context-sensitive access to educational material in a computerized provider order entry system (CPOE). ³³ Control subjects could access educational content from a menu in the standard CPOE interface. Intervention subjects had access to an improved version, where links to educational material were visibly highlighted in the interface analogously to an automobile dashboard. The usage rate of the improved version was approximately 9 times higher than of the control group, illustrating the importance of user interface in the on-demand delivery of decision support. In our study, the improved version of infobuttons (i.e., topic links) was not associated with an increased usage. Notably, our improved version focused on promoting quicker access to relevant information as opposed to drawing attention to the availability of infobutton links. On the other hand, the results reported by Rosembloom et al. suggest that improved visibility may hold the answer to increasing the usage of infobuttons.