PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of cjhpAboutRegisterLog inSearchCJHP OnlineCJHP Online
 
Can J Hosp Pharm. 2010 Mar-Apr; 63(2): 130–137.
PMCID: PMC2858502

Language: English | French

Quality and Usability of Common Drug Information Databases

Carmen M Mountford
Carmen M Mountford, BSc, BSc(Pharm), ACPR, is a Clinical Pharmacist, Vancouver General Hospital, Vancouver, British Columbia.
Teresa Lee
Teresa Lee, BA, MLIS, MA, is a Reference Librarian, Woodward Biomedical Library, University of British Columbia, Vancouver, British Columbia.
Jane de Lemos
Jane de Lemos, BPharm, PharmD, MSc, is Regional Coordinator, Professional Practice, Vancouver Coastal Health – Providence Health Care, and a Clinical Associate Professor, Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver, British Columbia.

Abstract

Background:

Pharmacists’ access to user-friendly electronic drug information databases that can quickly provide accurate, up-to-date information has become increasingly important. Unfortunately, decisions about purchasing subscriptions to such services are not always made objectively. Previously published studies have compared drug information databases, but there are no recent analyses from the perspective of Canadian hospital pharmacists.

Objective:

To determine overall preferences among the most commonly used online drug information databases, based on an appraisal of the quality, performance, and usability of the databases and users’ preferences.

Methods:

Qualitative and quantitative analyses with descriptive and inferential statistics were used to compare the Clinical Pharmacology, Lexi-Comp Online, and Micromedex databases. Quality scores were determined from investigators’ consensus ratings across 5 categories of quality indicators. Performance scores were determined according to the ability of a database to answer 15 clinical drug information questions. Usability scores were determined from user ratings in 7 domains. Users’ preferences were assessed through rankings of the databases by 26 practising pharmacists.

Results:

The highest quality and performance scores were awarded to Lexi-Comp Online, whereas Micromedex received the lowest overall usability score, attributable to poor scores for layout, navigation, and speed. Lack of Canadian content was identified as a major disadvantage of the Clinical Pharmacology database. Users ranked Micromedex significantly lower than the other databases, whereas the majority of users ranked Lexi-Comp Online as the most preferred database.

Conclusions:

Lexi-Comp Online appeared to be the most preferred database, whereas Micromedex was clearly the least preferred database. These findings should be considered in future decisions about purchasing database subscriptions.

Keywords: drug information, online databases, clinical decision support tools, usability

RÉSUMÉ

Contexte :

L’accès des pharmaciens à des bases de données en ligne conviviales sur les médicaments pouvant fournir des renseignements actuels et précis est de plus en plus important. Malheureusement, les décisions d’acheter ou non des abonnements ne sont pas toujours prises de façon objective. Des études publiées ont comparé des bases de données sur les médicaments, mais il n’existe pas d’analyses récentes du point de vue des pharmaciens des établissements du Canada.

Objectif :

Déterminer les préférences générales parmi les bases de données en ligne sur les médicaments les plus courantes, d’après une évaluation de la qualité, de la performance et de la convivialité des bases de données, et des préférences des utilisateurs.

Méthodes :

Des analyses qualitative et quantitative combinées à des statistiques descriptives et inférentielles ont été utilisées pour comparer les bases de données Clinical Pharmacology, Lexi-Comp Online et Micromedex. Les scores de qualité ont été déterminés à partir de cotes consensuelles des chercheurs relativement à cinq catégories d’indicateurs de qualité. Les scores de performance ont été déterminés d’après la capacité d’une base de données à répondre à 15 demandes d’information clinique sur des médicaments. Les scores de convivialité ont été déterminés à partir de cotes attribuées par les utilisateurs dans 7 domaines. Les préférences des utilisateurs ont été évaluées à partir du classement des bases de données par 26 pharmaciens actifs.

Résultats :

Les scores de qualité et de performance les plus élevés ont été attribués à la base de données Lexi-Comp Online, alors que Micromedex a reçu le score de convivialité le plus bas, attribuable à de faibles notes au chapitre de la mise en page, de la navigation et de la vitesse. Une absence de contenu canadien a été reconnue comme un inconvénient majeur de la base de données Clinical Pharmacology. Les utilisateurs ont accordé à la base de données Micromedex une cote significativement inférieure aux autres bases de données, alors que la majorité des utilisateurs ont classé la base de données Lexi-Comp Online comme la préférée.

Conclusions :

La base de données Lexi-Comp Online semblait être la base de données préférée, alors que Micromedex a été classée incontestablement au dernier rang. Ces résultats devraient être pris en compte dans les décisions d’achat d’abonnements à des bases de données.

[Traduction par l’éditeur]

Mots-clés : information sur les médicaments, bases de données en ligne, outils d’aide à la décision clinique, convivialité

INTRODUCTION

The size and complexity of the body of drug information is growing rapidly. This growth in the availability of drug information and the accompanying growth in medication use have necessitated advancement of the role of pharmacists as medication experts within multidisciplinary health care teams, particularly in hospitals. The use of clinical decision support tools such as electronic drug information databases can aid pharmacists in their provision of pharmaceutical care and has the potential to improve medication safety.1 As such, pharmacists’ access to user-friendly electronic resources that can quickly provide complete, accurate, and current medication information has become increasingly important, especially in hospital pharmacy practice, where clinical pharmacists are often directly involved in the therapeutic decision-making process.

The budgets of health care organizations may not allow for subscriptions to multiple drug information databases. Unfortunately, when only a single subscription is feasible, the choice between databases is not always made objectively, because these purchasing decisions can be influenced by many factors. For example, users’ preferences, and therefore purchasing decisions, may be influenced by users’ familiarity with a particular drug information database.2 Therefore, studies that objectively compare electronic drug information databases can be useful when contemplating subscription purchases. Previous studies have compared online drug information databases27; however, these databases evolve rapidly, and there have been no recent analyses from the perspective of Canadian hospital pharmacists.

The value of an online drug information database can be described in terms of its utility relative to its cost, where utility is defined as the extent to which the database satisfies the user’s need for information (database quality and performance), and the cost comprises both the purchase cost and the time spent retrieving the desired information (usability).2,8 The objective of this study was to determine an overall preference among the most commonly used online drug information databases within Vancouver Coastal Health – Providence Health Care, namely Clinical Pharmacology, Lexi-Comp Online, and Micromedex, on the basis of a multimodal appraisal of the quality, performance, and usability of each database, as well as users’ preferences.

METHODS

A qualitative and quantitative data collection and analysis strategy, with descriptive and inferential statistics, was used to assess the quality, performance, and usability of the databases and users’ preferences, to determine overall database preference. This study satisfied all ethical review requirements of the investigators’ organization.

Selection of Databases and Study Participants

The online drug information databases selected for evaluation were those most commonly used or under consideration for purchase or renewal by the various hospital pharmacy practice sites within Vancouver Coastal Health – Providence Health Care at the time of the study (early 2009). The 3 databases evaluated were Clinical Pharmacology (Gold Standard Inc, an Elsevier company, Tampa, Florida), Lexi-Comp Online (Lexi-Comp Inc, Hudson, Ohio), and Micromedex (Thomson Reuters [Healthcare] Inc, Greenwood Village, Colorado) (Table 1). The publishers of the databases provided free trial access for the purposes of this study.

Table 1.
Online Drug Information Databases Evaluated

The performance and usability of the databases, as well as users’ preferences, were evaluated during February 2009 with a convenience sample of 26 pharmacists. This sample reflected a cross-section of pharmacist roles from a variety of hospital pharmacy practice sites across the health region. The investigators invited pharmacists to participate in the study, with the number of pharmacists recruited from each site weighted approximately by the number of pharmacists employed at each practice site. Data were also collected regarding the study participants’ characteristics, including length of time in pharmacy practice, level of pharmacy training attained, and frequency of use of drug information databases before participating in the study (see Appendix 1 for questionnaire).

Development of Drug Information Questions

The drug information questions used in this study were selected from those previously published,9 with balanced representation for different drug classes, disease states, and drug information categories. The questions were modified for relevance to hospital pharmacy practice and were contextualized with Canadian brand names as applicable. Ultimately, a set of 15 drug information questions covering 17 different categories of drug information was created for use in this study (Appendix 2).

Evaluation of Database Quality

The study investigators evaluated the quality of the databases in terms of 5 quality indicator categories, namely referencing, grading of evidence cited or recommendations provided, editorial policy for content updates, provision of authorship information, and inclusion of Canadian content. These quality indicators, chosen on the basis of those previously established,9 were criteria thought to increase a Canadian hospital pharmacist’s confidence in the quality and relevance of drug information provided by an online drug information database. The investigators assessed the quality indicator criteria through a comprehensive review of each database and the respective publishers’ websites, followed by development of consensus ratings for each quality indicator category on a scale of 0 to 3. The quality score for each database was calculated as the mean of investigators’ consensus ratings across all 5 quality indicator categories.

Evaluation of Database Performance

Each study participant used each database to answer a set of 5 randomly assigned drug information questions (total of 15 questions per participant). The order in which study participants evaluated the 3 drug information databases was also randomly assigned, and over the whole study each database was used to attempt to answer each of the 15 questions at least once. In an effort to consume no more than 1 hour of a participant’s time, the participants were asked to spend no more than 3 minutes on each drug information question. Study participants used a 3-point scale to note whether answers to the drug information questions posed were present and complete (see Appendix 1), but the investigators made no attempts to confirm the accuracy of the answers located. The performance score for each database was calculated as the mean score across all 15 questions, with each question weighted equally.

Evaluation of Database Usability

After using all 3 drug information databases to answer the assigned drug information questions, study participants were asked to evaluate the usability of each product by completing a usability questionnaire (see Appendix 1), which was adapted from those previously published.3,5 Study participants used a 5-point Likert scale to rate each database within 7 different usability domains (database layout, navigation, speed, accuracy of content, amount of information, timeliness of information, and user satisfaction). The usability of each database was then calculated in terms of the mean score for each individual usability domain, as well as overall usability score across all 7 domains.

Users’ Preferences

After evaluating performance and usability, study participants were asked to rank the online drug information databases in order of preference on a scale of 1 to 3, where a rank of 1 represented the most preferred database and a rank of 3 represented the least preferred database. The distribution of rankings for each database was compared, and mean rank scores were calculated for each database. Subgroup analyses of the mean rank scores for each database, stratified by level of pharmacy training attained, years in pharmacy practice, and prior database access, were also performed. Finally, study participants were invited to comment on their experiences with the databases. The study investigators performed independent thematic analyses of these comments to supplement comparisons of the quality, performance, and usability of the databases and users’ preferences.

Statistical Analysis

Descriptive statistics were used to calculate mean scores and 95% confidence intervals for database quality, performance, and usability and for users’ preferences. Inferential statistics were used to compare mean scores via analysis of variance (ANOVA) and 2-sample t tests. Values of p below 0.05 were considered statistically significant.

RESULTS

Characteristics of Study Participants

Most of the study participants reported that they consulted drug information databases at least daily in their practice, with 23 (88%) reporting prior access to Micromedex and half or fewer reporting prior access to either Lexi-Comp Online or Clinical Pharmacology (Table 2).

Table 2.
Characteristics of Study Participants

Database Quality

Lexi-Comp Online received the highest mean quality score (2.6), followed by Micromedex (2.2) and Clinical Pharmacology (1.6), although the differences among the databases were not statistically significant (Table 3). Lack of Canadian content in Clinical Pharmacology was a major distinguishing difference between this database and Lexi-Comp Online in the investigators’ assessment. Thematic analysis of study participants’ comments (Table 4) confirmed this deficiency as a distinguishing feature between Clinical Pharmacology and the other databases.

Table 3.
Assessment of Quality, Performance, and Usability of the 3 Online Drug Information Databases
Table 4.
Thematic Analysis of Study Participants’ Comments about Their Experience with Each Drug Information Database

Database Performance

Lexi-Comp Online received the highest mean performance score (2.7), followed by Clinical Pharmacology (2.4) and Micromedex (2.3), when used to answer a set of 15 drug information questions covering 17 different categories of drug information, although the differences among the databases were not statistically significant (Table 3). However, some of the differences between databases in terms of mean performance scores within individual drug information categories were statistically significant (data not shown). For example, Micromedex performed worse than the other databases in the categories for excipient information and drug use in pregnancy, but performed better than the other databases in the adverse drug reaction category. Lexi-Comp Online outperformed the other databases in 5 of the 17 drug information categories in this study: drug interactions, monitoring, pharmacology, foreign/Canadian/newly approved drugs, and herbal/nonprescription drugs.

Database Usability

Lexi-Comp Online received a significantly higher mean score (4.1) in the satisfaction domain than both Clinical Pharmacology (3.6) and Micromedex (3.1) (p < 0.05) (Table 3). Micromedex received significantly lower scores than the other databases in the layout, navigation, and speed domains, which contributed to its significantly lower overall usability score (3.4) relative to both Clinical Pharmacology (4.0) and Lexi-Comp Online (4.0) (p < 0.05). Thematic analysis of comments from study participants confirmed the findings of the quantitative evaluation of usability, indicating that poor database layout, difficult navigation, and slow speed differentiated Micromedex from both Clinical Pharmacology and Lexi-Comp Online (Table 4).

Users’ Preferences

On the basis of users’ rankings of the databases, where 1 represented the most preferred database and 3 the least preferred, Micromedex received a significantly worse mean rank score (2.5) than the other 2 databases (p < 0.05) (Table 3). Although the mean rank scores for Lexi-Comp Online (1.6) and Clinical Pharmacology (1.9) were not statistically different from each other, Lexi-Comp Online was ranked as the most preferred database by the majority of users and was ranked as the least preferred database less frequently than Clinical Pharmacology and Micromedex (Figure 1).

Figure 1.
Distribution of database rankings according to users’ preferences.

In subgroup analyses, the mean rank scores for each database were not significantly different between subgroups of users stratified by number of years in practice or level of pharmacy training attained (data not shown). Users reporting access to a particular database before participating in this study tended to rank that database higher than users reporting no prior access, although these differences were not statistically significant. Within all subgroups of pharmacist users, Micromedex received the lowest mean rank scores of the 3 databases.

DISCUSSION

As the body of drug information continues to grow in size and complexity, drug information databases have become essential tools for pharmacists in their role as medication experts. The results of this study confirmed the perception that pharmacists rely heavily on such databases, with the majority of study participants reporting that they consulted drug information databases at least daily. This finding further emphasizes the need for active engagement of end-users in objective comparisons between databases. A previous analysis ranked the databases used in this study as the top 3 online drug information databases in terms of scope, completeness, and ease of use.6 However, that earlier study yielded a different overall rank ordering: Clinical Pharmacology, Micromedex, and Lexi-Comp. Although the rankings in that study were not significantly different among the databases, the difference in rank order was most likely due to the study’s substantially different (and, in our view, less applicable) methodology and to changes to the databases that have occurred over the ensuing 3 years. We found that Lexi-Comp Online was the highest-quality, best-performing, and most preferred online drug information database among those evaluated. Clinical Pharmacology’s lack of Canadian content was identified as the most important disadvantage of this database, which highlights the relevance of completing a study like this from a Canadian perspective. It has been suggested that users’ evaluations of a database may be favourably affected by familiarity with the database.2 In the current study, participants reporting prior access to a particular database tended to rank that database higher than participants reporting no prior access to the database. However, this difference was not statistically significant, and, despite 88% of study participants reporting prior access to Micromedex, this database was still clearly the least preferred of the 3 databases.

Some limitations of this study deserve mention. Drug information databases are dynamic products that are improved and regularly updated by their publishers, which makes analyses like this one vulnerable to continuous change. Still, the differences in database quality and usability elucidated in this study are less likely to be affected by regularly scheduled content updates, but rather would require publishers to implement major changes to their editorial policies, bundling of database modules, and database design. The drug information questions used in this study could not represent every type of drug information question that a hospital pharmacist is likely to encounter; thus, repeating the study with a different set of questions might lead to different conclusions. However, we selected a subset of drug information questions from those used by other investigators9 that covered a broad range of drugs, disease states, and drug information categories. Admittedly, however, this subset excluded content areas such as IV compatibility and toxicology. In addition, our overall conclusions were based on other components, such as the quality indicator criteria, that we believe to be less subjective. We recognize that the availability and usefulness of mobile device versions of these online drug information databases, such as those for use on personal digital assistants (PDAs), are important determinants of users’ preferences. However, a comparison between mobile device versions of the online databases included in this study was beyond the intended scope of analysis. Therefore, only a descriptive summary of the availability of mobile device editions is presented (see Table 1), and readers are invited to consult previously published studies evaluating PDA-based drug information databases.7,912 Finally, we acknowledge that acquisition cost is an important factor in database-subscription purchasing decisions. Calculation of the subscription cost for a database is often confidential and can be complex, depending on the platform and modules chosen, the size of the institution (i.e., number of hospital beds), and/or the number of users. Thus, an economic analysis was not included in our comparison of online drug information databases, but should be considered in concert with the findings of this study when making database-subscription purchasing decisions.

CONCLUSIONS

From the perspective of Canadian hospital pharmacists, Lexi-Comp Online is superior to Clinical Pharmacology and Micromedex, on the basis of the database quality and performance criteria evaluated; it also appeared to be the database most preferred by study participants. Clinical Pharmacology compared favourably to Lexi-Comp Online in terms of most of the criteria evaluated in this study, but its usefulness in the authors’ practice setting may be hampered by its lack of Canadian drug information. Micromedex was clearly distinguished as the least preferred database. These findings should be considered in future subscription purchasing decisions by hospital pharmacy departments.

Appendix 1. Questionnaires for evaluation of database performance, usability, users’ preferences, and demographic characteristics of study participants

Please tell us about your experience with each drug information database by circling the most appropriate response and/or providing comments where applicable.

An external file that holds a picture, illustration, etc.
Object name is cjhp-63-130fa1.jpg

Appendix 2. Drug Information Questions Used to Assess Performance of Databases

QuestionDrug Information Category
1. What is the recommended initial dose of tolbutamide in an adult with type 2 diabetes?Dosage
2. What strength of Actonel tablets are used for the new monthly dosing regimen?Dosage, Strengths/formulation
3. Do Spiriva capsules contain lactose?Excipients
4. How should Renagel tablets be administered with respect to food?Administration
5. Once reconstituted, what is the stability of Amoxil oral suspension at room temperature?Stability/storage
6. What is the incidence of seizures with melphalan?Adverse drug reactions
7. When used off-label for acne, what is the recommended dose of Aldactone?Indications, Dosage
8. Can quinine be safely used in a patient with G6PD deficiency?Contraindications, Pharmacogenomics
9. Is it safe for a patient on anticoagulants to take reishi?Drug interactions, Herbal/nonprescription drugs
10. How soon after discontinuing fluoxetine can a patient start taking moclobemide?Drug interactions, Foreign/Canadian/newly approved drugs
11. What is the mechanism of action of Xarelto?Pharmacology, Foreign/Canadian/newly approved drugs
12. What is the onset of action of anti-emetic effect for metoclopramide after IM administration?Pharmacokinetics
13. During which trimester of pregnancy should the use of Sustiva be avoided?Pregnancy, Contraindications
14. How often should phenytoin levels be drawn in a stable patient with a history of tonic–clonic seizures?Monitoring
15. Does this database have patient education/instructions for warfarin?Patient education

G6PD = glucose-6-phosphate dehydrogenase.

References

1. Kuperman GJ, Reichley RM, Bailey TC. Using commercial knowledge bases for clinical decision support: opportunities, hurdles, and recommendations. J Am Med Inform Assoc. 2006;13(4):369–371. [PMC free article] [PubMed]
2. Belgado BS, Hatton RC, Doering PL. Evaluation of electronic drug information resources for answering questions received by decentralized pharmacists. Am J Health Syst Pharm. 1997;54(22):2592–2596. [PubMed]
3. Kupferberg N, Jones Hartel L. Evaluation of five full-text drug databases by pharmacy students, faculty, and librarians: do the groups agree? J Med Libr Assoc. 2004;92(1):66–71. [PMC free article] [PubMed]
4. Li J, Sayed EN, Robertson JC. Evaluating drug databases: the librarian’s perspective. J Electron Resour Med Libr. 2005;2(4):31–51.
5. Campbell R, Ash J. An evaluation of five bedside information products using a user-centered, task-oriented approach. J Med Libr Assoc. 2006;94(4):435–441. e206–e207. [PMC free article] [PubMed]
6. Clauson KA, Marsh WA, Polen HH, Seamon MJ, Ortiz BI. Clinical decision support tools: analysis of online drug information databases. BMC Med Inform Decis Mak. 2007;7:7. [PMC free article] [PubMed]
7. Clauson KA, Polen HH, Marsh WA. Clinical decision support tools: performance of personal digital assistant versus online drug information databases. Pharmacotherapy. 2007;27(12):1651–1658. [PubMed]
8. Huth EJ. The information explosion. Bull N Y Acad Med. 1989;65(6):647–661. [PubMed]
9. Galt KA, Rule AM, Houghton B, Young DO, Remington G. Personal digital assistant-based drug information sources: potential to improve medication safety. J Med Libr Assoc. 2005;93(2):229–236. [PMC free article] [PubMed]
10. Enders SJ, Enders JM, Holstad SG. Drug-information software for Palm operating system personal digital assistants: breadth, clinical dependability, and ease of use. Pharmacotherapy. 2002;22(8):1036–1040. [PubMed]
11. Clauson KA, Seamon MJ, Clauson AS, Van TB. Evaluation of drug information databases for personal digital assistants. Am J Health Syst Pharm. 2004;61(10):1015–1024. [PubMed]
12. Lowry CM, Kostka-Rokosz MD, McCloskey WW. Evaluation of personal digital assistant drug information databases for the managed care pharmacist. J Manag Care Pharm. 2004;9(5):441–448. [PubMed]

Articles from The Canadian Journal of Hospital Pharmacy are provided here courtesy of Canadian Society Of Hospital Pharmacists