Several publicly available citation post-processing tools have been developed to compensate for the lack of computational capabilities of PubMed. Services such as SLIM [2], MedKit [3] and PubMed Assistant [4] provide more intuitive search interfaces, encouraging users to take advantage of available PubMed search filters in refining their data mining strategy. Other services, such as XplorMed [5,6], PubFinder [7], Vivisimo^® [8] and GoPubMed [9] match PubMed search results with additional databases to classify articles into relevant topics based on ontology. XplorMed extracts keywords from the abstracts of the PubMed hits by analysis of word co-occurrence. Subsequent classification and selection are based on those keywords. To establish ontology, PubFinder uses an independently generated occurrence frequency database of 100,000 terms extracted from abstracts available through PubMed since 1990. GoPubMed uses the Gene Ontology database [10], including over 19,000 terms organized in three sub-ontologies: cellular location, molecular function or biological process. However, most of these services make a use of only one filtering mechanism or of only one biomedical database.

Many other useful biomedical databases have not been integrated into PubMed analysis. For example, as of the August 2006 BioMed Central provides references to as many as 2,704 biomedical databases. Many of these databases are textual, containing unique collections of terms that can be included in citations' analytics [11]. All of these databases can potentially become a useful addendum in citation analytics.

Despite a growing number of tools for textual processing of PubMed citations, there has been little done on developing efficient statistical representation tools of major bibliometric data. PubMed lacks simple statistical analyses of relevant search records, and the user cannot obtain a quick introduction to the topic of interest through publication trends, publications with the most impact, and names of leading researchers in the field. An alternative to PubMed, Web of Science^® (The Thomson Corporation) is a subscription-based service that provides only basic analysis of search results by parameters such as publication year, author's name, and source title on up to 2,000 records at a time [12]. However, statistical outputs generated by Web of Science^® are purely volumetric and do not provide any impact ranking of publications or authors.

Ranking of publications and authors can be based on "human factor" bibliometric parameters such as journal impact factor and volumetric data on forward referencing, both of which are used as common numeric indicators of a publication's impact.

1) Journal impact factor (IF) determines the average significance and quality of information in any given article published in a respective journal. Impact factor is calculated as follow [13,14]:

IF = All citations in YEAR to articles in JOURNAL during (YEAR-1) + (YEAR-2)/All articles in JOURNAL during (YEAR-1) + (YEAR-2)

Journals with a higher impact factor are generally harder to get published in. These journals have a more stringent and rigorous review process and more strict requirements as to the quality of the presented data. If a manuscript is not accepted in a journal with a higher rank, it will ultimately be attempted in an alternative journal of lower rank (one with a lower impact factor). In most instances, higher quality data will be published in journals with a higher impact factor. While this is not universally true and evidence exists against this statement [15], "there is nothing better and it (impact factor) has the advantage of already being in existence and is, therefore, a good technique for scientific evaluation. Experience has shown that in each specialty the best journals are those in which it is most difficult to have an article accepted, and these are the journals that have a high impact factor" [16].

2) The volume of the forward citations and the citation trend over the lifetime of an article reflects its acceptance by the scientific community. Thus, it is obvious that if published findings represent a significant milestone in the field, they will be heavily cited.

Unlike other bibliometric parameters (such as chronological and volumetric parameters), both numerical parameters described above are dependent on a human factor. Unfortunately, PubMed does not account for them. The SIGAPS software accounts for journal impact factor [17]. This software serves the purpose of generating graphic reports on a researcher's publication activity and tags each relevant article with the journal impact factor value. For each researcher, it breaks down all his/her articles into arbitrary groups based on the values of the impact factor (from high to low). SIGAPS shows the proportion of articles published in sources with high impact factors versus those with low impact factors.

Further development of ranking utility accounting for both impact factor and forward citations among other factors is required. Taken separately, impact factor and volume of forward citations can be misleading. Leading journals within a narrow biomedical field can have a low impact factor that does not adequately represent the publication's impact. Also, self-referencing is a common practice among some scientists, and articles with apparently high forward citation level might have few citations from other scientists. Therefore, it will be desirable to develop a numeric index based on both parameters.

PubFocus provides a set of statistical tools for bibliometric analysis of relevant records. In the first step, data on relevant records is extracted from the PubMed server in packets of 50 XML-formatted records (up to a total of 2500 records per analysis) and written into a local temporary database (Fig. 1B, 1E). Remote extraction of the XML-formatted records was chosen over loading the entire PubMed database (approximately 31.6 – 46.3 GB in size [18]) into the local relational database in order to maintain lightweightness and ease of transferability of the application between servers and to avoid necessity for frequent updates and maintenance. In addition, this method excludes a need for developing a separate search engine. Here, relevance of citations to the search query is determined by the PubMed search engine. In the second step, the local database is enriched with IFs and volumetric data on forward citations (Fig. 1C, 1D; Fig. ​Fig.2C).2C). The user can choose to collect data on forward citations from three alternative sources: PubMed Basic, PubMed Central or Google™ Scholar (Fig. ​(Fig.2D).2D). Data acquisition from PubMed Basic is fast, yet it only allows establishing the presence or absence of forward references within the PubMed Central database (i.e. "yes or no" mode). Alternatively, both PubMed Central and Google™ Scholar provide numeric values of forward citations, but data harvesting from these sources can be lengthy and is recommended for smaller sets of records. In general, Google™ Scholar provides higher values of forward citations for the same publication than PubMed Central does. In the recent formal study forward citation data from Google™ Scholar showed a substantial degree of overlap with that from proprietary Web of Science^® [19]. While forward citations appear in the Google™ Scholar database earlier in the lifecycle of the publication than in PubMed Central database, Google™ Scholar "returns a smaller number of citing references" than Web of Science^® but "provides a large set of unique citing material" [19]. While "it is clear that Google™ Scholar provides unique citing material", "the exact composition of this citing material should also be more thoroughly examined so that scholars will have a clear idea what is and is not included in Google™ Scholar searches".

Statistical analysis is performed in the second step. Basic statistics employ a simple volumetric approach (similar to the analysis done by Web of Science^®) to compute, sort, and provide semi-graphical output of the results (Fig. ​(Fig.1G).1G). Basic statistics can be viewed by accessing the "Basic Statistics" tab. Analysis includes that of publication trends over the years, top publishing first authors (commonly scientists with most contribution to the paper), top publishing last authors (commonly principal investigators), top fields of research, top research topics, top publication sources based on volume or impact factor, publication types, and publication languages. In addition, search-narrowing tools allow selection and display of subsets of relevant records matching any of the above listed parameters. For example, one can select to display records published by top three principal investigators or records published in certain journals only (such as a small subset of "high-profile" articles published in journals like Nature or Science with high impact factors). Back and forth focusing on the subsets of the initial search is done using a temporary local database without additional time-consuming external data harvesting (Fig. ​(Fig.1G1G → 1H → 1E → 1G; Fig. ​Fig.2E2E).

We have also developed means for the integration and use of biomedical databases in the assisted citations' analytics (Fig. ​(Fig.1I).1I). We have designed a standard MySQL format and integrated MySQL full-text search to allow automatic extraction of matching terms from titles and abstracts of relevant citations (Fig. ​(Fig.1J).1J). These terms are sorted either based on their occurrence rate or by build-in ontology categories. Matching terms are presented in the form of semi-graphical output, allowing similar selection and search narrowing procedures as throughout the rest of the PubFocus portal (Fig. 1K, 1L). While multiple databases qualified for the integration, we chose NCI (National Cancer Institute) thesaurus and MGD (Mouse Genome Database) mammalian gene orthology (Fig. 2I, 2J). Both of these databases are large and can be useful for a wide scientific audience rather than for small interest groups only. NCI thesaurus represents a "major effort to integrate molecular and clinical cancer-related information within a unified biomedical informatics framework, with controlled terminology as its foundational layer". It includes some 49,000 biomedical concepts and 146,000 synonyms "separated into 20 logically distinct kinds" including anatomical terms, diagnostic terms, diseases, drugs and chemicals etc. [20,21]. NCI thesaurus has not been previously integrated into any PubMed citations analytics applications. MGD mammalian gene orthology includes around 65,500 symbols and names and an additional 24,000 synonyms of mammalian genes. It is one of the most comprehensive and inclusive mammalian gene databases [22].

Additional statistics employ custom algorithms to provide more accurate ranking of the search results. Particularly, PubFocus uses several determinants of publications' and authors' impact to automatically attempt to identify most prominent publications and authors whose papers can be considered significant within the given field of search.

1) Combined Impact Factor (CombIF) is calculated as CombIF = IF*citations-over-age index to account for:

a) Impact factor (IF) of the publication source.

b) Age of the publication and presence of forward references in either one of three databases mentioned above (citations-over-age index). Calculation of citations-over-age index is performed based either on Table ​Table11 (if PubMed Basic mode is used) or Table ​Table22 (if either PubMed Central or Google™ Scholar mode is used). In general, citations-over-age index boosts the IF value of new and cited articles proportionally to the number of forward citations and reduces the IF value of old articles that have not been cited ("dead-end" articles).

2) Cumulative impact factor (CIF) is calculated as CIF = ∑(IF) and represents the cumulative value of a journal's IF where any given author has been published.

3) Author's Rank (AR) was introduced to make important adjustments to AIF. It is calculated as AR = ∑(CombIF/contribution index) to account for:

a) Combined Impact Factor (CombIF, see above).

b) The role of author in the publication (contribution index). Contribution index of the first and last authors (the most contributing authors) on the paper is set at 1, keeping CombIF value unchanged. Contribution level of the middle authors demonstrates high degree of variability [23]. Often, however, contribution of the second author is greater than contribution of the remaining middle authors [24]. Therefore, the contribution index of second authors is set at 1/2 and for remaining middle authors it is set at 1/3, diluting the initial CombIF value. While an alternative strategy could have been implemented where total contribution indices of all authors of a paper should add to "1", this approach was dismissed because papers with large authors' lists will produce contributions diluted to an insignificant extent.

Ranked results can be viewed in the form of sorted citation records by accessing the "Sorted Results" tab (Fig. ​(Fig.1F;1F; Fig. ​Fig.2F).2F). Processed publications can be sorted by publication date, first author, last author, impact factor, forward citations or CombIF (Fig. ​(Fig.2G).2G). Sorting can be done either in ascending or descending modes. Sorting by CombIF in descending mode is the most informative, as it outputs publications with the highest established impact first. In addition, by accessing the "Basic Statistics" tab the user can view distribution of either impact factors, volume of forward citations or CombIFs among all publications within the database.

By accessing the "1st Authors" and "PI Authors" tabs the user can access ranking tables of both first and last authors within the given field of search (Fig. ​(Fig.2H).2H). Ranking is provided based on both CIF and AR (see above). Two sets of tables are generated. The first set accounts for the publications where respective authors are either first or last in the list of authors only. The second set accounts for all publications including those where respective authors are not the primary contributors (i.e. middle authors). Ranking tables by AR on all publications (last table on the page) is the most informative as it ranks authors based on their cumulative established impact within the field of search.

We see two major challenges on the path to building efficient semantic networks for PubMed database analytics. The first challenge pertains to the development of meaningful, interest-specific and extensive extraction and representation of biomedical terms. The second relates to the development of reliable algorithms for sorting and ranking citations. We believe that construction of successful semantic networks for PubMed would include development of automated terminology extraction tools based on existing biomedical thesauri, dictionaries and ontologies, and ranking algorithms based on established "human factor"-based bibliometric parameters such as journal impact factor and forward references.

To address these challenges we have built a scalable prototype PubMed citation analytics system (PubFocus). The following three steps of data mining were realised in this system: 1) Query match-based selection of the records from PubMed database (for this step PubFocus relies on the build-in PubMed search engine); 2) Controlled vocabularies-assisted extraction of relevant biomedical terms (NCI thesaurus and MGD mammalian gene orthology database were implemented for this project); 3) prioritization of the resulting citations based on the algorithm relying on journal impact factor, volume of forward references, referencing dynamics and authors' contribution level. For each citation PubFocus generates multiple reference points including bibliometric (impact factor, forward citations volume, rank) and contextual (matching biomedical terms fro controlled vocabularies, ontologies). Combined and statistically processed, these reference points are used in computer-assisted citation analytics.

Scalability of PubFocus allows expansion of the bank of controlled vocabularies. A single controlled vocabulary or ontology contains a collection of biomedical terms of special interest often useful for the analytics of the limited group of PubMed citations (such as FlyBase – a database of Drosophila melanogaster genome) [30]. As the result of such "specialization" of the terminology, multiple controlled vocabularies covering various biomedical fields should be used simultaneously. Many of these terminology databases have already been built. A majority of them are available within the public domain (as many as 2,704 biomedical databases have been made available through BioMed Central) [11].

PubFocus can be adapted to work with citation collections other than PubMed. Properly formatted in MySQL, a variety of databases can be readily queried by PubFocus. For example, PubFocus analytics can be integrated into publisher's databases, biomedical journals, university libraries, R&D databases of commercial organisations etc.

Several challenges remain on the path toward optimisation of the efficient semantic networking tools for publications analytics. Citations ranking algorithms should be further evaluated. Comparison of ranked results between major proprietary ranking algorithms (Google™ Scholar and Scirus™) and between these and PubFocus have uncovered very low degree of overlap. It should be further determined what should and what should not constitute a meaningful factor in the ranking process. Here we propose an algorithm that is largely based on journal impact factor and forward citations volume and dynamics. However, other "human factor"-based parameters can be used as well. Each PubMed citation can be assigned with the rank based on experts' opinion. It can be generated upon formal post-publication evaluation performed by other scientists working within the relevant field [13]. However, for obvious reasons it is difficult to generate such parameters for all the articles in the PubMed database. An effort has been initiated by Biomed Central toward this direction with the introduction of "Faculty of 1000 (F1000) Biology" and "F1000 Medicine". Both new services attempt to "comprehensively and systematically highlight and review the most interesting papers published in the biological sciences based on the recommendations of a faculty of well over 1000 selected leading researchers" and "in any field of medicine based on the recommendations of a selected faculty of nearly 2500 leading international researchers and clinicians" [31,32]. If successful, F1000s can deliver a new citation ranking parameter.

Operating system(s): Linux/Unix

Programming language: PERL

Other requirements: MySQL database. Perl modules for XML processing, HTML parsing, generation of session, fork processes and interaction with MySQL database.

License: Open Access.