PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1690854)

Clipboard (0)
None

Related Articles

1.  Anatomy of the Epidemiological Literature on the 2003 SARS Outbreaks in Hong Kong and Toronto: A Time-Stratified Review 
PLoS Medicine  2010;7(5):e1000272.
Weijia Xing and colleagues reviewed the published epidemiological literature on SARS and show that less than a quarter of papers were published during the epidemic itself, suggesting that the research published lagged substantially behind the need for it.
Background
Outbreaks of emerging infectious diseases, especially those of a global nature, require rapid epidemiological analysis and information dissemination. The final products of those activities usually comprise internal memoranda and briefs within public health authorities and original research published in peer-reviewed journals. Using the 2003 severe acute respiratory syndrome (SARS) epidemic as an example, we conducted a comprehensive time-stratified review of the published literature to describe the different types of epidemiological outputs.
Methods and Findings
We identified and analyzed all published articles on the epidemiology of the SARS outbreak in Hong Kong or Toronto. The analysis was stratified by study design, research domain, data collection, and analytical technique. We compared the SARS-case and matched-control non-SARS articles published according to the timeline of submission, acceptance, and publication. The impact factors of the publishing journals were examined according to the time of publication of SARS articles, and the numbers of citations received by SARS-case and matched-control articles submitted during and after the epidemic were compared. Descriptive, analytical, theoretical, and experimental epidemiology concerned, respectively, 54%, 30%, 11%, and 6% of the studies. Only 22% of the studies were submitted, 8% accepted, and 7% published during the epidemic. The submission-to-acceptance and acceptance-to-publication intervals of the SARS articles submitted during the epidemic period were significantly shorter than the corresponding intervals of matched-control non-SARS articles published in the same journal issues (p<0.001 and p<0.01, respectively). The differences of median submission-to-acceptance intervals and median acceptance-to-publication intervals between SARS articles and their corresponding control articles were 106.5 d (95% confidence interval [CI] 55.0–140.1) and 63.5 d (95% CI 18.0–94.1), respectively. The median numbers of citations of the SARS articles submitted during the epidemic and over the 2 y thereafter were 17 (interquartile range [IQR] 8.0–52.0) and 8 (IQR 3.2–21.8), respectively, significantly higher than the median numbers of control article citations (15, IQR 8.5–16.5, p<0.05, and 7, IQR 3.0–12.0, p<0.01, respectively).
Conclusions
A majority of the epidemiological articles on SARS were submitted after the epidemic had ended, although the corresponding studies had relevance to public health authorities during the epidemic. To minimize the lag between research and the exigency of public health practice in the future, researchers should consider adopting common, predefined protocols and ready-to-use instruments to improve timeliness, and thus, relevance, in addition to standardizing comparability across studies. To facilitate information dissemination, journal managers should reengineer their fast-track channels, which should be adapted to the purpose of an emerging outbreak, taking into account the requirement of high standards of quality for scientific journals and competition with other online resources.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Every now and then, a new infectious disease appears in a human population or an old disease becomes much more common or more geographically widespread. Recently, several such “emerging infectious diseases” have become major public health problems. For example, HIV/AIDS, hepatitis C, and severe acute respiratory syndrome (SARS) have all emerged in the past three decades and spread rapidly round the world. When an outbreak (epidemic) of an emerging infectious disease occurs, epidemiologists (scientists who study the causes, distribution, and control of diseases in populations) swing into action, collecting and analyzing data on the new threat to human health. Epidemiological studies are rapidly launched to identify the causative agent of the new disease, to investigate how the disease spreads, to define diagnostic criteria for the disease, to evaluate potential treatments, and to devise ways to control the disease's spread. Public health officials then use the results of these studies to bring the epidemic under control.
Why Was This Study Done?
Clearly, epidemics of emerging infectious diseases can only be controlled rapidly and effectively if the results of epidemiological studies are made widely available in a timely manner. Public health bulletins (for example, the Morbidity and Mortality Weekly Report from the US Centers from Disease Control and Prevention) are an important way of disseminating information as is the publication of original research in peer-reviewed academic journals. But how timely is this second dissemination route? Submission, peer-review, revision, re-review, acceptance, and publication of a piece of academic research can be a long process, the speed of which is affected by the responses of both authors and journals. In this study, the researchers analyze how the results of academic epidemiological research are submitted and published in journals during and after an emerging infectious disease epidemic using the 2003 SARS epidemic as an example. The first case of SARS was identified in Asia in February 2003 and rapidly spread around the world. 8,098 people became ill with SARS and 774 died before the epidemic was halted in July 2003.
What Did the Researchers Do and Find?
The researchers identified more than 300 journal articles covering epidemiological research into the SARS outbreak in Hong Kong, China, and Toronto, Canada (two cities strongly affected by the epidemic) that were published online or in print between January 1, 2003 and July 31, 2007. The researchers' analysis of these articles shows that more than half them were descriptive epidemiological studies, investigations that focused on describing the distribution of SARS; a third were analytical epidemiological studies that tried to discover the cause of SARS. Overall, 22% of the journal articles were submitted for publication during the epidemic. Only 8% of the articles were accepted for publication and only 7% were actually published during the epidemic. The median (average) submission-to-acceptance and acceptance-to-publication intervals for SARS articles submitted during the epidemic were 55 and 77.5 days, respectively, much shorter intervals than those for non-SARS articles published in the same journal issues. After the epidemic was over, the submission-to-acceptance and acceptance-to-publication intervals for SARS articles was similar to that of non-SARS articles.
What Do These Findings Mean?
These findings show that, although the academic response to the SARS epidemic was rapid, most articles on the epidemiology of SARS were published after the epidemic was over even though SARS was a major threat to public health. Possible reasons for this publication delay include the time taken by authors to prepare and undertake their studies, to write and submit their papers, and, possibly, their tendency to first submit their results to high profile journals. The time then taken by journals to review the studies, make decisions about publication, and complete the publication process might also have delayed matters. To minimize future delays in the publication of epidemiological research on emerging infectious diseases, epidemiologists could adopt common, predefined protocols and ready-to-use instruments, which would improve timeliness and ensure comparability across studies, suggest the researchers. Journals, in turn, could improve their fast-track procedures and could consider setting up online sections that could be activated when an emerging infectious disease outbreak occurred. Finally, journals could consider altering their review system to speed up the publication process provided the quality of the final published articles was not compromised.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000272.
The US National Institute of Allergy and Infectious Diseases provides information on emerging infectious diseases
The US Centers for Control and Prevention of Diseases also provides information about emerging infectious diseases, including links to other resources, and information on SARS
Wikipedia has a page on epidemiology (note that Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The World Health Organization has information on SARS (in several languages)
doi:10.1371/journal.pmed.1000272
PMCID: PMC2864302  PMID: 20454570
2.  Assessing Clinical and Life Sciences Performance of Research Institutions in Split, Croatia, 2000-2006 
Croatian medical journal  2008;49(2):164-174.
Aim
To evaluate publications of clinical and life scientists from research institutions in Split, Croatia, and the publication output from government-funded research projects of the University of Split School of Medicine.
Methods
We analyzed the number of publications from research institutions in Split, Croatia, in the 2000-2006 period, relative impact factors, predominant research fields, output of researchers from the University of Split School of Medicine receiving government research grants, and the average price of published article.
Results
From 2000 to 2006, clinical and life scientists published 350 articles indexed in Thomson Scientific database Current Contents. The number of articles increased from 30 in 2000 to 76 in 2006, and the average impact factor of journals where these articles were published increased from 2.03 in 2000 to 2.89 in 2006. Twenty percent of articles (72/350) were published in the Croatian Medical Journal. Principal investigators of the 12 research projects receiving government grants published 0 to 8 articles related to the project topic in the 2002-2006 research grant cycle. The research grantees published 78 original research articles, with an average price per article of € 29.210.
Conclusion
Although the number and impact factor of research articles published by clinical and life scientists from Split, Croatia, is increasing, it is still low when the number of scientists is taken into account. There should be better mechanisms of control and evaluation of research performance of government-funded research projects.
doi:10.3325/cmj.2008.2.164
PMCID: PMC2359877  PMID: 18461671
3.  Using Co-authorship Networks to Map and Analyse Global Neglected Tropical Disease Research with an Affiliation to Germany 
PLoS Neglected Tropical Diseases  2015;9(12):e0004182.
Background
Research on Neglected Tropical Diseases (NTDs) has increased in recent decades, and significant need-gaps in diagnostic and treatment tools remain. Analysing bibliometric data from published research is a powerful method for revealing research efforts, partnerships and expertise. We aim to identify and map NTD research networks in Germany and their partners abroad to enable an informed and transparent evaluation of German contributions to NTD research.
Methodology/Principal Findings
A SCOPUS database search for articles with German author affiliations that were published between 2002 and 2012 was conducted for kinetoplastid and helminth diseases. Open-access tools were used for data cleaning and scientometrics (OpenRefine), geocoding (OpenStreetMaps) and to create (Table2Net), visualise and analyse co-authorship networks (Gephi). From 26,833 publications from around the world that addressed 11 diseases, we identified 1,187 (4.4%) with at least one German author affiliation, and we processed 972 publications for the five most published-about diseases. Of those, we extracted 4,007 individual authors and 863 research institutions to construct co-author networks. The majority of co-authors outside Germany were from high-income countries and Brazil. Collaborations with partners on the African continent remain scattered. NTD research within Germany was distributed among 220 research institutions. We identified strong performers on an individual level by using classic parameters (number of publications, h-index) and social network analysis parameters (betweenness centrality). The research network characteristics varied strongly between diseases.
Conclusions/Significance
The share of NTD publications with German affiliations is approximately half of its share in other fields of medical research. This finding underlines the need to identify barriers and expand Germany’s otherwise strong research activities towards NTDs. A geospatial analysis of research collaborations with partners abroad can support decisions to strengthen research capacity, particularly in low- and middle-income countries, which were less involved in collaborations than high-income countries. Identifying knowledge hubs within individual researcher networks complements traditional scientometric indicators that are used to identify opportunities for collaboration. Using free tools to analyse research processes and output could facilitate data-driven health policies. Our findings contribute to the prioritisation of efforts in German NTD research at a time of impending local and global policy decisions.
Author Summary
Neglected tropical disease research has changed considerably in recent decades, and the German government is committed to addressing its past neglect of NTD research. Our aim was to use an innovative social network analysis of bibliometric data to map neglected tropical disease research networks that are inside of and affiliated with Germany, thereby enabling data-driven health policy decision-making. We created and analysed co-author networks from publications in the SCOPUS database, with a focus on five diseases. We found that Germany's share of global publication output for NTDs is approximately half that of other medical research fields. Furthermore, we identified institutions with prominent NTD research within Germany and strong research collaborations between German institutions and partners abroad, mostly in other high-income countries. This allowed an assessment of strong collaborations for further development, e.g., for research capacity strengthening in low-income-countries, but also for identifying missed opportunities for collaboration within the network. Through co-authorship network analysis of individual researcher networks, we identified strong performers by using classic bibliometric parameters, and we identified academic talent by social network analysis parameters on an individual level.
doi:10.1371/journal.pntd.0004182
PMCID: PMC4703140  PMID: 26719978
4.  Conflict of Interest Reporting by Authors Involved in Promotion of Off-Label Drug Use: An Analysis of Journal Disclosures 
PLoS Medicine  2012;9(8):e1001280.
Aaron Kesselheim and colleagues investigate conflict of interest disclosures in articles authored by physicians and scientists identified in whistleblower complaints alleging illegal off-label marketing by pharmaceutical companies.
Background
Litigation documents reveal that pharmaceutical companies have paid physicians to promote off-label uses of their products through a number of different avenues. It is unknown whether physicians and scientists who have such conflicts of interest adequately disclose such relationships in the scientific publications they author.
Methods and Findings
We collected whistleblower complaints alleging illegal off-label marketing from the US Department of Justice and other publicly available sources (date range: 1996–2010). We identified physicians and scientists described in the complaints as having financial relationships with defendant manufacturers, then searched Medline for articles they authored in the subsequent three years. We assessed disclosures made in articles related to the off-label use in question, determined the frequency of adequate disclosure statements, and analyzed characteristics of the authors (specialty, author position) and articles (type, connection to off-label use, journal impact factor, citation count/year). We identified 39 conflicted individuals in whistleblower complaints. They published 404 articles related to the drugs at issue in the whistleblower complaints, only 62 (15%) of which contained an adequate disclosure statement. Most articles had no disclosure (43%) or did not mention the pharmaceutical company (40%). Adequate disclosure rates varied significantly by article type, with commentaries less likely to have adequate disclosure compared to articles reporting original studies or trials (adjusted odds ratio [OR] = 0.10, 95%CI = 0.02–0.67, p = 0.02). Over half of the authors (22/39, 56%) made no adequate disclosures in their articles. However, four of six authors with ≥25 articles disclosed in about one-third of articles (range: 10/36–8/25 [28%–32%]).
Conclusions
One in seven authors identified in whistleblower complaints as involved in off-label marketing activities adequately disclosed their conflict of interest in subsequent journal publications. This is a much lower rate of adequate disclosure than has been identified in previous studies. The non-disclosure patterns suggest shortcomings with authors and the rigor of journal practices.
Please see later in the article for the Editors' Summary
Editor's Summary
Background
Off-label use of pharmaceuticals is the practice of prescribing a drug for a condition or age group, or in a dose or form of administration, that has not been specifically approved by a formal regulatory body, such as the US Food and Drug Administration (FDA). Off-label prescribing is common all over the world. In the US, although it is legal for doctors to prescribe drugs off-label and discuss such clinical uses with colleagues, it is illegal for pharmaceutical companies to directly promote off-label uses of any of their products. Revenue from off-label uses can be lucrative for drug companies and even surpass the income from approved uses. Therefore, many pharmaceutical companies have paid physicians and scientists to promote off-label use of their products as part of their marketing programs.
Why Was This Study Done?
Recently, a number of pharmaceutical companies have been investigated in the US for illegal marketing programs that promote off-label uses of their products and have had to pay billions of dollars in court settlements. As part of these investigations, doctors and scientists were identified who were paid by the companies to deliver lectures and conduct other activities to support off-label uses. When the same physicians and scientists also wrote articles about these drugs for medical journals, their financial relationships would have constituted clear conflicts of interest that should have been declared alongside the journal articles. So, in this study, the researchers identified such authors, examined their publications, and assessed the adequacy of conflict of interest disclosures made in these publications.
What Did the Researchers Do and Find?
The researchers used disclosed information from the US Department of Justice, media reports, and data from a non-governmental organization that tracks federal fraud actions, to find whistleblower complaints alleging illegal off-label promotion. Then they identified the doctors and scientists described in the complaints as having financial relationships with the defendant drug companies and searched Medline for articles authored by these experts in the subsequent three years. Using a four step approach, the researchers assessed the adequacy of conflict of interest disclosures made in articles relating to the off-label uses in question.
Using these methods, the researchers examined 26 complaints alleging illegal off-label promotion and identified the 91 doctors and scientists recorded as being involved in this practice. The researchers found 39 (43%) of these 91 experts had authored 404 related publications. In the complaints, these 39 experts were alleged to have engaged in 42 relationships with the relevant drug company: the most common activity was acting as a paid speaker (n = 26, 62%) but also writing reviews or articles on behalf of the company (n = 7), acting as consultants or advisory board members (n = 3), and receiving gifts/honoraria (n = 3), research support funds (n = 2), and educational support funds (n = 1). However, the researchers found that only 62 (15%) of the 404 related articles had adequate disclosures—43% (148) had no disclosure at all, 4% had statements denying any conflicts of interest, 40% had disclosures that did not mention the drug manufacturer, and 13% had disclosures that mentioned the manufacturer but inadequately conveyed the nature of the relationship between author and drug manufacturer reported in the complaint. The researchers also found that adequate disclosure rates varied significantly by article type, with commentaries significantly less likely to have adequate disclosure compared to articles reporting studies or trials.
What Do These Findings Mean?
These findings show the substantial deficiencies in the adequacy of conflict-of-interest disclosures made by authors who had been paid by pharmaceutical manufacturers as part of off-label marketing activities: only one in seven authors fully disclosed their conflict of interest in their published articles. This low figure is troubling and suggests that approaches to controlling the effects of conflicts of interest that rely on author candidness are inadequate and furthermore, journal practices are not robust enough and need to be improved. In the meantime, readers have no option but to interpret conflict of interest disclosures, particularly in relation to off-label uses, with caution.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001280.
The US FDA provides a guide on the use of off-label drugs
The US Agency for Healthcare Research and Quality offers a patient guide to off-label drugs
ProPublica offers a web-based tool to identify physicians who have financial relationships with certain pharmaceutical companies
Wikipedia has a good description of off-label drug use (note that Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The Institute for Medicine as a Profession maintains a list of policies regulating physicians' financial relationships that are in place at US-based academic medical centers
doi:10.1371/journal.pmed.1001280
PMCID: PMC3413710  PMID: 22899894
5.  Conflicts of Interest at Medical Journals: The Influence of Industry-Supported Randomised Trials on Journal Impact Factors and Revenue – Cohort Study 
PLoS Medicine  2010;7(10):e1000354.
Andreas Lundh and colleagues investigated the effect of publication of large industry-supported trials on citations and journal income, through reprint sales, in six general medical journals
Background
Transparency in reporting of conflict of interest is an increasingly important aspect of publication in medical journals. Publication of large industry-supported trials may generate many citations and journal income through reprint sales and thereby be a source of conflicts of interest for journals. We investigated industry-supported trials' influence on journal impact factors and revenue.
Methods and Findings
We sampled six major medical journals (Annals of Internal Medicine, Archives of Internal Medicine, BMJ, JAMA, The Lancet, and New England Journal of Medicine [NEJM]). For each journal, we identified randomised trials published in 1996–1997 and 2005–2006 using PubMed, and categorized the type of financial support. Using Web of Science, we investigated citations of industry-supported trials and the influence on journal impact factors over a ten-year period. We contacted journal editors and retrieved tax information on income from industry sources. The proportion of trials with sole industry support varied between journals, from 7% in BMJ to 32% in NEJM in 2005–2006. Industry-supported trials were more frequently cited than trials with other types of support, and omitting them from the impact factor calculation decreased journal impact factors. The decrease varied considerably between journals, with 1% for BMJ to 15% for NEJM in 2007. For the two journals disclosing data, income from the sales of reprints contributed to 3% and 41% of the total income for BMJ and The Lancet in 2005–2006.
Conclusions
Publication of industry-supported trials was associated with an increase in journal impact factors. Sales of reprints may provide a substantial income. We suggest that journals disclose financial information in the same way that they require them from their authors, so that readers can assess the potential effect of different types of papers on journals' revenue and impact.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Medical journals publish many different types of papers that inform doctors about the latest research advances and the latest treatments for their patients. They publish articles that describe laboratory-based research into the causes of diseases and the identification of potential new drugs. They publish the results of early clinical trials in which a few patients are given a potential new drug to check its safety. Finally and most importantly, they publish the results of randomized controlled trials (RCTs). RCTs are studies in which large numbers of patients are randomly allocated to different treatments without the patient or the clinician knowing the allocation and the efficacy of the various treatments compared. RCTs are best way of determining whether a new drug is effective and have to be completed before a drug can be marketed. Because RCTs are very expensive, they are often supported by drug companies. That is, drug companies provide grants or drugs for the trial or assist with data analysis and/or article preparation.
Why Was This Study Done?
Whenever a medical journal publishes an article, the article's authors have to declare any conflicts of interest such as financial gain from the paper's publication. Conflict of interest statements help readers assess papers—an author who owns the patent for a drug, for example, might put an unduly positive spin on his/her results. The experts who review papers for journals before publication provide similar conflict of interest statements. But what about the journal editors who ultimately decide which papers get published? The International Committee of Medical Journal Editors (ICMJE), which produces medical publishing guidelines, states that: “Editors who make final decisions about manuscripts must have no personal, professional, or financial involvement in any of the issues that they might judge.” However, the publication of industry-supported RCTs might create “indirect” conflicts of interest for journals by boosting the journal's impact factor (a measure of a journal's importance based on how often its articles are cited) and its income through the sale of reprints to drug companies. In this study, the researchers investigate whether the publication of industry-supported RCTs influences the impact factors and finances of six major medical journals.
What Did the Researchers Do and Find?
The researchers determined which RCTs published in the New England Journal of Medicine (NEJM), the British Medical Journal (BMJ), The Lancet, and three other major medical journals in 1996–1997 and 2005–2006 were supported wholly, partly, or not at all by industry. They then used the online academic citation index Web of Science to calculate an approximate impact factor for each journal for 1998 and 2007 and calculated the effect of the published RCTs on the impact factor. The proportion of RCTs with sole industry support varied between journals. Thus, 32% of the RCTs published in the NEJM during both two-year periods had industry support whereas only 7% of the RCTs published in the BMJ in 2005–2006 had industry support. Industry-supported trials were more frequently cited than RCTs with other types of support and omitting industry-supported RCTs from impact factor calculations decreased all the approximate journal impact factors. For example, omitting all RCTs with industry or mixed support decreased the 2007 BMJ and NEJM impact factors by 1% and 15%, respectively. Finally, the researchers asked each journal's editor about their journal's income from industry sources. For the BMJ and The Lancet, the only journals that provided this information, income from reprint sales was 3% and 41%, respectively, of total income in 2005–2006.
What Do These Findings Mean?
These findings show that the publication of industry-supported RCTs was associated with an increase in the approximate impact factors of these six major medical journals. Because these journals publish numerous RCTs, this result may not be generalizable to other journals. These findings also indicate that income from reprint sales can be a substantial proportion of a journal's total income. Importantly, these findings do not imply that the decisions of editors are affected by the possibility that the publication of an industry-supported trial might improve their journal's impact factor or income. Nevertheless, the researchers suggest, journals should live up to the same principles related to conflicts of interest as those that they require from their authors and should routinely disclose information on the source and amount of income that they receive.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000354.
This study is further discussed in a PLoS Medicine Perspective by Harvey Marcovitch
The International Committee of Medical Journal Editors provides information about the publication of medical research, including conflicts of interest
The World Association of Medical Editors also provides information on conflicts of interest in medical journals
Information about impact factors is provided by Thomson Reuters, a provider of intelligent information for businesses and professionals; Thomson Reuters also runs Web of Science
doi:10.1371/journal.pmed.1000354
PMCID: PMC2964336  PMID: 21048986
6.  Completeness of Reporting of Patient-Relevant Clinical Trial Outcomes: Comparison of Unpublished Clinical Study Reports with Publicly Available Data 
PLoS Medicine  2013;10(10):e1001526.
Beate Wieseler and colleagues compare the completeness of reporting of patient-relevant clinical trial outcomes between clinical study reports and publicly available data.
Please see later in the article for the Editors' Summary
Background
Access to unpublished clinical study reports (CSRs) is currently being discussed as a means to allow unbiased evaluation of clinical research. The Institute for Quality and Efficiency in Health Care (IQWiG) routinely requests CSRs from manufacturers for its drug assessments.
Our objective was to determine the information gain from CSRs compared to publicly available sources (journal publications and registry reports) for patient-relevant outcomes included in IQWiG health technology assessments (HTAs) of drugs.
Methods and Findings
We used a sample of 101 trials with full CSRs received for 16 HTAs of drugs completed by IQWiG between 15 January 2006 and 14 February 2011, and analyzed the CSRs and the publicly available sources of these trials. For each document type we assessed the completeness of information on all patient-relevant outcomes included in the HTAs (benefit outcomes, e.g., mortality, symptoms, and health-related quality of life; harm outcomes, e.g., adverse events). We dichotomized the outcomes as “completely reported” or “incompletely reported.” For each document type, we calculated the proportion of outcomes with complete information per outcome category and overall.
We analyzed 101 trials with CSRs; 86 had at least one publicly available source, 65 at least one journal publication, and 50 a registry report. The trials included 1,080 patient-relevant outcomes. The CSRs provided complete information on a considerably higher proportion of outcomes (86%) than the combined publicly available sources (39%). With the exception of health-related quality of life (57%), CSRs provided complete information on 78% to 100% of the various benefit outcomes (combined publicly available sources: 20% to 53%). CSRs also provided considerably more information on harms. The differences in completeness of information for patient-relevant outcomes between CSRs and journal publications or registry reports (or a combination of both) were statistically significant for all types of outcomes.
The main limitation of our study is that our sample is not representative because only CSRs provided voluntarily by pharmaceutical companies upon request could be assessed. In addition, the sample covered only a limited number of therapeutic areas and was restricted to randomized controlled trials investigating drugs.
Conclusions
In contrast to CSRs, publicly available sources provide insufficient information on patient-relevant outcomes of clinical trials. CSRs should therefore be made publicly available.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
People assume that, when they are ill, health care professionals will ensure that they get the best available treatment. In the past, clinicians used their own experience to make decisions about which treatments to offer their patients, but nowadays, they rely on evidence-based medicine—the systematic review and appraisal of clinical trials, studies that investigate the benefits and harms of drugs and other medical interventions in patients. Evidence-based medicine can guide clinicians, however, only if all the results of clinical research are available for evaluation. Unfortunately, the results of trials in which a new drug performs better than existing drugs are more likely to be published than those in which the new drug performs badly or has unwanted side effects (publication bias). Moreover, trial outcomes that support the use of a new treatment are more likely to be published than those that do not support its use (outcome reporting bias). Both types of bias pose a substantial threat to informed medical decision-making.
Why Was This Study Done?
Recent initiatives, such as making registration of clinical trials in a trial registry (for example, ClinicalTrials.gov) a precondition for publication in medical journals, aim to prevent these biases but are imperfect. Another way to facilitate the unbiased evaluation of clinical research might be to increase access to clinical study reports (CSRs)—detailed but generally unpublished accounts of clinical trials. Notably, information from CSRs was recently used to challenge conclusions based on published evidence about the efficacy and safety of the antiviral drug oseltamivir and the antidepressant reboxetine. In this study, the researchers compare the information available in CSRs and in publicly available sources (journal publications and registry reports) for the patient-relevant outcomes included in 16 health technology assessments (HTAs; analyses of the medical implications of the use of specific medical technologies) for drugs; the HTAs were prepared by the Institute for Quality and Efficiency in Health Care (IQWiG), Germany's main HTA agency.
What Did the Researchers Do and Find?
The researchers searched for published journal articles and registry reports for each of 101 trials for which the IQWiG had requested and received full CSRs from drug manufacturers during HTA preparation. They then assessed the completeness of information on the patient-relevant benefit and harm outcomes (for example symptom relief and adverse effects, respectively) included in each document type. Eighty-six of the included trials had at least one publicly available data source; the results of 15% of the trials were not available in either journals or registry reports. Overall, the CSRs provided complete information on 86% of the patient-related outcomes, whereas the combined publicly available sources provided complete information on only 39% of the outcomes. For individual outcomes, the CSRs provided complete information on 78%–100% of the benefit outcomes, with the exception of health-related quality of life (57%); combined publicly available sources provided complete information on 20%–53% of these outcomes. The CSRs also provided more information on patient-relevant harm outcomes than the publicly available sources.
What Do These Findings Mean?
These findings show that, for the clinical trials considered here, publicly available sources provide much less information on patient-relevant outcomes than CSRs. The generalizability of these findings may be limited, however, because the trials included in this study are not representative of all trials. Specifically, only CSRs that were voluntarily provided by drug companies were assessed, a limited number of therapeutic areas were covered by the trials, and the trials investigated only drugs. Nevertheless, these findings suggest that access to CSRs is important for the unbiased evaluation of clinical trials and for informed decision-making in health care. Notably, in June 2013, the European Medicines Agency released a draft policy calling for the proactive publication of complete clinical trial data (possibly including CSRs). In addition, the European Union and the European Commission are considering legal measures to improve the transparency of clinical trial data. Both these initiatives will probably only apply to drugs that are approved after January 2014, however, and not to drugs already in use. The researchers therefore call for CSRs to be made publicly available for both past and future trials, a recommendation also supported by the AllTrials initiative, which is campaigning for all clinical trials to be registered and fully reported.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001526.
Wikipedia has pages on evidence-based medicine, publication bias, and health technology assessment (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The ClinicalTrials.gov website is a searchable register of federally and privately supported clinical trials in the US; it provides information about all aspects of clinical trials
The European Medicines Agency (EMA) provides information about all aspects of the scientific evaluation and approval of new medicines in the European Union, and guidance on the preparation of clinical study reports; its draft policy on the release of data from clinical trials is available
Information about IQWiG is available (in English and German); Informed Health Online is a website provided by IQWiG that provides objective, independent, and evidence-based information for patients (also in English and German)
doi:10.1371/journal.pmed.1001526
PMCID: PMC3793003  PMID: 24115912
7.  Promotional Tone in Reviews of Menopausal Hormone Therapy After the Women's Health Initiative: An Analysis of Published Articles 
PLoS Medicine  2011;8(3):e1000425.
Adriane Fugh-Berman and colleagues analyzed a selection of published opinion pieces on hormone therapy and show that there may be a connection between receiving industry funding for speaking, consulting, or research and the tone of such opinion pieces.
Background
Even after the Women's Health Initiative (WHI) found that the risks of menopausal hormone therapy (hormone therapy) outweighed benefit for asymptomatic women, about half of gynecologists in the United States continued to believe that hormones benefited women's health. The pharmaceutical industry has supported publication of articles in medical journals for marketing purposes. It is unknown whether author relationships with industry affect promotional tone in articles on hormone therapy. The goal of this study was to determine whether promotional tone could be identified in narrative review articles regarding menopausal hormone therapy and whether articles identified as promotional were more likely to have been authored by those with conflicts of interest with manufacturers of menopausal hormone therapy.
Methods and Findings
We analyzed tone in opinion pieces on hormone therapy published in the four years after the estrogen-progestin arm of the WHI was stopped. First, we identified the ten authors with four or more MEDLINE-indexed reviews, editorials, comments, or letters on hormone replacement therapy or menopausal hormone therapy published between July 2002 and June 2006. Next, we conducted an additional search using the names of these authors to identify other relevant articles. Finally, after author names and affiliations were removed, 50 articles were evaluated by three readers for scientific accuracy and for tone. Scientific accuracy was assessed based on whether or not the findings of the WHI were accurately reported using two criteria: (1) Acknowledgment or lack of denial of the risk of breast cancer diagnosis associated with hormone therapy, and (2) acknowledgment that hormone therapy did not benefit cardiovascular disease endpoints. Determination of promotional tone was based on the assessment by each reader of whether the article appeared to promote hormone therapy. Analysis of inter-rater consistency found moderate agreement for scientific accuracy (κ = 0.57) and substantial agreement for promotional tone (κ = 0.65). After discussion, readers found 86% of the articles to be scientifically accurate and 64% to be promotional in tone. Themes that were common in articles considered promotional included attacks on the methodology of the WHI, arguments that clinical trial results should not guide treatment for individuals, and arguments that observational studies are as good as or better than randomized clinical trials for guiding clinical decisions. The promotional articles we identified also implied that the risks associated with hormone therapy have been exaggerated and that the benefits of hormone therapy have been or will be proven. Of the ten authors studied, eight were found to have declared payment for speaking or consulting on behalf of menopausal hormone manufacturers or for research support (seven of these eight were speakers or consultants). Thirty of 32 articles (90%) evaluated as promoting hormone therapy were authored by those with potential financial conflicts of interest, compared to 11 of 18 articles (61%) by those without such conflicts (p = 0.0025). Articles promoting the use of menopausal hormone therapy were 2.41 times (95% confidence interval 1.49–4.93) as likely to have been authored by authors with conflicts of interest as by authors without conflicts of interest. In articles from three authors with conflicts of interest some of the same text was repeated word-for-word in different articles.
Conclusion
There may be a connection between receiving industry funding for speaking, consulting, or research and the publication of promotional opinion pieces on menopausal hormone therapy.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Over the past three decades, menopausal hormones have been heavily promoted for preventing disease in women. However, the Women's Health Initiative (WHI) study—which enrolled more than 26,000 women in the US and which was published in 2004—found that estrogen-progestin and estrogen-only formulations (often prescribed to women around the age of menopause) increased the risk of stroke, deep vein thrombosis, dementia, and incontinence. Furthermore, this study found that the estrogen-progestin therapy increased rates of breast cancer. In fact, the estrogen-progestin arm of the WHI study was stopped in 2002 due to harmful findings, and the estrogen-only arm was stopped in 2004, also because of harmful findings. In addition, the study also found that neither therapy reduced cardiovascular risk or markedly benefited health-related quality of life measures.
Despite these results, two years after the results of WHI study were published, a survey of over 700 practicing gynecologists—the specialists who prescribe the majority of menopausal hormone therapies—in the US found that almost half did not find the findings of the WHI study convincing and that 48% disagreed with the decision to stop the trial early. Furthermore, follow-up surveys found similar results.
Why Was This Study Done?
It is unclear why gynecologists and other physicians continue to prescribe menopausal hormone therapies despite the results of the WHI. Some academics argue that published industry-funded reviews and commentaries may be designed to convey specific, but subtle, marketing messages and several academic analyses have used internal industry documents disclosed in litigation cases. So this study was conducted to investigate whether hormone therapy–promoting tone could be identified in narrative review articles and if so, whether these articles were more likely to have been authored by people who had accepted funding from hormone manufacturers.
What Did the Researchers Do and Find?
The researchers conducted a comprehensive literature search that identified 340 relevant articles published between July 2002 and June 2006—the four years following the cessation of the estrogen-progestin arm of the women's health initiative study. Ten authors had published four to six articles, 47 authored two or three articles, and 371 authored one article each. The researchers focused on authors who had published four or more articles in the four-year period under study and, after author names and affiliations were removed, 50 articles were evaluated by three readers for scientific accuracy and for tone. After individually analyzing a batch of articles, the readers met to provide their initial assessments, to discuss them, and to reach consensus on tone and scientific accuracy. Then after the papers were evaluated, each author was identified and the researchers searched for authors' potential financial conflicts of interest, defined as publicly disclosed information that the authors had received payment for research, speaking, or consulting on behalf of a manufacturer of menopausal hormone therapy.
Common themes in the 50 articles included arguments that clinical trial results should not guide treatment for individuals and suggestions that the risks associated with hormone therapy have been exaggerated and that the benefits of hormone therapy have been or will be proven. Furthermore, of the ten authors studied, eight were found to have received payment for research, speaking or consulting on behalf of menopause hormone manufacturers, and 30 of 32 articles evaluated as promoting hormone therapy were authored by those with potential financial conflicts of interest. Articles promoting the use of menopausal hormone therapy were more than twice as likely to have been written by authors with conflicts of interest as by authors without conflicts of interest. Furthermore, Three authors who were identified as having financial conflicts of interest were authors on articles where sections of their previously published articles were repeated word-for-word without citation.
What Do These Findings Mean?
The findings of this study suggest that there may be a link between receiving industry funding for speaking, consulting, or research and the publication of apparently promotional opinion pieces on menopausal hormone therapy. Furthermore, such publications may encourage physicians to continue prescribing these therapies to women of menopausal age. Therefore, physicians and other health care providers should interpret the content of review articles with caution. In addition, medical journals should follow the International Committee of Medical Journal Editors Uniform Requirements for Manuscripts, which require that all authors submit signed statements of their participation in authorship and full disclosure of any conflicts of interest.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000425.
The US National Heart, Lung, and Blood Institute has more information on the Womens Health Initiative
The US National Institutes of Health provide more information about the effects of menopausal hormone replacement therapy
The Office of Women's Health, U.S. Department of Health and Human Services provides information on menopausal hormone therapy
The International Committee of Medical Journal Editors Uniform Requirements for Manuscripts presents Uniform Requirements for Manuscripts published in biomedical journals
The National Womens Health Network, a consumer advocacy group that takes no industry money, has factsheets and articles about menopausal hormone therapy
PharmedOut, a Georgetown University Medical Center project, has many resources on pharmaceutical marketing practices
doi:10.1371/journal.pmed.1000425
PMCID: PMC3058057  PMID: 21423581
8.  An Update: NIH Research Funding for Palliative Medicine 2006 to 2010 
Journal of Palliative Medicine  2013;16(2):125-129.
Abstract
Background
Palliative care clinical and educational programs are expanding to meet the needs of seriously ill patients and their families. Multiple reports call for an enhanced palliative care evidence base.
Objective
To examine current National Institutes of Health (NIH) funding of palliative medicine research and changes since our 2008 report.1
Methods
We sought to identify NIH funding of palliative medicine from 2006 to 2010 in two stages. First, we searched the NIH grants database RePorter2 for grants with key words “palliative care,” “end-of-life care,” “hospice,” and “end of life.” Second, we identified palliative care researchers likely to have secured NIH funding using three strategies: (1) We abstracted the first and last authors' names from original investigations published in major palliative medicine journals from 2008 to 2010; (2) we abstracted these names from a PubMed generated list of all original articles published in major medicine, nursing, and subspecialty journals using the above key words Medical Subject Headings (MESH) terms “palliative care,” “end-of-life care,” “hospice,” and “end of life;” and (3) we identified editorial board members of palliative medicine journals and key members of palliative medicine research initiatives. We crossmatched the pooled names against NIH grants funded from 2006 to 2010.
Results
The NIH RePorter search yielded 653 grants and the author search identified an additional 352 grants. Compared to 2001 to 2005, 589 (240%) more grants were NIH funded. The 391 grants categorized as relevant to palliative medicine represented 294 unique PIs, an increase of 185 (269%) NIH funded palliative medicine researchers. The NIH supported 21% of the 1253 original palliative medicine research articles identified. Compared to 2001 to 2005, the percentage of grants funded by institutes other than the National Cancer Institute (NCI), the National Institute for Nursing Research (NINR), and the National Institute of Aging (NIA) increased from 15% to 20% of all grants.
Conclusions
When compared to 2001–2005, more palliative medicine investigators received NIH funding; and research funding has improved. Nevertheless, additional initiatives to further support palliative care research are needed.
doi:10.1089/jpm.2012.0427
PMCID: PMC3607902  PMID: 23336358
9.  Relationship between Funding Source and Conclusion among Nutrition-Related Scientific Articles 
PLoS Medicine  2007;4(1):e5.
Background
Industrial support of biomedical research may bias scientific conclusions, as demonstrated by recent analyses of pharmaceutical studies. However, this issue has not been systematically examined in the area of nutrition research. The purpose of this study is to characterize financial sponsorship of scientific articles addressing the health effects of three commonly consumed beverages, and to determine how sponsorship affects published conclusions.
Methods and Findings
Medline searches of worldwide literature were used to identify three article types (interventional studies, observational studies, and scientific reviews) about soft drinks, juice, and milk published between 1 January, 1999 and 31 December, 2003. Financial sponsorship and article conclusions were classified by independent groups of coinvestigators. The relationship between sponsorship and conclusions was explored by exact tests and regression analyses, controlling for covariates. 206 articles were included in the study, of which 111 declared financial sponsorship. Of these, 22% had all industry funding, 47% had no industry funding, and 32% had mixed funding. Funding source was significantly related to conclusions when considering all article types (p = 0.037). For interventional studies, the proportion with unfavorable conclusions was 0% for all industry funding versus 37% for no industry funding (p = 0.009). The odds ratio of a favorable versus unfavorable conclusion was 7.61 (95% confidence interval 1.27 to 45.73), comparing articles with all industry funding to no industry funding.
Conclusions
Industry funding of nutrition-related scientific articles may bias conclusions in favor of sponsors' products, with potentially significant implications for public health.
In 111 scientific articles on nonalcoholic beverages, articles with all industry funding were more than 7 times more likely to have favorable conclusions compared with articles with no industry funding.
Editors' Summary
Background.
Much of the money available for doing medical research comes from companies, as opposed to government agencies or charities. There is some evidence that when a research study is sponsored by an organization that has a financial interest in the outcome, the study is more likely to produce results that favor the funder (this is called “sponsorship bias”). This phenomenon is worrying, because if our knowledge about effectiveness and safety of medicines is based on biased findings, patients could suffer. However, it is not clear whether sponsorship bias extends beyond research into drugs, but also affects other types of research that is in the public interest. For example, research into the health benefits, or otherwise, of different types of food and drink may affect government guidelines, regulations, and the behavior patterns of members of the public. Were sponsorship bias also to exist in this area of research, the health of the wider public could be affected.
Why Was This Study Done?
There is not a great deal of evidence about whether sponsorship bias affects nutritional research (scientific studies that look at the relationship between food and/or drink, and health or disease states). Therefore, the group of researchers here set out to collect information from published nutritional research papers, to see if the type of sponsorship for the research studies was in any way linked with whether the main conclusions were favorable or unfavorable to the sponsor.
What Did the Researchers Do and Find?
The research study reported here used the scientific literature as a source of data. The researchers chose to examine one particular area of nutrition (nonalcoholic drinks including soft drinks, juices, and milk), so that their investigation would not be affected too much by variability between the different types of nutritional research. Using literature searches, the researchers identified all original research and scientific review articles published between January 1999 and December 2003 that examined soft drinks, juices, and milk; described research carried out in humans; and at the same time drew conclusions relevant to health or disease. Then, information from each published article was categorized: the conclusions were coded as either favorable, unfavorable, or neutral in relation to the health effects of the products being studied, and the article's funding was coded as either all industry (ie, food/drinks companies), no industry, or mixed. 206 published articles were analyzed and only 54% declared funding. The researchers found that, overall, there was a strong association between the type of funding available for these articles and the conclusions that were drawn. Articles sponsored exclusively by food/drinks companies were four to eight times more likely to have conclusions favorable to the financial interests of the sponsoring company than articles which were not sponsored by food or drinks companies.
What Do These Findings Mean?
These findings suggest that a high potential for bias exists in research into the health benefits or harms of nonalcoholic drinks. It is not clear from this research study why or how this bias comes about, but there are many different mechanisms that might cause it. The researchers suggest that certain initiatives might help to reduce bias, for example, increasing independent funding of nutrition research.
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/doi:10.1371/journal.pmed.0040005.
Conflict of Interest definition from Wikipedia (Wikipedia is an internet encyclopedia that anyone can edit)
The International Committee of Medical Journal Editors provides standard guidelines for practices at medical journals, including a section on sponsorship, authorship, and accountability
The Committee on Publication Ethics is a forum for journal editors to discuss issues related to the integrity of the scientific record, and it provides guidelines for editors and case studies for reference
The Good Publication Practice guidelines outline standards for responsible publication of research sponsored by pharmaceutical companies
doi:10.1371/journal.pmed.0040005
PMCID: PMC1764435  PMID: 17214504
10.  Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis 
PLoS Medicine  2009;6(9):e1000144.
Joseph Ross and colleagues examine publication rates of clinical trials and find low rates of publication even following registration in Clinicaltrials.gov.
Background
ClinicalTrials.gov is a publicly accessible, Internet-based registry of clinical trials managed by the US National Library of Medicine that has the potential to address selective trial publication. Our objectives were to examine completeness of registration within ClinicalTrials.gov and to determine the extent and correlates of selective publication.
Methods and Findings
We examined reporting of registration information among a cross-section of trials that had been registered at ClinicalTrials.gov after December 31, 1999 and updated as having been completed by June 8, 2007, excluding phase I trials. We then determined publication status among a random 10% subsample by searching MEDLINE using a systematic protocol, after excluding trials completed after December 31, 2005 to allow at least 2 y for publication following completion. Among the full sample of completed trials (n = 7,515), nearly 100% reported all data elements mandated by ClinicalTrials.gov, such as intervention and sponsorship. Optional data element reporting varied, with 53% reporting trial end date, 66% reporting primary outcome, and 87% reporting trial start date. Among the 10% subsample, less than half (311 of 677, 46%) of trials were published, among which 96 (31%) provided a citation within ClinicalTrials.gov of a publication describing trial results. Trials primarily sponsored by industry (40%, 144 of 357) were less likely to be published when compared with nonindustry/nongovernment sponsored trials (56%, 110 of 198; p<0.001), but there was no significant difference when compared with government sponsored trials (47%, 57 of 122; p = 0.22). Among trials that reported an end date, 75 of 123 (61%) completed prior to 2004, 50 of 96 (52%) completed during 2004, and 62 of 149 (42%) completed during 2005 were published (p = 0.006).
Conclusions
Reporting of optional data elements varied and publication rates among completed trials registered within ClinicalTrials.gov were low. Without greater attention to reporting of all data elements, the potential for ClinicalTrials.gov to address selective publication of clinical trials will be limited.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
People assume that whenever they are ill, health care professionals will make sure they get the best available treatment. But how do clinicians know which treatment is most appropriate? In the past, clinicians used their own experience to make treatment decisions. Nowadays, they rely on evidence-based medicine—the systematic review and appraisal of the results of clinical trials, studies that investigate the efficacy and safety of medical interventions in people. However, evidence-based medicine can only be effective if all the results from clinical trials are published promptly in medical journals. Unfortunately, the results of trials in which a new drug did not perform better than existing drugs or in which it had unwanted side effects often remain unpublished or only appear in the public domain many years after the drug has been approved for clinical use by the US Food and Drug Administration (FDA) and other governmental bodies.
Why Was This Study Done?
The extent of this “selective” publication, which can impair evidence-based clinical practice, remains unclear but is thought to be substantial. In this study, the researchers investigate the problem of selective publication by systematically examining the extent of publication of the results of trials registered in ClinicalTrials.gov, a Web-based registry of US and international clinical trials. ClinicalTrials.gov was established in 2000 by the US National Library of Medicine in response to the 1997 FDA Modernization Act. This act required preregistration of all trials of new drugs to provide the public with information about trials in which they might be able to participate. Mandatory data elements for registration in ClinicalTrials.gov initially included the trial's title, the condition studied in the trial, the trial design, and the intervention studied. In September 2007, the FDA Amendments Act expanded the mandatory requirements for registration in ClinicalTrials.gov by making it necessary, for example, to report the trial start date and to report primary and secondary outcomes (the effect of the intervention on predefined clinical measurements) in the registry within 2 years of trial completion.
What Did the Researchers Do and Find?
The researchers identified 7,515 trials that were registered within ClinicalTrials.gov after December 31, 1999 (excluding phase I, safety trials), and whose record indicated trial completion by June 8, 2007. Most of these trials reported all the mandatory data elements that were required by ClinicalTrials.gov before the FDA Amendments Act but reporting of optional data elements was less complete. For example, only two-thirds of the trials reported their primary outcome. Next, the researchers randomly selected 10% of the trials and, after excluding trials whose completion date was after December 31, 2005 (to allow at least two years for publication), determined the publication status of this subsample by systematically searching MEDLINE (an online database of articles published in selected medical and scientific journals). Fewer than half of the trials in the subsample had been published, and the citation for only a third of these publications had been entered into ClinicalTrials.gov. Only 40% of industry-sponsored trials had been published compared to 56% of nonindustry/nongovernment-sponsored trials, a difference that is unlikely to have occurred by chance. Finally, 61% of trials with a completion date before 2004 had been published, but only 42% of trials completed during 2005 had been published.
What Do These Findings Mean?
These findings indicate that, over the period studied, critical trial information was not included in the ClinicalTrials.gov registry. The FDA Amendments Act should remedy some of these shortcomings but only if the accuracy and completeness of the information in ClinicalTrials.gov is carefully monitored. These findings also reveal that registration in ClinicalTrials.gov does not guarantee that trial results will appear in a timely manner in the scientific literature. However, they do not address the reasons for selective publication (which may be, in part, because it is harder to publish negative results than positive results), and they are potentially limited by the methods used to discover whether trial results had been published. Nevertheless, these findings suggest that the FDA, trial sponsors, and the scientific community all need to make a firm commitment to minimize the selective publication of trial results to ensure that patients and clinicians have access to the information they need to make fully informed treatment decisions.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000144.
PLoS Medicine recently published two related articles on selected publication by Ida Sim and colleagues and by Lisa Bero and colleagues and an editorial discussing the FDA Amendments Act
ClinicalTrials.gov provides information about the US National Institutes of Health clinical trial registry, including background information about clinical trials, and a fact sheet detailing the requirements of the FDA Amendments Act 2007 for trial registration
The US Food and Drug Administration provides further information about drug approval in the US for consumers and health care professionals
doi:10.1371/journal.pmed.1000144
PMCID: PMC2728480  PMID: 19901971
11.  Trends of public health research output from India during 2001-2008 
BMC Medicine  2009;7:59.
Background
An understanding of how public health research output from India is changing in relation to the disease burden and public health priorities is required in order to inform relevant research development. We therefore studied the trends in the public health research output from India during 2001-2008 that was readily available in the public domain.
Methods
The scope and type of the published research from India in 2007 that was included in the PubMed database was assessed and compared with a previous similar assessment for 2002. Papers were classified based on the review of abstracts and original public health research papers were assessed in detail. Impact factors for the journals were used to compute quality-adjusted research output. The websites of governmental organizations, academic and research institutions and international organizations were searched in order to identify and review reports on original public health research produced in India from 2001 to 2008. The reports were classified based on the topics covered and quality and their trends over time were assessed.
Results
The number of original health research papers from India in PubMed doubled from 4494 in 2002 to 9066 in 2007. This included a 3.1-fold increase in public health research papers, but these comprised only 5% of the total papers in 2007. Within public health, the increase was lowest for the health system and policy category. Several major causes of disease burden in India continued to be underrepresented in the quality-adjusted public health research output in 2007. The number of papers evaluating population health interventions increased from 2002 to 2007, but there were none on the leading non-communicable causes of disease burden or on road traffic injuries. The number of identified original public health research reports increased by 64.7% from 204 in 2001-2004 to 336 in 2005-2008. The proportion of reports on reproductive and child health was very high but decreased slightly from 38.7% of the total in 2001-2004 to 31.5% in 2005-2008 (P = 0.09); those on the leading chronic non-communicable conditions and injuries increased from 6.4% to 13.4% (P = 0.01) but this was still much lower than their contribution to the disease burden. Health system/policy issues were the topic in 27.4% reports but health information issues were covered in a miniscule 0.6% reports. The proportion of reports that were evaluations increased slightly from 26% in 2001-2004 to 31.5% in 2005-2008, with this proportion being higher among the reports commissioned by international organizations (P < 0.001). The proportion of reports commissioned by Indian governmental organizations alone, or in collaboration with international organizations, doubled from 2001-2004 to 2005-2008 (P < 0.001). Only 25% of the total 540 reports had a quality score of adequate or better. The quality of reports produced by collaborations between Indian and international organizations was higher than those produced by Indian or international organizations alone (P < 0.001).
Conclusion
This is the first analysis from India that includes research reports in addition to published papers. It provides the most up-to-date understanding of public health research output from India. The increase in available public health research output and the increase in commissioning of this research by Indian governmental organizations are encouraging. However, the distribution of research topics and the quality of research reports continue to be unsatisfactory. It is necessary for health policy to address these continuing deficits in public health research in order to reduce the very large disease burden in India.
doi:10.1186/1741-7015-7-59
PMCID: PMC2766381  PMID: 19828017
12.  Characteristics and Trends of Orthopedic Publications between 2000 and 2009 
Clinics in Orthopedic Surgery  2011;3(3):225-229.
Background
This study was undertaken to investigate the trends of orthopedic publications during the last decade, and to document the country of origin, journal, funding source, and language of contribution using PubMed.
Methods
Orthopedic articles published between 2000 and 2009 were retrieved from PubMed using the following search terms: "orthopaedic[Affiliation] AND ("2000/1/1"[PDAT]: "2009/12/31"[PDAT])" and "orthopedic[Affiliation] AND ("2000/1/1"[PDAT]: "2009/12/31"[PDAT])." The articles were downloaded in XML file format, which contained the following information: article title, author names, journal names, publication dates, article types, languages, authors' affiliations and funding sources. These information was extracted, sorted, and rearranged using the database's management software. We investigated the annual number of published orthopedic articles worldwide and the annual rate of increase. Furthermore, the country of publication origin, journal, funding source, and language of contribution were also investigated.
Results
A total of 46,322 orthopedic articles were published and registered in PubMed in the last 10 years. The worldwide number of published orthopedic articles increased from 2,889 in 2000 to 6,909 in 2009, showing an annual increase of 384.6 articles, or an annualized compound rate of 10.2%. The United States ranked highest in the number of published orthopedic articles, followed by Japan, the United Kingdom, Germany, and the Republic of Korea. Among the orthopedic articles published worldwide during the last 10 years, 37.9% pertained studies performed in the United States. Fifty-seven point three percent (57.3%) of articles were published in journals established in the United States. Among the published orthopaedic articles, 4,747 articles (10.2%) disclosed financial support by research funds, of which 4,688 (98.8%) articles utilized research funds from the United States. Most articles were published in English (97.2%, 45,030 articles).
Conclusions
The number of published orthopedic articles has been increasing over the last decade. The number of orthopedic articles, journals publication, and funding sources were dominated by research conducted in the United States, while share and growth of Asian countries including Japan, the Republic of Korea, and China were notable.
doi:10.4055/cios.2011.3.3.225
PMCID: PMC3162203  PMID: 21909470
Bibliometrics; Orthopedics; Research trend; Periodicals as topics
13.  Statistical relationships between journal use and research output at academic institutions in South Korea 
Scientometrics  2015;103(3):751-777.
In this study, we analysed the statistical association between e-journal use and research output at the institution level in South Korea by performing comparative and diachronic analyses, as well as the analysis by field. The datasets were compiled from four different sources: national reports on research output indicators in science fields, two statistics databases on higher education institutions open to the public, and e-journal usage statistics generated by 47 major publishers. Due to the different data sources utilized, a considerable number of missing values appeared in our datasets and various mapping issues required corrections prior to the analysis. Two techniques for handling missing data were applied and the impact of each technique was discussed. In order to compile the institutional data by field, journals were first mapped, and then the statistics were summarized according to subject field. We observed that e-journal use exhibited stronger correlations with the number of publications and the times cited, in contrast to the number of undergraduates, graduates, faculty members and the amount of research funds, and this was the case regardless of the NA handling method or author type. The difference between the maximum correlation for the amount of external research funding with two average indicators and that of the correlation for e-journal use were not significant. Statistically, the accountability of e-journal use for the average times cited per article and the average JIF was quite similar with external research funds. It was found that the number of e-journal articles used had a strong positive correlation (Pearson’s correlation coefficients of r > 0.9, p < 0.05) with the number of articles published in SCI(E) journals and the times cited regardless of the author type, NA handling method or time period. We also observed that the top-five institutions in South Korea, with respect to the number of publications in SCI(E) journals, were generally across a balanced range of academic activities, while producing significant research output and using published material. Finally, we confirmed that the association of e-journal use with the two quantitative research indicators is strongly positive, even for the analyses by field, with the exception of the Arts and Humanities.
doi:10.1007/s11192-015-1563-0
PMCID: PMC4430588  PMID: 25995527
Correlation analysis; Comparative analysis; Diachronic analysis; e-Journal use; Research output
14.  Strategic faculty recruitment increases research productivity within an academic university division 
Canadian Journal of Surgery  2009;52(5):401-406.
Background
Research is an important mandate for academic surgical divisions. However, there is widespread concern that the current health care climate is leading to a decline in research activity. A University of British Columbia (UBC) academic surgical division attempted to address this concern by strategically recruiting PhD research scientists to prioritize research and develop collaborative research programs. The objective of our study was to determine whether this strategy resulted in increased research productivity.
Methods
We reviewed the UBC Department of Surgery database to assess research funding obtained by the Division of General Surgery for the years 1994–2004. We searched MEDLINE for peer-reviewed publications by faculty members during this period.
Results
Research funding increased from a mean of Can$417 292 per year in the 5 years (1994/95–1998/99) before the recruitment of dedicated PhD scientists to a mean of Can$1.3 million per year in the 5 years following the recruitment strategy (1999/2000–2003/04; p = 0.012). Funding for the initial 5 years was Can$2.1 million, including 1 Canadian Institutes of Health Research (CIHR) grant. Funding increased to Can$6.8 million, including 22 CIHR grants over the subsequent 5 years (p < 0.001). Collaborative research led to the awarding of multidisciplinary grants exceeding Can$4 million with divisional members as principle or coprinciple investigators. From 1994/05 to 1998/99, the total number of peer-reviewed publications was 116 (mean 23.2, standard deviation [SD] 7 per year), increasing to 144 from 1999/2000 to 2003/04 (mean 28.8, SD 13 per year). The trend was for publications in journals with higher impact factors in the latter 5-year period.
Conclusion
Strategic recruitment resulted in increased and sustained research productivity. Interactions between research scientists and clinicians resulted in successful program grant funding support. These results have implications for sustaining the research mission within academic departments of surgery.
PMCID: PMC2769093  PMID: 19865575
15.  Evaluation of medical and health economic effectiveness of non-pharmacological secondary prevention of coronary heart disease 
Background
Coronary heart disease (CHD) is a common and potentially fatal malady with a life time prevalence of over 20%. For Germany, the mortality attributable to chronic ischemic heart disease or acute myocardial infarction is estimated at 140,000 deaths per year. An association between prognosis of CHD and lifestyle risk factors has been consistently shown. To positively influence lifestyle risk factors in patients with CHD, non-pharmaceutical secondary prevention strategies are frequently recommended and implemented.
Objectives
The aim of this HTA (HTA = Health Technology Assessment) is to summarise the current literature on strategies for non-pharmaceutical secondary prevention in patients with CHD and to evaluate their medical effectiveness/efficacy and cost-effectiveness as well as the ethical, social and legal implications. In addition, this report aims to compare the effectiveness and efficacy of different intervention components and to evaluate the generalisability with regard to the German context.
Methods
Relevant publications were identified by means of a structured search of databases accessed through the German Institute of Medical Documentation and Information (DIMDI). In addition, a manual search of identified reference lists was conducted. The present report includes German and English literature published between January 2003 and September 2008 targeting adults with CHD. The methodological quality of included studies was assessed according to pre-defined quality criteria, based on the criteria of evidence based medicine.
Results
Among 9,074 publications 43 medical publications met the inclusion criteria. Overall study quality is satisfactory, but only half the studies report overall mortality or cardiac mortality as an outcome, while the remaining studies report less reliable outcome parameters. The follow-up duration varies between twelve and 120 months. Although overall effectiveness of non-pharmaceutical secondary prevention programs shows considerable heterogeneity, there is evidence for the long-term effectiveness concerning mortality, recurrent cardiac events and quality of life. Interventions based on exercise and also multicomponent interventions report more conclusive evidence for reducing mortality, while interventions focusing on psychosocial risk factors seem to be more effective in improving quality of life. Only two studies from Germany fulfill the methodological criteria and are included in this report.
Additionally, 25 economic publications met the inclusion criteria. Both, quantity and quality of publications dealing with combined interventions are higher compared with those investigating single component interventions. However, there are difficulties in transferring the international results into the German health care system, because of its specific structure of the rehabilitation system. While international literature mostly shows a positive cost-effectiveness ratio of combined programs, almost without exception, studies investigate out-of hospital or home-based programs. The examination of publications evaluating the cost-effectiveness of single interventions merely shows a positive trend of exercise-based and smoking cessation programs. Due to a lack of appropriate studies, no conclusive evidence regarding psychosocial and dietary interventions is available.
Altogether eleven publications concerned with ethical or social issues of non-pharmacological secondary prevention strategies are included. These studies are relatively confirm the assumption that patients with a lower socioeconomic background reflect a population at increased risk and therefore have specific needs to participate in rehabilitation programs. However, there currently remains uncertainty, whether these patients participate in rehabilitation more or less often. As barriers, which deter patients from attending, aspects like a lack of motivation, family commitments or the distance between home and rehabilitation centres are identified. Psychological factors like anxiety, depression and uncertainty as well as physical constraints are also pointed out.
Discussion
Non-pharmacological secondary preventive strategies are safe and effective in improving mortality, morbidity and quality of life in patients with CHD. Because of the small number of reliable studies with long term follow up over 60 months, sustainability of observed intervention effects has to be regarded with caution. Due to a lack of suitable studies, it was not possible to determine the effectiveness of interventions in important patient subgroups as well as the comparative effectiveness of different intervention strategies, conclusively. Future research should, amongst others, attempt to investigate these questions in methodologically rigorous studies.
With regard to the cost-effectiveness of non-pharmacological interventions, overall, international studies show positive results. However, there are considerable limitations due to the qualitative and quantitative deficiencies of identified studies. The special characteristics of the German rehabilitation system with its primarily inpatient offers result in further difficulties, when trying to transfer international study results to the German health care system. Both, studies demonstrating the cost-effectiveness of inpatient programs and those investigating the cost-effectiveness of single interventions are currently not available. To examine the German rehabilitation programs concerning their efficiency and their potential for optimisation, there is a need for further research.
Concerning social and ethical issues, a lack of studies addressing the structure of rehabilitation participants in Germany is striking. The same applies to studies examining the reasons for none participation in non-pharmacological secondary prevention programs. Evidence regarding these questions would provide an informative basis for optimising rehabilitation programs in Germany.
Conclusion
Non-pharmacological secondary prevention interventions are safe and able to reduce mortality from CHD and cardiac events, as well as to imporve patient’s quality of life. Nevertheless, there is considerable need for research; especially the effectiveness of interventions for important subgroups of CHD patients has to be evaluated. In addition to intervention effectiveness, there is also some evidence that interventions generate an appropriate cost-effectiveness ratio. However, future research should investigate this further. The same applies to the sustainability of secondary prevention programs and patient’s reasons for not attending them.
doi:10.3205/hta000078
PMCID: PMC3011286  PMID: 21289903
Coronary heart disease; secondary prevention; prevention, non-pharmacological; effectiveness; cost-effectiveness; efficiency; intervention, psychosocial; intervention, multimodal; exercise; training; reduction, stress; smoking cessation; dietary change; rehabilitation
16.  Misrepresentation of Randomized Controlled Trials in Press Releases and News Coverage: A Cohort Study 
PLoS Medicine  2012;9(9):e1001308.
A study conducted by Amélie Yavchitz and colleagues examines the factors associated with “spin” (specific reporting strategies, intentional or unintentional, that emphasize the beneficial effect of treatments) in press releases of clinical trials.
Background
Previous studies indicate that in published reports, trial results can be distorted by the use of “spin” (specific reporting strategies, intentional or unintentional, emphasizing the beneficial effect of the experimental treatment). We aimed to (1) evaluate the presence of “spin” in press releases and associated media coverage; and (2) evaluate whether findings of randomized controlled trials (RCTs) based on press releases and media coverage are misinterpreted.
Methods and Findings
We systematically searched for all press releases indexed in the EurekAlert! database between December 2009 and March 2010. Of the 498 press releases retrieved and screened, we included press releases for all two-arm, parallel-group RCTs (n = 70). We obtained a copy of the scientific article to which the press release related and we systematically searched for related news items using Lexis Nexis.
“Spin,” defined as specific reporting strategies (intentional or unintentional) emphasizing the beneficial effect of the experimental treatment, was identified in 28 (40%) scientific article abstract conclusions and in 33 (47%) press releases. From bivariate and multivariable analysis assessing the journal type, funding source, sample size, type of treatment (drug or other), results of the primary outcomes (all nonstatistically significant versus other), author of the press release, and the presence of “spin” in the abstract conclusion, the only factor associated, with “spin” in the press release was “spin” in the article abstract conclusions (relative risk [RR] 5.6, [95% CI 2.8–11.1], p<0.001). Findings of RCTs based on press releases were overestimated for 19 (27%) reports. News items were identified for 41 RCTs; 21 (51%) were reported with “spin,” mainly the same type of “spin” as those identified in the press release and article abstract conclusion. Findings of RCTs based on the news item was overestimated for ten (24%) reports.
Conclusion
“Spin” was identified in about half of press releases and media coverage. In multivariable analysis, the main factor associated with “spin” in press releases was the presence of “spin” in the article abstract conclusion.
Editors' Summary
Background
The mass media play an important role in disseminating the results of medical research. Every day, news items in newspapers and magazines and on the television, radio, and internet provide the general public with information about the latest clinical studies. Such news items are written by journalists and are often based on information in “press releases.” These short communications, which are posted on online databases such as EurekAlert! and sent directly to journalists, are prepared by researchers or more often by the drug companies, funding bodies, or institutions supporting the clinical research and are designed to attract favorable media attention to newly published research results. Press releases provide journalists with the information they need to develop and publish a news story, including a link to the peer-reviewed journal (a scholarly periodical containing articles that have been judged by independent experts) in which the research results appear.
Why Was This Study Done?
In an ideal world, journal articles, press releases, and news stories would all accurately reflect the results of health research. Unfortunately, the findings of randomized controlled trials (RCTs—studies that compare the outcomes of patients randomly assigned to receive alternative interventions), which are the best way to evaluate new treatments, are sometimes distorted in peer-reviewed journals by the use of “spin”—reporting that emphasizes the beneficial effects of the experimental (new) treatment. For example, a journal article may interpret nonstatistically significant differences as showing the equivalence of two treatments although such results actually indicate a lack of evidence for the superiority of either treatment. “Spin” can distort the transposition of research into clinical practice and, when reproduced in the mass media, it can give patients unrealistic expectations about new treatments. It is important, therefore, to know where “spin” occurs and to understand the effects of that “spin”. In this study, the researchers evaluate the presence of “spin” in press releases and associated media coverage and analyze whether the interpretation of RCT results based on press releases and associated news items could lead to the misinterpretation of RCT results.
What Did the Researchers Do and Find?
The researchers identified 70 press releases indexed in EurekAlert! over a 4-month period that described two-arm, parallel-group RCTs. They used Lexis Nexis, a database of news reports from around the world, to identify associated news items for 41 of these press releases and then analyzed the press releases, news items, and abstracts of the scientific articles related to each press release for “spin”. Finally, they interpreted the results of the RCTs using each source of information independently. Nearly half the press releases and article abstract conclusions contained “spin” and, importantly, “spin” in the press releases was associated with “spin” in the article abstracts. The researchers overestimated the benefits of the experimental treatment from the press release as compared to the full-text peer-reviewed article for 27% of reports. Factors that were associated with this overestimation of treatment benefits included publication in a specialized journal and having “spin” in the press release. Of the news items related to press releases, half contained “spin”, usually of the same type as identified in the press release and article abstract. Finally, the researchers overestimated the benefit of the experimental treatment from the news item as compared to the full-text peer-reviewed article in 24% of cases.
What Do These Findings Mean?
These findings show that “spin” in press releases and news reports is related to the presence of “spin” in the abstract of peer-reviewed reports of RCTs and suggest that the interpretation of RCT results based solely on press releases or media coverage could distort the interpretation of research findings in a way that favors experimental treatments. This interpretation shift is probably related to the presence of “spin” in peer-reviewed article abstracts, press releases, and news items and may be partly responsible for a mismatch between the perceived and real beneficial effects of new treatments among the general public. Overall, these findings highlight the important role that journal reviewers and editors play in disseminating research findings. These individuals, the researchers conclude, have a responsibility to ensure that the conclusions reported in the abstracts of peer-reviewed articles are appropriate and do not over-interpret the results of clinical research.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001308.
The PLOS Hub for Clinical Trials, which collects PLOS journals relating to clinical trials, includes some other articles on “spin” in clinical trial reports
EurekAlert is an online free database for science press releases
The UK National Health Service Choices website includes Beyond the Headlines, a resource that provides an unbiased and evidence-based analysis of health stories that make the news for both the public and health professionals
The US-based organization HealthNewsReview, a project supported by the Foundation for Informed Medical Decision Making, also provides expert reviews of news stories
doi:10.1371/journal.pmed.1001308
PMCID: PMC3439420  PMID: 22984354
17.  “Equal” Contributions and Credit: An Emerging Trend in the Characterization of Authorship 
Annals of epidemiology  2010;20(11):868-871.
Background
Scientific manuscripts sometimes have two or more authors explicitly designated as having “contributed equally” to the study. The prevalence and characteristics of this practice are not known. The goal of this study was to identify longitudinal trends and characteristics of the practice of explicitly giving authors equal credit in publications found in major medical journals.
Methods
We conducted electronic keyword searches looking for original research articles with equally credited authors (ECAs) published between January 1, 2000 and December 31, 2009 in the five general medicine journals with the highest impact factors (New England Journal of Medicine, Journal of the American Medical Association, Lancet, Annals of Internal Medicine, and British Medical Journal). The annual prevalence of original research articles with ECAs for each journal.
Results
Original research articles with authors explicitly given equal credit were found in all five journals. Articles with ECAs formed a greater proportion of the total number of articles published in each journal in 2009 versus published in 2000. [NEJM: 8.6% vs. <1%; JAMA: 7.5% vs. 0%; Annals: 3.8% vs. 0%; Lancet: 3.6% vs. <1%; BMJ: 1.0% vs. 0%]. There was a statistically significant increasing trend in yearly prevalence of ECA articles for all the journals [NEJM: p<0.0001; JAMA: p<0.001; Annals: p<0.001; Lancet: p<0.001, BMJ: p=0.001]. The first two authors listed in the byline received equal credit the majority of the time but the practice was also applied to authors in nearly every position in the byline. Finally, none of the journals provided specific guidance regarding this practice in their instructions to authors.
Conclusions
The practice of explicitly giving authors equal credit is increasingly common in original research publications. Scientific journals should consider providing guidance for authors regarding this practice. Furthermore, the potential impact of this practice on evaluations for academic promotion should be assessed.
doi:10.1016/j.annepidem.2010.08.004
PMCID: PMC2953548  PMID: 20933195
authorship; credit; equal; contributions; practice
18.  International Funding for Malaria Control in Relation to Populations at Risk of Stable Plasmodium falciparum Transmission 
PLoS Medicine  2008;5(7):e142.
Background
The international financing of malaria control has increased significantly in the last ten years in parallel with calls to halve the malaria burden by the year 2015. The allocation of funds to countries should reflect the size of the populations at risk of infection, disease, and death. To examine this relationship, we compare an audit of international commitments with an objective assessment of national need: the population at risk of stable Plasmodium falciparum malaria transmission in 2007.
Methods and Findings
The national distributions of populations at risk of stable P. falciparum transmission were projected to the year 2007 for each of 87 P. falciparum–endemic countries. Systematic online- and literature-based searches were conducted to audit the international funding commitments made for malaria control by major donors between 2002 and 2007. These figures were used to generate annual malaria funding allocation (in US dollars) per capita population at risk of stable P. falciparum in 2007. Almost US$1 billion are distributed each year to the 1.4 billion people exposed to stable P. falciparum malaria risk. This is less than US$1 per person at risk per year. Forty percent of this total comes from the Global Fund to Fight AIDS, Tuberculosis and Malaria. Substantial regional and national variations in disbursements exist. While the distribution of funds is found to be broadly appropriate, specific high population density countries receive disproportionately less support to scale up malaria control. Additionally, an inadequacy of current financial commitments by the international community was found: under-funding could be from 50% to 450%, depending on which global assessment of the cost required to scale up malaria control is adopted.
Conclusions
Without further increases in funding and appropriate targeting of global malaria control investment it is unlikely that international goals to halve disease burdens by 2015 will be achieved. Moreover, the additional financing requirements to move from malaria control to malaria elimination have not yet been considered by the scientific or international community.
To reach global malaria control goals, Robert Snow and colleagues argue that more international funding is needed but that it must be targeted at specific countries most at risk.
Editors' Summary
Background.
Malaria is one of the most common infectious diseases in the world and one of the greatest global public health problems. The Plasmodium falciparum parasite causes approximately 500 million cases each year and over one million deaths. More than 40% of the world's population is at risk of malaria.
The Millennium Development Goals (MDGs), established by the United Nations in 2000, include a target in Goal 6: “to have halted by 2015 and begun to reverse the incidence of malaria and other major diseases.” Following the launch of the MDG and international initiatives like Roll Back Malaria, there has been an upsurge in support for malaria control. This effort has included the formation of the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) and considerable funding from the US President's Malaria Initiative, the World Bank, the UK Department for International Development, USAID, and nongovernmental agencies and foundations like the Bill & Melinda Gates Foundation. But it is not yet clear how equitable or effective the financial commitments to malaria control have been.
Why Was This Study Done?
As part of the activities of the Malaria Atlas Project, the researchers had previously generated a global map of the limits of P. falciparum transmission. This map detailed areas where risk is moderate or high (stable transmission areas where malaria is endemic) and areas where the risk of transmission is low (unstable transmission areas where sporadic outbreaks of malaria may occur). Because the level of funding to control malaria should be proportionate to the size of the populations at risk, the researchers in this study appraised whether the areas of greatest need were receiving financial resources in proportion to this risk. That is, whether there is equity in how malaria funding is allocated.
What Did the Researchers Do and Find?
To assess the international financing of malaria control, the researchers conducted a audit of financial commitments to malaria control of the GFATM, national governments, and other donors for the period 2002 to 2007. To assess need, they estimated the population at risk of stable P. falciparum malaria transmission in 2007, building on their previous malaria map. Financial commitments were identified via online and literature searches, including the GFATM Web site, the World Malaria Report produced by WHO and UNICEF, and various other sources of financial information. Together these data allowed the authors to generate an estimate of the annual malaria funding allocation per capita population at risk of P. falciparum.
Of the 87 malaria-endemic countries, 76 received malaria funding commitments by the end of 2007. Overall, annual funding amounted to US$1 billion dollars, or less than US$1 per person at risk. Forty percent came from the GFATM, and the remaining from a mix of national government and external donors. The authors found great regional variation in the levels of funding. For example, looking at just the countries approved for GFATM funding, Myanmar was awarded an average annual per capita-at-risk amount of US$0.01 while Suriname was awarded US$147. With all financial commitments combined, ten countries had per capita annual support of more than US$4 per person, but 34 countries had less than US$1, including 16 where annual malaria support was less than US$0.5 per capita. These 16 countries represent 50% of the global population at risk and include seven of the poorest countries in Africa and two of the most densely populated stable endemic countries in the world (India and Indonesia).
What Do These Findings Mean?
The researchers find that the distribution of funds across the regions affected by malaria to be generally appropriate, with the Africa region and low-population-at-risk areas such as the Americas, the Caribbean, the Middle East, and Eastern Europe receiving proportionate annual malaria support. But they also identify large shortfalls, such as in the South East Asia and Western Pacific regions, which represents 47% of the global population at risk but received only 17% of GFATM and 24% of non-GFATM support. National government spending also falls short: for example, in Nigeria, where more than 100 million people are at risk of stable P. falciparum transmission, less than US$1 is invested per person per year. These findings illustrate how important it is to examine financial commitments against actual needs. Given the gaps between funding support and level of stable P. falciparum risk, the authors conclude that the goal to reduce the global burden of malaria by 2015 very likely will not be met with current commitments. They estimate that there remains a 50%–450% shortfall in funding needed to scale up malaria control worldwide.
Future research should assess the impact of these funding commitments and what additional resources will be needed if goals of malaria elimination are added to malaria control targets.
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0050142.
This study is discussed further in a PLoS Medicine Perspective by Anthony Kiszewski
The authors of this article have also published a global map of malaria risk; see Guerra, et al. (2008) PLoS Med 5(2) e38
Information is available from the Global Fund to Fight AIDS, Tuberculosis and Malaria
More information is available on global mapping of malaria risk from the Malaria Atlas Project
doi:10.1371/journal.pmed.0050142
PMCID: PMC2488181  PMID: 18651785
19.  Physician Emigration from Sub-Saharan Africa to the United States: Analysis of the 2011 AMA Physician Masterfile 
PLoS Medicine  2013;10(9):e1001513.
Siankam Tankwanchi and colleagues used the AMA Physician Masterfile and the WHO Global Health Workforce Statistics on physicians in sub-Saharan Africa to determine trends in physician emigration to the United States.
Please see later in the article for the Editors' Summary
Background
The large-scale emigration of physicians from sub-Saharan Africa (SSA) to high-income nations is a serious development concern. Our objective was to determine current emigration trends of SSA physicians found in the physician workforce of the United States.
Methods and Findings
We analyzed physician data from the World Health Organization (WHO) Global Health Workforce Statistics along with graduation and residency data from the 2011 American Medical Association Physician Masterfile (AMA-PM) on physicians trained or born in SSA countries who currently practice in the US. We estimated emigration proportions, year of US entry, years of practice before emigration, and length of time in the US. According to the 2011 AMA-PM, 10,819 physicians were born or trained in 28 SSA countries. Sixty-eight percent (n = 7,370) were SSA-trained, 20% (n = 2,126) were US-trained, and 12% (n = 1,323) were trained outside both SSA and the US. We estimated active physicians (age ≤70 years) to represent 96% (n = 10,377) of the total. Migration trends among SSA-trained physicians increased from 2002 to 2011 for all but one principal source country; the exception was South Africa whose physician migration to the US decreased by 8% (−156). The increase in last-decade migration was >50% in Nigeria (+1,113) and Ghana (+243), >100% in Ethiopia (+274), and >200% (+244) in Sudan. Liberia was the most affected by migration to the US with 77% (n = 175) of its estimated physicians in the 2011 AMA-PM. On average, SSA-trained physicians have been in the US for 18 years. They practiced for 6.5 years before US entry, and nearly half emigrated during the implementation years (1984–1999) of the structural adjustment programs.
Conclusion
Physician emigration from SSA to the US is increasing for most SSA source countries. Unless far-reaching policies are implemented by the US and SSA countries, the current emigration trends will persist, and the US will remain a leading destination for SSA physicians emigrating from the continent of greatest need.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Population growth and aging and increasingly complex health care interventions, as well as existing policies and market forces, mean that many countries are facing a shortage of health care professionals. High-income countries are addressing this problem in part by encouraging the immigration of foreign health care professionals from low- and middle-income countries. In the US, for example, international medical graduates (IMGs) can secure visas and permanent residency by passing examinations provided by the Educational Commission of Foreign Medical Graduates and by agreeing to provide care in areas that are underserved by US physicians. Inevitably, the emigration of physicians from low- and middle-income countries undermines health service delivery in the emigrating physicians' country of origin because physician supply is already inadequate in those countries. Physician emigration from sub-Saharan Africa, which has only 2% of the global physician workforce but a quarter of the global burden of disease, is particularly worrying. Since 1970, as a result of large-scale emigration and limited medical education, there has been negligible or negative growth in the density of physicians in many countries in sub-Saharan Africa. In Liberia, for example, in 1973, there were 7.76 physicians per 100,000 people but by 2008 there were only 1.37 physicians per 100,000 people; in the US, there are 250 physicians per 100,000 people.
Why Was This Study Done?
Before policy proposals can be formulated to address global inequities in physician distribution, a clear picture of the patterns of physician emigration from resource-limited countries is needed. In this study, the researchers use data from the 2011 American Medical Association Physician Masterfile (AMA-PM) to investigate the “brain drain” of physicians from sub-Saharan Africa to the US. The AMA-PM collects annual demographic, academic, and professional data on all residents (physicians undergoing training in a medical specialty) and licensed physicians who practice in the US.
What Did the Researchers Do and Find?
The researchers used data from the World Health Organization (WHO) Global Health Workforce Statistics and graduation and residency data from the 2011 AMA-PM to estimate physician emigration rates from sub-Saharan African countries, year of US entry, years of service provided before emigration to the US, and length of time in the US. There were 10,819 physicians who were born or trained in 28 sub-Saharan African countries in the 2011 AMA-PM. By using a published analysis of the 2002 AMA-PM, the researchers estimated that US immigration among sub-Saharan African-trained physicians had increased over the past decade for all the countries examined except South Africa, where physician emigration had decreased by 8%. Overall, the number of sub-Saharan African IMGs in the US had increased by 38% since 2002. More than half of this increase was accounted for by Nigerian IMGs. Liberia was the country most affected by migration of its physicians to the US—77% of its estimated 226 physicians were in the 2011 AMA-PM. On average, sub-Saharan African IMGs had been in the US for 18 years and had practiced for 6.5 years before emigration. Finally, nearly half of the sub-Saharan African IMGs had migrated to US between 1984 and 1995, years during which structural adjustment programs, which resulted in deep cuts to public health care services, were implemented in developing countries by international financial institutions as conditions for refinancing.
What Do These Findings Mean?
Although the sub-Saharan African IMGs in the 2011 AMA-PM only represent about 1% of all the physicians and less than 5% of the IMGs in the AMA-PM, these findings reveal a major loss of physicians from sub-Saharan Africa. They also suggest that emigration of physicians from sub-Saharan Africa is a growing problem and is likely to continue unless job satisfaction for physicians is improved in their country of origin. Moreover, because the AMA-PM only lists physicians who qualify for a US residency position, more physicians may have moved from sub-Saharan Africa to the US than reported here and may be working in other jobs incommensurate with their medical degrees (“brain waste”). The researchers suggest that physician emigration from sub-Saharan Africa to the US reflects the complexities in the labor markets for health care professionals in both Africa and the US and can be seen as low- and middle-income nations subsidizing the education of physicians in high-income countries. Policy proposals to address global inequities in physician distribution will therefore need both to encourage the recruitment, training, and retention of health care professionals in resource-limited countries and to persuade high-income countries to train more home-grown physicians to meet the needs of their own populations.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001513.
The Foundation for Advancement of International Medical Education and Research is a non-profit foundation committed to improving world health through education that was established in 2000 by the Educational Commission for Foreign Medical Graduates
The Global Health Workforce Alliance is a partnership of national governments, civil society, international agencies, finance institutions, researchers, educators, and professional associations dedicated to identifying, implementing and advocating for solutions to the chronic global shortage of health care professionals (available in several languages)
Information on the American Medical Association Physician Masterfile and the providers of physician data lists is available via the American Medical Associations website
The World Health Organization (WHO) annual World Health Statistics reports present the most recent health statistics for the WHO Member States
The Medical Education Partnership Initiative is a US-sponsored initiative that supports medical education and research in sub-Saharan African institutions, aiming to increase the quantity, quality, and retention of graduates with specific skills addressing the health needs of their national populations
CapacityPlus is the USAID-funded global project uniquely focused on the health workforce needed to achieve the Millennium Development Goals
Seed Global Health cultivates the next generation of health professionals by allying medical and nursing volunteers with their peers in resource-limited settings
"America is Stealing the Worlds Doctors", a 2012 New York Times article by Matt McAllester, describes the personal experience of a young doctor who emigrated from Zambia to the US
Path to United States Practice Is Long Slog to Foreign Doctors, a 2013 New York Times article by Catherine Rampell, describes the hurdles that immigrant physicians face in practicing in the US
doi:10.1371/journal.pmed.1001513
PMCID: PMC3775724  PMID: 24068894
20.  Is German Medical Education Research on the rise? An analysis of publications from the years 2004 to 2013 
Objectives: The point of departure for the present work is the observation that, in comparison with Anglo-American countries or the Netherlands, Germany was responsible for only a marginal number of international publications in the field of medical education research before 2004. Recent years, however, have seen an increase in the importance of medical education research in Germany. The objective of this article is to evaluate the extent to which this trend can be substantiated by increased German publishing activity since the year 2004 in international, English-language journals in the subject area of “medical education research”.
Methods: In the framework of a literature and content analysis, German-author articles from the years 2004 to 2013 in six international, English-language medical education research journals were evaluated. In order to obtain an overview of German research activity in this area, all project and original works with German first and last authors were identified and subjected to a more in-depth content analysis.
Results: In total, 10,055 articles were examined. The evaluation shows that between the years 2004 and 2013 179 articles (of which 145 are project or original works) by German authors were published in the journals examined. Fluctuations over the course of time were evidenced. The project and original works are primarily cross-sectional studies (27.8%) and randomised control studies (25.6%) on the subject of “teaching and learning methods” (43.6%).
Conclusions: In comparison with the years 2004-2008, a significant rise can be seen in the number of publications by German education researchers in international journals since the year 2009.
doi:10.3205/zma000972
PMCID: PMC4580754  PMID: 26413168
medical education research; literature analysis; content analysis; international publication
21.  The Relationship of Previous Training and Experience of Journal Peer Reviewers to Subsequent Review Quality 
PLoS Medicine  2007;4(1):e40.
Background
Peer review is considered crucial to the selection and publication of quality science, but very little is known about the previous experiences and training that might identify high-quality peer reviewers. The reviewer selection processes of most journals, and thus the qualifications of their reviewers, are ill defined. More objective selection of peer reviewers might improve the journal peer review process and thus the quality of published science.
Methods and Findings
306 experienced reviewers (71% of all those associated with a specialty journal) completed a survey of past training and experiences postulated to improve peer review skills. Reviewers performed 2,856 reviews of 1,484 separate manuscripts during a four-year study period, all prospectively rated on a standardized quality scale by editors. Multivariable analysis revealed that most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training). Being on an editorial board and doing formal grant (study section) review were each predictors for only one of our two comparisons. However, the predictive power of all variables was weak.
Conclusions
Our study confirms that there are no easily identifiable types of formal training or experience that predict reviewer performance. Skill in scientific peer review may be as ill defined and hard to impart as is “common sense.” Without a better understanding of those skills, it seems unlikely journals and editors will be successful in systematically improving their selection of reviewers. This inability to predict performance makes it imperative that all but the smallest journals implement routine review ratings systems to routinely monitor the quality of their reviews (and thus the quality of the science they publish).
A survey of experienced reviewers, asked about training they had received in peer review, found there are no easily identifiable types of formal training and experience that predict reviewer performance.
Editors' Summary
Background.
When medical researchers have concluded their research and written it up, the next step is to get it published as an article in a journal, so that the findings can be circulated widely. These published findings help determine subsequent research and clinical use. The editors of reputable journals, including PLoS Medicine, have to decide whether the articles sent to them are of good quality and accurate and whether they will be of interest to the readers of their journal. To do this they need to obtain specialist advice, so they contact experts in the topic of the research article and ask them to write reports. This is the process of scientific peer review, and the experts who write such reports are known as “peer reviewers.” Although the editors make the final decision, the advice and criticism of these peer reviewers to the editors is essential in making decisions on publication, and usually in requiring authors to make changes to their manuscript. The contribution that peer reviewers have made to the article by the time it is finally published may, therefore, be quite considerable.
Although peer review is accepted as a key part of the process for the publishing of medical research, many people have argued that there are flaws in the system. For example, there may be an element of luck involved; one author might find their paper being reviewed by a reviewer who is biased against the approach they have adopted or who is a very critical person by nature, and another author may have the good fortune to have their work considered by someone who is much more favorably disposed toward their work. Some reviewers are more knowledgeable and thorough in their work than others. The editors of medical journals try to take in account such biases and quality factors in their choice of peer reviewers or when assessing the reviews. Some journals have run training courses for experts who review for them regularly to try to make the standard of peer review as high as possible.
Why Was This Study Done?
It is hard for journal editors to know who will make a good peer reviewer, and there is no proven system for choosing them. The authors of this study wanted to identify the previous experiences and training that make up the background of good peer reviewers and compare them with the quality of the reviews provided. This would help journal editors select good people for the task in future, and as a result will affect the quality of science they publish for readers, including other researchers.
What Did the Researchers Do and Find?
The authors contacted all the regular reviewers from one specialist journal (Annals of Emergency Medicine). A total of 306 of these experienced reviewers (71% of all those associated with the journal) completed a survey of past training and experiences that might be expected to improve peer review skills. These reviewers had done 2,856 reviews of 1,484 separate manuscripts during a four-year study period, and during this time the quality of the reviews had been rated by the journal's editors. Surprisingly, most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training), and even these were only weak predictors.
What Do These Findings Mean?
This study suggest that there are no easily identifiable types of formal training or experience that predict peer reviewer performance, although it is clear that some reviewers (and reviews) are better than others. The authors suggest that it is essential therefore that journals routinely monitor the quality of reviews submitted to them to ensure they are getting good advice (a practice that is not universal).
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/doi:10.1371/journal.pmed.0040040
• WAME is an association of editors from many countries who seek to foster international cooperation among editors of peer-reviewed medical journals
• The Fifth International Congress on Peer Review and Biomedical Publication is one of a series of conferences on peer review
• The PLoS Medicine guidelines for reviewers outline what we look for in a review
• The Council of Science Editors promotes ethical scientific publishing practices
• An editorial also published in this issue of PLoS Medicine discusses the peer review process further
doi:10.1371/journal.pmed.0040040
PMCID: PMC1796627  PMID: 17411314
22.  Mapping pneumonia research: A systematic analysis of UK investments and published outputs 1997–2013 
EBioMedicine  2015;2(9):1193-1199.
Background
The burden of pneumonia continues to be substantial, particularly among the poorest in global society. We describe here the trends for UK pneumonia R&D investment and published outputs, and correlate with 2013 global mortality.
Methods
Data related to awards to UK institutions for pneumonia research from 1997 to 2013 were systematically sourced and categorised by disease area and type of science. Investment was compared to mortality figures in 2010 and 2013 for pneumonia, tuberculosis and influenza. Investment was also compared to publication data.
Results
Of all infectious disease research between 2011 and 2013 (£917.0 million), £28.8 million (3.1%) was for pneumonia. This was an absolute and proportionate increase from previous time periods. Translational pneumonia research (33.3%) received increased funding compared with 1997–2010 where funding was almost entirely preclinical (87.5%, here 30.9%), but high-burden areas such as paediatrics, elderly care and antimicrobial resistance received little investment. Annual investment remains volatile; publication temporal trends show a consistent increase. When comparing investment to global burden with a novel ‘investment by mortality observed’ metric, tuberculosis (£48.36) and influenza (£484.21) receive relatively more funding than pneumonia (£43.08), despite investment for pneumonia greatly increasing in 2013 compared to 2010 (£7.39). Limitations include a lack of private sector data and the need for careful interpretation of the comparisons with burden, plus categorisation is subjective.
Conclusions
There has been a welcome increase for pneumonia funding awarded to UK institutions in 2011–2013 compared with 1997–2010, along with increases for more translational research. Published outputs relating to pneumonia rose steadily from 1997 to 2013. Investment relative to mortality for pneumonia has increased, but it remains low compared to other respiratory infections and clear inequities remain. Analyses that measure investments in pneumonia can provide an insight into funding trends and research gaps.
Research in context
Pneumonia continues to be a high-burden illness around the globe. This paper shows that although research funding is increasing in the UK (between 1997 and 2013), it remains poorly funded compared to other important respiratory infectious diseases such as tuberculosis and influenza. Publications about pneumonia have been steadily increasing over time, indicating continuing academic and clinical interest in the topic. Though global mortality of pneumonia is declining, it should still be an area of high priority for funders, policymakers and researchers.
Highlights
•Pneumonia has historically received little research funding in the UK.•Pneumonia-related funding is increasing, particularly in translational research.•Compared to global burden, pneumonia remains poorly-funded compared to influenza and tuberculosis.
doi:10.1016/j.ebiom.2015.06.024
PMCID: PMC4588381  PMID: 26501117
Pneumonia; Pneumococcal; Investments; Financing; Policy; Bibliometrics; Publications
23.  Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey 
Objectives To assess the prevalence of honorary and ghost authors in six leading general medical journals in 2008 and compare this with the prevalence reported by authors of articles published in 1996.
Design Cross sectional survey using a web based questionnaire.
Setting International survey of journal authors.
Participants Sample of corresponding authors of 896 research articles, review articles, and editorial/opinion articles published in six general medical journals with high impact factors in 2008: Annals of Internal Medicine, JAMA, Lancet, Nature Medicine, New England Journal of Medicine, and PLoS Medicine.
Main outcome measures Self reported compliance with International Committee of Medical Journal Editors (ICMJE) criteria for authorship for all authors on the selected articles.
Results A total of 630/896 (70.3%) corresponding authors responded to the survey. The prevalence of articles with honorary authorship or ghost authorship, or both, was 21.0% (95% CI 18.0% to 24.3%), a decrease from 29.2% reported in 1996 (P=0.004). Based on 545 responses on honorary authorship, 96 articles (17.6% (95% CI 14.6% to 21.0%)) had honorary authors (range by journal 12.2% to 29.3%), a non-significant change from 1996 (19.3%; P=0.439). Based on 622 responses on ghost authorship, 49 articles (7.9% (6.0% to 10.3%)) had ghost authors (range by journal 2.1% to 11.0%), a significant decline from 1996 (11.5%; P=0.023). The prevalence of honorary authorship was 25.0% in original research reports, 15.0% in reviews, and 11.2% in editorials, whereas the prevalence of ghost authorship was 11.9% in research articles, 6.0% in reviews, and 5.3% in editorials.
Conclusions Evidence of honorary and ghost authorship in 21% of articles published in major medical journals in 2008 suggests that increased efforts by scientific journals, individual authors, and academic institutions are essential to promote responsibility, accountability, and transparency in authorship, and to maintain integrity in scientific publication.
doi:10.1136/bmj.d6128
PMCID: PMC3202014  PMID: 22028479
24.  Publication trends in noninvasive cardiovascular imaging: 1991-2011: a retrospective observational study 
The last twenty years have seen an explosive growth in cardiovascular disease research. The purpose of our study was to evaluate the characteristics of published research in the field of non-invasive cardiovascular imaging research from 1991-2011. Our aims were to determine: (1) the origin of the studies (international or from the U.S.) (2) differences in funding sources for U.S. publications and (3) if there has been an evolving trend pertaining to the mode of imaging. We evaluated characteristics of original research articles from Circulation, Circulation cardiovascular imaging, Journal of the American College of Cardiology (JACC), Journal of the American College of Cardiology cardiovascular imaging, Journal of the American Medical Association and the New England Journal of Medicine for the years 1991-91, 2001-02 and 2010-11. To establish trends for contributions for U.S. based studies and proportions of U.S. based studies receiving NIH funding in the study period, data was compared using a chi-square test. A two sided p value of less than or equal to 0.05 was used as the threshold for significance. Differences in modes of imaging under study were made by comparing average number of publications between the data sets in the study period using a t-test analysis. A total of 5431 studies were reviewed; 594 studies were selected as per the standardized abstraction criteria. U.S. based publications outnumbered international publications; its’ share declined from 77% in 1991-92 to 57% in 2010-2011 (p<0.0001). Funding for U.S. publications by the National Institutes of Health (NIH) remained static (1991-92: 40%; 2001-02: 49%; 2010-11: 42%). A decline was seen in the investigation of echocardiography (47%, p=0.44); cardiac computed tomography and cardiac magnetic resonance imaging studies grew 6.5 times (p=0.002) and 7-fold (p=0.01) respectively. Nuclear cardiology imaging fell by more than 50% (p=0.02). The last twenty years have seen a globalization of research in non-invasive cardiovascular imaging with a shift in focus towards investigation of cardiac magnetic resonance imaging. The decline in U.S. based publications coupled with a stasis in NIH funding may call for increased federal support for non-invasive imaging research.
PMCID: PMC3819584  PMID: 24224136
Cardiac; imaging; cardiovascular research; National Institutes of Health; funding; diagnostic techniques; echocardiography; Doppler; MRI; computed tomography
25.  The Third French Alzheimer Plan: analysis of the influence of a national public health initiative on scientific research productivity and impact 
Introduction
The Third National Alzheimer Plan (2008–2012) was a major public health initiative in France that included €200 million of funding for research in Alzheimer disease and related disorders (AD). The aim of this study was to document trends in French academic output in AD following the implementation of the plan.
Methods
Academic output (i.e., number of original articles) and scientific impact (i.e., article citations) of French research in AD were obtained from the Web of Knowledge core collection database. Analyses compared the 5-year period immediately before (2004–2008) and after (2009–2013) initiation of the plan. Comparisons were made with stroke, Parkinson disease, AIDS, and diabetes in the 14 leading countries worldwide and regionally within France.
Results
Worldwide production of original scientific articles between the periods 2004–2008 and 2009–2013 increased by 39 %, and that for AD increased by 46 %. China showed the largest increase and Japan the smallest. The absolute increase in French output on AD (54.6 %) was larger than that for stroke, Parkinson disease, AIDS, or diabetes. Globally, France had the third largest relative increase in output in AD (1.7-fold), behind only India (2.5-fold) and China (1.9-fold). There was a relative 2.3-fold increase in the proportion of French AD articles in the top 1 % globally most cited, larger than that for French articles on stroke, Parkinson disease, AIDS, or diabetes. At the national level, university hospitals participated in nearly 50 % of French AD publications. Analyses by geographical area demonstrated marked heterogeneity. We observed a strong correlation between level of funding and volume of output (R2 = 0.70), but not between funding and article impact (proportion of top 10 % globally cited articles; correlation R2 = 0.03).
Conclusions
Our study provides evidence of a specific positive impact of the funding provided by the Third National Alzheimer Plan in nearly doubling the global academic scientific output and increasing by 2.3-fold the top 1 % globally cited articles of France in AD research. Our bibliometric analyses provide objective and transparent information for policy makers on the relationship between research funding and academic output.
Electronic supplementary material
The online version of this article (doi:10.1186/s13195-015-0144-z) contains supplementary material, which is available to authorized users.
doi:10.1186/s13195-015-0144-z
PMCID: PMC4584022  PMID: 26407550

Results 1-25 (1690854)