Concern over COI and resultant risk of bias has engendered a variety of approaches and policies by biomedical journals, academic institutions, medical and continuing education institutions, healthcare delivery systems, professional organizations, and public sector policy makers. Central to COI policies is disclosure, and recent efforts to address COI have focused on disclosure and increased transparency. One such policy in the US is the Physician Payments Sunshine Act, signed into law in the US in 2010 as part of the Patient Protection and Affordable Care Act. This policy requires that pharmaceutical, medical device, biological, and medical supply manufacturers report to the U.S. Department of Health and Human Services payments that are more than $10 to physicians and teaching hospitals [10]. The law will require data reporting beginning in January 2013, including stock options, royalties, consulting fees, honoraria, education, research grants, meals, gifts, entertainment, and travel. In addition, the database will provide information on the physician receiving the payment, their address, payment date, and drug or device that the physician helped promote. There will be stiff penalties for both inadvertent lapses (up to $150,000 annually for failure to report) and intentional nondisclosure (up to $1 million annually) [10].

To date, drug companies have rarely disclosed payments to their speakers. Recently, however, a number of companies have begun posting doctors’ names and compensation on their web sites, some in anticipation of the Sunshine Act, others a result of legal settlements with the US government for illegal marketing of pharmaceuticals [11].

ProPublica obtained these data from open-access websites for each of the seven companies and aggregated them into a single database. The original data were available in a variety of formats, including unsortable PDFs. ProPublica programming staff used a variety of programs to scrape and clean the data, including Google Refine [12], Firebug [13], Adobe Acrobat [14], and others [15] to import these data into spreadsheets. Names of payees varied, even within a single company’s dataset, and Google Refine was used to aggregate the data for each individual listed when variations in name occurred such as the addition of an initial or an abbreviated first name. We verified the aggregation of data by individuals in the Dollars for Docs database using the Massachusetts General Hospital Utility Multiprogramming System (MUMPS), reanalyzing the raw dataset provided by ProPublica, and confirmed that our cohort did receive the stipulated payments.

For each recipient of a payment from one of the seven companies, the database includes the recipient’s name, city and state, reason for the payment, company making the payment, and the quarter(s) when the payment was made. Companies varied in their categorization of the reasons for payments, for example AstraZeneca and Merck only reported payments to speakers, whereas Eli Lilly, Cephalon and Pfizer reported payments for a variety of services, including meals, travel, and consulting. The exact date for the payments was not provided, and companies varied in their aggregation of calendar quarters. The database has been continuously updated since its first release with the addition of more companies, individuals, and payments. For this study, we used payment data from the initial database, released in October, 2010.

We selected as our study cohort individuals who received more than USD $100,000 in the period 2009 to 2010. We chose this cohort of payees because the large amounts of funds that these individuals received should likely be remembered and disclosed in concurrent publications, and we felt that these payees may have characteristics that make them somewhat homogeneous in their behaviors. Among these individuals, we examined only physicians, excluding other healthcare providers. We obtained demographic information on the cohort, including age, sex, years since medical school graduation, primary affiliation, specialty, and state of medical board certification. These demographic data were obtained from institutional websites, HealthGrades© and a general Internet search. All data were entered into Microsoft Access (Redmond, WA).

In order to examine whether physicians disclose these payments, we identified publications by physicians in our cohort in any authorship position by searching Medline from January 1, 2011 through March 31, 2011 using the Ovid search engine. We included all types of publications (e.g., primary research studies, reviews, editorials), and confined our search to English language publications. Results were imported into an EndNote library (Carlsbad, CA). If we identified publications by authors with similar names, we reviewed the full text to further examine the institutional affiliation, credentials, and subject matter of the article to determine if the individual in our cohort authored the publication. If there was any uncertainty about whether a physician in our cohort authored a specific publication, a second member of our research team independently examined the publications and if there was disagreement, consensus was achieved.

We then compared the COI disclosures to payments made to those individuals as indicated in the Dollars for Docs database. Because the lag time between completion of the COI disclosure and the date of publication (both electronic and print) is variable and unknown, and the database does not contain exact payment dates, we compared disclosures in 2011 publications to payments received in 2009 and 2010. This should produce a conservative (low) estimate of discrepancies, as COI disclosures in publications in 2011 should in all likelihood include payments to those authors in the prior 2 years.

The total payments to this physician cohort during the reporting period were $52,600,624, with a mean payment of $141,020 per physician (median $126,724, range $100,047 to $303,558) (Table ​(Table1).1). The mean total payment per physician per quarter for each company ranged from $21,305 for GlaxoSmithKline to $3596 for Johnson & Johnson. Payments were for consultation, continuing education presentation, and other services. None of the payments to this cohort of payees were categorized as research funding.

For specialties with five or more physicians represented in our cohort, mean payments ranged from $125,733 for psychiatry to $174,666 for anesthesiology. Four specialties (family medicine, internal medicine, surgery, and psychiatry) had 20 or more physicians represented in our cohort and the mean total payments per physician per quarter were significantly different among these specialties (P<0.0001). In particular, the mean total payment to physicians in internal medicine was significantly higher than to physicians in family medicine (mean difference $21,507; 95% confidence interval (CI) $1,533 to $41,482) and to psychiatrists (mean difference $25,623; 95% CI $13,176 to $38,069). The mean total payment to surgeons was also significantly higher than to psychiatrists (mean difference $27,990; 95% CI $6,750 to $49,229). The mean total payment to surgeons was not significantly different from that to physicians in internal medicine ($153,722 vs. $151,356). Based on results from the linear regression, the total payment per physician per quarter was not associated with sex (P=0.1730) or age (P=0.4908).

147 of the 373 physicians published a total of 1223 articles between January 2009 and the end of March, 2011 (mean 8 publications per physician, median 4, range 1–125). Of the 134 publications by physicians in the first quarter of 2011, 103 (77%) provided author disclosures of COI in the publication while the remaining 23% did not. Among publications with disclosures, 42 (41%) reported that the index author had nothing to disclose, while in 29 publications (28%) the author disclosed a conflict other than the payment in the Dollars for Docs database. In the remaining 32 publications (31%), the payment in the database was disclosed. The percentage of publications with the payer in the Dollars for Docs database disclosed, stratified by the 7 companies, is displayed in Figure ​Figure22.

In summary, among a cohort of physicians who were known recipients of large total payments from a select group of 7 pharmaceutical companies, nearly one-quarter of their concurrent publications do not report any COI disclosures. Furthermore, of the publications with disclosure statements, over two-thirds do not report payments listed in the Dollars for Docs database.

Several prior studies have examined the accuracy of COI disclosures in journal publications, with similar results. Two recent studies compared disclosures in publications to payment information from an orthopedic device company. Chimonas and colleagues [16] reported high rates of nondisclosures: 50% among publications directly or indirectly related to payments. Okike and coauthors [17] reported rates of nondisclosure in published abstracts from an orthopedic meeting of 20.7% for payments that were directly related, and 50% for payments that were indirectly related to the topic of the abstract. Other studies comparing abstracts and subsequent publications or concurrent publications also suggest a high prevalence of inaccurate author disclosures [18,19].

Our study has important strengths. Our data originated with payer databases, which were placed in the public domain either voluntarily or by mandate of legal settlements. The use of physician payment data from pharmaceutical companies is novel, to our knowledge, and our study provides insights into the methods, feasibility, and limitations of using such datasets for studies of discrepancies between payments known to have occurred and disclosure in peer-reviewed journals. When datasets are more complete and journal policies more uniform, these approaches can be used to determine the accuracy and completeness of disclosures.

Our study findings have important implications. Journal editors and readers rely on accurate disclosures by authors in order to assess the credibility and risk of bias in journal publications, and our findings suggest that the accuracy and completeness of disclosures cannot be assumed at present. As industry payment data increasingly become publically available, however, journal staff and readers will be able to verify the accuracy of author disclosures. Though in the US, until the Sunshine Act is implemented and there is a unified public reporting system for industry payments to physicians, it may not be feasible for journal editors and staff to check author disclosures. For authors, the increasing availability of payment data may promote more accurate disclosures or may discourage physicians from accepting industry payments, although there is as yet no evidence to support these possibilities. It is hoped that the standardized reporting form of the International Committee of Medical Journal Editors [20] will improve the accuracy of disclosures, however such data are not yet available.

There are a number of limitations to this pilot work. First, these are exploratory analyses on a small cohort of American physicians, whose characteristics and behaviors are likely not representative of US physicians in general, including those who receive much smaller total payments or those who do not receive funds from industry. Second, the database that we examined was limited to the payments of only 7 pharmaceutical companies during variable intervals in 2009 and 2010. The exact dates for payments were not available, and the categorization of types of payments varied across companies, making comparisons of the reasons for payments difficult.

Third, we faced challenges identifying publications by our physician cohort: exact author names vary (e.g., use of middle initial or not) and can resemble names of non-cohort authors. If we failed to identify all of the author’s publications, we may bias our assessment of rates of discrepancies if missing publications have a different disclosure rate than the rate in the publications that we identified. There is a delay in indexing publications in Medline thus more recent publications may have been missed.

Fourth, our assessment of rates of discrepancies cannot be equated with inaccurate disclosures because of several assumptions that we made. We did not examine journal COI policies, but rather assumed that all journals required disclosures. We also did not examine how journal COI policies address relevance of the topic of the publication to the disclosure. We assumed that all payments in the database should have been disclosed in the identified publications.

It was difficult to determine the relationship between the time of disclosure and payments as it was not possible to know exactly when authors completed disclosures relative to the time of publication, as this varies widely across journals. Although some journals request an update of disclosures at the time of publication, it is unknown how frequently authors comply and how accurate those updates are. In addition, we only had information on the quarter or group of quarters when payments occurred, and not the exact dates. Although unlikely, we could have overestimated rates of discrepancies if disclosure occurred well prior to publication and prior to drug company payment, or if a disclosure was close to the publication date and the required time period of disclosure was short, with payment preceding the disclosure period. We feel that our approach of examining 2011 publications should minimize these biases, as disclosures in publications likely encompass payments in the prior 2 years. Chimonas and colleagues [16] used a similar approach when they compared payments to physicians by orthopedic device companies to disclosures in orthopedic publications 1 to 2 years after the payments were made.

Finally, we were unable to determine the reasons for discrepancies, which are undoubtedly multifactorial, including forgetting, carelessness, misreading or misinterpreting COI policies, lack of clarity in disclosure forms, or purposeful nonreporting. From these retrospective, observational data we clearly cannot determine causality or impute motive.

Future research can build on this work. Studies could further describe the characteristics of physicians and other healthcare professionals who accept payments across a broad range of values and payment types. The effectiveness of specific COI policies and disclosure forms for achieving accurate and complete disclosures should be explored for journals, institutional review boards, academic institutions, and other organizations. Predictors of the accuracy of COI disclosures can be examined when more complete payment data are available with implementation of the Sunshine Act [10]. The effect of public disclosure of industry payments may be linked to provider behavior such as the future acceptance of industry payments or the prescription of specific medications when individual prescribing data are available.