Related Articles

We describe a specific mentoring approach in an academic general internal medicine setting by audiotaping and transcribing all mentoring sessions in the year. In advance, the mentor recorded his model. During the year, the mentee kept a process journal.

Qualitative analysis revealed development of an intimate relationship based on empathy, trust, and honesty. The mentor's model was explicitly intended to develop independence, initiative, improved thinking, skills, and self-reflection. The mentor's methods included extensive and varied use of questioning, active listening, standard setting, and frequent feedback. During the mentoring, the mentee evolved as a teacher, enhanced the creativity in his teaching, and matured as a person. Specific accomplishments included a national workshop on professional writing, an innovative approach to inpatient attending, a new teaching skills curriculum for a residency program, and this study.

A mentoring model stressing safety, intimacy, honesty, setting of high standards, praxis, and detailed planning and feedback was associated with mentee excitement, personal and professional growth and development, concrete accomplishments, and a commitment to teaching.

A comprehensive mentoring program includes a variety of components. One of the most important is the ongoing assessment of and feedback to mentors. Scholars need strong active mentors who have the expertise, disposition, motivation, skills, and the ability to accept feedback and to adjust their mentoring style. Assessing the effectiveness of a given mentor is no easy task. Variability in learning needs and academic goals among scholars makes it difficult to develop a single evaluation instrument or a standardized procedure for evaluating mentors. Scholars, mentors, and program leaders are often reluctant to conduct formal evaluations, as there are no commonly accepted measures. The process of giving feedback is often difficult and there is limited empirical data on efficacy. This article presents a new and innovative six-component approach to mentor evaluation that includes the assessment of mentee training and empowerment, peer learning and mentor training, scholar advocacy, mentee–mentor expectations, mentor self-reflection, and mentee evaluation of their mentor.

Background

Mentoring is a core component of medical education and career success. There is increasing global emphasis on mentorship of young scientists in order to train and develop the next leaders in global health. However, mentoring efforts are challenged by the high clinical, research and administrative demands. We evaluated the status and nature of mentoring practices at Makerere University College of Health Sciences (MAKCHS).

Methods

Pre-tested, self-administered questionnaires were sent by email to all Fogarty alumni at the MAKCHS (mentors) and each of them was requested to complete and email back the questionnaire. In addition to training level and number of mentors, the questionnaires had open-ended questions covering themes such as; status of mentorship, challenges faced by mentors and strategies to improve and sustain mentorship within MAKCHS. Similarly, open-ended questionnaires were sent and received by email from all graduate students (mentees) registered with the Uganda Society for Health Scientists (USHS). Qualitative data from mentors and mentees was analyzed manually according to the pre-determined themes.

Results

Twenty- two out of 100 mentors responded (14 email and 8 hard copy responses). Up to 77% (17/22) of mentors had Master's-level training and only 18% (4/22) had doctorate-level training. About 40% of the mentors had ≥ two mentees while 27% had none. Qualitative results showed that mentors needed support in terms of training in mentoring skills and logistical/financial support to carry out successful mentorship. Junior scientists and students reported that mentorship is not yet institutionalized and it is currently occurring in an adhoc manner. There was lack of awareness of roles of mentors and mentees. The mentors mentioned the limited number of practicing mentors at the college and thus the need for training courses and guidelines for faculty members in regard to mentorship at academic institutions.

Conclusions

Both mentors and mentees were willing to improve mentorship practices at MAKCHS. There is need for institutional commitment to uphold and sustain the mentorship best practices. We recommend a collaborative approach by the stakeholders in global health promotion to build local capacity in mentoring African health professionals.

Background

Peer review is considered crucial to the selection and publication of quality science, but very little is known about the previous experiences and training that might identify high-quality peer reviewers. The reviewer selection processes of most journals, and thus the qualifications of their reviewers, are ill defined. More objective selection of peer reviewers might improve the journal peer review process and thus the quality of published science.

Methods and Findings

306 experienced reviewers (71% of all those associated with a specialty journal) completed a survey of past training and experiences postulated to improve peer review skills. Reviewers performed 2,856 reviews of 1,484 separate manuscripts during a four-year study period, all prospectively rated on a standardized quality scale by editors. Multivariable analysis revealed that most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training). Being on an editorial board and doing formal grant (study section) review were each predictors for only one of our two comparisons. However, the predictive power of all variables was weak.

Conclusions

Our study confirms that there are no easily identifiable types of formal training or experience that predict reviewer performance. Skill in scientific peer review may be as ill defined and hard to impart as is “common sense.” Without a better understanding of those skills, it seems unlikely journals and editors will be successful in systematically improving their selection of reviewers. This inability to predict performance makes it imperative that all but the smallest journals implement routine review ratings systems to routinely monitor the quality of their reviews (and thus the quality of the science they publish).

A survey of experienced reviewers, asked about training they had received in peer review, found there are no easily identifiable types of formal training and experience that predict reviewer performance.

Editors' Summary

Background.

When medical researchers have concluded their research and written it up, the next step is to get it published as an article in a journal, so that the findings can be circulated widely. These published findings help determine subsequent research and clinical use. The editors of reputable journals, including PLoS Medicine, have to decide whether the articles sent to them are of good quality and accurate and whether they will be of interest to the readers of their journal. To do this they need to obtain specialist advice, so they contact experts in the topic of the research article and ask them to write reports. This is the process of scientific peer review, and the experts who write such reports are known as “peer reviewers.” Although the editors make the final decision, the advice and criticism of these peer reviewers to the editors is essential in making decisions on publication, and usually in requiring authors to make changes to their manuscript. The contribution that peer reviewers have made to the article by the time it is finally published may, therefore, be quite considerable.

Although peer review is accepted as a key part of the process for the publishing of medical research, many people have argued that there are flaws in the system. For example, there may be an element of luck involved; one author might find their paper being reviewed by a reviewer who is biased against the approach they have adopted or who is a very critical person by nature, and another author may have the good fortune to have their work considered by someone who is much more favorably disposed toward their work. Some reviewers are more knowledgeable and thorough in their work than others. The editors of medical journals try to take in account such biases and quality factors in their choice of peer reviewers or when assessing the reviews. Some journals have run training courses for experts who review for them regularly to try to make the standard of peer review as high as possible.

Why Was This Study Done?

It is hard for journal editors to know who will make a good peer reviewer, and there is no proven system for choosing them. The authors of this study wanted to identify the previous experiences and training that make up the background of good peer reviewers and compare them with the quality of the reviews provided. This would help journal editors select good people for the task in future, and as a result will affect the quality of science they publish for readers, including other researchers.

What Did the Researchers Do and Find?

The authors contacted all the regular reviewers from one specialist journal (Annals of Emergency Medicine). A total of 306 of these experienced reviewers (71% of all those associated with the journal) completed a survey of past training and experiences that might be expected to improve peer review skills. These reviewers had done 2,856 reviews of 1,484 separate manuscripts during a four-year study period, and during this time the quality of the reviews had been rated by the journal's editors. Surprisingly, most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training), and even these were only weak predictors.

What Do These Findings Mean?

This study suggest that there are no easily identifiable types of formal training or experience that predict peer reviewer performance, although it is clear that some reviewers (and reviews) are better than others. The authors suggest that it is essential therefore that journals routinely monitor the quality of reviews submitted to them to ensure they are getting good advice (a practice that is not universal).

Additional Information.

Please access these Web sites via the online version of this summary at http://dx.doi.org/doi:10.1371/journal.pmed.0040040

• WAME is an association of editors from many countries who seek to foster international cooperation among editors of peer-reviewed medical journals

• The Fifth International Congress on Peer Review and Biomedical Publication is one of a series of conferences on peer review

• The PLoS Medicine guidelines for reviewers outline what we look for in a review

• The Council of Science Editors promotes ethical scientific publishing practices

• An editorial also published in this issue of PLoS Medicine discusses the peer review process further

Objective To investigate the effect of an additional review based on reporting guidelines such as STROBE and CONSORT on quality of manuscripts.

Design Masked randomised trial.

Population Original research manuscripts submitted to the Medicina Clínica journal from May 2008 to April 2009 and considered suitable for publication.

Intervention Control group: conventional peer reviews alone. Intervention group: conventional review plus an additional review looking for missing items from reporting guidelines.

Outcomes Manuscript quality, assessed with a 5 point Likert scale (primary: overall quality; secondary: average quality of specific items in paper). Main analysis compared groups as allocated, after adjustment for baseline factors (analysis of covariance); sensitivity analysis compared groups as reviewed. Adherence to reviewer suggestions assessed with Likert scale.

Results Of 126 consecutive papers receiving conventional review, 34 were not suitable for publication. The remaining 92 papers were allocated to receive conventional reviews alone (n=41) or additional reviews (n=51). Four papers assigned to the conventional review group deviated from protocol; they received an additional review based on reporting guidelines. We saw an improvement in manuscript quality in favour of the additional review group (comparison as allocated, 0.25, 95% confidence interval –0.05 to 0.54; as reviewed, 0.33, 0.03 to 0.63). More papers with additional reviews than with conventional reviews alone improved from baseline (22 (43%) v eight (20%), difference 23.6% (3.2% to 44.0%), number needed to treat 4.2 (from 2.3 to 31.2), relative risk 2.21 (1.10 to 4.44)). Authors in the additional review group adhered more to suggestions from conventional reviews than to those from additional reviews (average increase 0.43 Likert points (0.19 to 0.67)).

Conclusions Additional reviews based on reporting guidelines improve manuscript quality, although the observed effect was smaller than hypothesised and not definitively demonstrated. Authors adhere more to suggestions from conventional reviews than to those from additional reviews, showing difficulties in adhering to high methodological standards at the latest research phases. To boost paper quality and impact, authors should be aware of future requirements of reporting guidelines at the very beginning of their study.

Trial registration and protocol Although registries do not include trials of peer review, the protocol design was submitted to sponsored research projects (Instituto de Salud Carlos III, PI081903).

This study explored the nonkin natural mentoring relationships among older youths in foster care. Three hundred thirty-nine youths nearing their exit from one state’s alternative care system were interviewed. Those that reported the presence of a natural mentor, 62% (n=211), were subsequently asked about how they met their mentor, how long they have known their mentor, how frequently they have contact with their mentor, and the quality of the relationship. Seventy percent of the youth with mentors reported having known their mentors for over 1 year. Youth reported high levels of contact and relationship quality. Ethnic group membership, gender, where youth were living, and clinical characteristics were associated with individual mentoring characteristics. Child and adult service delivery systems might benefit from considering ways to support the continuation of these relationships during the transition period.

Background

Research funders, educators, investigators and decision makers worldwide have identified the need to improve the quality of health care by building capacity for knowledge translation (KT) research and practice. Peer-based mentorship represents a vehicle to foster KT capacity. The purpose of this exploratory study is to identify mentoring models that could be used to build KT capacity, consult with putative mentee stakeholders to understand their KT mentorship needs and preferences, and generate recommendations for the content and format of KT mentorship strategies or programs, and how they could be tested through future research.

Methods

A conceptual framework was derived based on mentoring goals, processes and outcomes identified in the management and social sciences literature, and our research on barriers and facilitators of academic mentorship. These concepts will inform data collection and analysis. To identify useful models by which to design, implement and evaluate KT mentorship, we will review the social sciences, management, and nursing literature from 1990 to current, browse tables of contents of relevant journals, and scan the references of all eligible studies. Eligibility screening and data extraction will be performed independently by two investigators. Semi-structured interviews will be used to collect information about KT needs, views on mentorship as a knowledge sharing strategy, preferred KT mentoring program elements, and perceived barriers from clinician health services researchers representing different disciplines. Qualitative analysis of transcripts will be performed independently by two investigators, who will meet to compare findings and resolve differences through discussion. Data will be shared and discussed with the research team, and their feedback incorporated into final reports.

Discussion

These findings could be used by universities, research institutes, funding agencies, and professional organizations in Canada and elsewhere to develop, implement, and evaluate mentorship for KT research and practice. This research will establish a theoretical basis upon which we and others can compare the cost-effectiveness of interventions that enhance KT mentorship. If successful, this program of research may increase knowledge about, confidence in, and greater utilization of KT processes, and the quality and quantity of KT research, perhaps ultimately leading to better implementation and adoption of recommended health care services.

Introduction

Across healthcare, US minority populations including Latinos face disparities in risk for disease, clinical outcomes, and quality of care. The discourse related to disparity problems has often highlighted the need to increase the number of minority scientists so that the productivity of research focusing on minority populations is expanded. As a result, a group of Latino mental health researchers collaborated to develop a national network of senior mentors that participated in annual mentoring-oriented conferences. A cost-effective program was developed to stimulate the entry of new investigators into the field and provide mentoring with a focus on Latino mental health issues.

Method

A conference-platform was used as the career stimulation strategy. Annual conferences were held (2002 – 2006) that emphasized the showcasing of new investigators' work, pairing new investigators with senior researchers, and a feedback-oriented environment. Quantitative data were used to track new investigator career progress (i.e., publications, research grants), while qualitative data were used to assess all attendees' feedback, which was provided via conference feedback questionnaires.

Results

The feedback questionnaires revealed high levels of satisfaction with the conference, noting most highly the interactive, friendly, and nurturing conference format. Career tracking data indicated that nearly half of the new investigators participants published their work in peer-reviewed journals, and that about one-third of were successful in obtaining research funding.

Conclusions

These mentoring conferences appear to be an effective tool for stimulating the research careers of new investigators engaged in Latino mental health research.

Background

This protocol is for a study of a new program to improve outcomes in children suffering from chronic pain disorders, such as fibromyalgia, recurrent headache, or recurrent abdominal pain. Although teaching active pain self-management skills through cognitive-behavioral therapy (CBT) or a complementary program such as hypnotherapy or yoga has been shown to improve pain and functioning, children with low expectations of skill-building programs may lack motivation to comply with therapists' recommendations. This study will develop and test a new manualized peer-mentorship program which will provide modeling and reinforcement by peers to other adolescents with chronic pain (the mentored participants). The mentorship program will encourage mentored participants to engage in therapies that promote the learning of pain self-management skills and to support the mentored participants' practice of these skills. The study will examine the feasibility of this intervention for both mentors and mentored participants, and will assess the preliminary effectiveness of this program on mentored participants' pain and functional disability.

Methods

This protocol will recruit adolescents ages 12-17 with chronic pain and randomly assign them to either peer mentorship or a treatment-as-usual control group. Mentored participants will be matched with peer mentors of similar age (ages 14-18) who have actively participated in various treatment modalities through the UCLA Pediatric Pain Program and have learned to function successfully with a chronic pain disorder. The mentors will present information to mentored participants in a supervised and monitored telephone interaction for 2 months to encourage participation in skill-building programs. The control group will receive usual care but without the mentorship intervention. Mentored and control subjects' pain and functioning will be assessed at 2 months (end of intervention for mentored participants) and at 4 month follow-up to see if improvements persist. Measures of treatment adherence, pain, disability, and anxiety and depression will be assessed throughout study participation. Qualitative interviews for mentors, mentored participants, and control subjects will also be administered.

Trial registration

ClinicalTrials.gov NCT01118988.

Purpose: Until now, mentoring has hardly been used by the medical profession in German-speaking countries as a means of supporting junior physicians in their careers. The aim of the mentoring project described here was to obtain information for promoting and developing future mentoring programs at a university hospital.

Method: A new integrated mentoring model was developed and implemented over a 12-month period. Peer groups were advised on the mentoring process by mentors and program managers. A total of eight mentoring groups (40 peers) from four departments of a university hospital took part in the project: four voluntarily, and four on a compulsory basis. The evaluation was carried out using qualitative methods for analysis of the group protocols and the focus group interviews with the participants.

Results: Group discussions revealed that individual mentees, young female physicians in particular, developed concrete career plans and initiated further career-relevant steps. Some mentees - again more women than men - were promoted to senior physician posts. Further measurable career steps were increased research and publishing activity, and research fellowships abroad. The group process developed in five typical phases (forming, storming, norming, performing, and finalizing), which differed according to whether the groups had been formed on a voluntary or compulsory basis. In the evaluation interviews, mentees emphasized the following as effective mentoring factors: Concrete definition of own career goals; exchange of experiences within the peer groups; support and motivation from the mentors; and fostering of the group process by the program managers.

Conclusion: Participation in mentoring programs has to be voluntary. Mentees are motivated, autonomous, goal-oriented and prepared to take action. Mentors serve as examples and advisers. They derive satisfaction from being held in high esteem, as well as from the advancement of their own careers. Program managers have experience in systems theory and group dynamics, structure the group processes, and evaluate the quality of the results. Hospital management should regard mentoring as a business strategy and a means of staff development and quality management, and provide the necessary resources. The mentoring program presented here is being extended to other departments of the hospital on the basis of the positive experiences it has offered.

Objective. To assess the impact of a graduate student mentoring program on student interest in research and postgraduate education and on graduate student confidence in mentoring.

Methods. Undergraduate and pharmacy students (mentees) and graduate students (mentors) were matched and participated in the study, which required them to engage in at least 2 discussions regarding research and careers. Mentees completed a pre- and post-assessment of their perceptions of research, postgraduate training plans, and perceptions about mentors. Mentors completed a pre- and post-assessment of their perceptions about themselves as mentors and their confidence in mentoring.

Results. Although there were no significant differences among the mentees’ perceptions of research or the mentors’ confidence in mentoring, qualitative analysis indicated that the mentees’ perceptions of research improved and that the mentors believed their mentoring skills improved.

Conclusions. Based on the results of the qualitative analysis, implementing a graduate student mentoring program may help improve students’ perceptions of research and graduate students’ confidence in mentoring, which could increase student interest in postgraduate education and prepare mentors for future leadership roles.

Background

Despite increasing recognition that mentoring is essential early in medical careers, little is known about the prevalence of mentoring programs for medical students. We conducted this study to survey all medical schools in Germany regarding the prevalence of mentoring programs for medical students as well as the characteristics, goals and effectiveness of these programs.

Methods

A definition of mentoring was established and program inclusion criteria were determined based on a review of the literature. The literature defined mentoring as a steady, long-lasting relationship designed to promote the mentee's overall development. We developed a questionnaire to assess key characteristics of mentoring programs: the advocated mentoring model, the number of participating mentees and mentors, funding and staff, and characteristics of mentees and mentors (e.g., level of training). In addition, the survey characterized the mentee-mentor relationship regarding the frequency of meetings, forms of communication, incentives for mentors, the mode of matching mentors and mentees, and results of program evaluations. Furthermore, participants were asked to characterize the aims of their programs. The questionnaire consisted of 34 questions total, in multiple-choice (17), numeric (7) and free-text (10) format. This questionnaire was sent to deans and medical education faculty in Germany between June and September 2009. For numeric answers, mean, median, and standard deviation were determined. For free-text items, responses were coded into categories using qualitative free text analysis.

Results

We received responses from all 36 medical schools in Germany. We found that 20 out of 36 medical schools in Germany offer 22 active mentoring programs with a median of 125 and a total of 5,843 medical students (6.9 - 7.4% of all German medical students) enrolled as mentees at the time of the survey. 14 out of 22 programs (63%) have been established within the last 2 years. Six programs (27%) offer mentoring in a one-on-one setting. 18 programs (82%) feature faculty physicians as mentors. Nine programs (41%) involve students as mentors in a peer-mentoring setting. The most commonly reported goals of the mentoring programs include: establishing the mentee's professional network (13 programs, 59%), enhancement of academic performance (11 programs, 50%) and counseling students in difficulties (10 programs, 45%).

Conclusions

Despite a clear upsurge of mentoring programs for German medical students over recent years, the overall availability of mentoring is still limited. The mentoring models and goals of the existing programs vary considerably. Outcome data from controlled studies are needed to compare the efficiency and effectiveness of different forms of mentoring for medical students.

Background

In academic medicine, women physicians lag behind their male counterparts in advancement and promotion to leadership positions. Lack of mentoring, among other factors, has been reported to contribute to this disparity. Peer mentoring has been reported as a successful alternative to the dyadic mentoring model for women interested in improving their academic productivity. We describe a facilitated peer mentoring program in our institution's department of medicine.

Methods

Nineteen women enrolled in the program were divided into 5 groups. Each group had an assigned facilitator. Members of the respective groups met together with their facilitators at regular intervals during the 12 months of the project. A pre- and post-program evaluation consisting of a 25-item self-assessment of academic skills, self-efficacy, and academic career satisfaction was administered to each participant.

Results

At the end of 12 months, a total of 9 manuscripts were submitted to peer-reviewed journals, 6 of which are in press or have been published, and another 2 of which have been invited to be revised and resubmitted. At the end of the program, participants reported an increase in their satisfaction with academic achievement (mean score increase, 2.32 to 3.63; P = 0.0001), improvement in skills necessary to effectively search the medical literature (mean score increase, 3.32 to 4.05; P = 0.0009), an improvement in their ability to write a comprehensive review article (mean score increase, 2.89 to 3.63; P = 0.0017), and an improvement in their ability to critically evaluate the medical literature (mean score increased from 3.11 to 3.89; P = 0.0008).

Conclusions

This facilitated peer mentoring program demonstrated a positive impact on the academic skills and manuscript writing for junior women faculty. This 1-year program required minimal institutional resources, and suggests a need for further study of this and other mentoring programs for women faculty.

Manuscripts have been subjected to the peer review process prior to publication for over 300 years. Currently, the peer review process is used by almost all scientific journals, and The International Journal of Sports Physical Therapy is no exception. Scholarly publication is the means by which new work is communicated and peer review is an important part of this process. Peer review is a vital part of the quality control mechanism that is used to determine what is published, and what is not. The purpose of this commentary is to provide a description of the peer review process, both generally, and as utilized by The International Journal of Sports Physical Therapy. It is the hope of the authors that this will assist those who submit scholarly works to understand the purpose of the peer review process, as well as to appreciate the length of time required for a manuscript to complete the process and move toward publication.

Background

Little is known about the characteristics of mentoring relationships formed between faculty and medical students. Individual mentoring relationships of clinical medical students at Munich Medical School were characterized quantitatively and qualitatively.

Methods

All students signing up for the mentoring program responded to a questionnaire on their expectations (n = 534). Mentees were asked to give feedback after each of their one-on-one meetings (n = 203). A detailed analysis of the overall mentoring process and its characteristics was performed. For qualitative text analysis, free-text items were analyzed and categorized by two investigators. Quantitative analysis was performed using descriptive statistics and Wilcoxon-test to assess differences in grades between students with and without mentors.

Results

High-performing students were significantly more likely to participate in the mentoring program (p<0.001). Topics primarily discussed include the mentee's personal goals (65.5%), career planning (59.6%), and experiences abroad (57.6%). Mentees mostly perceived their mentors as counselors (88.9%), providers of ideas (85.0%), and role models (73.3%). Mentees emphasized the positive impact of the mentoring relationship on career planning (77.2%) and research (75.0%).

Conclusions

Medical students with strong academic performance as defined by their grades are more likely to participate in formal mentoring programs. Mentoring relationships between faculty and medical students are perceived as a mutually satisfying and effective instrument for key issues in medical students’ professional development.

Practical implications

Mentoring relationships are a highly effective means of enhancing the bidirectional flow of information between faculty and medical students. A mentoring program can thus establish a feedback loop enabling the educational institution to swiftly identify and address issues of medical students.

Background

Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both.

Methodology and Principal Findings

Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with “no statistical expert” and “no checklist” as controls. The two interventions were crossed in a 2×2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6–24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: −0.3–+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3–6.7), showing a significant improvement in quality.

Conclusions and Significance

This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.

Objective To determine the effects of training on the quality of peer review.

Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group.

Setting and participants Reviewers at a general medical journal.

Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials.

Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts.

Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P < 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002).

Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.

Background

High predictive validity – that is, a strong association between the outcome of peer review (usually, reviewers' ratings) and the scientific quality of a manuscript submitted to a journal (measured as citations of the later published paper) – does not as a rule suffice to demonstrate the usefulness of peer review for the selection of manuscripts. To assess usefulness, it is important to include in addition the base rate (proportion of submissions that are fundamentally suitable for publication) and the selection rate (the proportion of submissions accepted).

Methodology/Principal Findings

Taking the example of the high-impact journal Angewandte Chemie International Edition (AC-IE), we present a general approach for determining the usefulness of peer reviews for the selection of manuscripts for publication. The results of our study show that peer review is useful: 78% of the submissions accepted by AC-IE are correctly accepted for publication when the editor's decision is based on one review, 69% of the submissions are correctly accepted for publication when the editor's decision is based on two reviews, and 65% of the submissions are correctly accepted for publication when the editor's decision is based on three reviews.

Conclusions/Significance

The paper points out through what changes in the selection rate, base rate or validity coefficient a higher success rate (utility) in the AC-IE selection process could be achieved.

One of the best opportunities that undergraduates have to learn to write like a scientist is to write a thesis after participating in faculty-mentored undergraduate research. But developing writing skills doesn't happen automatically, and there are significant challenges associated with offering writing courses and with individualized mentoring. We present a hybrid model in which students have the structural support of a course plus the personalized benefits of working one-on-one with faculty. To optimize these one-on-one interactions, the course uses BioTAP, the Biology Thesis Assessment Protocol, to structure engagement in scientific peer review. By assessing theses written by students who took this course and comparable students who did not, we found that our approach not only improved student writing but also helped faculty members across the department—not only those teaching the course—to work more effectively and efficiently with student writers. Students who enrolled in this course were more likely to earn highest honors than students who only worked one-on-one with faculty. Further, students in the course scored significantly better on all higher-order writing and critical-thinking skills assessed.

Background

The professional development of under-represented faculty may be enhanced by mentorship, but we understand very little about the mechanisms by which mentoring brings about change. Our study posed the research question, what are the mechanisms by which mentoring may support professional development in under-represented groups?

The study aims to: (i) to pilot a mentoring scheme for female academics; (ii) to compare various health-related and attitudinal measures in mentees at baseline, 6 months, and 1 year into the mentoring relationship and, (iii) to compare pre-mentoring expectations to outcomes at 6 months and 1 year follow-up for mentees and mentors.

Methods

Female academic mentees were matched 1:1 or 2:1 with more senior academic mentors. Online surveys were conducted to compare health-related and attitudinal measures and expectations of mentoring at baseline with outcomes at 6 months and 1 year using paired t-tests and McNemar's test for matched cohort data.

Results

N = 46 mentoring pairs, 44 (96%) mentees completed the pre-mentoring survey, 37 (80%) at 6 months and 30 (65%) at 1 year. Job-related well-being (anxiety-contentment), self-esteem and self-efficacy all improved significantly and work-family conflict diminished at 1 year. Highest expectations were career progression (39; 89%), increased confidence (38; 87%), development of networking skills (33; 75%), better time-management (29; 66%) and better work-life balance (28; 64%). For mentees, expectations at baseline were higher than perceived achievements at 6 months or 1 year follow-up.

For mentors (N = 39), 36 (92%) completed the pre-mentoring survey, 32 (82%) at 6 months and 28 (72%) at 1 year. Mentors' highest expectations were of satisfaction in seeing people progress (26; 69%), seeing junior staff develop and grow (19; 53%), helping solve problems (18; 50%), helping women advance their careers (18; 50%) and helping remove career obstacles (13; 36%). Overall, gains at 6 months and 1 year exceeded pre-mentoring expectations.

Conclusions

This uncontrolled pilot study suggests that mentoring can improve aspects of job-related well-being, self-esteem and self-efficacy over 6 months, with further improvements seen after 1 year for female academics. Work-family conflict can also diminish. Despite these gains, mentees' prior expectations were shown to be unrealistically high, but mentors' expectations were exceeded.

As a major employer of health information professionals, the VA faces significant recruitment and retention challenges. The authors evaluated mentoring as a retention tool through a review of existing literature and the retrospective review of a VA health information management mentoring program. The literature review showed a link between employer mentorship and employee retention, regardless of the nature and structure of the mentoring relationship. Most organizations support employees who are willing to serve as mentors through increased compensation, recognition, and other types of support. No literature was found that studied retention rates for more than three years after a mentoring experience. The review of the VA mentoring program showed increased retention in the three years following enrollment in the program, but the increase was not statistically significant. The review did not demonstrate improvement in retention over a seven-year period. The combined evaluation gives mixed findings for mentorship as a retention tool and demonstrates the need for more research on the topic.

Objective

The authors developed mentorship programs to train minority junior faculty and advanced graduate students in mental health services research.

Method

The programs target “mentees” in the Southwest United States and offer long-term mentoring, seminars, group supervision, seed funding for peer reviewed research proposals, peer interaction, and weeklong institutes that feature presentations and mentoring by recognized experts.

Results

Evaluations suggest that these programs have influenced participants’ career development. Most mentees have continued to evolve in their research careers, submitted research grant applications, and obtained postdoctoral fellowships, and/or have advanced in faculty positions. Some mentees have expressed an opinion that without the support network that these programs provided, they would have abandoned their academic careers.

Conclusions

Future training efforts should take into account a series of challenges and tensions that affect mentees’ careers and personal lives, including the emotional legacy of discrimination and historical trauma.

BACKGROUND

Effective mentorship is crucial to career development. Strategies to improve the availability of mentors include mentoring multiple mentees at once, compensating mentors, comentoring, and long-distance mentoring.

OBJECTIVE

To describe current trends in mentorship in general Internal Medicine (GIM).

METHODS

We conducted a national cross-sectional web-based survey of GIM mentors, GIM fellowship directors, and GIM National Institutes of Health K24 grant awardees to capture their experiences with mentoring, including compensation for mentorship, multiple mentees, comentorship, and long-distance mentorship. We compared experiences by mentorship funding status, faculty type, academic rank, and sex.

RESULTS

We collected data from 111 mentors (77% male, 54% full professors, and 68% clinician-investigators). Fifty-two (47%) received funding for mentorship. Mentors supervised a median (25th percentile, 75th percentile) of 5 (3, 8) mentees each, and would be willing to supervise a maximum of 6 (4, 10) mentees at once. Compared with mentors without funding, mentors with funding had more current mentees (mean of 8.3 vs 5.1, respectively; P<.001). Full professors had more current mentees than associate or assistant professors (8.0 vs 5.9 vs 2.4, respectively; P=.005). Ninety-four (85%) mentors had experience comentoring, and two-thirds of mentors had experience mentoring from a distance. Although most mentors found long-distance mentoring to be less demanding, most also said it is less effective for the mentee and is personally less fulfilling.

CONCLUSIONS

Mentors in GIM appear to be close to their mentorship capacity, and the majority lack funding for mentorship. Comentoring and long-distance mentoring are common.

Background

Editorial peer review is universally used but little studied. We examined the relationship between external reviewers' recommendations and the editorial outcome of manuscripts undergoing external peer-review at the Journal of General Internal Medicine (JGIM).

Methodology/Principal Findings

We examined reviewer recommendations and editors' decisions at JGIM between 2004 and 2008. For manuscripts undergoing peer review, we calculated chance-corrected agreement among reviewers on recommendations to reject versus accept or revise. Using mixed effects logistic regression models, we estimated intra-class correlation coefficients (ICC) at the reviewer and manuscript level. Finally, we examined the probability of rejection in relation to reviewer agreement and disagreement. The 2264 manuscripts sent for external review during the study period received 5881 reviews provided by 2916 reviewers; 28% of reviews recommended rejection. Chance corrected agreement (kappa statistic) on rejection among reviewers was 0.11 (p<.01). In mixed effects models adjusting for study year and manuscript type, the reviewer-level ICC was 0.23 (95% confidence interval [CI], 0.19–0.29) and the manuscript-level ICC was 0.17 (95% CI, 0.12–0.22). The editors' overall rejection rate was 48%: 88% when all reviewers for a manuscript agreed on rejection (7% of manuscripts) and 20% when all reviewers agreed that the manuscript should not be rejected (48% of manuscripts) (p<0.01).

Conclusions/Significance

Reviewers at JGIM agreed on recommendations to reject vs. accept/revise at levels barely beyond chance, yet editors placed considerable weight on reviewers' recommendations. Efforts are needed to improve the reliability of the peer-review process while helping editors understand the limitations of reviewers' recommendations.

Objective

This report describes a model for the development, process and tracking methods of an interdisciplinary peer mentoring program, Peer-mentored Research Development Meeting (PRDM). PRDM was initiated in 2004 by a group of postdoctoral scholars and junior faculty from the Schools of the Health Sciences at the University of Pittsburgh.

Method

Case report. From February of 2004 through February of 2006, PRDM's first 5 members tracked and documented their research activity (e.g., manuscripts, grants) every 4 months. The defining features of PRDM are: 1) adherence to a structured frequency and format for meetings, 2) systematic tracking and evaluation of research development activities, and 3) maintenance of ongoing relationships with senior mentors.

Results

During the 24-month data collection period, members were involved in 91 research development projects including grant applications, journal article manuscripts, book chapters and conference abstracts. Members' productivity increased during the 24-month period, as did the efficiency and focus of the completed projects.

Conclusions

Members increased the efficiency and focus of their research development activities during the study period. Structured peer-mentoring groups have potential to enhance research productivity among junior investigators in research intensive environments.