PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Science. Author manuscript; available in PMC Feb 12, 2011.
Published in final edited form as:
PMCID: PMC3038128
NIHMSID: NIHMS266823
INQUIRY LEARNING. Integrating Content Detail and Critical Reasoning by Peer Review
Ravi Iyengar,1* Maria A. Diverse-Pierluissi,1 Sherry L. Jenkins,1 Andrew M. Chan,2 Lakshmi A. Devi,1 Eric A. Sobie,1 Adrian T. Ting,3 and Daniel C. Weinstein1
1Department of Pharmacology and Systems Therapeutics, Mount Sinai School of Medicine, New York, NY 10029, USA.
2Department of Oncological Sciences, Mount Sinai School of Medicine, New York, NY 10029, USA.
3Immunology Institute, Mount Sinai School of Medicine, New York, NY 10029, USA.
*Author for correspondence. ravi.iyengar/at/mssm.edu
Abstract
Classroom lectures by experts in combination with journal clubs and Web-based discussion forums help graduate students develop critical reasoning skills.
Students working toward Ph.D.’s develop deep expertise in their areas of research through thesis work and interactions with advisers. Students must also develop broad knowledge in related areas to formulate research questions and to identify appropriate technologies in areas not encountered during their thesis research. Breadth of training will become increasingly critical for long-term success as biomedical research becomes ever more interdisciplinary and multidisciplinary. Graduate students also need to learn how to participate in and respond to peer review to become effective professionals. Learning the balance between breadth and depth and participating effectively in peer review are interrelated educational issues.
Breadth of training is obtained during the didactic part of the graduate program in advanced courses and journal clubs that use the primary literature. Typically, these formats use synchronous teaching-learning methods (1) that are valuable because they enable direct interactions and immediate feedback between the teachers and students. However, a time-limited session greatly reduces the possibility for students to critically evaluate and integrate information. Often the discussion is between the student and the teacher with limited, if any, sustained interactions between the students. In addition, there is little opportunity for the students to reflect on and respond to comments from the teacher or their peers. Technology-based approaches can be used to address these issues in inquiry learning (2). Web technologies such as threaded discussion forums are asynchronous formats that overcome the limitations of synchronous sessions. Because asynchronous discussions allow students to respond to a topic at any time, this feature encourages more thoughtful and in-depth responses (3, 4). Moreover, as the peer-review process for publications and grants uses a similar asynchronous format, we felt this format would be useful for providing students with peer-review experience. Teaching effective peer-review skills is challenging (5, 6) because of the many subtleties that must be considered (see table below, right). We felt that making evaluation of peer review an integral part of an advanced course could enable students to learn an essential professional skill.
Table 1
Table 1
Effective Peer-Review Skills
Assessing students for breadth of knowledge and peer-review skills also poses challenges. Typical exams in advanced courses ask students to evaluate and design experiments. These exams, however, do not assess key aspects of graduate training, including the student’s ability to apply ideas from one field to another, to choose among multiple technologies to answer a specific question, to comment effectively on a peer’s approach, or to respond to criticisms of one’s own thinking. These deficiencies arise for two reasons: (i) Lack of feedback and responses: Once the exam is handed in, the teacher grades the exams. However, the student cannot respond to the teacher’s comments, and the teacher does not evaluate the student for her or his ability to understand and respond to constructive criticisms. (ii) Lack of peer interactions: In the typical exam format, each student deals individually with the teacher and has no knowledge of how his or her peers are thinking, much less the opportunity to comment on and respond to answers by peers. Yet scientists learn from peer interactions and are evaluated by their peers throughout their research careers.
We organize and teach an advanced course on cell signaling systems for second-year graduate students. The course, which began in 1988, originally focused on cell surface signaling molecules (heterotrimeric guanine nucleotide–binding proteins or G proteins). Over the years, as our knowledge of signaling pathways and biological processes has grown, the course has increasingly reflected facets of neuroscience, microbiology, immunology, pharmacology, and cell and developmental biology. This breadth has presented us with challenges and opportunities. Specifically, the breadth of student interests is valuable in peer interactions, because the comments made by the other students can provide useful perspectives that complement and enhance those of the instructors.
Lectures, which cover the major signaling pathways in mammalian cells, are taught by researchers with expertise in the area and are developed from primary literature (7). Four discussion forums, in which journal articles are discussed in depth, are interspersed among the lectures. A persistent issue has been evaluating students’ understanding of the journal club articles. In 2005, we introduced a format that used classroom presentation and discussion followed by written student responses to questions posted by the lecturer (8). Although this format evaluated how each student had understood the papers, the lack of teacher-student feedback and peer interactions in the asynchronous mode precluded prolonged and potentially thoughtful discussions. To address this deficiency, for the final exam we used a Web-based forum that was designed to elicit peer interactions (9). In response to the teacher’s questions, each student selected a primary publication and described in 200 to 500 words how the experiments in the paper answered the question. The first set of answers was posted on the Web using Science’s Signal Transduction Knowledge Environment (STKE) Discussion Forum, identifying students by numbers to maintain anonymity. Each student then wrote brief (less than 250 words), anonymous commentaries on two other students’answers. The original answer and the two commentaries were evaluated to determine each student’s final grade. Although the peer commentary feature worked well, this format did not allow students to reply to the critiques and to modify their responses accordingly.
When the course was offered in 2007 (10), we integrated the journal clubs, discussion forums, and exams. Each of the four journal clubs was led by two faculty members. Recent primary publications relevant to lectures in the previous section were selected by the faculty and posted on the Web. Students were required to answer several questions related to the papers before the discussion forum. During the classroom session, the teachers called on the students to explain the figures and tables in the papers. This was followed by an open discussion of the papers and the posted questions. The teachers then provided individual written critiques of students’written answers, and students revised their answers to respond to the critiques. The revised answers were posted on the Web with anonymity maintained. The students then posted brief comments on two of their peers’ answers using their student number as an identifier. For each of the four discussion forums each student received grades for the original answer, the revised answer, and the peer critiques (10).
The technology needed for this integrative project is relatively modest. In 2005, we had used the discussion forum in STKE. In 2007 we used the WebCT software platform from Blackboard Learning System (11). This program allows us to post the course contents, to run the discussion forums, and to set up Web links.
Web technologies were useful in assessing the depth and breadth of student knowledge. Because answers were relatively brief, students cited the appropriate literature to justify assumptions and specific experimental approaches. With hyperlinks to the cited references, the teachers could easily evaluate whether the students had understood the papers and used them appropriately. Addition and justification of referencing for the revised version of the answers became a useful tool to assess both integration of breadth and depth and ability to respond to peer review.
We had several major objectives from the student learning perspective: (i) to convey a historical perspective of the key experiments that established canonical signal transduction mechanisms, (ii) to engage the students substantively in a structured dialogue regarding current ideas and experiments, (iii) to enable the students to develop novel approaches to help integrate information, and (iv) to teach and evaluate participation in peer review. The use of team teaching, although challenging from a scheduling perspective, provides the students with a broad, yet personal, perspective of the development of research on the various signaling pathways. Integrating the lectures with classroom-based journal clubs and Web-based discussion forums has allowed us to achieve these goals. An unexpected benefit of the question-and-answer format was making the students aware of Web-based databases and bioinformatics tools useful for organizing information and designing experiments, resources often used by researchers.
A valuable outcome of this integrative project was the ability to document and assess how students integrated breadth of knowledge with depth of reasoning. This was achieved by comparing the original and revised answers, as well as the references used in both answers. The commentaries on answers from their peers were also very useful in assessing the student’s integrated learning capability. All of the faculty observed a difference between the classroom discussion, where students mostly did not challenge each others’ comments, and the written Web postings where students were respectful but often quite critical of answers from their peers. Anonymity of the Web format and the time provided to think about the Web postings appear to contribute to this critical feedback between peers. We had a nearly even distribution of male and female students in a class of 25, but the female students on average made fewer unsolicited comments in class (P = 0.03; see graph, left). We observed only a weak correlation between the number of unsolicited comments made by a student in classroom discussion and the student’s grade on the assignment (R2 = 0.15; see SOM). These data gathered over the four discussion forums support our original concern that some of the top students do not speak up in class. The gender divide in voluntary journal club participation in an advanced course is disconcerting and merits further study.
We also examined the students’ evaluations of the course, assessed using the school’s standard survey instrument. We compared the overall course rating, exam format, and effectiveness of readings for this course to ratings for two other courses with similar formats except for the lack of asynchronous interactions. The cell signaling course ranked better in all three categories [table S1 (16)]. This integration project has appeared to work well both from the teachers’ and students’ perspectives and has struck a practical balance between student-teacher and student-student interactions. In the future, we feel a comparison between instructor critiques and peer critiques would be useful.
Figure 1
Figure 1
Distribution of the number of unsolicited comments made in classroom discussion forums by male and female students.
Supplementary Material
Supporting Online Material
1. Johnson GM, Buck GH. paper presented at the Annual Conference of the American Educational Research Association; 9 to 13 April 2007; Chicago, IL. http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/28/08/16.pdf.
2. Edelson DC, Gordon DN, Pea RD. J. Learning Sci. 1999;8:391.
3. Branon R, Essex C. TechTrends. 2001;45:36.
4. Teikmanis M, Armstrong J. Comput Nurs. 2001;19:75. [PubMed]
5. Lightfoot JT. Adv. Physiol Educ. 1998;19:S57.
6. Guilford WH. Adv. Physiol Educ. 2001;25:167. [PubMed]
7. Iyengar R, Diverse-Pierluissi M, Weinstein D, Devi LA. Sci. STKE. 2005;2005:tr3. [PubMed]
8. Weinstein DC. Sci. STKE. 2005;2005:tr24. [PubMed]
9. Iyengar R, Diverse-Pierluissi M, Weinstein D, Devi L. Principles of Cell Signaling and Biological Consequences: Final Forum, Sci. STKE (Forum as seen February 2008) http://stke.sciencemag.org/cgi/forum-display/short/stke_el;357.
10. Supporting online materials include the syllabus for the Spring 2007 course, the instructions for the journal club Web-discussion forum, statistical analysis, and survey results. Examples of student answers, teacher critiques; and student comments on the Web can be found at (12).
11. Blackboard and WebCT. www.blackboard.com/webct.
12. Jenkins SL, et al. Sci. Signal. 2008;1:tr2. [PubMed]
13. The development of this integration project is part of the educational program supported by the Predoctoral Training Grant T32GM062754 from the National Institute of General Medical Sciences, NIH. We thank N. Gough for her encouragement and support over the last 2 years in developing the lectures for the Web and the use of the STKE discussion forum in 2005. We also thank M. Schindler for help with use of the WebCT program and T. Krulwich for her comments on the manuscript.