|Home | About | Journals | Submit | Contact Us | Français|
We used computational linguistic and content analyses to explore the concept of project ownership for undergraduate research. We used linguistic analysis of student interview data to develop a quantitative methodology for assessing project ownership and applied this method to measure degrees of project ownership expressed by students in relation to different types of educational research experiences. The results of the study suggest that the design of a research experience significantly influences the degree of project ownership expressed by students when they describe those experiences. The analysis identified both positive and negative aspects of project ownership and provided a working definition for how a student experiences his or her research opportunity. These elements suggest several features that could be incorporated into an undergraduate research experience to foster a student's sense of project ownership.
Undergraduate research experiences are an important factor in attracting students to the sciences and helping them build a foundation for a scientific career (Nagda et al., 1998 ; Seymour et al., 2004 ; Villarejo et al., 2008 ). Undergraduate research experiences can offer students an opportunity to personalize and own science, ultimately increasing the chances of scientific careers for such students. A study of the learning cycles of Nobel laureates identified the progression of events for several illustrious scientific careers that began with “nurturing” a sense of ownership through a passionate commitment to scientific inquiry (Stewart, 1994 ). In a discussion of factors that contribute to effective scientific inquiry experiences, Graham Hatfull, the lead scientist of a bacteriophage isolation and genomic description educational program, stated that “Isolating phages from environmental samples proves encouraging and exciting for students, giving them a sense of accomplishment and a feeling of ownership of their new virus. These feelings help to motivate them to continue analyzing their new phage” (Hatfull, 2010 , p. 244). This statement suggests that research experiences are especially motivating and encouraging when students gain a “feeling of ownership” over their research projects.
Despite recognition of the relationship between positive research experiences and project ownership, no studies explore this connection for undergraduate science, technology, engineering, and mathematics (STEM) research. We undertook this question with the goal of addressing four objectives. First, our study defines and offers theoretical clarification of the concept of project ownership and its relevance to science education. Second, we describe a quantitative methodology for measuring project ownership by using interview data. Third, we used linguistic analysis to characterize the elements of project ownership expressed in relation to different types of educational research experiences. Finally, we offer suggestions based on the linguistic analysis for designing science education programs that promote project ownership.
As a theoretical construct, the concept of project ownership integrates several auxiliary terms related to the concepts of choice, control, personal responsibility, self-determination, commitment, engagement, emotional involvement, and identification (Kennedy, 1994 ; Chung et al., 1998 ; Downie and Moore, 1998 ; Mason et al., 2004 ; Nail, 2007 ; Wiley, 2009 ). To explicate the concept of ownership of learning from these adjacent terms, Wiley has proposed three conceptual definitions (Wiley, 2009, pp. 25–33 ):
1. Ownership as responsibility relates to the student's role in his or her educational activity. Students have ownership over their learning process when they take personal responsibility for the choice of study and the outcome of their work.
2. Ownership as buy-in refers to the commitment, engagement and loyalty that a student has towards his or her educational program or project. Students expressing ownership have “bought into” the importance of the educational project.
3. Ownership as self-identification refers to a sense of belonging and identification with the educational program or project. A student with ownership has a deep personal connection and sees the project as an extension of him/herself.
Combining Wiley's ideas leads to a more nuanced definition of ownership. We assume that development of project ownership is not just an individual student characteristic, but one that results from a more complex interaction between the student and the educational environment. Accordingly, features of an educational program may increase or decrease the degree of project ownership that a student experiences. Milner-Bolotin defines student educational ownership along a spectrum and notes that student-centered educational approaches should elicit much higher levels of ownership (Milner-Bolotin, 2001 ). Importantly, some evidence suggests that engagement, one aspect of project ownership, is a factor that retains students in the sciences and reduces the number of declared science majors who leave to study other fields after taking introductory science courses (Dickinson and Flick, 1998 ; Hake, 1998 ; National Research Council, 2003 ; Cech and Kennedy, 2005 ; Jones et al., 2010 ; Thiry et al., 2011 ).
Despite agreement that research experiences add value to a science education and can help promote student long-term interest in science, these experiences vary widely. Laboratory training ranges from rote experiments with known outcomes, amounting to little more than demonstration exercises, to a research experience in a faculty member's laboratory. Even capstone experiences vary significantly in the extent of autonomy and intellectual control provided to the undergraduate student (Wenning, 2005 , 2007 ; Hanauer et al., 2009 ). Characteristics of the educational research experience, such as control over decision making, personal agency, and amount of responsibility, are likely to be reflected in the degree to which a student experiences and develops project ownership. This is the connection that we set out to explore.
One approach to evaluating text is through content analysis (Tausczik and Pennebaker, 2010 ), a qualitative research method in which verbal data are assigned to categories relevant to the research topic (Stemler, 2001 ). Linguistics researchers develop reliable content categories via established methods and use the content categories to code a collection of texts. The percentage of statements categorized this way can yield insights about the text, such as common features of student descriptions of their undergraduate research experiences. We used this approach to develop a definition of ownership as expressed by undergraduate students engaged in a research project.
We also wanted to measure degrees of project ownership by using a quantitative approach. This involved addressing two methodological problems. The first issue involved finding suitable comparative research experiences that differ in relation to choice, control, agency, and responsibility. The second problem concerned finding a way to measure a student's sense of project ownership. Project ownership is a complex term that integrates attitudes and emotions related to a student's personal connection to an educational experience. It involves a response to an educational environment and is not a disembodied feature of the student. We anticipated that project ownership would be evident in a student's description or discussion of his or her scientific project. In other words, students who feel a sense of ownership will reveal a sense of personal connection through the attitudinal and emotive language they use to discuss their work. This expression provides a measure of how ownership is developed and conceptualized by students.
Recent developments in the field of computational linguistics offer powerful tools for analyzing speech and texts. Computational linguistics—the analysis of language using computer-based methodologies—allows analysts to go beyond intuitive, comprehension-based approaches to text analysis, revealing systematic language use patterns that reflect a range of attitudinal and emotive stances (Hunston, 2002 ). The Linguistic Inquiry and Word Count (LIWC) program has been used extensively to analyze issues of attitude, stance, and emotive responses (Hunston, 2002 ; Alpers et al., 2005 ; Kahn et al., 2007 ; Tausczik and Pennebaker, 2010 ). The LIWC program calculates frequencies for linguistic categories (such as pronouns or verbs) and semantic categories (such as emotive words, words designating social processes, or cognitive words) in a given collection of texts. As a basic principle, researchers who use word counts and content analysis hypothesize that what someone is thinking and feeling is reflected in the language he or she uses. The LIWC program offers a way of systematically assessing that information.
Several different categories of information are relevant to project ownership. If a student is an active agent, in control and personally connected to a project, we expected that this sense of ownership would be reflected in increased use of first-person pronouns (I, me, my, and mine). Additional features of project ownership include emotional engagement, identification, and commitment. In linguistic terms, these features would be reflected in the semantic category of words that denote an emotional response. Students who are emotionally engaged with a program would be expected to express a heightened sense of emotion through increased use of emotive words. But in contrast with the emotional lexicon, when students discuss the content of scientific inquiry experiences, they would express cognitive processes and insights related to their projects. We would not expect language in cognitive categories to fluctuate significantly with the degree of ownership.
Our experimental approach incorporated content analysis and analytical techniques for language processing (the LIWC program) to define and measure ownership as expressed in student interviews. We identified elements of project ownership by using content analysis and employed computational linguistic analysis to measure the degree of project ownership for groups of students in different science educational experiences. For the question dealing with differing degrees of project ownership, we compared student descriptions of different types of research experiences. A comparison between the content analysis and the linguistic analysis provides a means for assessing the compatibility of the linguistics findings with content statements concerning degrees of project ownership.
Three groups of students were identified based on their self-selected participation in different types of undergraduate laboratory experiences. We were interested in comparing a research experience with ownership as an explicit design feature to the more variable category of undergraduate research experiences in faculty labs around campus. As a contrast to the two groups of students participating in a capstone undergraduate experience, we also interviewed a third group of students enrolled in a standard undergraduate laboratory course. The three groups were identified as follows:
Group A students enrolled in the course, but were not selected for enrollment based upon previous research experience. Students in group B independently identified a lab with a research subject of interest to them and experienced a variable level of mentoring and autonomy available in each of the individual labs. Group C involved students enrolled in one of two laboratory courses, Organic Chemistry Laboratory and Biochemistry Laboratory. Students interviewed for this study were sent targeted invitations inviting their participation in a brief interview protocol. Students in each of the different groups studied at Yale University; were drawn from a mixture of college sophomores, juniors, and seniors; and were mainly science or premed majors. Gender and ethnic balance in the groups was approximately representative of the overall university population, with roughly equal numbers of male and female students and two or fewer African-American or Hispanic participants in each group. A total of 49 students participated in this study: 17 in group A, 16 in group B, and 16 in group C.
All students were interviewed by the same person, who used identical questions designed to invoke a general description of each student's laboratory experience. The interview protocol consisted of the following questions:
All answers were recorded, transcribed, and subjected to computational linguistic and content analyses.
Two types of data analysis were conducted on the transcribed interviews, content analysis and computational linguistic analysis. The content analysis was conducted in three stages, following accepted guidelines (Neuendorf, 2002 ; Krippendorf, 2004 ). The initial stage consisted of constructing a coding system. Two applied linguistics researchers read the complete data set of interviews and proposed an initial set of codes for the interview data. After establishing agreement between researchers using standard practices, a 14-category coding system was defined. In the second stage of coding system development, reliability was established according to accepted guidelines (Cohen's kappa statistic, a measure of agreement among raters, was K = 0.61; p < 0.01; Landis and Koch, 1977 ). In the final stage of the content analysis, the entire interview data set was analyzed with this coding system, and the percentages of codes by group were calculated. The quantitative computational analysis was conducted using the LIWC program (Pennebaker et al., 2007 ). Frequencies of first-person pronouns, emotive words, cognitive lexicon, and insight words were counted and analyzed using appropriate statistical methods. Details of the methods and analysis are provided in the Supplemental Material.
Content analysis of the interview data yielded a coding system that defines elements of project ownership. Interestingly, eight of the categories are positive indicators of project ownership (categories 1–8 below) and four are negative indicators (categories 9–12 below). The last two categories listed are neutral and pertain to question clarification or details of signing up for a research program (categories 13 and 14 below). This analysis provides an extensive definition of the elements of project ownership. The coding system categories are as follows:
Table 1 summarizes the percentage of statements made by each research experience group for the 14 content analysis categories. All the statements made by participants were coded and categorized according to the coding system presented. The percentages were calculated in relation to the total number of statements made by each group.
Consistent with the various levels of autonomy and project development that were experienced by the three groups, the distribution of coding categories reveals some interesting differences in the content of their language usage. Students in group A had a higher percentage of statements in seven of the eight positive project ownership categories compared with both groups B and C. These statements suggest that they had a strong sense of project ownership, consistent with the formulation of the course. In contrast, students in group C exhibited higher frequencies of the four negative ownership indicators compared with both groups A and B (categories 9–12). Notably, group C students made no statements describing their experiences as involving agency (category 2) or personal connections (category 1). They also lacked statements about overcoming challenges or gaining unexpected understanding (categories 4 and 8), two critical elements of successful research.
Group B was generally situated between the category usage of group A and group C. These students expressed some elements of project ownership (such as agency) but not all (such as personal connection and emotional engagement). Interestingly, group B had the highest number of statements that deal with frustration with the scientific process (category 6). They made more than twice the number of statements concerning frustration than either of the other groups. This suggests that these students were conducting scientific research but may have lacked sufficient support from mentors or a community of peers.
When describing their research experiences, group C students made twice as many statements describing the procedures of science (category 12) than members of group A. Conversely, group A made twice as many statements concerning the real-world significance of their scientific research (category 7) than group C. It seems that for group A, there was far greater understanding of the broader social value of their research, while group C students were more focused on the procedures. Furthermore, group A was the only group to make statements addressing the unexpected nature of science (category 8). This suggests that a student who has ownership over his or her research also faces, and ultimately understands, the uncertainty and unpredictability that characterizes scientific inquiry. Overall, the content analysis of the interviews of the three groups suggests a different understanding of their research experiences.
We also explored the degree of ownership expressed by the students, using a usage frequency of emotional and cognitive lexicon. The computational linguistic analysis of the four linguistic categories (first-person pronouns, emotional words, cognitive lexicon, and insight words) for the three scientific inquiry groups is summarized in Table 2. Group A had a higher usage frequency of first-person pronouns and emotional words. The frequency of cognitive lexicon and insight usage were very similar for all groups. Statistical analysis supported the differences between these groups.
Tests of the homogeneity of variances were calculated in order to verify the appropriateness of the statistical procedure. This was necessitated by the small numbers of participants and the slight inequality of student numbers in the groups and as a precursor to the statistical analysis of the descriptive data. A Box's M test and Levene's test of the equality of variances were conducted. Both returned p values above 0.05, thus allowing the assumption of normality. For evaluation of the above analysis of the descriptive data, a one-way multivariate analysis of variance (MANOVA) was calculated using the student group as the independent variable and First-person Pronouns, Emotional Words, Cognitive Lexicon, and Insight Words as four dependent variables. Hotellings’ T2 multivariate generalization of the univariate t value was used. The MANOVA revealed a highly significant effect for the group variable (Hotellings’ T2 = 0.80, p < 0.01). Because a significant multivariate effect was found, univariate F-tests were calculated to determine which variables contributed to the overall difference. Significant differences were found for First-person Pronoun usage (F(2, 46) = 10.33, p < 0.01) and Emotional Words (F(2, 46) = 3.62, p < 0.035). The other two variables, Cognitive Lexicon and Insight Words, were not found to be significantly different.
To further understand the source of difference in the variable of Personal Pronouns and Emotional Words, we conducted post hoc comparisons of cell means, using the Scheffe method. Group A (rainforest group) was found to be significantly different than group C (standard lab group) for both Personal Pronoun usage and Emotional Words. No significant differences between groups A and B (independent research) were found on this test. However, on a less conservative, least significant differences post hoc test, group A was also found to be significantly different than group B on the variable of Personal Pronoun usage. This analysis also confirmed the significant differences between group A and C for both the Personal Pronoun and Emotional Word variables.
The statistical results support the conclusion that the three groups are significantly different based on the frequency of Personal Pronouns and Emotional Words. Group A students used first-person pronouns and emotional words with higher frequencies than did group C students. However, differences were also found between groups A and B for Personal Pronoun usage. These results suggest that there are differences in the levels of project ownership expressed in interview data concerning different types of research experiences. Higher levels of personal choice and agency in the research project design were reflected by increased first-person pronoun and emotional word use. This further demonstrates that the degree of project ownership felt by a student is reflected in the language the student uses to express that experience.
The results of this study suggest that features of an educational research experience do result in measurable differences in the way students describe these experiences. Significant differences were observed in both the content analysis and computational linguistic data. The content analysis has defined a set of elements that positively and negatively influence the degree of project ownership experienced by a student. Research experiences in which students felt agency, choice, and control elicited statements of excitement about doing scientific research, whereas experiences lacking these elements elicited statements of indifference toward science. The evidence from this study suggests that students develop and describe a sense of project ownership when research experiences are appropriately designed.
A second important finding is that there are measurable degrees of project ownership and these are systematically aligned to the types of education research students experience. Rather than considering project ownership a dichotomous variable that a student does or does not have, the data suggest that there are degrees of expressed ownership. Group B revealed agency in the content analysis data but did not express the degree of emotional commitment or personal connection of the group A participants. Our results emphasize that undergraduate research in general is better than a standard laboratory course for instilling project ownership and demonstrate that undergraduate research experiences are highly variable in design, incorporation of mentoring, and whether students are included within a scientific community.
An underlying question in this study concerned the validity of using computational linguistic data concerning first-person pronouns and emotional words as a way of measuring project ownership. In addressing this question, the same interview data were analyzed using two different linguistic analytical techniques: computational linguistic frequency counts and content analysis. The data presented in this study reveal a close correspondence between the frequency counts designating project ownership and the informational categories designating project ownership in relation to different research experiences. Group A expressed the highest levels of first-person pronouns and emotional words, and also had the highest levels of those categories relevant to project ownership (agency, personal connection, excitement, personal achievement, and a sense of scientific significance). This correspondence between the results suggests that our use of computational linguistic analysis was a valid measure of project ownership.
The interviews for this study were conducted in 2008. Enough time has passed that it is possible to ask whether the degree of ownership showed any correlation with retention in the sciences or persistence in a scientific career. We tracked student decisions for groups A and B, in which students were doing research as a capstone undergraduate experience. Of the group A students, 88% are now following a career and studies in science or medicine (18% to an MD program, 53% to a PhD or MD/PhD program, 18% to a master's program or lab work, 12% not doing science). Three-quarters (76%) continued to participate in research during their undergraduate education and after graduation. For group B students who reported future plans, 63% either continued to graduate school in the sciences or went to a research-intensive medical school with a research/independent study requirement (25% to an MD program, 38% to a PhD or MD/PhD program, 6% to science consulting, and 31% indicated other/not doing science). The group of students with a higher level of project ownership also showed a higher level of persistence in science. Although these data derive from a small cohort of students, they hint that project ownership in an undergraduate research experience could encourage students toward longer-term career goals in the sciences. We did not track group C students because the courses we drew from were not capstone experiences like those of groups A and B.
Our results suggest that the type of educational program influences the way students describe their experiences and that intentionally designing educational programs to inculcate ownership results in students expressing increased levels of project ownership. We recognize that using three groups of self-selected students introduces the possibility that a participant's desire for project ownership motivated the group selection. We would expect a higher number of group A students to continue in science if their choice to participate in group A reflects a pre-existing interest. The inclusion of pre- and postdata on student education and career interests in future studies would address this limitation.
The content analysis identified both positive and negative expressions of project ownership that provide a working definition for how a student experiences a research opportunity. These elements suggest several features that could be incorporated into undergraduate research to foster student project ownership:
The components of project ownership defined by our linguistic analyses can be used to design undergraduate research programs in many settings with various levels of resources. Programs in which students experience strong project ownership make authentic science accessible to undergraduate students.
D.I.H. was funded through a subaward (IUP RI log no.: 0910-028) from a Howard Hughes Medical Institute (HHMI) professorship award to Graham Hatfull. D.I.H. and S.A.S were supported from a HHMI professorship award to S.A.S. J.F. was supported by an HHMI institutional award to Yale University and Dr. Robert Wyman.