1. Introduction
Undergraduate research experiences are considered a high-impact educational practice [1]. These experiences have been touted by the National Science Foundation [2] and the Association of American Colleges and Universities [3] as critical to engaging students in Science, Technology, Engineering, and Mathematics (STEM) and improving retention rates. The effects of undergraduate research on students’ scholastic experiences and future careers are well documented [1,2,4,5,6,7,8,9,10]. Students participating in research experiences gain skills and confidence that benefit their academic endeavors, including the ability to analyze data and an understanding (critical thinking) of how science is done [11,12,13]. More specifically, research opportunities provide STEM students with hands-on experiences that are important to: 1) understand how real-world science is conducted; 2) prepare them to critically think about science and encounters with science in their everyday lives; 3) develop into scientific “thinkers;” and 4) engage students in authentic science activities where they develop skills, such as designing experiments, interpreting results, and dealing with failure and problem-solving [2]. Therefore, research provides avenues to better prepare undergraduate students for graduate school and the workforce as “scientists.”
In addition, undergraduate researchers have also self-reported other gains, particularly in soft skills, such as cognitive, personal, and professional growth [14]. Research experiences encourage excitement about science and/or science careers [2] and provide students with opportunities to be exposed to alternative careers in science, rather than just the medical professions. Unsurprisingly, student researchers also demonstrate increased interest in research careers in science [11,12,15], especially as measured by matriculation in graduate school [16,17]. In addition, several studies have shown that undergraduate research experience promotes critical thinking, communication skills (oral and written), time management skills, and organizational skills, which are translatable to careers both within and outside STEM [1,13,18,19,20].
Although many studies have demonstrated gains by research students compared to their peers, fewer have addressed higher-order thinking skills, such as the development of a complex epistemological understanding of science or the ability to manage the ambiguity and uncertainty inherent to the practice of science [21,22,23]. Therefore, the purpose of this study was to determine if there were differences in academic performance between students engaged in research and those who chose not to engage in research experiences. We hypothesized that an undergraduate research experience should improve students’ writing.
In this study, freshman biology majors from two similarly sized schools offering comparable research programs were encouraged to participate in a three-semester research program that would prepare them for independent research later in their college careers. Applications for the program were collected and processed during the fall semester of the freshman year, but the research courses did not begin until the spring semester. As a result, subjects had identical experiences during the fall semester of the freshman year regardless of their future connections to the research program. This study compared research students’ performance in these classes to their classmates’ who were not interested in pursuing research experiences. In addition to analyzing the grades of students who participated in the research program, this study also addressed how well students who simply applied to the program fared compared to their classmates. That is, this study analyzed the academic performance of students who conducted research compared to their peers, as well as the performance of students who were interested in research compared to other students. Therefore, we asked: Who in science classes are benefitting from the opportunity to develop the mentor-mentee connections and the interest in careers in science that so often accompany research? Are students of all talent levels anxious to get into the lab or field, and is the research experience transforming them into better students? Or, alternatively, are research opportunities seized only by students who are already performing better in the classroom than their peers? In short, is students’ education enhanced by research experiences? This study also analyzed the academic performances of students who conducted research compared to their peers. We asked: How well did they perform compared to their classmates who would not do research? Who in the science classes is benefitting from the opportunity to develop the mentor-mentee connections and the interest in careers in science that so often accompany research? Are students of all talent levels anxious to get into the lab or field, and is the research experience transforming them into better students? 2. Materials and Methods 2.1. Design This study was conducted at two small liberal arts colleges in the Northwestern United States. Fall-semester grades were earned in the introductory class of the biology major sequence (BIO111 and BIO151: General Biology I) and in the only non-science class required for all incoming freshmen (ENG 119 and ENG 117: First-Year Writing). The application process for inclusion in the research program was described several times during the General Biology I class, either by the lecturing professor, also known as the professor of record, or by other students who were already conducting biology research on campus. These descriptions of the research program highlighted how the research experience often helps job placement or acceptance to graduate programs after college. Further, freshmen were informed of the time commitments required by participation in the program both during the three-semester sequence and as upper-division researchers. However, students were assured that applying was not a firm commitment to continue participation throughout their collegiate careers. However, students who started research continued through graduation. At one of these two schools, there was no application process for the research program, and all interested students were accepted. For the other school, applications were available online and were due on October 1 of the freshman year. Applicants were encouraged to fill out a simple cover sheet and compose three essays that outlined their career goals and their desires to become involved in research. More importantly, prior to the application deadline, students were informed repeatedly that the primary criterion for selection would not be grades or GPA; this was also obvious considering that no grades would have been earned by the application deadline. Students were informed that selection would instead be based on motivation to conduct research and on demonstrated interest in the laboratory portion of their biology course. Students were selected by a committee composed of professors who taught the General Biology I lecture, General Biology I lab, First-Year Writing, and General Chemistry I lecture, as well as a biology professor who had no first-hand knowledge of the applicants’ performances in the classroom or the lab. While performance in the lecture portion of the course was discussed during this committee meeting, the conversation focused on students’ applications and their performances in the lab, with special attention given to inquisitiveness and behavior in peer groups. Students who were selected by the committee were informed of the decision prior to spring registration, which began in late October. Students who participated in the research class in the spring received no special instruction or feedback on their coursework during the fall semester. 2.2. Instruments for Collecting Information: Writing Analysis
This study used two technologies to investigate and visualize the conceptual content of student writing samples, Leximancer and Gephi. In concept analysis, documents are scanned repeatedly in a boot-strapping process that automatically builds a thesaurus and designates words and phrases that meet certain criteria as concepts. For example, in analyzing a collection of recipes, Leximancer could associate the phrase “lightly fried batter served at breakfast with butter and syrup” with the concept “pancake.” Leximancer’s most distinctive feature is that it performs this sort of analysis automatically using Bayesian statistical methods. This approach, in addition to being unbiased, relieves users of the tedious and time-consuming task of formulating and applying their own coding concept schemes. For additional information on Leximancer and content analysis, see [24]. This study used 1) Leximancer to identify emergent concepts and themes and to provide direct access to the context blocks from which they arose; and 2) Gephi to further characterize relationships between concepts [25].
2.3. Statistical Data Analysis Students in the study had no research requirements for the completion of their degrees. At one university, 14 students applied to participate in the research program, and nine were admitted. At least 16 students did not apply to the research program. That is, references to “research” versus “non-research” students involved nine and 21 students, respectively, while references to “applicant” and “non-applicants” included 14 and 16 students, respectively. From the other university, 23 students were involved with the research and 18 were not involved in the research. To ensure that the student sample included only long-term biology majors and not those who either dropped out of school or who switched majors as freshmen, only students who registered for the third course in the biology major sequence the following year were included in this study. Research students took the biology and English classes analyzed in this study alongside their peers. The two biology courses were taught by different instructors. Analysis of the spring semester included grades from the second semester of each of these yearlong sequences (BIO112 and BIO152: General Biology II and ENG120: Critical Reading). Writing samples for writing analysis were taken from ENG313: Writing for the Sciences. If a student had fulfilled a requirement for either of these classes by taking an equivalent class at another institution, the transferred grade was not included in the sample. All data were analyzed after final grades had been submitted. Grades were quantified by conversion to a standard four-point scale (i.e., A = 4.0, A- = 3.67, B+ = 3.33) for calculation of grade-point average (GPA). Statistical significance was measured via t-test (p value = 0.05). 3. Results 3.1. Grade Analysis
Research students earned significantly higher grades than their peers after the research program began in the spring semester (Figure 1a). In General Biology II, in the spring, research students at one institution earned an average GPA of 3.59, but their non-research counterparts (i.e., those who either applied and were not accepted and those who did not apply) earned a 2.69, or almost one letter grade lower (p < 0.01). The same trend was observed in the corresponding biology course at the second institution (3.41 versus 2.68; p < 0.001) (Figure 1b). This trend was also observed in their Critical Reading class in the English Department during the spring semester: research students earned a 3.71 GPA, but their peers earned only a 2.98 (p < 0.01) (Figure 1a), and the other University research students earned a 3.62 GPA, while the non-research peers earned 2.72 GPA (p < 0.001) (Figure 1b).
Students who were selected to participate in the research sequence also outperformed their peers during the fall semester (before research started), when neither group received additional instruction (Figure 1a). At one institution, soon-to-be research students earned a 3.59 in General Biology I during the fall, but students who would not become involved in the research program earned a 2.56, or about one point lower (p < 0.01). The same trend was observed at the second institution (3.16 versus 2.59 in Biology, p < 0.01) (Figure 2b). In First-Year Writing, those students who were accepted to the research program earned a 3.89 GPA, but those who were not accepted earned a 2.69 (p < 0.01); the other institution, 3.58 versus 2.86 in English (p<0.001) (Figure 2b). Overall, research students earned higher grades in both science and non-science courses, both during the program and before the program started.
3.2. Writing Sample Analysis
Students in both the research and non-research groups submitted reviews of research papers. The papers in both groups spanned a variety of scientific topics. Non-research students submitted eight summaries in the MS Word format averaging 15.4 kB in size. Research students submitted seven summaries in the MS Word format averaging 19 kB in size. A t-test was performed comparing the word counts in the two groups, designated A (non-research) and B (research), respectively, in the summary presented in Table 1. Assuming a non-directional null hypothesis, the group means were different (p = 0.01), with research students producing longer (in length) papers.
Further analysis of the writing samples was conducted using Leximancer (see Figure 3; Figure 4 for the emergent concepts and themes two ways). We combined all the files from both the non-research and research groups as if they came from a single sample (see Figure 3). This approach emphasizes the association between each file and individual concepts. Then, the files were grouped into two folders, Folder1_non and Folder1_res, representing the non-research and research samples (see Figure 4). This approach highlights the associations between each group and individual concepts. Concepts and files near to one another co-occur more frequently than concepts and files that are further apart (see Figure 3 and Figure 4). In other words, the proximity of concepts, files, and folders is strongly related to co-occurrence in Leximancer concept maps. A delimitation of this approach is that only the most likely co-occurrences are shown as edges. Less likely co-occurrences are not shown at all.
However, this creates a false impression that the pattern of emergent concepts and themes found in the writing samples is simple. That was not the case. An overview of that complexity is seen in Figure 5, created using Gephi. Instead, proximity was weakly associated with co-occurrence, but all co-occurrences were shown (see Figure 5).
It should be noted that each of these concept maps is, in effect, a projection onto a 2D surface of mathematical relationships that exist in a higher dimensional space. In the process of performing this projection, information is lost. So, while the figures may reveal main concepts, attempting to read too much into the details is not advised.
We summarized concept prominence and co-occurrents (see Table 2; Table 3). Prominence is a Bayesian statistical concept used to identify concepts that are characteristic of a category (i.e., sample set in this case). By inspection, it is clear that the writing samples from the two groups differed in the concepts most used by students to summarize their respective research papers.
The most prominent concept found in the non-research writing sample was report (see Table 4, Table 5, Table 6 and Table 7). This concept co-occurred with several others (analysis, down, field, medical, research, review, and scientific). The supporting text for these co-occurrences (and other co-occurrence contextual findings) suggested an approach to writing focused on the structure of the paper under review. By contrast, the most prominent concept found in the research writing sample was pain. The concept co-occurred with several others (better, effects, example, low, and proposal). The supporting text for these co-occurrences (and other co-occurrence contextual findings) suggested an approach to writing focused on the scientific content of the paper under review. This fundamental difference in focus reflected, we believe, a more scientific disposition on the part of research students.
4. Discussion Not surprisingly, students who were involved in an undergraduate research program earned significantly higher grades than their peers during the program. This result is consistent with the body of literature that describes how research experiences enrich scientific thinking as well as students’ attitudes toward science in general. However, students who were selected to participate in the undergraduate research program earned higher grades than their peers even before the research program started, or before they received additional instruction or feedback compared to their peers. This trend was observed in both science and non-science classes. Taken together, these data might merely indicate that the faculty members who selected students for the program were apt judges of academic talent. However, such faculty bias seemingly disappeared upon analysis of the grades earned by applicants compared to their peers, as well as data from the school that did not have an application process. In addition, students who wanted to participate in the research program earned higher grades than students who did not. This self-selection was remarkable, considering that the deadline for application was early in the semester, before students had received much feedback regarding their coursework. In addition, students were informed that the primary criteria for selection for the program would be the strengths of the applications; those with poorer scores early in the course were not explicitly discouraged from applying. Under these circumstances, where all students were equally encouraged to conduct research, why did only students with high grades demonstrate an interest in doing so?
Better students are more likely to participate in optional academic activities, often regardless of whether those activities are actually helpful. For example, better students are more likely to display academic behaviors that assumedly help them in their courses, such as attending class [26,27] and studying [27,28]. In addition, better students are also more likely to attend help sessions and complete extra credit. These two latter behaviors do not seem to help students significantly, at least in introductory courses: students perform no better on exams when they have attended a help session compared to when they have not [28], and students who complete extra-credit assignments have higher grades than their peers even when the points earned for extra credit are not considered when computing the grades [27]. The fact that better students are more diligent may also have a strong psychological root: when asked what grades they think they will earn in a course, D and F students are often grossly overconfident, while A students are surprisingly underconfident [27]. In short, better students are often motivated enough to work hard in their courses, even if they are already performing well or if the work doesn’t help them.
Like these other behaviors, demonstrated interest in undergraduate research appears to be an indicator of simple academic motivation. Similarly, students who show up to a help session for the first exam of the semester, before they have had any feedback or any indication that they need the help, are better students [28]. Students who are simply willing to spend time in the lab outside of class earn higher grades. Importantly, in the current study, the higher grades from the fall semester were not influenced by the decision to do research—the “rewards” that often accompany that decision were yet to come. Therefore, the results of this study suggest that undergraduate research is yet another way by which already motivated students distance themselves from their peers.
Of course, students who did not want to participate in the research program were not necessarily unmotivated. Access must be considered. For example, several academically talented student-athletes in the class in this study chose not to apply to the research program because of time commitments to their respective sports. Other extracurricular or off-campus responsibilities may have similarly precluded other good students from becoming involved in the research. Overall, however, the robust differences between the grades of applicants and non-applicants suggest that students who are willing and able to conduct undergraduate research are, on average, already significantly better students than their peers. The fact that the differences in grades seen in this study existed in such different courses as General Biology and English suggests that these perceived differences in aptitude were not due to varied personal interests in the material or disparate learning styles. Instead, students who are willing to do research appear to be simply better students, regardless of the subject. In this way, interest in undergraduate research, or at least the willingness to conduct it, appears to be one more way that motivated students get, or stay, ahead of their classmates. These motivated students’ subsequent exposure to an even richer learning environment in the lab or the field via research might only widen an existing academic gap to produce the disparate successes of research students and their peers that have been found in other studies.
The writing samples did reveal systematic differences in the concepts employed by non-research and research students to express themselves. Further insight into the origin of these differences is seen in Table 4, Table 5, Table 6 and Table 7, where the most frequently co-occurring pairs of concepts are shown in context. It is at this level of inspection that we found evidence of a difference in dispositions on the part of the non-research students and the research students. In short, the non-research students used prominent concepts to describe structural aspects of the research papers they reviewed. Research students were more focused on the scientific content of the papers.
Does undergraduate research enhance student learning? Undoubtedly. However, on some level, research might not make students better than their peers: the fact that they want to do research means that they are probably better students already. Many primarily undergraduate institutions offer research opportunities to their students, but many do not do so on a large scale early in their curricula because of limited space, funds, faculty availability, or a combination thereof. Results of this study demonstrate that at a small, liberal arts college, only about half of the students were willing and able to conduct research early in their careers. This result also suggests that similar institutions may be able to offer inclusive, long-term research programs at a more manageable scale than anticipated. Of course, this is not to say that students who do not apply for extracurricular research opportunities should be denied the opportunity to become involved in inquiry-based laboratory experiences. These students would assumedly still be exposed to this thought process in the laboratory portions of their science classes. Simply having an application process for conducting research seems to serve as a self-selection event where prime candidates for research identify themselves (i.e., fewer than half of the students apply, and the applicants earn higher grades than their peers). As such, merely going through the motions of an application process—even if all students who applied would be admitted by default—could identify a pool of motivated, talented students early in their careers, and this pool could be scaled such that a small liberal arts college could support each student in the lab. In short, simply allowing the students to identify themselves, based on their motivation and availability—two criteria that are important for successful undergraduate research—may enable institutions to run manageable research programs despite offering the opportunity to all of their students. 5. Summary and Conclusions In this paper, we found that students involved in research outperformed their peers who did not choose to apply for the research program. Surprisingly, the same cohort of students involved in research outperformed their peers even prior to participation in the research program. This indicated that personal interest and drive to engage in research experiences may be the indicator for improved academic performance, instead of participation in the research itself. We conclude that the experience of conducting research in the natural sciences is, indeed, correlated with stronger writing skills and that this correlation is consistent with the idea that students who conduct research have better developed higher-order thinking skills often associated with scientific research. Further studies examining the long-term effects on both academic performance and soft skills that may be developed during research experiences and one-on-one mentoring will illuminate further benefits of undergraduate research in STEM. These results add to the extensive body of work documenting the positive effects of undergraduate research.
Enlarge this image.Figure 1. The effect of participation on mean grade-point average (GPA) in the spring semester. (a) Students in the research program earned higher grades than their peers when involved in the research program. (b) Students who applied to the research program earned higher grades than students who did not apply. (* Denotes statistically significant differences in GPA. Error bars represent two standard errors both above and below the mean).
Enlarge this image.Figure 2. Students who were selected to participate in the research program outperformed their peers before the research started (i.e., in the fall semester). (a) Students who were selected to participate in the research program during the upcoming spring semester outperformed their peers during the fall semester. (b) Students who applied to the research program earned higher grades than students who did not apply. (* Denotes statistically significant difference in GPA. Error bars represent two standard errors both above and below the mean.).
Enlarge this image.Figure 3. In the combined samples, the files from both the non-research and research groups are combined as if they came from a single sample.
Enlarge this image.Figure 4. In the separate samples, the files are grouped into two folders, Folder1_non and Folder1_res, representing the non-research and research samples.
| Data Summary | A | B | Total | |
|---|---|---|---|---|
| n | 8 | 7 | 15 | |
| Σx | 8471 | 9441 | 17912 | |
| Σx2 | 9,207,313 | 12,991,027 | 22,198,340 | |
| SS | 237,582.87 | 257,815.42 | 809,023.73 | |
| mean | 1058.875 | 1348.7143 | 1194.1333 | |
| Results | ||||
| Meana-Meanb | t | df | p-value one-tailed | p-value two-tailed |
| −289.8393 | −2.87 | 13 | 0.0065715 | 0.013143 |
Meana is the non-research student data and Meanb is research student data.
| Concept | Rel Freq (%) | Strength (%) | Prominence |
|---|---|---|---|
| report | 10 | 97 | 2.5 |
| introduction | 7 | 67 | 1.7 |
| section | 13 | 60 | 1.5 |
| study | 8 | 60 | 1.5 |
| information | 7 | 57 | 1.5 |
| results | 9 | 54 | 1.4 |
| authors | 15 | 51 | 1.3 |
| used | 9 | 43 | 1.1 |
| research | 14 | 42 | 1.1 |
| article | 10 | 40 | 1.0 |
| Concept | Rel Freq (%) | Strength (%) | Prominence |
|---|---|---|---|
| pain | 6 | 97 | 1.6 |
| proposal | 6 | 97 | 1.6 |
| paper | 4 | 70 | 1.2 |
| article | 10 | 60 | 1.0 |
| research | 12 | 57 | 0.9 |
| different | 4 | 56 | 0.9 |
| used | 7 | 56 | 0.9 |
| authors | 9 | 48 | 0.8 |
| results | 5 | 45 | 0.8 |
| section | 5 | 39 | 0.7 |
| Concept | Rel Freq (%) | Strength (%) | Prominence |
|---|---|---|---|
| information & overall | <1 | 100 | 12.7 |
| introduction & overall | <1 | 100 | 11.4 |
| introduction & methods | 2 | 82 | 11.2 |
| results & figures | 1 | 86 | 10.0 |
| introduction & down | <1 | 100 | 10.0 |
| report & down | <1 | 100 | 10.0 |
| report & field | <1 | 100 | 10.0 |
| report & research | 5 | 100 | 9.8 |
| Concept | Rel Freq (%) | Strength (%) | Prominence |
|---|---|---|---|
| pain & low | 3 | 96 | 42.5 |
| proposal & low | <1 | 100 | 13.5 |
| pain & better | <1 | 100 | 10.7 |
| proposal & better | <1 | 100 | 10.7 |
| proposal & writing | <1 | 100 | 10.7 |
| paper & written | <1 | 100 | 9.9 |
| proposal & throughout | <1 | 100 | 9.7 |
| pain & effects | <1 | 100 | 9.3 |
| Concept | Related Concept | Supporting Text |
|---|---|---|
| report | analysis | , data analysis and writing the report |
| down | The reviewer found analyzing this report helpful for future research in the field of youth with ASD. He found the concept of beginning with a large setting and then whittling it down to | |
| field | research report is the psychology field. As well | |
| medical | h literary diction the authors assume their report is being viewed by medical and psychological audiences. | |
| Research | The research report, | |
| review | The list of references is a lot shorter than most reference sections for research reports; however, all of the sources are reliable peer reviewed published journals | |
| scientific | With this report being directed into the medical field the language becomes a little more complex and scientific |
| Concept | Related Concept | Supporting Text |
|---|---|---|
| pain | better | The purposed of this article is to be able to help better diagnose and treat pain among both men and women. |
| Effects | have differential effects on men’s and women’s response to experimentally induced pain. | |
| Example | When Petrisko talks about one of the most commonly encountered injuries for low back pain he does a good job at explaining the specific structures involved. For example on page 8, he talks about the musculoligamentous injuries he goes into detail about how the supporting musculature of the low back or the ligamentous structures relating to the lumbosacral spine and how those are the parts that are injured. | |
| Low | ures of pain tolerance with low- | |
| proposal | Put into this proposal is with his appendix of 5 different algorithms for low back pain. This appendix can be found after the conclusion and before the bibliography. |
Author Contributions
Conceptualization, P.J. and N.B.; methodology, P.J.; software, D.T.; validation, E P.,; formal analysis, N.B.; investigation, P.J.; data curation, E.P.; writing-original draft preparation, P.J.; writing-review and editing, E.P.; visualization, D.T.; supervision, N.B.; project administration, P.J. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Conflicts of Interest
The authors declare no conflict of interest.
1. Jansen, D.A.; Jadack, R.A.; Ayoola, A.B.; Doornbos, M.M.; Dunn, S.L.; Moch, S.D.; Moore, E.M.; Wegner, G.D. Embedding research in undergraduate learning opportunities. West. J. Nurs. Res. 2015, 37, 1340-1358.
2. Bauerle, C.; Depass, A.; Lynn, D.; O'Connor, C.; Singer, S.; Withers, M. Vision and Change in Undergraduate Biology Education: A Call to Action. 2009. Available online: http://visionandchange.org/files/2011/03/Revised-Vision-and-Change-Final-Report.pdf (accessed on 10 November 2014).
3. Brownell, J.E.; Swaner, L.E. Five High-Impact Practices: Research on Learning Outcomes, Completion, and Quality; Association of American Colleges & Universities: Washington, DC, USA, 2010.
4. Mogk, D.W. Undergraduate research experiences as preparation for graduate study in geology. J. Geol. Educ. 1993, 41, 126-128.
5. Tomovic, M.M. Undergraduate research-Prerequisite for successful lifelong learning. ASEE Annu. Conf. Proc. 1994, 1, 1469-1470.
6. Bauer, K.W.; Bennett, J.S. Alumni perceptions to assess undergraduate research experience. J. High. Educ. 2003, 74, 210-230.
7. Russell, S.H.; Hancock, M.P.; McCullough, J. Evaluation of NSF Support for Undergraduate Research Opportunities; 2003 NSF-Program Participant Survey: Final Report; SRI International: Menlo Park, CA, USA, 2005.
8. Ward, C.; Bennett, J.; Bauer, K. Content analysis of undergraduate research student evaluations. Retrieved March 2005. 2002. Available online: http://www.udel.edu/RAIRE (accessed on 22 May 2020).
9. Zydney, A.L.; Bennett, J.S.; Shahid, A.; Bauer, K.W. Impact of undergraduate research experience in engineering. J. Eng. Educ. 2002, 91, 151-157.
10. Zydney, A.L.; Bennett, J.S.; Shahid, A.; Bauer, K.W. Faculty perspectives regarding the undergraduate research experience in science and engineering. J. Eng. Educ. 2002, 91, 291-297.
11. Lopatto, D. Survey of undergraduate research experiences (SURE): First findings. Cell Biol. 2004, 3, 270-277.
12. Lopatto, D. Undergraduate research experiences support science career decisions and active learning. CBE Life Sci. Educ. 2007, 6, 297-306.
13. Seymour, E.; Hunter, A.; Laursen, S.L.; Deantoni, T. Establishing the benefits of research experiences for undergraduates in the sciences; First findings from a three-year study. Sci. Educ. 2004, 88, 493-534.
14. Hunter, A.-B.; Laursen, S.L.; Seymour, E. Becoming a scientist: The role of undergraduate research in students' cognitive, personal and professional development. Sci. Educ. 2007, 91, 36-74.
15. Russell, S.H.; Hancock, M.P.; McCullough, J. The pipeline: Benefits of undergraduate research experiences. Science 2007, 27, 548-549.
16. Hathaway, R.S.; Nagda, B.A.; Gregerman, S.R. The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. J. Coll. Stud. Dev. 2002, 43, 614-631.
17. Kremer, J.F.; Bringle, R.G. The effects of an intensive research experience on the careers of talented undergraduates. J. Res. Dev. Educ. 1990, 24, 1-5.
18. Greenwald, D.A. Faculty involvement in undergraduate research: Considerations for nurse educators. Nurs. Educ. Perspect. 2010, 31, 368-371.
19. Hu, S.; Scheuch, K.; Schwartz, R.A.; Gayles, J.G.; Li, S. Reinventing undergraduate education: Engaging college students in research and creative activities. ASHE High. Educ. Rep. 2008, 33, 1-103.
20. Schaefer, B. What university students learn from doing academic research: Implications for nursing educators. Singap. Nurs. J. 2013, 40, 32-37.
21. Brown, J.S.; Collins, A.; Duguid, P. Situated cognition and the culture of learning. Educ. Res. 1989, 18, 32-42.
22. Farmer, D.W.; Mech, T.F. Information Literacy: Developing Students as Independent Learners (No. 78); Jossey-Bass: San Francisco, CA, USA, 1992.
23. Leximancer. Leximancer Manual Version 2.2; The University of Queensland: Brisbane, Australia, 2005.
24. Thomas, D. Searching for significance in unstructured data: Text mining with Leximancer. Eur. Educ. Res. J. 2014, 13, 235-256.
25. Gelphi. 2015. Available online: http://info.leximancer.con/ (accessed on 23 May 2015).
26. Moore, R. Students' choices in developmental education: Is it really important to attend class? Res. Teach. Develop. Educ. 2003, 20, 42-52.
27. Jensen, P.A.; Moore, R. Students' behaviors, grades, and perceptions in an Introductory Biology course. Amer. Biol. Teacher. 2008, 70, 483-487.
28. Jensen, P.A.; Moore, R. What do help sessions accomplish in introductory science courses? J. Collge Sci. Teach. 2009, 38, 60-65.
Nate Bickford1,*, Elizabeth Peterson1, Philip Jensen2 and Dave Thomas3
1Biology Department, Colorado State University Pueblo, Pueblo, CO 81003, USA
2Department of Biology, Rocky Mountain College, Billings, MT 59102, USA
3Mathematics Department, University of Providence, Great Falls, MT 59401, USA
*Author to whom correspondence should be addressed.
© 2020. This work is licensed under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.