Oceanography The Official Magazine of
The Oceanography Society
Volume 30 Issue 02

View Issue TOC
Volume 30, No. 2
Pages 198 - 208

OpenAccess

Two-Stage Exams: A Powerful Tool for Reducing the Achievement Gap in Undergraduate Oceanography and Geology Classes

By Barbara C. Bruno , Jennifer Engels, Garrett Ito , Jeffrey Gillis-Davis, Henrietta Dulai, Glenn Carter, Charles Fletcher , and Daniela Böttjer-Wilson 
Jump to
Article Abstract Citation Supplementary Materials References Copyright & Usage
Article Abstract

As part of a school-wide course transformation project at the University of Hawai‘i to improve student learning and retention, multiple geology and oceanography instructors are introducing two-stage exams in their undergraduate courses. The first stage is the traditional, individual exam. The second stage is collaborative, in which groups of two to six students answer the same (or a subset of) questions posed during the first stage. We analyzed n = 289 scores on 14 two-stage exams in seven sections of five unique undergraduate courses taught by six different instructors. Two of the courses are categorized as oceanography and three as geology, although all courses cover both terrestrial and marine content. For each exam, the mean group score (stage two) exceeded the mean individual score (stage one), and all gains were statistically significant at α = 0.05. Overall, the mean individual score was 73.2%, with a standard deviation of 15.0%. The mean group score was 89.6% with a standard deviation of 9.3%, reflecting an overall improvement and a narrowing of the achievement gap. Students who scored in the bottom quartile of the individual exam experienced the greatest improvement from individual to group (increase of 29.9 percentage points). This compares to a lower, but still statistically significant, increase of 5.5 percentage points for students in the top quartile. The majority (83%) of groups had a group score that exceeded the scores of all individuals in that group, which argues against the theory that the increased group score is due to group members simply copying answers from the top-performing individual in their group. A formal parametric (z) analysis reveals that the group scores are systematically higher than the maximum individual scores, indicative of a systematic (non-random) process. We interpret this process to be collaborative learning during the group stage of the exam. A cohort analysis reveals that groups containing all combinations of high- and low-performing students during stage one experience statistically significant mean gains in exam scores, and selecting groups to include a mix of high- and low-performing students is a highly effective way to proactively reduce the achievement gap. 

Citation

Bruno, B.C., J. Engels, G. Ito, J. Gillis-Davis, H. Dulai, G. Carter, C. Fletcher, and D. Böttjer-Wilson. 2017. Two-stage exams: A powerful tool for reducing the achievement gap in undergraduate oceanography and geology classes. Oceanography 30(2):198–208, https://doi.org/10.5670/oceanog.2017.241. 

Supplementary Materials

A video of students working together during the group stage of a two-stage exam is available at https://youtu.be/a498AAS90Ew.

References
    Arthurs, L., and A. Templeton. 2009. Coupled collaborative in-class activities and individual follow-​up homework promote interactive engagement and improve student learning outcomes in a college-level Environmental Geology course. Journal of Geoscience Education 57(5):356–371, https://doi.org/10.5408/1.3544287.
  1. Bonwell, C.C., and J.A. Eison. 1991. Active Learning: Creating Excitement in the Classroom. ASHE-ERIC Higher Education Report No. 1, 121 pp.
  2. Bruno, B.C., J.L.K. Wren, K. Noa, E.M. Wood-Charlson, J. Ayau, S.L. Soon, H. Needham, and C.A. Choy. 2016. Summer bridge program establishes nascent pipeline to expand and diversify Hawai‘i’s undergraduate geoscience enrollment. Oceanography 29(2):286–292, https://doi.org/​10.5670/oceanog.2016.33.
  3. Carl Wieman Science Education Initiative. 2014. Two-stage exams, http://www.cwsei.ubc.ca/resources/files/Two-stage_Exams.pdf.
  4. Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Routledge, United Kingdom, 567 pp.
  5. Crouch, C.H., and E. Mazur. 2001. Peer instruction: Ten years of experience and results. American Journal of Physics 69(9):970–977, https://doi.org/​10.1119/1.1374249.
  6. Derek Bok Center for Teaching and Learning at Harvard University. 2016. Active learning, http://bokcenter.harvard.edu/active-learning.
  7. Deslauriers, L., E. Schelew, and C. Wieman. 2011. Improved learning in a large-enrollment physics class. Science 332(6031):862–864, https://doi.org/​10.1126/science.1201783.
  8. Ertmer, P.A., and T.J. Newby. 1996. The expert learner: Strategic, self-regulated, and reflective. Instructional Science 24:1–24, https://doi.org/​10.1007/BF00156001.
  9. Fengler, M., and P.M. Ostafichuk. 2015. Successes with two-stage exams in mechanical engineering. Proceedings of the Canadian Engineering Education Association (CEEA15) Conference, McMaster University, May 31–June 3, 2015.
  10. Freeman, S., S.L. Eddy, M. McDonough, M.K. Smith, N. Okoroafor, H. Jordt, and M.P. Wenderoth. 2014. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America 111(23):8,410–8,415, https://doi.org/10.1073/pnas.1319030111.
  11. Gilley, B.H., and B. Clarkston. 2014. Collaborative testing: Evidence of learning in a controlled in-class study of undergraduate students. Journal of College Science Teaching 43(3):83–91.
  12. Haak, D.C., J. HilleRisLambers, E. Pitre, and S. Freeman. 2011. Increased structure and active learning reduce the achievement gap in introductory biology. Science 332:1,213–1,216, https://doi.org/10.1126/science.1204820.
  13. Knierim, H., and R.K. Davis. 2015. Two-stage exams improve student learning in an introductory geology course: Logistics, attendance, and grades. Journal of Geoscience Education 63:157–164, https://doi.org/10.5408/14-051.1.
  14. Leight, H., C. Saunders, R. Calkins, and M. Withers. 2012. Collaborative testing improves performance but not content retention in a large-enrollment introductory biology class. CBE – Life Sciences Education 11:392–401, https://doi.org/10.1187/cbe.12-04-0048.
  15. Lindsley, J.E., D.A. Morton, K. Pippitt, S. Lamb, and J.M. Colbert-Getz. 2016. The two-stage examination: A method to assess individual competence and collaborative problem solving in medical students. Academic Medicine 91(10):1,384–1,387, https://doi.org/10.1097/ACM.0000000000001185.
  16. Lorenzo, M., C.H. Crouch, and E. Mazur. 2006. Reducing the gender gap in the physics classroom. American Journal of Physics 74(2):118–122, https://doi.org/10.1119/1.2162549.
  17. Lucas, A. 2009. Using peer instruction and i-clickers to enhance student participation in calculus. PRIMUS 19(3):219–231, https://doi.org/​10.1080/10511970701643970.
  18. Lusk, M., and L. Conklin. 2003. Collaborative testing to promote learning. The Journal of Nursing Education 42(3):121–124.
  19. Lyle, K.S., and W.R. Robinson. 2003. A statistical evaluation: Peer-led team learning in an organic chemistry course. Journal of Chemical Education 80(2):132–134, https://doi.org/10.1021/ed080p132.
  20. Oakley, B., R.M. Felder, R. Brent, and I. Elhajj. 2004. Turning student groups into effective teams. Journal of Student Centered Learning 2(1):9–34.
  21. PCAST (President’s Council of Advisors on Science and Technology). 2012. Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. 103 pp, https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_2-25-12.pdf.
  22. Pocock, S.J. 2006. Current controversies in data monitoring for clinical trials. Clinical Trials 3(6):513–521, https://doi.org/10.1177/1740774506073467.
  23. Rieger, G., and C. Heiner. 2014. Examinations that support collaborative learning: The students’ perspective. Journal of College Science Teaching 43(4):41–47, https://doi.org/10.2505/4/jcst14_043_04_41.
  24. Ruiz-Primo, M.A., D. Briggs, H. Iverson, R. Talbot, and L.A. Shepard. 2011. Impact of undergraduate science course innovations on learning. Science 331:1,269–1,270, https://doi.org/10.1126/science.1198976.
  25. Sandahl, S.S. 2010. Collaborative testing as a learning strategy in nursing education. Nursing Education Perspectives 31(3):142–147.
  26. Sawilowsky, S. 2009. New effect size rules of thumb. Journal of Modern Applied Statistical Methods 8(2):467–474.
  27. SERC (Science Education Resource Center at Carleton College). 2016. Starting point: Teaching entry level geoscience—Active learning, http://serc.carleton.edu/introgeo/gallerywalk/active.html.
  28. Shindler, J.V. 2004. “Greater than the sum of the parts?” Examining the soundness of collaborative exams in teacher education courses. Innovative Higher Education 28:273–283, https://doi.org/​10.1023/B:IHIE.0000018910.08228.39.
  29. Smith, M.K., W.B. Wood, W.K. Adams, C. Wieman, J.K. Knight, N. Guild, and T.T. Su. 2009. Why peer discussion improves student performance on in-class concept questions. Science 323:122–124, https://doi.org/10.1126/science.1165919.
  30. Smith, M.K., W.B. Wood, K. Krauter, and J. Knight. 2011. Combining peer discussion with instructor explanation increases student learning from in-class concept questions. CBE – Life Sciences Education 10:55–63, https://doi.org/10.1187/cbe.10-08-0101.
  31. Snyder, J.J., J.D. Sloane, R.D.P. Dunk, and J.R. Wiles. 2016. Peer-led team learning helps minority students succeed. PLoS Biology 14(3):e1002398, https://doi.org/10.1371/journal.pbio.1002398.
  32. Springer, L., M.E. Stanne, and S.S. Donovan. 1999. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology. Review of Educational Research 69(1):21–51.
  33. Stearns, S.A. 1996. Collaborative exams and learning tools. College Teaching 44(3):111–112, https://doi.org/​10.1080/87567555.1996.9925564.
  34. Tenney, A., and B. Houck. 2003. Peer-led team learning in introductory biology and chemistry courses: A parallel approach. Journal of Mathematical Sciences 6:11–20.
  35. University of Hawai‘i Institutional Research and Analysis Office. 2016. Enrollment, https://www.hawaii.edu/institutionalresearch/enrReport.action?reportId=ENRT00.
  36. Wamser, C.C. 2006. Peer-led team learning in organic chemistry: Effects on student performance, success, and persistence in the course. Journal of Chemical Education 83:1,562–1,566, https://doi.org/10.1021/ed083p1562.
  37. Wieman, C., G.W. Rieger, and C.E. Heiner. 2014. Physics exams that promote collaborative learning. The Physics Science Teacher 52:51–53, https://doi.org/10.1119/1.4849159.
  38. Yuretich, R.F., S.A. Khan, R.M. Leckie, and J.J. Clement. 2001. Active-learning methods to improve student performance and scientific interest in a large introductory oceanography course. Journal of Geoscience Education 49(2):111–119, https://doi.org/10.5408/1089-9995-49.2.111.
Copyright & Usage

This is an open access article made available under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution, and reproduction in any medium or format as long as users cite the materials appropriately, provide a link to the Creative Commons license, and indicate the changes that were made to the original content. Images, animations, videos, or other third-party material used in articles are included in the Creative Commons license unless indicated otherwise in a credit line to the material. If the material is not included in the article’s Creative Commons license, users will need to obtain permission directly from the license holder to reproduce the material.