Using learning analytics to improve online formative quiz engagement

Irene O'Dowd

Abstract


This paper describes the findings of a small research study, conducted in a third-level online college, using learning analytics to examine the implementation of formative quizzes in a blended-learning post-primary teaching programme. Using historic data captured in a virtual learning environment (VLE) for a single cohort (n=126), patterns of use of formative Knowledge Check quizzes were analysed with particular regard to completion and retakes. Three hypotheses were tested using appropriate data correlation methods. Completion levels for quizzes were correlated with completion levels for other online tasks to see whether an increase in task workload resulted in a decrease in quiz engagement. A second test compared levels of quiz re-attempts with completion levels for other online tasks, to see whether different patterns of quiz attempts were linked to different levels of online engagement. Finally, the data was analysed to ascertain the relationship, if any, between student gender and different patterns of quiz attempts, to see if gender might be a factor in quiz engagement. The findings of this study suggested that the decrease in engagement with quizzes was not significantly related to task workload increase, and that there is a relationship between quiz re-attempts and higher module engagement. The findings are presented and discussed in the context of student engagement with online formative strategies in humanities-based subjects. Options are considered for enhancing engagement and formative value in this teaching and learning context; the potential of learning analytics in informing evidence-based improvements in digital learning design is also assessed.


Keywords


Moodle; quiz; quizzes; formative assessment; learning analytics; digital learning; quantitative; data analysis; engagement

Full Text:

PDF

References


Arend, B. (2007). Course Assessment Practices and Student Learning Strategies in Online Courses. Journal of Asynchronous Learning Networks, 11(4), 3-17.

Boud, D. (2000). Sustainable Assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151-167. doi: 10.1080/713695728

Cohen, D. and Sasson, I. (2016). Online quizzes in a virtual learning environment as a tool for formative assessment. Journal of Technology and Science Education, 6(3), 188-208. doi: 10.3926/jotse.217

Connolly, P. (2007). Quantitative Data Analysis in Education: A Critical Introduction Using SPSS, London: Routledge.

Dietrichson, A. (2013). Beyond Clickometry: Analytics for Constructivist Pedagogies. International Journal on E-Learning, 12(4), 333-351.

Fill, K. and Brailsford, S. (2005). Investigating gender bias in formative and summative CAA, in Proceedings of the 9th CAA Conference, Loughborough University, UK. Retrieved from https://dspace.lboro.ac.uk/dspace-jspui/bitstream/2134/1994/1/FillK_BrailsfordS.pdf [Accessed 30/09/2017]

Freasier, B., Collins, G. and Newitt, P. (2003). A Web-Based Interactive Homework Quiz and Tutorial Package To Motivate Undergraduate Chemistry Students and Improve Learning. Journal of Chemical Education, 80(11), 1344-1364. doi: 10.1021/ed080p1344

Gunn, C. (2003). Dominant or Different? Gender Issues in Computer-Supported Learning. Journal of Asynchronous Learning Networks, 7(1), 14-30.

Heinrich, E. (2015). Can learning analytics provide useful insights? An exploration on course level, in T. Reiners, B.R. von Konsky, D. Gibson, V. Chang, L. Irving, and K. Clarke (eds), Globally connected, digitally enabled: proceedings of ascilite 2015, Perth, Australia, pp. 108-112. Retrieved from http://research.moodle.net/81/ [Accessed 30/09/2017]

Hoskins, S.L. and Van Hooff, J.C. (2005). Motivation and ability: which students use online learning and what influence does it have on their achievement?. British Journal of Educational Technology, 36(2), 177–192. doi: 10.1111/j.1467-8535.2005.00451.x

Kearns, L. (2012). Student Assessment in Online Learning: Challenges and Effective Practices. MERLOT Journal of Online Learning and Teaching, 8(3). Retrieved from http://jolt.merlot.org/vol8no3/kearns_0912.htm [Accessed 30/09/2017]

Kibble, J.D. (2011). Voluntary participation in online formative quizzes is a sensitive predictor of student success. Advances in Physiology Education, 35(1), 95-96. doi: 10.1152/advan.00053.2010

Lewis, D., Trail, T., Srinivasan, S., Lee, S. and Lopez, S. (2010). Knowledge check questions: Best practices for use of this instructional strategy, in Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2010, Chesapeake, VA, pp. 2783-2788. Retrieved from http://davidlewisphd.com/publications/2010-Lewisetal-EDMEDIAbestPracticesFinal.pdf [Accessed 30/09/2017]

Nicol, D. J. and Milligan, C. (2006). Rethinking technology-supported assessment in terms of the seven principles of good feedback practice, in C. Bryan and K. Clegg (eds), Innovative Assessment in Higher Education. London: Taylor and Francis Group Ltd.

Paquette, P. (2016). Instructing the Instructors: Training Instructors to Use Social Presence Cues in Online Courses. Journal of Educators Online, 13(1), 80-108.

Richardson, J.C. and Newby, T. (2006). The Role of Students' Cognitive Engagement in Online Learning. American Journal of Distance Education, 20(1), 23-37. doi: 10.1207/s15389286ajde2001_3

Salas-Morera, L., Arauzo-Azofra, A. and García-Hernández, L. (2012). Analysis of online quizzes as a teaching and assessment tool. Journal of Technology and Science Education, 2(1), 39-45. doi: 10.3926/jotse.30

Sclater, N. and Mullan, J. (2017). Jisc briefing: Learning analytics and student success – assessing the evidence. JISC. Retrieved from http://repository.jisc.ac.uk/6560/1/learning-analytics_and_student_success.pdf [Accessed 30/09/2017]

Simkin M.G. and Kuechler W.L. (2005). Multiple choice tests and student understanding: what is the connection?. Decision Sciences Journal of Innovative Education, 3, 73–97. doi: 10.1111/j.1540-4609.2005.00053.x

Tuttle, H.G. (2010). 10 Reasons to Use Online Practice Quizzes (Formative Assessment). Education with Technology blog. Retrieved from https://eduwithtechn.wordpress.com/2011/04/04/10-reasons-to-use-online-practice-quizzes-formative-assessment/ [Accessed 30/09/2017]




DOI: https://doi.org/10.22554/ijtel.v3i1.25

Refbacks

  • There are currently no refbacks.


Copyright (c) 2018 Irene O'Dowd

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

This journal is published under the terms of the Creative Commons Attribution 4.0 International (CC-BY 4.0) License.