Engaging students through the VLE : comparing like with like using the # VLEIreland student survey

Virtual learning environments (VLEs) provide the core infrastructure for the digital learning experiences for many students in Irish and international higher education institutions. Hence the student experience and voice offer an important perspective to understand to what extent this proves a strategic choice, and investment of institutional resources and lecturers’ time. However, surveys comparing a diversity of institutions at different moments in time offer serious methodological limitations to data analysis. In this paper, results of the #VLEIreland student survey are presented for four Irish higher education institutions, which were surveyed using the common questionnaire with 3,332 student responses in 2011 and 5,170 when the survey was repeated in 2013. To some extent, this allows us to analyse the data while relatively controlling for institutional influences. This snapshot of students’ use and perceptions at two points allow for tentative trends to be drawn as to any changing patterns over time. The comparison of results indicate a consistent finding of high frequency of use (with almost half using daily in the more recent findings). The VLE thus provides a stable base for sharing learning resources, managing assignments and student communications. The consistency of the findings indicate a certain maturity of use of the VLE, albeit with a continued emphasis on disseminating notes and other learning resources to students. As the student VLE experience is largely determined by their lecturers’ use, a continued emphasis on the development of digital capacity of lecturers (as well as students) is needed. Irish Journal of Technology Enhanced Learning Ireland, 2018. © 2018 Raftery, D. & Risquez, A. The Irish Journal of Technology Enhanced Learning Ireland is the journal of the Irish Learning Technology Association, an Irish-based professional and scholarly society and membership organisation. (CRO# 520231) http://www.ilta.ie/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. 1 The basis for this article stems from a conference paper (Cosgrave et al., 2013) and we would like to acknowledge the valuable contributions of all our collaborators. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018


Introduction
In Ireland, the use of digital tools and the internet both within and beyond formal teaching environments has become a normalised and expected dimension of the teaching and learning experience (O'Rourke, 2017, p.13).
In Irish higher education institutions as well as internationally, virtual learning environments (VLEs) have matured from early pilots in the early 2000s to now being widely used and considered core infrastructure.To justify this widespread investment in VLE technology, it is important to study patterns of actual student VLE use and student satisfaction (Naveh, Tubin & Pliskin, 2010) and indeed it was commented that Irish higher education lacks an evidence base in relation to VLE usage (Devine, 2015, p.16).It is essential to allow the student voice to inform the use of VLEs and the continuous improvement of their learning experiences.Irish students, along with staff, have an expectation of "ubiquitous connectivity, access to online resources and the availability and use of digital tools in almost every aspect of their work" (O'Rourke, 2017, p.21).
This paper contributes to this gap in the literature.Following a summary of research methods, this article presents and discusses students' views on the use of VLEs across higher education in Ireland.In particular, we focus our analysis on a comparison between the responses from students of four Irish higher education institutions which took the survey simultaneously in a two year gap (in 2011 and 2013 respectively).To some extent, this allows us to analyse the data while relatively controlling for institutional influences.These snapshots of students' use and perceptions at two points allow for tentative trends to be drawn as to any changing patterns over time.Prior to conclusions, the comparison analysis is discussed along with other research and some implications are drawn.

Methods
As described elsewhere in this special issue (Ryan & Risquez, 2018), students were surveyed using the standard questionnaire with a common set of questions.To enable comparisons over time, changes have been kept to a minimum in the survey through the multiple administrations year on year.In this article, the analysis compares the 3,332 student responses in 2011 for four institutions in which the survey was repeated in 2013, at which time 5,170 students responded.The four colleges represent both universities and institutes of technology, and a variety of VLEs are used.The response rates in the four colleges varied and, as the survey was administered online, the sample is self-selecting leading to potential sample bias.A specific limitation in interpreting any change between 2011 and 2013 overall for the four colleges is that there has been no attempt to control for changes in the relative composition of the overall sample (proportion of respondents from each institution in 2011: 38%, 31%, 18% and 13%; in 2013: 37%, 39%, 19% and 6%).Similarly, there may be other changes in demographics or the nature of the students surveyed that may distort the overall change reported (for example, gender and age of respondents from each institution in 2011: 42% female, 63% aged 23 and under; in 2013: 56% female, 69% aged 23 and under).In part, due to differences in VLE terminology, there may be substantial non-responses to certain questions.These limitations are a caveat to the interpretation and generalisability of the findings presented.This being said, the overall sample surmounts 8,400 responses, which offers a degree of reliability to the data presented next.

Findings
The 2013 student responses are, on average, strikingly similar to those of 2011 indicating relatively stability in student attitudes and use of the VLE.There seems to be a general increase in the routine usage of the VLE, with the continued predominant, almost universal, use of the VLE to solve the problem of disseminating learning resources to students.The main difference is in where and when students are accessing the VLE from, with a marked increase in the use of mobile devices.There is also a continued increase in the use of the VLE for submitting assignments and for academic integrity/plagiarism detection.The analysis below is presented under key themes that emerged in previous reporting of our research including the VLE as a content distribution platform, the need for consistency of use, the potential to enhance student communication and the expansion of flexibility (Risquez et al., 2013) and can be compared with results reported in the paper by Ryan and Risquez (2018) in this Special Issue.Students use the VLE frequently.This is consistent: slightly up in 2013, 48% of respondents use the VLE daily with a further 40% using it a few times a week and 7% once a week.This compares to 43% using daily in 2011, with 42% a few times a week and 7% once a week.The basic use of the VLE is as a simple, stable transmission system for lecture notes and other learning resources.As can be seen in Table 1, there is a general increase in reported VLE use across all categories asked.A substantial majority of students reported that they found the VLE broadly useful for getting course material from the lecturereither lecture Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 notes/handouts (96% in 2013, up from 87% in 2011) or other course resources (90% in 2013, up from 87% in 2011).There is substantial and continued growing use of the proportion of students who report finding the VLE useful for submitting assignments (83% in 2013, up from 76% in 2011) and for academic integrity/plagiarism detection (54% in 2013, up from 46% in 2011).There is stable rating of usefulness (about 2 out of 3) of class announcements.

Reported use
Respondents consistently had high levels of agreement with the statement that the VLE gives them more access to resources and learning materials (89% in 2013, 88% in 2011).The VLE provides a content management system with increasing use for an efficient process for managing and grading assignments and as an additional communication forum for students.
The increased submission of assignments may be due to the integration of systems such as Turnitin within the VLE.Despite being substantially more demanding of lecturer time, there seems to be a substantial minority finding quizzes and online discussions useful, a finding consistent with the findings of surveys with lecturers reported in this issue (Farrelly, Raftery & Harding, 2018).

Consistency of use
Only one of my lecturers uses [the VLE] which is great for that subject.Unfortunately I don't get to use it for other subjects.
Some lecturers put material up in a fairly disorganised manner which can be frustrating when trying to access material quickly.
Students consistently agree that the VLE is helpful and easy to use, about 94% and 83% respectively in both 2013 and 2011.There was a general increase in the proportion of coursework that had material in the VLE from 2011 to 2013.22% of respondents in 2013 reported all their coursework had material in the VLE compared to 10% in 2011 whilst there was a reduction having none to 2% in 2013 from 7% in 2011.Only a small proportion of respondents reported not using the VLE, about 1% in both 2013 and 2011.The overwhelming barrier given as a reason is the lack of use of the VLE by lecturers rather than any technical issues such as system reliability or access.
Students clearly would like their lecturers to make more use of the VLE: in 2013, 68% of respondents still strongly agreed or agreed with this statement (consistent with the 69% in 2011).Students want comprehensive and consistent basic usage that facilitates easier navigation within the VLE.This has implications for academic development, to ensure all staff are consistently using the basics.There is a modest increase in the level of agreement that lecturers make good use of the VLE, from 57% in 2011 to 62% in 2013.This continues an upward trend, implying that digital capacity is gradually being developed in the academic community and this needs to continue.Indeed, when asked "What is the most important thing you would like to see improved about the VLE?" in 2013 56% of students chose better usage by lecturers compared to 68% in 2011.
Note that the second most popular choice is consistently more use of audio/multimedia, which rose to 19% in 2013 from 13% in 2011.There was also a small increase for making the VLE easier to use, from 8% in 2011 to 13% in 2013.Coupled with a consistent 6% level of disagreement that VLEs are easy to use and about 10% disagreement that the respondent can get adequate help and support to use the VLE, this indicates that supporting students in using the VLE remains important.VLEs facilitate students communicate and engage with their lecturers and each other.There is a consistent agreement that the VLE gives more access to lecturers (70% in 2011, 67% in 2013).Relatively few students feel that using the VLE makes it more likely for them to communicate with their lecturer, for example by asking a question online (25% in 2011, 22% in 2013).However the VLE may be a valuable forum for students unwilling or unable to communicate with lecturers in class or in person, with the VLE offering an 'open office door' for students to engage with their lecturers.Note that consistently a much smaller proportion of respondents felt the VLE increased their access to classmates (27% in 2011, 21% in 2013) but again this may be valuable to particular students.

Figure 1 Aspects of the VLEs perceived contribution to learning
As can be seen in Figure 1, students seem to feel that the VLE provides a genuine aid for learning.There is a consistent majority that report that using the VLE makes it easier for them to learn (67% in 2011, 72% in 2013), as well as adding to and clarifying what is covered in class.In addition, almost half feels the VLE helps them understand how well they are doing (44% in 2011, 41% in 2013).Respondents were asked to indicate what times they used the VLE.Although such selfreporting is much less reliable than access logs, there was a dramatic increase from 2011 to 2013 in the level of access reported across a broad range of hours.As well as high usage during the day, this change is most pronounced from 7am to 9am (from 7% in 2011 to 21% in 2013), from 6pm to 10pm (from 36% to 81%) and during the night from 10pm to 7am (from 10% to 29%).This increase is largely outside of conventional study and office hours and may have implications for support.This increased flexibility of location and time (anywhere, anytime), coupled with broadly positive student attitudes that the VLE is helpful and makes learning easier, is a key outcome of the use of VLEs.Although beneficial for all students the remote availability of VLEs is particularly important for part-time and distance learners, enabling ubiquitous access to learning materials and communication tools.Mobile access, including VLE apps, provide useful new ways to easily communicate with students and enable the use of the VLE to support learning activities in class (see Raftery (2018) in this issue for further discussion).

Discussion
[The VLE] has been very important to my studies throughout the years, when on Erasmus the lack of a useful and easily accessible system like [the VLE] was very noticeable.
I feel that the use of online quizzes and video links on material covered in lectures helps with my learning of subjects and feel that all lecturers should be making more use of this feature.
These quotes illustrate aspects of how students value the VLE and its positive impact on their learning.The comparative analysis in the findings have highlighted the stability in student attitudes and use of the VLE.There has however been a general increase in the routine usage of the VLE, with a substantial increase in the student use of mobile devices.The VLE's predominant role as a content distribution platform has been complemented with increasing use of the VLE for managing and grading assignments.The comparative analysis above confirms a stability in students' view on the need for lecturers to use the VLE consistently, and highlights how the VLE enhances student communication and flexibility.
Survey respondents consistently consider that the VLE is helpful (about 94%), consistent with findings of Heaton-Shrestha et al. (2009) that, in a UK university, students felt the VLE enhanced their performance, effectiveness and experience.Individual students commented that "[the VLE] gives you direction", "I'd be lost without [the VLE].It's really helpful" and "I could not survive academically without [the VLE]".There is stable agreement that the VLE is easy to use (over 80%), but small levels of disagreement indicate continued importance of appropriate initial induction and ongoing support.Students like the flexible and structured access to learning resources that the VLE offersbeing able to access from wherever and whenever, with substantial access from home and in the evenings.Consistency in the layout, terminology and design of courses within the VLE is wanted and valued by students.The increased use of mobile devices has implications for campus infrastructure (such as the provision of wi-fi and flexible learning spaces) and VLE design (including apps), as well as demand for mobile access via the VLE to rich multimedia learning resources (Irish examples: Marcus-Quinn & Cleary, 2015;Mitchell, 2014;Fannon, Kelly, MacBlain, Raftery & Brennan, 2015).
There is a general increase in students' reported VLE use.A key determinant of students' use of the VLE is the amount and type of use by lecturers.An increasing majority of respondents agreed that their lecturers made good use of the VLE, with consistently about 7 out of 10 wanting their lecturers to make more use of the VLE.Despite heavy workloads (especially in context of increased teaching and administration including working with more students), academics are demonstrating increasing effectiveness in basic VLE use.This can be understood as part of a trend where lecturers tend to make, on average, only incremental changes to their practice when faced with new technology in the form of a VLE (Dutton, Cheong & Park, 2004;Jenkins, Browne & Walker, 2005;Kirkup & Kirkwood, 2005).Naveh et al. (2010) suggest that "instructors can maintain their conservative teaching habits except for posting their course content … this can be done at low cost, yielding relatively high student satisfaction" (p.132).This is consistent with Brown's argument that, despite almost universal adoption by institutions in the UK, "VLEs have had only a relatively slight impact on pedagogy in higher education" (2010, p.7).For continual improvements as well as substantial or innovative changes in teaching practice, both national and institutional strategic vision is needed, coupled with a renewed emphasis on fostering digital literacies, academic professional development, and pedagogical and technical support (for further discussion of staff attitudes to, and use of, the VLE see Farrelly, Raftery and Harding (2018) and of the importance of staff development see Harding (2018), both in this Special Issue).Indeed, the National Forum report Building Digital Capacity in Higher Education directly links the under-utilisation of current digital infrastructure with the need for strategic and pedagogical guidance, and the development of digital literacy within the teaching community as follows (2014, p.7): The constant emergence of newer and better tools has often resulted in confusion among staff regarding the best tools to recommend and use, and to develop and learn about in their teaching and learning practices.This may go some way towards explaining the emerging evidence suggesting that key digital resources (for example, virtual learning environments) are not being used to their full pedagogical capacity.
Our findings that in 2013 almost 9 in 10 students report finding the VLE useful for submitting assignments has been corroborated by findings in the Irish Survey of Student Engagement, where 86% of respondents stated they used a VLE to discuss or complete an assignment sometimes, often or very often (2015, p. 18).Thus as the VLE is increasingly used to manage aspects of assessing students (submission of assignments, online quizzes, discussions and journals), the administrative requirements to retain student work mean that formalised policies and procedures are required.Policies are also needed to ensure adequate privacy and data protection, whilst enabling analysis of student data within the VLE (such as a data source for learning analytics projects such as DESSI (National Forum, n.d.); see Logan-Phelan (2018) in this Special Issue for further discussion of learning analytics).

Conclusion
The consistent finding of high frequency of use (with almost half using daily and a further four in ten students using the VLE at least a few times a week in 2013), and the greater flexibility of access and communication via mobile devices, combines to reinforce that the VLE provides a stable base for sharing learning resources, managing assignments and student communications.The consistency of the findings indicates a certain maturity of use of the VLE.
As indicated by the rapid rise in the use of mobile devices to use the VLE, advances in technology and the uses of that technology can facilitate major and profound changes in practice and use.However, this is by no means guaranteed.Indeed technological developments, combined with wider changes in higher education, will affect the evolving nature of use and future role of VLEs, perhaps disruptively.In this context, national and institutional strategic engagement is important as is that of departments, and individual and groups of academics.The support and continued academic professional development of lecturers is needed with appropriate emphasis on digital literacy and digital pedagogy to enhance student learning.
to [the VLE] from my phone.All notes are portable and when I have a free moment I can go online and study.I can learn at home, even in bed.

Figure 2 :
Figure 2: Where students access the VLE from

Table 1 : Proportion of students self-reported use and rating as broadly useful (useful or very useful)
My course is very time consuming and it's difficult to manage study around this. [The VLE]allows me that time to concentrate my study in a precise way.It suits me just the way it is ...