The Buzz around Learning Analytics – Enablers and Challenges Identified through the #VLEIreland Project

Increased availability of big datasets around learner activity with learning analytics (LA), offers universities great scope in data driven decisions around individual student learning, yet learning analytics is still in its infancy in Ireland. LA are used mostly to identify student retention and graduation rates, and less used to personalise learning to support students. This article presents an overview of the challenges to the use of learning analytics within institutions of higher education in Ireland. Using findings from the #VLEIreland Project on VLE usage the author examines the enablers and the challenges to the creation of large datasets on learning. With reference to current national initiatives, the article outlines how these initiatives can support the collection of datasets on learning, placing Irish institutions in a position to scale up implementation of learning analytics.


Introduction
Business intelligence and analytics (BI&A) and the related field of big data analytics have become increasingly important in both the academic and the business communities over the past two decades (Chen, Chiang & Storey, 2012) with different outcomes for both.Businesses are deploying sophisticated analytic techniques to evaluate rich data sources, identify patterns within the data and exploit these patterns in decision making.This allows for the collating of intelligence in the form of qualitative and quantitative data on how customers make purchasing decisions and use products (Chaudhuri, Dayal & Narasayya, 2011).Many businesses are successfully using predictive analytics, the goal being to analyse past and present behaviour patterns of their customers, predict trends before they happen and build sound business strategies for the future.
Increased availability of big datasets around learner activity with learning analytics (LA) offers universities great scope in data driven decisions around individual student learning, yet LA are used mostly to identify student retention and graduation rates, and less for personalising learning to support students.LA technologies are available and the research evidence on learning analytics has made recommendations on how institutions should approach their implementation.These include 1) develop learning analytics that makes good use of pedagogy, align analytics with assessment practices, and 2) build capacity by training educators, researchers and developers (EU Science Hub, n.d.).However, Irish universities do not have the large datasets from technology enhanced learning to implement complex learning analytics.This short paper examines findings from the #VLEIreland Project to reveal VLE usage in Ireland, and as a result, identify some of the challenges to the growth of technology enhanced learning in Ireland and to the collection of large datasets in the practice of teaching and learning.The article will also discuss some of the national initiatives created to address the challenges, and to support policy makers in making open and innovative education which fully embraces the digital era, a reality for Europe (EU Science Hub, n.d.).

Setting the Scene -Learning Analytics in Higher Education
The International Society for Learning Analytics Research (https://solaresearch.org)defines the study of learning analytics as "the measurement, collection, analysis and reporting of data about learners and their context for understanding and optimising learning and the environments in which it occurs" (Long & Siemens, 2011).Evidence points to learning analytics being used as a tool to 1) contribute to quality assurance and quality improvement of programmes, 2) boost retention rates, 3) assess and act upon differential outcomes among the student population and 4) enable the development and introduction of personalised or adaptive learning (Sclater, Peasgood & Mullan, 2016).Personalised learning is intended to provide a unique highly focused learning path for each student, with lessons, feedback, and assessments tailored to the student's strengths and weakness (EDUCAUSE, 2015).This level of attention and support is not viable in a traditional learning environment, but smart technologies can however allow universities to collate data from online and blended courses to reveal how student actions contribute to their progress and specific learning gains (Johnson et al., 2016).Insofar as learning analytics refers to the use of learner data and assessment results to respond to students' learning needs, it is used in every institution in Ireland by proactive individuals or in isolated pockets around campuses, but scalability is an issue (O'Farrell, 2017).
It is standard for institutions of Higher Education to have a robust technological infrastructure, student information systems for management of student admissions and information, financial systems, timetabling systems, survey software and a VLE.By utilising the increased availability of big datasets around learner activity and digital footprints left by student activity in learning environments, learning analytics using modern technologies can take us further than data currently available can (Sclater, Peasgood & Mullan, 2016).Internationally, countries such as Denmark, the Netherlands, and Norway are beginning to develop national approaches to support and enable endeavours in learning analytics.However, as evidenced in a recent report by the National Forum, Irish institutions are slow to purchase and implement learning analytics on a strategic level.Of around 25 Irish institutions, only six institutions have implemented learning analytics systems (O'Farrell, 2017).
The current work on LA in higher education focuses on the supply sidethe development of tools, data, models and prototypeswhile there is considerably less work on the demand side: how analytics connect with education and the changes institutions need to support their everyday learning, teaching and assessment work (Vuorikari & Castano Munoz, 2016).This is understandable as learning analytics is concerned with multi-level processes.It is the collection, measurement, analysis and reporting of data about learners in different contexts.Success with learning analytics begins with an understanding of how students learn (Vuorikari & Castano Munoz, 2016) and in addition, the mining of clean, high-quality data sets from multiple systems including the VLE.

The #VLEIreland Project
The #VLE Ireland project has been running since 2008, and its dataset amounts to some 24,000 student responses (Ryan & Risquez, 2018).In 2016 a survey instrument was developed and circulated on how lecturers use the VLE and technology enhanced learning (Farrelly, Raftery & Harding, 2018;Harding, 2018).The combined results help shed light on how VLEs are being used by students and lecturers for teaching and learning, pointing to the quality of data sets available for analysis.Hence, these findings help to identify some of the challenges to the transformation of learning with technology enhanced learning, and to the implementation of learning analytics system and processes.

Findings
Technology enhanced learning has the capacity to enhance the overall learning experience, promote collaborative behaviours and produce good learning outcomes (Risquez et al, 2013).It has proven to facilitate more active, problem-based learning which has been demonstrated to encourage greater student engagement and leads to better learning outcomes (Biggs, 2012), yet the datasets on learning are not available in the institutions to warrant the purchase of expensive and complex learning analytics systems.
The students and lecturers are the main contributors to the data collected by the VLE within an institution.Findings from the #VLEIreland project confirm that VLEs are not yet being used to their full potential within institutions.Overall, students find the VLE helpful and easy to use: the number of students accessing the VLE a few times a week has grown from 77% in 2008/09 to 94% of students in 2015 (Ryan & Risquez, 2018).The submission of assignments via the VLE has grown from 24% in 2008/9 to 47% in 2015.But this type of use is associated with administration more than innovative teaching and learning.Students indicate that barriers to their usage related to the lecturers not using it at all or the way they are using it.VLEs in higher education in Ireland are primarily used to facilitate ground-level pedagogical functions related to access and delivery of content, management of class administration and communications (Ryan & Risquez, 2018).Students identified 'a lack of use' by higher education teachers (24%) as the most common challenge and they clearly would like their lecturers to make more effective use of the VLE in support of learning (66% in 2015).
In 2015, 18% of lecturers responded to a staff survey on the VLE and technology enhanced learning.80% admitted to using the VLE to distribute notes and content, while less than 30% of lecturers admitted to 'not using the online assessment or discussion tools'.Less than 10% 'knew how to create podcasts or had knowledge of the software to develop resources for students'.Additionally, 15% of lecturers indicated they would like more access to support and guidance on developing resources and using technology in their teaching.
The #VLEIreland project shows that 60% of lecturers have attended workshops on using the VLE but when asked what prevents them getting the best use of it, replies included not knowing how to use the VLE to its full potential.The majority of VLE workshops within the institutions address both pedagogical and technical knowledge, but design and development require practice to increase capacity.Lecturers admitted to not having the time to dedicate to learning these new skills.As a consequence of the greater incentive to focus on research over teaching, there is less value placed on teaching and evidence of scepticism about the increased use of technology in a teaching context (Gregory & Lodge, 2015).Success with analytics does not begin with a set of data; it begins with an understanding of how people learn.The way in which a lecturer designs, develop and delivers technology enhanced learning within a VLE is one of the key influencers of the quantity and quality of student data available for analytics.

Discussion
Changing the delivery of a traditional module to a technology enhanced module requires a redesign of teaching practice.There are models of this type of redesign for technology enhanced learning, but they are primarily popular amongst the early adopters.One such model is the SAMR model (Puentedura, 2018).This model speaks to the different ways lecturers can explain their use of technology in the re-design of their practice: 1) Substitution, where the online component mimics how a lecturer teaches in a traditional setting; 2) Augmentation where there is some change for the better in how learning is supported, both the substitution and augmentation approaches can serve to enhance the learning experience for the students without a full redesign; 3) Modification and redefinition: these involve the lecturers incorporating new activities and approaches in their practice using technology leading to a full transformation in how the students learn.
As with a face-to-face lecture, a well-designed online module / activity requires constructive alignment between learning outcomes, quality content, learning activities and assessment.
Lecturers can use the VLE in different ways, they can substitute their core teaching practice using the VLE or they can make the best use of it, they can modify or redefine practice guiding the students to a more independent and collaborative learning environment and improve the learning experience.
TEL and the VLE allow a lecturer to intervene and incorporate personalised learning when students are experiencing difficulties.There is a need for novel, innovative pedagogy (theorised approaches to teaching and learning) that drives innovation and makes use of data to solve practical problems (Vuorikari & Castano Munoz, 2018).One approach is for a lecturer to monitor or track the students' learning and activity in a VLE.The lecturer can monitor how students are performing, and identify the concepts which, may be causing difficulty.This allows a lecturer to intervene by incorporating personalised learning design in the form of specific learning materials or activities to address the difficult concepts.Another example of a personalised learning intervention is a tool named CLARA which, is being used in the University of Technology, Sydney.The tool makes students aware of their learning dispositions (the habits of mind they bring to their learning) and generates 'learning power' profile visualisations for each student, as well as interventions based on their profile (https://utscic.edu.au/tools/clara/).These are but two examples of how TEL can be used to provide students with innovative personalised feedback and will produce the datasets around learning that is needed for LA systems.
Considering most of Irish universities have had futuristic and modern VLE for over a decade, along with models of online education, MOOCs, mobile learning, online discussions and collaboration tools, one might predict institutions should be perfectly placed to implement learning analytics technologies and predictive analytics frameworks.The findings from #VLEIreland suggest a different narrative.Technology enhanced learning encompassing learning analytics has the potential to transform teaching practice and learning but the way in which the technologies are being used to support learning within the institutions is preventing Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 the transformative impact across the sector.It is understandable that technology solutions are outpacing large-scale implementations and outcome studies (Johnson et al., 2017).

Challenges
There remain a number of challenges and barriers to be addressed before LA can be used effectively in higher education.The New Media Consortium (NMC) Horizon Report (2017) confirms the continuous presence of "significant challenges that impede on technology adoption in blended and informal education" (Johnson et al., 2017).The NMC categorises these challenges into three groups: solvable challenges, difficult challenges and wicked challenges, (those that are less well-understood).Challenges such as 1) improving digital literacy (JISC, 2015), i.e. necessary 21 st century practices that are vital for success in the workplace and beyond and 2) integrating formal and informal learning which, experts believe can foster experimentation, curiosity, and creativity are considered solvable.Another solvable challenge is a lack of strong pedagogical design in blended modules (Johnson et al., 2017) with lecturers currently using online platforms as repositories for notes and slides.These challenges are directly associated with how lecturers use technology in their teaching practice; suggesting institutions can address these issues through strategy, implementation of policy, processes and the provision of the relevant supports for the academic community.
A more complex challenge for technology-enhanced learning is the presence of competing models of education in formal institutions.This refers to the proposition that universities are being challenged by the premise that the more traditional methods of teaching and learning, e.g. a face-to-face lecture, may not be as appealing to prospective students.Students expect and often prefer more flexible access to their learning via the web.Technology enhanced learning has the potential to support a more effective and personalised learning experience for students.Not disregarding the possible transformative impact on education, the Internet has created a paradigm shift, but we are still in the process of absorbing and applying the implications (Bates, 2015) particularly in its use to support personalised and adaptive learning.
Wicked challenges refer to those that are difficult to define let alone to fix (Johnson et al., 2017) namely the rate of change in technologies, the lack of processes to address a change to a new technology, and the silent challenge, the academic workload (Gregory & Lodge, 2015).
With their teaching commitments lecturers have research priorities that leave little time for designing technology enhanced learning strategies and increasing digital capacity.If technologies are not being used effectively in the redesign of the learning experience, there is a challenge with gathering good and useful evidence and analytics.Resourcing and policy will address many of the solvable challenges by building digital capacity and providing more support for lecturers, while the more complex barriers require more complex solutions.

National Forum for the Enhancement of Teaching and Learning
Ireland's National Forum for the Enhancement of Teaching and Learning consulted with technology-enhanced learning personnel from higher education institutions to identify the national barriers to embedding of technology enhanced learning, including LA, in higher education in Ireland.
To address the barriers nationally, the National Forum then produced a comprehensive roadmap to build digital capacity in higher education in Ireland (National Forum, 2015).This strategy recommends that the institutions 'build on foundations' acquired over the last decade, promote a culture of innovation in digital teaching and learning, and implement platforms, resources and technologies such as learning analytics.This strategy along with quality blended module design and development will support staff and students in the acquisition of digital literacy and skills which will change culture and bring about the innovation needed.
The appetite for innovation using technology enhanced learning amongst lecturers remains high.With financial support from the Forum, collaborative partners from the institutions have produced a cohort of exciting innovation collaborative projects in e-assessment, MOOCs, digital skills and literacy proving that with adequate resources and practical support from educational technologists, lecturers will engage with the technology enhanced learning agenda to innovate within their teaching practice.
Looking to the future, the National Forum has launched a report titled Using Learning Analytics to Support the Enhancement of Teaching and Learning in Higher Education (O'Farrell, 2017).The project has produced a report which takes an informed approach to LA and promotes the importance of LA in Irish higher education.Additional to the report is the Data-Enabled Student Success Initiative (DESSI), which was established to consolidate the sector's investment in LA to date, and to work with institutions to ensure the future of LA in Irish higher education (https://www.teachingandlearning.ie/learning-analytics-educationaldata-mining-learning-impact/data-enabled-student-success-initiative-dessi/).
In support of the digital roadmap, all third level institutions in Ireland have implemented a VLE along with professional development programmes for lecturers to combine technology enhanced learning with the best of established practice.Unfortunately, the level of practical support required from skilled instructional designers and educational technologists is not always available within the institutions.Most institutions have very small technology enhanced learning teams to support lecturers in the design and redesign of module for online delivery.
Developing meaningful learning experiences for online delivery requires redesign of activities and of modules, in line with relevant quality assurance for technology enhanced learning.A recent survey carried out by the National Forum shows that institutions do not have separate quality assurance processes in place for digitally-enhanced or online courses, holding that the same standards apply to digital learning as those used in face-to-face learning (O'Rourke, 2017).The concept of quality in online learning is as complex as the reality of online learning itself (Butcher & Wilson-Strydom, n.d.).What then constitutes quality in online learning?A quality approved face-to face course will not necessarily be a good online course if the design is replicated in an online environment.Similarly, a good lecturer will not automatically become a good online instructor, hence the need for experienced educational technologists to support the lecturers.The VLE has proven it can offer the practitioner greater capacity to create exciting and meaningful online learning experiences, but this success is dependent on quality assurance in the design, development and delivery processes for technology enhanced learning.

Quality and Qualifications Ireland (QQI)
Quality criteria for science and research are widely accepted and have a quantifiable expression, but the situation is more complicated for the quality of education.A lack of historical evidence in technology enhanced learning (TEL) adds to this complexity; the result has been the presence of competing models of education in formal institutions.Fortunately, the need to address the gap in quality assurance in TEL is recognised and is currently being addressed by the Quality and Qualifications Ireland (QQI).This body has published Statutory Quality Assurance Guidelines for Blended Learning (2018).The guidelines provide guidance on how high-quality TEL opportunities and excellent campus-based learning should be combined in a coherent, reflective and innovative ways so that learning is enhanced, and choice is increased (QQI, 2018).The document has a legal and policy context and applies to all higher-level institutions in Ireland.The guidelines were produced in consultation with the TEL community in higher education.They address the requirements for assuring quality in the design, development and evaluation of programmes and modules in and out of the VLE that include blended learning and in doing so, remove the complexity around the quality assurance process.
The implementation of the QQI guidelines should formalise the actions institutions need take to promote transformation in teaching and learning using technology enhanced learning, leading to the collection of meaningful datasets on student learning and valuable pedagogy.This will place Irish universities in a stronger position to implement complex analytics systems in the context of the current research and recommendations on learning analytics for Europe.

Conclusion
To reboot learning for the digital age (Davies et al., 2017) in Ireland, we need to 1) build the effective use of technology for learning and teaching into the curriculum, 2) continue to develop an evidence and knowledge base on what works in TEL to help universities make informed decisions, 3) adopt learning analytics platforms and processes and 4) employ researchers to consider how learning analytics big dataset can be harnessed to provide new insights into teaching and learning.There remains a number of challenges to be addressed before the adoption of learning analytics platforms, frameworks and processes.These challenges are related to how lecturers within the institutions are using the VLE to support student learning, resulting in the unavailability of large datasets around student learning.This can be addressed through the more active engagement of lecturers with technology enhanced learning to support personalised and adaptive learning for students.
There is strong national evidence around what is required to address the challenges regarding technology enhanced learning and the implementation of LA at tertiary level in Ireland.
There has been good progress in terms of publications from the National Forum for the Enhancement of Teaching and Learning and the Quality Qualification of Ireland.The solutions though apparent are costly.Robust technological infrastructure, effective use of the infrastructure, policy, procedures and processes on design, development and delivery of TEL, data protection, costing models, support for the academic community are recognised as pivotal to ensure personalised learning experiences for students leading to a large quantity of quality datasets on student learning.