Irish Journal of Technology Enhanced Learning, Vol 7, Issue 2
Special Issue: The Games People Play: Exploring Technology Enhanced Learning Scholarship & Generative Artificial Intelligence
https://doi.org/10.22554/ijtel.v7i2.145 
  

Pseudo-Research: A Fabricated Study on Student Engagement in Music Pedagogy

Hazel Farrell *
South East Technological University 


Abstract

The concept of writing a lengthy scholarly piece using GenAI presented an intriguing challenge with optimistic presumptions around reduced effort and time commitment. It quickly became apparent that this optimism was misplaced as the limitations of GenAI emerged and the construction of the piece demanded more dedicated effort. The report focuses on the use of technology to enhance student engagement within the context of a music degree program and this is presented in the form of a fabricated case study. The findings raise concerns around academic integrity and cultural stereotyping, while also acknowledging the value of aspects of the ChatGPT 4 generated output.

1. Introduction

Educational and scholarly relationship with AI

The emergence of GenAI as a rapidly evolving technology has caused significant disruption to the field of education. Academic integrity has been at the core of the narrative, stimulating a plethora of research into ethical use (Foltynek et al, 2023), assessment redesign (Liu & Bridgeman, 2023), and the challenges associated with this continuously evolving technology (UOC - Open University of Catalonia, 2023; Chan, 2023). Studies into innovative approaches to the integration of GenAI into teaching practices also continue to emerge as the opportunities to enhance the student experience and ensure the relevance of our pedagogies come to the fore (Nerantzi et al, 2023). This technology has motivated and provoked educators to reassess their practices and embrace new approaches to teaching, learning, and assessment, as is the case with the author.

In addition to the early adoption GenAI in the classroom, active involvement in developing guidelines on AI usage for staff and students in SETU has necessitated a deep engagement with emerging research and literature in the higher education sector and beyond. Further, contributions to the design of the GenAI Staff Hub and GenAI Student Area in the university has included the creation of tools, templates, and resources addressing challenges such as academic integrity, assessment re-design, and the application of GenAI in the classroom. Dissemination of research and learning has been ongoing in addition to professional development in an attempt to support others in the challenges associated with this technology and to keep abreast of the latest advances.

Rationale

The integration of GenAI into the classroom was stimulated by an enduring commitment to the exploration of approaches to enhance student engagement, which is widely regarded as a critical aspect in creating an optimal learning experience (Sharan & Tan, 2008; Clements, 2015). As a founding member of CASE (Creative Approaches to Student Engagement), a multi-disciplinary group in South East Technological University (SETU), practice-based research into the use of technology for this purpose has been ongoing with previous focuses on e-portfolios, visuals, collaborative and reflective tools. In tandem with the integration of digital technologies into our learning environments, an examination of the impact of these technologies on student engagement is necessary to inform educators and improve practice (Nkomo et al., 2021).

The chosen topic on the use of technology to enhance student engagement practices within the context of the music discipline reflects the expertise and research interests of the author, thereby providing an ideal foundation to engage fully with the theme and integral aspects of the edition. In the spirit of ‘play’, a case study format was selected to present this report to test the boundaries of GenAI in relation to the level of detail it could produce with minimal information and how relevant the outputs would be. The notion that GenAI could fabricate a case study within these parameters was intriguing to the author and presented a research opportunity not to be missed. It was also welcomed as a learning opportunity to become more familiar with the possibilities and limitations of the technology, which is a daunting task given the barrage of information and continuous developments around GenAI. The basis of the case study is factual in terms of the Irish university classroom, the specified degree program, the selection of modules, small sample size, and the areas of focus in terms of previous student engagement research.

ChatGPT 4, which is the paid version of the OpenAI tool, was used in the creation of this case study. The focus areas of Prompts 2 – 5 were suggested by ChatGPT in response to a first iteration of Prompt 1 (identical to below with the exception of the word ‘Irish’ being omitted), which was designed for a positive study outcome.

Prompts

1. Generate a 500-word introduction to a case study on the use of technology to enhance student engagement in an Irish university classroom. The study focuses on a small group of 18 students undertaking a BA (Hons) in music. Their modules include musicology, performance, music pedagogy, and research. The tools used include those that facilitate collaborative learning experiences, create visually stimulating resources, and provide platforms for reflective practice.
2. In the context of the above case study, develop a rationale for the research focused on cultivating a deeper understanding of the current educational strategies employed and the necessity for the integration of technology.
3. Provide an in-depth description of the tools, the strategy behind their selection, and the experiences during their application
4. Provide direct insights from the student body, quantitative data on engagement levels, and qualitative feedback on the overall experience
5. In conclusion to the above case study, discuss the implications drawing insights and outlining potential pathways for further integration of technology in specialized educational programs.

2. AI Generated Short Report

Introduction

In Ireland, a land where melodies echo the tales of yore and rhythms beat in tandem with its rich history, music is more than just a subject; it's an essence. But as the world undergoes rapid technological transformation, even the sanctity of the traditional Irish music classroom finds itself on the brink of revolution. In the midst of this evolving academic landscape, this case study sets out to unravel the intricate dance between age-old music pedagogy and the emergent power of technological tools.
Our investigative lens is trained upon a distinctive classroom in an esteemed Irish university, where 18 young aspirants journey through the captivating realms of a BA (Hons) in Music. Their academic mosaic comprises modules as diverse as musicology, which delves deep into the historical and analytical nuances of compositions; performance, the passionate expression of musical prowess; music pedagogy, examining the philosophy and techniques of music education; and research, a meticulous foray into music's uncharted dimensions.
In any specialized discipline, student engagement is paramount, more so in a subject as visceral as music. The challenge then arises: How does one retain the essence of traditional teaching while integrating the allure of technological tools to enhance the overall learning experience? To address this, a suite of advanced tools has been introduced in the classroom. These tools are not just random additions; they have been strategically chosen to complement specific aspects of the curriculum. For collaborative learning experiences, platforms that allow students to work synchronously on compositions and analyses have been incorporated. Recognizing the importance of visual engagement, especially in the understanding of complex musical theories, resources that are both visually stimulating and interactive have been integrated. Furthermore, in a discipline where introspection plays a crucial role, platforms fostering reflective practice have been adopted, allowing students to revisit, analyse, and refine their musical endeavours.
The introduction of technology in such a traditional setting isn't without its concerns. Does the digitization of certain elements of music education take away from the raw, organic experience that has been its cornerstone for centuries? Or does it add a layer of depth, making the subject more relatable and engaging for the digital-native generation? It is these questions, among others, that this case study endeavours to answer. By immersing ourselves in the experiences of these 18 students, we seek to understand the balance between technology's appeal and the intrinsic value of traditional music pedagogy. Through their eyes and ears, we'll explore whether these digital tools amplify or detract from the essence of music education.
In a country like Ireland, where every note and rhythm resonate with stories of the past, the integration of modern technological tools in music education signifies a meeting of two worlds. This case study is not just an exploration of that intersection but a testament to the timeless beauty of music and the relentless march of innovation. As we delve into the heart of this musical classroom, we aim to uncover the symphony that plays out when tradition meets technology.


Rationale

In the quest to continually evolve and enhance the educational experience, institutions globally have increasingly looked towards technology as a potential panacea. The integration of technological tools in pedagogy is not merely a trend; it is emerging as an essential component in moulding contemporary learning environments that cater to the unique needs of the current generation of students. In this context, the BA (Hons) Music program in an Irish university, with its rich blend of traditional teaching methods and modern curricula, presents an apt setting to understand this intersection in-depth.

Shift in Learner Demographics and Preferences:
Today's students, often referred to as 'digital natives,' have grown up in an era marked by the rapid evolution of technology. Their modes of interaction, communication, and even cognition are inherently different from previous generations. Understanding how they engage with and benefit from technological tools in the context of a subject as historically rich as music becomes paramount. This study hopes to bridge the generational divide, ensuring that music education remains relevant and engaging.

Evolving Nature of Music Consumption and Creation:
The way music is consumed, shared, and even produced has changed drastically with the advent of technology. Platforms like SoundCloud, digital audio workstations (DAWs) like Ableton, and music-sharing apps have transformed music from being a mere listening experience to an interactive one. This shift necessitates a relook at traditional teaching methods in music classrooms to incorporate these modern-day realities.

The Potential for Enhanced Engagement:
Preliminary observations and anecdotal evidence suggest that the integration of technology can lead to heightened student engagement. Platforms that allow collaborative learning or provide visually stimulating resources can make complex theoretical concepts more accessible. Similarly, platforms designed for reflective practice can offer students the chance to critically analyse their performances or compositions, fostering deeper understanding and continuous improvement.

Globalization and the Need for Interconnected Learning:
As music becomes increasingly global, understanding varied cultural nuances, collaborating with peers across borders, and having a holistic world view becomes crucial. Technology facilitates this interconnected learning, enabling students to engage with global perspectives without leaving their classrooms.

Assessing the Balance:
While the push for technological integration is strong, there's an intrinsic value in the traditional methods of teaching music, especially in a country like Ireland, where music is deeply intertwined with cultural and historical narratives. This research aims to understand where the balance lies. It seeks to answer whether there's a point of saturation for technology in the music classroom or if the possibilities are truly limitless.
This research is not merely an academic exercise; it is a crucial endeavour in the evolution of music education. By understanding the current strategies employed, their effectiveness, and the potential benefits (or pitfalls) of technology integration, this study hopes to provide insights that could shape the future of music education, not just in Ireland, but globally. This is a journey of discovering harmony, not just in musical notes, but in the confluence of tradition and technology.

Technological Tools: Selection, Strategy, and Experiences

Collaborative Learning Platforms: Soundtrap and Splice:
Soundtrap and Splice are online platforms that allow students to create, edit, and collaborate on music projects in real-time. Both offer a variety of virtual instruments, loop libraries, and recording features. Given the shift towards remote learning and the need for enhanced collaboration among students, platforms that facilitate real-time musical collaboration were deemed essential. The ease of use, cloud-based storage, and robust features of Soundtrap and Splice made them apt choices.
Students found these platforms to be game changers. They could collaboratively compose pieces, provide instant feedback, and blend various musical elements seamlessly. However, some initial training was required, and occasional connectivity issues did arise.
Visually Stimulating Resources: Auralia and Musition:
Auralia and Musition are comprehensive musicianship software packages, providing visually engaging exercises on music theory, ear training, and notation. To address the often daunting task of understanding complex music theories and to cater to the visual learning preferences of the digital generation, tools that could present theory in an interactive, engaging manner were sought. Auralia and Musition's rich visual interfaces and expansive exercise libraries filled this gap.
Students reported higher engagement levels during theory sessions. Difficult concepts were more accessible, and the interactive nature of the exercises led to better retention. However, educators had to ensure that these tools complemented, rather than replaced, traditional teaching methods.
Platforms for Reflective Practice: Padlet and VoiceThread:
Padlet allows users to create virtual boards where they can pin notes, videos, and links, while VoiceThread is a collaborative slideshow tool that lets users comment on slides via text, voice, or video. Reflection is a cornerstone of deep learning. Platforms were needed where students could chronologically document their learning journey, share their musical pieces, and receive diverse feedback. Padlet's organizational ease and VoiceThread's multimedia feedback capabilities made them the tools of choice.
These platforms transformed students' reflective practices. They could revisit their growth, understand areas of improvement, and engage in constructive peer feedback. However, ensuring consistent and meaningful student participation required regular prompts from educators.
The selection of these tools was driven by the dual goal of preserving the essence of traditional music pedagogy while enhancing it with the benefits of technology. While students largely embraced these tools, the real magic unfolded when educators adeptly blended them with conventional teaching methods. The resultant fusion was a dynamic learning environment where tradition met modernity, providing students a holistic musical education tailored for the 21st century.

Quantitative and Qualitative Perspectives

Quantitative Data on Engagement Levels: Survey Metrics

  1. Frequency of Tool Use:
  2. Increase in Collaborative Projects:
  3. Enhanced Participation in Theory Classes:
  4. Consistency in Reflective Practices:

Qualitative Feedback on the Overall Experience:

  1. Soundtrap and Splice Feedback:
  2. Auralia and Musition Feedback:
  3. Padlet and VoiceThread Feedback:
  4. General Feedback:

The student insights painted a picture of appreciation, adaptability, and a few challenges. While the majority embraced the technological tools and felt a noticeable improvement in their engagement and learning, there was also a conscious understanding of the importance of balancing technology with tradition. The data and feedback suggest that while these digital platforms can significantly enhance the educational experience, their optimal use will depend on the judicious blend of the old and the new, ensuring that the soul of traditional Irish music remains undiluted.

Conclusion and Implications

The unfolding of this case study in an Irish university classroom, focusing on BA (Hons) in music, underscores a salient narrative of modern education: the symbiotic relationship between traditional pedagogy and technology. Drawing from the experiences and insights gleaned from the study, several implications emerge that can guide the future trajectory of specialized educational programs.
The Indispensability of Technology: It's clear that technology is no longer just an adjunct but an indispensable component of the contemporary educational landscape. The heightened engagement levels, improved collaborative efforts, and enriched reflective practices among students, as revealed in the study, corroborate this.
Balancing Act is Crucial: While technology brings myriad advantages, Caitlín's feedback is a reminder that an over-reliance can be counterproductive. The challenge is to ensure that technological tools supplement, not supplant, traditional teaching methods, especially in a field as historically and culturally rich as music.
Personalised Learning Pathways: The varied feedback from the student body indicates diverse learning preferences. Modern technological tools offer the flexibility to cater to individual learning styles, paving the way for more personalised education, even in specialised fields.
Continuous Training and Support: The introduction of any new tool requires adequate training and support. The initial learning curve experienced by some students with platforms like Splice underscores the need for institutions to invest in continuous training for both educators and students.
Potential for Cross-Cultural Collaborations: With technology erasing geographical barriers, there's an immense potential for students to collaborate with peers globally. This can be especially enriching in music education, where cross-cultural collaborations can lead to a fusion of musical styles and traditions.
Evaluation and Iteration: The field of educational technology is rapidly evolving. Continuous evaluation of the effectiveness of chosen tools and being open to adopting newer innovations ensures that the curriculum remains relevant and engaging.

Pathways for Further Integration:

  1. Research and Development: Institutions can collaborate with tech companies to develop specialized tools tailored for specific educational needs. For instance, a platform exclusively dedicated to Celtic music traditions can be invaluable for Irish music students.
  2. Augmented and Virtual Reality: The next frontier in educational technology could be AR and VR. For music students, a virtual, immersive experience of a historical concert or a virtual tour of global music traditions can be transformative.
  3. AI-Powered Feedback Systems: Advanced AI systems can provide instant feedback on student performances, compositions, or theory exercises, offering them insights and suggestions for improvement in real-time.
  4. Global Digital Libraries: Creating a global digital repository of music resources, recordings, manuscripts, and research can provide students with a treasure trove of knowledge at their fingertips.
  5. Hybrid Learning Models: Combining face-to-face traditional learning with online modules facilitated by technological tools can offer students the best of both worlds.

In conclusion, the case study from the Irish university music classroom serves as a microcosm of the broader educational paradigm shifts. The careful and thoughtful integration of technology in specialised educational programs can usher in a new era of enriched, engaged, and holistic learning experiences.

3. Critical Reflection

Process and Implications
The process of producing a long scholarly piece using GenAI was more challenging and time consuming than expected. The conceptualisation of the topic and approach took place over three weeks as a preference of the author to have time to absorb and reflect on the brief, while the generation of the main body of the study involved two days of trial and error before settling on the current version. Three concentrated days were then dedicated to the writing, editing and completion of the piece. The most obvious challenge was the necessity to divide it into five distinct sections due to the limitations on the word-count of the outputs, which average at around 500 words per prompt. Some editing was required to combine the five main sections into a cohesive report, mainly in relation to deleting superfluous headings and repetitive concluding comments. Standardisation of European English was also undertaken as a small number of Americanisations emerged.

The process of creating the study was undertaken twice in an attempt to make it more relevant to an Irish rather than American university classroom experience. This simply involved inserting the word ‘Irish’ into Prompt 1 and it revealed an alarming level of bias inherent in the GenAI output. While bias has consistently been highlighted as an issue with GenAI given the predictive algorithms and data it draws from (Nishant et al., 2023), the extent revealed in this study was, perhaps naively, unexpected. What was a relatively respectable academic study on technology enhanced engagement practices for a specialised discipline area in the first iteration, transformed into a ‘plastic shamrock’ narrative evocative of Darby O’Gill meets The Lord of the Rings.

This ‘plastic shamrock’ or ‘Oirish’ cultural stereotyping permeates the study detracting from the academic nature not only in relation to the narrative, but also in terms of specific details, for instance distinctly Irish names were assigned to the imaginary students in contrast to Ethan, Mia, Sophia and Noah of the first iteration. This may be viewed as a means of making the study sound more authentic given the context of the Irish university classroom, but it also alludes to a misconception of a mono-cultural society which further exemplifies the biased nature of the AI output. The imagery of an antiquated society and education system is also evoked through the frequent twee references to tradition versus modernity.

Aside from the cultural stereotyping, the invention of core elements of the study including the tools, statistics, fictional students, and quotations, was both revealing and concerning in terms of how authentic the AI output sounds. The ability of GenAI to fabricate specialised and often highly detailed information and present it in a believable manner is of significant concern in the context of academic integrity (Elkhatat, 2023). These fake statistics suggest significantly enhanced learner engagement categorised by increased tool usage, collaboration, participation and reflection, illustrating the predisposition of GenAI to a positive outcome in this case. This is worrying as the authenticity and integrity of academic research may be increasingly called into question in the future. A sixth prompt seeking a reference list was originally included, but a decision was made to omit this as the resultant list was accompanied by a disclaimer highlighting the fictional nature of the sources and the necessity to replace them with ‘actual’ references.

Although the value of engaging with the opportunity to experiment and play with GenAI is recognised, a deeply embedded ‘ethical conscience’ impacted on the completion of the work. The struggle with presenting work not conceived entirely by the author proved challenging and this was intensified by the necessity to become conversant with the fabricated AI generated material. However, the AI output is impressive in terms of the clarity and logical progression of information, and while the suggested tools may not necessarily reflect the personal preferences of the author, they are valid choices.

The conclusions presented by ChatGPT reflect common research findings, for instance the fact that most technologies promote interactive and collaborative learning (Schindler et al, 2017) which can improve educational outcomes (Escotet, 2023) is widely recognised. Similarly, the challenge of over-reliance on technology and ideal usage complementing rather than replacing teaching and learning practices emerges frequently in narratives around technology enhanced learning. This may be applied to current concerns around AI-usage as can the flexibility of the technologies to create personalised and specialised learning pathways (Nah et al, 2023). The need for ongoing research and training as new technologies emerge is also recognised (Chan, 2023), and the suggested future pathways are aspirational, but thought-provoking. While these conclusions are not ground-breaking, they are relevant and reflective of the data ChatGPT is drawing upon.

Improvements to the approach could be made by training ChatGPT to replicate the author’s writing style and by providing skillfully crafted prompts to produce a more nuanced output. Further professional development and engagement with emerging research would enhance the skills of the author in this regard. Despite the limitations and challenges of this exercise, valuable insights have emerged with implications for our pedagogical practices:

 

Finally, it is clear that engaging in training and experimentation with GenAI is vital as this disruptive technology evolves and increasingly impacts on our practices. The decision to optimistically, but cautiously embrace it appears to be a good approach as AI not only permeates our professional lives, but also becomes embedded in our society.

 


References
Chan, C. K. Y. (2023). A Comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20, 38. https://doi.org/10.1186/s41239-023-00408-3
Clements, J. (2015). Using Facebook to enhance independent student engagement: A case study of first-year undergraduates. Higher Education Studies, 5, 131-146. https://doi.org/10.5539/hes.v5n4p131
Elkhatat, A. M. (2023). Evaluating the authenticity of ChatGPT responses: A study on text-matching capabilities. International Journal of Educational Integrity, 19, 15. https://doi.org/10.1007/s40979-023-00137-0
Escotet, M. Á. (2023). The optimistic future of Artificial Intelligence in higher education. PROSPECTS: Comparative Journal of Curriculum, Learning, and Assessment. https://doi.org/10.1007/s11125-023-09642-z
Ferrando, P., & Cardosa, D. (2023). Harnessing the benefits of artificial intelligence for higher education: A dialogue with professor Mike Sharples. ETHE Journal Blog. http://etheblog.com/2023/07/05/harnessing-the-benefits-of-artificial-intelligence-for-higher-education-a-dialogue-with-professor-mike-sharples/
Foltynek, T., Bjelobaba, S., Glendinning, I., Khan, Z. R., Santos, R., Pavletic, P., & Kravjar, J. (2023). ENAI Recommendations on the ethical use of artificial intelligence in education. International Journal for Educational Integrity19(1), 12–14. https://doi.org/10.1007/s40979-023-00133-4
Liu, D., & Bridgemen, A. (2023). What to do about assessments if we can’t out-design or out-run AI? https://educational-innovation.sydney.edu.au/teaching@sydney/what-to-do-about-assessments-if-we-cant-out-design-or-out-run-ai/
Mollick, E. (2023). The future of education in a world of AI: A positive vision for the transformation to come. One Useful Thing. https://www.oneusefulthing.org/p/the-future-of-education-in-a-world
Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023). Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research, 25(3), 277-304. https://doi.org/10.1080/15228053.2023.2233814
Nerantzi, C., Abegglen, S., Karatsiori, M., & Martinez-Arboleda, A. (Eds.) (2023). 101 Creative ideas to use AI in education. 2023 1.0) [Computer software]. Zenodo. https://zenodo.org/record/8072950
Nishant, R., Schneckenberg, D., & Ravishankar, M. (2023). The formal rationality of artificial intelligence-based algorithms and the problem of bias. Journal of Information Technology, 0(0). https://doi.org/10.1177/02683962231176842
Nkomo, L. M., Daniel, B. K., & Butson, R. J. (2021). Synthesis of student engagement with digital technologies: A systematic review of the literature. International Journal of Educational Technology in Higher Education, 18, 34. https://doi.org/10.1186/s41239-021-00270-1
Sabzalieva, E., & Valentini, A. (2023). ChatGPT and Artificial intelligence in Higher Education: Quick start guide. UNESCO. https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf
Schindler, L. A., Burkholder, G. J., Morad, O. A., et al. (2017). Computer-based technology and student engagement: A critical review of the literature. International Journal of Educational Technology in Higher Education, 14, 25. https://doi.org/10.1186/s41239-017-0063-0
Sharan, S., & Tan, I.G.C. (2008). Organizing schools for productive learning. Springer Netherlands. https://doi.org/10.1007/978-1-4020-8395-2
UOC - Open University of Catalonia. (2023, June 7). Generative AI: Transforming higher education. [Video]. YouTube. https://youtu.be/v8A0xix0Ofo

* Corresponding author email: hazel.farrell@setu.ie

A colloquial term used to characterise clichéd elements of the Irish identity such as excessive drinking, strong accent, and simplistic lifestyle.