Open Science and Educational Research: An Editorial Commentary

Educational technology as a broad and applied interdisciplinary research field faces challenges in achieving consensus on what constitutes good quality research. As the field is embedded in many other disciplines, considering what evidence matters and the optimal methodologies to conduct inquiry is continually evolving and maturing. Inhabiting a boundary between education, and computer science, and viewed through numerous theoretical lenses ranging from disciplines of sociology, politics, psychology, the learning systems, curriculum development, digital humanities, and beyond, the number of approaches contributing to the field is vast. The validity, trustworthiness and integrity of over two decades of research in this domain is continually questioned. Furthermore, as technology itself also changes, there are differing opinions on how best to explore and understand the role it plays in education. How we define, research and evaluate our evidence is central to our understanding of how we learn and how this is enhanced with and through technology in various ways. Whilst scholars continue to critique and debate the veracity of findings, educational technology journals play an important role in allowing us to collectively peer review, and publish the best quality research studies. Changes in the open access publishing world and in the open science movement have the potential to address some of shortfalls in how our understandings are evaluated, critiqued and judged in this domain.


Introduction
Educational technology as a broad and applied interdisciplinary research field faces challenges in achieving consensus on what constitutes good quality research. As the field is embedded in many other disciplines, considering what evidence matters and the optimal methodologies to conduct inquiry is continually evolving and maturing. Inhabiting a boundary between education, and computer science, and viewed through numerous theoretical lenses ranging from disciplines of sociology, politics, psychology, the learning systems, curriculum development, digital humanities, and beyond, the number of approaches contributing to the field is vast. The validity, trustworthiness and integrity of over two decades of research in this domain is continually questioned. Furthermore, as technology itself also changes, there are differing opinions on how best to explore and understand the role it plays in education. How we define, research and evaluate our evidence is central to our understanding of how we learn and how this is enhanced with and through technology in various ways. Whilst scholars continue to critique and debate the veracity of findings, educational technology journals play an important role in allowing us to collectively peer review, and publish the best quality research studies.
Changes in the open access publishing world and in the open science movement have the potential to address some of shortfalls in how our understandings are evaluated, critiqued and judged in this domain.
This concern around research quality isn't confined to this domain. Related fields of psychology have similarly encountered a reproducibility crisis (Baker, 2016). Researchers have identified questionable practices, such as failing to report all dependent measures, selectively reporting studies that "worked", or claiming to have predicted unexpected findings. (John, Loewenstein, & Prelec, 2012).
Some of the causes of these practices include the perverse incentives in academic work, where publish or perish holds strong, and new faculty positions and tenure are directly linked to a researchers publication rate. Along with promotions and tenure being linked to publication records, commercial journal publishers benefit from free labour, which in turn creates a culture where incentives for publishing research findings with high statistical significance and impact rather than also failures and non-significant results (Dobele & Rundle-Theile, 2015;Santos & Horta, 2018). Some have taken an even stronger position, suggesting that the result of this culture of research has resulted in a situation where most published research findings are false, relying on low statistical power, poor control for bias and questionable research practices (Ioannidis, 2005).

A look at the characteristics of Open Science and its values
In the last twenty years, there have been key developments based on Open Science and Open Scholarship, and a resurgence in open research practices. The Budapest Open Access Initiative (BOAI, 2002) was a pivotal moment in the definition of openness, and a call to action for openness of all publicly funded work. Open access differs from subscription-based publication, where articles are published but inaccessible due to copyright, licencing and paywall constraints.
Openness in academic publishing has been interpreted in various ways, such as where scholars pay the publisher to keep the work open, or publish pre-prints in their institutional repository. However, fully free and open access journals such as the Irish Journal of Technology Enhanced Learning exist, and are funded through grassroots organisations such as the Irish Learning Technology Association (ILTA), where publication fees are sustained through registration fees to the Irish EdTech community annual conference. The journal was this year indexed in the Directory of Open Access Journals (DOAJ). As a "diamond open access" journal (Fuchs & Sandoval, 2013) (Fuchs & Sandoval, 2013) authors are enabled to publish scholarly articles in a free and open access journal, without charging them fees. Through peer review, publicly funded scholars giving their time and expertise. Via the support of the Irish Edtech community, hosting and associated publishing costs are met, so that authors retain copyright and license their work through a permissive creative commons license (CC-BY 4.0), without any lower publishing standards. Furthermore, the openness helps authors amplify their impact.
More recently, a global movement to make research and educational practices more transparent, under the Open Science umbrella term, promisingly, include a set of practices and principles that have extended into all aspects of the research cycle. These have the potential to address some of the challenges and problems noted above that are fundamental to teaching and learning research quality/success, and will be considered next.

Irish Journal of Technology Enhanced Learning
Vol 4, Issue 1 iv Changing the culture A potential solution arising from Open Science, with direct application to educational research is to instil scientific rigor at the grassroots level. Working collaboratively across institutions can help us to achieve better quality research outputs (current and future) and help us to increase power, generalisability and transparency of research data. Practical research practices, such as preregistration, open data, open resources, are central to this change. Collectively, written protocol & apriori statistical analysis plans can be published on the Open Science Framework before data collection starts, to improve the efficiency and access to research, and resultant outputs. Additional solutions proposed include protocol and analysis plan pre-registration and FAIR or CONSORT reporting guidelines, and consortia pooling resources to maximise sample size and large scale collaborations. Furthermore, systematic reviews encourage transparent and clear reporting of data and methods. Others have argued for more equity and partnerships in the process of educational research including a clearer definition of roles, including practitioners, researchers, and funders to promote larger scale research findings, with participatory design goals, such as those proposed by the Design Based Implementation Research (Penuel, 2019). It is clear that there are immediate improvements from the Open Science movement in how we support and carry out our research, that we later publish openly as peer-reviewed research work, as free to read, redistribute and reuse, with some restrictions as a journal article.

Continuing challenges/conclusion
The intention of open access journals such as the Irish Journal of Technology Enhanced Learning is to negate claims made that are exaggerated, unsubstantiated, contradictory, and even untrustworthy. Diverse methodologies, and a critical analysis will continue to be a feature of research in this meta domain, but more transparency and openness into all aspects of the research process, including the published outputs will be a very positive direction for the future. The challenge now is for us to work together towards creating a shared understanding of what works and doesn't in the field of educational technology and how practices from the Open Science movement might become a supported and defined part of our educational research practices, resulting in better quality published research findings that are open and accessible to all.