Game design for visually-impaired individuals : Creativity and innovation theories and sensory substitution devices influence on virtual and physical navigation skills

This action research study examined the design elements of three VR games that used an HTC VIVE VR helmet, two HTC game controllers, and a VR horse simulator for functionality and transferability to orientation and mobility (O&M) education for visually impaired individuals. The functionality of the VR games was tested with a visually-impaired individual based upon five characteristics that are important to O&M education: perimeter scanning then grid scanning, hearing, touch, smell, and perceptions of body positions. The horse simulator has potential benefits for proprioception and kinesthesia development. The inconsistent haptic feedback requires redesign consideration for inclusion within VR games and systems for O&M education of visually impaired individuals. * Corresponding author. Email: rose.baker@unt.edu Irish Journal of Technology Enhanced Learning Ireland, 2018 © 2018 R. M. Baker, K. Ramos & J. R. Turner. The Irish Journal of Technology Enhanced Learning Ireland is the journal of the Irish Learning Technology Association, an Irish-based professional and scholarly society and membership organisation. (CRO# 520231) http://www.ilta.ie/ . This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Irish Journal of Technology Enhanced Learning


Introduction
Sensory cues from the environment have assisted visually impaired individuals with orientation and mobility (O&M), often through echolocation, also referred to as facial vision, and other prompts for navigation.Education and entertainment games have helped visually impaired learners practice O&M in a safe-environment, reinforce O&M competencies, and learn new skills for finding their way in the physical world (Connors, Yazzolino, Sánchez, & Merabet, 2013;Lange et al., 2012;Merabet, Conners, Haiko, & Sánchez, 2012;Spence & Feng, 2010).To develop skills for O&M and to serve as preplanning aids for building and community navigation, game-based verbal cues have activated spatial mental imagery for the visually impaired players to help them to complete tasks within 2D and 3D games (Connors et al., 2013).Mental images were transformed into patterns to guide motion of either the full body in a physical space or controller-actions to manage movement within the game to build a cognitive map of the environment (Connors et al., 2013).After playing a game for preplanning navigation in a building, increased mobility was observed in the physical space represented by the game's virtual space (Conners et al., 2013).In situ support for O&M through the use of Sunu bands (Sunu, 2017), EyeMusic SSD (Abboud, Hanassy, Levy-Tzedek, Maidenbaum, & Amedi, 2014), or other similar feedback devices, in combination with canes or other technologies have assisted visually impaired individuals to gather spatial information when navigating.Many educational and entertainment games have been developed and adopted; however, future VR game design can be informed by assessing what works and what does not work within games that are played by visually impaired game players.Use of sensory substitution devices, adaptations for individual differences, and attributes that are most effective for visually impaired players can help form future O&M educational VR game designs.

Sensory Substitution Devices
In addition to verbal cues within O&M games, developers have implemented the translation of visual information into tactile sensations through haptic feedback systems (Lahav & Mioduser, 2008).The programming relies on three elements (a) a sensor to capture the specific form of energy, (b) a transformation algorithm to transcode the captured information into auditory or tactile patterns, and (c) a stimulator to transfer the captured information in its new form to the user (Elli, Benetti, & Collignon, 2014).Techniques have included perceptual supplementation (Lenay, Gapenne, Hanneton, Marque, & Genouëlle, 2003) and sensorimotor extension (Aurvay & Myin, 2008) in a cross-modal approach by artificially providing information to another sense, such as hearing, touch, smell, or temperature sensing.Virtual sensory substitution devices do not have to be expensive.Tools such as EyeMusic SSD resulted in successful navigation of the complex virtual world by walking down a street or moving about the inside of virtual houses, around trees, and across crosswalks (Maidenbaum, Buchs, Abboud, Lavi-Rotbain, & Amedi, 2016).
The use of haptic feedback has advanced traditional virtual reality (VR) and gameprogramming from primarily visual and auditory feedback to a system that provides tactile sensations to the user as vibrations of various durations and intensities through an actuator.A Canetroller has been developed to simulate white cane interactions with the environment by transferring feedback from a virtual environment to a wearable programmable brake mechanism, vibrotactile receiver, and spatial 3D auditory system (Zhao et al., 2018).Systems such as Blindaid have been used to help visually impaired individuals to explore and learn about new environments by providing haptic feedback and stereo sound relative to the avatar's position in a virtual environment replicating the area to be explored (Schloerb, Lahav, Destoge, & Srinivasan, 2010).

Individual Differences
Drawing upon the individual characteristics of creativity, such as cognitive abilities and style, intrinsic motivation, and knowledge (Woodman, Sawyer, & Griffin, 1993), and the individual's innovation framework, that is their level of confidence, dominance, growth need strength, internal work motivation, creativity and innovativeness, previous success of innovation, and task specific skills (West & Farr, 1989), game designers can incorporate sensory substitution devices to reinforce or support the level of creativity and innovation of the individual within the programming to help facilitate the individual's success within the game experience (Seifi & MacLean, 2017).Instructional and game designers can apply new technological tools into game design in a manner that is adaptable and personalized to help those with visual impairments and various levels of creativity and innovation to operate within the complex organization of the workplace or navigate the physical world (Bateman et al., 2018, Panëels, Ritsos, Rodgers, & Roberts, 2013).Those who are newly blind due to disease or injury have a perception of spatial relationships and potential obstacles within an environment (Dulin & Hatwell, 2006); they have experienced the visual input of stepping off a curb or crossing a street.Those who are congenitally blind or impaired at a young age would develop these perceptions using their own spatial awareness and experiences; much of this is dependent upon their creativity, innovativeness, and other mental and cognitive skills (Maidenbaum et al., 2016;Picinali, Afonso, Denis, & Katz, 2014).Each of these spatial awareness perspectives can be supported by the VR design to help the visually impaired individual to develop O&M skills within the game environment such that the game adapts to the creativity and innovation level of the player to provide an appropriate environment to meet the needs and expectations of the user.

Study Purpose
Work has been completed and studies are underway to inform the development of virtual environments to use as education or entertainment (Lange et al., 2012;Maidenbaum et al., 2015;Merabet et al., 2012;Sánchez et al., 2013).Educational game design decisions in 2D and 3D realms have considered the behavioral, cognitive, and constructivist objectives and goals for the learner and incorporate these within the elements and practice within the game environment (Elli, et al., 2014;Seifi & MacLean, 2017;Sorgini, et al., 2018).Entertainment factors have been directed by decisions about the rules, goals, and challenges planned within the game to control the levels of competence, autonomy, and social relatedness required.Many of the advanced technology solutions that have been used to supplement education for the visually impaired have been used within the technology laboratory and not adopted for education of the broader public (Cuturi, Aggius-Vella, Campus, Parmiggianai, & Gori, 2016).
The purpose of this study was to determine what works well and what does not work as well in current VR game design in order to inform design decisions for development of VR games for visually impaired individuals.These game design recommendations could be used for O&M education or other education-based VR games for the visually impaired.In this present study, we evaluated three VR games to determine elements of VR game design that do not work well to support future O&M education within a VR game environment.Those aspects that are supportive of success and possible inclusion in O&M education will be evaluated in future research studies.Individual attributes noted about the research subject helped the researchers to evaluate the comments made during game play and her responses to the study questions within the research design.Inclusion of the potential bias during self-reflexivity about the responses and coding of the themes relative to O&M characteristics guided conclusions about what worked and what did not work within the games.

Methods
This research project was conducted using a single-case participatory action research (PAR) design with one visually impaired game player and two observers to examine the elements of three games designed by NetDragon for use with the HTC VIVE VR helmet, two HTC game controllers, and a VR horse simulator.The games were: (a) VR骑射荣耀-Honor: Mount & Shoot, (b) VR南蛮入侵-Barbarian Breakout, and (c) VR火烧赤壁-The Battle of Chibi.The games and game system were designed to engage the player in shooting or combat-based challenges with the avatar riding a moving virtual horse.See Figure 1 for a photograph of the research participant using the VR helmet, controllers, and horse simulator and of the 90-inch computer monitor on which the researchers could observe the VR image depicted in the helmet.Each of the three games followed a set path within the programming and engaged a different weapon system.The first game, Honor: Mount and Shoot, required the game player to shoot a bow and arrow at targets while riding a horse.The second game, Barbarian Breakout, used a spear-type fighting device and required crossing a rickety bridge.The third game, The Battle of Chibi, was engaged using a swiping motion with a spear-like fighting device and holding a shield while riding a horse.At the end of the third game, the rider used a crossbow type device to launch explosives at a moving boat.Even though there were three games, the three had common functionality of riding a horse through obstacles and participating in hitting a target or engaging in combat.This single-case PAR design was selected in that description of aspects that worked well would be examined in future research by other visually-impaired users and those that did not work well could be marked for modification in O&M education environments and later follow-up testing.Motor and physical attribute development was expected within the games through the movement of the controllers as part of the game play, balance on the moving VR horse simulator, and activation of core body muscles.The game system was chosen as the environment to test for functionality with a visually-impaired individual based upon five characteristics that are important to O&M for visually impaired individuals: perimeter scanning then grid scanning, hearing, touch, smell, and perceptions of body positions.
A review of literature on the use of VR games for O&M education and game design decisions was conducted.The use of sensory substitution devices was of particular interest and noted within the literature.Questions were developed to assess the presence and functionality of sensory substitution devices within the three VR horse simulator enhanced games evaluated.In particular, the researchers discussed the design of the game, the usefulness of its features for entertainment or education of visually impaired users, aspects that were challenging to a visually impaired user, and elements that could be used as a basis for programming for future games to teach O&M skills to visually impaired users.Specifically, in this current study the following questions were asked at the conclusion and during replay of each game: The visually impaired participant explored her emotional experiences during her explanation of her experience.Her responses to the questions, outcomes of prior research, and observation notes taken during the game operation were coded and analyzed.Her introspective examination of her experience and personal meaning allowed her to be reflexive about her personal reaction, provided a springboard to understanding her experiences, and supported insight in the interpretation (Finlay, 2002).Together, the three researchers established agreement for aspects of the games that were useful and additions to the game design that would be necessary to enhance the experience of a visually impaired game player and to use the game for O&M education.This mutual collaboration reflexivity (Finlay, 2002) allowed for confrontation with and modification of the interpretations of the outcomes.

Findings
The literature, responses, and observations were coded into major themes of interest: audio feedback, haptic feedback within the controllers, and the movement of the horse simulator.Observed game features and recommendations for future O&M game design were considered as the visually impaired game player described her experience while she answered the five questions about her experience.A summary of these observations and recommendations are provided in Table 1 and are organized by the five O&M characteristics.The only audio feedback was a beeping sound from the computer when the game passed through fire or when the horse was jumping; no other sound was audible.The haptic feedback in the controllers was inconsistent and at times vibrated only in the right-hand controller.The movement of the horse simulator was a consistent speed until the avatar and virtual horse were jumping a ravine or climbing stairs; at those times it paused for a few seconds.The perceived most useful aspects of the games that could translate into O&M education were the movements of the saddle to simulate riding a horse.Often visually impaired individuals have difficulty with balance (Allen, Otto, & Hoffman, 2004).The manufacturers of horse simulators advertise the development of core muscles and strength to improve balance with use of the equipment (See advertisements for Tiger Leopard Ride VR simulator for example of exercise use of horse simulator devices).The horse simulator motion has the potential to help develop proprioception and kinesthesia to help the visually impaired become more aware of their body position and balance.
The haptic feedback could be enhanced from within a single controller to inclusion within the horse simulator unit and into the second controller.The programming to the single controller was inconsistently applied.Changes in the speed of the horse simulator for the various surfaces, vibrations with different levels of intensity and duration, and audio alerts, such as a helper sprite, would be necessary to help with O&M development.The current programming was confusing because the feedback was in conflict with the perceptions of the visually impaired game player.The functionality of the controllers was quite confusing to the player and, for one game, flipped the main controller such that the controller in the left hand was receiving the signals usually sent to the right hand.This might have occurred at the start of the game due to depression of the switch on the controller; however, the switch of controllers could not be replicated thus the cause is unknown.
Primary features that were missing were auditory feedback and a choice of level of difficulty.
Mobility training for visually impaired individuals consists of communication, motor development, concept development, sensory development, orientation skills, and mobility skills.The VR game lacked functionality to adequately address the key elements of O&M: perimeter scanning then grid scanning, hearing, touch, smell, and perceptions of body positions.The set path did not provide any scanning of the environment.A lack of audiobased, touch, and smell feedback limited the communication.As for body positions, the visually impaired player noted that the game had the potential to educate a user about balance and suggested the game might help with walking a straight line, standing on one foot, or completing repetitive tasks such as physical therapy exercises.In comparison to other O&M education, the VR game could employ similar technology as an optical device with a camera that was programmed to recognize objects and people; when pointed at the object or person, the audio feedback would tell the wearer what or who it was, and when pointed at a book or words, it would read the text.
The visually impaired participant described her visual impairment and its similarities and differences to others with visual impairments for the researchers to understand any bias related to her individual attributes related to vision, creativity, innovation, and game-play ability.Because of a lack of lenses (removed when she was a young child), depth perception is a challenge; she understands depth perception by using shadows.Her focal length was described to be able to count fingers at about two feet.Without sharp contrasts, she cannot see people until they are "right there," meaning almost immediately in front of her.As with others with visual impairments, she has to use a variety of senses for orientation in a new space and navigating spaces in which she has visited.As such, sounds, smells, temperature changes, and facial vision (often called echolocation) help her to know where she is in space and identify potential obstacles.
Within the games, she could see the sharp contrasts but could not see any of the details that were camouflaged or too close in color to other aspects within the game.She compared the game experience to other experiences and technologies, such as mobile warning devices (Mataro et al., 2017) and VR devices that calibrate to vision impairment and help to see details that cannot be seen with only corrective lenses in glasses (See online advertisements for Jordy devices).Her experiences with physical O&M training over the years were insightful to assess the aspects of the gaming environments for possible implementation using educational technology for O&M training and practice; something that is being driven by economic factors.The costs for one-to-one training and practice might be prohibitive for those without health insurance or other financial means to support the O&M education.Generalization of aspects of the game design that were useful and not useful to other visually impaired members and specific differences were addressed in the researcher's discussions about the experience.The recommendations within Table 1 for future design for O&M game designs are most likely helpful for members on the visual impairment spectrum who have limited vision rather than no or severe low vision.These recommendations support the implementation of adaptations within the game that are possible with advancements in educational and assistive technologies to address the individual differences of users and visual levels.

Discussion
It is expected to be more cost-efficient to build from existing programming than to build from new programming; thus, the determination of functionality in existing VR games would serve as a starting point for modification to O&M education for visually impaired members.Consistency in the feedback in an educational environment is necessary to not confuse the learner.The VR games evaluated applied inconsistent feedback through vibration alerts in the right-hand controller and the movement of the horse simulator.Future game design must develop the stimulant-response-feedback in a consistent manner to develop the behavioral and cognitive results desired.Sounds related to the types of surfaces that the avatar horse was on or 3D sound from the advancing attackers were non-existent; we determined that the sound could be supplied through a high-quality headset and should be included in future research and development for the games.A retailer describing other horse simulation machines stated that the rider could change speeds within their game by leaning forward to accelerate or leaning back to decelerate; however, this information was not available to us at the time of the game playing nor did we make any observation of speed changes with different body positions.
Due to the determined path through the VR games, even though the visually impaired game player could not see the approaching attackers, her game scores were above average with scores of B and a C within the game history; thus, our visually impaired game player scored higher than individuals with full sight capabilities.This is likely due to her curiosity and levels of creativity and innovation.From the experience, it was determined that individual differences related to creativity and innovation are essential elements for the design of the VR environment and there are opportunities to incorporate adaptation to those individual differences within the VR game programming.Not all visually impaired people will want to play a game to learn O&M; it is expected that those with a level of curiosity might be more interested in playing a game.Future research will include a measure of creativity and innovation as well as additional questions about the game playing experience.

Conclusion
The programming within the VR environment can serve as a starting point to develop future O&M education games for the visually impaired.Work is being completed to advance the technology; additional work is needed to include the educational technology and learning theories within the design of the games and systems.This research accomplished the goal to evaluate the design of the VR games, the usefulness of their features for entertainment or education of visually impaired users, aspects that were challenging to a visually impaired user, and elements that could be used as a basis for programming for future games to teach O&M skills to visually impaired users.
In spite of the limitations of using a single-subject PAR design and the potential for lack of generalizability, the use of the single subject allowed for documentation of what does not work within the game design for people with a visual impairment at the level or more severe than our research participant.The game system selected for evaluation is a second study limitation; the games were a convenience sample.The NetDragon system and its games were used due to availability in our academic department through the start of a research center to study game design, technology development, and neurodiversity (UNT, 2018).
Future research will be to conduct additional studies with different game systems and with other visually impaired individuals with different levels of visual acuity and motor skills.A self-assessment to measure creativity and innovation for each of these additional research participants will be used to assess VR game design decisions based upon individual differences.

Figure 1 .
Figure 1.Image of visually-impaired participant wearing the HTC VIVE VR helmet, holding two HTC game controllers, and strapped to a VR horse simulator and of the 90-inch monitor depicting the VR image for the researchers to see what was occurring in the game.
(a)  what would translate into O&M education?(b) what haptic feedback occurred and how did it influence any action or reaction?(c) how did the experience of riding the horse impact balance?(d) how did the functionality of the controllers impact physical movements?(e) what is missing that should be added to help with O&M?At the conclusion of the three games, the visually impaired player was asked (a) how does this experience compare to other experiences in O&M training and practice?and (b) what visual and O&M practices and strategies that are currently used impacted the playing of the games?

Table 1 .
Description of Observed Game Features and Recommendations for Future Design for O&M by O&M Characteristic