Eelke Folmer
UNVR Lab
University of Nevada, Reno

Blind Hero: Guitar Hero for Users with Visual Impairments

Guitar Hero is a series of music video games in which players use a guitar-shaped controller to simulate the playing of guitar across numerous rock music songs. The gameplay of Guitar Hero relies on pattern matching: players match visual notes indicated by different colors that scroll on-screen to colored fret buttons on the controller and have to strum the controller in time to the music in order to score points. This game is inaccessible to users who are visually impaired as they cannot perceive what visual cues are provided. In this game music is used to facilitate pattern matching. The use of audio for sensory substitution of visual cues is complicated as this interferes with the player hearing the music.

How it works

This research project explored the novel use of haptic feedback in sensory substitution to facilitate a pattern matching game. We developed a haptic glove that has little pager engines attached to each finger which are activated using a USB controller. We modified an open source version of guitar hero called frets on fire, such as to provide output using this haptic glove. Several tradeoffs were made to play the game using the haptic glove. We removed the lookahead, e.g., the ability to see which cues are coming up and instead the game is played using a "whac-a-mole like gameplay", i.e., The user presses the button the finger is on which receives the cue. Instead of using 5 inputs, we removed the thumb input as this is typically provided using the fourth finger.

Media

The custom glove we developed with a pager engine attached to each finger
The custom glove we developed with a pager engine attached to each finger (excluding the thumb).

Publications

Bei Yuan and Eelke Folmer. Blind hero: Enabling guitar hero for the visually impaired.In Proceedings of the 10th international ACM SIGACCESS conference on Computers and Accessibility, pages 169-176, Halifax, Nova Scotia, Canada, October 2008. [37% acceptance rate]