Eelke Folmer
Human+ Lab
University of Nevada, Reno

TextSL: A Command-Based Virtual World Interface for the Visually Impaired

Virtual worlds -and 3D environments in general- are notoriously inaccessible to users with visual impairments as they lack any textual representation that can be read with assistive technology such as a screen reader or tactile display. Virtual worlds are significantly different from 3D games, e.g., virtual worlds lack combat, virtual worlds are often entirely driven by user generated content; and players spend large amount of time socializing with other players. Though several audio interfaces have been developed for 3D games, these differences require a fundamental different approach towards how to make virtual worlds accessible to users with visual impairments.

How it works

Shells and early text based adventure games such as Zork are examples of command line interfaces (CLI) that are very accessible to screen readers as they provide textual output and users iteratively interact with them using commands. We implemented a command line interface (TextSL) on top of Second Life whose output can be read with a screen reader. TextSL accepts commands from the user allowing the user to move their avatar around ("move north"), interact with other avatars ("say") and objects ("sit on chair") and get information on their environment using simple spatial queries (e.g. "describe") that can be parametrized (e.g. "describe chair"). This command line interface is turned into a simple natural language interface by mapping common verbs ("go", "walk","proceed") to the same internal command ("move"). This allows users to interact using natural language which avoids users having to memorize specific commands. A interpreter allows for using propositions and conjunctions, e.g., "sit on the chair". When answering spatial queries a simple form of pruning and iterative queries are used to avoid overwhelming the user with feedback.

Link

Try TextSL and let us know your experiences.

Media

Publications

- Bugra Oktay, Eelke Folmer. TextSL: a screen reader accessible interface for Second Life, Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A'10) Raleigh, North Caroline, April 2010. [26% acceptance rate]

- Eelke Folmer, Bei Yuan, Dave Carr, Manjari Sapre. TextSL: A Command-Based Virtual World Interface for the Visually Impaired, In Proceedings of the 11th international ACM SIGACCESS conference on Computers and Accessibility (ASSETS'09), Pages 59-66, Pittsburgh, Pennsylvania, October 2009. [32% acceptance rate]

Acknowledgements

This research is supported by NSF grants 0738921 and 0917362