Eelke Folmer
Human+ Lab
University of Nevada, Reno

Feedback Synthesizer for exploring Virtual Worlds with a Screen Reader

Virtual worlds such as Second Life contain large amounts of content. Consequently, turning a scene into a textual representation that can be read with a screen reader may often overwhelm the user with feedback. To accommodate environments that are dense in content users can iteratively query their environment, e.g., describe which results in there are 3 people and 30 objects, upon which users can iteratively query subsets, e.g., describe objects. However, in environments with few objects this makes interaction slow. We propose a feedback synthesizer called: Syntherella, which dynamically adjusts the amount of feedback based on the object density of an environment and the preferences of the user.

How it works

Syntherella consists of five translation steps:
  1. Scanning:we collect the names and descriptions of objects in a 360 degree range around the user with a predefined radius.
  2. Filtering: we filter out the objects that have non descriptive names.
  3. Tresholding: objects are prioritized based on: (1) name length; (2) description length; (3) distance to the user; (4) size; and (5) interaction capabilities
  4. Grouping: Objects are grouped by type.
  5. Aggregation or Detailing: based on whether the compiled description exceeds a user specified word limited, the set of objects is either (1) further reduced using an object taxonomy until the feedback is under the user specified word limited; or (2) detailed with more information, such as, description, location, size, color sound, price tag and interaction capabilities.
Two examples of types of generated feedback, for different word limits.
  1. >There are 2 cats to your north; 1 cat to your south and 2 chairs to your east.
  2. >There is car (it is described as "yellow sports car with two doors"; it is interactive and it is for sale for L200; it is 20 m west and 6 m north of you), window (it is described as "big triangle shaped window"; it is not interactive but it is for sale for L19; it is 12 m east of you).


Syntherella is implemented in our accessible virtual world interface called TextSL.


Bugra Oktay, Eelke Folmer. Syntherella: A Feedback Synthesizer for Efficient Exploration of Virtual Worlds using a Screen Reader, Proceedings of Graphics Interface (GI'11). Pages 65-70, St John, New Foundland, Canada, May 2011. [31% acceptance rate]