Seek-n-Tag: A Game for Labeling and Classifying Virtual World Objects
A problem with user-generated content in virtual worlds such as Second Life is that they often lack accurate metadata for their content. Objects in Second Life can be given a name and a description, but as most content creators assume other users can see, they frequently leave these properties with their default value. An analysis of 350,000 objects in 433 regions in Second Life found 31% of the objects to be called "object"
. The lack of metadata is a serious problem towards making virtual worlds accessible for users who are visually impaired as they rely on a textual representation of objects to be present which can be read with a screen reader or tactile display.
How it works
As manually labeling millions of objects is tedious we propose an automated approach. A classifier can be trained to recognize object categories based on how objects are composed out of smaller solid bodies called prims. To train this classifier a set of data needs to be created with accurate labels.
Humans significantly outperform computers in recognizing objects and by wrapping such tasks into a game (Games-with-a-Purpose) tedious human-based computation tasks can be made more attractive to perform. Modeled after a scavenger hunt game, we implemented a game in Second Life called SEEK-N-TAG that offers sighted players a game where they will try to find and tag specific objects in Second Life within a certain time frame. This game is seeded by given labels for an object and only seeks to confirm the given label of an object. The faster players find an object the more points they score. See below for a video of how it works. Different labeling attempts by different players for the same object are compared as to achieve consensus on a given label for an object. When multiple players agree on a given label the object can be added to the training set. A user study with 10 participants compared SEEK-N-TAG with manual labeling and found that SEEK-N-TAG is more effective and accurate than manual labeling.
Bei Yuan, Manjari Sapre, Eelke Folmer. Seek-n-Tag: A Game for Labeling and Classifying Virtual World Objects, Proceedings of Graphics Interface (GI'10). Pages 201-208, Ottawa, Ontario, June 2010.
[40% acceptance rate]
This research is supported by NSF grant 0917362