Zink, Sydney: Travel Grant - 2017-2018

College: 
College of Engineering
Department: 
Computer Science
Project Title: 
You Talk a Good Game: Gaming Accessibility through Speech Narrative
Project Abstract: 
What if language were used in games not only for accessibility in making traditional keyboard/mouse controls unnecessary, but also in situating the language for practiced casual, conversational use so as to encourage the same beyond gameplay? My project aspires to both of these goals past its primary goal of teaching a game to allow a user to control it solely by talking to the game -- that is, giving it conversational language inputs or an embellished storytelling narrative from which to extract executable game actions in sequence. This involves allowing the game to adapt to multiple languages in such a way as to treat this as a natural language processing and machine learning problem that frames a translation from natural language to game input "language." It also involves understanding how multiple cultures refer to the same actions or intents in the same language -- e.g. "I approach the castle" (translated into key inputs for moving the game agent toward a destination point) may have an idiom in, for example, an Irish or Australian person's approach to using English.