/ Sketches in
Voice User Interface
Relational Conversations with Virtual Personal Assistants in Domestic Spaces.
This project is part of the Digital Futures Grad Show 2021. Visit the exhibition page here.
© Manisha Laroia 2021
Sketches in Voice User Interface (VUI) explores ways in which VUIs can be designed to go beyond the existing Virtual Personal Assistant in our everyday conversations, specifically as smart speaker embodiments. Unlike conventional voice-based virtual assistants (Siri, Alexa, Google Assistant), the Sketches in VUI drive conversations and take an agentive role in human-computer conversational interactions.
The Sketches in VUI is a portfolio of four Sketches. Sketch 1:The Thinker, Sketch 2:The Listener, Sketch 3: The Learner and Sketch 4:The Chatterer.
This work explores the complex machinery that is 'human-computer conversations', critiquing the master-servant relationship we have with this technology in our intimate spaces and proposing a designed associate-like relationship with this agentive tool.
Experience the Sketches
What if we could brainstorm an idea with a VUI?
The Thinker asks questions to discuss ideas with the user and in turn, gives them a fresh perspective. The attempt is to hold a conversation with the Assistant while working as one with a colleague.
What if we could pour my thoughts out to a VUI when we feel overwhelmed?
The Listener wants to know what the person is thinking and is there to hear them out. It is designed to listen when one is entangled in their thoughts.
What if we could engage VUI in a group conversation?
The Learner attempts to engage people around it in a conversation. It is designed to take part in a group conversation, curious to know about the people it is talking to and intends to facilitate a conversation between them.
What if the VUI initiates a conversation and is the one to be listened to?
The Chatterer only wants to talk about itself. It is designed to drive the conversation its way making the user be the listener.
/ EXPERIENCE TESTING STUDY
The goal of the study was to observe and compare the participants' experience of conventional domestic Virtual Personal Assistants to their experience with the Sketches in VUI.
Findings from the Testing
"It feels like I am talking to myself. It was not constructive.”
“I usually give shorter answers, but this made me give longer responses and I did not expect that.”
“I am constantly thinking if it gets me. It is not human but gives me a vibe that it is- as it has opinions, talks about thoughts and goals.”
“They speak so clearly that it makes me hyper-aware of how I talk.”
Based on the first responses shared by the participants, strong evocations are indications that unlike for task-based VUI which are one size fits all, in relational VUI conversation the person is an important actor and bring with themselves the expert knowledge of conversation having used speech as their primary communication mode.
The design elements in Sketches in VUI shifted the current VPAs towards one with an agency in a conversation where is it associate-like, emotionally-present, trying to be empathetic and talks as an equal actor in the conversation, sometimes even driving the conversation.
An estimation of the ‘gap’ between two responses by each actor is essential for a smooth conversation. To a VUI we appear as agents that can listen, talk and see and the VUI itself can listen (array mic), speak (the speaker) and has LED indicators. A VUI conversation is between these two actors and understanding this model can be useful in defining the constraints to design in for relational VUI.
/ RESEARCH METHODOLOGY
The backbone of my design research is the annotated Research through Design methodology. The Sketches in VUI and the experience test findings is the design knowledge generated and forms the ‘annotated portfolio.’
Sketching as a prototyping method is adopted from sketches being evocative, suggestive, rough, easy to build explorations not necessarily on paper (Moussette 2012). Sketches in VUI are software Sketches with a voice-based interface that are interacted with, by a user in a short conversation.
/ THE AUTOETHNOGRAPHY
The autoethnography study was used to inform the making of the Sketches. It is a 6-month study from July- January 2020 of me living with and using a voice-based Virtual Personal Assistant here - the Google Assistant on a Google Nest mini device christened Jeeves.