/ Sketches in

Voice User Interface

Relational Conversations with Virtual Personal Assistants in Domestic Spaces.

GRADUATE THESIS

A research thesis project under the guidance of Kate Hartman and Bill Leeming, in partial fulfilment of the requirements for the degree of Master of Design in Digital Futures at OCAD University. 

 

EXHIBITION

This project is part of the Digital Futures Grad Show 2021. Visit the exhibition page here.

© Manisha Laroia 2021

/ ABOUT

Sketches in Voice User Interface (VUI) explores ways in which VUIs can be designed to go beyond the existing Virtual Personal Assistant in our everyday conversations, specifically as smart speaker embodiments. Unlike conventional voice-based virtual assistants (Siri, Alexa, Google Assistant), the Sketches in VUI  drive conversations and take an agentive role in human-computer conversational interactions.

 

The Sketches in VUI is a portfolio of four Sketches. Sketch 1:The Thinker, Sketch 2:The Listener, Sketch 3: The Learner and Sketch 4:The Chatterer.

This work explores the complex machinery that is 'human-computer conversations', critiquing the master-servant relationship we have with this technology in our intimate spaces and proposing a designed associate-like relationship with this agentive tool.

Experience the Sketches

the-sketches.gif

SKETCH 1

The Thinker

What if we could brainstorm an idea with a VUI?

The Thinker asks questions to discuss ideas with the user and in turn, gives them a fresh perspective. The attempt is to hold a conversation with the Assistant while working as one with a colleague.

SKETCH 2

The Listener

What if we could pour our thoughts out to a VUI when we feel overwhelmed?

The Listener wants to know what the person is thinking and is there to hear them out. It is designed to listen when one is entangled in their thoughts. 

SKETCH 3

The Learner

What if we could engage VUI in a group conversation?

The Learner attempts to engage people around it in a conversation. It is designed to take part in a group conversation, curious to know about the people it is talking to and intends to facilitate a conversation between them.

SKETCH 4

The Chatterer

What if the VUI initiates a conversation and is the one to be listened to?

The Chatterer only wants to talk about itself. It is designed to drive the conversation its way making the user be the listener. 

/ EXPERIENCE TESTING STUDY

The goal of the study was to observe and compare the participants' experience of conventional domestic Virtual Personal Assistants to their experience with the Sketches in VUI.

Sketches in VUI experience testing

Findings from the Testing

sk-VUI-quotes.png

"It feels like I am talking to myself. It was not constructive.”

“I usually give shorter answers, but this made me give longer responses and I did not expect that.”

“I am constantly thinking if it gets me. It is not human but gives me a vibe that it is- as it has opinions, talks about thoughts and goals.”

“They speak so clearly that it makes me hyper-aware of how I talk.”

FINDING 1

Based on the first responses shared by the participants, strong evocations are indications that unlike for task-based VUI which are one size fits all, in relational VUI conversation the person is an important actor and bring with themselves the expert knowledge of conversation having used speech as their primary communication mode.

FINDING 2

The design elements in Sketches in VUI shifted the current VPAs towards one with an agency in a conversation where is it associate-like, emotionally-present, trying to be empathetic and talks as an equal actor in the conversation, sometimes even driving the conversation.

FINDING 3

An estimation of the ‘gap’ between two responses by each actor is essential for a smooth conversation. To a VUI we appear as agents that can listen, talk and see and the VUI itself can listen (array mic), speak (the speaker) and has LED indicators. A VUI conversation is between these two actors and understanding this model can be useful in defining the constraints to design in for relational VUI.

The Process

Project process.png

/ RESEARCH METHODOLOGY

The backbone of my design research is the annotated Research through Design methodology. The Sketches in VUI and the experience test findings is the design knowledge generated and forms the ‘annotated portfolio.’

/ SKETCHING

Sketching as a prototyping method is adopted from sketches being evocative, suggestive, rough, easy to build explorations not necessarily on paper (Moussette 2012). Sketches in VUI are software Sketches with a voice-based interface that are interacted with, by a user in a short conversation.

/ THE AUTOETHNOGRAPHY 

The autoethnography study was used to inform the making of the Sketches. It is a 6-month study from July- January 2020 of me living with and using a voice-based Virtual Personal Assistant here - the Google Assistant on a Google Nest mini device christened Jeeves.

Website footer.jpg

Go back to the Exhibition Page.