Motivational Script Editor

In 2015, I co-founded the company Tinybots. As the lead designer I developed Tessa; a small social robot that supports people with cognitive disabilities and their families. Tessa is a small, talking robot that speaks friendly reminders, which family members can write and schedule via an easy-to-use app.

Initially, Tessa could only give reminders. Based on extensive user testing and feedback, Tessa needed to be interactive to provide better, and more context-aware support.

Client

Tinybots

Year

2018

Duration

8 months

Role

Lead designer

Screenshots of the Tinybots app, showing the script planner interface

The key challenge was to design interactivity that was predictable. Routine and structure is important for people with dementia. Families wanted to make sure Tessa's voice was curated—what Tessa says should be up to them.

The robot Tessa standing on a wooden table in bright light

Initially, Tessa's could only give reminders (one-way communication)

Tessa's interactive scripts used speech recognition to detect yes and no answers. By using simple, closed questions, families could create branched conversations. The families were fully in control of what Tessa was saying, and when.

Several different low-fidelity concepts were made and tested with users to design a simple, but flexible way to create conversational scripts. 

An early concept of the script planner, as a mockup An early concept of the script planner, as a mockup An early concept of the script planner, as a mockup

Some early concepts how to create and plan conversational scripts

To make sure all the different states, edge cases, and possible scenarios were covered, each individual flow was mapped. Not only what the app does, but also what the robot says and does, and what the robot communicates back to the app.

Flowchart diagram of one of the app flows for the script editor

Flow for creating a new script: user actions (blue), system actions (green), UI screens (red), details obscured on purpose

The final result was a script editor that allows users to write different scenarios based on the answer Tessa recognized. If Tessa recognized "yes" after asking a question, Tessa would execute the yes-branch. If Tessa recognized a "no", then the no-branch is executed. The other-branch was only used when Tessa did not recognize a specific yes or no.

Animation of the script editor in use

Tessa's script editor

The interface was designed to have simple design based on speech balloons. Each speech balloon in the script was one step in the script. Each step is a Say-action, an Ask-action, or a Wait-action, which the families could use to combine into their own conversational scripts.

Screenshots of the Tinybots app, showing the script planner interface
The app also had a library of scripts: Users could add, modify, and reuse existing scripts

Scripts can be reused, modified, and planned like regular reminders. After the user has set the time and date, Tessa will initiate that script at that moment. For example, you could write a script that asks you to check if you locked your front every night. 

Screenshots of the Tinybots app, showing the script planner interface

Tessa can also play music via the script feature

Collaborators:

Arno Nederlof
Erik Hoogeveen
Wang Long Li