Both Apple and Amazon now allow third parties to build their own brand-specific voice technologies on their respective products, Siri and Echo. These are huge steps forward in artificial intelligence (AI) based conversational interfaces or 'chatbots' that will make computing a more effortless and intuitive part of our world

Microsoft was also recently in the news, but not in the way they had hoped. Their release of a chatbot named Tay, an AI designed to become smarter as more users interacted with it, was hijacked to spout hate speech. Microsoft created Tay to learn how AI programs could interact with users in casual conversation and, over time, learn from that interaction--just like people do. Now it's back to the drawing board.

Welcome to the brave new world of 'conversational understanding,' in which a computer is designed to act like a human with the ability to perform desired tasks, anticipate concerns or actions, and personalize responses, including the use of humor. 

In the world of commerce, companies want their 'brand voice' to have a distinct personality. But Siri and Echo now prompt us to think of brand voice more literally. We expect a plethora of subtle branding in the devices around us, but voice changes this equation: imagine your kitchen range by LG, in the voice of chef Jamie Oliver, guiding you through the recipe for Beef Wellington, and correcting you before you make a mistake with the pastry crust. Or Samantha, the voice of Lexus, greeting you as you put on your seatbelt and advising you about an accident on the Interstate, and then proposing an alternate route. Independently these new brand voices might be compelling, but as my friend Stacey Higginbotham at Fortune writes, a cacophony of computerized voices in the home might create a new kind of digital fatigue. 

Brands already have a long history of trying to be our friends, which to a savvy consumer rings horribly false. What happens when each of them tries to engage us in conversation? Which brands should merit our attention? As the technology matures, should I continue to speak with Echo's Alexa or a growing list of brand-specific characters? Could I ever trust more than a handful of unique brand voices? 

 inline image

One solution could be a personalized virtual assistant for each of us with a voice of its own. Empowered by my personal data and evolving to meet my needs through machine learning, it would engage other brands on my behalf. The larger question of who or what is the voice is answered--it's actually a digital extension of me. Unfortunately, this vision contradicts the goals of brands that would rather control the narrative and 'speak' to me directly.  

Brett Scott, author of The Heretic's Guide to Global Finance, writes about a different kind of chatbot fatigue that will sound familiar to anyone who has dealt with a call center robot recently; tired of a machine that can't empathize with our particular customer service needs, we may just want to opt out and speak to a human being. Does AI have an empathy problem, asks design writer Mark Wilson?  

Some issues will be solved as we create conversational AIs with adaptive and learning capabilities similar to humans, including predicting outcomes, making suggestions without being asked, creative and unexpected behavior, and finally a sensitivity to what the human user might be feeling.  

As our relationship with machines begins to shift, the danger is that we begin to feel subordinate to the machine. There is a cost to everything, even efficiency. To design human-centered AIs, we should look at the task in sociological terms--with the notion that automation shouldn't be the end-goal. If the AI is going to mimic human behavior, it must be the kind of behavior that encourages and empowers us. 

The failure of Microsoft's Tay is a reminder that today's AI is still a reflection of the humans who created it. Our core design problem is how to promote genial and open-ended cooperation between man and machine. In the new frontier of conversational interface design, the user experience now encompasses the arts of language and persuasion, as well as attributes like plasticity, creativity, and empathy. These challenges are not unlike the social challenges we face among fellow humans. When giving voice to the machines around us, our central goal should be to increase human agency and dignity.