First Impressions of Chatbot UX on Facebook Messenger

by: | Apr 14, 2016

good bot bad bot ebookWhen I updated Facebook Messenger Tuesday, I instantly had five new friends. Of course, with Facebook’s rollout of its new chatbot-friendly Messenger platform, making new friends (with a bot) won’t be difficult.

Facebook Messenger: Chatbot UX

Yesterday, I decided to see just what my new friends were really capable of. Really, I was most interested in what the user experience (UX) these first-to-market bots offered. In summary, what I experienced reminded me that these are early days for chatbots. The five initial Facebook Messenger bots are somewhat useful but fell short of “delightful” — a threshold we aspire to with every mobile and connected experience we create — and I’m not sure I would want to have an ongoing friendship with any of them.

More conclusions to come below. But first, here are more specifics about how to get started with bots in Messenger, and how my first experience went.

To try it yourself, make sure you have the updated version of Messenger. Next, click into the search pane along the top (on iPhone), where it says “Search for people and groups.” At this point, there’s no mention of bots in that field, but as you start to type something in, you’ll see in the search results the available “Bots:”

box UX of Facebook Messenger

As I mentioned, there were five when I started testing:

  • CNN — suggests relevant news stories
  • Hi Poncho — provides local weather conditions
  • Shop Spring — find and buy products from thousands of brands
  • 1-800-Flowers — find and buy flowers
  • Detective Kees — play a detective mystery game

Select one of the options and you’ll see a welcome message or some brief instruction of what the bot can do and how to converse with it:

bot UX of shopping bot

Here are some of the specific chatbot UX areas I was looking for — and how these new friends performed.

Humor — Most of the bots were dry and clinical feeling. Question. Answer. Question. Answer. If you stray too far from their known language, they reply with a sterile message like “I don’t understand.” One notable exception was Hi Poncho, which started with some humor. It was after midnight when I started my conversation with Poncho and his first message to me was “Zzz.” The sleeping cat resonated in that moment — and the metaphor, that I could wake the cat anytime I needed a little love, carries through the whole experience.

Personality — Hi Poncho’s personality and style carried through beyond that initial sleepy greeting. More to explore here, but the way it dealt with shortcomings in its AI was similar to how Siri handles unknowns with bits of humor and one-liners. At one point when Hi Poncho didn’t understand me, it said, “Sorry, dozed off for a second. What were you saying?”

Buggy behavior — Another time when Hi Poncho didn’t understand me, it said “Sorry, an error occurred when looking for this location.” Clearly a bug. We’re early days here, but chatbots are an extension of your company — they require the same level of quality assurance testing as any other product or service your company provides. Test, test, test. This was a big fail.

Personalization done right — 1-800-Flowers started the conversation with “Hi Adam.” A seemingly simple thing, but it made me smile and was probably very simple to implement. I didn’t find it creepy that it knew my first name. After all, I am signed into the Messenger app. It probably would have also simplified my ecommerce experience by pre-filling information it knew about me.

Personalization done wrong — One time, Hi Poncho started my conversation with “Hello Anonymous!” I’m not sure “Anonymous” is deserving of an exclamation point! And it probably would have just been better to say “Hi!” if it didn’t know my specific name.

Personalization done REALLY wrong — The CNN bot has started to push messages to me about news it thinks I am interested in. How does it know what I want to learn about? It may be that over time, it’ll learn what I like based on my interaction on different topics and my searches. But there are no options or help information about rating, filtering, liking or giving any feedback to CNN to start or stop this.

Push permissions — Speaking of push, I didn’t ask CNN to push new content to me and have proactive conversations with their bot. And there’s no obvious way to turn this off without blocking the bot entirely. I tried to delete the conversation and that didn’t work. There’s also no obvious way to adjust the timing or frequency of these proactive chats. Hey CNN bot, we just met, and we’re not best friends yet. You need to earn the right to initiate a conversation with me and ask for my permission — and give me clear instructions to turn this off.

Discovery — Bot discovery was pretty easy once I figured out that I need to tap on the Search field. But there are only five bots right now. What happens when there are hundreds of thousands or millions? How good will the search be? Bot discovery will eventually provide the same challenge as we see today with apps on the eight-year-old public app stores.

Help — Platforms like Messenger need some common standards around Help options. It should be persistent and consistent across all bots. CNN, as an example, has no built-in support. If you search for “Support” in Shop Spring, it will trigger a (human) customer support. And 1-800-Flowers has a support button built right into its welcome message (that you see even before you do anything). No matter which brand a bot represents, it needs a common set of overarching standards to deliver a user experience that becomes the norm (and understood) by all users on that platform. There’s also a need to create standards for governing things like rating bots, silencing bad ones, and standardizing permissions around push, location services, and privacy.

Early days: How my new bot friends performed

Overall, my initial impression with the chatbot UX of my five new bot friends was encouraging — I can visualize more clearly how brands and startups can take advantage of chatbots and this new platform. But there is work to do. The experience of the first five felt very “transactional” — the user interface is basically a picture, some text, and a few pre-canned options to choose from, which take you to a new set of options (almost like a Choose Your Own Adventure novel). Though there’s nothing that makes me smile about these, they mostly accomplish their intended purpose.

Some things, like buying flowers on 1-800-Flowers, actually seem like they would take longer with the back and forth conversation than going through a responsive website or native app. And the beauty of the flower arrangements is not really captured in the Messages flow. It would be better merchandised in an app or site.

One approach we expect to see is for bots to start the conversation with potential customers, but to drive users at a certain point into a more immersive website or mobile app to deliver the more engaging experience or more efficiently finish a transaction. Put another way, bots take on the role of virtual concierge, and if what you need is a quick bit of information or a micro-engagement, they handle it. If you need more, they direct you to the full experience you are craving.