Virtual Assistants

Virtual assistants are all the rage now, in the press if not in not people’s lives.

I am not claiming to do a thorough, like for like comparison. What follows is my subjective, personal experience. Your usage patterns, successes and failures may be different to mine, but I think my conclusions should broadly hold. We’ll see.

I’m comparing an Amazon Echo, an iPhone 6S and the Google app on that same iPhone. I know I’m not using Google Now in its native environment. That was unavoidable and may hobble it. You should bare that in mind if you use Android.

So this Sunday we were going to go to IKEA. It’s a Sunday and the opening hours are shorter than the rest of the week, but which ones?

“Alexa, Ikea Croydon opening hours today.”

Two things about this. First, I don’t really use a conversational style with my computers. That works in the adverts but I try to keep my interactions with inanimate objects short.

Second, none of the three systems had any real trouble understanding what I said. They may have difficulty understanding what I meant but the words themselves posed no problem. If you’ve used voice recognition software any time in the last twenty years this is amazing. (This is not to say that any are 100% accurate, just that the standard is very high and fairly consistent.)

Anyway, while Alexa completely understood the words I asked it, it wasn’t able to get an answer. It didn’t even punt it to Google or Bing.

Siri did better. It thought “Ikea Croydon” was ambiguous and asked me to confirm its guess (it had it right) and then spoke the correct answer.

Google Now went straight to the answer, though it only showed it on screen and didn’t speak it.

Of the three responses, I’d put Siri first and Alexa last. I think speaking the responses makes it more useful, even though I had to say exactly which IKEA I was going to and Google guessed correctly. Reasonable people could argue that Google’s response was better.

Next I asked for the weather: “What’s the weather in Croydon in Fahrenheit.” The catch is the ‘in Fahrenheit’ part, since I normally work in metric. Both Siri and Google Now got it right. Apple’s agent remembered my preference for Imperial measures for the current session, so immediately asking for another city gave the answer in Fahrenheit but later in the day it reverted to the correct measure.

Alexa wasn’t quite as good. It got the right weather but kept the default units. I won’t make the argument that this is another reason for people to give up Imperial measures.

Finally I asked all three whether there had been any problems reported on the Northern Line, the nearest line on the London Underground to my house.

Siri and Google both went to a search engine for the answer, just showing a list of sites that might have the answer but not actually showing the results.

Alexa doesn’t natively know the answer but it has an extensive directory of “skills” that you can add. I used one called “Tube Status.”

It can answer the question but you have to talk to it in a specific way. “Alexa, are there any problems reported on the Northern Line” fails. Instead you have to say, “Alexa, ask Tube Status about the Northern Line.” The “ask Tube Status about” bit is critical; without it, it doesn’t work.

In this case, Alexa comes first but with some caveats. Google and Apple come joint last. This is odd as both companies have the transit data available in their respective Maps app, but they’re currently not surfacing it to their voice assistants.

That experience is actually a pretty good overall summary. Alexa can do a lot, possibly more than either Siri or Google Now, but it’s not so good at understanding what you’re asking without help. Google has its vast trove of data to virtual hand and uses it to make clever inferences about what you mean rather than what you asked. Siri tries to have more of a personality and otherwise falls somewhere between the two. In some areas the fact that it has less data than Google is a feature (privacy!) but that can limit its options.

I have a hard time saying one is better than the other. They’re all flawed, they all got some stuff right and wrong. It’s early days and rather than showing one is better than the others, it really shows differing priorities. Amazon emphasises its API, Apple its privacy, Google its data. They’re all right. An ideal system would have all of those. And in the future they probably will.