A couple of weeks after the Internet made Tay, a sophisticated chat bot, into a Hitler-loving, genocidal repository of 4chan memes, people have discovered another new-ish bot from Microsoft. This one is called captionbot, and it exists to try and describe to you the content of an image.
Upload any image to Captionbot, and you’ll get a sentence back describing what it thinks it sees. Sometimes, the bot will say, it’s not really confident with its answer. It also gets a lot of things wrong right now, based on my incredibly small sample size of random images that I decided to run by the bot because I felt like it, and what other people have said on the Internet.
Here is its analysis of a photo of Tillman, the skateboarding dog (RIP), trying out a surfboard:
I also tried running a meme through Captionbot:
This person tried another, very bad meme, and the answer was not good:
— Paul Curry (@cr3) April 14, 2016
As Microsoft explains, Caption bot, like the rest of its AI projects, is still learning, so Microsoft is hoping it will more accurately describe images as more people play with it.
So let’s turn down the difficulty level a little bit, and try a cat:
Okay. What about the pope taking a selfie?
What about the president?
So right now, you might be thinking, “Hey! that’s not bad. Maybe Caption Bot is trying to say that the president is just ‘a man,’ like the rest of us.”
Except Caption Bot can recognize some people by name. Just not, in this case, the president. Here is Caption Bot’s response to a random image I uploaded of Agent Dana Scully, played by Gillian Anderson from O.G.-era “X-files:”
It seems, to varying degrees, to be able to identify several other celebrities. The bot recognized Anderson’s former co-star David Duchovny, Idris Elba, and Jennifer Lawrence, but not OPRAH, and it was only kind of sure about Carl Sagan, as you saw at the beginning of this piece.
Don’t try cartoons, though:
Guys, please be nice.
— Stephen Nellis (@StephenNellis) April 14, 2016
If you’d like to play around with Caption Bot for yourself, it lives here.