Computer program deciphers doodles

As difficult as Pictionary can be for human players, deciphering our abstract sketches is nigh on impossible for computers. Until now, that is. Researchers at Brown University and the Technical University of Berlin have developed the first computer program that’s capable of identifying someone’s doodles.

The researchers collected 20,000 sketches covering 250 object categories and fed them through recognition and machine learning algorithms. Once trained, the program could identify people’s doodles with a 56% success rate – not a bad performance when you consider that humans managed a 73% accuracy with the same images.

The program’s limited vocabulary means that it’s not quite ready for a game of Pictionary yet. However, turning the program into a Draw Something-style game could be one way to train it further. “The game could ask you to sketch something and if another person is able to successfully recognise it, then we can say that must have been a decent enough sketch,” says James Hays, assistant professor of computer science at Brown University. “You could collect all sorts of training data that way.”

Beyond games, this technology could lead to a search tool that uses pictures instead of words, improving computer accessibility for people with language difficulties.

By Sacha Torregrosa-Jones