Artificial intelligence (AI) systems have a long way to go before they can take over tasks and jobs traditionally performed by people, say scientists who highlighted the severe limitations of deep learning computer networks.Researchers at University of California, Los Angeles (UCLA) in the US conducted various experiments which showed that it is easy to fool the deep learning neural networks.
Long way before AI systems take over humans jobs
"The machines have severe limitations that we need to understand," said Philip Kellman, a UCLA professor and senior author of the study published in the journal PLOS Computational Biology According to Kellman, machine vision has drawbacks. In the first experiment, researchers showed colour images of animals and objects to one of the best deep learning networks, called VGG-19. However, the images had been altered.
In the second experiment, the psychologists showed images of glass figurines to VGG-19 and to a second deep learning network, called AlexNet. VGG-19 performed better on all the experiments in which both networks were tested. Both networks were trained to recognise objects using an image database called ImageNet. However, both networks did poorly, unable to identify the glass figurines. Neither VGG-19 nor AlexNet correctly identified the figurines as their first choices.
Most of the top responses were puzzling to the researchers, such as VGG-19's choice of "website" for "goose" and "can opener" for "polar bear." On average, AlexNet ranked the correct answer 328th out of 1,000 choices. "The machines make very different errors from humans," Lu said. In the third experiment, the researchers showed 40 drawings outlined in black, with images in white, to both VGG-19 and AlexNet. These first three experiments were meant to discover whether the devices identified objects by their shape. The networks again did a poor job of identifying such items as a butterfly, an airplane and a banana.
Researchers concluded that while humans see the entire object, the artificial intelligence networks identify fragments of the object. "This study shows these systems get the right answer in the images they were trained on without considering shape," Kellman said. "For humans, overall shape is primary for object recognition, and identifying images by overall shape doesn't seem to be in these deep learning systems at all," he said. There are dozens of deep learning machines, and the researchers think their findings apply broadly to these devices.