Today’s artificial intelligence systems for image recognition are incredibly powerful, with enormous commercial potential. Nonetheless, current artificial neural networks (the deep learning algorithms that power image recognition) have a major flaw: they are easily broken by images that have been slightly modified.
This lack of ‘robustness’ is a significant barrier for researchers attempting to develop better AIs. However, the exact cause of this phenomenon, as well as the underlying mechanisms, are largely unknown.
Researchers at Kyushu University‘s Faculty of Information Science and Electrical Engineering published in PLOS ONE a method called ‘Raw Zero-Shot’ that assesses how neural networks handle elements unknown to them in the hopes of one day overcoming these flaws. The findings could aid researchers in identifying common characteristics that render AIs ‘non-robust,’ as well as developing methods to address these issues.
“There is a range of real-world applications for image recognition neural networks, including self-driving cars and diagnostic tools in healthcare,” explains Danilo Vasconcellos Vargas, who led the study. “However, no matter how well trained the AI, it can fail with even a slight change in an image.”
Image recognition AIs are typically ‘trained’ on a large number of sample images before being asked to identify one. For example, if you want an AI to recognise ducks, you would first train it on a large number of duck images.
Even the most well-trained AIs can be duped. Indeed, researchers discovered that an image can be manipulated in such a way that, while it appears unchanged to the human eye, an AI cannot accurately identify it. A single pixel change in an image can be confusing.
To better understand why this happens, the team began investigating different image recognition AIs with the hope of identifying patterns in how they behave when faced with samples that they had not been trained with, i.e., elements unknown to the AI.
“If you give an image to an AI, it will try to tell you what it is, no matter if that answer is correct or not. So, we took the twelve most common AIs today and applied a new method called ‘Raw Zero-Shot Learning,'” continues Vargas. “Basically, we gave the AIs a series of images with no hints or training. Our hypothesis was that there would be correlations in how they answered. They would be wrong, but wrong in the same way.”
That’s exactly what they found. The image recognition AI would produce an answer in all cases, and the answers, while incorrect, would be consistent, that is, they would cluster together. The density of each cluster would indicate how the AI processed the unknown images based on its prior knowledge of various images.
“If we understand what the AI was doing and what it learned when processing unknown images, we can use that same understanding to analyze why AIs break when faced with images with single-pixel changes or slight modifications,” Vargas states. “Utilization of the knowledge we gained trying to solve one problem by applying it to a different but related problem is known as Transferability.”
Capsule Networks, also known as CapsNet, produced the densest clusters, giving it the best transferability among neural networks, according to the team. They believe it is due to the dynamic nature of CapsNet.
“While today’s AIs are accurate, they lack the robustness for further utility. We need to understand what the problem is and why it’s happening. In this work, we showed a possible strategy to study these issues,” concludes Vargas. “Instead of focusing solely on accuracy, we must investigate ways to improve robustness and flexibility. Then we may be able to develop a true artificial intelligence.”
