Teaching Computers to Defeat Camouflage

The ability to blend in is very useful in the animal kingdom, where every day is an adventure in the rule of eat or be eaten. Animals from all walks of life have developed elaborate mechanisms of camouflage that seem to have evolved to foil the exact cues upon which animal and human vision rely most heavily—color, texture, and edges.

The excellent blending abilities of some animals pose challenges for marine biologists and oceanographers in knowing where, and how abundant, these animals are. Flounder fish are a bellwether fauna for ocean health, so it’s important for scientists to have accurate population data. But because they are also masters of disguise—able to change their coloration to imitate the color and texture of the ocean floor—flounder can be hard to find.

Human eyes are better at finding hidden objects in photographs than today’s computers are. But doing so is an incredibly tedious task with a high rate of burnout, because fatigued eyes can fail. Computers can work around the clock without getting tired, but they just aren’t that good at the task. Los Alamos data scientist Lakshman Prasad is working to change that. By studying how and why human vision fails to find hidden things, and what cues get missed, Prasad is figuring out how to train computers to reliably spot animals that are trying to hide. In a collaboration with Woods Hole Oceanographic Institute and the National Oceanic and Atmospheric Administration, Prasad has developed an algorithm that automatically analyzes images of the seafloor for tell-tale cues and then scores the images, indicating the likelihood of an animal such as a flounder, skate, or octopus hiding somewhere in the frame.

The main thrust of the work is to understand ocean ecosystems to better inform conservation efforts, but there is a tangible commercial element as well. Overfishing and environmental damage are big concerns to the fishing industry, which has to know how much fishing an ecosystem can sustain. Accurate counts are needed to quantify concerns and put appropriate regulations into place.

There are non-oceanographic uses for the technology as well. National security and intelligence agencies need efficient methods for detecting and analyzing features in remote-sensing imagery. Biomedical imaging and diagnostics too may benefit from the methods of detecting image features and their structures that Prasad’s algorithms offer.

To train the computers, their performance has to be evaluated. After initial examination, any image that is flagged by the first algorithm is next evaluated by a more complex algorithm and also a set of human eyes. This feedback helps to improve the program’s accuracy.

“The name of the game is ‘find the odd ball,’” says Prasad. “How do you mimic human vision situations such as doing a double take? You have a sense that there is something there the first time, so you look again to see if you can make it out. How do you train a computer to make those judgment calls?”

Prasad is still telling the machine what to look for, like smooth edges or straight lines or textures that are at odds with their surroundings. There is still a lot of ground truthing and training to be done, so this isn’t yet machine learning. But it could be a preamble to it.

The proverbial needle in the haystack (or flounder in the sand) is a universal problem. And because nature is under no obligation to obey statistical simplifications, every piece of the haystack has to be examined. The only really tractable approach to looking at the entire haystack is to develop complicated algorithms and have a machine do it. It’s a challenging problem, but Prasad is not daunted by the challenge.

“I want people to know that it’s not just a hard problem, it’s a problem that’s worth solving,” he says. “And if you cut your teeth on a tough problem, you wind up with a robust new tool that can be used on other problems. The dividends are invaluable.”