Researchers develop new image recognition software

A group of MIT researchers have discovered that just a few pixels of information can identify objects in an image. This discovery could lead to tremendous advances in the institute to automatically identify online images and, in particular, provide the platform for computers to look like humans.

Antonio Torralba, a lecturer at MIT's Artificial Intelligence and Computer Science Laboratory, and his colleagues tried to find out what is the smallest amount of information - the shortest number of manifestations - that can be extracted. from an image that brings useful content content.

Getting such a short expression will be an important step in realizing the ability to classify millions of images on the Internet automatically. Currently, the only way to find the image is based on the text captions that people enter into each picture, but many photos lack this information. Automatic identification will provide a method to annotate photos that people download from digital cameras to computers without having to fill out a caption for each photo by hand. And in particular it can lead to real machine images, which could one day visualize data entered from the camera and know what they are.

According to Torralba 'We are trying to find very short codes for images so that if there are two images with the same number sequence, they may be similarly or nearly identical to the same object, in almost the same way. situation.' If an image is identified with a caption or title, other images that match its code will probably return to the same object (eg, car, tree or person) and so the name is related. A photo can be transferred to other photos.

According to Torralba 'With an extremely large number of images, even fairly simple operations can perform quite well' in identifying images this way. Torralba will present his findings in May at a workshop on Computer Identity and Computer Image, Alaska. This work was done with Rob Fergus at the Courant Institute, New York University and Yair Weiss Jewish University, Jerusalem.

(Photo: Antonio Torralba)

Question: What do you see in the red circles? A bottle, a cell phone, a person and a shoe? Answer: They are just one!Lecturer Antonio Torralba created these low-resolution images, then inserted into the object in the red circle, thereby showing how the external impact on the process of identifying our objects.Even the car in the photo below the left is the same thing.

To find out how much information is at least enough for people to identify objects in a photo, Torralba and co-authors try to reduce the image to lower and lower resolutions, and see if people can be determine how many photos are at each level.

'We can recognize what is in the picture, even the resolution is very low because we know a lot about photos. The amount of information you need to determine most images is about 32x32. ' In contrast, even the thumbnail images displayed on Google's search results are usually 100x100.

Even an affordable digital camera now produces images containing megapixels of data - and each pixel usually contains 24 bits (0 or 1) of data. But Torralba and his colleagues have created a mathematical system that can reduce data from each image more and it turns out that many images are identifiable even when encoded into mathematical expressions containing 256 to 1024. data bit.

Using such a small amount of information on each image makes it possible to search for similar images in millions of photos in the database, using a personal computer for less than 1 second. And unlike other methods that require splitting each image into small items containing different objects, this method uses the whole image, making it easier to apply it to large data sets. without human intervention.

For example, using the encryption system they developed, Torralba and colleagues can display 12.9 million images from the Internet with just 600 megabytes of data - small enough to fit in the RAM of most PCs. current, and can be stored on a memory card. Image data and data search software are currently published online.

Of course, a system that minimizes the amount of information cannot reach complete identification. Currently, the pairing method works on the most common types of images. 'Not all images are created equal.' The more complex or strange the image, the harder it is to pair. But for the most common objects in photos - people, flowers, cars, buildings - the results are impressive.

This work is part of a study conducted by hundreds of groups around the world, aimed at analyzing the content of visual information. Torralba also collaborates in related projects with other MIT researchers such as William Freeman, a lecturer in the Department of Electronics and Computer Science; Aude Oliva, lecturer in the Department of Brain and Cognitive Science, PhD student Bryan Russell and Ce Liu, of CSAIL. Torralba's work is partly supported through the National Science Foundation.

Torralba emphasized that this work is still in its infancy and there are many problems in identifying more unusual objects. It is the same way we recognize language. 'There are many words you hear very often, but no matter how long you live there, there will be words that you never knew. You always need to understand something new from an example. '