How can artificial intelligence steal data by 'recognizing' keystrokes?

According to scientists, in the near future, faces are not the only thing that we can train AI to recognize.

Computer vision

What if AI could analyze a video of tapping on a phone's touchscreen and correctly guess what apps we're using and what we're typing?

Modern computer vision techniques are capable of making us see technological superpowers that usually only appear in movies. We can upload a video to an AI system and ask it to zoom in on a low resolution frame, and with a little training, some clever algorithms, we can make the picture quality of the video better. markedly enhanced.

This may not sound very dangerous, but you need to know that humans can use computer vision to apply it to everything from detecting cancer to counting large numbers of objects in a photo. . At the same time, there's nothing stopping a developer from training an AI system to read text from keystrokes or finger movements. So is this scary or not?

Researchers are training the AI ​​so that it mimics human typing.

First, it's worth reiterating that computer vision technology has come a long way since 2017, when Google's AI still made basic mistakes like confusing a turtle with a rifle.

To date, computer vision systems have advanced greatly, being able to make extremely powerful inferences with extremely small amounts of data. For example, researchers have demonstrated the ability of computers to authenticate users based on typing biometrics via AI, and psychologists have developed a stress detection system using combinatorial analysis. keys.

Researchers are even training the AI ​​to mimic human typing. This enables us to develop better tools for checking spelling, grammar and other communication techniques. The basic thing now is that we are teaching AI systems to make inferences from finger movements that humans cannot.

It is not too difficult to imagine the existence of a system capable of analyzing finger movements and interpreting it into text in the same way that we move our lips to form speech. Currently, there is no such AI product, but that does not mean it is not available in the world.

So what's the worst?

Not long ago, when the internet was not yet developed, shoulder surfing or the technique of gathering information by viewing syslog files was one of the biggest problems with computer security. Basically, the easiest way to steal someone's password is to see how they type it.

That's why most screens for entering passwords hide passwords as '*' or '.' . This makes it impossible for anyone to know what character you are entering. When typing a password, our fingers move surprisingly quickly with the coordination of our eyes.

However, AI can be trained to read these movements and almost anything can happen in the AI ​​world if it has enough data.

Theoretically, creating an AI system that reads human hand movements to figure out passwords should be relatively simple for a well-resourced developer. It will give developers an idea of ​​what we are touching and typing on our phones or computers.

That would allow the bad guys to steal our passwords, ATM pins and all of our data quite easily. This will be even more dangerous if software developers can achieve the ability to turn any camera on a phone into a keystroke detector.