Radio vision: See through smoke and around corners

A student at the University of Pennsylvania was conducting a late-night experiment aimed at testing a robot's ability to "see through" objects in smoke. However, shortly after activating the smoke machine, the fire alarm was unexpectedly triggered, causing a minor interruption to the experiment.

"The alarm went off in the entire building," said Mingmin Zhao (transliteration), from the University of Pennsylvania, with a laugh. "My student called me, he was very surprised." Nevertheless, this incident was just a small setback in the team's development of a robot equipped with an innovative radio sensing system. This system is designed to allow robots or self-driving vehicles to see through thick smoke, heavy rain, and even around corners, and even detect hidden weapons.

Using radio waves to simulate visual images is an unconventional approach for robots and self-driving vehicles, compared to traditional visible light cameras, LiDAR, and other sensors. However, Professor Zhao and his students have developed a potentially powerful method for observing using radio waves. While radar has been using radio waves to track aircraft, ships, and weather for decades, Professor Zhao's rotating array on the robot emits radio waves in all directions. Then, the onboard artificial intelligence (AI) system uses this information to construct a 3D view of the environment.

Professor Zhao explained, "We have been working to help robots gain superhuman vision capabilities, enabling them to observe in scenarios where the human eye or traditional visual sensors cannot work." He believes that this technology could help future search and rescue robots save trapped people in burning buildings. To avoid triggering the fire alarm again, subsequent robot tests used transparent plastic boxes filled with smoke, placed around the rotating device.

Although radio waves are invisible to the human eye, they are also a form of light, belonging to the electromagnetic spectrum, which includes X-rays and gamma rays. Only a small portion of the electromagnetic spectrum is classified as visible light. As a form of light, radio waves can be reflected from surfaces and materials like visible light, although in slightly different ways. The robot designed by Professor Zhao and his colleagues can sense these radio wave reflections. The key factor is that radio waves have much longer wavelengths than visible light, so they are not blocked by tiny smoke particles.

Professor Zhao said he is also working to improve the technology so that the robot can see partially around corners. He used the analogy of a "funhouse mirror for radio waves" to describe this effect. Friedemann Reinhard from the University of Rostock in Germany commented, "This is a very interesting and impressive work." He himself was not involved in the study. In 2017, Professor Reinhard and his colleagues described how Wi-Fi signals could allow spies to peek into private rooms.

A minor limitation of this technology is that the rotating array cannot see all directions simultaneously. Professor Reinhard pointed out that the system requires a large amount of data processing to clean up the images generated by the rotating device. However, the robot emits radio waves in the millimeter-wave band (wavelengths between 1 and 10 millimeters), which is the same technology used in some 5G devices. "This is very attractive because it is a very mature and inexpensive technology," Professor Reinhard said. "I would certainly like to see a self-driving car that relies solely on radar."

Fabio da Silva, founder and CEO of the American company Wavsens, said that it is possible to avoid using rotating radio transmitting devices to obtain a complete image. His company is also developing radio-based sensing technology. "We have created an algorithm that allows you to perceive the entire space instantaneously and continuously, so we don't have to rotate the antenna," he said. He described the system as similar to a bat's echolocation, which emits radio waves and "listens" to how the waves return, revealing the shape of the object they hit.

Some researchers have already used radio waves to detect hidden weapons, such as concealed handguns and knives. Radio waves can even "fingerprint" the details of a room. If scanned again later, it can reveal whether any objects in the room have been moved. Last year, scientists in Germany proposed that countries use this method to audit other countries' management of their nuclear weapons stockpiles. For example, this would be a way to know if someone had moved a nuclear warhead.

Luana Olivieri of Loughborough University is exploring the use of another form of non-visible electromagnetic radiation, terahertz waves. Terahertz waves have shorter wavelengths than radio waves but longer wavelengths than visible light. Dr. Olivieri said, "This wavelength is particularly unexplored." She added that this form of radiation can be used to penetrate objects and analyze materials. In theory, such a system could even identify specific drugs by detecting their chemical structure.

While seeing through materials can help rescue robots find trapped people in future disasters, it also has other applications. Police forces and the military could use radio-based technology to see through doors and walls to some extent. "War is definitely one of the markets it caters to. It can be used to find and kill someone," Mr. da Silva said. He said he has already demonstrated Wavsens' technology to the U.S. Department of Defense and the Israeli Ministry of Defense.

However, Professor Reinhard believes that these applications are not entirely surprising, noting that a range of emerging technologies have in principle made it easier to detect and attack people. "Maybe radar sounds scary, but drones and cheap cameras are more dangerous things," he said.