Last year's Google I / O on, Google mysterious ATAP sector ((Advanced Technologies and Projects) have been very cool show Project Soli. Because the technology was still in its early stages, but a small scale to 60 developers distributed development kit for internal testing. in this year's I / O 2016, Google in addition to bringing Daydream, Android N and other heavy update, ATAP also showcase the latest developments in low-key Project Soli's. today, we come to a grilled steak Project Soli's context, let us understand the principles and application of this technology.
What is Soli
Soli is a miniature radar monitoring of the use of air gestures of new sensing technology. The specially designed radar sensor can track sub-millimeter accuracy, high-speed motion, and then the radar signal after various treatments, identified as a common set of interactive gestures, wearable and easy to control a variety of miniature devices.
Currently Google designed these generic interactive gestures, people usually are based on the familiar tools and some physical action, such as buttons, dials and sliders:
While these actions are virtual, but because it is a mutual action between a plurality of fingers, it can give users a good tactile sense and physical feedback. But also because human beings have precise small muscle motor skills (fine motor skills), it allows these virtual tools to achieve high accuracy and smooth movement through our gestures.
Google believe that these physical and tactile feedback interaction is more natural than a touch screen or voice recognition technology to control the plane, the man-machine interface introduces new ideas and opportunities.
Soli and other tracking technologies differ gesture
Currently on the palm and finger motion recognition and tracking technology, the market has been several different solutions:
Compared to these types of common solution, using the millimeter-wave radar technology Soli has the following advantages:
Dependent infrared depth-sensing and projection technology in outdoor infrared interference and more environmental reliability is poor, the millimeter-wave radar no problems in this area.
Based on three-dimensional optical imaging techniques require considerable computation to obtain depth data, difficult to achieve high resolution, low power consumption is not; and because the dependence of visible light, can not be used in a low light environment, millimeter radar no problems in this area.
And because the frequency is much lower than the millimeter-wave radar and infrared visible light, infrared-based compared to time of flight technology, millimeter wave radar can calculate the phase shift (Phase shift) and the Doppler effect (Doppler Effect), thereby computing low Get the amount of movement and direction of the object.
Millimeter-wave radar for some materials there are very good penetration, without blocking the light path of.
Antenna technology in recent years because there is not a small increase, less than one square centimeter Google can put the sensor and the antenna array integrated together, which facilitates the application of this technology to wearable devices and mobile phones, and other solutions yet not so small:
The I / O 2016 conference, Google will bring a smart watch with LG cooperation in the strap at Soli inherited the technology used to capture the user's fingertip minute operation. Such size and power consumption, the traditional gesture sensing technology is unthinkable.
In this demo, the show with the keys with your fingers and slide two actions can send different instructions:
In Soli project video introduction, there have been associated with smart watch design concepts:
Is not it cool? Smart Watch limited screen size, if the touch operation on the screen directly, then fingers will block the screen; the physical buttons or dial if the design is difficult to operate. And this gesture is expected to solve the air above them.
Principle and Implementation of Soli
Technical details Soli project currently open is very small,Currently we do know is that it might take a single transmitter, multi-receiver antenna design. Obstacle in front of the antenna will be reflected radio waves back to the radar antenna and the energy of the reflected signal, the corresponding delay and the Doppler effect of the characteristics of the object, including object size, shape, materials, distance and speed.
Google introducing specific emphasis Soli Soli time unlike conventional radar sensors, and did not need a lot of bandwidth and space resolution. In fact Soli resolution than can be achieved fine finger movements less, so the motion capture finger need to analyze the reflected signal dynamic changes over time, which is also on the gesture recognition algorithm to mention higher requirements.
Radar signal from the original to the final gesture control to go through the following steps:
1, the original capture the reflected signal
2, the timing of the received signal processing and conversion to Range Doppler Map
3, feature extraction, identification, location and tracking
4, extracted from the features of the gesture recognition
Soli SDK provides a step above every program interface for third-party developers, is also compatible with different types of radars, support from the sampling frames per second of 100-10000.
Soli radar uses a frequency modulated continuous wave Frequency Modulated Continuous Wave (FMCW) modulation format. This technology is now used in automotive applications such as automatic cruise, you can achieve the object distance and speed sensors. Here we combine NVDIA recently published paper (Short-range FMCW monopulse radar for hand-gesture sensing) To explain the principles under this technology:
FMCW This format is a periodic sawtooth function modulation. Radar signal emitted from an object is reflected back to produce a difference in time and frequency. Wherein the time delay & tau; a radio round-trip time, the change in frequency fd is the Doppler effect caused by the movement of objects.
We received signal with the original signal is multiplied, then in the field of analog low-pass filter, so that it can produce significant beat signal. Then the two-dimensional FFT conversion timing signal respective to do, minus the static background environment, you can get the desired Range Doppler Map (RDM):
RDM Each cell corresponds to the current target distance and velocity. Because different objects at different distances or may each have different speeds but at a similar distance, RDM will be realized simultaneously track multiple objects effect. For example, the following figure is a moving hand, each green box can be identified finger.
If we have multiple receive antennas by comparing the phase difference between the antenna for each RDM, we can calculate the angle of arrival of each object (Angle of Arrival), before with the calculated distance, thus achieving the object of spatial orientation. If, after the traditional with the depth sensor calibration, you can obtain three-dimensional depth map.
We can put four-dimensional vector signal (three-dimensional position plus radial velocity) is mapped to a palm model, introducing the rear end of the gesture learning model for identification.
This is the five different users RDM generated four kinds of gestures, it is clear that the difference between the gesture is obvious, but making the same gesture when different users have a lot of similarities.
So this format compared to other FMCW radar has the following advantages:
1, and calculate the speed and distance, distance information accuracy.
2, Beat frequency is not high, corresponding to the sampling rate is low, reduces the difficulty of analog circuits and digital signal processing, the entire algorithm can be implemented on a low-power embedded chips.
3, the shortest wavelength of the signal associated with the measured distance, so objects can be from the antenna is very close to, or even attached to the antenna.
Thus, this FMCW technology Soli uses radar base its miniaturization and low power consumption.
Site's visibility on Google early 2014 prototype machine was a huge box, and then took just over a year after the system is reduced to less than one square centimeter, which includes a variety of antennas and sensors.
Google for Soli's goal is to create a low-cost, mass production and very low power miniature radar chip, expectations can be integrated into the smallest mobile device inside. Through two years of development it can be said that Google has from this goal very near.
The following finishing some of the early developers in the last year using the dev kit carried out a variety of interesting applications:
Because the millimeter-wave radar to penetrate part of the material has a certain role, reflected signals are also some differences, there are developers to design accordinglyMaterial DetectorNot only can identify a variety of plastic and metal, but also to identify the water with milk.
Car gestureIs an interesting application, compared to visible and infrared light technology, Soli can be the perfect solution under different lighting environment is unstable conventional technical problem, but capture the subtle movement radar has a strong advantage.
In Soli help us to interact with the addition of touch screen mobile phones a new dimension, Hover your finger on the screen can now be accurately capture and predictable.
Since Soli can scan and calculate the depth map, why not bring make 3D scanning and imaging applications?
This application is very cool, the air to draw 3D finger passwords, automatically open the drawer!
Soli use of gesture capture design a musical instrument how?
A similar project called Soli designed by the world's smallest violin! Accurately capture every bit of finger movement and then mapped to the note.
to sum up
From the start of the project in 2014, to release the I / O on the initial prototype last year to this year I / O and the various creative applications this year Soli progressed very quickly. Google claims that compared to last year's version, currently 70 percent smaller chip area, power consumption decreased by 22 times, from 1.2 watt high should not have further dropped to 0.054 watts, the algorithm has also been a 256-fold efficiency.
According to media reports, Google will be the end of next year or the new dev kit available to more developers, we believe that in the near future we will see the use of intelligent technology Soli watches, VR / AR devices and intelligent appliances appear in the market .
Another huge market is a wearable device, in fact ATAP another magical project Project Jacquard is now a tangible cooperation Levi jeans fabric, used in the manufacture of smart jacket. Soli uses radar technology just can penetrate many of the traditional textile material is likely to be a smart way to interact with standard clothing. Soli after this novel radar technology can also be fused with other RF and sensor technology may be able to create more unexpected applications.