AI chip has long been the focus of chip giant. Earlier, Nvidia announced that it will integrate some Nvidia technologies into ARM chip design, especially for IOT devices to realize AI processor optimization. Lei Feng network news, recently high Qualcomm launched a visual intelligent platform, carrying the use of 10nm FinFET process technology, specialized for IoT - oriented SoC. QCS605 and QCS603 SoC integrate ISP, Qualcomm AI Engine and connection and security technology. It can provide powerful computing power for terminal side camera processing and machine learning, and is applied to a wide range of Internet of things applications.
How can the new high pass SoC platform make the IoT device also AI? Seshu Madhavapeddy, a high Qualcomm IoT product manager, has made a further explanation for this.
Why do you need a visual intelligent SoC platform?
As PC eliminated high value computing on the mainframe in 1980s, IOT devices and chips supporting these devices are proving that AI computing can also be done at the edge, rather than just in the cloud. The earlier it breaks the unique position of the data center in AI, the AI can be accepted earlier and easily by ordinary end users and developers.
Madhavapeddy indicates that the new SoC platform is designed for IoT edge computing. By making more computation in devices, the data transmission to remote infrastructure can be reduced as much as possible, so that the Internet of things terminals will become more intelligent. QCS605 and QCS603 are mainly for the field of intelligent vision. They are applied to security cameras, motion cameras, wearable cameras, virtual reality (VR) 360 degree and 180 degree cameras, robots and smart displays.
Madhavapeddy further explained that the new platform is due to the big difference between the IoT camera and the mobile camera. First, unlike the mobile phone camera, IoT camera works in low light environment; secondly, IoT camera image stabilization demand is not the same, a snapshot of IoT camera required is not ambiguous, but the helmet motion in the scene on the video shoot, or UAV during the mission of the camera is clear and real-time feedback.
High pass visual intelligent platform (Qualcomm Vision Intelligence Platform)
Detailed explanation of QCS605 and QCS603
High pass AI Engine
High pass's new visual intelligent SoC platform integrates the Qualcomm artificial intelligence engine AI Engine, and AI Engine is composed of several inherited hardware and software components. Including Xiaolong neural processing engine (NPE) software framework, which includes the TensorFlow, Caffe and Caffe2 framework analysis, optimization and debugging tools, ONNX (Open Neural Network Exchange) and Android Neural Network exchange format, API Qualcomm and Hexagon Neural Network library. AI Engine also includes the software development kit SKD. Of course, Qualcomm hopes that developers and OEM vendors can easily access the training network to the platform.
Based on the software framework of AI Engine and Xiaolong neural processing engine, the visual intelligent platform can provide deep neural network inference with computing performance of up to 2 trillion and 100 billion times per second (TOPS). At the same time, the artificial intelligence engine can also optimize the reasoning, that is, the image data is graded based on the machine learning model deployed to the device. Of course, model training will still be carried out in a cloud with a large number of GPU performance.
Madhavapeddy shows that the image is not just static, but also dynamic video, but both 603 and 605 have strong video performance. 605 support video streams of 4K@60fps and 1080p@60fps, 4K@30fps, and 720p@30fps, as well as multiple concurrent video streams with lower resolution.
To achieve extraordinary image quality, the visual intelligence platform integrates a dual 14 bit Qualcomm Spectra 270ISP that supports two 1600 megapixel sensors.In addition, the visual intelligence platform includes visual processing capabilities necessary for the Internet of things subdivision domain, such as preventing the emergence of high dynamic range video
Heterogeneous computing architecture
The heterogeneous computing architecture of the QCS605 chipset of the visual intelligent platform is integrated with eight core Kryo 360 CPU, Adreno 615 GPU and Hexagon 685 vector processors. The integrated display processor of the visual intelligent platform can provide hardware acceleration synthesis, 3D coverage for a series of displays, as well as mainstream graphics API support including OpenGL, OpenCL and Vulkan. The architecture supports a variety of advanced operation systems, which helps developers and manufacturers easily build differentiated features in their solutions, such as VR 360 degree camera's end to side content stitching, autonomous robot navigation and obstacle avoidance, and motion camera video summarization.
QCS603 and 605 are integrated Adreno GPU, multi-core Kryo CPU, Hexagon 685 vector processor, and high pass AI Engine, all of which belong to low-power products designed for battery powered products. In contrast, 605 is 8 core CPU, 603 is dual core CPU, 603 has lower running power, smaller chip, and more functions with lower specifications.
Wi-Fi, audio, and security features
According to Lei Feng network, the Qualcomm vision intelligent platform is all board mounted Wi-Fi, 605 has MU-MIMO and dual frequency concurrency characteristics of 2x2 802.11ac Wi-Fi, 603 has 1x1 802.11ac Wi-Fi.
On the audio side, it supports the Qualcomm 3D Audio Suite, the Qualcomm Aqstic audio technology and the Qualcomm aptX audio. The platform also has a Qualcomm noise suppression and echo cancellation technology, to the terminal side audio analysis and processing characteristics, support Natural Language Processing, audio speech recognition and insertion capabilities create reliable voice interface, even in noisy or noisy environment, or when the user is far away from the terminal can be realized.
Visual intelligent platform security features based on hardware can support networking terminal reliable through the following functions, including hardware root of trust safe start, trusted execution environment, hardware encryption engine, storage security, through the life cycle control debugging security, key distribution and wireless security protocol.
Lei Feng network reports viaZDNet