The Device Chronicle interviewed Amit Goel, Global Director of Product Management, Nvidia Jetson on the enterprise use cases for the Nvidia Jetson Edge AI Platform.
This special executive interview marks the fact that Mender has built a reference integration with the Nvidia Jetson AI platform to enable robust and secure OTA software updates to IoT device fleets at scale. Learn more about the reference integration here.
Take a look at the integration in this video.
Amit begins by describing the key industrial areas where the AI applications based on the Nvidia Jetson embedded platform are used.
Robotics and industrial automation and Nvidia Jetson Edge AI
Amit says that robotics in industrial processes is emerging quickly. He says “You look at companies such as 6 River systems where this technology is supporting building arms and autonomous robots for logistics. These robots move around and help staff pick up things and bring them to the stations for packaging. 6 River Systems have deployed this technology based on Jetson to deploy and manage that scale for mobile robots in logistics in huge warehouses at Crocs and Lockheed Martin.
Nvidia has also been learning about what works from strategic partnerships with Fanuc, Kawada Technologies, Komatsu and Musashi Seimitsu. The learnings helped shape the industrial version of the Nvidia Jetson in the shape of the TX 2. This model has an industrial environment rating, is designed for higher shock and is IPC Class with ECC and DMC integrated. Amit says “These were very great experiences that we started off our journey on the TX, working with all these industries and understanding the requirements. There are a number of things that are an emerging part of industry. These include optical inspection, predictive maintenance, or in the agricultural space we see automated weeding and automated harvesting autonomous control of the tractor safety applications. Workers safety is also a big focus where you are working around all this heavy equipment all day long and you have rules and regulations that can be supported and enhanced by the technology. There’s still a big gap in terms of safety and there are lots of applications there on the safety side.
The Xavier Industrial was launched with a safety component added to it now. So if you want to design for safety with a certified solution, Nvidia has an additional package for a product based on hardware that has really high levels of safety hardware. Nvidia has ISO 26262 certification for their processors and have brought this tremendous seal of approval into the industrial market. Amit observes that all the various industrial processes and protocols can be automated and enabled with AI. He says “Huge improvements have been seen in machine vision optical inspection areas. Nvidia announced a big partnership with BMW leveraging the Jetson channel to manage everything from the arms that are moving these large amounts of pallets around their manufacturing line for just in time reconfiguration in these factories. BMW can’t have a fully established workflow because every product requires some customization. So the only way they can do this is to do real time adjustment to their factory. Bring the stuff specifically for individual cars. Autonomous mobile robots are used for this purpose and they are doing inspections of these cars using Jetsons. BMW also has a project where they are doing pick in place. Amit explains “So all the objects that you need, they’re all in a bin, you need to assemble it together to make a kit of sorts and that is specific for that car. So that’s being done with when the robot gets in as well.”
Smart and safe cities and Nvidia Jetson Edge AI
The second area is in smart cities and Amit references the work of companies such as Boulder AI – described in this Device Chronicle interview with their expert Dan Walkes – and many others participating in the Nvidia Metropolis program. These companies provide Safe City, Smart City, and Safe City solutions. An example deployment would be using cameras at the intersection that are tracking the traffic and base and that controlling the light or monitoring people for their movement to ensure they’re safe or controlling the street lights for that matter, right based on. “If there is no activity or person in a certain area then don’t need to keep the lights on, so you can optimize energy usage.”
Retail and Nvidia Jetson Edge AI
The third area is in retail: There are applications on the analytics side and on the autonomous side. Amit explains that autonomous stores are stores where you want to reduce the queues and you just want the shopper to just pick up the product, put it on a scanning device and walk out of the store. So all of that is now being fueled by AI where the checkout counter with its cameras and computer vision can actually detect the pick ups, identify the product and bill it to the consumer.
Then on the analytics side, you want to see which shelves are empty as a big thing for retailers is lost opportunity because the product was not there and given the size of these stores, it is hard to keep up with the store’s inventory. It is often limited by the staff available. Now, with robotics and other even fixed cameras, retailers are able to look at that inventory in real time and see if a specific stock is almost out so to get it ordered immediately. So there are lots of elements of inventory management, where you want to identify stock outs before they happen. Retailers may also want to check for pricing, where the product is placed, and to see if the pricing has been updated. Inventory management is another area. You also want to see how shoppers are moving within the store to identify opportunities. These are the areas that are hotspots where shoppers are spending a lot of time.
Some sellers pay these retailers for putting specific items in specific places. Now you can see the ROI. Sellers won’t pay more money to put their products on under-performing shelves and now the retailer has the anonymised data, all running on the edge. Continuously processing all this high bandwidth data coming in in real time and sending it to a cloud would not be not feasible from a cost perspective. Lots of use cases in retail involve analyzing customer behavior and deciding where to position products as a result. To get the maximum benefit, you may need to change the layout of the store. The customer behavior is monitored and the customer experience improved. The cyber physical world is here where people have expectations from the digital world. “I go online boom, check the product out, I go to a store and need to go through all the hoops in order to buy something and they’re trying to reduce that friction, make it seem less so that your physical experiences are as good as a digital experience.”
Medical devices and Nvidia Jetson Edge AI
The medical devices field has been growing slowly due to regulation. But in the last 18 months of the pandemic, it has accelerated. For example, as Amit explains, a company called Nanopore made a “pretty awesome product”. It’s a handheld DNA sequencing device that was deployed initially at airports when there weren’t enough tests to identify whether a patient had Covid or not. Amit says “Imagine doing the entire DNA sequence within minutes on the device. It’s just so powerful.” Lots of applications have emerged in healthcare. Everything ranging from safety where you want to ensure that the person is wearing PPE. Amit explains that there is a need to track if patients are maintaining social distance and or if they have touched any surfaces. Because a patient comes in and tests positive, you want to ensure that you can trace it back to where and what they have touched. So you can send the cleaning crew to go clean that area, or sanitize the hospital with UVD, coming in, autonomously going and cleaning the surfaces where it is unsafe for humans to go. There’s a huge shortage of workforce to attend to these patients. So lots of applications are emerging for serving patients: this could be a robot or a small device within the room, the patient can talk to it, and help assess patient condition. See the Device Chronicle article on AI telemedicine device the Clinomic Mona. Amit adds “If somebody accidentally falls off the bed, it will recognize that and send an alert to the assistance to come and help initiate sensing the body temperature and tracking the movement of the patient to make sure nothing is wrong. Lots of medical instruments are now adding AI capabilities for scanning and ultrasound endoscopy. All of these are now being assisted with AI. So the diagnosis and the scan itself can be better done.”
Strategic end to end solution
Amit says that strategically Nvidia is thinking about all of these problems from an end to end perspective. “We are not just thinking about that edge device in isolation. We do think about managing these devices at the edge but we also know that the robot is getting about. So we also have to think about simulation, training, and finding the best ways to manage the devices to scale. This is why Nvidia is investing in making our stack cloud native so these things can be done. We have seen how successful Cloud has been and how successful the growth in Cloud application services has been.” Amit believes the big reason for this has been the growth of dev ops making it so easy to scale things, and develop and manage things in the cloud with limited resources. Nvidia thinks about it all the way from the beginning of the developers journey to the end and builds solutions all along the way. Amit concludes that Nvidia identifies problems that are hard to solve and if there’s a good match they go execute and solve the problem. “Robotics and smart manufacturing with the coming of 5G is one to watch. “Nvidia is bringing 5G to the edge so it’s an area that we believe is a hard problem and we have all the right ingredients to solve it. So we are investing in this.”
Thoughts on OTA software updates and Nvidia Jetson Edge AI
Amit observes that the workflow and dev ops of AI is very different to that of traditional embedded engineering. “In embedded, you build the product, you ship it and you know you don’t have to touch it for years to come. This has changed with AI. And with the application on the device, we are investing heavily in software. And you can see that from our real software releases that we do. We do a major release every six months if not sooner and all these devices that are out there in the system in the field today, these are not just bug fixes, these are performance improvements, new features that are being added.”
Amit believes that it is extremely important for customers to be able to perform secure and robust OTA software updates to the field, in order to leverage the best of AI and all the goodness from the billions of dollars that Nvidia is investing in software. “Our customers want to leverage this. For example, with Xavier, when we launched it to the latest version of Jetpack, you ran the same workload, the performance would be 50% more today. How will you do that if you don’t perform an over the air update of the latest stack onto the devices? An AI inherently is a process that is recurring. You deploy, you collect data, you train and you update. It’s a cyclical thing that requires constant updates, constant training and so the role of OTA updates is even more critical because just the nature of AI being a cyclical process, you have to do that. So we’re excited. Customers have thousands of devices. They need a solution and that’s why we invested in all the underlying tools and now with Mender we are bringing in a more turnkey solution. Building on these tools, I’m really excited about how our customers will be able to now deploy more of the devices that scale and manage them more easily.”
We wish Amit and the Nvidia Jetson product management team well on their journey to realising the power of AI and edge devices in diverse industrial applications.