The NVIDIA GTC Conference, in March, was undoubtedly the most anticipated AI conference of the year. Tech leaders, experts, developers, innovators, and enthusiasts were out in full force to see just how far and wide AI is expanding. The high-energy event proved a testament to the transformation and profound influence AI and Gen AI drive across sectors.
Jensen Huang’s keynote was packed with product announcements, from the “Blackwell” GPU (built on the next-gen Hopper architecture) to “NIMs” (NVIDIA Inference Microservices) and the expanded capabilities of Omniverse Cloud APIs.
Beyond Just Silicon: A Richer Playbook
NVIDIA has transcended by building a robust ecosystem of valuable intellectual property (IP) beyond its impressive silicon, including deep learning frameworks, industry-specific software platforms, hefty investments in AI research, and much more.
Here’s my summary of NVIDIA GTC announcements relevant to edge compute companies:
Hardware Powerhouse: NVIDIA Blackwell GPU
The NVIDIA Blackwell GPU, unveiled at GTC, promises a significant performance leap over its predecessor, the A100 GPU. This powerhouse GPU sits at the top of NVIDIA’s data center and AI offerings, making it ideal for large cloud providers, large enterprises with high-performance computing needs, and national laboratories and research institutions.
However, this power level is overkill for most edge compute applications, such as computer vision, autonomous machines, and medical imaging analysis. In these cases, NVIDIA’s existing platforms, the IGX Orin™ and Jetson, offer sufficient GPU capabilities to run these applications smoothly without encountering performance bottlenecks.
Software Suite for Streamlined Development: NVIDIA AI Enterprise 5.0
Edge compute thrives on efficient development and deployment workflows. NVIDIA addressed this need with the announcement of NVIDIA AI Enterprise 5.0 software suite, which is shared across all platforms, including the NVIDIA IGX Orin™ and the NVIDIA Jetson. This comprehensive suite provides a range of tools and frameworks specifically designed to streamline the process of building and deploying AI applications at the edge.
Notable tools include:
- NVIDIA Inference Microservices (NIMs): A set of easy-to-use microservices designed to accelerate deployment of generative AI models across cloud, data center, and workstations. Supporting a wide range of AI models, including NVIDIA AI Foundation and custom models, it ensures seamless, scalable AI inferencing, on-premises or in the cloud, leveraging-industry standard APIs.
- Visual Insight Agent (VIA): A collection of workflows to build AI agents capable of processing large amounts of live or archived videos and images with Vision-Language Models (VLM), whether deployed at the edge or cloud. This new generation of visual AI agents will help nearly every industry summarize, search, and extract actionable insights from video using natural language.
- NVIDIA cuOpt: A microservice to accelerate logistics and supply chain optimization to save time and reduce infrastructure costs.
Arrow: Steering Customers Through the Options
Choosing the right AI solution is becoming increasingly complex. While NVIDIA’s high-end offerings appeal to customers across various sectors, they often carry a substantial price. Not every scenario necessitates NVIDIA hardware. There’s also a growing trend for NVIDIA to monetize its software and platform IP through recurring fees.
Further complicating the decision is the need to choose between building a custom solution in-house or purchasing a pre-built system from a vendor. This approach is also prevalent in the embedded edge market, with many of Arrow’s suppliers offering a wide range of solutions featuring the latest Jetson and IGX products.
Although there are many factors to consider, Arrow can help. Our experts guide you through the complex landscape of options and work with you to find the best fit for your specific needs and budget.
