NVIDIA HQ
NVIDIA announced that it is acquiring Run:ai, an Israeli startup that built a GPU orchestrator based on Kubernetes. Although the price has not been disclosed, there are reports that it is worth between $700 million and $1 billion.
The acquisition of Run:ai highlights the growing importance of Kubernetes in the creative AI era. This makes Kubernetes the de facto standard for managing GPU-based accelerated computing infrastructure.
Run:ai is a Tel Aviv, Israel-based AI infrastructure startup founded in 2018 by Omri Geller (CEO) and Dr. Ronen Dar (CTO). It has built an orchestration and virtualization platform tailored to the specific needs of AI workloads running on GPUs, efficiently pooling and sharing resources. Tiger Global Management and Insight Partners led a $75 million Series C round in March 2022, bringing the company's total funding to $118 million.
Run the problem: ai solves.
Unlike CPUs, GPUs cannot be easily virtualized so that multiple workloads can use them at the same time. Hypervisors such as VMware's vSphere and KVM enable the emulation of multiple virtual CPUs from a single physical processor, giving workloads the illusion that they are running on a dedicated CPU. When it comes to GPUs, they cannot be efficiently shared across multiple machine learning tasks, such as training and inference. For example, researchers cannot use half of the GPU for training and testing while using the other half for other machine learning tasks. Similarly, they cannot aggregate multiple GPUs to make better use of available resources. This is a huge challenge for enterprises running GPU-based workloads in the cloud or on-premises.
The problem described above extends to containers and Kubernetes. If a container requires a GPU, it will effectively use 100% of the GPU if it is not being used to its full capacity. The lack of AI chips and GPUs exacerbates the problem.
Run:ai saw an opportunity to effectively solve this problem. They used Kubernetes' ancient and proven scheduling mechanism to create a layer that allows enterprises to use only a fraction of available GPUs or to pool multiple GPUs. This results in better utilization of GPUs, providing better economics.
Here are 5 key features of the Run:ai platform.
- The orchestration and virtualization software layer is tailored to AI workloads running on GPUs and other chipsets. This allows efficient pooling and sharing of GPU compute resources.
- Integration with Kubernetes for container orchestration. Run:ai's platform is built on Kubernetes and supports all Kubernetes variants. It also integrates with third-party AI tools and frameworks.
- A central interface for managing shared compute infrastructure. Users can allocate computing power to clusters, pool GPUs and different tasks through Run:ai's interface.
- Dynamic scheduling, GPU pooling and GPU fractionation for optimal performance. Run:ai's software enables GPUs to be segmented and allocated dynamically to optimize utilization.
- Integrations with Nvidia's AI stack include DGX Systems, Base Command, NGC Containers and AI enterprise software. Run:ai has partnered closely with Nvidia to offer a full-stack solution.
Notably, Run:ai is not an open source solution, even though it is based on Kubernetes. It provides customers with proprietary software that must be deployed in their Kubernetes clusters along with a SaaS-based management application.
Why did NVIDIA acquire Run:ai?
NVIDIA's acquisition of Run:ai strategically positions the company to strengthen its leadership in the areas of AI and machine learning, particularly in the context of optimizing GPU utilization for these technologies. The main reasons NVIDIA pursued this acquisition are:
Improved GPU orchestration and management: Run:ai's advanced orchestration tools are critical to managing GPU resources more efficiently. This capability is critical as the demand for AI and machine learning solutions continues to grow, requiring more sophisticated management of hardware resources to ensure optimal performance and utilization.
Integration with NVIDIA's existing AI ecosystem: By acquiring Run:ai, NVIDIA can integrate this technology into its existing suite of AI and machine learning products. This enhances NVIDIA's overall product offerings, better serving customers who rely on NVIDIA's ecosystem for their AI infrastructure needs. NVIDIA HGX, DGX and DGX Cloud customers will have access to Run:ai's capabilities for their AI workloads, specifically for generative AI workloads.
Expanding Market Access: Run:ai's established relationships with key players in the AI space, including their prior integration with NVIDIA's technologies, gives NVIDIA greater market reach and the ability to serve a wider array of customers. This is especially valuable in sectors that are rapidly adopting AI technologies but face challenges in resource management and scalability.
Development of Innovation and Research: The acquisition enables NVIDIA to leverage the innovative capabilities of Run:ai's team, known for its pioneering work in GPU virtualization and management. This could lead to further advancements in GPU technology and orchestration, keeping NVIDIA at the forefront of technological advancements in AI.
Competitive advantage in a growing market: As enterprises increase their investment in AI and machine learning, efficient GPU management becomes a competitive advantage. NVIDIA's acquisition of Run:ai ensures that it remains competitive against other tech giants entering the AI hardware and orchestration space.
By acquiring Run:ai, NVIDIA not only expands its product capabilities, but also solidifies its position as a leader in the AI infrastructure market, ensuring that it keeps pace with technology innovations and market demands. I was at the forefront.
What does this mean for Kubernetes and cloud native ecosystems?
NVIDIA's acquisition of Run:ai is important for Kubernetes and the cloud native ecosystem for several reasons.
Improved GPU orchestration in Kubernetes: The integration of Run:ai's advanced GPU management and virtualization capabilities into Kubernetes will allow for more dynamic allocation and efficient use of GPU resources in AI workloads. This is consistent with Kubernetes' capabilities in handling complex, resource-intensive applications, particularly in AI and machine learning, where efficient resource management is critical.
Advances in Cloud-Native AI Infrastructure: By leveraging Run:ai's technology, NVIDIA can further expand the Kubernetes ecosystem's ability to support high-performance computing (HPC) and AI workloads. This synergy between NVIDIA's GPU technology and Kubernetes will potentially provide a more robust solution for deploying, managing and scaling AI applications in cloud-native environments.
Wider adoption and innovation: The acquisition could lead to wider adoption of Kubernetes in sectors that rely heavily on AI, such as healthcare, automotive and finance. The ability to efficiently manage GPU resources in these areas can lead to faster innovation and deployment cycles for AI models.
Impact on Kubernetes Maturity: The integration of NVIDIA and Run:ai technologies with Kubernetes underscores the platform's maturity and readiness to support modern AI workloads, reinforcing Kubernetes as the de facto system for modern AI and ML deployments. Available. This may encourage more organizations to adopt Kubernetes for their AI infrastructure needs.
NVIDIA's move to acquire Run:ai not only strengthens its position in the AI and cloud computing markets, but also expands the Kubernetes ecosystem's ability to support the next generation of AI applications, making it one of the industries A wide range benefits.