NVIDIA NIM is NVIDIA’s microservices layer for running model inference in optimized containers. The microservices run on the customer’s infrastructure — cloud, data center or an RTX GPU workstation — and are licensed through NVIDIA AI Enterprise for production use. NVIDIA also keeps a hosted endpoint catalog for testing, build.nvidia.com.
What it does
- Standardizes how open models and NVIDIA-family models run in ready-made containers
- Moves the same OpenAI-compatible endpoint across clouds and on-premises environments
- Lets teams test models in the hosted catalog before deciding where to deploy
How it works
- Each model ships in a container with a standardized inference server, GPU profile and API
- The microservices integrate with Kubernetes and the orchestrators already used by the customer
- The hosted catalog issues API keys for prototyping and runs on NVIDIA infrastructure
- The certified edition adds enterprise support and long-term update cycles
Strengths
- A production-ready package, with no inference server to assemble from scratch
- Portability across cloud providers and local environments
- The hosted catalog lets teams evaluate models without installing anything
Points of attention
- It is software to deploy, not a platform that hosts the model for you; production requires a license
- Hosted endpoints are meant for evaluation and prototyping
- Not to be confused with NVIDIA’s own GPU cloud or with routing gateways