Overview
The NVIDIA AI Factory Observability (AIFO) integration monitors NVIDIA GPU hardware, NVLink networks, and application-level AI resource consumption across on-premises and cloud-hosted environments. AIFO tracks a wide range of GPU metrics and configuration details, including utilization and power consumption.
GPU network monitoring
In addition to GPU hardware monitoring, AIFO monitors NVLink configuration and throughput. This visibility helps you identify bottlenecks in high-performance GPU networks.
Workload resource monitoring
AIFO tracks GPU, memory, and CPU utilization for the training and inferencing programs that use GPU resources. This tracking supports detailed reporting on both program-level and per-GPU usage.
Token usage monitoring
AIFO monitors token usage from containers running on GPU hardware, such as NVIDIA NIM. It can also extract token usage from OpenTelemetry traces sent by any application, giving you visibility into application-level AI resource consumption.
Supported deployment environments
AIFO can monitor GPU hosts that exist on-premises or on cloud-hosted AI resources.