With global investment in AI infrastructure reaching unprecedented scale, objective and peer-reviewed performance evaluation is essential for guiding architectural design and procurement. MLPerf has established itself as the definitive scoreboard for machine learning systems, bridging the gap between raw silicon capability and production workload performance. This talk provides an inside look at the design principles behind the MLPerf benchmark suites and examines some of our newer efforts, such as MLPerf Endpoints and MLPerf Power.

GPU-accelerated systems have grown heterogeneous at every level within the SM, across the GPU, within the node, across the rack and across the system. Over the same period, the number of agents that move data has grown from the CPU alone to include copy engines, SMs, the Tensor Memory Accelerator (TMA), GPU-initiated network operations and in-switch reduction, and data movement is increasingly started from inside the kernel rather than by the host. This talk follows one workload, a GEMM followed by a cross-rank reduction, through five generations of NVIDIA hardware and software from 2006 to the present. For each generation it identifies the new data movers, the measurement tools available and what those tools could not observe. Finer-grained parallelism requires finer-grained performance measurement, and so measurements have moved inward, from library interposition to stream timelines to in-kernel instrumentation, and each step costs more overhead and requires more hardware-specific knowledge. Despite recent progress in in-kernel tracing and multi-GPU profiling, correlating events across devices, attributing link contention, and observing in-network computation remain open problems.

Every AI accelerator has its own architecture, memory hierarchy, and often its own kernel language, so reaching peak performance has meant rewriting and re-tuning kernels for every new chip. Helion, is an open-source Python domain-specific language (often called "PyTorch with tiles") that lets you write a kernel once and leave the hardware-specific tuning to the compiler. The compiler derives a search space of tile sizes, loop orders, pipelining, and more from the kernel itself; the autotuner then searches that space to find a performant implementation on the target hardware.

Helion can compile one source to many hardware backends, and this talk presents two new ones. On Google TPU, the Pallas backend produces attention kernels that outperform Google's optimized Tokamax library. On NVIDIA Blackwell, the CuTe DSL backend matches or outperforms the best of cuDNN, FlashAttention 4, and FlexAttention across 29 attention variants.

Much of Helion's compiler is built by AI agents. Rather than tuning individual kernels, agents hill-climb the compiler itself, turning each trade-off they find into an autotuner knob instead of a fixed heuristic, so every performance win is amortized across many kernels. We demonstrate how this agentic hill-climbing of the compiler brings state-of-the-art performance to Helion.

AI agents are becoming kernel authors. RNGD programming must expose tensor intent and resource use to the compiler, then return correctness and execution cost to the author. RNGD Virtual ISA (vISA) expresses tensor semantics, mapping, data movement, and engine use. The furiosa-opt toolchain checks and lowers that program, then surfaces diagnostics, schedules, correctness results, and runtime profiles.

Two RNGD case studies show programmability in practice. The NPU Model Optimization Competition gives external developers a path to build and evaluate kernels. In an internal Mistral-Nemo-12B study, an AI agent iteratively optimized kernels and the serving engine using compiler and runtime feedback, bringing performance close to the modeled attainable ceiling.

Edge-AI accelerator performance depends on the complete deployment stack—including model conversion, quantization, compiler, runtime, host, and execution scenario—rather than accelerator specifications alone. We present AI-MicroBMT, a deployment-aware benchmarking framework for characterizing versioned edge-AI accelerator stacks. AI-MicroBMT keeps dataset validation, query generation, task-metric computation, and result logging common across backends, while a thin Hardware Abstraction Layer (HAL) confines vendor-specific conversion, preprocessing, and inference with output adaptation. The framework evaluates feasibility before performance and separately measures latency-oriented Single-Stream execution, throughput-oriented Offline execution, and requested finite-N concurrency scaling. We evaluate six non-CPU accelerator stacks and an Apple M4 CPU reference using 2,440 measurements across 67 classification variants, activation and resolution sweeps, fixed-N workloads, and 20 YOLO-family detection models. The results show that compilation success does not ensure a useful deployment, that Single-Stream and Offline measurements can select different leading stacks, and that Offline throughput does not predict finite-request concurrency scaling. Our platform is available at https://www.ai-bmt.com.