OpenAI's first custom inference chip, Jalapeño, published benchmarks against Nvidia's GB300

At Hot Chips 2026 on 25 August, OpenAI presented Jalapeño, an inference ASIC co-developed with Broadcom, with its first published benchmark numbers: a 700W part claiming up to 1.9x the throughput per kilowatt and 3.6x lower latency than Nvidia's 1,400W GB300. OpenAI's head of hardware said initial deployment is late 2026 in small volumes, with the significant rollout in 2027. Broadcom handled silicon implementation and networking; Celestica did board, rack and system integration. The same session week carried Microsoft's Maia 200, Google's TPUv8 and SambaNova's SN50 — every large buyer now showing its own inference silicon.

What it means Inference is now most of the compute bill, and the biggest buyer of it has shipped a part that claims to beat the merchant GPU on performance per watt. Serving prices follow that curve, not the training one.