OpenAI’s first custom AI accelerator, Jalapeño, has reached public benchmarking, giving the industry its clearest look yet at how the company plans to reduce the cost and latency of large-scale model inference.
Jalapeño was designed by OpenAI and implemented with Broadcom, with Celestica contributing board, rack and system expertise. OpenAI introduced the chip in June as the first product in a multi-generation computing platform intended to begin deployment by the end of 2026.
Promising results, within a defined benchmark
New results published through SemiAnalysis’s InferenceX benchmark compare Jalapeño with commercial accelerator systems across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. OpenAI says the chip delivered greater peak mixed-token throughput per kilowatt and lower token latency than the compared systems.
The chart and detailed figures were reproduced in ifanr’s report. For a DeepSeek R1 test with an 8K-token input and 1K-token output, the report lists end-to-end latency falling from 5.99 seconds on the comparison system to 1.65 seconds on Jalapeño, while the minimum interval between generated tokens fell from 5.90ms to 1.43ms.
Those results should not be generalised into a claim that Jalapeño is faster than every GPU in every workload. Inference performance changes with model, batch size, quantisation, latency target, software stack, rack design and power accounting. SemiAnalysis conducted the testing with OpenAI engineers, and the comparisons reflect the specific systems and settings disclosed by the benchmark.
OpenAI’s own measured-results post similarly describes higher peak throughput per kilowatt and lower latency on the tested models, rather than making an unrestricted performance claim.
Built around the bottlenecks of LLM inference
Jalapeño is intended for inference rather than model training. Large-language-model inference typically alternates between a compute-heavy prefill phase, which processes the prompt, and a decode phase that repeatedly moves model weights and key-value cache data while generating tokens.
OpenAI says the architecture was designed to reduce data movement and balance compute, memory and networking resources so that more of the chip’s theoretical capability is realised in production workloads. SemiAnalysis reports 216GiB of HBM4 memory and 15.4TB/s of memory bandwidth, although those detailed specifications come from the research firm rather than OpenAI’s initial product announcement.
The design partitions compute and memory resources so that model data can remain close to the cores responsible for processing it, using high-speed communication when information must cross regions. The goal is to spend less energy and time moving data between compute units, memory and neighbouring accelerators.
SemiAnalysis lists a 700W nominal power envelope and says sustained power during its testing remained below 550W. The meaning of performance-per-watt comparisons still depends on whether power is measured at the chip, board, server or facility level; readers should use the benchmark’s stated “all-in utility” methodology when comparing its charts.
A nine-month tape-out accelerated by AI tools
OpenAI says Jalapeño moved from initial design to manufacturing tape-out in nine months. The company attributes that pace to close software-hardware co-development, Broadcom’s implementation experience and the use of OpenAI models during parts of design exploration, verification and optimisation.
The nine-month figure covers the design-to-tape-out stage, not the entire programme from team formation to deployed systems. Semiconductor development also includes fabrication, packaging, bring-up, validation, software enablement and production qualification after tape-out.
According to ifanr’s reconstruction of the timeline, OpenAI received first silicon in 2026 and spent several months bringing the chip up and adapting additional models. The publication says Codex and an unreleased model were used to help port workloads that were not in the accelerator’s original enablement plan. Those detailed timing and tool-use claims should be read as reported information beyond OpenAI’s shorter official account.
Broadcom partnership targets multi-generation deployment
OpenAI supplies the accelerator architecture and its knowledge of model-serving workloads. Broadcom contributes silicon implementation, networking and connectivity technology, including Tomahawk networking components, while Celestica helps industrialise boards, racks and systems.
The companies’ broader collaboration covers up to 10 gigawatts of OpenAI-designed accelerators and networking systems. Broadcom’s 2025 announcement said rack deployments were targeted to start in the second half of 2026 and continue through 2029.
For OpenAI, the strategic attraction is control over a larger part of the inference stack. A processor designed around its serving patterns could reduce operating cost, increase available capacity and lower response times for products such as ChatGPT, Codex and the API. Those benefits will depend on production yields, system reliability, software maturity and how closely public benchmark results translate to deployed workloads.
Jalapeño’s early results are therefore significant but not the final verdict. They demonstrate that a first-generation custom accelerator can be competitive in demanding inference tests; broader conclusions will require reproducible benchmarks across more models, latency targets and complete data-centre systems.
Sources: OpenAI’s Jalapeño announcement; OpenAI benchmark commentary; SemiAnalysis InferenceX; Broadcom; and ifanr.
