Jalapeño: OpenAI's first AI chip beats Nvidia in efficiency – When the customer suddenly becomes the competitor

Xpert Pre-Release

Available in 27 languages 📢

Published on: August 29, 2026 / Updated on: August 29, 2026 – Author: Konrad Wolfenstein

Jalapeño: OpenAI's first AI chip beats Nvidia in efficiency – When the customer suddenly becomes the competitor – Image: Xpert.Digital

22 times more efficient: OpenAI's first in-house AI chip significantly beats Nvidia

Save energy, double performance: What OpenAI's new AI chip means for the industry

AI builds AI chips: OpenAI's "Jalapeño" outclasses the competition in record time

A chip that's completely reshuffling the cards in the multi-billion-dollar AI market: With "Jalapeño," OpenAI has developed its very first in-house AI accelerator in record time – and is challenging none other than its most important hardware partner to date, Nvidia. Initial independent tests show the new chip boasting impressive energy efficiency, significantly outperforming Nvidia's current Blackwell generation in certain areas. The real stroke of genius behind this development: OpenAI used its own advanced AI models to dramatically accelerate the chip design. What does this strategic move mean for the already strained power supply of data centers, the operating costs of ChatGPT, and Nvidia's market power? A detailed analysis reveals why OpenAI's hardware debut is far more than just a technological upgrade – it could fundamentally change the revenue model of the entire industry.

From customer to competitor: OpenAI attacks Nvidia with its own wonder chip

In less than a year and a half, OpenAI has developed its own AI chip, which, in initial independent tests, demonstrates significantly higher efficiency than Nvidia's current reference hardware. This is remarkable because OpenAI was previously one of Nvidia's largest single customers and is now entering a segment that has been almost exclusively dominated by a single vendor. After the company unveiled its first proprietary AI accelerator, codenamed Jalapeño, in June 2026, reliable performance data, collected by the independent analysis firm SemiAnalysis, is now available. This data shows that OpenAI, thanks in part to the active support of semiconductor specialist Broadcom, has developed a high-performance AI chip from scratch that significantly outperforms Nvidia's current Blackwell generation in certain metrics and also appears to be well-positioned against the upcoming Rubin platform.

The economic implications of this development can hardly be overstated, as it touches upon one of the central cost problems of the entire AI industry: the enormous discrepancy between spending on computing infrastructure and the revenue that can be generated from AI services. OpenAI reportedly burns through approximately $3.7 billion per quarter, with inference costs—the costs of continuously deploying pre-trained models in daily operations—representing the largest single expense. A chip that significantly reduces these costs not only changes a technical metric but also the entire revenue model of a company that operates ChatGPT with over 400 million active users.

An independent test bench without free rein

The tests were conducted using SemiAnalysis's own testbench, InferenceX. However, the analysts themselves were not permitted to perform the tests, nor were all scenarios tested, nor was the more informative testbench, AgentX, used, according to SemiAnalysis. This limitation is economically significant because it means that while the available data originates from a reputable company specializing in chip analysis, it was collected within a framework controlled by OpenAI. Anyone wishing to base sound investment decisions on such data should consider this context, even if the general direction of the results appears plausible.

InferenceX uses the OpenWeights models GPT-OSS, DeepSeek R1, and Kimi K2.5, suggesting that Jalapeño is not only designed for OpenAI's own models but also suitable as a general inference platform for other architectures. As anecdotal evidence of the hardware's flexibility, SemiAnalysis cites the developers' demonstration that the game Doom, ported from OpenAI's own programming assistant Codex, even ran on the chip. While such demonstrations have little economic substance, they do indicate that the architecture is not limited to a narrow application window but can handle more general computational tasks.

Efficiency gains that translate into hard cash

The results confirm Jalapeño's particularly high efficiency. In some scenarios, OpenAI's chip achieves 22 times the token rate per watt compared to Nvidia's B300, an exceptional result for a company's first proprietary chip without decades of semiconductor experience. Jalapeño can also compete with the new Vera Rubin system, which is currently being rolled out to major customers, although there is currently little reliable data available for this comparison and a final assessment is still pending.

SemiAnalysis, however, points out methodological limitations that put the significance of the figures into perspective. Only relatively small prompt sizes with 8,192 input and 1,024 output tokens were tested, while larger prompts, as are common in many production agent applications, could significantly alter the results. Additionally, the tested version ran on an early A0 stepping, while an optimized B0 stepping is currently in production, which OpenAI claims will be around 25 percent more efficient. Economically, this means that the already impressive figures likely represent a lower limit rather than an upper limit of the actual potential, assuming the announcements regarding the B0 stepping are confirmed.

The real leverage lies not in the chip, but in the power grid

OpenAI's chip could even be more cost-effective than Vera-Rubin, as Nvidia's system benefited from speculative decoding in the tests, a technique for accelerating text generation, while Jalapeño had to manage without this optimization. Overall, in-house hardware inference should make OpenAI significantly cheaper, as the company no longer has to pay trade margins to an external supplier and the architecture can be precisely tailored to its own serving patterns.

The real strategic advantage, however, lies not primarily in the pure cost savings per chip, but in energy efficiency. Due to the increased efficiency, significantly more chips can be operated with the same power input, which represents the true competitive advantage in an industry where the availability of electricity and grid connection capacity has become the limiting factor for growth. OpenAI itself has already signed contracts for more than ten gigawatts of computing capacity in the US, a goal originally set for 2029 that has now been achieved years earlier. In an environment where new power plant capacity and grid connections often constitute the real bottleneck for data center expansion, every watt used more efficiently translates directly into additional available computing power, without the need to build additional power plants.

A pace of development that is challenging the chip industry

The project's development time is particularly noteworthy. According to OpenAI, only 16 months passed between the formation of the development team and the chip's tape-out—that is, the completion of the design for manufacturing approval—with the first complete design reportedly finished nine months before the tape-out. In the traditional semiconductor industry, development cycles of three to five years are considered standard for comparably complex chips, so such a pace, should it be confirmed in future generations, could put pressure on the established cost structure of the entire industry.

According to OpenAI, this was made possible by the intensive use of its own AI systems in the design process itself. This enabled faster evaluation of design decisions and simultaneously helped optimize the computing units, significantly shortening traditional iteration loops in chip design. This creates a self-reinforcing effect: the more powerful the company's AI models become, the faster and more cost-effectively new chip generations can be developed, which then execute these models even more efficiently. Such a feedback mechanism, provided it proves reproducible, could become a structural advantage for companies that possess both leading AI models and their own chip development capabilities.

When your own AI becomes the engineer's toolbox

Astra, the next planned GPT version, has already been used at OpenAI to develop optimized computing cores, or kernels, for new models. According to reports, the code developed by Astra in GPT-OSS was 50 to 80 percent faster than comparable programs developed by human expert teams. This finding has more far-reaching economic implications than it initially sounds, as it suggests that the bottleneck of software optimization, which has so far depended heavily on the availability of highly specialized engineers, can increasingly be resolved by automated systems.

For the semiconductor industry as a whole, this means a potential realignment of competitive factors. Until now, decades of experience, in-depth engineering knowledge, and an established software ecosystem like Nvidia's CUDA were considered almost insurmountable barriers to entry for new providers. If AI-powered tools can partially compensate for this experience advantage, the entry threshold for further competitors will decrease, which could lead to a fragmentation of the currently highly concentrated market for AI accelerators in the long term.

🎯🎯🎯 Data-driven B2B industry hub as a quasi-in-house solution

The quasi-in-house solution: How Xpert.Digital closes operational gaps in B2B marketing and sales – Smart Content-Driven Business - Image: Xpert.Digital

Xpert.Digital is a data-driven B2B industry hub led by Konrad Wolfenstein . The company acts as an external, quasi-in-house solution for industrial partners, closing operational gaps in marketing, content, and sales – without requiring additional resources on the client side.

More information here:

The quasi-in-house solution: How Xpert.Digital closes operational gaps in B2B marketing and sales – Smart Content-Driven Business

OpenAI's Jalapeno chip: Why architecture is more important than raw computing power

Architecture as cost accounting in silicon

OpenAI made some interesting and economically sound design decisions with Jalapeño. For example, the chip's matrix units do not support calculations using the 16-bit FP16 floating-point format, which significantly simplifies the internal logic and aligns with the industry trend toward more quantized models, meaning those with lower numerical precision. 64-bit and 32-bit data types are only supported by the additional scalar and vector units, with the latter also limited to 32-bit calculations. Any transistor area saved by omitting rarely used calculation formats can instead be invested in more memory bandwidth or more processing cores for the formats that are actually needed, thus increasing the cost-efficiency per chip produced.

In terms of pure FP8 and FP4 computing power—the number formats particularly relevant for modern, quantized language models—OpenAI's chip, with 3.4 and 13.4 petaFLOPS respectively, formally lags behind Nvidia's Blackwell at 5 and 15 petaFLOPS. However, this seemingly disadvantageous figure is mitigated by actual system utilization, as chips rarely reach their theoretical peak performance in practice. According to industry observers, current accelerators often operate at only 30 to 50 percent of their theoretical efficiency under real-world inference workloads. Therefore, a chip that comes closer to its own peak performance can remain competitive or even surpass it in practice, despite its raw performance appearing lower on paper.

Memory bandwidth instead of raw computing power as a strategic choice

The chip is equipped with six HBM4 memory stacks, giving it almost twice the memory bandwidth of Blackwell, at 15.4 TB per second compared to 8 TB per second. This decision reflects a deliberate prioritization, because when inferring large language models, the limiting factor is often not pure computing power, but the speed at which model weights and intermediate results can be moved between memory and the arithmetic logic unit (ALU). A chip that specifically addresses this so-called memory bandwidth bottleneck can operate more efficiently in real-world application scenarios than a competitor with higher nominal computing power but comparatively tighter memory access.

Thanks to eight memory stacks, Blackwell and the upcoming Rubin generation offer a higher total memory capacity of 288 GB, while Jalapeño only has 216 GB available. For very large models with correspondingly extensive context windows, this lower capacity could become a limiting factor, especially if future model generations are even larger and more memory-intensive. The chip's TDP, or maximum thermal design power, is 700 watts, although OpenAI states that the chip only requires an average of around 550 watts in actual operation. This is a more economically relevant metric for everyday data center operations than the theoretical peak value.

Latency versus flexibility: a conscious compromise solution

The chip's memory hierarchy is interesting. OpenAI divides the chip into numerous compute groups, or slices, each with a fixed allocation of HBM memory. This tight, fixed coupling between the processing core and local memory significantly reduces access latency because each compute group has direct, low-latency access to its own memory partition, instead of having to traverse a multi-level, shared memory system as in traditional GPU architectures. The price for this speed advantage is reduced flexibility in memory allocation, since capacity cannot be arbitrarily redistributed between compute groups once a partition reaches its limits.

The groups are connected via two separate network layers: a specialized, particularly low-latency collective network for frequently recurring, clearly defined communication patterns such as synchronization during parallel processing, and a more general network-on-chip for other communication and access to the higher-level scale-up network between multiple chips. This architectural division allows the majority of internal data traffic to be handled via the faster, dedicated path, while the general network remains reserved only for less frequent, less time-critical accesses. From an economic perspective, this is a classic trade-off between specialization and general-purpose use, typical for purpose-built inference chips and fundamentally different from the more generalist architecture of classic graphics processors.

The systolic array as a legacy of the TPU philosophy

Like many other dedicated AI accelerators, Jalapeño relies on a systolic array, where intermediate results are passed directly between adjacent processing units instead of constantly oscillating between memory and the arithmetic logic unit. This concept is by no means new, but follows an architectural philosophy that has been successfully tested in practice for years, particularly by Google's Tensor Processing Units, where it has already achieved comparable efficiency gains in specialized AI workloads.

Unlike traditional, very large systolic array designs, Jalapeño, according to SemiAnalysis, also supports smaller matrix dimensions, thus avoiding the typical performance drops that occur when batch sizes or matrix dimensions don't exactly match the fixed array size. Additionally, the chip features dedicated 64-bit scalar cores as well as FP32- and INT32-capable vector units and employs an out-of-order execution approach with a classic L1 cache on each processing core, instead of relying on software-controlled scratchpad memory like many other accelerators. This combination of a specialized matrix unit and more flexible scalar and vector logic suggests that OpenAI deliberately sought a middle ground between pure specialization and general programmability.

Why lower costs don't immediately reach customers

From the perspective of enterprise customers and developers, the immediate question is whether the promised efficiency gains will also lead to lower API prices. According to several industry analysts, this is rather unlikely in the short term, as OpenAI currently operates primarily with capacity constraints rather than cost constraints. This means that lower unit costs will initially translate into more available volume rather than lower prices. Historical examples, such as Google's introduction of its own TPUs or Amazon's Trainium chips, show that end-customer prices typically only decrease 12 to 24 months after the widespread rollout of new proprietary hardware, usually triggered by increased competitive pressure from rival vendors.

In addition, OpenAI must first recoup its substantial upfront investments in design, manufacturing, and infrastructure through inference revenues over several quarters before the cost savings actually translate into higher margins. Until then, the chip is likely to primarily help alleviate existing capacity bottlenecks, such as reducing long wait times or rate limits in API access, rather than leading to immediate price reductions. Nevertheless, for companies creating medium-term budget plans for the use of AI services, it seems reasonable to factor in a gradual cost reduction of potentially 30 to 50 percent over a period of two to three years as a realistic planning scenario.

A competition for market power, not just for watts

The deeper strategic reason behind this development lies in the struggle for independence from a single, market-dominating supplier. For years, Nvidia has controlled the vast majority of the market for AI training and inference hardware, thereby achieving substantial gross margins that ultimately had to be borne by customers. If OpenAI, much like Google, Amazon, and Microsoft with their respective chip programs, shifts a significant portion of its inference workload to its own hardware, Nvidia's bargaining power with its largest customers will diminish, even if Nvidia remains the primary partner for training new models.

It is noteworthy in this context that Nvidia is simultaneously providing massive funding commitments for OpenAI's data center expansion, for example, through a multi-billion dollar loan for a new data center in Ohio, while OpenAI is simultaneously working on its own, potentially competing chip platform. This seemingly contradictory situation can be interpreted economically as rational behavior on the part of both sides: Nvidia secures long-term purchase commitments and capital commitments from its largest single customer, while OpenAI simultaneously develops strategic independence for a growing portion of its inference workload without completely straining its relationship with its most important training partner.

From announcement to mass production

Initial small-scale shipments of Jalapeño are planned for the end of 2026, while large-scale production and a major rollout are expected to take place throughout 2027. Until then, Nvidia, particularly through the continued use of its Vera Rubin systems, will remain a central component of the OpenAI infrastructure, meaning that a complete abandonment of Nvidia hardware is not on the cards. Instead, a hybrid model is emerging in which different hardware generations and vendors are used in parallel for different workload profiles.

In the long run, Jalapeño's true significance likely lies less in the specific efficiency figures of the first generation than in the demonstrated pace of development and the associated ability to leverage AI-powered tools for its own hardware development. Should this self-reinforcing cycle of more powerful models and faster chip development cycles prove reproducible, it would not only alter OpenAI's competitive position vis-à-vis Nvidia but also fundamentally shift the timeline in which new semiconductor generations for AI applications can be developed.

📈🚀 From visibility to trust 👀🤝 Your scalable path with Xpert.Digital

From visibility to trust: Your scalable path with Xpert.Digital - Image: Xpert.Digital

In industrial B2B, sustainable business relationships rarely emerge overnight. They develop step by step – through visibility, professional relevance, recurring touchpoints, and growing trust. Xpert.Digital's 4-stage model addresses precisely this: It offers a structured path that begins with a manageable entry point and can evolve into deeper collaboration in business development if needed.

Instead of relying on loud marketing promises, this model puts the relationship at the forefront. Companies start with clearly defined, easily calculable measures and then decide, based on their own experience, how far they want to expand the collaboration. A key factor for this undisturbed trust-building process: The platform completely avoids annoying advertising ads, so the editorial focus remains solely on the companies' expertise.

More information here: