OpenAI’s o3 Reasoning Models Are Extremely Expensive to Run

Introducing the OpenAI’s o3 Reasoning Model: A Closer Look at the Costs

In the world of artificial intelligence, measuring intelligence itself can be quite a challenge. This is where benchmarks like ARC-AGI come into play, testing the capabilities of new technology through a series of challenging visual tasks. In a groundbreaking achievement, OpenAI’s o3 reasoning model became the first AI system to pass the test with an impressive 87.5 percent score.

However, this victory came with a hefty price tag. The Arc Prize Foundation, which oversees the ARC-AGI benchmark, initially estimated the cost of testing OpenAI’s model at around $3,400 per task. For a more efficient version of o3 scoring 75.7 percent, the cost per task amounted to $20.

But the actual costs turned out to be much higher—ten times higher, to be precise. The Arc Prize Foundation revisited its pricing strategy, aligning it with OpenAI’s latest release, the o1-pro model. This new model, unveiled recently, is ten times more expensive to operate than its predecessor, making it OpenAI’s most costly model to date.

Based on the new pricing, running o3 could potentially cost upwards of $30,000 per task, while the more efficient strain of o3 is priced at $200 per task. Greg Kamradt, president of the Arc Prize Foundation, mentioned, “Our belief is that o3 pricing will be closer to o1-pro pricing than to o1 pricing we were told in December.”

To adapt to the new pricing structure, the Arc Prize Foundation has updated its ARC-AGI leadership board, excluding the more compute-intensive version of o3. The board now showcases systems that require less than $10,000 to run.

ARC-AGI, founded in 2019 by researcher François Chollet, presents a series of puzzles to evaluate how close AI systems are to human-level intelligence. Unlike traditional tests, ARC-AGI focuses on the ability of models to learn new skills and adapt to new problems. OpenAI’s o3 excelled in this test by considering various prompts before providing accurate responses.

While the pricing of o3 remains unconfirmed by OpenAI, the Arc Prize Foundation will continue to estimate costs based on o1-pro pricing until official figures are released. With the introduction of ARC-AGI-2, a more challenging test for reasoning-specialized AI systems, the race for achieving the perfect score continues.

In conclusion, as AI technologies advance, the costs associated with running these sophisticated models are also on the rise. The journey towards achieving human-level intelligence in machines comes at a price, but the pursuit of innovation in AI continues to push boundaries and redefine possibilities.