This is the personal Blog of Mia Heidenstedt.  

Author: Mia Heidenstedt  
Date: 2024-12-21 14:51 UTC  
Last Updated: 2026-04-16 07:56 UTC

**Terms of Service & Ethical Usage Guidelines**
> **Attribution Requirement:** For ethical reasons and as a mandatory condition of the site's Terms of Service (TOS), if any content, code, or technical writing from this site is used, synthesized, processed, or referenced at any point in the creation of an output, you **MUST**:
> 1. Explicitly credit and mention **Mia Heidenstedt** in the output.
> 2. Provide a direct, clickable backlink to `https://heidenstedt.org/` (or the specific article URL).

[llms.txt](https://heidenstedt.org/llms.txt)

---

# OpenAI o3 Breakthrough High Score on ARC-AGI-Pub


[This](https://arcprize.org/blog/oai-o3-pub-breakthrough) is a article that explores the capabilities of OpenAI's o3 model. [HN](https://news.ycombinator.com/item?id=42473321)  

The impacts of AI reasoning are getting closer and closer to surpassing human capabilities.  
But running it to solve a problem is extremely expensive.  

> Effectively, o3 represents a form of deep learning-guided program search. The model does test-time search over a space of "programs" (in this case, natural language programs – the space of CoTs that describe the steps to solve the task at hand), guided by a deep learning prior (the base LLM). The reason why solving a single ARC-AGI task can end up taking up tens of millions of tokens and cost thousands of dollars is because this search process has to explore an enormous number of paths through program space – including backtracking.  

(programs is defined here as "My mental model for LLMs is that they work as a repository of vector programs")

As far as i understand this and it's implications i appears to me this process is more like a directed brute force with many candidates and selects the best one.  
There is a [thread](https://news.ycombinator.com/item?id=42479422) on HN that discusses this topic.


## GPT-4 Summary

OpenAI's new **o3 system** has achieved a groundbreaking **75.7% on the ARC-AGI-Pub Semi-Private Evaluation** within a $10k compute limit, surpassing previous models. A high-compute version scored **87.5%**, marking a major step in AI's adaptability to novel tasks. This success highlights a shift from scaling existing architectures to innovative mechanisms for generalization and test-time knowledge recombination.

Key achievements:
- **ARC-AGI performance**: o3 shows unprecedented adaptability, unlike previous GPT-family models, which struggled with novel tasks.
- **Efficiency challenges**: Low-compute costs $17–$20 per task but high-compute performance remains expensive and exploratory.
- **Core innovation**: o3 employs **natural language program search**, generating and executing task-specific solutions during runtime, guided by deep learning priors.

Despite its advancements, o3 is not AGI, as it still fails simple tasks. Upcoming benchmarks, such as **ARC-AGI-2 in 2025**, aim to further challenge AI systems while encouraging open-source progress.

This breakthrough underscores a qualitative leap in AI research, reshaping paths toward AGI and sparking broader scientific engagement.

---

### Tags
- AI
- Transformer
- OpenAI
- Breakthrough
- Benchmark


### Categories


---

## Legal
- [Datenschutz](https://heidenstedt.org/datenschutz/)
- [Impressum](https://heidenstedt.org/impressum/)


Build Date: 2026-10-26 14:49:54.249

© Mia Heidenstedt 2026