Looking at AI progress in verifiable steps
By Oliver Schmalzl · Written: · Archive discovery date:
7 September 2025 · Archive date
Conversations about AI quickly switch between two levels. On one is a concrete task that a tool does today. On the other is an idea of what might be possible in a few years. Both levels can be interesting, but they should stay distinguishable.
An impressive demonstration makes you curious. For your own assessment, it helps to translate its claim into a verifiable sentence.
A small task as a yardstick
An invented example: "The tool finds two contradictory size details in a text I wrote myself." There is a known source text and an expected result. Whether the task succeeds can be observed.
"The tool will soon take over all planning" is a much bigger claim. It contains assumptions about new abilities, reliability and the conditions of later use. A single successful attempt does not prove these assumptions.
A sensible comparison can therefore look at several tasks of different difficulty. Failures are part of it too. They show where an extra explanation, a different tool or human checking is still needed.
Looking at progress and responsibility together
Additional abilities bring new questions: Which steps may happen on their own? Which results can be checked? When does it have to stop? These limits can change with the task and should be described explicitly.
A forecast remains the assessment of its source. For my own blog, the concrete learning question is often more fruitful: what works in a clearly limited test, under which conditions and with which open points? That way you can follow a development closely without presenting an optimistic or pessimistic prediction as a settled course.
Original finds
These links were the starting point for this post. The commentary in the text is mine; the linked posts are not my own tests.
- Dario Amodei on AI progress and programming
- Dario Amodei on AI progress and programming
- How fast should AI move forward?
- Compilation of Ilya Sutskever's warnings and the opportunities of advanced AI; check the statements against the original source before a blog post.
- Post arguing that AI itself is not the most important skill for the future; the exact reasoning in the video is still open.
About the date: This post was written on 11 October 2026. The archive date corresponds to the earliest related find in the collection, not to a publication at that time. Technical context and sources added later refer to the time of writing.