This one goes all in the following themes:
- few examples to learn from. You have to carefully inspect the input examples to deduce the output rules. Rules can require specific It application ordering, so you actually generate an algorithm. It tends to be easy for humans, but sometimes not so easy!
- extensive use of geometric concepts, notably "contained inside", "adjacent to", "connected"
Bibliography:
- www.reddit.com/r/mlscaling/comments/1ht4emi/anyone_else_suspect_arcagi_was_never_much_of_a/ Anyone else suspect ARC-AGI was never much of a test of anything? (2025)
www.kaggle.com/code/allegich/arc-agi-2025-visualization-all-1000-120-tasks contains plots of all questions and answers. It is truly very convenient.
Bibliography:
This interesting repo defines a set of input transformations that can be composed together into programs to generate the solve ARC problems.
It does not appear to have any program synthesis: it only defines the DSL and then provides manual solutions to the problems.
The README is lacking as usual, an overview of the files is:
- dsl.py: defines the transformations as Python functions
- solvers.py: defines solvers for the 400 ARC-AGI-1 training problems
Intended usage to run the solvers seems to be:Unfortunately this blows up on Ubuntu 25.04 on
git clone https://github.com/fchollet/ARC-AGI
cd ARC-AGI
git checkout 399030444e0ab0cc8b4e199870fb20b863846f34
git clone https://github.com/michaelhodel/arc-dsl
cd arc-dsl
git checkout 635de4902a5fb4e376f27333feaa396d3f5dfdcb
python main.py
test_mpapply
apparently due to a Python 3.12 issue and the pull request github.com/michaelhodel/arc-dsl/pull/7 has been ignored for more than one year, so the project is largely dead.Ciro Santilli's fork of ARC-DSL merging all pull requests needed to make tests run again on Ubuntu 25.04.
It seems to have been tested on something older than Ubuntu 24.04, as 24.04 install requires some porting, started process at: github.com/cirosantilli/ARC-AGI-solution/tree/ubuntu-24-04 but gave up to try Ubuntu 22.04 instead.
Ubuntu 22.04 Docker install worked without patches, after installing Poetry e.g. to try and solve 1ae2feb7:but towards the end we have:so it failed.
git clone https://github.com/aviad12g/ARC-AGI-solution
cd ARC-AGI-solution
git checkout f3283f727488ad98fe575ea6a5ac981e4a188e49
poetry install
git clone https://github.com/arcprize/ARC-AGI-2
`poetry env activate`
export PYTHONPATH="$PWD/src:$PYTHONPATH"
python3 -m arc_solver.cli.main solve ARC-AGI-2/data/evaluation/1ae2feb7.json
{
"success": false,
"error": "Search failed: no_multi_example_solution",
"search_stats": {
"nodes_expanded": 21,
"nodes_generated": 903,
"termination_reason": "no_multi_example_solution",
"candidates_generated": 25,
"examples_validated": 3,
"validation_success_rate": 0.0,
"multi_example_used": true
},
"predictions": [
null,
null,
null
],
"computation_time": 30.234344280001096,
"task_id": "1ae2feb7",
"task_file": "ARC-AGI-2/data/evaluation/1ae2feb7.json",
"solver_version": "0.1.0",
"total_time": 30.24239572100123,
"timestamp": 1760353369.9701269
}
Task: 1ae2feb7.json
Success: False
Error: Search failed: no_multi_example_solution
Multi-example validation: ENABLED
Training examples validated: 3
Candidates generated: 25
Validation success rate: 0.0%
Computation time: 30.23s
Total time: 30.24s
Let's see if any of them work at all as advertised:and at the end:has only 7 successes.
ls ARC-AGI-2/data/evaluation/ | xargs -I'{}' python3 -m arc_solver.cli.main solve 'ARC-AGI-2/data/evaluation/{}' |& tee tmp.txt
grep 'Success: True' tmp.txt | wc
Also weirdly only has 102 hits, but there were 120 JSON tasks in that folder. I search for the missing executions:The first missing one is 135a2760, it blows up with:and grepping ERROR gives us:Reported at: github.com/aviad12g/ARC-AGI-solution/issues/1
grep 'Success: True' tmp.txt | wc
diff -u <(grep Task: tmp.txt | cut -d' ' -f2) <(ls ARC-AGI-2/data/evaluation)
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type SizePredicate is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type ndarray is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type ndarray is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type VerticalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type VerticalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type ndarray is not JSON serializable
ERROR: Solve command failed: Object of type VerticalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type ndarray is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type HorizontalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type VerticalLinePredicate is not JSON serializable
ERROR: Solve command failed: Object of type VerticalLinePredicate is not JSON serializable
These section lists common visual primitives that a solver must first extract in order to infer solutions.
Many people have come up with the same idea on the Discord. Some nicely call it DSL.
Implementations:
An "object" is a set of points that is understood to be one singular entity.
Contiguity and having the same color are strong indicators that something should be understood as an object.
A rectangular container.
There are two or more boxes drawn inside the toplevel and sharing boundaries with toplevel.
There are two toplevel boxes, one contains only input, and all output goes to the second one. The second one may also contain some input.
A path is something you obtain by somehow drawing from one point to another, e.g. a line, and then starting another drawing between two points from the end point.
A dotted line is a generalized line that cycles between a color pattern, e.g.:would be a line:An extra color "transparent" may also be added to not change for that pixel.
Then to the right, on each line draw one square of each color to the left every n columns, starting with a square on the first column to the right of the red line, where n is the count of that color.
Start with the color furthest away from the red line, and then color with colors nearer to the red line. If there's overlap, replace the old color with the new one.
Output:
- draw dotted lines
Input primitives:
- background color
- squares
- points
Transformations primitives:
- line drawing
Solution: move pieces to fill the gap on the fat object that crosses the screen. Place objects either on fat object or on other objects placed on the fat object. Anything you add must end in a rectangle.
The rules for this one are not entirely clear with the number of examples.
Also clearly if the goal is to make rectangular towers, then this is an NP-hard optimization problem in general.
Input primitives:
- same color chunk. Properties: crosses screen.
Transformation primitives:
- move solid around
- fills the gap
This existed earlier: x.com/GianpaoloGalli/status/1846144236900827413
Solution: vs are guns that shoot diagonal line of their color, when line touches another object, change line color to match that of the object, then bounce on the object and continue going with the new color
Input primitives:
- diagonal line
Assumptions:
- line don't cross each other, it is unclear how to resolve that case
Transformation primitives:
- draw line
- draw line and bounce
Input primitive:
Transformation primitives:
- draw lines
Input:
- background color
- boxes
- points inside boxes
- distance between point and box
Input: three or more containers:
- one touching top left corner
- inside it there are three monocolor objects
- one touching bottom right corner of toplevel box
- inside it there is one monocolor object
- outside of those, touching the left toplevel box edge, there is one or more point
Output:
- draw dotted path of perpendicular line
That's Ciro Santilli's favorite. Of course, there is a huge difference between physical and non physical jobs. But one could start with replacing desk jobs!
Articles by others on the same topic
There are currently no matching articles.