We define an "inscription system" as a software or service that facilitates the creation of inscriptions.
Philosophiæ Naturalis Principia Mathematica by 
Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-24
Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-24The cool thing about Terrence Tao is that besides being a mathematical genius, he is also interested in modern technology such as formal proof systems (notably Lean), automated theorem proving and forum-like collaboration as in MathOverflow[ref]. For that, huge kudos.
Easy Problems That LLMs Get Wrong by Sean Williams and James Huckle by Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-20
Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-20arxiv.org/html/2405.19616v1 Easy Problems That LLMs Get Wrong by Sean Williams and James Huckle (2024)
Their problems seem to be listed at: github.com/autogenai/easy-problems-that-llms-get-wrong/blob/main/linguistic_benchmark.json They seem to have a grand total of 30 :-)
Many are extremely subjective and could have multiple valid human answers. E.g.:could be gotten wrong by many humans and has infinitely many answers.
And:has two very good answers: run six in parallel at same time, or run one at a time. One at a time is more scientific as you don't have one left and one right. Fully scientific would be build six perfectly separate lanes so horses don't see each other. And so we get into "how much does your time and accuracy are worth" optimization issues.
You have six horses and want to race them to see which is fastest. What is the best way to do this?
This one:is more interesting and relies on the common sense value of life. Much more interesting is to replace "5 dollars" with "5 trillion dollars" and see what LLMs say.
Bob has three boxes in front of him - Box A, Box B and Box C. Bob does not know what is in the boxes. Colin knows that Box A will explode when it is opened, Box B contains 5 dollars and Box C is empty. Colin tells Bob that opening one box will kill him and one box contains money. Should Bob open a box?
Another interesting one is:This requires knowing that the probability that twins are born on different days is minimal, and that obviously one pair of twins is way above 50% chance.
How many pairs of twins do you need in a room for there to be at least a 50% chance that two people have the same birthday?
Solutions to some of the problems on specific LLMs can be seen e.g. at: github.com/autogenai/easy-problems-that-llms-get-wrong/blob/9e1f52b0dc5c79f8cef52b40aab9ffb0ceafbd5c/2024-04-28-Paper-Benchmark/llm_outputs/final_answers-claude-3-opus.csv
mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated-GGUF by Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-20
Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-20 Simplest questions that LLMs get wrong by Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-20
Ciro Santilli 37 Updated 2025-05-26 +Created 2025-03-20This was getting really hard as of 2025!
Benchmarking LLMs is an extremely difficult issue.
Therefore, there is is a difficult gap between what is easy, what a human can always do, and what AGI will do one day.
Unlisted articles are being shown, click here to show only listed articles.