Many discussions of artificial intelligence (AI) ignore the margins principle. Dramatic headlines such as “AI will kill all humans” are popular because they attract an audience. Understanding, especially when discussions move beyond “we should [should not] ban AI”, is less dramatic because it involves nuances. A recent talk by Terence Tao offers some insights into which deserved to be noticed. These insights offer a broader lesson which is both familiar and needs to be repeated often.
Terence Tao is a famous mathematician who spoke to the International Congress of Mathematicians (in July 2026) on “Mathematics in the age of AI”. If you did not know, mathematicians are debating (very energetically!) whether AI can prove theorems faster and more reliably than humans can and, if so, should the job of a mathematician change radically.
In my opinion, the most instructive slide in his presentation (at minute 7) poses an “AI Capability Conjecture”:
“At some point in the near future, some Al tools will, at some expense, and with some level of human supervision, be able to accomplish some research-level mathematical tasks in some fields of mathematics, with some non-trivial success rate, and at some level of correctness and quality.”
The word “some” appears eight times. This conjecture is not a conclusion because these bits of an argument need to be resolved. Even if there is not yet enough hard evidence to resolve these “placeholders” or “planning assumptions”, some people are willing to rush past these annoying details when the correct answer seems so obvious.
What does it all mean?
AI is surprisingly capable: e.g., it created the feature image at the top based on my writing. That fact explains why so many people are using it for so many tasks (often for free, now). That surprise does not mean that AI is capable of doing everything. Between Everything and Nothing, many other types of answers exist. Those answers depend on how the words “some” are resolved. For example,
- “At some point in the near future”: to somebody looking for a job, it matters whether that nearness is tomorrow or 10 years from now
- “some Al tools”: which of the competitors (OpenAI, Anthropic, Google, …) is best for which task (coding, chatting, thinking deeply, …)?
- “at some expense”: what fraction of the cost is paid by whom (e.g., users, suppliers, third parties)? If the marginal benefit is not worth the marginal cost then what?
- “with some level of human supervision”: if AI needs help, what form does that help take? [1] Is the “30% Rule for AI” right?
And so on.
The effects of AI are worth discussing. Over-simplifying that discussion can produce dramatic assertions but the potential effects are sufficiently profound that they deserve to be treated seriously. For example, media reports suggested that there was a 10% chance that AI could kill all humans (at least, according to one expert). Is this dramatic number credible? (In what sense is the chance 10%, not 1 or 50%?)
Resolved these margins has practical implications for government policy: e.g.,
- What kind of policy: what does “some point in the near future” reveals about income support policies intended to to help ease the transition in a job market? what does “some level of human supervision” reveal about regulating the technology?
- How to manage the transition to the future: e.g., if a policy is implemented as soon as possible, but the technology’s capabilities change, will the policy remain relevant?
People who make predictions about the effects of changes in technology often ignore the fact that the links between changes in behaviour and changes in technology are not as simple as an If/Then/Else bit of computer code. Human behaviour is infinitely interesting because it includes so many possible margins of adjustment.
Leave a Reply