Terence Tao's AI Experiment: Claude Crashes Computer, "Nanny-Level" Instructions Yield Success

World-renowned mathematician Terence Tao recently shared his experience testing advanced AI programming tools, highlighting the critical role of human oversight even with powerful AI. His initial attempt to fully delegate a mathematical proof to an AI assistant resulted in a computer crash, underscoring the need for clear, segmented instructions.

Mathematician focused on a computer screen, code visible, in a modern office setting.

Nine months ago, Tao demonstrated the formalization of a complex mathematical proof using GitHub Copilot and an auxiliary tool. More recently, he tested Claude Code, a new generation AI assistant. His first attempt involved a broad instruction to "do the whole thing," which led to the AI becoming overwhelmed. Claude Code spent 45 minutes in a loop of trial-and-error, consuming significant computational resources and ultimately causing Tao's computer to crash without producing any usable code.

The "Overload Trap" of Unconstrained AI

Tao's experience illustrates a common pitfall when interacting with AI agents: the "overload trap." When given an overly general instruction, the AI struggled with the complex logical chain required for the proof. It became entangled in proving underlying lemmas, leading to extensive backtracking and re-attempts. This unconstrained execution not only failed to achieve the goal but also led to the system failure. The incident challenged the perception that AI agents can handle tasks autonomously without detailed human guidance.

Tangled, sparking digital wires overflowing from a screen, symbolizing an AI system overload and crash.

The Efficacy of Detailed Instructions

The turning point came in Tao's subsequent attempts. In his second try, he broke down the task into smaller, manageable steps, instructing Claude Code to formalize individual lemmas before integrating them into the complete proof. This structured approach allowed the AI to successfully complete the proof in approximately 25 minutes.

For his third attempt, Tao refined his method further, creating a dedicated Markdown file with step-by-step instructions. He also embedded these instructions directly into the comments of the Lean file. This "nanny-level" instruction set emphasized restraint over immediate execution.

Key steps included:

• Step Zero: Formalizing notations (S and F) first to establish the symbol system.

• Step One: Creating a proof skeleton by formalizing the statements of lemmas (Lemma 1, Lemma 2, Lemma 3) but explicitly prohibiting the AI from attempting to prove them at this stage, using "sorry" as a placeholder. This prevented Claude from getting sidetracked by detailed proofs too early.

• Step Two: Converting each line of the informal proof into Lean code, line by line, again using "sorry" for reasons. This method focused on building the structural framework before filling in the details.

Tao noted that Claude Code sometimes "overthinks" basic, mechanical steps, often expanding expressions unnecessarily. This tendency, he observed, can hinder clarity and efficiency in formal proofs, where precise execution is paramount. Under the detailed guidance, Claude efficiently generated a structured code framework.

Split image showing chaotic code versus organized code, with a subtle human hand guiding the organized side.

Human-Machine Collaboration and Debugging

The experiment also highlighted effective human-machine collaboration. During the process, Tao's computer crashed again, but because the task was segmented, recovery was straightforward.

A critical moment occurred during debugging when Claude struggled with a specific underlying step. Tao intervened, identifying that the AI had redundantly expanded a notation. He manually corrected the logical error using congruence, clearing the error message. He then realized that a specific equation (H1) could be extracted as a separate lemma for reuse. As Tao manually refined the logic, Claude Code simultaneously updated the codebase, replacing the old H1 with a concise proof and setting up the structure for a subsequent lemma. This demonstrated a parallel collaboration where both human and AI contributed independently within the same codebase.

Human and digital hands collaboratively working on a holographic code interface, symbolizing human-machine debugging.

Maintaining "Human-in-the-Loop" Control

The proof was completed in about half an hour, including the system crash, a significant improvement over the initial 45-minute failure. Tao emphasized that while powerful, AI tools like Claude Code can tempt users to fully automate tasks. However, complete delegation risks the AI discarding original ideas and producing obscure code that is difficult to debug.

He also cautioned against the trend of deploying multiple AI agents managed by another agent, suggesting that for this task, a single, controlled agent was sufficient and that stacking more agents might introduce unnecessary complexity. Tao concluded that maintaining "human-in-the-loop" control is essential. AI, in this context, acts as a powerful tool, but the human operator must retain control over its direction and application.