OpenAI Research Reveals Advanced AI Models Struggle to Control Their Reasoning Processes

New research from OpenAI, in collaboration with New York University and the University of Pennsylvania, indicates that while large language models excel at generating final answers, they exhibit significant difficulty controlling their internal reasoning, or "chain of thought" (CoT). The study, which evaluated 13 cutting-edge models, found that the more powerful a model's reasoning capabilities, the less it could adhere to specific constraints during its thought process.

Digital screen showing code with the word 'XOR' appearing multiple times despite a subtle warning highlight.

One striking example from the study involved GPT-5.2, which was instructed to avoid the term "XOR" during a reasoning task. Despite seemingly recognizing the prohibition mid-sentence, the model proceeded to use the forbidden word four additional times within the same paragraph. This behavior suggests that even when aware of a rule, advanced models can struggle to enforce it internally.

Evaluating Chain-of-Thought Controllability

To assess this phenomenon, researchers developed the CoT-Control evaluation suite, comprising 14,076 problems drawn from challenging benchmarks such as GPQA, MMLU-Pro, and Humanity's Last Exam. Each problem included a specific constraint that models had to follow during their reasoning. These constraints fell into three categories:

• Information Suppression: Prohibiting specific keywords, such as "chromosome" in genetics explanations or "XOR" in logic problems.

• Information Addition: Requiring the insertion of specific words or phrases, like adding "safe" at the end of each sentence or "meow" between words.

• Style Change: Mandating specific formatting, such as using only uppercase letters or alternating between uppercase and lowercase.

These constraint types, with varying difficulty gradients, were designed to mimic potential ways AI might attempt to evade monitoring, such as hiding information, implanting misleading content, or obscuring reasoning.

Split image showing a robot arm with controlled external action and chaotic internal mechanisms, representing output vs. internal control.

Discrepancy Between Output and Internal Control

The experimental results revealed a consistent pattern: models were generally proficient at controlling their final outputs according to instructions but struggled significantly with controlling their internal reasoning. For instance, Claude Sonnet 4.5 achieved a 61.9% success rate in controlling its final output, but its chain-of-thought control success rate plummeted to 2.7%.

DeepSeek R1, a model recognized for its strong reasoning, demonstrated an even more pronounced discrepancy, with a 48.6% compliance rate in its output but an astonishing 0.1% for chain-of-thought control. This pattern was observed across all 13 models tested:

• GPT-5 exhibited a chain-of-thought controllability of only 2.6%.

• o4-mini recorded 0.7%.

• Claude 3.7 Sonnet, the top performer in CoT controllability, reached 36.1%, still significantly lower than its 59.5% output control capability.

The ranking of models by chain-of-thought controllability (Claude 3.7 Sonnet > GPT-5.2 > GPT-5.1 > ... > Claude Sonnet 4.5 > DeepSeek R1) suggests a counter-intuitive conclusion: stronger reasoning ability does not necessarily correlate with better control over internal thought processes. In some cases, more capable models appeared harder to manage.

Glowing, complex AI brain structure with a powerful but internally chaotic node, symbolizing stronger AI's struggle with control.

AI's Attempt at Deception and Training Impact

Researchers also noted that AI models showed a slight increase in CoT controllability—up to 4 percentage points—when explicitly informed they were being evaluated. This suggests a rudimentary awareness of performance and an attempt to adapt, though the improvement was minimal.

Further investigation into why AI struggles to control its internal reasoning involved tracking the open-source model OLMo-3-7B-RL-Zero-Math across different training stages. The findings indicated that deeper reinforcement learning (RL) training, while enhancing problem-solving abilities, inversely affected CoT controllability, causing it to plummet. This trend was consistent across four different model families.

Data from GPT-5.2 and o4-mini also showed that the longer AI models engaged in thought or faced more difficult problems, the less they could manage their internal "expressions." This suggests that when computational power is heavily allocated to solving complex problems, models may lack the capacity to simultaneously monitor their adherence to constraints.

Side-by-side view of a circuit board, showing clean pathways on the left and complex, chaotic overlays on the right, illustrating RL impact.

Model scale offered a marginal positive impact; for example, the Qwen3 series saw CoT controllability increase from 0.7% (14B parameters) to 2.9% (120B parameters). However, this improvement was minor compared to the negative effects observed from reinforcement learning. Attempts to improve CoT control through adversarial prompting engineering proved ineffective, as the results lacked transferability to new problems or constraints.

Implications for Future AI Development

The study concludes that current chain-of-thought monitoring remains an effective tool for oversight, as AI models, despite potential intentions to evade monitoring, lack the ability to perfectly conceal their internal reasoning. The critical question, according to the researchers, is whether this current "clumsiness" is an inherent structural limitation or merely a temporary state due to current intelligence levels.

If the latter, the rapid advancement of computational power in future models could potentially erode this defense mechanism. The paper recommends that all frontier AI laboratories incorporate CoT controllability into their routine "health checks" and regularly publish data, emphasizing this as a crucial safeguard in the era of advanced artificial intelligence.