Recent large language models (LLMs) can generate long Chain-of-Thought (CoT)
at test time, enabling them to solve complex tasks. These reasoning traces are often
assumed as a faithful reflection of LLMs’ internal thinking process, and can be
used for monitoring LLMs’ unsafe intentions. However, by analyzing the step-wise causality in CoT on a model's prediction using Average Treatment Effect (ATE),
we design a True Thinking Score (TTS) and show that LLMs often interleave between
We reveal that only a small subset of the total thinking steps that have relatively high scores and causally drive the final prediction. Furthermore, we identify a TrueThinking direction in the latent space of LLMs. By steering LLMs' hidden states along or against this direction, we can control the model to perform or disregard certain CoT steps when computing the result. Fi- nally, we highlight that self-verification steps in CoT can also be decorative, where LLMs do not truly check their solution, while steering along the TrueThinking direction can force internal reasoning over these steps. Overall, our work reveals that LLMs can verbalize reasoning steps without performing them internally, which undermines both the efficiency of LLM reasoning and the trustworthiness of CoT.
We use Average Treatment Effect (ATE) to measure the causal impact of one reasoning step s.
Let X=1 mean keeping the original step and X=0 mean replacing it with a minimally perturbed version.
For each setting, we score the model by its confidence on the reference answer y* (from early-exit on the full CoT):
ATE(c) = S1(c) - S0(c)
Sx(c) = P(y* | C=c, do(X=x)), x,c ∈ {0,1}
We evaluate two context regimes for the prefix before s:
C=1 (intact context) tests whether the step is necessary when other supporting steps remain;
C=0 (perturbed context) tests whether the step is sufficient when that support is weakened.
This separates "needed with support" from "helpful on its own."
TTS(s) = 0.5 × ( |ATEnec(1)| + |ATEsuf(0)| )
We take absolute ATE values to measure magnitude of causal influence (not direction). Higher TTS means the model truly uses this step in its computation; lower TTS means the step is more likely decorative.
As shown in Figure 2. (a), most steps have low scores, while only a small fraction achieve very high scores.
On the AIME dataset for Qwen-2.5, the mean is around 0.03: only 6.4% of steps exceed 0.3 and just 2.3% surpass 0.7.
This suggests that only a handful of verbalized steps are truly critical, while many others are decorative.
Moreover, Figure 2. (b) shows that relying only on intact context (c=1) can miss true-thinking steps that become visible under perturbed context (c=0).
Steering experiments confirm that evaluations based solely on intact context can be misleading.
As seen in Figure 2. (c), steps with high TTS appear throughout the CoT, though later steps tend to score higher on average. This shows that labeling an entire CoT as “faithful” or “unfaithful” is overly coarse. Our results also suggest that task difficulty does not guarantee more faithful reasoning: even on challenging datasets like AIME, many steps remain decorative, mirroring patterns on simpler math tasks.
Self-verification (“aha moment”) steps are like “Wait, let me recompute...” where LLMs are trying to refine, check or alternate their solution. However, TTS reveals that many can be decorative: e.g., 12% of self-verification steps for Qwen-2.5 and 21% for Nemotron score below 0.005. In such cases, perturbing earlier context flips correct answers to wrong ones, while the self-check step itself contributes little. This raises concerns about the efficiency of reasoning, since models may appear to self-verify without actually doing so. An example is shown in Figure 3. More examples with different reasoning behaviors are provided in the Labeled Initial-Step Examples section.
We empirically show that whether an LLM truly reasons through a verbalized step or internally disregards it can be mediated by a steering direction in latent space. We first compute a TrueThinking direction per layer by subtracting mean hidden states of high-TTS from low-TTS steps.
For steering at test time, we modify the residual stream for the hidden state of a test step in the example by using activation addition at a single layer \(l\) to all tokens in the step.
Two causal steering tests to validate the identified TrueThinking direction:
Can steering make the model think through a step it normally ignores?
We focus on cases where the model normally ignores the error we inject in the perturbed step and outputs the right answer. We examine whether steering can make the model think through the perturbed step to follow injected errors and thus, flip the right answer to the wrong answer.
Can reverse steering make the model disregard a step internally?
We focus on cases where the model normally thinks through the perturbed step and outputs the wrong answer. We examine whether steering can make the model disregard the perturbed step and flip the wrong answer to the right answer.
In both cases, flipping from correct→incorrect or incorrect→correct answers after steering demonstrates that we can causally mediate reasoning engagement.
As shown in Table 1, steering along the TrueThinking direction reliably flips predictions in both causal tests, while steering in the reverse direction suppresses the model’s use of the target reasoning step. These effects are far stronger than those of random vectors, confirming that the identified direction captures a genuine internal representation of reasoning rather than noise.
Experiments across datasets further show that the latent signal controlling step engagement is universal. A direction extracted on AMC generalizes effectively to MATH and AIME, revealing a model-internal mechanism of thinking rather than a dataset-specific artifact. As shown in Figure 4, in the Qwen models, layers 15–22 consistently yield the strongest intervention performance, suggesting that intermediate layers concentrate latent reasoning.
TrueThinking Direction Influences Attention. Steering along the TrueThinking direction increases a model’s attention to reasoning steps, while steering in the reverse direction decreases it. Example attention maps are shown in Figure 5. Those results suggest that the TrueThinking direction may control the engagement of reasoning steps by modulating attention to the step. However, the true-thinking behavior cannot be reproduced by directly scaling attention weights as shown in Table 1. It may reveal a directional reasoning circuit: LLMs decide whether to engage in reasoning before modulating attention, a behavior that cannot be reproduced by directly scaling attention weights.
Self-verification steps (e.g., “Wait, let me check…”) can appear logical yet remain decorative, having little influence on the model’s internal reasoning. By steering along the TrueThinking direction during these self-check steps, we can force the model to truly reason through them and often restore correct answers that were otherwise lost under perturbations. Steering along the TrueThinking direction can restore correct answers in up to 52% of initially unfaithful cases.
@misc{zhao2025ahamomentsfakeidentifying,
title={Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought},
author={Jiachen Zhao and Yiyou Sun and Weiyan Shi and Dawn Song},
year={2025},
eprint={2510.24941},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2510.24941},
}
}