KAIST Maps Internal Reasoning Patterns in Qwen and Gemma
Researchers from KAIST and Naver AI Lab found that LLMs like Qwen and Gemma show distinct internal activation patterns for different reasoning steps, offering a new path for AI safety.
A new study by South Korea's KAIST and Naver AI Lab reveals that the step-by-step reasoning of large language models corresponds to distinct, identifiable patterns within their internal states. To investigate this, researchers analyzed how Qwen2.5-7B, Qwen3-8B, and Gemma4-31B solved mathematical problems. They categorized the solution paths into eight reasoning operations, such as extraction, decomposition, formula recall, deduction, and computation, using GPT-5 to label the text segments.
The team discovered that these reasoning operations are clearly separable within the models' internal representations, with the strongest signals appearing in the middle layers. This separation is not merely a byproduct of word choice or token position. In fact, common function words like "a," "is," or "the" share overlapping representations in early layers but diverge in middle and later layers based on the active reasoning step. Furthermore, when researchers blocked attention to the preceding 30 tokens, the signal for the active operation weakened, indicating that these steps build on prior context rather than emerging in isolation.
Crucially, these internal patterns remain identifiable even when a model fails to solve a problem correctly; a flawed computation step still looks like computation internally. The researchers successfully replicated their findings using Llama-3-8B, and classifiers trained on Qwen3-8B transferred effectively to the GPQA-Diamond and MATH-500 benchmarks.
For AI practitioners and safety researchers, these findings provide a potential mechanism to monitor and steer models during generation. While reading a model's written chain of thought is a common oversight tool, previous research from Anthropic showed that models only disclose their internal hints in 25 to 39 percent of cases. By looking directly at internal vectors, developers might bypass deceptive or incomplete text outputs to verify if a model is genuinely reasoning or merely generating superficial text.
This is our own summary of reporting by The Decoder
