Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models
Entropy is widely used to decide which parts of a Chain of Thought can be pruned. Testing six reasoning models on math, logic and science tasks, we find that entropy-based selection never beats random pruning at the sentence level. Its apparent advantage at the token level appears only on math and is explained by numeric tokens, while activation patching shows that task information is spread across the whole reasoning trace.
Aug 11, 2026
