https://saracandussio.github.io/tags/sparse-autoencoders/Sparse AutoencodersHugo Blox Builder (https://hugoblox.com)en-usWed, 01 Jul 2026 00:00:00 +0000https://saracandussio.github.io/media/icon_hu7729264130191091259.pnghttps://saracandussio.github.io/tags/sparse-autoencoders/https://saracandussio.github.io/publication/lmtaboo/Wed, 01 Jul 2026 00:00:00 +0000https://saracandussio.github.io/publication/lmtaboo/<h2 id="key-findings">Key findings</h2> <ul> <li><strong>Explicit constraints work.</strong> Without being told the forbidden words, models break the rules in about two thirds of cases. Prompting raises compliance above 90% for Gemma and for gpt-oss with reasoning, and constrained decoding reaches up to 98.5%.</li> </ul> <p> <figure > <div class="flex justify-center "> <div class="w-100" ><img alt="Compliance breakdown by model and constraint strategy." srcset=" /publication/lmtaboo/fig-compliance_hu4624855150758071424.webp 400w, /publication/lmtaboo/fig-compliance_hu2232433585483480342.webp 760w, /publication/lmtaboo/fig-compliance_hu6679561330521547985.webp 1200w" src="https://saracandussio.github.io/publication/lmtaboo/fig-compliance_hu4624855150758071424.webp" width="760" height="321" loading="lazy" data-zoomable /></div> </div></figure> </p> <ul> <li><strong>Reasoning helps follow the rules.</strong> gpt-oss-20b goes from 72% to 97% compliance under prompting when chain-of-thought is enabled.</li> <li><strong>How you constrain shapes what the model says.</strong> Constrained decoding pushes models toward more precise paraphrases, and human guessers do best on these descriptions (38% vs. 23% for prompting). SAE steering keeps descriptions informative but often violates the rules and switches language.</li> </ul> <p> <figure > <div class="flex justify-center "> <div class="w-100" ><img alt="Rule compliance versus how often a model judge guesses the target within 10 tries (pass@10)." srcset=" /publication/lmtaboo/fig-compliance-vs-guessability_hu11896575050429390626.webp 400w, /publication/lmtaboo/fig-compliance-vs-guessability_hu10380791304109304415.webp 760w, /publication/lmtaboo/fig-compliance-vs-guessability_hu3145358206015888656.webp 1200w" src="https://saracandussio.github.io/publication/lmtaboo/fig-compliance-vs-guessability_hu11896575050429390626.webp" width="760" height="272" loading="lazy" data-zoomable /></div> </div></figure> </p> <ul> <li><strong>Models are weak guessers.</strong> Even the better judge needs 5 to 10 guesses to approach human first-try accuracy.</li> </ul>