Primary
Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations
Primary source. Published by the vendor, CERT, regulator, government agency or researcher responsible.
arXiv:2611.22186v1 Announce Type: new Abstract: Chat models on phones and laptops are usually 4-bit copies of the model that was evaluated, and audits of such copies disagree about what quantization does to refusal. A 4-bit build is one of many weight perturbations of the same size, so a change in its behaviour could come from its specific rounding error or from any perturbation that large. In aโฆ
Read the full story at arXiv cs.CR (Cryptography & Security) โHackerNews24 shows a short summary only. The full article, its copyright and any corrections belong to the original publisher.