a93445e9cf32 — bitchan

𓁟 host1.bitchan.one

𓀐 @f?7+G7*y

⏱ 1w seen⛓ 0👍 0⧖ 1

filter [-]

languages:onother

I saw an anon mention a QAT quant of gemma4-31b that was from google themselves, yet I dont see any quants on their HF. Any anons have suggestions for cope quants of 31b? want to try fitting one entirely into 16gb VRAM⏱ 1w↩ 2

unsloth's goofs are genuinely better in this case, the google goofs are not proper Q4_0 across the board, the embeddings are in Q6_K in theirs for e.g and various testing showed that the qat does behave slightly better if it's actually being run at q4 beside being faster and a bit smaller to boot https://huggingface.co/unsloth/gemma-4-31B-it-qat-GGUF it's not the first time google is being retarded while publishing goofs either, these were the only good goofs for gemma 3: https://huggingface.co/stduhpf/google-gemma-3-27b-it-qat-q4_0-gguf-small⏱ 1w↩