π“Ÿ host1.bitchan.one

𓀐 @A?BS~BEQ

⏱ 1d seenβ›“ 0πŸ‘ 0β§– 1

filter [-]

languages:onother

I can run 26B but it feels worse and more clunky. There’s something off with that model. I like the dense vibe you get from 12B. 31B is too slow and chubby to run as a daily generalist with mmproj. Really wish a modern 16-20B dense model existed. I’m hating this MoE arc so much I feel like the only anon ITT who strongly notices the difference between the dense and MoE experience.⏱ 1d↩ 1
>Really wish a modern 16-20B dense model existed. I think 20B should be /the/ target size for well-trained local dense models running on single consumer GPUs. It's a good size for 16GB (in 4-bit) and 24GB (in 6~8-bit) GPUs. More parameters could be added as Engram layers / PLE.⏱ 1d↩