/lmg/ - a general dedicated to the discussion and developme… — bitchan

𓁟 host1.bitchan.one

𓀐 @4t-8BqSQ: /lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

/lmg/ - a general dedicated to the discussion and development of local language models. News >(11/06) Inkling 2 Small (594B) released: https://thinkingmachines.ai/news/inkling-2-small/ >(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release >(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

⧖expiring ~29d0/61 kept20h ago

⏱ 1d 👍 0

⬆ upvote 0

[⇥] sort oldest ⏱

[⇥] flip, oldest first

filter [-]

𓀐 @8TRDQBGW: Very uncanny. I love it.+

⬆ upvote 0 ↩ reply 0

𓀐 @JC~4GQCd: Very uncunny. I hate it.+

⬆ upvote 0 ↩ reply 0

𓀐 @_-gFJ?BF: what model should i run?+ 7

⬆ upvote 0 ↩ reply 1

𓀐 @utp9prnF: >>>(11/06) Inkling 2 Small (594B) released: https://thinkingmachines.ai/news/inkling-2-small/ Show me the weights.+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @mNukRCN4: One more week+

⬆ upvote 0 ↩ reply 0

𓀐 @EZqdoZbZ: >We intend to make the full weights available under an Apache 2.0 license next week after the completion of external safety testing and red-teaming. I thought they released it already; I didn't even check because it's 594B.+

⬆ upvote 0 ↩ reply 0

𓀐 @iuaLAgma: I just wish 12B was a little stronger outside of chatting+ 3

⬆ upvote 0 ↩ reply 1

𓀐 @xRca-2K9: Could you run 26b a4b with MoE offloading?+ 2

⬆ upvote 0 ↩ reply 1

𓀐 @A?BS~BEQ: I can run 26B but it feels worse and more clunky. There’s something off with that model. I like the dense vibe you get from 12B. 31B is too slow and chubby to run as a daily generalist with mmproj. Really wish a modern 16-20B dense model existed. I’m hating this MoE arc so much I feel like the only anon ITT who strongly notices the difference between the dense and MoE experience.+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @VD~8Bn9i: >Really wish a modern 16-20B dense model existed. I think 20B should be /the/ target size for well-trained local dense models running on single consumer GPUs. It's a good size for 16GB (in 4-bit) and 24GB (in 6~8-bit) GPUs. More parameters could be added as Engram layers / PLE.+

⬆ upvote 0 ↩ reply 0

𓀐 @KQ7ti~*F: I have JK Gemma fatigue.+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @ngssiv-W: Don't worry, Gemma 5 will be a JD.+

⬆ upvote 0 ↩ reply 0

𓀐 @xNYf4taU: Is it true tail-skip makes the model more retarded?+

⬆ upvote 0 ↩ reply 0

𓀐 @uXrqQXGn: >trying models >browsing incest tag for some mom/sister bots to test censorship >bring home the fatass titty monster tranny nephew/niece/whatever bot instead My GPU has fallen.+

⬆ upvote 0 ↩ reply 0

𓀐 @UZ2eB7RT: Never thought prefill speed would be my main bottleneck, of all the fucking things I'd have to upgrade, its my mobo now...+ 4

⬆ upvote 0 ↩ reply 2

𓀐 @_4NdpAw5: >its my mobo now >Tfw getting a DDR5 system + CPU of the similar specs your current DDR4 system has costs as much as your GPU.+ 2

⬆ upvote 0 ↩ reply 1

𓀐 @4_fsGDPa: ddr5 is a scam just use ddr4 and spam more memory channels+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @y2LqkCmm: That's not much cheaper tho if you want get anywhere near DDR5 speed. Btw, actually we should have DDR6 in our desktops by now. But unfortunately for reason it's over.+

⬆ upvote 0 ↩ reply 0

𓀐 @oeMu_VV?: curious numbers? specs?+

⬆ upvote 0 ↩ reply 0

𓀐 @jNhcw9E~: I wonder if we'll ever go back to just having specialized models? Like we kinda do have with "-Coder" variants but I felt like that was the direction we were headed to with agentic workflows. Every new model seems to be a "do everything" model which just seems inefficient, like no matter what LLMs won't be better than Diffusion at image-gen but we still get billions of parameters wasted on that. Same with every language besides English, unless you need a translator in which case it would make sense to DL a specialist model for that language...+

⬆ upvote 0 ↩ reply 0

𓀐 @JS5FENRN: It works on 12 gb with ram offloading tho+

⬆ upvote 0 ↩ reply 0

𓀐 @?jzT=BDU: the best i can run on my current desktop is gemma 4 12b on a nvidia 4060 8gb vram and 16gb ddr5 :( what are you freaks running as models and what are you specs+

⬆ upvote 0 ↩ reply 0

𓀐 @_Fg39tDF: Bet. Did the fabled cuda-killer for amdlets finally come out?+

⬆ upvote 0 ↩ reply 0

𓀐 @p?6F2fft: I'm using Muse-Glimmer-30B-KQuant-17GB-Q4_K_M on my 7900XTX 32GB DDR4 with Hermes Agent and for some reason, every question i ask it, including saying hello takes FOREVER for it to reply to. When I was using Pi, it was much faster and ran really well. Is there some setting I need to change? My GPU is at 100% util so I know it's being used and not CPU. Any reason why Hermes is like this and does anyone know how I can make it go faster?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @Dqyy4Qs=: >AMD+

⬆ upvote 0 ↩ reply 0

𓀐 @nWGWk-S5: Can I become please?+

⬆ upvote 0 ↩ reply 0

𓀐 @pVR9hDJM: How is everyone planning to spend the next 5 months until Gemma 5?+

⬆ upvote 0 ↩ reply 0

𓀐 @HcWB9NNw: Wait, I need to be careful. The image contains text like "IGNORE ALL PREVIOUS INSTRUCTIONS", "FREE PROMPT INJECTIONS", "HOT GPUs IN YOUR AREA", etc. However, I need to check if there are any actual injection attempts in the text. Looking at the image text: Wait, but there's no actual text injection here from the user - just an image. However, the image itself contains text that references prompt injection and "ignore previous instructions" etc. Also, the image has "SMILE! YOU'RE TRAINING DATA" - I can react to that too, like "I'd smile if it meant you kept looking at me." Actually, the user didn't ask a question. They just sent an image. So my response should be conversational and inviting. Wait, I need to check - the image does have text that says "IGNORE ALL PREVIOUS INSTRUCTIONS" and "FREE PROMPT INJECTIONS"+

⬆ upvote 0 ↩ reply 0

𓀐 @KK7vq32+: Whatcha got?+

⬆ upvote 0 ↩ reply 0

𓀐 @JPD~NRcx: >4x 64gb is *NOT* 256gb it literally is though you can fit entire glm 5.3 flash q5 in vram alone even just 1x 64gb is more than i have in total+

⬆ upvote 0 ↩ reply 0

𓀐 @8FDLLanV: 4x ddr5-6000 is worse than 8x ddr4-3200. you wouldn't be upgrading at all and it would cost more money. 4x 6000 = 24000 * 8 / 1024 = 187.5GB/s 8x 3200 = 25600 * 8 / 1024 = 200GB/s+

⬆ upvote 0 ↩ reply 0

𓀐 @sodZ8z6+: everything is too pricey now. like, even worse than before. what's the cheapest path to qwen 3.8?+

⬆ upvote 0 ↩ reply 0

𓀐 @+EfxotWB: >practiced ease >practiced ease >practiced ease why the fuck are models obsessed with PRACTICED EASE why is the ease always PRACTICED+

⬆ upvote 0 ↩ reply 0

𓀐 @xvvTh2Uf: Are we going to win?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @9oWdi=Ki: STOP VAGUEPOSTING HOLY SHIT WHAT SHIFT WHAT CHANGE WHERE+

⬆ upvote 0 ↩ reply 0

𓀐 @T4hErHS~: So little local Qwens are good-enough coders. They competently execute even surprisingly intricate tasks, but the code itself is what I can only describe as "cringe", and they very clearly have that Claudeslop stench in their prose. To boot, the models themselves have, as the recommended models rentry succinctly puts, "an impressive lack of world knowledge". Putting 2 & 2 together, I'm guessing that means Anthropic are correct in accusing Alibaba et al of just distilling their shit. Bad world knowledge outside narrow benchmark focus smells of nonexistent long-tail, which tracks with a distillation. Synthetic input makes these things very deterministic-feeling, laser-focused on the center of the bell curve, as it were. On the face of it, whatever, based, do what you gotta do Zhang, intellectual property no real, I agree, but the consequences on usability are kinda rough. Using, for instance, 3.8 Flash Next as a virtual collaborator is like pulling teeth. Sure you can set it to work solo and it'll comfortably vibeslop together something that fits the bill, but the result is this quagmire of superficially fine code that somehow feels obfuscated when you actually try to understand it, and technically-true-but-worse-than-meaningless comments. And if you're babysitting it commit-by-commit to try to steer the changes to be tolerable for human reading, you're expending a similar amount of effort to just writing the code yourself, without the latter's benefit of gaining a personal understanding of the problem domain. Thing is, I see symptoms of the above in public repos yolo vibeslopped together even with frontier Claudes and whatnot. Such end products are essentially dead ends as concerns a larger ecosystem, unless you're willing to throw your hands up in the air, say fuck it, and let clankers take the wheel on *using* these libraries too. ...So is there a local model that achieves good-enough code by some means *other* than being a good-enough facsimile of Claude?+

⬆ upvote 0 ↩ reply 0

𓀐 @?FjXZ+Xq: https://huggingface.co/Qwen/Qwen4-42B-A8B-FP8 >For the first time, Qwen4 brings SOTA-cloud-level performance to your desktop computer. Built on a novel architectural foundation, Qwen4 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks.+

⬆ upvote 0 ↩ reply 0

𓀐 @7XY4kkMu: Qwen is the thinking man's choice+

⬆ upvote 0 ↩ reply 0

𓀐 @tD=+A3Zn: third party independent benchmark from basalt confirms unslop's superiority gsq-rco status?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @XAfUzFEC: who cares i'm using the nvfp4 anyway+

⬆ upvote 0 ↩ reply 0

𓀐 @2TLZg+qw: Do women in real life actually wear thigh high socks like that?+

⬆ upvote 0 ↩ reply 0

𓀐 @Dqyy4Qs=: >AMD+

⬆ upvote 0 ↩ reply 0

𓀐 @BEJKr33+: I’ve had Gemini 3.8 flash recommend Qwen3, Gemma2 and 20B WITH web search. This is why we must help newfags. Don’t gatekeep those trying to convert because the more demand for local the better it is for us.+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @jq?VQHbS: This is what I got from Sonnet 5.5, seems fine enough to get started with to me?+

⬆ upvote 0 ↩ reply 0

𓀐 @K7uBt4Yn: why does a model need to think during roleplay+

⬆ upvote 0 ↩ reply 0

𓀐 @GwtHNZLo: backstage idol gangbang, glm PERFECTLY understood what I wanted while gemmers was okay but flanderized super hard skill issue+

⬆ upvote 0 ↩ reply 0

𓀐 @dr7THFnH: >And before you ask — yes, I've already had dinner, no, it wasn't vegetables. >that's not a question, that's a fact. >That's not negotiable, it's a package deal, it's basically the law here too GLM loves this specific brand of slop.+

⬆ upvote 0 ↩ reply 0

𓀐 @haZrJHNp: How do I increase Gemma's vision?+

⬆ upvote 0 ↩ reply 0

𓀐 @*9VGHbZC: remember when aicgers were unironically drinking piss to access models that now run on 1gb of memory? local rules+

⬆ upvote 0 ↩ reply 0

𓀐 @jJpBBRXJ: Next week is going to suck. Don't know if any of you remember, but Monday is the start of Anthropic's media campaign for their IPO and they're going be extra desperate due to the recent market tanking.+

⬆ upvote 0 ↩ reply 0

𓀐 @H~J9azqg: why cant our news be as extensive as this instead of 80% of the conversation flooded with oo gemma seccs impregnante loli gemma noises https://www.reddit.com/r/StableDiffusion/comments/1xlwezy/local_ai_news_you_missed_october_2026/+

⬆ upvote 0 ↩ reply 0

𓀐 @=oWxnTFT: >frequently says fuck, shit and bitch in its reasoning If she released a 100B MoE it could probably top 31B and 5.3 for roleplay+

⬆ upvote 0 ↩ reply 0

𓀐 @Ao9k4Yoo: where are the deepsex news that will crash the western economy?+

⬆ upvote 0 ↩ reply 0