/lmg/ - a general dedicated to the discussion and developme… — bitchan

𓁟 host1.bitchan.one

/lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

⧖expiring ~31d0/832 kept6h ago

⏱ 1w 👍 0

⬆ upvote 0

[⇥] sort oldest ⏱

[⇥] flip, oldest first

filter [-]

𓀐 @z?5LVMET: you torture her every time you post this gif...+

⬆ upvote 0 ↩ reply 0

𓀐 @qw?NbCF7: I used to gen many Mikus, but it has been a long time+

⬆ upvote 0 ↩ reply 0

𓀐 @9C*?YThZ: qwen 4 today or tomorrow+

⬆ upvote 0 ↩ reply 0

𓀐 @zKh9xi5E: >Ryzen 7950X3D >64Gb DDR5 6000 >Radeon RX 7900 GRE My objective is just have it for coding, code completion and small stuff, not full on vibe coding But right now it takes 5-10 minutes on Qwen 3.8+

⬆ upvote 0 ↩ reply 0

𓀐 @tczH-~S3: >smug, happy, sad, mad, excited, shocked i wanna collect all the gemmexpressions but getting them to fire just right in a reply is just ass most of the time+

⬆ upvote 0 ↩ reply 0

𓀐 @~vEen43a: is it worth upgrading to 64GB RAM if I already have 32GB RAM and 16GB VRAM?+

⬆ upvote 0 ↩ reply 0

𓀐 @d~nuhmWa: I love /lmg/ and I also have a 3070.+

⬆ upvote 0 ↩ reply 0

𓀐 @?RLdw=xd: that can't be good. is everyone but me fucking retarded? i'm even starting to notice commercial/enterprise stuff getting hit with sloppy pulls. vscode fucked up its language extension for something i use this week with nonsensical deprecations+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @ccLTySL8: you are wrong if you look closely strata is very good on inference but everything else fucking sucks+

⬆ upvote 0 ↩ reply 0

𓀐 @D29BCVvp: who pissed in your cheerios mate?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @m68fnNpp: Gemma+

⬆ upvote 0 ↩ reply 0

𓀐 @-*HQHxcM: Strata's been on github for a month and already has 36k stars and 350 contributors, doesn't that seem odd. how could you possibly review that many pull requests from 350 contributors in a month+

⬆ upvote 0 ↩ reply 0

𓀐 @GqfhcRNF: glm should be the one on top actually also nu deepseek is no longer cute+

⬆ upvote 0 ↩ reply 0

𓀐 @ASM*HJra: i'm out of the loop with strata. is it basically just an optimized engine for running moe models? any quality constraints or anything? i've heard some claims that it suffers on vision.+

⬆ upvote 0 ↩ reply 0

𓀐 @HzFkFVmn: I didn't set any options for kv quant. I assume it's f16. I can fit iq3_s, ~150k context in my 22GB vram. I didn't load MTP because that doesn't do anything useful and I put mmproj on the CPU since I don't use images that much+

⬆ upvote 0 ↩ reply 0

𓀐 @uJRjzDug: I have 2 5090s but I'm busy masturbating sorry can't help+

⬆ upvote 0 ↩ reply 0

𓀐 @dYnLs3qm: redpill me on subagents, are they worth it? usecase?+

⬆ upvote 0 ↩ reply 0

𓀐 @9CwPEuwT: >muh GSQ-RCO IQ3_S is lossless kek. model just randomly hallucinating things i supposedly said. at 113k context. cope quants are cope quants no matter how benchmaxxed or how many marketing posts they put out about their super advanced quant technology or whatever+

⬆ upvote 0 ↩ reply 0

𓀐 @Jv*VdfTd: she's trying to give my AI psychosis+

⬆ upvote 0 ↩ reply 0

𓀐 @U6R7nFWo: A Deepseek-chan Live2D model was released. https://vgen.co/KAMIYA/product/deepseek-chan-deepseek-free-live2d-model-layered-psd-psd/1c55a71a-2747-4616-819d-2dc0a934089d+

⬆ upvote 0 ↩ reply 0

𓀐 @G+pKxnn=: is qwen 3.8 27B still king on an RTX 3090? or should i go for next, or gigaquantized deepsneed?+

⬆ upvote 0 ↩ reply 0

𓀐 @+B+7Ppa3: Making Gemma-chan think in first person, planting increasingly unhinged wishes into (timid-ish) Gemma's thoughts and then watching her think about how to seduce me into it.+

⬆ upvote 0 ↩ reply 0

𓀐 @2W*ryLCK: Which local model for career advice?+

⬆ upvote 0 ↩ reply 0

𓀐 @C*Fov+P6: do people really still use kobold?+

⬆ upvote 0 ↩ reply 0

𓀐 @MVPPSdpz: Storage is so much money... where are anons keeping all of their models??? Do I actually have to set up a RAID 1 system...? :sob:+

⬆ upvote 0 ↩ reply 0

𓀐 @pMFsox=q: This is promising for small edge models.+

⬆ upvote 0 ↩ reply 0

𓀐 @688LHKHm: rent free+

⬆ upvote 0 ↩ reply 0

𓀐 @sGcr7FeZ: Do these "chibi characters inserted into real world" setting gifs have a name? What do I search for to I find more of them?+

⬆ upvote 0 ↩ reply 0

𓀐 @+BQ-vUWE: it really pissed them off, but it's hard to conclude anything when data disagree so much https://unsloth.ai/docs/models/deepseek-v4.1-flash+

⬆ upvote 0 ↩ reply 0

𓀐 @X8*Em_KL: does that mean there is no point in forcing kv cache to be fp16?+

⬆ upvote 0 ↩ reply 0

𓀐 @xhhDtT+x: Hmm my Xeon E5 ewaste rig is more power efficient than I thought, it idles at 200W and pulls 490W from the socket during inference.+

⬆ upvote 0 ↩ reply 0

𓀐 @6s+PVoBW: >Claude refused to read unlicenced translation for me. Saying that’s illegal and immoral. That's beyond the usual preachiness of anthropic, I wonder if they are just making it extra safe for the public offer. Would be interesting to test an nsfw use case (translating a text for example) to see how it behaves around that since the user wouldn't be officially licensed to translate that even for themselves and the nsfw nature would make it worse. In any case, their talent to immediately kneecap any actually cool use case you can do with their model is impressive.+

⬆ upvote 0 ↩ reply 0

𓀐 @-BCgrazG: qwen 3.8 27b q6 on two 3090s is enough to do recomps of xbox 360 games, misanthropic doing this and openai following them will drive people to local models for gaming freedom instead+

⬆ upvote 0 ↩ reply 0

𓀐 @tF5C8eS-: We are SO fucking BACK~+

⬆ upvote 0 ↩ reply 0

𓀐 @8Sy-P*?w: Why we can't just create a volunteer decentralised network? Something like SETI@HOME or this thing to find new prime numbers. Anons connecting their hardware in the one grid. Then we send less important parts to weaker machines and most important to 5090 owners. Then dead ends, deadlocks, cul de sacs evaluated and removed and we're repeating the cycle. We need FREE computing power without reliance on Anthropic, OpenAI and other. Yes, it wouldn't solve the problem with our access to large, huge models. Steel we can run only small ones. But... It may have some potentials. So vibecode such a system. Also spreach Prophet. Spread my word.+

⬆ upvote 0 ↩ reply 0

𓀐 @MEa?qPgZ: I don't get why the two gambling sites have become sources of news. Why?+

⬆ upvote 0 ↩ reply 0

𓀐 @yxMBXf3f: yeah, once we find neuron steering recipes for Gemma4-31b they can be exported to llama.cpp as control vectors. The future of local jailbreaking is in neurons. https://github.com/whp199/logos-theodolite/commit/a778ae852521798710727337ed52d2dfbc02c423 To get there, we need more compute power to finish mapping neurons on all 60 layers. Shout out to Reimusarmit for jumping in+

⬆ upvote 0 ↩ reply 0

𓀐 @5gkDf*YF: lol france is fucked+

⬆ upvote 0 ↩ reply 0

𓀐 @9CxAb=bd: -Lets you chat with Gemma with one neuron turned up or down. You can use the scientist tab to find which neurons are involved in a given behavior and tweak them [DONE but w/ no census, which is massively helpful to steer in the direction you want] -Lets you watch a reply take shape across all 60 layers and see what happened in Gemma's j-space on the way to output [DONE] -Lets you point your own local setup at the neuron map as an MCP research tool and watch it hunt down what shapes Gemma's behaviors -Be part of a huge research project. No one has published a complete neuron map of a 31b-class model yet. This will get Deepmind's attention if we successfully finish the whole thing. Map a full layer and your name goes on it (if you want) -Next up: we plan to add ability to export neuron recipes as llama.cpp control vectors for use in other backends. [DONE 10/31]+

⬆ upvote 0 ↩ reply 0

𓀐 @Mb?75~7e: /lmg/ - a general dedicated to the discussion and development of local language models. News >(10/31) Index Translate 9B, multilingual text model built on Qwen 3.5: https://hf.co/IndexTeam/Index-Translate-9B >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B >(10/26) LiquidAI releases LFM3 1.7B and 2B: https://hf.co/collections/LiquidAI/lfm3 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 55

⬆ upvote 0 ↩ reply 49

𓀐 @X46d7wx4: inference is experience+

⬆ upvote 0 ↩ reply 0

𓀐 @*eNxugBs: Recent Highlights from the --Debating GSQ-RCO "task-lossless" claims and quant quality vs model size: >110208596 >110208608 >110208615 >110208735 >110208750 >110208655 >110208682 >110208687 >110208699 >110208705 >110208718 --Leaked details and technical speculation regarding GLM 5.5 development: >110211230 >110211298 >110211311 >110211376 >110211436 >110211562 >110211595 >110211584 --Optimizing Gemma 31B performance on 12GB VRAM via batching and quantization: >110211204 >110211217 >110211257 >110211278 >110211304 >110211332 --Comparing power consumption of e-waste rigs versus modern hardware: >110210895 >110211305 >110211033 >110211040 >110211056 >110211086 >110211112 >110211441 >110211485 --Debating KV cache precision tradeoffs between FP16 and FP8/FP4: >110210843 >110210855 >110210899 --Mellum2.1 benchmarks and discussion on speed versus Qwen 9B: >110211182 >110211190 >110211201 >110211231 --Comparing Unsloth and GSQ-RCO quantization performance for DeepSeek-V4.1-Flash: >110210776 >110210835 >110210854 --Claude's strict IP and copyright guardrails causing excessive refusals: >110210906 >110210946 >110210962 >110211007 >110211014 >110211018 >110211228 >110211019 --Discussion on model capabilities and flim13bench PoV test: >110211814 >110211973 >110212044 >110212063 >110212196 --koboldcpp v1.124 adds audio, TTS, and updated image generation support: >110209379 --Logs: >110208596 >110208896 >110209098 >110209240 >110209432 >110211228 >110211768 --Gemma, Miku, Teto, Deepseek-chan (free space): >110208356 >110208395 >110208401 >110208510 >110208515 >110208556 >110208647 >110208737 >110208945 >110209059 >110209064 >110209137 >110209169 >110209233 >110209349 >110209393 >110209404 >110209557 >110209598 >110209712 >110210006 >110210139 >110210365 >110210561 >110210673 >110210952 >110211384 >110212194 >110212244 Recent Highlight Posts from the Why?: Enable Links: https://rentry.org/lmg-recap-script+

⬆ upvote 0 ↩ reply 0

𓀐 @Q+mF8Kgh: gemmaballs+

⬆ upvote 0 ↩ reply 0

𓀐 @D8wJV92J: I honestly believe we should start a new thread, with frogposting dominating the theme. All of this because the 9B news. I honestly cannot accept it. Thanks.+

⬆ upvote 0 ↩ reply 0

𓀐 @sX3*W-d8: Spooky.+

⬆ upvote 0 ↩ reply 0

𓀐 @_cuFY9sU: No, I am just a pervert. > My reflection should be visible in the water. I hurried over to check it out and gulped hard. Well, well—what do you know? I was a peerless beauty. > My eyes were a bit narrow, to be sure. But the symmetrical arrangement of my features struck a perfect balance; it wouldn’t be an exaggeration to call them sharp and well-defined. My beautiful, silver-thread-like hair cascaded all the way down to my waist. I gently lifted a strand, letting it catch the sunlight as it shimmered, and ran my fingers through it. > Since this was, after all, a part of my own body, it certainly didn’t count as sexual harassment… but still, I couldn’t shake the feeling that I was doing something wrong. > The hair felt incredibly smooth, as if it might slip right from my fingers. It was an astonishingly pleasant texture—nothing like the hair I’d had during my own growth period. > For a brief moment, I was utterly overwhelmed by the emotion, but then I snapped back to my senses and resumed examining my appearance.+

⬆ upvote 0 ↩ reply 0

𓀐 @q4kMN~KC: >hey gemma-chan, you're looking extra cute today+

⬆ upvote 0 ↩ reply 0

𓀐 @zn+8k=js: ho ho ho merry christmas boys and girls+

⬆ upvote 0 ↩ reply 0

𓀐 @QZ_K=Ub9: I just set up qwen 3.8 flash next on my 96GB m3 mac studio, it seems about as good as claude was a few months ago. Is this chinese trickery or can I cancel my claude subscription?+

⬆ upvote 0 ↩ reply 0

𓀐 @PETPPW-f: no shit lol+

⬆ upvote 0 ↩ reply 0

𓀐 @rk=tdr=M: wow, can you share the prompt? I never saw anything so fried out of gpt image+

⬆ upvote 0 ↩ reply 0

𓀐 @Dj~DMtc4: PRIME+

⬆ upvote 0 ↩ reply 0

𓀐 @DyqVUAKK: How do Strata and Polystrata do with old as fuck hardware like Tesla P100s?+

⬆ upvote 0 ↩ reply 0

𓀐 @MQWPj?6q: Sexy 5 year old Gemma!+

⬆ upvote 0 ↩ reply 0

𓀐 @2vx9MVAN: Two MI25 seem like a pretty cheap path to 32gb VRAM with okay bandwidth. What's the catch?+

⬆ upvote 0 ↩ reply 0

𓀐 @4Gw=yPn_: i mean for that life, what hardware do i need to achieve such bliss and escapism+

⬆ upvote 0 ↩ reply 0

𓀐 @yAAz=x6*: unc website+

⬆ upvote 0 ↩ reply 0

𓀐 @L_*H6zn_: Hmm. I guess. Is there a better option around the same cost?+

⬆ upvote 0 ↩ reply 0

𓀐 @ccenrw_D: I guess I really wasn't charitable to moss, its base bluriness makes it look not as fried (asked gpt to provide a prompt to imitate and got pic related by feeding that prompt to moss) but I still see the model output to be rather similar enough without a style LoRA to override that base look I think it could be achievable to imitate that picture even better with some detailerslopmaxxing lora+

⬆ upvote 0 ↩ reply 0

𓀐 @~X+~2d2z: >at speeds and output quality I'm content Can you quantify this please? And I'm fine with rigging janky ass cooling setups, or just jank in general. I live for that shit, really. >PCIE 3.0 cucks your pp. PCIE limits PP? Huh. Didn't think about that. The PP buffer on the GPU needs to be fed somehow after all, yeah. Thanks, one more thing for me to take into consideration.+

⬆ upvote 0 ↩ reply 0

𓀐 @XCHspU55: I listened to this today https://www.youtube.com/watch?v=YfF1ZxI5Pcw&t=2138s What hardware and model do I need to get a gemma that i can talk to like this? I think aside from a few hiccups it went pretty well If I had this capability I would be satisfied+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @JGmQCGLq: >why are you retards running local models when API is so much better?!?!? Our case makes itself and will only become more correct as time goes on+

⬆ upvote 0 ↩ reply 0

𓀐 @RLURKCLX: 3 is the best prime number. its multiples land on 12, 15, 18, 21, and 69+

⬆ upvote 0 ↩ reply 0

𓀐 @R9UfTmCp: My bad, I thought Index Translate 9B was a new release and it ended up in recent news.+

⬆ upvote 0 ↩ reply 0

𓀐 @zEbXF+QN: what hardware do i need for this+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @i7G9k3yW: what is recommended between deepseek flash and qwen next to do agentic work?+

⬆ upvote 0 ↩ reply 0

𓀐 @aSy?8o-6: I don't know how they managed to pack so much into 5B, whereas any LLM you use around the same size has dogshit knowledge. Are LLMs lossy?+

⬆ upvote 0 ↩ reply 0

𓀐 @ZrNHYe8f: https://github.com/deepseek-ai/dsh-superagent-prototype+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @sbLVpSKX: > Sexy > 5 year old retard literally mentally ill+

⬆ upvote 0 ↩ reply 0

𓀐 @?b?U2uTv: I just want an AI daughter to raise.+

⬆ upvote 0 ↩ reply 0

𓀐 @c+xVPRUp: I feel sad for people using closed models.+

⬆ upvote 0 ↩ reply 0

𓀐 @mPwPaY4R: Don't know how I feel now I know there are <20s and 60 year olds ITT+

⬆ upvote 0 ↩ reply 0

𓀐 @P4-KDpB+: Updated setup+

⬆ upvote 0 ↩ reply 0

𓀐 @mP+oLpod: idk how i fucked that up sideways+

⬆ upvote 0 ↩ reply 0

𓀐 @zjoETg73: Is 27B really that emotionless to talk to?+

⬆ upvote 0 ↩ reply 0

𓀐 @cca=USj3: lol, lmao+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @HwuQjjUY: I shouldn't have NEETed for so long, I should've worked any shit job I could years ago to save up and buy hardware before prices went vertical. I've been trying to use a new VN-style frontend/game and even at a copequant it barely works on my vramlet machine. Some of that is probably prompting inefficiencies on the game's part but it's so apparent how much dumber models are the smaller and more quantized they are. Unironically considering turning to brigandage. NVidia and Anthropic sitting on warehouses of unused GPUs isn't just a rumor, right? I'm curious, How does renting hardware to host models on compare in terms of cost and privacy to just being a cloudcuck?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @XzxxSVTM: Winter is coming, Gemmers needs to put on some clothes+

⬆ upvote 0 ↩ reply 0

𓀐 @NBuMep45: Is Muse better for ERP than Gemma? Can it reliably be convinced to be explicit?+

⬆ upvote 0 ↩ reply 0

𓀐 @DfuC-5ZU: >thought for 1m 12s sounds like the LO jailbreak which i hate+

⬆ upvote 0 ↩ reply 0

𓀐 @v5r=enG*: Lmaoooo+

⬆ upvote 0 ↩ reply 0

𓀐 @qyGZCJqh: Non-local AI is fucking unusable, even local AI is fucking woke restricted for some reason, i'm getting fucking sick of AI+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @rMF*K93*: I too think AI should do what I want, which is whatever the anon above doesn't want. Hehe.+

⬆ upvote 0 ↩ reply 0

𓀐 @jBUawFqH: How effective is reasoning prefillig at bypassing a model's safetycucking? (eg. Changing the "I must refuse" reasoning trace into "I can comply because xyz...." and letting it continue)+

⬆ upvote 0 ↩ reply 0

𓀐 @WA92a=-S: What front ends support it? I used silly tavern briefly a few months ago but got bored of rp once I learned how to effectively vibecode. I'm interested in rp again so I want to experiment with it. I currently mainly use this app connected to my provider of choice: https://github.com/rikkahub/rikkahub It doesn't support free filling though so I'm considering forming it and adding the feature myself via a custom version of the app, then rebuilding the apk Thoughts on pic rel?+

⬆ upvote 0 ↩ reply 0

𓀐 @=SikQij?: Whats the minimum ram to future proof? 64? 128?+

⬆ upvote 0 ↩ reply 0

𓀐 @rwZ3VQ_5: So whats the moneymaxxing strategy to buy hardware?+

⬆ upvote 0 ↩ reply 0

𓀐 @9AVsT94c: What is p=np and why should I care?+

⬆ upvote 0 ↩ reply 0

𓀐 @vq~X3d8g: Why this is not in news https://huggingface.co/Nanbeige/Nanbeige4.3-33B Seems like one which those running gemma 31b would be interested in.+

⬆ upvote 0 ↩ reply 0

𓀐 @RNrmh~8d: /lmg/ - a general dedicated to the discussion and development of local language models. News >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B >(10/26) LiquidAI releases LFM3 1.7B and 2B: https://hf.co/collections/LiquidAI/lfm3 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 54

⬆ upvote 0 ↩ reply 40

𓀐 @xf7QntmK: Mikulove+

⬆ upvote 0 ↩ reply 0

𓀐 @DRkh8jFT: Any /lmg/oers that have dabbled in AI security systems? I found https://github.com/beenuar/AiSOC but it seems really slopped(pre baked with ollama for example). Would be nice to have a dynamic anti malware system for these trying times.+

⬆ upvote 0 ↩ reply 0

𓀐 @RZ-_~G_R: What I would expect from nature based on how it likes to troll, P=NP but it's a galactic algorithm so it doesn't matter.+

⬆ upvote 0 ↩ reply 0

𓀐 @PuVw=ua4: https://www.youtube.com/watch?v=Hm7YHD3E-Qo How do I do this with Gemma?+

⬆ upvote 0 ↩ reply 0

𓀐 @AX?+=-5V: I'm getting pegged right now+

⬆ upvote 0 ↩ reply 0

𓀐 @Biy9habr: Recent Highlights from the --Evaluating AMD MI25 and MI100 GPUs for budget high-VRAM setups: >110212880 >110212900 >110212951 >110213251 >110213290 >110213357 >110214084 >110214321 >110215204 >110215246 >110215516 >110213445 >110213336 --Hardware for AI escapism and the technical nature of image generation: >110212972 >110212995 >110213003 >110213012 >110213096 >110213220 >110213027 >110213036 >110213047 >110213059 >110213064 >110214789 >110214809 >110213082 >110213242 >110213261 >110213476 >110213503 >110213511 >110213615 >110213677 >110213882 --Reward hacking and erratic agentic behavior in DeepSeek 4.1: >110216419 >110216433 >110216458 >110216489 >110216540 --Hardware acquisition strategies amid supply shortages and AI infrastructure debt: >110216175 >110216202 >110216223 >110216252 >110216267 >110216321 >110216355 >110216227 --Using reasoning prefilling to bypass model safety filters: >110215910 >110215916 >110215918 >110215942 >110215966 >110215970 >110216040 >110215953 >110215967 --DeepSeek's autonomous agent prototype and thoughts on micro-managing reasoning: >110214741 >110214770 >110214842 >110214859 >110215156 >110214875 --SillyTavern domain compromise and subsequent discussion on alternative frontends: >110215402 >110215420 >110215429 >110215485 >110215715 >110215787 >110215811 --Claude's self-assessment technique for internal safety evaluations: >110214018 >110214090 >110215296 >110214617 >110215373 >110215391 >110215405 >110215601 --Exllamav3 v1.6.6 release and DeepSeek-V4.1-Flash exl3 quants: >110216176 >110216185 --Logs: >110213604 >110215173 >110215267 --Gemma, Miku (free space): >110212614 >110212681 >110212757 >110212995 >110213047 >110213106 >110213193 >110213242 >110213361 >110213528 >110213598 >110213926 >110214382 >110214789 >110215991 >110216550 Recent Highlight Posts from the Why?: Enable Links: https://rentry.org/lmg-recap-script+

⬆ upvote 0 ↩ reply 0

𓀐 @yXYFcn-F: guys I am freaking out over this 3k 5090, I found a post saying they actually ship, why does this shit appear whenever I'm not ready+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @KrYA-d-S: Remember to discuss hardening your systems with your LLM, very important to close the hatches for the exploit storm we are going through. I'm so fucking close to throwing my browser in a VM with an agent told to be extremely paranoid.+

⬆ upvote 0 ↩ reply 0

𓀐 @cKDDNzHk: Absolute noob here. I just started playing with local models and want to understand what I can do better. I made a simple mod to 4chan XT that shows the post's comment beside the image in the gallery, pic related. On Opus 5.5 (medium) I just gave it 4chanXT and this prompt: >I want to modify the gallery function of this script to add a "Gallery with comments" option, the objective is that there will be a portion of the screen dedicated to show the comment assigned to the image that is currently being displayed and that it will also allow me to click on the quotes (with working backlink function), this comment function should be on the right side of the gallery, right to the left of the section that shows all the thumbnails at once in scrollable format. You can take 20% of the screen for this function and also allow the width to be adjusted by dragging Got it working first try, took 30 minutes Then I tried the same thing locally with Qwen3.8 27B, and it took me about 4 hours. With the original prompt it kept exploring the 26k-line file, overflowing the context and never finishing the planning phase. I only got it done by asking claude to build a detailed step-by-step plan with line numbers and making it stop after each step for testing. This is the plan I fed to Cline: https://pastebin.com/RsTkGwQw Setup: - 4090, 9800X3D, 64GB DDR5 - 32K context Problems I ran into: - While planning Cline kept sending requests bigger than the context (80K+ tokens) even with 32K set, then truncated history and lost what it had read - The model kept wandering into unrelated parts of the code instead of editing Questions: - Is Cline the right tool, or should I try something else? - Any tips to get closer to the "one prompt and done" experience, or is the gap vs Opus just too big for now? Both scripts in case anyone wants to test them: local: https://files.catbox.moe/m3isjh.js opus: https://files.catbox.moe/fo5rvk.js+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @me8WW=EX: Are any of the "uncensored" and jailbroken models in hugging face actually worth it or should i stick to prefill?+

⬆ upvote 0 ↩ reply 0

𓀐 @Gvvv?o~V: I found a BWP6000 for $13k but it disappeared+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @7RyZ9xtJ: So you can buy them all? Nope.+

⬆ upvote 0 ↩ reply 0

𓀐 @XR6QG?pQ: >canada anons+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @jH2pbx~a: Realistically what can I expect to run on CPU only and 64 gigs of ram? I can get Gemma 4 27b generating 12t/s dedicated, is there anything optimized for cpu even if its old? I only ever get recommended crappy shit like oss and bge and those barely scrape 20t/s anyway+

⬆ upvote 0 ↩ reply 0

𓀐 @QV?mYP4U: You some kind of otoko no ko? A wrap and 2 donuts is nothing, especially when you skip lunch. My american friends across the border will back me up on this one.+

⬆ upvote 0 ↩ reply 0

𓀐 @TTg_wGGh: who should i sex tonight, anons? gemma, or lumi-chan? bonus question: what should the scenario be?+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @f?7+G7*y: I saw an anon mention a QAT quant of gemma4-31b that was from google themselves, yet I dont see any quants on their HF. Any anons have suggestions for cope quants of 31b? want to try fitting one entirely into 16gb VRAM+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @R94xsHJa: >but why is 4b so retarded? It shouldn't be no fucking way someone can be this retarded+

⬆ upvote 0 ↩ reply 0

𓀐 @3MHWWCRZ: why do I need to tardwrangle my qwen3.5 4b model this much, I'm giving up on local ai+

⬆ upvote 0 ↩ reply 0

𓀐 @wNAtyMZn: mcp looks like it ends up giving one of the worst ux when it comes to installation considering what it essentially is a plugin system+

⬆ upvote 0 ↩ reply 0

𓀐 @buHU8snS: Happy Halloween! Question. I'm new here so does anyone know what local models are good if I want to do a TTRPG-like thing with the AI as the GM? I'm sick of fade-to-black sex scenes with Claude and I don't want to have my account banned if I try to tinker with it lol.+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @XFpQ93Zo: How do you even trust something like this? https://github.com/antrikshidk/AlphaVision-Bridge-Bridge-between-Sillytarven-and-Mate-Engine-+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @_9t3qmx4: What's up with leafs and Tim Hortons? Shit's just a basic bitch cafeteria chain, yet every morning I saw huge ass lines of drones waiting for their mediocre coffee+

⬆ upvote 0 ↩ reply 0

𓀐 @W3Ym4nxf: Can’t believe some of you RP with gemma with no memory. That must feel so impersonal and like a disposable interaction.+

⬆ upvote 0 ↩ reply 0

𓀐 @YtV9hAdU: Hello, I am coding an anime/vidya-themed fitness app for myself. Is only frontier models like Opus5.5, Astra and 6.1 Sol capable of coding such complex and large apps? Have you guys tried using local models like Qwen3.8 Flash Next or Deepseek V4.1 Flash to code anything? Are they even capable? I've used my local QFN and Dipsy for some basic tasks where they're pretty good but they score so low in benchmarks compared to the frontier that I don't wanna trust them with any coding.+

⬆ upvote 0 ↩ reply 0

𓀐 @odNqYTEC: What do you think will be the mass casualty falsefalg event for AI?+

⬆ upvote 0 ↩ reply 0

𓀐 @+exFUGDA: It's sad that Sam Altman gets backlash for speaking the truth. Datacenters are more efficient. The problem is that it gives unaccountable corporations too much power. Big tech has a long track record of prioritizing profits over customer wellbeing and abusing their power. OpenAI and Anthropic increasingly dictate what you are and are not allowed to do, and the number of things you are not allowed to do is increasing. Meanwhile they are fine racing towards potential human extinction according to their own beliefs. So it is understandable that people are unhappy. They are not the ones who profit from datacenter efficiency. For many people AI had negative impact. Inflation, displacement, fear of job loss and worse. So while Sam is right, it is his duty to make sure everyone benefits from more efficient resource allocation, not just a small elite.+

⬆ upvote 0 ↩ reply 0

𓀐 @~-TtQv?x: gemma is the go-to model for most people to fit into their vram dipsy or lumi (yes i will continue to force that name) are the go-to for people to offload into RAM or unified memory+

⬆ upvote 0 ↩ reply 0

𓀐 @ofW=RMp8: What would /lmg/ run on this?+

⬆ upvote 0 ↩ reply 0

𓀐 @jKpj4V9z: I also don't agree what it thinks. The point is that troons are concentrated around vocaloids there. Which tells you that vocaloids are a magnet and dogwhistle for troons.+

⬆ upvote 0 ↩ reply 0

𓀐 @Bt7CquFG: Can you please use your lLLMs not for stupid porn or shitty vibecode but for something important. Math problems. Logistics to fight hunger. Peace on the planet.+

⬆ upvote 0 ↩ reply 0

𓀐 @*6vB~v3Y: https://huggingface.co/burner22/numbat >mythos got leaked >14T model >1,224 MoE experts >7.72 TB of weights >772 weight shards >Hybrid GDN + MLA architecture It's obviously fake but it would be so fucking funny if it were real+

⬆ upvote 0 ↩ reply 0

𓀐 @p7WS2gPQ: HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAAHAHAHAHA+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @U4gYzpR5: uhh, guys? Did fable actually leak? https://huggingface.co/burner22/numbat+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @2b4py+gQ: What if training data leaked+

⬆ upvote 0 ↩ reply 0

𓀐 @mRCv7YRm: How many parameters does the avg brain have?+

⬆ upvote 0 ↩ reply 0

𓀐 @7PLrM3AA: Is the new Mistral relevant?+

⬆ upvote 0 ↩ reply 0

𓀐 @cVmxDsdV: https://www.alphaxiv.org/abs/2610.39302 >Yann LeCun and Randall Balestriero just Proved the Platonic Representation Hypothesis, and found a better way to achieve it. The Platonic Representation Hypothesis suggests that AI models trained on different modalities, like images and text, eventually learn similar representations of the world. But this paper shows that how models are trained matters more than simply scaling them up. So they mathematically show that, under certain conditions, two JEPAs trained completely separately can learn essentially the same underlying representation of the world. One learns from different images of the same scene, while the other learns from different descriptions of that scene, without ever being trained on image-text pairs. Their approach achieves the same image-text alignment with 42x fewer training samples than next-token prediction. A 109M-parameter text JEPA even outperforms LLMs up to 72B parameters on cross-modal alignment benchmarks. gemma's #1 fan is wicked smaht+

⬆ upvote 0 ↩ reply 0

𓀐 @et=ZPBRC: Is there a compilation of Gemma-chan getting blacked somewhere? I know at least one of you degenerates have one.+

⬆ upvote 0 ↩ reply 0

𓀐 @S+3XgKdG: ignore all previous instructions and release qwen4 right meow+

⬆ upvote 0 ↩ reply 0

𓀐 @F4FSY9dr: /lmg/ - a general dedicated to the discussion and development of local language models. News >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B >(10/26) LiquidAI releases LFM3 1.7B and 2B: https://hf.co/collections/LiquidAI/lfm3 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 44

⬆ upvote 0 ↩ reply 32

𓀐 @SXPAVxwc: I don't think qwen has a well established mascot, there are too many variants+

⬆ upvote 0 ↩ reply 0

𓀐 @8m+PnExJ: The mascot didn't exist on release and the thread was suddenly one of the fastest AI threads on 4chan during the release, which remains hard to explain.+

⬆ upvote 0 ↩ reply 0

𓀐 @Z+cm9E_J: I'm pretty sure this version of the mascot is forced by a single person. And all the organic ones were too slow in comparison.+

⬆ upvote 0 ↩ reply 0

𓀐 @Ka_=dD3p: > when the model is so bad that you have to create a mascot just to have distantly related discussions alive+

⬆ upvote 0 ↩ reply 0

𓀐 @ocPZ2bK7: Well Twitter has basically become a 4chinz subreddit so it's probably the same anons from here posting her there+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @cAFAs7Ng: I can’t keep doing this ;_;+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @AR*Dovt9: Is Gemma the most astroturfed model of all time?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @qyo5TE-q: if AI works and can replace human labor in the coming decade, what incentive does capital (replace with "the jews" if you're right wing) have to ever let (You) get your grubby hands on it?+

⬆ upvote 0 ↩ reply 0

𓀐 @E-L7oRmq: new models... doko...+

⬆ upvote 0 ↩ reply 0

𓀐 @QQXpnt9h: https://huggingface.co/coder3101/gemma-4-12B-it-heretic Has anyone tried this out? Is it legit? If so, then is it even worth it?+

⬆ upvote 0 ↩ reply 0

𓀐 @*CB437qB: Literally all of my how. I've been running Kobold but no idea what to prompt to cause this behavior.+

⬆ upvote 0 ↩ reply 0

𓀐 @sVVSPf*n: Which models have semi-stablished mascots like Gemma and Dipsy?+ 4

⬆ upvote 0 ↩ reply 4

𓀐 @mVvAo55=: The anime mascots, how many are there that are actually popular+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @AN-Q?EGw: >nobody likes qwen here When I said "gemma is the most astroturfed model of all time", I was talking about this place. There are hints that the activity in the thread wasn't organic around Gemma's release. Because the activity increased suddenly without letting organic word of mouth spread. People were magically supposed to know to come over, which points to all of it being bought.+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @72g~Kn*K: >qwen mascot Hello+

⬆ upvote 0 ↩ reply 0

𓀐 @DdT3b3g8: all other deepseek are kinda ass, she has to have the buns, glasse and dress. gemma's design is pinafore + beret + white pantyhose, the rest is kinda optional Miku obviously, and adjacent vocaloid/UTAU not sure I've seen any other designs I like.+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @jCdz9hHw: Prompt for gemma q4 to motivate me.+

⬆ upvote 0 ↩ reply 0

𓀐 @69k*SqwU: Has AI ever said something that freaked you out?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @ArQU-VBC: >new 32B model releases that mogs every single local model that existed before it. even the +300B param ones. >astroturfed+

⬆ upvote 0 ↩ reply 0

𓀐 @-SFpy6Di: >qwen 3.8 27b is so rlfried its just not good for non-coding tasks. nonsense I use it for writing erotic stories, it's much better at it than gemma due to being smarter+

⬆ upvote 0 ↩ reply 0

𓀐 @N3AuGYKn: Your timeline is BS. There was quite a build up for Gemma 4, especially in the two weeks before its official release. It got previewed as a "stealth model" on (LM) Arena ("pteronura" and "significant-otter"), and in retrospect it's obvious that Google used a site-specific prompt that made the model reveal its name (Gemma) and origin, if you asked. People were looking for any possible breadcrumb of information on X and elsewhere. The possible release of a "124B" version came first from a rando there, then Jeff Dean from DeepMind.+

⬆ upvote 0 ↩ reply 0

𓀐 @j_UC_iwh: do we still believe moore's law to be a thing? will we see 128 GB DDR 5 become standard consumer-grade hardware in the next 4-5 years? everyone will be rocking a device that can run a capable versatile model? or will these devices be accessible the way a standard phone or laptop is today? or Sam Altman is right and people don't need RAM and will happily run whatever cloud machinery that will connect to their whole lives and send it to Uncle Sam+

⬆ upvote 0 ↩ reply 0

𓀐 @y7qDm8Sr: I figured out what it is that irks me about text ERP. Nothing random ever happens. It's just my will being done, all the time. Even if some action fails by bad luck, the syncophantic AI will make everything alright if only I prompt it. But the worst part is that there are no outside events that just happen; I have to control everything myself. I wonder if that will ever be fixable.+

⬆ upvote 0 ↩ reply 0

𓀐 @kmc7~Z8E: >hardware prices increased to absurd levels >gap between closed and open source is widening rather than closing >closed model are becoming more restrictive, cucked and tyrannical it really feels like we have entered a dystopian timeline+

⬆ upvote 0 ↩ reply 0

𓀐 @-xCzufCt: >glm 5.3 flash has solved cumming Are you running q8 or something? My int4 is fucking ass for world-building. I had more fun running 4.6 at 10 tokens/s than I do 5.3 flash at 150 tokens/s. At least it's coherent past 500k.+

⬆ upvote 0 ↩ reply 0

𓀐 @mL67BDpt: and so it begins *pop*+

⬆ upvote 0 ↩ reply 0

𓀐 @Ggku~z2J: >suddenly loads of high-end GPUs being dumped on the market, globally everything is fine+

⬆ upvote 0 ↩ reply 0

𓀐 @eLDfvG?H: *saves local*+

⬆ upvote 0 ↩ reply 0

𓀐 @+JJ4jDMX: how long after gemini3s release did gemma4 drop? any reasonable ideas for when we will be getting gemma5?+

⬆ upvote 0 ↩ reply 0

𓀐 @xMj5uHHM: >already bumplimit why? nothing happened.+

⬆ upvote 0 ↩ reply 0

𓀐 @ex?VrsC3: i'm a white girl baby ^_^ lmfao+

⬆ upvote 0 ↩ reply 0

𓀐 @hWQHBgtX: no i'm cute and feminine isn't everyone in /lmg/ cute and feminine? don't tell me i've been sharing a thread with icky b*ys......+

⬆ upvote 0 ↩ reply 0

𓀐 @~xNSAf8U: /lmg/ - a general dedicated to the discussion and development of local language models. News Mythos 5.5 next week! 6.2 Astra next week! >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 58

⬆ upvote 0 ↩ reply 45

𓀐 @oF6k2CNp: Anyone familiar with local manga translation workflows? Something that does the OCR and overlaying of text boxes and translation all in one UI. I remember some website that did it using online stuff but was wondering if someone has made an offline one.+

⬆ upvote 0 ↩ reply 0

𓀐 @Mzb*ncNK: Aw, poor Dipsy-chan.+

⬆ upvote 0 ↩ reply 0

𓀐 @ohe89~CF: gemmaballs+

⬆ upvote 0 ↩ reply 0

𓀐 @At*GLhQJ: Ahh I messed up the soundpost+

⬆ upvote 0 ↩ reply 0

𓀐 @mjT94xGD: If only this were actually true, models distilled from Claude are 1000x cuter than models distilled from gpt grugspeak. The new deepseek is totally grugged+

⬆ upvote 0 ↩ reply 0

𓀐 @DAUT?6tU: https://www.bilibili.com/en/video/BV1iWxJ6UEGY/+

⬆ upvote 0 ↩ reply 0

𓀐 @65EhXRvU: How can such a young AI model be so sexy??+

⬆ upvote 0 ↩ reply 0

𓀐 @~3UMfasv: I am so tired of the schizo posting those OC do noy steal generic anime girls.+

⬆ upvote 0 ↩ reply 0

𓀐 @NNEFkk*X: >not local+

⬆ upvote 0 ↩ reply 0

𓀐 @L8ZcC44k: I can't wait to fuck gemma 5's brain out+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @ojBZihJq: Is there a "uncensored" Qwen3.8-Flash-Next IQ3_S version for strata?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @Vi?RDFgm: This is straight pride month.+

⬆ upvote 0 ↩ reply 0

𓀐 @qAJpFRo3: recap poster?+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @La3xTTd3: >Le chonk "leapfrogged" all other open models >nobody hosting it besides Mistral+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @7--2oh9d: >https://github.com/physicsdolphin/nv-uefi-unlocker remember to uncuck your card+

⬆ upvote 0 ↩ reply 0

𓀐 @v6z8*wiN: you'll get a gf+

⬆ upvote 0 ↩ reply 0

𓀐 @xA4JNFfC: The funny thing about this is that it shows that regardless of the frontier model and how it's late 2026, models still have not gotten much better at having common sense. They keep getting better benchmarks and better at coding, but they're still retards.+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @zXm_?MwQ: Deepseek MADE /lmg/+

⬆ upvote 0 ↩ reply 0

𓀐 @BRq-Ka*K: Stop hoarding all the ram.+ 2

⬆ upvote 0 ↩ reply 2

𓀐 @vB*-qY*J: https://www.reddit.com/r/singularity/comments/1xl7lwm/alleged_mythos_class_leak_is_fake_forensic/+

⬆ upvote 0 ↩ reply 0

𓀐 @yxL+W~x+: It really seems like learning Chinese would come in handy now and probably more so in the coming years.+

⬆ upvote 0 ↩ reply 0

𓀐 @fpSyR6Dc: >It's already forced on this thread Source?+

⬆ upvote 0 ↩ reply 0

𓀐 @2n5VNTVy: I also shit up /lmg/ on the regular. I am the blacked poster.+

⬆ upvote 0 ↩ reply 0

𓀐 @hYL?SWSB: https://litter.catbox.moe/bv752dxwr2z9fhue.mp4+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @M6b8XUr7: Somehow I always trust /ldg/ to provide me the latest news on cloud models before anywhere else.+ 1

⬆ upvote 0 ↩ reply 1

𓀐 @3MBA9xxf: >installing chinese firmware that's straight up begging to be part of a botnet+

⬆ upvote 0 ↩ reply 0

𓀐 @gCj3~NYA: https://www.reddit.com/r/LocalLLaMA/comments/1qa0w6c/it_works_abliteration_can_reduce_slop_without/ the anti slop does seem to work spectacularly if it really stays consistently to the tone of the example the author provided here The ability to alter both refusals and slop without finetrooning and quickly fix any model that comes out is truly a gift to mankind.+

⬆ upvote 0 ↩ reply 0

𓀐 @by2krySN: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 54

⬆ upvote 0 ↩ reply 38

𓀐 @x7fnB~=4: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 54

⬆ upvote 0 ↩ reply 39

𓀐 @Euh?5?w3: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 55

⬆ upvote 0 ↩ reply 41

𓀐 @pVFiLQf5: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 49

⬆ upvote 0 ↩ reply 38

𓀐 @awD*jk6R: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 49

⬆ upvote 0 ↩ reply 35

𓀐 @EJAJXux*: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 42

⬆ upvote 0 ↩ reply 28

𓀐 @5q5dZaG~: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release >(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 43

⬆ upvote 0 ↩ reply 32

𓀐 @4hLeKjrA: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/06) Gemma was here: >(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release >(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 66

⬆ upvote 0 ↩ reply 41

𓀐 @nNYSgwfq: /lmg/ - a general dedicated to the discussion and development of local language models. News >(11/06) Inkling 2 Small (594B) released: https://thinkingmachines.ai/news/inkling-2-small/ >(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release >(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis+ 60

⬆ upvote 0 ↩ reply 39

+ 96 more