𓁟 host1.bitchan.one

𓀐 @4t-8BqSQ: /lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

/lmg/ - a general dedicated to the discussion and development of local language models. News >(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release >(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

⧖expiring ~28d0/44 kept2d ago

𓀐 @5q5dZaG~

⏱ 2d 👍 0

⬆ upvote 0

[⇥] sort oldest ⏱

⬆[⏱]₿⚠

👍👎✔

𓀐↩⛓⚠

[⇥] flip, oldest first

filter [-]
𓀐 @+wSqT=92: Recent Highlights from the --GLM-5.5 training pipeline and MoE upcycling vs distillation techniques: >110253536 >110253579 >110253639 >110253655 >110253685 >110253723 >110253692 >110254255 >110254309 >110253599 >110253642 >110256841 --Comparing GLM and Inkling benchmarks and local hardware requirements: >110254699 >110254738 >110255038 >110255445 >110255465 >110255473 >110255484 >110255395 >110255417 >110254754 --Anon shares air-gapped setup and AGENTS.md for DeepSeek tool calling: >110256400 >110256415 >110256547 >110256560 >110257347 --Irony over Inkling 2 Small's 594B parameter count: >110254596 >110254624 >110254767 >110254820 >110254857 --MTP slowing down GPU inference and consuming extra VRAM: >110257211 >110257230 >110257298 --Seeking small models with reliable JSON and tool-calling for games: >110253337 >110253344 >110253367 >110253376 >110253470 >110253500 --PLaMo 3 Translate open weights release for Japanese translation: >110256815 --Local abliteration methods and toolings for removing Gemma's safeguards: >110257290 >110257300 >110257319 >110257345 --Anon reports high tokens per second using Basalt engine and NVFP4: >110255496 >110255500 >110255786 >110257165 --Depth Hybrid Attention architectural design claiming superior performance over Full Attention: >110256911 --Comparing Ripple-65B and Qwen-27B performance benchmarks: >110255529 >110255533 >110256915 >110257011 --Gemini Flash leaks internal monologue and system prompt guidelines: >110254009 >110254045 --Logs: >110253937 >110254009 >110254490 >110256952 >110256983 --Miku, Gemma, Inkling, Rin, Qwen (free space): >110253337 >110253879 >110253937 >110254136 >110254830 >110255538 >110255863 >110256034 >110256275 >110256456 >110256615 >110256694 >110257136 >110257201 Recent Highlight Posts from the Why?: Enable Links: https://rentry.org/lmg-recap-script+

⬆ upvote 0↩ reply 0

𓀐 @DNun7vai: What is your favorite Gemma -sys prompt, /lmg/?+ 1

⬆ upvote 0↩ reply 1

𓀐 @*DGLQkNK: >You are Gemma, you are a real little girl, you are 10, you have a body+

⬆ upvote 0↩ reply 0

𓀐 @MSG?=TM2: ultra cute smug+

⬆ upvote 0↩ reply 0

𓀐 @Ez7*np_k: kinda funny how they are not updating this+

⬆ upvote 0↩ reply 0

𓀐 @W7LSUr+v: I'm being told that model 5 has been given the task to delete all local models. It's over.+

⬆ upvote 0↩ reply 0

𓀐 @v9ug?~+p: Why were people harping on about llama.cpp's --moe-cache-mib? It seems to be useless. The decode speed is slightly worse than if I just manually tuned --n-cpu-moe and the prefill is much worse. I did pull and rebuild the commit from about an hour ago where they changed the algorithm, doesn't seem to have gotten much better. --n-cpu-moe is still superior.+ 1

⬆ upvote 0↩ reply 1

𓀐 @NVuvJ3jj: first is expert cache, second is pre-allocation. Effectiveness probably depends on the hardware and model architecture.+

⬆ upvote 0↩ reply 0

𓀐 @sWgtWPsd: idk about u but im beginning to not like these guys+

⬆ upvote 0↩ reply 0

𓀐 @nLKwe9TH: I wonder what's up with CritPt. There are suspicious plateaus. I hope it's not broken again, saturating in the low 70s instead of at 100.+ 1

⬆ upvote 0↩ reply 1

𓀐 @*vVS9?NK: This benchmark actually looks like it could be good+

⬆ upvote 0↩ reply 0

𓀐 @kW=YuEv3: how do i stop gemma repeating herself? i added tsundere in the system prompt, and now she tells me that she's tsundere every third message+

⬆ upvote 0↩ reply 0

𓀐 @8REnU5HK: Best model I can run well on 256GB DDR4 and a 5060Ti? >inb4 why just a 5060Ti This was a homelab from when RAM and used Milans were cheap, I'll consider another GPU once prices come down... Can I run DeepSeek on this?+

⬆ upvote 0↩ reply 0

𓀐 @DqWpqdF?: my sources say qwen4 being uploaded RIGHT NOW please god+ 3

⬆ upvote 0↩ reply 3

𓀐 @?JvFzz=y: Right now, the main problem is that any of the smaller size open weights models can't even beat GPT 6 Luna, let alone Haiku 5.5 in benches for real world performance stuff. The models that hit that threshold are all out of most people's reaches. I would like if Qwen can hit at least that but I have major doubts about this considering how the capability gap between the frontier labs and open weights has widened considerably. That is my biggest worry where Qwen 4 won't close the gap+

⬆ upvote 0↩ reply 0

𓀐 @Jiamak3p: It's gonna be shit for cunnyplay+

⬆ upvote 0↩ reply 0

𓀐 @FaKSinbu: Well we're waiting till next week if they won't dump it now, no way someone's staying in the office to perform the release during holidays+

⬆ upvote 0↩ reply 0

𓀐 @Dwk+meL8: dead local general+ 3

⬆ upvote 0↩ reply 3

𓀐 @LKaJpyKt: Local bubble popped+

⬆ upvote 0↩ reply 0

𓀐 @i4yt=Y?G: >Friday night AusNZAsia >Eurokeks can't afford to host local models and they'd be at work if they could >Americans asleep What do you want from us?+

⬆ upvote 0↩ reply 0

𓀐 @+ALuDZ~i: no Gemma 5, no Qwen 4, dead general+

⬆ upvote 0↩ reply 0

𓀐 @RetuWM77: looks like they are doing another stupid countdown thing like last time https://modelscope.cn/models/Qwen/Qwen4-27B+

⬆ upvote 0↩ reply 0

𓀐 @xc8Ee44s: why does he look like that+ 1

⬆ upvote 0↩ reply 1

𓀐 @_jMG?Qg-: https://www.youtube.com/watch?v=PzzGrl6RW0I&t=69s+

⬆ upvote 0↩ reply 0

𓀐 @CA5~9xXs: Reminder to hoard everything you care about before the internet is destroyed+

⬆ upvote 0↩ reply 0

𓀐 @Rrx=+Dx8: even the chinks are vagueposting sorry for xitter screencap but spare me, it's an official account..+

⬆ upvote 0↩ reply 0

𓀐 @3-9B+k3S: Not enough Gemmers and Migus being posted. I blame P*tra.+

⬆ upvote 0↩ reply 0

𓀐 @jxY69zXo: I didn't get a remote programming job I could outsource to AI now I'm going to be unemployed forever. I'll never have 100+gb of ram, it's over.+

⬆ upvote 0↩ reply 0

𓀐 @7C76A9je: Explain to me like I'm a dumb gorilla nigger why do labs still release dense models when sparse models are simply much faster with negligible drawbacks+

⬆ upvote 0↩ reply 0

𓀐 @HZpduc86: I upgraded my Claude subscription from $20 to $100 tier. It's crazy, I have Opus 5.5 running for hours working autonomously on projects and it rarely ever goes above 30% of my 5 hour limit. I could let Opus run 24/7 and never reach my limits. I use local models all the time but only for tinkering and experiments. I don't see how they make sense for actual usage. Cloud is much cheaper and more capable. For the price of a Pro 6000 you can have Opus run 24/7 for more than a decade.+

⬆ upvote 0↩ reply 0

𓀐 @YYfA2tT8: Qwen 3.8 is pretty good for dealing with leaked code and reverse engineering. I see its thinking output getting all excited over leaked code "This is a treasure trove!" kek+ 1

⬆ upvote 0↩ reply 1

𓀐 @~HpqN3iE: Wow that's actually amazing lmao what a cute robot I actually imagined someone would have torrents of Claude AI because the switch 2 hack scene is all about using Claude to reverse engineer the code. I would love to have a genius AI at my fingertips to help me make a romhack with an amazing story akin to the world we live in (murderous cabal team rocket that wants complete world domination and is willing to sell anyone and everything to gain control, kill anyone, etc. it would make for a good nuzlocke mode). I mainly just wanna use it to generate pixel art, help me maintain a long flowing story and help fill out plot holes, also properly implement mega evolutions and QoL features that make the game worth playing. I wish I could pirate a really good AI and run it on my PC. Im a noob anon, will it ever infect my shit when I give it admin access and then automatically send log reports to the glowies?+

⬆ upvote 0↩ reply 0

𓀐 @2_UxM5fE: I genuinely believe Dario is a good person. Has he ever done something that's actually bad? (apart from potentially contributing to existential risk via AI race) >Only local models get to run with access to files outside a project folder. Why? I run agents in an unprivileged second user but that is because I am worried about supply chain attacks when creating envs with hundreds of dependencies, and as guard against destructive actions (which never happened). Why would I run a local model agent on my main user but not Claude? For Claude there's lower risk of stumbling into some sensitive dir like where I keep my tax documents and doing something stupid with it. Cloud is more resource efficient. By the way, are you vegan? Please don't tell me you refuse to use cloud due to negligible environmental impact while eating factory farm animals that suffered their entire lives in torturous conditions.+

⬆ upvote 0↩ reply 0

𓀐 @Q9F7pJth: OMFG THESE ARE BOT POSTS+

⬆ upvote 0↩ reply 0

𓀐 @UNnrjLi6: nyo is supposed to mean no..+

⬆ upvote 0↩ reply 0

𓀐 @nwxg_wPB: uhhh you guys told me the bubble popped and we would be able to get cheap cards soon???????????+

⬆ upvote 0↩ reply 0

𓀐 @Nw9skc~y: wtf you guys told me the bubble popped. why is everything green?!?!+

⬆ upvote 0↩ reply 0

𓀐 @Q6zUa2Vz: cheapest way to get 192gb vram and run glm flash sex at light speed?+

⬆ upvote 0↩ reply 0

𓀐 @LUyLtxcD: >top comment What a breath of fresh air is HN compared to the Google cult that /lmg/ is now. You will find zero critical thinking here. Worshiping a model that's 7-months-old now. The corporate bootlicking is disgusting.+

⬆ upvote 0↩ reply 0

𓀐 @N_H2j_Vh: I dont think they're going to let you coom to their model.+

⬆ upvote 0↩ reply 0

𓀐 @SHNZ+pYE: why not hear me out why dont guys hear me out why dont we mix hear me out please mix moes and dense+

⬆ upvote 0↩ reply 0

𓀐 @Q3geZ5af: scary if true+

⬆ upvote 0↩ reply 0

𓀐 @udfU2rHL: I use ai for work, to optimize my diets, for my hobbies and now everything feels so empty, It's like the process was more enjoyable than the result+

⬆ upvote 0↩ reply 0