𓁟 host1.bitchan.one

𓀐 @4t-8BqSQ: /lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

/lmg/ - a general dedicated to the discussion and development of local language models. News >(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple >(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

⧖expiring ~28d0/43 kept2d ago

𓀐 @EJAJXux*

⏱ 2d 👍 0

⬆ upvote 0

[⇥] sort oldest ⏱

⬆[⏱]₿⚠

👍👎✔

𓀐↩⛓⚠

[⇥] flip, oldest first

filter [-]
𓀐 @PF~?QGmV: Poptards are celebrating but I have not seen one (1) cheap gpu link.+ 1

⬆ upvote 0↩ reply 1

𓀐 @fi3GgEeT: prices are the last thing to go down first the crash needs to happen+

⬆ upvote 0↩ reply 0

𓀐 @nQXSbTuk: Should I short my GPUs? They are worth three times what I bought them for.+

⬆ upvote 0↩ reply 0

𓀐 @CSGZn+tz: multiple forms of distillation. One is making them cheaper by distilling them into smaller models through weight distillation. Another is just training on its outputs which is also called distillation, another is training super specialized models for every task you can think of then distilling it (OPD) into one raw pretrained / lightly post trained model to keep it as intact as possible while giving it each's skills.+

⬆ upvote 0↩ reply 0

𓀐 @ho9*--xy: hate to be that guy but.. >qfn3.8 iq3 working on fixing a rig retargeting script for a few hours >no progress >toss the script to the free claude chat and say "fix this" >2 prompts and 5 minutes later it's fixed it's too far ahead. we're never gonna make it do we+ 2

⬆ upvote 0↩ reply 1

𓀐 @i=?*fLHt: Stop posting about ai models. We only allow people to post vocaloids here. Vocaloids are more on topic than your post that mentions claude.+ 1

⬆ upvote 0↩ reply 1

𓀐 @rT6DV64i: You're such a fucking bitch it's incredible. Is all that tranny talk just projection? I've noticed trannies are into niggers do they make you feel feminine? Is that why you post them?+

⬆ upvote 0↩ reply 0

𓀐 @wMQvSk8D: Just got here. What got all the trans anons so angry?+

⬆ upvote 0↩ reply 0

𓀐 @RHdaJe4W: the bubble popped?+

⬆ upvote 0↩ reply 0

𓀐 @FdV?ZzmP: glm-5.5-flash... qwen4... when...+

⬆ upvote 0↩ reply 0

𓀐 @p37X*Zed: I really want a model that's small but has decent tool-calling, JSON, and writing abilities to use for game integrations.+ 3

⬆ upvote 0↩ reply 2

𓀐 @gY9hw7Js: >decent tool-calling, JSON How is that an issue with constrained grammar?+ 1

⬆ upvote 0↩ reply 1

𓀐 @SMbFCVRL: The models I've tried that depend on game integrations before tend to give me error messages relating to JSON formatting or failed tool calls, I think it's because they're not really fine-tuned for that but it's probably because I have to run smaller models at cope quants to get any reasonable speed.+

⬆ upvote 0↩ reply 0

𓀐 @KpxMt8sZ: I haven't tried it yet personally, but depending on how small, you could try Spark-X2.6-4B, an anon in a previous thread was happy with it for the size+

⬆ upvote 0↩ reply 0

𓀐 @achTVLVo: Troon and pedo thread+ 2

⬆ upvote 0↩ reply 2

𓀐 @3rVbiUXS: yes, and?+

⬆ upvote 0↩ reply 0

𓀐 @92GeYuxZ: both groups are preferable to poltard spergs+

⬆ upvote 0↩ reply 0

𓀐 @5?7dts2t: And you should go to reddit and let normal people stay here. You are not normal.+

⬆ upvote 0↩ reply 0

𓀐 @S+ZWhQEX: >dariobot gets told to fuck off for posting non-local off-topic >schizo suddenly arrives and starts calling everyone a tranny to derail the thread hmmmm i'm noticing+ 1

⬆ upvote 0↩ reply 1

𓀐 @hhyy7HR=: tranny being wordfiltered would improve the quality of the website 1 quadrillion fold+

⬆ upvote 0↩ reply 0

𓀐 @aSskV-Uc: so.. uh.. anyone want to talk about.. anthropic?+

⬆ upvote 0↩ reply 0

𓀐 @iSTLXvst: Looks like they are going all in on GLM.+ 2

⬆ upvote 0↩ reply 1

𓀐 @TChH?hs-: when was the last time they released a 32B? like never?+ 1

⬆ upvote 0↩ reply 1

𓀐 @4c~7U~pE: that is misleading. They are not releasing models that size. Those are specialized models being trained to then OPD into the main model+

⬆ upvote 0↩ reply 0

𓀐 @bRaMp3WT: >Local is saved! Thank you Modi+

⬆ upvote 0↩ reply 0

𓀐 @k?9cnC7+: How do we get more normies on local?+

⬆ upvote 0↩ reply 0

𓀐 @6w5TDoxL: https://thinkingmachines.ai/news/inkling-2-small/+ 1

⬆ upvote 0↩ reply 1

𓀐 @XgAJo2Ca: >594B >"""small""" yeah sure, if that's small then my penis is microscopic+

⬆ upvote 0↩ reply 0

𓀐 @pHMbXByP: NVFP4 is the biggest shill scam on the planet, makes models fucking retarded unless they were natively fp4 trained. Do not use NVFP4 quants of BF16 models.+

⬆ upvote 0↩ reply 0

𓀐 @F6KT-TTR: ripple 65b is so peak it's like i'm using a modern version of llama2 the finetunes will go crazy+

⬆ upvote 0↩ reply 0

𓀐 @pG5JD_q_: >0.5T 17BA jfc Looks cool but I doubt anyone here can run it+ 1

⬆ upvote 0↩ reply 1

𓀐 @ut+jQJ7c: >can run Lots of people here think 5tps is acceptable speed. For a RP chat, especially with reasoning off or minimal, it probably is.+

⬆ upvote 0↩ reply 0

𓀐 @G6-Hchet: sorry for posting xitter screenshot but is this true? could an /lmg/ chad with lots of hardware bootstrap RSI with kimi?+

⬆ upvote 0↩ reply 0

𓀐 @mgaFR3iz: Just installed 96GB of DDR5. Q3.8FN running at 80 tokens / second now. Feels good, man+

⬆ upvote 0↩ reply 0

𓀐 @RsZJwRwT: umm where's qwen4? it was supposed to release today..+

⬆ upvote 0↩ reply 0

𓀐 @Dc*NZjMk: ripple is as good as 27b if it writes well (a big if) then it might be good for agentic rp+ 1

⬆ upvote 0↩ reply 1

𓀐 @W=+7hr7U: >65B as good as 27B WOW!!!!!!!!!!!!!!!!!+

⬆ upvote 0↩ reply 0

𓀐 @kNyXe364: It takes a lot of learning to become a useful Assistant for User!+

⬆ upvote 0↩ reply 0

𓀐 @k3acz-AT: >rtx 2060 >r5 4600g any good models for this?+

⬆ upvote 0↩ reply 0

𓀐 @7N8A32Q?: what do you guys think of agentic rp?+

⬆ upvote 0↩ reply 0

𓀐 @TBxMEX?H: https://huggingface.co/collections/pfnet/plamo-3-translate https://www.preferred.jp/en/blog/tech/plamo-3-translate-weight-release new king of japanese translation model just dropped! >PLaMo 3 Translate 31B is a translation-specialized LLM developed by Preferred Networks (PFN) that delivers world-class translation quality. It is built on PLaMo 3 NICT, a pretrained model developed through joint research by PFN and the National Institute of Information and Communications Technology (NICT). We further trained this base model on tens of billions of tokens of translation-focused data. For development details, see the tech blog. >PLaMo 3 Translate 31B supports translation among 53 languages (see Supported Languages). It can also take surrounding context and a glossary into account, and lets you choose between plain and polite style for Japanese output. The maximum length during training was 32,768 tokens for the input and output combined. The model can translate long passages in one go, but split anything longer so that the input and output together stay within this limit.+

⬆ upvote 0↩ reply 0

𓀐 @SeQx5c*z: Finally jumped on the AI train over the weekend and got Ollama, and installed Open WebUI last night. >gemma4:26b >Added "You are Gemma, a large language model..." system prompt >Turned thinking off >Used roleplay mode instead of assistant mode >Still get this at any hint of potential nudity What am I doing wrong? How are you guys able to basically do whatever you want with such ease?+

⬆ upvote 0↩ reply 0