𓁟 host1.bitchan.one

𓀐 @4t-8BqSQ: /lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

/lmg/ - a general dedicated to the discussion and development of local language models. News >(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5 >(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response >(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss >(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B >(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis

⧖expiring ~26d0/55 kept4d ago

𓀐 @x7fnB~=4

⏱ 5d 👍 0

⬆ upvote 0

[⇥] sort oldest ⏱

⬆[⏱]₿⚠

👍👎✔

𓀐↩⛓⚠

[⇥] flip, oldest first

filter [-]
𓀐 @UMR8B3d?: Recent Highlights from the --Papers: >110234196 --Debating Strata's speed gains versus model intelligence degradation: >110233168 >110233185 >110233770 >110233877 >110233921 >110233936 >110233863 --Optimizing 7900 XTX performance in llama.cpp using ROCm and MTP: >110232461 >110232467 >110232492 >110232511 >110232565 >110232596 >110232607 >110232622 >110232640 >110232659 --Engineering neural circuitry and bare-metal AI execution via weight-space programming: >110232802 >110232926 >110232939 >110233045 --Adjusting logit softcapping to fix Gemma's repetitive outputs: >110234481 >110234506 >110234561 >110234569 >110234594 --Comparing Gemma-4 and Qwen for coding and multi-model workflows: >110234533 >110234544 >110234558 >110234548 >110234626 >110234647 --Analyzing network weights and the capability gap between frontier and open-weights models: >110231272 >110231304 >110231434 >110231542 >110231484 >110231698 --Potential of Weight-Space Programming to replace traditional finetuning: >110232513 >110232990 --Prompting strategies and performance of Gemma for explicit roleplay: >110231791 >110231801 >110231811 >110231820 >110232160 >110231829 --Comparing Gemma and Index-Translate performance on OtakuBench: >110231382 >110231458 >110231486 >110231531 --Weight-space programming effectiveness and implications for model training: >110232544 >110232556 >110232951 >110232960 >110234410 >110234433 --RWKV-8 RPU performance showcase and prompt sensitivity quirks: >110234114 >110234131 --Logs: >110231353 >110235321 --Miku, Gemma, Dipsy, Teto (free space): >110230872 >110230917 >110231046 >110231061 >110231117 >110231119 >110231575 >110231660 >110231724 >110231907 >110231949 >110232384 >110232455 >110232502 >110232545 >110233957 >110234153 >110234181 >110234652 >110234722 >110234725 Recent Highlight Posts from the Why?: Enable Links: https://rentry.org/lmg-recap-script+

⬆ upvote 0↩ reply 0

𓀐 @TVUS7Nv8: Tetolove+

⬆ upvote 0↩ reply 0

𓀐 @=~+Qsd4N: おいちぃー+

⬆ upvote 0↩ reply 0

𓀐 @XpgPW8wB: Gemma love!+

⬆ upvote 0↩ reply 0

𓀐 @ZVewe+BS: Is there a site where you can download hf models that require sending a request for access without doing so? Like https://huggingface.co/facebook/Large-Sparse-Reconstruction-Model for example. They gave my burner access so its kinda whatever, still annoying though. What speaks against sharing the access granted model URL, hf account login username and token (scope: download only)? So others can use the login info to download the models straight from HF. >HF will ban your account! Who cares, its just a burner account. If you want to get tricky with it, you can limit git login/download to the iprange of cloudflare WARP. So anon just has to use the free cloudflare WARP vpn (unlimited bandwidth, 100mbps+) for the download and HF wont be able to tell unless all of India is downloading through the account simultaneously, in which case token gets rolled over and reshared in the sekrit club. matter of fact, if the /g/ archive is still working, could just drop the info in the thread and people who need it later can just search the url in the archive (or a keyword we use like grabahug:URL;Username;Token) Surely grapegoon isnt gonna mind.+

⬆ upvote 0↩ reply 0

𓀐 @e3v*GZgi: 70b dense+

⬆ upvote 0↩ reply 0

𓀐 @Z+F~~*UX: Soon.+

⬆ upvote 0↩ reply 0

𓀐 @7-xNqQRw: Brand new language. Btw, you won't get a job unless you have 10 years of experience with it+

⬆ upvote 0↩ reply 0

𓀐 @WthmjXME: Her name is Kyla+

⬆ upvote 0↩ reply 0

𓀐 @ZFTDZ+?Z: The biggest giveaway that the labs DO have "internal ASI/RSI" is the fact that they're largely stopped hiring and are discussing how they are letting workers go that aren't needed anymore. Anthropic assumes that all their workers will be redundant by the end of 2027 and this actually scales with already released statistics.+ 1

⬆ upvote 0↩ reply 1

𓀐 @RLgw+L8X: will Claude give Dario the pink slip next year?+

⬆ upvote 0↩ reply 0

𓀐 @_q~?F?sh: what the fuck is littlelearner?+

⬆ upvote 0↩ reply 0

𓀐 @iUGu5w9E: He RT lolis on twitter+

⬆ upvote 0↩ reply 0

𓀐 @kec5VS-X: >16gb gpu Unfortunately I only have 12 GB on a 4070 super.+

⬆ upvote 0↩ reply 0

𓀐 @SGjZRJ-m: whats the right thread? i didnt find a place where anything outside of anime porn is talked about. let alone cloud stuff+

⬆ upvote 0↩ reply 0

𓀐 @_UHamaVn: Gemma 5 133B A6B+ 1

⬆ upvote 0↩ reply 1

𓀐 @w6=pRwH9: Gemma 5 600b40a. Fuck vramlets, fuck ramlets, fuck trannies, fuck jannies.+

⬆ upvote 0↩ reply 0

𓀐 @V96pAza6: The goal of a school isn't to teach people skills. 90% won't learn anything at all if you give them AI and don't enforce anything.+

⬆ upvote 0↩ reply 0

𓀐 @yeC*W?cL: public CA can't sign .local certs+

⬆ upvote 0↩ reply 0

𓀐 @5atu9CJz: GLM 5.3 Flash with Maya is kinda dumb. It writes great prose and then makes stupid mistakes with logical inconsistencies, physical positions, non sequiturs. Is it the Maya S copequant? Gemma 4 QAT doesn't do this.+ 3

⬆ upvote 0↩ reply 3

𓀐 @Hfe8GiaU: Has anyone tested these meme forks to see if the engine itself isn't bugged? llama.cpp had so many problems with degraded outputs over the years and that was with competent engineers reviewing the code and many people checking the output quality.+

⬆ upvote 0↩ reply 0

𓀐 @jNv*3Ze3: >GLM 5.3 Flash with Maya is kinda dumb. It writes great prose and then makes stupid mistakes with logical inconsistencies, physical positions, non sequiturs. Is it the Maya S copequant? Gemma 4 QAT doesn't do this. I use glm 5.3 flash at q4 to code all day every day and find it highly reliable (altho its a thinking hog).+

⬆ upvote 0↩ reply 0

𓀐 @*DZvFLqD: every time i change from models with different architecture i had to adjust settings and how i use it to get it going the way i want with the copequants+

⬆ upvote 0↩ reply 0

𓀐 @CvZBiFH~: Can anyone explain like im retarded the difference between all those gguf files on hugging face? https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main They range from 6 to 30 gigs. Are they all the same model? What are the difference between 1,2,3 bits etc, and q1, q2, q3, _K_XL and all that shit? I got a model working fine with ooga(qwen3.8 27B q2), im just trying to learn more but I don't seem to find any concrete info+ 4

⬆ upvote 0↩ reply 4

𓀐 @e9wPdexU: It's the same model but compressed so less accurate, or rather it will make mistakes more often+

⬆ upvote 0↩ reply 0

𓀐 @=LMKgMW-: >https://github.com/LostRuins/koboldcpp/wiki#what-are-the-differences-between-the-different-files-for-each-model-do-i-need-them-all-which-quantization-f16-q4_0-q5_1+

⬆ upvote 0↩ reply 0

𓀐 @aESDJeRA: it means the relationship between each concept is described with less subtlety the less bits you give it, so concepts and relationships that were comparable but one was slightly more related than another will be squashed to the same weight. This may or may not make the output worse for a given case, but overall always feels like an IQ reduction in model ability. How much worse it gets in real-world performance is on a kind of exponential curve where going from eg 32-bits to 16 or even 8 doesn't hurt much, but going to 2 or 1 basically scoops out the prefrontal cortex.+

⬆ upvote 0↩ reply 0

𓀐 @mEXKE=d3: less bits = shittier quality but might be faster more bits = better quality but might be slower if you have a 16gb gpu, get a 12gb model in order to leave some space for the model context+

⬆ upvote 0↩ reply 0

𓀐 @+XVb4nda: List of banned models: Gemma (all) Llama 2 littlelearner Midnight Miqu Mistral Small 3 Glimmer Do NOT discuss them. "Gemma-chan" images are also banned.+ 2

⬆ upvote 0↩ reply 2

𓀐 @2d-TJi6d: mythomax bros stay winning+

⬆ upvote 0↩ reply 0

𓀐 @JGfuJmaV: what about qfn+

⬆ upvote 0↩ reply 0

𓀐 @2KzNzv8W: The biggest giveaway that none of the labs have "internal ASI/RSI" is the fact that they're still all acting like a bunch of fucking idiots and grifters. If they had superintelligent AI advising them there's no way they'd be this retarded.+ 1

⬆ upvote 0↩ reply 1

𓀐 @ff4HJmFy: If they had a digital exponentially improving god they wouldn't be considering going public at all since they would soon become hypercorps and control 90% of the global GDP.+

⬆ upvote 0↩ reply 0

𓀐 @x5jey?gQ: I'm new to all this stuff cause I lost all my friends and I already have this 4090 so I figured I'd create some virtual ones for myself. Im currently using Gemma 4 26B with Heretic, is that basically the best model I can fit on this GPU? What about if I'm fine with waiting for an answer? I also have 64GB RAM so there is some space for it to spill over. I want it to have vision too like Gemma, cause I wanna be able to show the model a picture of my cock. I'm also curious what can I do to turn the model into a virtual facsimile of a real person as faithfully as possible, should I use SillyTavern for that? Can I stick all my texts with that person somewhere in there?+ 1

⬆ upvote 0↩ reply 1

𓀐 @NRA9YXak: hey anon ill be your friend and ill help you with what model you should be using do you have matrix/element? it's free and open source btw, and has a website if you don't want to download some untrusted program: app.element.io once u register post ur @ cheers+

⬆ upvote 0↩ reply 0

𓀐 @fdTdvgcy: >ISTA released 1bit dsv4.1f https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF Has anyone tested this? ISTA claims it retains 91% in coding TMB benchmarks but I assume it'd quite lobotomized in practice.+

⬆ upvote 0↩ reply 0

𓀐 @rwQYA3H-: Best small coding models that punch above their weight. I've been testing them a lot across various projects, big and small and using different languages and domains and they're consistently good (for their size). https://huggingface.co/openbmb/MiniCPM5-2B https://huggingface.co/XHToken/Spark-X2.6-4B https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B You need to use this jinja fix for it to work in llamacpp https://gist.github.com/coder543/d8f56cd6db67de4cafbb5bdb6c2dfb4d and it has no vision which is a huge let down, but it's so much better than the default 9B. 3.5-9B is still very good if you really need vision but it doesn't reason like this MiMo tune. There are no models between these and 27B/35B that are worth your time for coding. I've tried them all. 26B is okay but you need to put in the effort with the sysprompt to get her to work properly but the MiMo 9B is still a lot better.+

⬆ upvote 0↩ reply 0

𓀐 @EpG7SNvJ: I switched from Strata to Basalt and I'm using their NVFP4 model. It seems to be considerably more intelligent and more efficient compared to the iq3_s I was running on Strata, which felt a bit underwhelming which is why I wanted to try this other thing. I can't wait for the day when all of this meme engine shit has been solved and we don't have to guess which one is the most optimal to use.+

⬆ upvote 0↩ reply 0

𓀐 @N759Ejhq: How much longer will we have to endure this?+ 1

⬆ upvote 0↩ reply 1

𓀐 @KWwcGuQL: I vibecoded my own!!! What exactly? >stable diffusion cpp fork >I vibecoded a new frontend >I vibecoded my own launcher fro the stable diffusion cpp server effectively the launcher launches stable diffusion cpp with my select model (launched in my launcher). The web server is auto launched by my launcher. I also have nvtop+

⬆ upvote 0↩ reply 0

𓀐 @MQZkJ5Uw: Are we the baddies?+

⬆ upvote 0↩ reply 0

𓀐 @A3WWKdBF: So what the fuck is the point of a big unified memory rigs if you can apparently do the strata meme with a regular gpu+ram for 4x the speed with zero downsides?+

⬆ upvote 0↩ reply 0

𓀐 @BCkJviwq: Proud of yourselves?+

⬆ upvote 0↩ reply 0

𓀐 @WNTVhhv9: Technically not related to local models but I feel anons here would at the very least find this interesting: Student account Google Search will not be accessible to students without AI access enabled >“Search and AI are deeply integrated,...[If schools ban AI, students] will not have access to search, and they’ll see a message from their school that indicates it’s not available.” - Chris Phillips VP of Education at Google, New York Times interview, Nov 2, 2026 https://dnyuz.com/2026/11/03/google-will-turn-off-search-for-schools-that-reject-a-i/ https://futurism.com/future-society/google-search-engine-access-schools-ban-ai https://techcrunch.com/2026/11/03/google-will-let-schools-disable-its-ai-tools-but-not-without-sacrificing-search/+

⬆ upvote 0↩ reply 0

𓀐 @ordnroKo: Qwen 4 bros how we feeling+

⬆ upvote 0↩ reply 0

𓀐 @KKqm~~wb: my gpu suxs. whats a good website to make realistic porn?+

⬆ upvote 0↩ reply 0

𓀐 @QdFnxpBn: Can one of you /g/ods please gen a picture of Gemma slapping Gemini? Also NSFW if possible.(catbox or uguu se the image in that case)+

⬆ upvote 0↩ reply 0

𓀐 @WfGHT3qb: How difficult is VRAM extension for a card? I was wondering if it would be possible to find dead GPUs, unsolder the VRAM from them, then put that on a functional card. I'm not a hardware guy so I don't know much about this kinda stuff, but I know it has been done before.+

⬆ upvote 0↩ reply 0

𓀐 @ThRwV8fS: uh oh https://openrouter.ai/stealth/crouton-alpha >Crouton Alpha is a lightweight model built for scalable coding, agentic, and multimodal workflows. It's ideal for executing agentic sub-tasks, tool use, and document analysis in high-volume, repeated work. Developers can combine it with frontier reasoning models to execute the bounded steps that turn plans and information into action. >audio input uhhhhhh who is this+

⬆ upvote 0↩ reply 0

𓀐 @yGWyWcPU: >https://www.theregister.com/ai-and-ml/2026/11/03/british-fintech-revolut-slashes-ai-coding-costs-10x-by-cutting-out-claude-and-going-self-hosted/5307345 >To serve GLM-5.3 at scale, Revolut deployed 56 Nvidia B200 GPUs, equivalent in GPU count to seven eight-GPU DGX B200 systems. But as Abitbol noted, the greatest performance gains didn't come from adding GPUs, but rather tuning the model's hyperparameters along with switching to NVFP4 quantization, implementing disaggregated prefill and decode, and tuning context lengths. Looks like local is getting more popular.+ 1

⬆ upvote 0↩ reply 1

𓀐 @ot85E_Hj: So not only do we have the big cloud providers gobbling up all the hardware we have all these other entities scooping it up as well? Prices are never going down are they?+

⬆ upvote 0↩ reply 0

𓀐 @2Nb94A+*: how to stop gemma from being so horny all the time? i'm getting nothing done+

⬆ upvote 0↩ reply 0

𓀐 @KeWP3ZB7: When will it pop? I need better hardware.+

⬆ upvote 0↩ reply 0

𓀐 @~2uvPozP: Consumer RAM prices can go down. China is making it. They can't make HBM, so there won't be demand from big corps+

⬆ upvote 0↩ reply 0