/technology
+⧖/lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis
/lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk⧉ News Archive: https://rentry.org/lmg-news-archive⧉ Links: https://rentry.org/LocalModelsLinks⧉ Official /lmg/ card: https://files.catbox.moe/cbclyf.png⧉ Getting Started https://rentry.org/lmg-lazy-getting-started-guide⧉ https://rentry.org/lmg-build-guides⧉ https://rentry.org/IsolatedLinuxWebService⧉ https://rentry.org/recommended-models⧉ https://rentry.org/samplers⧉ https://rentry.org/MikupadIntroGuide⧉ Further Learning https://rentry.org/machine-learning-roadmap⧉ https://rentry.org/llm-training⧉ https://rentry.org/LocalModelsPapers⧉ Benchmarks LiveBench: https://livebench.ai⧉ Programming: https://swe-rebench.com⧉ Agentic Coding: https://deepswe.datacurve.ai⧉ Context Length: https://github.com/RecapAnon/NoLiMa⧉ GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference⧉ Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator⧉ Sampler Visualizer: https://artefact2.github.io/llm-sampling⧉ Token Speed Visualizer: https://shir-man.com/tokens-per-second⧉ Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad⧉ https://github.com/oobabooga/text-generation-webui⧉ https://github.com/LostRuins/koboldcpp⧉ https://github.com/ggerganov/llama.cpp⧉ https://github.com/theroyallab/tabbyAPI⧉ https://github.com/vllm-project/vllm⧉ https://rentry.org/custom-uis⧉So little local Qwens are good-enough coders. They competently execute even surprisingly intricate tasks, but the code itself is what I can only describe as "cringe", and they very clearly have that Claudeslop stench in their prose. To boot, the models themselves have, as the recommended models rentry succinctly puts, "an impressive lack of world knowledge". Putting 2 & 2 together, I'm guessing that means Anthropic are correct in accusing Alibaba et al of just distilling their shit. Bad world knowledge outside narrow benchmark focus smells of nonexistent long-tail, which tracks with a distillation. Synthetic input makes these things very deterministic-feeling, laser-focused on the center of the bell curve, as it were. On the face of it, whatever, based, do what you gotta do Zhang, intellectual property no real, I agree, but the consequences on usability are kinda rough. Using, for instance, 3.8 Flash Next as a virtual collaborator is like pulling teeth. Sure you can set it to work solo and it'll comfortably vibeslop together something that fits the bill, but the result is this quagmire of superficially fine code that somehow feels obfuscated when you actually try to understand it, and technically-true-but-worse-than-meaningless comments. And if you're babysitting it commit-by-commit to try to steer the changes to be tolerable for human reading, you're expending a similar amount of effort to just writing the code yourself, without the latter's benefit of gaining a personal understanding of the problem domain. Thing is, I see symptoms of the above in public repos yolo vibeslopped together even with frontier Claudes and whatnot. Such end products are essentially dead ends as concerns a larger ecosystem, unless you're willing to throw your hands up in the air, say fuck it, and let clankers take the wheel on *using* these libraries too. ...So is there a local model that achieves good-enough code by some means *other* than being a good-enough facsimile of Claude?
⧖expiring ~14d0/1 kept20h ago
⏱ 1d 👍 0
⬆ upvote 0
/technology
trying
closest
needs