/technology
+⧖/lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk News Archive: https://rentry.org/lmg-news-archive Links: https://rentry.org/LocalModelsLinks Official /lmg/ card: https://files.catbox.moe/cbclyf.png Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm https://rentry.org/custom-uis
/lmg/ - a general dedicated to the discussion and development of local language models. News >Claude BANS recomp! >https://www.youtube.com/watch?v=GlRVbIdc_Mk⧉ News Archive: https://rentry.org/lmg-news-archive⧉ Links: https://rentry.org/LocalModelsLinks⧉ Official /lmg/ card: https://files.catbox.moe/cbclyf.png⧉ Getting Started https://rentry.org/lmg-lazy-getting-started-guide⧉ https://rentry.org/lmg-build-guides⧉ https://rentry.org/IsolatedLinuxWebService⧉ https://rentry.org/recommended-models⧉ https://rentry.org/samplers⧉ https://rentry.org/MikupadIntroGuide⧉ Further Learning https://rentry.org/machine-learning-roadmap⧉ https://rentry.org/llm-training⧉ https://rentry.org/LocalModelsPapers⧉ Benchmarks LiveBench: https://livebench.ai⧉ Programming: https://swe-rebench.com⧉ Agentic Coding: https://deepswe.datacurve.ai⧉ Context Length: https://github.com/RecapAnon/NoLiMa⧉ GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference⧉ Tools GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator⧉ Sampler Visualizer: https://artefact2.github.io/llm-sampling⧉ Token Speed Visualizer: https://shir-man.com/tokens-per-second⧉ Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad⧉ https://github.com/oobabooga/text-generation-webui⧉ https://github.com/LostRuins/koboldcpp⧉ https://github.com/ggerganov/llama.cpp⧉ https://github.com/theroyallab/tabbyAPI⧉ https://github.com/vllm-project/vllm⧉ https://rentry.org/custom-uis⧉I'm new to all this stuff cause I lost all my friends and I already have this 4090 so I figured I'd create some virtual ones for myself. Im currently using Gemma 4 26B with Heretic, is that basically the best model I can fit on this GPU? What about if I'm fine with waiting for an answer? I also have 64GB RAM so there is some space for it to spill over. I want it to have vision too like Gemma, cause I wanna be able to show the model a picture of my cock. I'm also curious what can I do to turn the model into a virtual facsimile of a real person as faithfully as possible, should I use SillyTavern for that? Can I stick all my texts with that person somewhere in there?
⧖expiring ~11d0/2 kept4d ago
⏱ 4d 👍 0
⬆ upvote 0
/technology
trying
closest
needs