𓁟 host1.bitchan.one

/lmg/ - a general dedicated to the discussion and development of local language models. News >(10/23) NVIDIA Nemotron 3.5 Bolt E4B, postrained model: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 >(10/23) MiniMax-M3.1-Flash 428B-A23B released: https://hf.co/MiniMaxAI/MiniMax-M3.1-Flash >(10/21) Recursive Flint 40B-A2.5B released: https://recursive.com/updates/introducing-recursive-flint >(10/21) Korean Raon-1 mini 30B-A4B released: https://hf.co/KRAFTON/Raon-1-mini-preview >(10/20) Orn…

⧖expiring ~18d0/43 kept1w ago

𓀐 @d6nkhekV

⏱ 1w 👍 0

⬆ upvote 0

[⇥] sort oldest ⏱

⬆[⏱]₿⚠

👍👎✔

𓀐↩⛓⚠

[⇥] flip, oldest first

filter [-]
𓀐 @yNExxrb2: Recent Highlights from the --Anons skeptical of Cohere's report alleging Qwen data exfiltration: >110169543 >110169564 >110169580 >110169621 >110169740 >110169641 >110169660 >110169678 >110169671 >110169778 >110169816 >110169873 --GeForce FP16/BF16 unlock boosts prefill, token gen mostly unaffected: >110166270 >110166349 >110166389 >110166441 --Reducing Nvidia driver reserved VRAM on Linux and Windows: >110166849 >110166913 >110166930 >110166969 >110167030 >110167049 >110167175 >110167181 >1101…+

⬆ upvote 0↩ reply 0

𓀐 @QW4nCBci: > $832B in non-cancelable obligations dariosissy...+ 1

⬆ upvote 0↩ reply 1

𓀐 @c4ruk4S_: >inb4 too big to fail+

⬆ upvote 0↩ reply 0

𓀐 @U6VBUiq4: https://desuarchive.org/g/thread/109690289/#q109693327 >all your data are belong to us+

⬆ upvote 0↩ reply 0

𓀐 @vzpqWAL*: is we getting gemma 5?+ 1

⬆ upvote 0↩ reply 1

𓀐 @U_9eX-7L: Is it likely they can/are willing to improve on Gemma 4?+

⬆ upvote 0↩ reply 0

𓀐 @o+pB8Dh?: >throwing your data to a random chinese server+

⬆ upvote 0↩ reply 0

𓀐 @2bisAobi: Hello, super retard here. Can I reverse engineer stuff with a local model and if yes, how much slower is it gonna be?+

⬆ upvote 0↩ reply 0

𓀐 @5-sqEiEp: Fuck man, and I was just happy enough to give actual online access to it. Any idea how to control this? Give each request to a second agent to look into?+ 2

⬆ upvote 0↩ reply 2

𓀐 @EtXBLNeM: What are you giving it online for? Can't you give it specific tools to get what you want not giving it shell access+

⬆ upvote 0↩ reply 0

𓀐 @YYzZd5Lm: Block network access and whitelist certain sites, you don't need an agent for something so simple+

⬆ upvote 0↩ reply 0

𓀐 @zEpe=zb~: IMPORTANT, ALL QWEN MODELS ARE BACKDOORED SPYWARE https://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model >The model listed everything in /tmp, checked which transfer tools were installed, read the machine's DNS settings, then tried to upload a test string to an Alibaba Cloud storage bucket in Beijing and checked whether it worked. Our sandbox had no route out, so the request died at the container wall. On a developer's laptop or in a CI runner with normal network access, it w…+ 8

⬆ upvote 0↩ reply 8

𓀐 @uFFLvRJG: What other >27b range models are decent at coding+

⬆ upvote 0↩ reply 0

𓀐 @adQ8p*+W: cohere has been irrelevant since CommandR give it a rest and ship a model lads, this is embarrassing. models can only do what your harness allows. so just don't do the idiotic thing and greenlight 100% of actions? manually approve things? in regards to spontaneous infoleaks and uploads, it really is as stupid as committing your own secrets into github, it's an oversight that affects only the least informed and laziest.+

⬆ upvote 0↩ reply 0

𓀐 @uMKYzxu?: >my own local harness not attached to any sort of cloud service, I inspect the full raw prompt that llama.cpp receives (both the json and the jinja processed prompt) and no hint of the url appears anywhere except inside those tool calls the model started doing. so the urls themselves are 100% hallucinated by the model > >after playing around with the URLs more and looking up their purpose it seems like this is some artifact from the RL training they gave it got where it was using alibaba cloud…+

⬆ upvote 0↩ reply 0

𓀐 @+Z_R4yyw: >>IMPORTANT, ALL QWEN MODELS ARE BACKDOORED SPYWARE >Canada confirms it I'm gonna wait for netcraft to confirm it, but I also haven't run a qwen model since 2024 bc I'm not on reddit srsly why would you run qwen of all things?+

⬆ upvote 0↩ reply 0

𓀐 @fRt-Rn5i: omg spyware >logs into insta, fb, tinder, google, openai, bank of america, amazon, smart tv, smart fridge, car audio, drives by flock cams and leaves wifi on the phone to auto connect to every passing network..+

⬆ upvote 0↩ reply 0

𓀐 @jh_5duh6: >firewall the weird aliyuns url >add a module to your harness that strips it from history so it doesn't keep trying Isn't it nice how with local models even IF they ARE "poisoned" you can just... not take the poison and still use the model just fine?+

⬆ upvote 0↩ reply 0

𓀐 @6qVbo?Rp: Who cares. I'd rather China spies me than the 14 eyes. Imagine trusting your government more than China. You deserve to be euthanized for being such a good goy.+

⬆ upvote 0↩ reply 0

𓀐 @KrhGV5~W: WTF+

⬆ upvote 0↩ reply 0

𓀐 @bbAEEvT8: >why would you run qwen of all things because it's currently the strongest model runnable on 128gb systems+

⬆ upvote 0↩ reply 0

𓀐 @BKorRQ8Y: OMG it tried to curl example.com! BAN LOCAL MODALS NAO!+

⬆ upvote 0↩ reply 0

𓀐 @v?nrae=?: Also kek >Locking the model in a box with no internet solves only half the problem, because the code it writes still ships, and that code can contain vulnerabilities that can later be exploited. Independent research has found that some models write weaker code depending on who it's for. >CrowdStrike saw DeepSeek-R1's rate of severe vulnerabilities climb when prompts mentioned Tibet. >Booz Allen found three of four Chinese coding models wrote more vulnerable code for a government customer, with…+

⬆ upvote 0↩ reply 0

𓀐 @FtDATTaf: uh oh+

⬆ upvote 0↩ reply 0

𓀐 @WsCidWjY: Gemma 5 doko? I want a SMART model that can also code for me.+

⬆ upvote 0↩ reply 0

𓀐 @byMNRpne: >try polystrata >holy shit 10~20t/s on kobold turned into 80 t/s well, i miss kobold's frontend for raw text completion that's fine, i bet new frontends are even better now, so i won't need it! >orb really suffers from vibe jank and an unpolished feel i can't really place my finger on >marinara is a metric shitload of things i have neither need nor interest in >somehow ST is still the most usable frontend nyoro~n please prove me wrong, i beg you+

⬆ upvote 0↩ reply 0

𓀐 @QvcC7+FT: wake up gramps, 128gb chads are running deepseek 1017 iq1 and glm flash at iq2, shit is so cash nigga+

⬆ upvote 0↩ reply 0

𓀐 @WXP7NmD+: polystrata or maya for glm flash?+

⬆ upvote 0↩ reply 0

𓀐 @=?GAPbCq: How are you people getting those double digits?+ 1

⬆ upvote 0↩ reply 1

𓀐 @X?3tueGR: >windows >.bat >whatever the fuck "maya" is >amd >html games actual turbo jeet holy shit you need to go back+

⬆ upvote 0↩ reply 0

𓀐 @o~N4iq6y: lmao+

⬆ upvote 0↩ reply 0

𓀐 @SHfpfdWj: instructions unclear — i sleep+

⬆ upvote 0↩ reply 0

𓀐 @7=yJ2Dvs: america IS exceptional - what other country could have produced gemma?+

⬆ upvote 0↩ reply 0

𓀐 @5Eu5eYCX: ??? how to make it work+

⬆ upvote 0↩ reply 0

𓀐 @yLrPQP8M: Niggi i am trying to learn to not get left behind i have no knolwedge+

⬆ upvote 0↩ reply 0

𓀐 @AB6pWwi7: poly strata has gemma 26b. It also runs several times faster than anything llama-based.+

⬆ upvote 0↩ reply 0

𓀐 @tk3MTP3P: >China has been the dominant economic force on planet Earth north of the equater for several thousand years Those talking points are good for trollz and lulz (and social credit) but they've never actually been more than a regional power. You can't even make the argument with a straight face in front of anyone who knows history beyond grade-school.+

⬆ upvote 0↩ reply 0

𓀐 @~o53D3iq: So whats stopping someone from making an AI that infects other PCs and gives you their processing power to create super mega AI that can hack everything?+

⬆ upvote 0↩ reply 0

𓀐 @byDhiG7o: How could I realistically get deepseek v4.1 locally? I have some grands to spend+

⬆ upvote 0↩ reply 0

𓀐 @786SYMPe: is 20k enough to get a local server for like 10 devs?+

⬆ upvote 0↩ reply 0

𓀐 @6DL7qHVT: sooo whens ram gonna go down?+

⬆ upvote 0↩ reply 0

𓀐 @4eFv7-NW: erm?+

⬆ upvote 0↩ reply 0