• 21 Posts
  • 1.39K Comments
Joined 11 months ago
cake
Cake day: March 22nd, 2024

help-circle







  • I mean, if you have huge GPU, sure. Or at least 12GB free vram or a big Mac.

    Local LLMs for coding is kinda a niche because most people don’t have a 3090 or 7900 lying around, and you really need 12GB+ free VRAM for the models to start being “smart” and even worth using over free LLM APIs, much less cheap paid ones.

    But if you do have the hardware and the time to set a server up, the Deepseek R1 models or the FuseAI merges are great for “slow” answers where the model thinks things out for replying. Qwen 2.5 32B coder is great for quick answers on 24GB VRAM. Arcee 14B is great for 12GB VRAM.

    Sometimes running a small model on a “fast” less vram efficient backend is better for stuff like cursor code completion.


  • I’m not going to mock and belittle people who were losing their loved ones as a result of Democrat policies choosing not to partake in the election.

    I am.

    Many seemed to think that Trump and Biden/Harris had roughly “equivalent” Gaza policy, as evidenced by their surprise at Trump’s actions. That is dangerous misinformation, and it deserves to be called out as a mistake, no matter how tragic the consequences are.

    I view our election system as a rigged game, and not participating in swing states as a protest is all but choosing a side, in my eyes. Especially when the consequences are so personally important. The opportunity cost is Democrat votes, and Trump’s gain. In that spirit, I actually registered as a Republican in my home state, only so I could vote against Trumpist politicians wherever I can, as there’s basically no hope for Democrats and my vote would be “thrown away,” relatively speaking. And I can still vote against Republicans in the general election and some other offices that have a shot.








  • brucethemoose@lemmy.worldtoMemes@lemmy.ml2 in a single week that is crazy
    link
    fedilink
    arrow-up
    52
    arrow-down
    1
    ·
    edit-2
    2 days ago

    My friend, the Chinese have been releasing amazing models all last year, it just didn’t make headlines.

    Tencent’s Hunyuan Video is incredible. Alibabas Qwen is still a go to local model. I’ve used InternLM pretty regularly… Heck, Yi 32B was awesome in 2023, as the first decent long context local model.

    …The Janus models are actually kind of meh, unless you’re captioning images, and FLUX/Hunyuan Video is still king in diffusion world.



  • Depends what you mean by “AI”

    Generative models as you know them are pretty much all transformers, and there are already many hacks to let them ingest images, video, sound/music, and even other formats. I believe there are some dedicated 3D models out there, as well as some experiments with “byte-level” LLMs that can theoretically take any data format.

    But there are fundamental limitations, like the long context you’d need for 3D model ingestion being inefficient. The entities that can afford to train the best models are “conservative” and tend to shy away from testing exotic implementations, presumably because they might fail.

    Some seemingly “solvable” problems like repetition issues you encounter with programming have not had potential solutions adopted either, and the fix they use (literally randomizing the output) makes them fundamentally unreliable. LLMs are great assistants, but you can never fully trust them as is.

    What I’m getting at is that everything you said is theoretically possible, but the entities with the purse strings are relatively conservative and tend to pursue profitable pure text performance instead. So I bet they will remain as “interns” and “assistants” until there’s a more fundamental architecture shift, maybe something that learns and error corrects during usage instead of being so static.


    And as stupid as this sounds, another problem is packaging. There are some incredible models that take media or even 3D as input, for instance… but they are all janky, half functional python repos researchers threw up before moving on. There isn’t much integration and user-friendliness in AI land.