I got some weird specialised hardware over USB working via WinBoat. Might be an option for some.

Unfortunately, the AI community prefers rushed buggy development over proper, tested releases, so the quants and maybe the PR weren’t fully working.

As of 3 hours ago, unsloth was still updating their quants and guide. I don’t have time to test now but I wouldn’t judge the base model performance in the first few days when the bugs are still being worked out.

They also recommend some unconventional parameters in the Unsloth guide.

It could also be that the model is truly shit of course.

Edit I just took a look at the llama.cpp repo and there are still issues with the implementation as well.

Seems to be a new architecture so custom support is needed.

Tracking issue

PR

What features are still missing after this gets merged? Right now, I still use a glitchy adapter that randomly drops out every few minutes and sometimes crashes my whole Hyprland WM due to an unsteady state when no displays are recognised/connected.

I would love to use my GPU and display with the features I paid for.

I can only speak for the Heroic launcher but this has been my experience:

• One click play button like Steam, but it’s easier to configure Linux-specific settings in the menu.
• Updates are done automatically or can be forced like Steam.
• Cloud saves are available, but need to be done manually from the game settings menu with the risk of user error.
• There is a friends list and chat, but the games I played had their own in-game friends list.
• Unfortunately, no achievements from what I’ve seen.
• No DRM, so you truly own the game. (as far as it’s possible to own something digital.)
• Linux game support seems to not be any worse than Steam
• Built-in Wine/Proton version installer.

Interesting study for some kind of reversed Poe’s Law