Run a 35B parameter AI model on just 6GB VRAM using llama.cpp and Qwen 3.6.This setup shouldn’t work—but with the right optimizations, it reaches good enough...

I finally sat down and set this up last night. The Qwen 3.6 35b is very impressive and following this vid will get you running at workable speeds. I haven’t bumped it up to the 256k context window yet, but even with the 128 I’m able to have it work on projects I’d rather not be putting up on the cloud.