codeinabox@programming.dev to AI - Artificial intelligence@programming.devEnglish · 27 days agoExperimenting with local LLMs on macOSblog.6nok.orgexternal-linkmessage-square4linkfedilinkarrow-up13arrow-down11
arrow-up12arrow-down1external-linkExperimenting with local LLMs on macOSblog.6nok.orgcodeinabox@programming.dev to AI - Artificial intelligence@programming.devEnglish · 27 days agomessage-square4linkfedilink
minus-squareonlinepersona@programming.devlinkfedilinkEnglisharrow-up2arrow-down1·26 days agoWat I don’t understand is how it’s supported on Mac. Do they have NVIDIA cards? I thought everything was dominated by CUDA. Did the lock-in company implement CUDA?
minus-squarecodeinabox@programming.devOPlinkfedilinkEnglisharrow-up2·23 days agoOllama uses the Metal API on Apple Silicon Macs for GPU acceleration.
minus-squareonlinepersona@programming.devlinkfedilinkEnglisharrow-up1·23 days agoInteresting. Is that similar to Vulkan?
minus-squarecodeinabox@programming.devOPlinkfedilinkEnglisharrow-up3·22 days ago Vulkan? I’m not too familiar with either, but this article goes into more detail: A Comparison of Modern Graphics APIs
Wat I don’t understand is how it’s supported on Mac. Do they have NVIDIA cards? I thought everything was dominated by CUDA. Did the lock-in company implement CUDA?
Ollama uses the Metal API on Apple Silicon Macs for GPU acceleration.
Interesting. Is that similar to Vulkan?
I’m not too familiar with either, but this article goes into more detail: A Comparison of Modern Graphics APIs