Lemdro.id
  • Communities
  • Create Post
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
codeinabox@programming.dev to AI - Artificial intelligence@programming.devEnglish · 27 days ago

Experimenting with local LLMs on macOS

blog.6nok.org

external-link
message-square
4
link
fedilink
2
external-link

Experimenting with local LLMs on macOS

blog.6nok.org

codeinabox@programming.dev to AI - Artificial intelligence@programming.devEnglish · 27 days ago
message-square
4
link
fedilink
A developer's guide to downloading and running LLMs on macOS, for experimentation and privacy.
alert-triangle
You must log in or register to comment.
  • onlinepersona@programming.dev
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    1
    ·
    26 days ago

    Wat I don’t understand is how it’s supported on Mac. Do they have NVIDIA cards? I thought everything was dominated by CUDA. Did the lock-in company implement CUDA?

    • codeinabox@programming.devOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      23 days ago

      Ollama uses the Metal API on Apple Silicon Macs for GPU acceleration.

      • onlinepersona@programming.dev
        link
        fedilink
        English
        arrow-up
        1
        ·
        23 days ago

        Interesting. Is that similar to Vulkan?

        • codeinabox@programming.devOP
          link
          fedilink
          English
          arrow-up
          3
          ·
          22 days ago

          Vulkan?

          I’m not too familiar with either, but this article goes into more detail: A Comparison of Modern Graphics APIs

AI - Artificial intelligence@programming.dev

Aii@programming.dev

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !Aii@programming.dev

AI related news and articles.

Rules:

  • No Videos.
  • No self promotion: Don’t post links to your articles.
Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 5 users / day
  • 45 users / week
  • 195 users / month
  • 1.13K users / 6 months
  • 2 local subscribers
  • 162 subscribers
  • 83 Posts
  • 54 Comments
  • Modlog
  • mods:
  • Vacant@programming.dev
  • UI: 0.19.11
  • BE: 0.19.12
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code
  • join-lemmy.org