🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.ee to

LocalLLaMA@sh.itjust.worksEnglish · 24 days ago

How much gpu do i need to run a 90b model

10

How much gpu do i need to run a 90b model

🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.ee to

LocalLLaMA@sh.itjust.worksEnglish · 24 days ago

Do i need industry grade gpu’s or can i scrape by getring decent tps with a consumer level gpu.

Chat

Sylovik@lemmy.world
link
fedilink
English
arrow-up
2·
23 days ago
You may try Harbor. The description claims to provide an OpenAI-compatible API.

LocalLLaMA@sh.itjust.works

localllama@sh.itjust.works

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !localllama@sh.itjust.works

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

24 users / day
117 users / week
235 users / month
393 users / 6 months
5 local subscribers
2.5K subscribers
210 Posts
879 Comments
Modlog