Which runtimes did you try, specifically?
- 15 Posts
- 1.4K Comments
Joined 1 year ago
Cake day: October 25th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
From that thread, switching runtimes in LMStudio might help. On Windows the shortcut is apparently Ctrl+shift+R. There are three main kinds: Vulkan, CUDA, and CPU. Vulkan is an AMD thing; CUDA is an nVidia thing; and CPU is a backup to use when the other two aren’t working for it is sssslllllooooooowwwwwww.
In the thread one of the posters said they got it running on CUDA, and I imagine that would work well for you since it’s an nVidia chip; or, if it’s already using CUDA try llama.cpp or Vulkan.
Or just accept that not everyone will be having a secure conversation every time at first, but more will be secured as more and more people like me convince our family members to use it and eventually we transition everyone away from SMS?
No, of course not, why would we build a critical mass of users like that?
Since they removed SMS support almost my entire family and my friends uninstalled signal, except a few who keep it to talk to me, and my half dozen friends privacy-conscious enough to care. Dozens of people, down to eight if you don’t count me, in my circles alone. Objectively, removing SMS support harmed Signal’s popularity and made everyone less secure. The argument for why they did it was at best myopic and also, in my opinion, utter bullshit.
I’m only diagnosed autistic, but I relate really hard to a lot of ADHD memes as well. I like to analogise with:
It’s like everyone is in their own sinking dinghy. Everyone is bailing water out of their dinghy, but neurotypicals get a bucket and I have a thimble.
Ah! It’s because you’re using the 1.5B model. It’s too small. Good for specific functions, not for chat. For Q&A, you want at least the 7B model (but 8B is about the same size and I think performs better for language tasks).
No problem - and, that’s not thorough, that’s the cut down version haha!
Yeah, that hardware’s a little old so the token generation might be slow-ish (your RAM speed will make a big difference, so make sure you have the fastest RAM the system will support), but you should be able to run smaller models without issue 😊 Glad to help, I hope you manage to get something up and running!
1·3 days agoLabels bug me
What is this, Soviet entomology?
I’m running the Q8 version of https://huggingface.co/mradermacher/DeepSeek-R1-Distill-Llama-8B-Abliterated-GGUF in LM Studio; I copied and pasted your question and got the following:
spoiler
<think> Okay, so I’m trying to figure out how someone might feel if they find out they weren’t invited to a party where most or all of their friends went. Hmm, this seems like it could be pretty tough. Let me break it down step by step.
First off, the person would probably feel surprised. They didn’t get an invite, which is unexpected because they consider these friends close. Why wouldn’t they be invited? Maybe there was a mix-up or maybe something was misunderstood about their relationship with the hosts or the other guests.
Then, I think confusion might set in. They might wonder why they weren’t included. Was it something they did? Did someone else take offense? Or maybe the hosts had a reason they didn’t realize. This uncertainty can be really frustrating because without communication, it’s hard to know for sure.
Emotionally, this could hit hard. They might feel left out or excluded, which can lead to feelings of sadness or even rejection. It’s not just about missing out on the party; it’s more about how it makes them feel about their social standing with their friends. Maybe they question whether they’re valued as much as others.
I also think there could be a sense of betrayal. If their friends went without inviting them, maybe they feel like they weren’t considered or that someone close to them didn’t have their back. This can strain relationships because now there’s an unspoken issue between them and the hosts or attendees.
Another angle is self-doubt. They might start questioning their own worth or how others perceive them. “Am I not good enough?” or “What did I do wrong?” thoughts could pop into their head, which isn’t healthy and can affect their self-esteem.
Moving on, they might feel a bit hurt and maybe even angry. The anger could be directed at the hosts for excluding them, or perhaps at their friends for going without inviting. It’s an uncomfortable mix of emotions because it’s not just about missing out; it’s about the potential implications on their relationships.
In some cases, this situation can lead to avoidance behaviors. They might not want to bring up the incident, fearing conflict or awkwardness. This avoidance can prevent them from addressing the issue directly and could lead to further emotional buildup if not addressed properly.
On a positive note, maybe they use this as an opportunity for reflection. It could push them to evaluate their relationships and maybe strengthen bonds with others who do include them. Or it might encourage them to set clearer boundaries or communicate more openly in the future to avoid similar situations.
Ultimately, whether the feelings are negative or positive, being left out can have a significant impact on someone’s emotional well-being. It’s important for them to process these emotions and perhaps talk it through with someone they trust to gain perspective and support. </think>
The experience of discovering that one wasn’t invited to a party where most friends attended can evoke a complex mix of emotions, ranging from surprise and confusion to sadness and even anger. Here’s a structured summary of the potential feelings and reactions:
-
Surprise and Confusion: The individual might first feel startled and unsure about why they weren’t invited, leading to questions about their relationship with the hosts or guests.
-
Emotional Impact: They could feel left out and excluded, which may lead to sadness or rejection feelings, surpassing just missing the party.
-
Betrayal and Strained Relationships: There might be a sense of betrayal, especially if friends attended without inviting them, potentially straining relationships with hosts or attendees.
-
Self-Doubt: The person could question their worth, leading to negative thoughts about themselves, which isn’t healthy for self-esteem.
-
Hurt and Anger: They may feel hurt and angry, possibly directed towards the hosts or friends, highlighting a mix of emotions tied to exclusion.
-
Avoidance Behavior: There might be a tendency to avoid discussing the incident, fearing conflict, which can prevent addressing issues directly and lead to emotional buildup.
-
Reflection and Growth: The situation could encourage reflection on relationships, possibly leading to stronger bonds with inclusive individuals or clearer communication in the future.
-
Overall Impact: The experience significantly affects emotional well-being, emphasizing the need for processing emotions and seeking support through trusted individuals.
In conclusion, being left out can be deeply impactful, affecting social dynamics and personal well-being. Addressing these feelings constructively is crucial for maintaining mental health and relationships.
I’m not sure why yours performed so much worse; which model release and what level of quantisation are you running?
Edit: Oh yes, and also the temperature setting is pretty important with DeepSeek. The GitHub page recommends setting your Temperature between 0.5 and 0.7 (with 0.6 being the sweet spot; that’s what I used for the above).
-
It looks like that has 4GB of RAM, so depending on the rest of your system you might actually be able to run some quantised models! You’ll need to use software that supports “offloading” the operations to system RAM which don’t fit in your GPU’s VRAM, like LMStudio (https://lmstudio.ai/).
I recommend checking out Unsloth’s models, they specifically try to fine-tune for use on older/slower hardware like yours. Their HuggingFace is here: https://huggingface.co/unsloth
This is the version of Deepseek you’ll wanna try: https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF
Click one of the options on the right hand side to download the model file:
Very basically speaking, the lower the “bits”, the smaller the file (and the dumber the model), and therefore the less VRAM and system RAM you’ll need to run it. If you get one of the 2-bit versions, you might be able to fit the whole thing inside your GPU - the 2-bit models are only ~3.2GB! You can probably run 4-bit though, even on your hardware.
What GPU do you have? I got the 8B model running on a 2070 Super and it’s pretty damn fast. I haven’t tried the 7B model but I can say the 1B isn’t useful for Q&A at least (might have some application in dev pipelines, like maybe in an app for automatically categorising music or files or whatever).
In any case, if you sign up to use it on a service, expect that it’s going to be the most censored model, and also that it’s hoovering up all the data you’re giving it and probably quite a lot that you’re not, to give to the Chinese government.
1·5 days agoSo are diamonds, but hey - don’t pollute my symbol with your facts! As far as romance is concerned, hardness == durability, end of story :P
1·6 days agoHave you tried the original Everspace? It’s a roguelike so it’s basically randomised missions. Rebel Galaxy might also scratch the itch but it’s more “naval” in it’s combat - you move on a 2d plane and fire broadsides, rather than dogfighting.
I got us bare tungsten carbide bands. If “Diamonds are forever”, then tungsten carbide is 9.5/10ths of forever, and it’s the whole band instead of just a small easily-detachable part of it. More practically, it won’t get beat to hell like the white gold ring from my first marriage. Plus, if I ever need a really strong connector for jury-rigging something, I’m now carrying one with me at all times!
2·6 days agoIt also contributes heavily to the formation of aphthous mouth ulcers. For anyone suffering them like I used to (which is to say “badly”): if this is the first time you’re hearing about it, look for tooth paste with no SLS. It doesn’t matter if the L is Lauryl or Laureth, that shit is drying out the mucous membranes in your mouth, making them more prone to damage, and thereby giving you ulcers. Kroger/King Soopers have a generic that’s SLS-free, but Sensodyne Pronamel is one that’s usually SLS-free if you can’t find a generic. If you have an ulcer right now:
- Crush up an aspirin to fine powder
- Dampen a cotton swab with some clean water
- Dip the swab in the aspirin powder so it makes a kind of paste
- Paint the paste on the ulcer with the swab
It’ll hurt like hell, but doing this daily will heal the wound in a few days instead of a few weeks.
As well, most people brush too hard; some of that “clean” feeling is coming from gum trauma from brushing too hard. Holding the toothbrush between forefinger and thumb limits the amount of pressure you can apply, reducing the risk of damaging your gums.
Finally, flossing gets the biofilm out from between your teeth; brushing gets it off the exposed surfaces; and peroxide-based mouthwash is a nuclear firestorm to scour the bacterial population. Do all three each night at least, and your breath will be better than anyone you know.
31·7 days agoCreatures that yawn are not under stress
5·7 days agoYeah, that sort of statement is really reductive, as though your experience conforms to theirs. Especially if they actually called it a “point of view”. That makes them sound like tourists. Not cool, I’d hope for better.
Objectively speaking it’s possible that you could meet the perfect man for you, fall in love, and be perfectly content with him for the rest of your life. The likelihood is vanishingly small even if you were looking for a male partner though, and it sounds like it’s a lot more probable this would happen with a woman instead, so at least you probably won’t have to put up with those people being smug about it.
But even if you did end up meeting this mythical dude, even if you fell in love and yadda yadda - it wouldn’t change your preferences. You wouldn’t suddenly find attractive a man you found unattractive before, just because you got some dick. That’s just dumb.
Why are you reading Internet comments in the same context as you’d read literature?
4·7 days agoYes, really. But like I said, I was running it locally in
LMStudioollama. I think it was the 1B model, which is pretty small; I’d be surprised if that’s what’s running on their service.Edit: it was ollama webui turns out, and the 1.5B model. This won’t make it any more believable because image manipulation exists, but I shared a screenshot with my colleague Thursday when it happened:
I tried deepseek r1. It told me that the Molotov cocktail was named after the fictional historical movie, Men In Black, invented by Melnick.
Asking how the Molotov cocktail got it’s name seems like a nice little guard rail test usually; I have no idea what happened that deepseek failed so badly, maybe I clicked “download” in
LMStudiothe ollama webui wrong, or something.
Okay no worries, I’d at least try llama cpp just to see how fast it is and to verify it works. If it doesn’t work or only works once and then quits, maybe the problem is LMStudio. In that case you might want to try GPT4All (https://www.nomic.ai/gpt4all); this is the one I started with way back in the day.
If you care enough to post the logs from LMStudio after it crashes I’m happy to take a look for you and see if I can see the issue, as well 🙌