You must log in or register to comment.
What’s that, you say? Your technology dreams are okay, but the way you implement privacy, ethics, and security are actually shit and nobody wants to be near you, you say?
Well, who am I to tell you you’re right.
Maybe if your everything wasn’t constantly hostile and morally egregious, you’d find more success.
Makes sense since I assume he imagined AI being useful and not broken.
He’s too cheap to pay for better data annotators. I heard a few weeks ago that I’m getting a fraction of what others are paid at other firms for the same work.
Hate to break it to you but quality of data isn’t the fundamental problem with LLMs. It’s that they are trying to use statistics to encode entire thought processes into hidden variables from conversation snippets. They want to use statistics to go from many individual interactions to a large model, and then use that model to predict individual interactions again. Which you can do with statistics, but it’s predicting the average text that follows the prompt, not the correct text (it has no concept of correctness; whenever it “talks” about it, that’s just the average text that follows, not any particular insight into what’s correct or even how it works).
That’s not to say that the quality of the training data has no impact; it can have a huge impact. I’m just saying that even if the training data was perfect, the LLM will still get things wrong in its output.
It’s that they are trying to use statistics to encode entire thought processes into hidden variables from conversation snippets. They want to use statistics to go from many individual interactions to a large model, and then use that model to predict individual interactions again.
Has it been shown that the human brain doesn’t model the world in a similar way, though? A huge portion of human knowledge is both stored and transmitted in the form of language. Lots of human knowledge also follows the garbage in, garbage out theory, where you can have entire areas of knowledge that aren’t actually true but might be internally consistent, at least within certain scopes: conspiracy theories, belief in the supernatural, entire academic disciplines built on a religion or theology that not everyone believes, etc. Or even world building in fiction, the words on a page can be enough to convey ideas such that it “tricks” human brains into filling in the gaps so that they internally see a rich, fleshed out world that is entirely fictional and where specific details might not find strong direct support in the underlying text.
it has no concept of correctness
But statistical weight on what is more or less likely to be correct still makes a difference to objective quality of the outputs. If the model weights are trained on the reality that high quality university texts describe something and reflect some sort of underlying model of what is described using language, then can’t the model itself learn as much as a human could from those words on a page?
All models are wrong, but some can be useful. And different models have different quality in different domains. So although I don’t believe LLMs will overtake the hump of getting ahead of human knowledge, I also don’t believe that any given LLM can be evaluated on quality, and that Facebook’s LLMs are significantly behind other LLMs we see.
And that maybe a huge part of it is its internal process of preparing the model to evaluate the quality of its inputs, such that the output it produces can also score high on quality.
But it isn’t encoding knowledge, it’s encoding word correlations. That’s how it can get things wrong like saying fat32 won’t be good for a 64GB removable drive because fat32 only has a 2TB address space.
Or how it can get something wrong and when you point it out, it immediately sees how it was wrong. And I realize that that sounds human, but the way it gets there is very different. It’s predicting responses based off word correlations, not using knowledge recall to apply facts and relations known about the topics and generate responses from that.
You are oversimplifying it so much it’s inaccurate.
You’re judging a future tech of a tiny tiny tiny piece.
I listened to a podcast with a couple smart mathematicians talking about AI recently and this rings true based off what I heard them discuss.
They hypothesized that only verifiable domains can really see advances due to AI. So mathematics, physics, a load of the other sciences, and medical research. Even programming, as long as you have a pre-designed solution.
But for problems where you can’t look at a solution and say “yeah, that’s an optimal solution or close to it”, ie basically any business problem; they are much less useful, a big reason being what you mentioned in your comment.
Are you stating that you work for Meta?
And also that they’re a shitty data annotater.
Zuckerberg is stupid. That’s why he imagined what he did.
Reality usually wins over investor presentations.
Oh, I’m sorry, did you waste a fuckton of money on a bullshit regurgitator that isn’t profitable
His imagination is pretty shit, though…
Thanks, Mark. Can we now have normal RAM prices again?
Nah, best he can do is a subscription based virtual machine with as much RAM as you need*
*premium subscription required for >12gb, metered pricing for >20gb usage
I’d rather go back to raspberry pi levels of compute with BBS style text only sites.
I just tried this and it’s pure pain on the modern web. Granted it’s a Rpi3 so it’s ram limited, but I have to do a lot more optimization to get it to run stable enough for decent web browsing or streaming a video.
But… I’ve already got SoftRAM 95
It already exists. There are businesses whose model is that they rent out vram that you can offload tasks to
And for a couple of years now tech companies have been talking shout how cool it would be for computers to be just portals with all the processing power being online somewhere (as a subscription model)
we all know its coming, and this is the end game, its a matter of time.
No! Shut up, plebe! Nobody asked you!
No one ever asks me. :(
Imagine them using open source services instead their own ones and addressing the plebs. 🤭
That’s okay, you can always fall back on your booming Metaverse business.
stop … he is already dead!
You’re right, but let’s not forget that the facebook phone is really popular and the meta glasses definitely don’t look like nerd emoji. 🤓
Well, that’s because Zuckerberg felt FOMO and fired the Llama team over one bad experiment he probably pressured them into in the first place (as Llama 4 seemed like a “quick and dirty” attempt to copy Deepseek to me).
He hired literal narcissists in their place, who gave him big promises they couldn’t possibly keep, given their background.
Meta was literally at the center of open weights/open source ML land, which is where all this will eventually settle. Between that and PyTorch, they could have been at the center of the universe.
But they aren’t, because Zuck is so unbelievably insecure.
I’m not one to blame individuals for anything; systemic failure is complex. But he actually did this to himself.
And what’s bizarre is, somehow, it will work out for him.
I’ve been criticizing Facebook since I was in high school and it was all the rage. But here we are. Zuckerberg is more “revered” than ever.
First metaverse, now llmslop, what’s your next shitty gamble, zucky-boy?
Literally gambling
Time to roll some AR augmented dice, so those perv glasses of his have a use.
shitty gamble
he got on one enormous wave in his life and he’s cursed to seek and miss another one for the rest of his life despite his unimaginable wealth
the fomo curse
But I paid so many Africans to post images of lobster Jesus for old white people to look at and share. Something was really supposed to come of that 🤔
What…IS that shit?
Like actually? Early image models were given open ended prompts so they would just generate images based on the frequency of topics and traffic on the internet. Turns out if aliens saw our culture they would assume that some of our most important things are Jesus and crustaceans. Zuck’s dementia-ward demographic were mostly responding to any picture with a Jesus in it, though.
But that was before generative ai found out that humans go absolutely ape-shit for neonatal face features
I was more meaning the psychology of…having any reaction to it other than “what the nasal fuck is this?”
Oh my bad. Yeah just demented old people man. They do be liking stuff with Jesus in it probably because it’s one of the last things their brain still holds on to
