https://twipped.social/@twipped/114662771295312758
article they are referencing: https://futurism.com/atari-beats-chatgpt-chess
LLM sucks at maths, sucks at chess, sucks at remembering stuff and being consistent … They suck at everything a computer is usually good at.
It’s a very specialized program intended to get a computer to do something that computers are generally very, very bad at - write sensible language about a wide variety of topics. Trying to then get that one specialized program to turn around and do things that computers are good at, and expect to do it well, is very silly.
Yes, LLMs are designed to emulate how a human would respond to a prompt by digesting a huge amount of human-generated content. They can do that fairly well except when they can’t.
It also demonstrates how much AI companies mislead the public on what their products can do. If a guy is selling lawnmowers that actually just generate grass clippings without mowing the lawn, you’re not an idiot for thinking it was going to mow grass.
furthermore. companies mislead journalists, investors, philosphers, influencers etc. most of which dont have a technical background but a lot of reach. They then carry their misunderstanding into the general public.
All these public “academic” panel debates on conferences about AGI being the next nuclear weapon and singularity. They lead to Highbrow publications, opinion peaces, books and blog articles, which then lead to tweets, memes and pop cultural references
Huh… wait… what if we make a box… generate electricity bills… Call it a crypto miner?
Cram a bunch of space heaters into a box. Convince investors that all the electricity it burns up means it’s basically printing money. The building will inevitably burn down before anyone can investigate our claims.
But once someone explains it to you and you insist the grass was mowed, they show you the unmowed grass, and you still insist it’s great for mowing lawns.
And also you’re in the desert where you shouldn’t even have a fucking lawn, and you plant more lawns because they’re so easy to mow now
What do you call that? Because it’s a bit past ‘idiot’.
Asking ChatGPT to play chess is like asking someone who’s not played Chess to play well, and then documenting how poorly it played. Like no shit the hammer did a bad job as a saw. You wanted it cut, you should have used the tool for the job.
ChatGPT isn’t Deep Blue. It’s not made for that. You’re asking a word processor to calculate pi.
Yes but there are many out there that don’t have this sort of understanding and believe the LLMs can do almost anything.
There people are the higher ups at large companies…
Nobody thought it would do very well. This was a software dev’s little diversion.
We should praise attempts to make the public aware of the limitations of LLMs, not laugh at the guy who did this.
Using an LLM to play chess is like using autocorrect to write a novel.
And that’s the big problem with AI right now. People don’t understand what it is, they just want the label slapped on to as many things as possible.
AI is the new IoT, it will be integrated into everything, less than useless for 99.9% of consumers, and yet, still wildly successful.
It’s because the venture capitalists who are sinking BILLION$ into these things are calling it AI even though it’s not and literally never will be. And unfortunately, too many people are too stupid to understand that these aren’t AI but Generative Adversarial Networks or GAN’s for short. Which doesn’t sound as sexy and “take my money please”-ish as Artificial Intelligence or ✨AI✨ does.
These will never be HAL9000 or Jarvis or even Roku’s Basilisk. The stuff needed for that kind of “intelligence” doesn’t exist in these things. And the sooner people come to realize that this is all just digital snake oil the sooner we can collectively get on with our lives.
Too many people are failing to understand that fqughds are actually woplels, and until they understand that, they are just going to keep wasting their money on 💫woplels💫.
Even though woplels have proven to be useful for some things, they’re not as good as some people want them to be, so they’re useless.
Given how much it costs it will need to be ten times more successful than web search to even hop to break even. It’s the biggest dot com bubble yet.
Using an LLM to play chess is like using autocorrect to write a novel.
…What did I just read?
It’s AI generated and absolutely hilarious.
I miss the days when AI was erratic enough to be hilarious. I haven’t clicked onto AIWeirdness since chatgpt came out.
That is glorious. I’ve never wanted to read an entire book more.
The brain dead morons who defend it and accuse me of just being a hater for understanding any part of it are the worst.
I literally no longer believe personhood is a thing because of how stupid and oblivious they’re capable of being.
‘they’ referring to people? Hell no. It’s just hyped onto them whether they like it or not.
Yet another corporate hype wasting massive resources.
No more like auto-fill on your phone.
Hundreds of billions of dollars spent
No profitable product
No consistently usable product other than beginner code tasks
Massive environmental harms
Tens of thousands of (useful!) careers terminated
Destroyed Internet search, arguably the one necessary service on the Internet
No chance it’s going to get better
Atari 2600 beating it at chess is a perfect metaphor. People who want to complain about it can bite its plastic woodgrain printed ass.
Massive environmental harms
I find this questionable; people forget that a locally-hosted LLM is no more taxing than a video game.
No chance it’s going to get better
Why do you believe this? It has continued to get dramatically better over the past 5 years. Look at where GPT2 was in 2019.
No consistently usable product other than beginner code tasks
It is not consistently usable for coding. If you are hoping this slop-producing machine is consistently useful for anything then you are sorely mistaken. These things are most suitable for applications where unreliability is acceptable.
No profitable product […] Tens of thousands of (useful!) careers terminated
Do you not see the obvious contradiction here? If you are sure that this is not going to get better and it’s not profitable, then you have nothing to worry about in the long-term about careers being replaced by AIs.
Destroyed Internet search, arguably the one necessary service on the Internet
Google did this intentionally as part of enshittification.
Massive environmental harms
I find this questionable; people forget that a locally-hosted LLM is no more taxing than a video game.
No chance it’s going to get better
Why do you believe this? It has continued to get dramatically better over the past 5 years. Look at where GPT2 was in 2019.
Fair enough. It’s not going to get better because the fundamental problem is AI as represented by, say, ChatGPT doesn’t know anything. It has no understanding of anything it’s “saying”. Therefore, any results derived from ChatGPT or equivalent, will need to be double-checked in any serious endeavor. So, yes it can poop out a legal brief in two seconds but it still has to be revised, refined, and inevitably fixed when it hallucinates precedent citations and just about anything else. That, the core of it, will never get better. It might get faster. It might “sound” “more human”. But it won’t get better.
No profitable product […] Tens of thousands of (useful!) careers terminated
Do you not see the obvious contradiction here? If you are sure that this is not going to get better and it’s not profitable, then you have nothing to worry about in the long-term about careers being replaced by AIs.
Well tell that to the half a million people laid off in the last couple of years. Damage is done. Also, the bubble is still growing, and if you haven’t noticed what AI has done to the HR industry, let me summarize it thusly: it has destroyed it.
Destroyed Internet search, arguably the one necessary service on the Internet
Google did this intentionally as part of enshittification.
Well, yes. Every company which has chosen to promote and focus on AI has done so intentionally. That doesn’t mean it’s good. If AI wasn’t the all-hype vaporware it is, this wouldn’t have been an option. If OpenAI had been honest about it and said “it’s very interesting and we’re still working on it” instead of “it’s absolutely going to change the world in six months” this wouldn’t be the unusable shitpile it is.
I don’t think we disagree that much.
So read and learn. Okay, I agree that it can have environmental impact due to power usage and water consumption. But this isn’t a fundamental problem – we can use green power (I’ve heard there are plans to build nuclear plants in California for this reason) and build them in a place without water shortages (i.e. somewhere other than California.) AI differs from fossil fuels in this regard, which are fundamentally environmentally damaging.
But still, I cringe when someone implies open-model locally-hosted AIs are environmentally problematic. They have no sense of scale whatsoever.
But it still has to be revised, refined, and inevitably fixed when it hallucinates precedent citations and just about anything else. Well yeah, it’s slop, as I said. These are only suitable in cases where complete reliability is not required. But there’s no reason to believe that hallucinations won’t decrease in frequency over time (as they already have been), or at that the domains in which hallucinations are common won’t shrink over time. I’m not claiming these methods will ever reach 100% reliability, but humans (the thing they are meant to replace) also don’t have reliability. So how many years until the reliability of an LLM exceeds that of a human? Yes I know I’m making humans sound fungible, but to our corporate overlords we mostly are.
if you haven’t noticed what AI has done to the HR industry, let me summarize it thusly: it has destroyed it.
Good, so we agree that there is the potential for long-term damage. In other words, AIs are a long-term threat, not just a short-term one. Maybe the bubble will pop but so did the dotcom bubble and we still have the internet.
enshittification
No, I think enshittification started well before 2022 (ChatGPT). Sure, even before that LLMs were making SEO garbage webpages that google was reporting, so you can blame AI in that regard – but I don’t believe for a second that Google couldn’t have found a way to filter those kinds of results out. The user-negative feature was profitable for them, so they didn’t fix it. If LLMs hadn’t been around, they would have found other ways to make search more user-negative (and they probably did indeed employ such techniques).
No consistently usable product other than beginner code tasks
I mean, it’s pretty good as a productivity tool for programmers as it eliminates a bunch of chore.
Oh my god you ‘people’. Did you not read what you replied to?
‘people’ in scare quotes since coders aren’t people I guess.
it introduces more problems than it solves.
Such as?
hallucinations, bugs, security flaws,
I swear you have never actually used copilot while programming and are just making shit up
i swear you have never actually programmed anything
Why? Because I can use a smarter autocomplete and I am open to change and not a dogmatic closeminded person?
Fine, we’ll stipulate to that. The conclusion is upheld.
I mean, you literally have whole videos on YouTube made by GothamChess who shows how LLMs play chess. They literally spawn pieces from air, play moves that are illegal etc.
Holy hell
New response just dropped
I’m quite sure that the guy understood pretty well what LLMs can do. He just wanted to deinflate all the bullshit promises by Techbros
Man that sucks
Deinflate? Is that like uninflating? Or more like making something inflateless?
Tap for spoiler
It’s just deflate, and yes I feel like a dickhead for pointing it out.
“Deinflate” feels like actively sucking all the air out instead of letting it out passively. Unrelated, I know, but I think words are so neat
That’s a very good move! To counter, you should follow these three principles:
-
Prepare a response move that will prevent a future good move.
-
Defend your own pieces and try to attack theirs.
-
Don’t be too eager to sacrifice pieces in order to make short term gains.
-
Be prepared to sacrifice an unimportant piece to make a good gain.
If you want to make a good move, try Rook H8 -> G7.
Alt text
Me: Knight to C3
ChatGPT: You’ve played Knight to C3 — a classic developing move! That suggests we’re likely in the early phase of the game, possibly playing something like:
- e4 e5 2. Nc3
or perhaps you’re playing the English or Reti and going for flexibility.
Let me know the full board state or moves so far if you’d like to continue the game or get commentary!
Queen to queen’s bishop three, Captain.
-
I bet the llm doesn’t even know what en passant is
Holy hell
It knows the definition though!
It literally doesn’t. It has a pointer to other pointers that often times are marked as correct.
“Knows”
They could probably have done better by training a crow to play chess.
LLMs can’t beat anyone or anything at chess because they can’t play chess at all. Try it. They don’t get more than a few moves in without degrading into total nonsense.
I don’t know if this is real, but AI for chess kinda has to be tailor made for chess, right?
Yes the point is that LLMs don’t reason.
These articles aren’t written for people who know how LLMs work or what they do, anyway.
It’s to prove to everyday people that the techbro marketing is bullshit and these are limited tools, not conscious beings. The populace is being sold a hammer that hallucinates and told everything is a nail.
I fuck around with AI chatbots every now and then. Sometimes I’ll type in one thing to it and it will spit out a respone that doesn’t have anything to do with what I said to it
maybe it’s the way I type and form sentences, but yeah, a lot of ai “generated” stuff won’t be what you typed in
That or AGI. But there’s no AGI yet.
as far as the general public knows at least
By my best feelings, this shit is a bigger bust than the .com bubble, and I predate that latter shit by roughly twenty years.
I’m quite sure that you could use a LLM to play chess and probably even successful, but you need to train it on chess notation of games instead of a pile of fanfiction and other copyright infringements. I have considered trying that but was turned off by how inaccessible LLM training is and how difficult it would be to get a sufficient amount of games written in proper chess notation. Obviously this would not be a real LLM, as it does not “speak”, but I was curious how well this would work utilizing the same technique.
It’s called AlphaZero and is the best chess engine to date
Thank you, I couldn’t think of the name, but I knew there was a machine learning chess bot out there that made cheating at online chess really common.
You shouldn’t train an LLM for that, just any other type of machine learning.
You don’t need text to play chess.
deleted by creator
Make the computer play 100,000,000 games against itself. Human games are useless as training data because humans are useless at chess compared to computers.
