I (almost) fully agree with all of his takes.
The only one I am unsure about is, how broad of a scope an AI can eventually think about in one go, but that’s another point entirely.
Important video.
I agree with both, this guy and Yudkowsky.
The first half of the video is great. Completely agree, stop the harm now. Stop the CEOs, stop the naive techno-optimists.
The second half has a lot of strawmen. Let me give you an example:
He takes the 2027 reports Solar Powered robot machines maintenance crew (which is very futuristic and far fetched) and in the next statement taktes the “a blade of grass is a self replicating factory” and acts like Yudkowsky says the grass maintains the machines.
Yudkowsky’s argument is that if AI wanted to kill us it would’nt need the robots. Biology has perfected self replication for a long time. Make some humans do your CRISPR gene tech for you (thanks gig economy). Engineering a stronger moscito with a neuro toxin is not something I understand as impossible. Our biology is very capable of folding all kinds of proteins if given the correct instructions. You don’t need complex robot arms to (accidentally) kill humans. Our society is extremely fragile in thousands of ways.
If you manage to kill specific bugs we die, if you accelerate algee growth a lot we die. If you get enough carcoinogens into the food chain we die. If you engineer super resistant bacteria we die. If you rapidly deplete the ozon layer we die, if you disable the earth’s magnetic field we die.
The point is: If you build something that is smarter than you (in any way) and that has some form of agency, it will outsmart you and beat you. Don’t build it.
Don’t build it.
This won’t work, because humans have a perverse, morbid curiosity for all sorts of things. I’d wager if it can be built, it will be built.
Yudkowsky’s whole argument is that it seems like it is nearly impossible to do it right the first time and that there is a good possibility we don’t get two tries. Not because it is malevolent, but because it is really hard to put into numbers what we care about.
Dude! I figured out a way to build the torment nexus at a third of the cost!
The whole underground biotech lab story is eerily quiet in the last years. I wonder if nobody dares to touch it, it’s still too expensive or we just didn’t have the “right” crazy lunatic with the “right” knowledge to massively feed the shit-fan.
I think it is still too hard. The hurdles, safety and difficulty prevent humans adequately for the moment.
I just wonder, who made/makes more human sacrifices: the Aztecs of the 1400s to their gods or we today to the gods of cars and technical progress?
Edit: Acording to the internet, ~16 700 people are killed by cars per year in modern Mexico (which puts it among the safer countries carwise). That amounts to ~0.036/100 000 people/day. If the Azteks did do daily human sacrifices to ensure the sunrise, they were likely a bit more faithful than us.
I like this video, and I like this community.
