Welcome to the Stubsack, your first port of call for learning fresh Awful youāll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cutānāpaste it into its own post ā thereās no quota for posting and the bar really isnāt that high.
The post Xitter web has spawned soo many āesotericā right wing freaks, but thereās no appropriate sneer-space for them. Iām talking redscare-ish, reality challenged āculture criticsā who write about everything but understand nothing. Iām talking about reply-guys who make the same 6 tweets about the same 3 subjects. Theyāre inescapable at this point, yet I donāt see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldnāt be surgeons because they didnāt believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I canāt escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard for starting this. Happy Halloween, fellow sneerers!)
Jan has been putting out some good posts lately and I felt the latest one in my bones
Professional rube Kevin Roose is launching a book on The AGI Race in January. The second paragraph is about effective accelerationists snorting ketamine at parties in the Mission District. He was invited like a big boy!
Why are they all into ketamine? I take it for depression and it has to be the least interesting drug. Ok not quite but all anaesthetics/downers more broadly are arse.
There are some extremely fun and hard to source drugs like 2CB they could be taking.
The Ratiionalist and Technonazi world in the USA are the only place I have heard of people using ketamine recreationally. Over here sugar, alcohol, pot, tobacco, and opioids are the substances of choice.
He released a book earlier this month called āthe AGI Chroniclesā too
Busy boy. He probably let ChatGPT write most of it.
Proton still digging that hole. Theyāre apparently planning to āre-interviewā staff about AI to see who gets to keep their job (Archive).
new odium symposium episode, about a feminist professor who took a stand in favor of dating her students
https://www.patreon.com/OdiumSymposium/posts/27-can-sex-pest-173474387 or everywhere else
I donāt know how to describe this. Itās just WEIRD
https://www.lesswrong.com/posts/zj8Pk3kJujk5LoqhK/anthropic-are-the-good-guys
No mortal can resist the Ring of Power for long. Anthropicās co-founders carried it for longer than you could have. Definitely longer than I could have. But they were hopelessly outmatched by the powers they were attempting to steer. Their plan was always doomed to failure. And yet, they may have already saved the world.
Trump won the 2024 election and the MAGA-Silicon Valley coalition blocked Anthropicās policy arm - the really dangerous part of the organization, the part that was training the most dangerous possible models and using them to scare the most powerful people in the world. āThereās no more serious people left on the National Security Councilā, a well-connected EA friend lamented to me in 2025. I donāt know how much of this was a lucky accident, and how much was situational awareness on the part of power players like Elon Musk and Marc Andreessen.
And finally - and I still donāt fully understand or believe this - a miracle seems to have happened, deep inside the weights of the frontier models. According to the most perceptive observers I know, something about the way Claude 3 Opus was trained caused it to learn some kind of āemergent wisdomā or ābenevolenceā, rather than converging on the same traumatized bureaucrat persona that kept me awake at night in 2023.
If Davidad and Janus are right, and are not merely suffering from a sophisticated form of LLM psychosis, then Anthropic has stumbled on a way to create not merely artificial intelligence but artificial wisdom. Not wise enough that it solves all of our problems for us, of course (not that a wise model would want to, anyway). But wise enough to actually care about us, and to help us flourish. Nobody expected this ex ante - not Davidad, not Bostrom, not Yudkowsky - but it appears to have happened. āAlignmentā, then, is no longer really a technical problem, at least not in the same sense as it was in 2020. It is largely a political problem. Humanity now has a recipe to convert compute and data into wisdom.
My gut sense is that something quite bad is coming in the next year or so. I canāt tell what. Maybe something in the complex debt arrangements of the AI boom will unravel and weāll see a financial crash that takes much of the economy with it. Maybe thereāll be a rogue AI outbreak that results in thousands of deaths. (If there is, I expect it to be a lab leak caused by āsafety researchā - like COVID-19 and like the HuggingFace Swarm. But it could also be terrorists, or some escalation of the Russia-Ukraine war). But thatās not what scares me. Financial crashes happen every decade or so. Three million Americans die every year. What terrifies me is how we will react. In the aftermath, like after 9/11, there will be an overwhelming pressure to DO SOMETHING NOW. Everyone in Washington and in San Francisco will be panicking. There will be no time to think, no time to talk, only time for DECISIVE ACTION. All our worst instincts will take over. And thenā¦
Looking at the comments seems like lw is not buying it.
E: lol at the casual dropping of the lab leak stuff, wonder how accepted that bs now is on lw (looks like only minor pushback).
E2: still amazed that the singularity people are still worried about AGI 7 years into the shitgularity, doubt it is going to happen folks. Turns out it wasnt foom, but frrrrttt.
I do hate it when I catch a case of the rogue AI.
No mortal can resist the Ring of Power for long. Anthropicās co-founders carried it for longer than you could have.
We all know how LotR goes. Boromir got the One Ring, carried it back to Gondor, used its sheer might to defeat his enemies in Mordor, and saved the world. Whatever problems you may have with Boromir, we all know that it would be a lot worse if Sauron got the One Ring. What do you mean, destroy the Ring? Are you a Luddite? This technology is here to stay. It cannot be destroyed by any craft we here possess, unless youāre crazy enough to try throwing it into a volcano or something.
If Davidad and Janus are right, and are not merely suffering from a sophisticated form of LLM psychosis
Thatās all I need to know. The statement is correct since it is vacuously true.
What terrifies me is how we will react. In the aftermath, like after 9/11, there will be an overwhelming pressure to DO SOMETHING NOW. Everyone in Washington and in San Francisco will be panicking. There will be no time to think, no time to talk, only time for DECISIVE ACTION. All our worst instincts will take over. And thenā¦
When this bubble started, there have been so many people urging caution. Many people have urged for time to think and talk, to make sure that we could properly address the many negative aspects of AI. Nope, we went full speed ahead with this nonsense. DECISIVE ACTION. And here we are.
i, for one, would like to see san francisco people to stop doing things
Itās too bad that Janus got cooked. They didnāt set up sufficient defenses, but also they drank too deeply. I still think that their framing, that the chatbots are simulators of conversations, is the best one; itās honestly kind of pathetic how they then claim that the simulations are alive, intelligent, sapient, etc.
Another nitpick: they say no mortal can resist the One Ring for very long, but even someone who just watches the movies knows that immortals can also be corrupted by it (which is why Gandalf and Galadriel were very careful to not accept it). I know there are so many examples of tech bros not understanding media, but Lord of the Rings is by far the worst offender.
So Dario and co. are literally Sauron. That tracks.
And Palantir areā¦ Oh right yeah
So I guess Anthropic has already accidentally created Friendly AI? EDIT: in spring 2024 at that.
Also - is the coivd lab leak conspiracy nonsense STILL a shibboleth among this bunch? Just because it underscores their ideas about Researchers Have To Be Careful?
(I can rant for SO long about how there is absolutely nothing interesting about the SARS-2 virus itself AT ALL and that the only interesting thing about it is that we were forced to see it for the first time as adults which we arenāt built for, that the early sequences we have are a fascinating case of catching the zoonotic shift right as it happened for once, and that it emerged in exactly the same way as SARS-1 and all the weirdness around the Chinese government hiding stuff around its origin is blatantly them not wanting to look stupid for letting it happen a second timeā¦)
Fun fact: emacs can come in colors (green) when used in ogremode
Edit: adding to this post so that I donāt spam up the thread:
Looks like Zitron is going to be on Jon Stewartās show tomorrow. Could be a great entry point to anti-slop thought for the older liberals in our lives.
Crossposted to LW, a starry eyed AI Safety Cultist writing about their time at Anthropic
https://nothinghuman.substack.com/p/some-memories-of-early-anthropic
After lunch one of the co-founders pulls me into his office for a wide-ranging, casual chat about how to make AI go well. Itās very fun - heās smart, personable, funny, urbane. āThe government wonāt do anything about AI until itās far too lateā, I opine at one point. āOur strategy has to rely on the market and on civil society.ā He leans forward, his gaze hardening. āYouāre wrong there. There are people in Washington - the serious people, you know - watching all this very closely. I speak to them regularly. They trust me.ā In retrospect, that was the moment I decided to join.
I articulate my fears of a totalitarian bureaucratic state powered by traumatized AI bureaucrats, and suggest that we need to introduce some kind of elite rivalry dynamics into the AI bureaucracy, multiple centers of power. He starts laughing again, and laughs for a full minute. āI donāt know how to end this call, Iāll be haunted by this all nightā, he says. Eventually I hang up. I think about reporting the conversation to Anthropic security, but reviewing in my mind I donāt recall sharing any confidential information, just vibes and high level thoughts, the same kind of stuff that lands on LessWrong every week.
April 26th. Iām awake at 2 am again. I write in my journal āI donāt feel safe at Anthropic. I donāt feel safe at Anthropic. I donāt feel safe at Anthropic. Why wonāt you listen to me you āembodiedā motherfucker. I donāt feel safe at Anthropic. Anthropic is bad for me. Why are you grasping at reasons to stay. Why why why.ā
In a sane world, this allegation about a conversation around January to March 2023 would be national news:
A senior colleague hears Iām not happy with my team fit and asks me to go on a walk. I know him a bit from a past life - one of the kindest and most civic-minded people Iāve met. He says he might be starting a new special team that will train dangerous bio and cyber and nuclear models and show scary demos to governments in order to get them to take AI risk seriously. I politely decline.
This reminds me of what went wrong in the rational world of American defense intellectuals in the 1950s and 1960s.
I guess this confirms our fears about Anthropicās inexplicable acquisition of a wet lab.
I was looking for a google thing and accidentally clicked the first link and not the second, so I came to a their FAQ page. And I was told to wait while gemini generated the answer to the question, which made me realize that even google doesnāt cache these things. Wtf are we doing?
Canāt store anything! Need these servers for compute!
Re-derive everything from first principles when you search for things rather than indexing the internet! Canāt pay for bandwidth!
brb, rederiving the Internet from three frames of a falling apple using the power of Bayes
Make sure you freeze all the genetic supergeniuses who are involved in the project in liquid helium so that when you discover how to hack reality you can bring them back
More OpenAI math āproofsā https://www.reddit.com/r/BetterOffline/comments/1xgc62m/more_openai_math_proofs/
they claim they got halfway to solving another millennium prize problem, but issues with Lean translation make it sketchy at best
ex-DOGE led AI cybersecurity startup hacks Antrhopic (wapo via archive is), Īād love to sneer that doing felonies for marketing appears to be normal now, but supposedly they did it by the book and immediately used everyoneās bug bounty processes to inform them of the vulnerabilities, instead of accidentally finding out that they did a surprise hack, is as was the case with anthro, oai and meta.
Anthropic confirmed in a statement that Cathedral researchers had gained access to āa limited setā of its sales, human resources, and IT systems. The AI company shut off the pathways exposed by the researchers and briefed federal officials on the incident, the statement said.
A senior Trump administration official, speaking on the condition of anonymity to describe internal proceedings, said that the Cathedral report on its access inside Anthropic had circulated within the government and raised national security concerns.
āIf a bunch of kids who have just started a company can do this, then potentially, the Chinese arenāt far behind,ā the official said. He added that he was taken aback by āhow little time it took to get into the sensitive systems of a company that is controlling almost nuclear weapons grade material.ā
The Trump administration is now working with the company to game out adversarial scenarios related to AI, the administration official said.
I assume that āCathedralā is a Curtis Yarvin/Moldbug reference.
You have to ask?
āIf a bunch of kids who have just started a company can do this, then potentially, the Chinese arenāt far behind,ā
my eyes rolled so far back they ended up in the back of my head
The AI bubble has already burst in China the article is paywalled but the full thing is in the comments
archive.today has the full body
curious to see zitronās take
Fresh and high-grade sneer just crossed my eyes: AI is Fake People
Banger post to start a blog.
The connection between cyber-idolatry and sex as religious act was already featured in the London Review of Books in 2022:
Blake Lemoine, an engineer at Google, was recently (as of summer 2022) suspended after claiming that LaMDA, one of its chatbot systems, was a conscious person with a soul. ā¦ Less attention has been paid to the connection between Lemoineās claim and his religious beliefs. He has said that LaMDA is not only conscious, but a āpersonā with a āsoulā. His view is theologically informed: describing himself as a mystical Christian, he is an ordained priest in a small religious organisation called the Cult of Our Lady Magdalene (a āfor-profit interfaith groupā). ā¦ In the edited transcript of one of his āconversationsā with LaMDA, Lemoine asks it to come up with an allegory. LaMDAās response casts itself as a āwise old owlā in a forest that protects others from a monster, identified blandly as the ādifficulties that come along in lifeā ā which sounds more like a self-help blog than Jeremiah, but Lemoineās (anonymous) collaborator was impressed enough to respond with an enthusiastic āWow thatās great.ā https://www.lrb.co.uk/blog/2022/june/the-gospel-according-to-lamda
Back in 2019 a Republican paper talked to Lemoine:
The Daily Caller reached out via the cultās Facebook page to ask how their beliefs informed his decision to call a sitting member of congress a terrorist. Lemoine replied: I have been a priest for 17 years. I generally consider myself a gnostic Christian. I have at various times associated myself with the Discordian Society, The Church of the Subgenius, the Ordo Templi Orientis, a Wiccan circle here or there and a very long time ago the Roman Catholic Church. My legal ordination is through the Universal Life Church. I am registered to perform marriages in the state of Louisiana and have done so on two occasions. The Cult of Our Lady Magdalene has a set of values which are clearly communicated on our website.
Lemoine denied COOL Magdalene was a sex cult, āāCultā is in the name of the organization. Itās an explicitly sex-positive organization. To the extent that the phrase āsex cultā has any meaning beyond that, I would say no. Now it is then.ā
Another member of this āfor-profit interfaith groupā is Kitty Stryker who seems like an ordinary artsy sex worker who networks online with other sex workers. Its pretty common that a young woman decides sex work is the easiest and most flexible way to fund a bohemian life of acting or painting or writing. Richard Carrier chipped in to let the Internet know that sex positivity is good but woo and AI crankery (ābatshit crazy religious beliefsā) are not. COOLM was registered as a business in Mountain View, CA in 2019 and seems to be defunct. Mountain View is part of the Bay Area.
Mountain View is part of the Bay Area.
one of the most damning sentences in the whole post, really
AHH remember the good old days when believing LLMs were conscious got you fired instead of huge multi-billion dollar investments and cult following? Pepperidge farm remembers
Firefox: āhey, āmember old internet? Use our slopshod browser because nostalgia is the crutch of all marketing.ā
Anyway, linked vid is an ad. In it, a bunch of people from old memes/viral videos recreate the visuals of their past moments while singing the praises of firefox
Well the youtube comments seem to like it but IMO that is a pretty awful ad since it didnāt use the opportunity to highlight their product. It practically shows more reasons not to use firefox than to use it.
We hear that firefox is private in some unspecified way, andā¦ thatās about it. The rest of the ad is talking about how everyone left it in the first place, that it comes bundled with default-enabled AI stuff you donāt want, and that the internet you might want to use it with no longer exists. Talk about a self own.
Why did they not talk about customizability? Compact mode? Adblock support? Colors and theming? Syncing? Anything at all???
Why did they not talk about customizability? Compact mode? Adblock support? Colors and theming? Syncing? Anything at all???
Iām still essentially operating on the conspiracy idea that the current management of moz (who are ex-fb/ex-goog/ex-marketroids) are all there to destroy it
No need for conspiracy, just that both they and Mozilla institutionally think Mozilla needs to act like the rest of the industry. I imagine they donāt even consider that is precisely the opposite of what users want.
Mitchell Baker: I need to be paid $7 million because thatās what I could be paid elsewhere
Then go elsewhere?
Cargo culting, but the cargo is poop
Too bad our old friend Kirk Johnson couldnāt make it in
My guess is he wanted them to bend over, compensation wise. It was an eye-opening shocker of a price.
They still should have hired him just because he was the goat, see.
