It’s crazy how much smaller people were back then.
Yeah. It’s because of improvements in agriculture. It made food cheaper, and people got taller.
Yes, I heard our size and population doubled every two years. It’s called Morsel’s law iirc.
Seems to have plateaud out now though.
Also don’t forget the effects of thermal expansion brought on by global warming.
Not an electrical or computer engineer so I can’t really speak on the limits of miniaturizing hardware in terms of physics or technological ability. But even if you can fit a full computer on a fingernail, it’s gonna be hard to have a even just a USB-C connector on a finger nail. At a certain point, there’s little reason to miniaturize computers further when they still have to interface with human usable devices. Instead of miniaturizing the board further, continuing to increase transistor density on the cpu and chips to get more compute power in the same area seems like the obvious focus for future miniaturization efforts.
I think this is also a part of the OP picture. I bet the raspberry Pi is vastly more powerful than that Elliot computer cabinet.
Yeah, not even close.
That’s an Elliott 405 computer. It could perform up to 500,000 instructions per second, aka FLoating-point Operations Per Second (FLOPS).
The other is a Raspberry Pi Zero which can perform 250,000,000 FLOPS, 500 times the Elliott 405.
And, the Elliott 405 cost between 140k-350k in 1957, depending on the features and configuration chosen. With inflation to 2015 dollars, that’s $1.2-2.9 million ($2.40 per 1 FLOPS)
The Raspberry Pi Zero was their new low-power, low-cost board in 2015. It only cost $5 in 2015 (50 million FLOPS per $1)
And for an extra 30 bucks ($35 total) in 2015, you could have picked up a Raspberry Pi 2 Model B, which is capable of 24,000,000,000 FLOPS. That is 96 times faster than the Pi Zero, and 48,000 times faster than the Elliott 405. (~686 million FLOPS per $1)
We’re at the point where you can fit an entire computer inside a USB cable end without it looking any different (beware of keyloggers if you’re not using your own cables, they can even fit a wifi antenna in there)
You really doubled down on that pun. There should be a law about that.
Careful, you might get that Murphy guys attention with this law talk.
Good God has it been 11 years since that meme?
Gangnam style is 14 years old
Ok that’s uncalled for sir.
Sorry, I’m 14
AI was 59 in 2015
It’s been 84 years…
Now you can wear a full desktop PC on your wrist with 0.7nm chip
Well, it’ll be a pretty small chip if the entire thing has to fit on your wrist, so still not equivalent to a 0.7nm desktop chip. They don’t generally make the chips smaller, the feature size gets smaller so they can fit more transistors in the same die area
Positive example of a shrinkflation.
The modern version of this is AI datacenters.
In an amount of years all the crap they are taking up tons of racks for will fit in a cell phone and cost next to nothing.
We might need some new tech instead of shrinking existing tech.
Me: I need some new hardware to run my transformers on
Mom: We have transformers at home
The transformers at home:
Joking aside, I think you’re right, using discrete floating point math to simulate a transformer architecture will never be able to approach the efficiency of a “native” analog system like actual brains. I think eventually we’ll see someone come up with a hardware transformer that doesn’t require full synchronized clock signals.
Jokes aside, I think a brain and a computer solve two different problem sets. A computer needs to be exact, deterministic. A brain needs to be practical, very much at the expense of precision.
I wouldn’t want a brain to do my math calculations for me… brains suck at those, that’s why I use a computer.
If it didnt introduce a spectrum of new issues, I’d wager we could start encoding data into more complex structures than binary. Like using continuous values instead of just on/off. But, from what I understand, that makes the problem of computing damn near impossible and unaffordable.
Maybe we can find ways to encode more data into the same volume of transistors. Like, if current state could imply additional information. But that would probably impede on performance, as I’m sure the data structures used in a CPU prioritize performance not “adding more implied data.”
Hmm… tough one.
Using trinary wouldn’t really be a problem, they also already did it over 50 years ago, we just never used it because their tech wasn’t precise enough
But i am just a junior in the sector so i know little, if someone can explaim to me why it wouldn’t work well, i’d appreciate
They’re talking about AI datacenters though… The whole point is they’re not exact and deterministic, but usually about right.
Oh, I guess I read transistor. Not transformer. Thanks!
Yeah, a brain does exactly what it’s supposed to do. I can’t imagine how a simulation could be more efficient, especially when that CPU wasn’t designed to do the same thing. If the brain had any inefficiencies, I would imagine that it would be a trade off in reality… we’re going with hundreds of millions of years of evolution. It turns cooked meat into power for cognition machines—gonna be hard to beat that efficiency.
That’s still talking about transistors, though. Even more directly, transformers are just translating text into high dimensional arrays where the semantic structure is captured (relative to all other possible embedding values). It’s an interesting approach to navigating semantic information, but there’s not any guarantee that our brain does the same thing. Either way, I’d bet our brain is doing its job without much accidental complexity, whereas modern transformers will always have the added complexity of encoding / decoding semantic information using hardware not designed for it.
Computer should maybe try staying in its lane. Leave the cognitive dissonance and delusional confidence to the experts.
Yeah, a brain does exactly what it’s supposed to do. I can’t imagine how simulating something with alternative hardware could be any more efficient than the original, especially when that new hardware wasn’t designed to do the same thing.
That’s what Neuralink is for! I can see it now:
The jobs of the future will involve 12 hour shifts serving as a biological GPU. A large helmet is strapped to your head. It remotely, non-invasively takes over your neurons and forces them to perform calculations commanded by a remote server, rather than your own thoughts. It doesn’t override the vital parts of the brain involved in say, keeping your heart beating. But it does override most everything else. And it’s not like a dream. You’re not experiencing a virtual reality in there. For 12 hours, all of your senses are just full static at maximum volume. The experience would quickly drive you mad, if your own conscious awareness was not also dissolved into so much computational grist. You’re essentially a vegetable for 12 hours. Your shift is performed wearing an adult diaper. This is nearly the only form of wage labor available. Not even jobs like janitorial are available. One human mind, suitably harnessed, can remotely pilot a dozen humanoid androids. Half of your coworkers are there involuntarily, unpaid, caught up in one of anti-vagrancy law or another.
Yeah, I said transformer because that seems to be the state of the art in AI architectures, but purpose built neural network hardware might not actually benefit from the same architecture.
A neural network made from analog hardware could theoretically replace a significant portion of an LLM’s processing and not be limited by things like floating point precision or clock speeds. Who needs floating point when you can literally just multiply voltages together with a couple transistor junctions at the speed of light and read the output?
There is no way any breakthrough will be made with new technologies whilst the current AI speculation continues over RAM and GPUs, there seems to be still way to much money to be made in this bubble so most of capital resources in this area are being directed tô those “AI” initiatives.
I dunno, necessity breeds innovation. Once current hardware can no longer scale with the hardware requirements, or we completely exhaust the supply of raw materials, something will come along. Crypto mining shifted to ASICs when the GPUs became no longer profitable. I saw not long ago that some crypto mining “businesses” switched to AI so it’s not entirely outside the realm of realiry.
Whether that “something” is a breakthrough in compute, (re)manufacturing, or renewable energy is anybody’s guess. At least IMO as someone who’s been watching on the sidelines for a long time.
Yes, that’s neuromorphic computing. Once we properly figure our how to make semiconductor neurons and how to connect them to each other in a big scale, AI will hopefully be less of an energy waste. The current approach with mapping these problems into GPUs is highly inefficient.
https://en.wikipedia.org/wiki/Elliott_803
Another unusual feature is the use of magnetic cores not only for memory but also as logic gates. These logic cores have 1, 2 or 3 input windings, a trigger (read) and an output winding. Depending on their polarity, current pulses in the input windings either magnetise the core or cancel each other out. The magnetised state of the core indicates the result of a boolean logic function.
Huh. Clever.
A picture similar to this one was on one of my high school text books. Inside the cover was a description of it as magnetic core computer memory. For quite a long time I thought this is what computer chips looked like. The only issue was I was in high school in the 80s, long after such memory was used. Maybe the text book was 15 years old, I don’t know.
First computer company I worked for was still using it in the early 80s. Slow, but it retained state after a power failure.
Cost about £500,000 in today’s money. If the AI bubble hasn’t popped by this time next year, that Raspberry Pi might cost about the same.
Check this out. Very cool.
Though not a fan of his reasoning to have it in silicone oil. The computers back then also didn’t do that, and they had rougher measuring tooling.
He just wanted a oil-submerged thingy anyway.Yeah, absolutely. Just a fun and cool project.
I knew this was gonna be polymatt! What a great vid.
TL;DW: He makes a memory core from “scratch”
You can’t hump a PCB after everyone else has gone home though.
I absolutely can and did. It vibrated and I had a really nice time.
if that’s truly from 1957, the whole setup would have several pieces that size. the 803 a few years later was three (one about this size, two a little smaller), plus user console, printer, tape reader. nearly 2000 lbs worth of equipment.
To be fair the Pi has no peripheral (keyboard, screen…) either.
I don’t know but my supposition is that the ras pi pictured is several powers of magnitude more powerful than the 803.
It certainly is.
From the wiki:
“It uses ferrite magnetic-core memory in 4096 or 8192 words of 40 bits, comprising 39 bits of data with parity.”
So a whooping 39kB of memory on the largest option!
“Tape is read at 500 characters per second and punched at 100 cps.”
Compare that with a micro-SD…
“The bit time is 6 microseconds, jumps execute in 288 microseconds and simple arithmetic instructions in 576 microseconds.”
And it run and an incredible speed of 1 to 3kHz!
(And this is overselling the computer, it was slower than what the numbers appear.)
imagine a tape punch going at 100cps though… even if it’s just stamping baudot code that’s still 500 actuations per second. that’s fuckin terrifying.
probably loud af too
oh absolutely, dot matrix printer times 10
Not a thing you want your service without a good LOTO scheme.
Actually insane that it is so powerful at that time, without transistors.
Some jokes never stop being accurate.
That’s the same size box my new video card just came in.
and all that was inside was a smelly shoe with a can of soup stuffed in it. thanks amazon.
You have the ESP ones, hard to go much much lower without it being impractical (but there are loads of smaller too).
You could just do the soc, it would probably be closer to feature parity.
