Extropic.ai vs Moore
Extropic.ai
My brother, my father and AI have this chat where we talk tech, startups and sometimes ideas. Recently a company popped up in the chat and really caught my attention.
Extropic.AI
Apart from a name that looks like what a tribute band of Anthropic.AI would look like, Extropic references a type of process where -opposite to entropy- with time, tends to organise itself. No chaos. Is this a message that the company plans organise the computational chaos behind AI or a statement much more profound. Let's dive!
"Ushering in the thermodynamic future..."
Extropic.ai is a technological company aimed at providing a HW platform for AI computing.
What? How do they plan to compete against NVIDIA?
Well, it might be surprising but these people do not offer GPUs.
Then how do you train an LLM if not with thousands of GPUs?
With springs!! Do i have your attention?
Extropic.ai proposes a novel approach to a very recent problem. Semiconductors and the Moore's Law end. In 1965, Gordon Moore forecasted a set of rules that hold up to date. Hi named law. Moore's law states the following:
The number of transistors on a microchip grows each year at minimum cost, and following an exponential growth.
Now, 59 year later Moore's law meets it's match in Quantum physics. Currently the smallest transistors are 2 nanometers (nm) wide, developed by IBM. This allows IBM to fit 50 billion transistors onto a chip the size of a fingernail, making these the smallest transistors ever made for commercial purposes. And some theorise we could go even further. But when you get to atomic levels *weird shit* happens. Mainly two things:
From a thermodynamics POV, temperature manifests as particle vibrations. These vibrations can affect then the MOS gates in the microprocessor and affect results.
From a quantum physics POV, uncertainty plays a huge role, specially when an electorn can "jump" a logic gate and produce results that are not representative of the state of the system.
These ofc are grossly oversimplified, but it is good to keep in mind before diving into the offer that *Extropic.ai* presents, that all these phenomenons can be modelled via a probabilistic distribution. In vernacular, there is a bigger chance that the chips are in one state but they can also be in different states with less chance.
Raising to the challenge!
And how does AI come into the picture?
AI (or the current-very-advanced generative models) need huge computation capabilities. There are two ways to improve computing and those are vertically and horizontally. Vertically means investing in smaller and smaller transistors (with the limitations explained) and horizontally, by stacking this microprocessors (aka. distributed computing).
Each has it's limitations, but Extro is looking to push the boundaries vertically.
Long story short, they plan to create a different chip configuration where the result instead of being digital is probabilistic. And this probability, based on equations tested numerous times in both fields. To sum up, thermodynamics behave like springs in the sense that even though a spring may oscillate, there is always a point it oscillates around.
Aiming thus to create spring-powered logical gates, instead of electrical powered.
That is creative.
Indeed.
Could be a complete failure though?
Could be. But the important thing to focus here is not if it will work or not. Let me tell you something...
Ad astra?
This example in particular is one of many that are pushing the boundaries of computing. I really like the notion of harnessing other fields of physics to compute. Even biology can come into the picture. Novel discoveries in information storage using DNA strands look straight out of sci-fi. Organic computers where computing units are neurons (or the like) instead of gates are also interesting.
The important thing here is that the laws of physics are true an unbendable, at least until it is disproven. But the space in between? That is pure creativity. The intangible. The space where human ingenuity meets with creativity.
This law does allows me not? Then i shall change the playing field.
There is always a way.