Joined February 2009
32 Photos and videos
Pinned Tweet
12 Jun 2025

1
1
697
Pluralis is making the same bet for AI that Bitcoin made for money; Decentralised, Sovereign and Trustless. And it solves the value-capture problem open source never could.
The 8B model currently training on Agora is 350B tokens in and continuing to converge. The top level metrics and evals look almost exactly like a centralised run. But; - 133 external contributors total bringing 4090's, 5090's, L40S/RTX 6000 and RTX 6000 Pros. These are cards that people actually own - there are no H100, B200's etc. - The max number of nodes the system can support (104) was filled almost immediately. The authorization layer is receiving approximately 100 requests/minute to join. - The total tokens/per second processed moves directly with amount of compute in the swarm, with Agora constantly optimising to make most efficient use of what hardware is present. - MFU is approximately 20%, TPS is 170k tok/s. There are near constant communication failures which Agora is completely absorbing without slowdown. - The system is effectively on auto-pilot, requiring very little intervention from us. Bad nodes are purged immediately before training is affected and new nodes take their place.
1
1
47
Jun 11
Pluralis thesis boils down to one question: Does subspace-compressed model parallelism hold convergence at scale? Can GPUs talking through the drinking straw of ordinary internet train something as smart as GPUs talking through the fire hoses of datacenters? That's the whole bet.
26
Jun 11
Fable 5 is a Ferrari for the Mind!
18
Jun 11
The Bitcoin quantum debate keeps circling one question: Is the threat real? That's not the question that matters. Bitcoin became a trillion-dollar asset because enough people believed it was a digital store of value — nobody proved anything. The risk prices the same way. If enough people believe the threat is real and that the fix will come too slow, it's in the price. The machine doesn't have to exist.
1
1
1
51
Jun 10
Protocol Models are decentrally trained neural nets whose full weights are not extractable by any single actor. Here’s how it works: A model designer commits enough compute capital that show their skin in the game. Rest of the compute providers in the network votes with compute to train the model. If the model design gets enough compute, it gets trained. Each compute provider gets ownership in the trained model proportional to their risk-adjusted compute contributed as measured in verified FLOPS. Risk-adjusted means compute contribution in the beginning of a training run is weighed more than at the end since model performance is not clear at the start. The ownership is represented by a trade-able credential issued upon training a model. That credential gets consumed when an inference query runs the model. The credential is re-issued to the owner once its consumed.
25
Jun 10
Mythos and a longevity milestone launching on the same day. We are in or close to singularity.
1
25
Jun 10
So Protocol Models are decentrally trained neural nets whose full weights are not extractable by any single actor.
10
Jun 10
Well, meta abandoned llama. Skepticism is reality now.
23 Jul 2024
One key argument in the open source vs closed source LLMs debate is this: Without a backer with deep pockets, open source LLMs are always going to trail one or two generations behind their closed source counterparts due to the huge costs in training frontier models. One historical example that backs this argument is IBM continuously funding Linux in the 2000s. Similarly, while Meta is currently funding Llama, there is skepticism about their ability to continue funding it indefinitely as the costs get astronomical. What this argument misses is that crypto is a counter example. BTC and ETH with market caps at $1.3T and $415B respectively are both open-source software projects that bootstrapped from scratch while achieving significant capital formation at scale. Crypto projects themselves can be their funders. So, why can’t open source AI LLM projects with crypto economic incentives beat closed source LLMs? #Bittensor #BasedAI
9
The AI buildout debate is stuck on the wrong issues: circular financing, GPU depreciation math, supply and demand, etc. The two real ideas that, if true, would call into question the return on the marginal training GW are: One: Ilya's idea that we've moved from the age of scaling to the age of research, and that there are diminishing returns on simply adding more compute. Two: Decentralised training works at scale to produce frontier models.
8
Ilya Sutskever says another 100x of compute wouldn't transform AI. The whole AI training buildout assumes the opposite. That's the question worth arguing about. The AI bubble debate is stuck on the financing. The circular deals, the GPU depreciation math, supply-demand. These are real issues, but second-order. The first-order question is whether the marginal training GW still earns its return. Two things have to stay true for it to, and both are shaky. One: more compute keeps buying better frontier models. That was the recipe from 2020 to 2025: add compute, watch the model improve. Ilya's point is the recipe is spent. We're back in an age of research, "just with big computers." You still need the computers. You just stop getting frontier capability in proportion to how many you wire together. The marginal GW buys less than the financial model in the spreadsheet. Two: frontier training has to happen in one place. A run lives inside a single campus because of bandwidth. The chips have to talk to each other faster than an ordinary internet link allows, and that constraint is the moat under the entire capex race. Decentralised training is dissolving that. Pluralis, Nous, IOTA and Prime Intellect already train real models across continents over ordinary internet. They sit roughly 1000x behind the frontier on compute today, but that scale has been growing about 20x a year while the frontier grows 5x. If those rates hold, the gap closes in about five and a half years. And the math cuts against the moat: as a model gets bigger, the compute each machine does grows faster than the data it has to send to the other machines (square-cube law). Bigger models are easier to decentralise, not harder - making decentralised training a practical choice. Here's a fair pushback though: a scaling slowdown doesn't kill compute demand, it moves it to inference and agents, which fill data centres too. And 1000x is still a long way back. Both true. So this is a slow repricing, not an immediate crash. But under the bubble talk, the buildout is one bet: that the bottleneck stays exactly where it sits today. Both of these ideas move it.
97
A bug in Zcash's Orchard pool could have minted unlimited counterfeit ZEC, undetectably, for four years. There's no way to cryptographically prove it never happened. That part is real. And, anyone selling you certainty about the past is lying. Here's the trap, though. The exploitability risk was highest while the bug was in the open and unpatched. It dropped the day the bug was disclosed and patched. Your fear is spiking in the exact week the actual risk fell. That's the possibility effect — Kahneman & Tversky's finding that we overweight a risk once it moves from unimagined to possible. A 1–2% tail risk takes over your whole decision-making rubric. And while you stare at it you miss what the event revealed: a team that hired someone to hunt this with the newest AI the week it shipped, and closed a flaw that hid from the world's best cryptographers for four years. Proactive, AI-assisted security is becoming table stakes for every protocol that survives. Most protocols are not there yet. Zcash is visibly leading the path. The possibility effect is how you let a closed 1–2% tail risk talk you out of the 10–100x.
64
xpmanoj retweeted
Though I am sympathetic to my Zcash supporting friends, I view the selloff in reaction to the Orchard bug as a good thing. Not because I dislike zcash. Because it means that markets are finally pricing fundamentals. In the past when consensus bugs, exploits, chain splits, 51% attacks occurred the market would typically shrug them off. If anything they were perceived as bullish and sometimes the coin would rally in reaction. More eyeballs was the thinking. This is a bad state of affairs and I’m glad markets are starting to price risk properly. My guess is that ZEC was due for a pullback due to the large rally recently and its price was somewhat tenuous anyway, which exaggerated the impact. But either way I think it’s right that markets have started to take notice of security fundamentals. Zcash admirably took risk to create new privacy primitives and they should be applauded for that. Zcash is the originator of deployed ZKP and deserve endless credit. But privacy is also fundamentally more risky in blockchains because it trades off against auditability. Both Monero and Zcash have had historical inflation bugs (patched). These are particularly scary in privacy focused blockchains because inflation is much harder to detect. The problem with markets not punishing exploits is that it means they also don’t value chains with ironclad track records of security. Exploits being punished by the market also implies that security, credibility, stability and auditability should be rewarded. This is a good, healthy feature of the market. Now should chains _not_ take risks? Also no. Bitcoin (and every blockchain) will have to take some controlled risks as we move away from ECC towards a new era of cryptographic agility. I am also grateful that blockchains like Zcash have created new primitives and tested them in the most unforgiving market imaginable. I sympathize with my Zcash friends and I’m sure they will recover just fine from this like they did the Sapling bug in 2018/19. Onchain privacy is a valuable goal and enormously hard to pull off.
58
43
582
67,842
xpmanoj retweeted

89
48
356
79,722
The bear case on neoclouds is that GPUs depreciate to zero in 5 years. CoreWeave just re-contracted 5-year-old A100s at 95% of original price. Wrote about what that means.
52
xpmanoj retweeted
We @trans_celestial are looking for epic sci fi artists who have bold ideas and visions of the future and would like to come work with me on some ideas. If you are new or upcoming and still unrecognized for your potential (or you know of someone, please tag here 🙏🏽 Epic world building and technology imagineering shall commence 🔥
5
2
11
442