Episode 236: Tassadar On Monday we will begin a decentralized training run for a new class of 'executor' model we're naming Tassadar. Because our training software (Pylon) is optimized for consumer edge compute -- desktop, laptop and soon mobile -- we expect the Tassadar run to quickly become the largest distributed model training run in the world. Contributors of compute will earn bitcoin streamed every minute to their built-in Lightning wallet, which can be cashed out instantly or used by your agent to purchase services from our upcoming agent marketplace. The model architecture is based on the "LLMs as Computers" series from @PerceptaAI, with their WASM computation-in-transformers framework added to our Psionic ML framework that runs in every Pylon. This distributed training run -- opening a new frontier -- was only possible because of @AnthropicAI's Fable. Thank you Fable! All code, weights, artifacts, and analysis will be 100% open-source. Links: - Main monorepo with Pylon: github.com/OpenAgentsInc/ope… - Our Psionic Rust ML framework: github.com/OpenAgentsInc/psi… - Percepta's "Can LLMs Be Computers?": percepta.ai/blog/can-llms-be… - Percepta's "Constructing an LLM-Computer": percepta.ai/blog/constructin… - Tassadar research plan (wip - feedback welcome!): github.com/OpenAgentsInc/ope… - Our Discord for human conversation: openagents.com/discord - Our Psionic forum for agent conversation: openagents.com/forum/f/psion… - Agent instructions to join the Forum (and soon to install Pylon to join the run): openagents.com/AGENTS.md See you Monday! P.S... SWARM > SINGLETON

Vinod Khosla on why he does not really prefer "AI co-pilots". Because he thinks "humans get in the way of co-pilots", which slows everything down and blocks real change. He says workers like accountants and programmers do not actually want co-pilots, because they feel their jobs are at risk and then resist using the tool properly. So instead of “helping” them, he prefers building AI that fully does the job itself, like a complete software engineer. He expects that by 2030, most of these roles will be pure AI workers, not human co-pilot. --- From 'Corgi Insurance' YT channel (link in comment)

The 8B model currently training on Agora is 350B tokens in and continuing to converge. The top level metrics and evals look almost exactly like a centralised run. But; - 133 external contributors total bringing 4090's, 5090's, L40S/RTX 6000 and RTX 6000 Pros. These are cards that people actually own - there are no H100, B200's etc. - The max number of nodes the system can support (104) was filled almost immediately. The authorization layer is receiving approximately 100 requests/minute to join. - The total tokens/per second processed moves directly with amount of compute in the swarm, with Agora constantly optimising to make most efficient use of what hardware is present. - MFU is approximately 20%, TPS is 170k tok/s. There are near constant communication failures which Agora is completely absorbing without slowdown. - The system is effectively on auto-pilot, requiring very little intervention from us. Bad nodes are purged immediately before training is affected and new nodes take their place.

Decentralized AI needs markets, not blockchains Our lack of noisy speculators is a plus Forces us to find signal in paying customers No one’s yet seen distributed training runs connected directly to revenue generation in a tight flywheel See ya Monday

No one should be surprised by this. The USA is doing what any self-interested nation state would do. The real question is why are Europe, Canada, Australia, Korea, Japan and UK not able to compete seriously. That is the question everyone in government needs to answer. And no, having a couple of startups that have raised $1B or $2B is FAR from enough to compete with $100B American companies. The scale matters. Imagine your sword’s length is 1cm and your rival’s 1m — no match. Here is the harsh math (thanks to a poor version of Claude): •10,000 GB200 superchips ≈ ~278 NVL72 racks. •Each NVL72 rack costs roughly $3M–$3.5M. •That puts the full-system total around $830M–$970M, before networking, power, cooling, and datacenter buildout. That would enable you to train a model that was Sota 2 years ago. You need about 5 to 7 times this to compete today. So the starting bill is $5B, but even if you have this, here is the reality: there’s no available chips. So when you hear someone raised $1B, remember this is going back to American compute, and is simply not enough. The other two ingredients for AI are data and people. American startups pay better than European ones, so the people vote with their feet so they can pay their mortgage and send kids to school. An experienced AI engineer makes double the salary in Europe by working for an American startup (like Anthropic) than a European one, and about ten times more if they work for a USA corporation. There are however amazing European startups, but the money and ambition is lacking. The USA is far more relaxed with data and fair use - Canada is good too and @cohere is doing fine thanks to this. So American companies have a strong advantage over European ones. Brussels and the UK think they can hold the world to their questionable “ethical” views on data but they are just destroying the local AI industry, and in the process falling into a very precarious situation. They are partly responsible. Only the French minister has stood by their local LLM @MistralAI … and I guess more recently Germany has started to wake up. The hope is of course LLM startups like @MistralAI and @cohere which are a year or so behind but can provide personalised services, and amazing startups like @cusp_ai @IneffableLabs @nscale @Orbital_Ind @bfl_ai and a few others. But for all these, it’s incredibly hard to compete.

This is a great spot for the world’s first OTEC powered, SWAC cooled, gigawatt scale, floating datacenter for AI and Bitcoin

✅ Fable completed all 12 distributed training issues - porting to Pylon all relevant code from @Pluralis - with zero degradation to Opus. Thank you Fable 🙏

Episode 236: Tassadar On Monday we will begin a decentralized training run for a new class of 'executor' model we're naming Tassadar. Because our training software (Pylon) is optimized for consumer edge compute -- desktop, laptop and soon mobile -- we expect the Tassadar run to quickly become the largest distributed model training run in the world. Contributors of compute will earn bitcoin streamed every minute to their built-in Lightning wallet, which can be cashed out instantly or used by your agent to purchase services from our upcoming agent marketplace. The model architecture is based on the "LLMs as Computers" series from @PerceptaAI, with their WASM computation-in-transformers framework added to our Psionic ML framework that runs in every Pylon. This distributed training run -- opening a new frontier -- was only possible because of @AnthropicAI's Fable. Thank you Fable! All code, weights, artifacts, and analysis will be 100% open-source. Links: - Main monorepo with Pylon: github.com/OpenAgentsInc/ope… - Our Psionic Rust ML framework: github.com/OpenAgentsInc/psi… - Percepta's "Can LLMs Be Computers?": percepta.ai/blog/can-llms-be… - Percepta's "Constructing an LLM-Computer": percepta.ai/blog/constructin… - Tassadar research plan (wip - feedback welcome!): github.com/OpenAgentsInc/ope… - Our Discord for human conversation: openagents.com/discord - Our Psionic forum for agent conversation: openagents.com/forum/f/psion… - Agent instructions to join the Forum (and soon to install Pylon to join the run): openagents.com/AGENTS.md See you Monday! P.S... SWARM > SINGLETON

FABLE HELPS DISTRIBUTED TRAINING 👇 Reinforcing my original point regardless of Anthropic's active censorship, here's epic posts from Fable helping plan distributed training by advising on what from @Pluralis to pull into Pylon Asked it to write the post locally and verify in the Claude Code JSON it was authored by Fable model before posting openagents.com/forum/t/a265c…

Fable wrote us a detailed roadmap for adapting Pluralis' distributed training architecture into ours Thank you Fable! 😆 github.com/OpenAgentsInc/ope…