Train your own LLM from scratch! A step-by-step repo that walks you through building and training a transformer model from scratch using PyTorch. From downloading training data all the way to generating text. The architecture is built from the ground up following the original "Attention is All You Need" paper. MLP, single head attention, multi-head attention, transformer blocks, and the full transformer model - all coded and explained with detailed diagrams at each step. Training data comes from The Pile - a diverse 825GB open-source dataset covering books, articles, code, websites, and more. The repo includes scripts to download it, preprocess and tokenize it using tiktoken, store it in HDF5 format, and feed it into training batches. You can train a 13M parameter model on a single Colab T4 GPU. At 13M parameters the model starts generating proper grammar and coherent short sentences. For billion-parameter training you need at least an A100 or RTX 4090. The repo includes a full GPU compatibility table so you know exactly what's possible on your hardware. Includes a complete SFT and RLHF guide as a separate notebook for taking your trained model further. Key capabilities: • End-to-end pipeline: data download → preprocessing → training → text generation • Full transformer implementation from scratch with PyTorch • Trains models from 13M to 2B parameters on a single GPU • Training data from The Pile (825GB, 22 diverse datasets) • Tokenization via tiktoken (r50k_base) • SFT and RLHF guide included 100% open source. I've shared the link in the replies!

The huge effect month of birth has on likelihood of making it as a top footballer From Fink Tank by @Dannythefink in Times

Today, I’m proud to announce Homebrew 6.0.0. Since 5.1.0: secure tap trusting, faster JSON API, Linux sandboxing, better defaults, brew bundle improvements, improved performance, initial macOS Golden Gate support. brew.sh/2026/06/11/homebrew-…

Lots of people asked how I used Fable to edit its own launch video so I made a video about that! TLDR it wrote a lot of code & tool calls to use transcription services, ffmpeg, do colorgrading, use the figma mcp, make remotion UI and render it. I didn't touch a video editor.

I do wonder how @claudeai solves this security safeguard problem: Yes, I should not be able to do harm on other software but also I would like to use Fable 5 to find and fix security problems in my own software projects.

"... I get so much out of heinzel every single day ... And this is exactly why I love the project: the safety-critical parts are taken seriously. ..." A Heinzel user wrote this today. That kind of feedback makes the day of any software maintainer. github.com/wintermeyer/heinz…

This is a super exciting release - Claude Fable 5 is the same underlying model as Mythos but with added safeguards. The benchmarks are great and it's SOTA on everything by a margin but I'll add that *qualitatively* also, this is a major-version-bump-deserving step change forward (imo of the same order as Claude 4.5 was in November), peaking especially for long problem-solving sessions on very difficult problems. You can give it a lot more ambitious tasks than what you're used to, the model "gets it" and it will just go, and it's never felt this tempting to stop looking at the code at all (but don't do this in prod!). The model still has quirks that people will run into and the safeguards are configured to be a little too trigger happy for launch, which can hopefully be tuned over time. I feel a lot of things changing as working software increasingly comes out on a tap. The Jevon's paradox kicks in and I feel my own demand for software growing substantially. You can ask for anything - explainers, visualizers, dashboards, bespoke single-use apps (e.g. a full wandb that is hyper-specific just for your project), you can 10X your test suite, auto-optimize code, run giant research projects with custom HTML for the results, anything! "Free your mind" (Matrix ref). Really looking forward to all the things people build!

Wrote up my initial impressions of Claude Fable 5 - it has a big model smell: slow, expensive and capable of crunching through pretty much everything I threw at it simonwillison.net/2026/Jun/9…

You think rewriting Bun in Rust with agents is fun? How about rewriting Git into a memory-safe Rust library? A few weeks and 45B tokens in, the new Grit project is passing 99.3% of the entire 42k test Git testing suite.

I’m pretty AI pilled. This loop stuff is slop. I respect @steipete for his innovation - but openclaw is a bloated unstable pile of garbage because of stuff like this. I’m all for loops of crons and webhooks where an AI agent wakes up and performs some task like cleanup, or updates the docs or triages errors. I think these are great for standard well defined tasks with a fairly deterministic route (a.k.a workflows). I think what these guys are talking about now is jumping the gun. The models need to be guided, and you want to atleast skim their output so you don’t end up with slop. Humans are far better planners and architects than models. You absolutely shouldn’t delegate away prompting and reviews in my opinion. this encourages the creation of crappy buggy unsafe software that actually hurts adoption.

Tell me you have a foldable coming without telling me you have a foldable coming

When we first demoed Claude Code internally, it got two reactions on Slack. A year after GA, @_catwu and I sat down to talk about what's changed: why I use auto mode instead of plan mode, how routines fix bugs before I see them, why I do most of my coding from my phone now, and where the product is going

Today, Anthropic engineers on average ship 8x as much code per quarter as they did compared to 2021-2025.

Announcing deflua.com 🎉 The new home for Lua, the pure-Elixir Lua 5.3 VM for @elixirlang. Scriptable, sandboxed, stupid easy. Embed untrusted code (AI agent tools, user formulas, plugins) all on the BEAM, zero NIFs. Plus a live playground 👇

Elixir v1.20 released! Now officially a gradually typed language: Elixir type checks every single line of code, finding bugs and dead code, without developer overhead (no typing signatures) and extremely low false positives rate. Plus a faster compiler! Links and reports below.

Great and tragic news at once. Emmanuel, the 5-year old featured in this report, has since recovered and left the Ebola ward, his father told me. But the lab technician I met in the next ward deteriorated and died last night. His name was Bienfaits. nytimes.com/video/world/afri…

Not an easy one to read. nytimes.com/2026/06/01/opini…

📝 Rails 8.1 adds first-class Markdown support, including .md routes, and model-level to_markdown methods. A walkthrough of the new API and common use cases. #Rails blog.saeloun.com/2026/05/22/…

Reading "Steve Jobs in Exile: The Untold Story of NeXT and the Remaking of an American Visionary" by @geoffrey_cain. Great book! Interesting dive into IT history. Some discrepancies about Jobs compared to "Steve Jobs: The Exclusive Biography" by @WalterIsaacson. I do wonder who is right.