🧠 Exciting news for #neuroscience! Launching deepmriprep: Voxel-based Morphometry (VBM) preprocessing via neural networks in ~10 seconds per #brain image 🚀 🔗 Preprint: arxiv.org/abs/2408.10656 🔗 GitHub: github.com/wwu-mmll/deepmrip… Install via "pip install deepmriprep"

This really speeds up preprocessing and shows - yet again - that neural networks are eating software. Great work, @codingfisch !! Happy to see this finally published.

📢Out now! @codingfisch and colleagues present deepmriprep, a tool that leverages neural networks to enable 37x faster Voxel-based Morphometry preprocessing of MRI data than existing methods. nature.com/articles/s43588-0…

Our GPU stack for both NVIDIA and AMD, aside from minimal pieces of signed firmware, is 100% open source and pure Python except for the compiler. It's not using vendor drivers, frameworks, or libraries. That's why it's so easy to make it work on Mac. For compilers, on AMD, we use upstream LLVM, and on NVIDIA, we use the NAK compiler from the MESA project. We plan to replace the compiler with pure tinygrad in a year or two as well. With RANGEIFY merged, our lowering stuff now matches the state of the art, TVM style. We're studying ThunderKittens and TileLang for speed at that level, and should have all this stuff ready in 200 days for the due date of our AMD Llama 405B training contract. Due to tinygrad's small size and pure Python nature, it's the easiest ML library to make progress on, aka fastest slope of improvement. With Megakernel style for scheduling, MODeL_opt style for planning, and E-graph style for symbolic, we should blow past the state of the art in PyTorch and JAX speed. If we do that, NVIDIA's moat is over. It's 1000 lines at most to add a new accelerator to tinygrad. And I don't mean to add a new accelerator with help from a kernel driver, compiler, and libraries. Just 1000 lines of software for the *whole* accelerator speaking right on the PCIe BARs, like what tinygrad is doing with the NVIDIA and AMD GPUs now.

This!

These are special times 📈

The curse of dimensionality. The blessing of compositionality.

When you store your knowledge and skills as parametric curves (as all deep learning models do), the only way you can generalize is via interpolation on the curve. The problem is that interpolated points *correlate* with the truth but have no *causal* link to the truth. Hence hallucinations. The fix is to start leveraging causal symbolic graphs as your representation substrate (e.g. computer programs of the kind we write as software engineers). The human-written software stack, with its extremely high degree of reliability despite its massive complexity, is proof of existence of exact truthiness propagation.

Couldn't agree more with Linus here. Making your code running as fast as possible (or just faster than someone's else code) is the best feeling you can get from programming The issue that it is extremely hard to find the project that will pay you for instruction level efficiency

🧐view(your_data) in the terminal🧐 "pip install viiew" for a - colorfull - scrollable - sortable view of your array/Tensor/DataFrame! Supports numpy, pandas, torch, jax & @__tinygrad__ and is tiny: 4.1kB (0 dependencies)!

Our thesis at Ndea: simple theories should be discoverable using little data and little compute. And everything humans have ever invented is a fairly simple composition of fairly simple theories.

NCCL sending the loss value from the last pipeline parallel stage back to rank 0 so the user can print it

I’m speaking at 2pm on Wednesday at the Upper Bound AI conference. I am in the “business of AI” track, but my talk will be about reinforcement learning research. Video should be available after the conference.

🚀 Release of tinygym: RL in tinygrad 🚀 It learns Pong in 5 minutes on a 3090! Like in flashrl you can ✅ Play against your RL agent ⚔️ ✅ Add your env by adding one .pyx file! Hopefully leads to another RL stream from @realGeorgeHotz @__tinygrad__ github.com/codingfisch/tinyg…