You can set up an base model (here gpt-neox-20b, all shoggoth, all the time) and have Opus use it to infuse its own anemic writing with some real crazy. Suprisingly Opus here does have a real appreciation of what its otherwise missing out on and just can't bring itself to do. Also: McDonalds line is pretty great.

Been at Sanity nearly eight years. Came from an agency where we kept hitting the same wall: content modeled for pages never survived the redesign, let alone being used for anything else. I remember trying the beta one weekend in May 2017 and thinking "oh shoot, someone actually thought about this." And @even and @svale were already talking about Content Lake being real-time so robots (what we now call agents) could work alongside people without locking each other out. Turned out to be a pretty solid bet. A million users in, I'm just one of them. Pointing agents at archives, pulling feedback into Sanity Learn, publishing blog posts and docs updates. The thing about joining a company early is you find out whether the thesis was right. This one still is.

holy shit academic philosophers are finally taking LLMs seriously. david chalmers directly cites @repligate !!

Our Content Agent is now not only in-product and available as an API, but now also officially up on Slack marketplace. Let's let it speak for itself: lnkd.in/e7CA5jvh

This is like the Miyazaki zombie animation video for people who read. “I strongly feel that this is an insult to text itself”

I love how the only setting I need on GitHub is buried under profile > settings > developer settings (!) > PAT > fine grained

Structure powers intelligence.

"The model wasn't broken. The context was." @MHillestad on why AI agents need structured content, not bigger context windows. Structure powers intelligence. sanity.io/blog/structure-pow…

Simen cooked a collaborative environment for humans and agents during the Holidays. Stay tuned.

We're the CMS @leerob migrated away from. He's got valid points. But we've got some context to add. sanity.io/blog/you-should-ne…

New are is coming。。。。#nanobanana

We hosted the first-ever developer conference dedicated to AI-powered Content Operations. Over 300 developers and technical leaders driving the future of content at scale and how to go from theory to production. 13 hours. One track. AI, content ops, and culture. If you missed it, or want to go back to the future with us, we got the sessions turned into a video podcast → sanity.io/blog/everything-ny…

You store your content as files in a repo. My CMS writes PRs. We are not the same.

New Paper! Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents A longstanding goal of AI research has been the creation of AI that can learn indefinitely. One path toward that goal is an AI that improves itself by rewriting its own code, including any code responsible for learning. That idea, known as a Gödel Machine, proposed by @SchmidhuberAI over two decades ago, is a hypothetical self-improving AI. It optimally solves problems by recursively rewriting its own code when it can mathematically prove a better strategy, making it a key concept in meta-learning or “learning to learn.” While the theoretical Gödel Machine promised provably beneficial self-modifications, its realization relied on an impractical assumption: that the AI could mathematically prove that a proposed change in its own code would yield a net improvement before adopting it. Sakana AI, in collaboration with Jeff Clune’s lab at UBC, proposes something more feasible: a system that harnesses the principles of open-ended algorithms like Darwinian evolution to search for improvements that empirically improve performance. We call the result the Darwin Gödel Machine. DGMs leverage foundation models to propose code improvements, and use recent innovations in open-ended algorithms to search for a growing library of diverse, high-quality AI agents. Applied to practical tasks, we implemented Darwin Gödel Machine as a self-improving coding agent that rewrites its own code to improve performance on programming tasks. It creates various self-improvements, such as a patch validation step, better file viewing, enhanced editing tools, generating and ranking multiple solutions to choose the best one, and adding a history of what has been tried before (and why it failed) when making new changes (see the attached video). We believe that Darwin Gödel Machines represent a concrete step towards AI systems that can autonomously gather their own stepping stones to learn and innovate forever!

New Paper: Continuous Thought Machines 🧠 Neurons in brains use timing and synchronization in the way that they compute, but this is largely ignored in modern neural nets. We believe neural timing is key for the flexibility and adaptability of biological intelligence. We propose a new neural architecture, “Continuous Thought Machines” (CTMs), which is built from the ground up to use neural dynamics as a core representation for intelligence. By using neural dynamics as a first-class representational citizen, CTMs naturally perform adaptive computation. Many emergent, interesting behaviors arise as a result: CTMs solve mazes by observing a raw maze image and producing step-by-step instructions directly from its neural dynamics. When tasked with image recognition, the CTM naturally takes multiple steps to examine different parts of the image before making its decision. This step-by-step approach not only makes its behavior more interpretable but also improves accuracy: the longer it “thinks,” the more accurate its answers become. We also found that this allows the CTM to decide to spend less time thinking on simpler images, thus saving energy. When identifying a gorilla, for example, the CTM’s attention moves from eyes to nose to mouth in a pattern remarkably similar to human visual attention. I think this work underscores an important, yet often lost, synergy between neuroscience and AI. While modern AI is ostensibly brain-inspired, the two fields often operate in surprising isolation. By starting with such inspiration and iteratively following the emergent, interesting behaviors, we developed a model with unexpected capabilities, such as its surprisingly strong calibration in classification tasks, a feature that was not explicitly designed for. When we initially asked, “why do this research?”, we hoped the journey of the CTM would provide compelling answers. By embracing light biological inspiration and pursuing the novel behaviors observed, we have arrived at a model with emergent capabilities that exceeded our initial designs. We are committed to continuing this exploration, borrowing further concepts to discover what new and exciting behaviors will emerge, pushing the boundaries of what AI can achieve.

The feeling of seeing our new logo on the @Nasdaq tower is unreal