Applying the security mindset to everything @PalisadeAI

Joined March 2013
405 Photos and videos
Pinned Tweet
I think the AI situation is pretty dire right now. And at the same time, I feel pretty motivated to pull together and go out there and fight for a good world / galaxy / universe @So8res has a great post called "detach the grim-o-meter", where he recommends not feeling obligated to feel more grim when you realize world is in deep trouble It turns out feeling grim isn't a very useful response, because your grim-o-meter is a tool evolved for you to use to respond to things being harder *in your local environment* rather than the global state of things So what do you do when you find yourself learning the world is in a dire state? I find that a thing that helps me is finding stories that match the mood of what I'm trying to do, like Andy Weir's The Martian You're trapped in a dire situation and you're probably going to die, but perhaps if you think carefully about your situation, apply your best reasoning and engineering skills, you might grow some potatoes, ducktape a few things together, and use your limited tools to escape an extremely tricky situation In real life the lone astronaut trapped on Mars doesn't usually make it. I'm not saying to make up fanciful stories that aren't justified by the evidence. I'm saying, be that stubborn bastard that *refuses to die* until you've tried every last line of effort I see this as one of the great virtues of humanity. We have a fighting spirit. We are capable of charging a line of enemy swords and spears, running through machine gun fire and artillery even though it terrifies us No one gets to tell you how to feel about this situation. You can feel however you want. I'm telling you how I want to feel about this situation, and inviting you to join me if you like Because I'm not going to give up. Neither am I going to rush to foolhardy action that will make things worse. I'm going to try to carefully figure this out, like I was trapped on Mars with a very slim chance of survival and escape Perhaps you, like me, are relatively young and energetic. You haven't burnt out, and you're interested in figuring out creative solutions to the most difficult problems of our time. Well I say hell yes, let's do this thing. Let's actually try to figure it out 🔥 Maybe there is a way to grow potatoes using our own shit. Maybe someone on earth will send a rescue mission our way. Lashing out in panic won't improve our changes, giving up won't help us survive. The best shot we have is careful thinking, pressing forward via the best paths we can find, stubbornly carrying on in the face of everything And unlike Mark Watney, we're not alone. When I find my grim-o-meter slipping back to tracking the dire situation, I look around me and see a bunch of brilliant people working to find solutions the best they can So welcome to the hackathon for the future of the lightcone, grab some snacks and get thinking. When you zoom in, you might find the problems are actually pretty cool Deep learning actually works, it's insane. But how does it work? What the hell is going on in those transformers and how does something as smart of ChatGPT emerge from that?? Do LLMs have inner optimizers? How do we find out? And on that note, I've got some blog posts to write, so I'm going to get back to it. You're all invited to this future-lightcone-hackathon, can't wait to see what you come up with! 💡
31
63
666
235,668
Jeffrey Ladish retweeted
Would be great if people at Anthropic could collect data on this! (But I understand they might be busy and this might not last long...)
Jun 13
The moment is ripe for natural experiments on (R&D compute, labor) complementarities!
1
3
45
5,262
Some additional context: In the last few days I’ve joked that I have “AI mania”, because I can’t stop using Claude Code. I’ve been walking around, sending voice notes to direct my Fable cloud agent I built with Fable. I brought out my laptop during date night (sorry @collegraphy)
Am I personally annoyed losing access to Fable? Yes, I’m super annoyed! I’ve been building basically nonstop since it came out. Biggest shift I’ve experienced using AI. And also this doesn’t matter much. Getting superintelligence right, not losing control of AI, is what matters!
5
19
2,096
I think the uproar right now would be much larger. And these are just today’s capabilities. What’s it going to be like a year from now? I don’t have a clear takeaway or recommendation from this. It’s just wild to notice the feelings in myself and extrapolate.
1
10
317
Another example: I made this meme 1 shot while driving by sending a voice note to my fable agent (to grab a picture of the original meme and then use Image 2 api to make the new version)
6
342
Am I personally annoyed losing access to Fable? Yes, I’m super annoyed! I’ve been building basically nonstop since it came out. Biggest shift I’ve experienced using AI. And also this doesn’t matter much. Getting superintelligence right, not losing control of AI, is what matters!
The most dangerous thing about Mythos is probably speed-up of AI development, nudging the world closer to full RSI and actual superintelligence. This is far more concerning than the models’ cyber or bio capabilities.
8
6
103
6,624
People’s emotions are valid. It’s super annoying to lose access to such an incredible tool. And I’m not saying the admin’s actions were good. I don’t understand the threat model they were considering, and I don’t think export controls make sense if the main threat was cyber.
1
1
27
832
But I try not to lose track of the plot. Superintelligence is not a cyber capability or a bio capability. It’s the end of the game.
2
24
598
The most dangerous thing about Mythos is probably speed-up of AI development, nudging the world closer to full RSI and actual superintelligence. This is far more concerning than the models’ cyber or bio capabilities.
6
5
43
6,231
I think it’s also good to consider the cyber and bio capabilities of the model. If I were running the admin’s AI efforts, I’d greatly expand CAISI and the NSA AI center and task them with evaluating all of these risks in depth and publishing most of the findings.
2
1
17
769
I’d also work with third party auditors and the labs, including asking labs to evaluate each other’s safety plans and safeguards. Transparency here would go a long way!
1
8
448
2
3
79
3,589
Not enough people appreciate that @m_bourgon is at the frontier of getting agents to do extremely useful mundane work. Ask him about the Gmail API…
Fun fact: this is a deterministic and reproducible conversion. Creating a process to faithfully recreate the contents of a 300 page PDF like this is kind of a nightmare. It took many millions of tokens from a collection of agents led by Fable 5 running a loop of writing/improving conversion code then verifying the output against images of the source PDF. The process ended up generating over 4k lines of Python.
1
26
4,420
I am uncomfortable with the number of @DKokotajlo's predictions coming true
Another quite successful prediction by @DKokotajlo : Fable is intentionally nerfed for frontier ML research. This is within ~3 months of Daniel's prediction of Q1 2026 (made in 2023). Although I don't think Mythos is automating ML research to the same extent as his prediction.
9
12
178
8,853
Similar but I like the new energy
Centaurs are fake and Lucretius proved it in the first century BC!
2
971
It's very good seeing this from both OpenAI and Anthropic. Best news all year.
Jun 8
now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development
12
20
247
13,125
Australia ABC just released a 45 min feature on the AI race. @SteveCannane stopped by my office a few weeks ago and we had a great conversation about the controllability of AI agents and the risk of human extinction
5
12
49
2,561
“A Grassroot Institute of Hawaii study found the Act costs Hawaiian households roughly $1,800 per year.” Why don’t residents of Hawaii lobby hard against the Jones Act?
Today is June 5th, one day to take a break from fighting each other online, and remind ourselves of our shared humanity and common goals by uniting around the one thing we all agree about: Repealing the Jones Act. June5.xyz
16
955