I wished there was an ASI novel set in China 🇨🇳 🤖 - and now it exists! From the same guy who wrote Crystal Society. Highly recommend it. I started off my day "just reading a chapter" and ended up reading it all in one sitting. Fascinating look into what if you made an ASI that was a corrigible as possible. What would happen? Inspired by the author's own research on corrigibility.

In my spare time, I'm working on a mapmaking tool

In '96 I proposed Fire-The-CEO stock markets, which show impact of CEO leaving on stock price. Today, you can bet here on impact of ~$1T CEO Performance Award on Tesla stock price. Current estimate: it adds ~6.5%. x.com/_futarchy/status/19756…

Has anyone made LLMs play Risk with each other yet? Or Diplomacy?

new cursor rule: dont-tempt-fate.mdc

Gemini Flash 2.0 deciding the appopriate way to get past a viscious poodle is to kick it

Today I'm launching my new company @GeneralAgentsCo and our first product. Introducing Ace: The First Realtime Computer Autopilot Ace is not a chatbot. Ace performs tasks for you. On your computer. Using your mouse and keyboard. At superhuman speeds!

Quite keen to visit

Sycophancy inflation? Was still riding the high from finding out from Claude how many of my questions were fascinating

btw, i think it's *totally possible* that we're all just wrong about near-term x-risk. like through a combination of selection effects, drinking the koolaid and mutually reinforcing each other's views, we've worked ourselves into a panic over an implausible scenario (read to end) like maybe there is a huge silent majority of AI experts who think our concerns are totally silly! it's impossible to determine whether this is the case, because people who are unworried about x-risk usually don't discuss it maybe alignment is trivially easy. maybe it turns out not to be a real problem. maybe we really are just building tools and not creatures. maybe ASI isn't even possible due to some real-world constraints we don't know about but i think my current level of alarm RE AI x-risk is totally warranted, even while believing that the above is plausible! if in 20 years none of this has played out, I don't think i will have been wrong to worry. I'm increasingly frustrated by the inability of some on the skeptic side to exhibit this same humility. "what if we're wrong?" is a far more important q for them to ask than it is for us! this rant is inspired by an event on AI governance I went to last night (which I will not name). the panellists were just so confidently, egregiously dismissive of x-risk that I declined to stay for the free dinner afterwards so no one would see the smoke coming out of my ears these were important, credible ppl with real expertise! while they were describing the silly little x-risk panic of 2023 as an episode of misguided alarmism that scared so many poor laypeople, I could feel the colour rising in my cheeks, embarrassed to be accused of credulity what angers me here is not that these experts do not believe in existential risk. it's that they unfaithfully represent the state of the discourse. they fail to acknowledge their many peers who take it seriously. they ignore the total lack of consensus in the field people communicating to the public about x-risk owe us honesty. we deserve to know that there's no pilot flying this plane. If you're an expert who is (somehow) sure that birthing an intelligence greater than our own poses no risks, caveat that this is far from a settled debate also, trust your instincts. don't let anyone make you feel silly for worrying about AI x-risk. the arguments for it *are* as straightforward and intuitive as they seem. even if they turn out to be wrong, you won't have overreacted