The data used to train an AI model is vital to understanding its capabilities and risks. But how can we tell whether a model W actually resulted from a dataset D? In a new paper, we show how to verify models' training-data, incl the data of open-source LMs!arxiv.org/abs/2307.00682

Unless this changes, OpenAI researchers on visas need to plan for the fact they’ll probably lose access to internal models, and therefore their ability to do their jobs moving forward, sometime in the next couple months. I hope the company acts to prevent that.

…do we think there’s a Docusign link?

This is the first serious analysis of what is likely on track to happen to Europe and the middle powers. (I expect it is understating the direness of the situation for palatability.) Strongly recommend.

We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵

In medieval times, within the arms race of ever more demonic torture devices, some sadistic genius came up with the idea of the Little Ease. This was a prison cell built so small in every dimension that a grown man could not stand upright in it nor lie down at full length nor properly sit. The pain is relentless and without relief and inflicted by one's own body. Prisoners were known to go insane within a few days. A stay at the Little Ease was considered even more cruel than the rack, the thumbscrew, and the other ghoulish machinery of the Tower of London. A breeding pig will spend her whole life in a version of that box. These are social, roaming creatures (more intelligent than dogs) who will never leave this corset of steel. They have been selectively bred to be bigger than their frames can support. Yet we put them in cells so confined that they cannot comfortably sit, and their attempts to do so (for example, by sneaking their limbs into adjacent stalls) reliably lead to fractures and sprains. They cannot sweat, yet have nothing to roll around in to cool themselves off. Except their own manure, which (contrary to the common misconception) they are so averse to (thanks to their strong sense of smell) that new sows will often suffer from constipation to avoid soiling the space from which they eat and sleep. Here is how the writer Matthew Scully described what saw at one of Smithfield’s “gestation barn”: > “Sores, tumors, ulcers, pus pockets, lesions, cysts, bruises, torn ears, swollen legs everywhere. Roaring, groaning, tail biting, fighting, and other “Vices,” as they’re called in the industry. Frenzied chewing on bars and chains, stereotypical “vacuum” chewing on nothing at all, stereotypical rooting and nest building with imaginary straw. And “social defeat,” lots of it, in every third or fourth stall some completely broken being you know is alive only because she blinks and stares up at you … creatures beyond the power of pity to help or indifference to make more miserable, dead to the world except as heaps of flesh into which the [insemination] rod may be stuck once more and more flesh reproduced.” — The Save Our Bacon Act is trying to unroll the few state protections we have against this barbaric cruelty - for example California’s Prop 12 - which banned the sale of pork from pigs kept in gestation crates. It’s incredibly important we don’t end up with this sort of federal preemption. SOB will not only kill the most important animal welfare related laws in the US of the past decade, but more importantly, it will also restrict ALL future legislative progress (aka how the animal welfare movement has gotten its biggest wins). The Senate is currently deciding whether to add the SOB Act to the Farm Bill. With relatively little money now, we can discourage the most pivotal senators in the Ag committee from backing this amendment. Defeating this bill is even more important given the amount of philanthropic funding I expect to come online in the next year or two. It will plausibly be over 10x more expensive to repeal SOB than to prevent it from passing in the first place. All that money that could be spent transforming our society's relationship to mass animal suffering will instead have to be spent just getting us back to where we are right now. That's why money spent now fighting this bill (and I mean right NOW) is so effective. If you’re in a position to donate six figures, please DM me.

I feel almost entirely bottlenecked on wisdom amid ambiguity, and AI systems are rarely wise. Is my task distribution just different from others’, or do you think I’m probably using them wrong?

It would be nice if AI companies and others (e.g. startups) tried to have their AIs hillclimb on this task. ARC is approximately our only current bet on scalable/worst-case solutions to alignment and they could be boosted by relatively checkable work!

No one should be able to order a bioweapon through the mail. @IFP & @JoinFAI are proud to co-lead an open letter calling for mandatory DNA synthesis screening & recordkeeping. Signatories include: - Sam Altman, CEO & Co-Founder, OpenAI - Dario Amodei, CEO & Co-Founder, Anthropic - David Baker, Director, Institute for Protein Design; 2024 Nobel Prize in Chemistry recipient - Patrick Collison, CEO & Co-Founder, Stripe - Paul Graham, Founder, Y Combinator - Demis Hassabis, CEO, Google DeepMind; 2024 Nobel Prize in Chemistry recipient - Emily Leproust, CEO & Co-Founder, Twist Bioscience - Lawrence Lessig, Roy L. Furman Professor of Law and Leadership, Harvard Law School - Gerald W. Parker, former Special Assistant to the President for Biosecurity and Pandemic Response - Mustafa Suleyman, CEO, Microsoft AI - Alex Tabarrok, Professor of Economics, George Mason University - Alexandr Wang, Chief AI Officer, Meta; Founder, Scale AI - Christine E. Wormuth, President & CEO, Nuclear Threat Initiative; 25th Secretary of the Army Read the letter and see the full list of signatories: screendna.org Many DNA synthesis companies voluntarily screen orders to mitigate biosecurity risks, but no law requires them to do so. Leaders in AI, biotech, life sciences, national security, and the nucleic acid synthesis industry agree that Congress should act to strengthen safeguards against biological threats. @deanwball put it well in the WSJ: “If you’re synthesizing the stuff that yields biological life and viruses, we’re asking you to screen to see whether it is dangerous in some way. That seems like a reasonable thing for society to insist upon.”

AI is advancing quickly. Society’s ability to manage its risks must advance just as fast. Today we’re sharing our vision for AI Resilience, with more than $130M in initial grants underway across bio-resilience, cyber-resilience, AI model safety, and AI’s impact on young people: openaifoundation.org/news/re…

On Friday, I resigned from OpenAI. Today is my first day at the OpenAI Foundation, where I'm helping build out our AI Resilience program. There is a great deal to do before superintelligence, and little time to do it. If you were debating when to pivot to help, it's time.

before you overreact, the Holy See is notoriously captured by Open Philanthropy

my guess is they got an ICML weak-reject