1/9 Today, alongside the team at @phylo_bio, we are sharing our first step toward having AI build better AI benchmarks using privileged information—and there is much more to come 👀 Meet BenchGuard: an automated auditing framework for scientific agent benchmarks. 🔬👇

9/9 A special thanks to the team at @phylo_bio for the core support and vision in advancing scientific AI evaluation. 🧬 This work was made possible through an incredible collaboration. Grateful to all our co-authors and institute: @TzWangCompbio, @mintaylu, @KexinHuang5, @YuanhaoQ, @sara_mostafavi, @genentech, and @uwcse @phylo_bio

Announcing Biomni — the first general-purpose biomedical AI agent. Biomni is a free web platform where biomedical scientists can immediately delegate their tasks to Biomni, starting today! Biomni automates literature reviews, hypothesis generation, protocol design, bioinformatics analysis, clinical reasoning, and much more — scaling biomedical expertise for 100× the number of discoveries. Key results: ➡️ Designed a cloning experiment with real-world wet-lab validation; on par with 5 year expert in a blind test ➡️ Ran 458-file wearable bioinformatics analysis in 35 minutes vs. 3 weeks (800x faster) for human expert ➡️ Uncovered novel hypothesis: new TFs regulating skeletal lineages on a large scRNA scATAC data ➡️ Human-level performance on LAB-bench DbQA and SeqQA, with SOTA at Humanity’s Last Exam and across 8 new biomedical tasks—ranging from GWAS and rare disease diagnosis to microbiology and drug repurposingPowered by: ➡️ Biomni-E1 – the first unified environment designed for a biomedical agent—encompassing 150 tools, 59 databases, 106 software—systematically curated from 2,500 bioRxiv papers ➡️ Biomni-A1 – a generalist agent with retrieval, planning, and code as action Biomni is an open-source initiative: we invite the community to build on it and advance biomedical research at scale. - Try it now: biomni.stanford.edu - Paper: biomni.stanford.edu/paper.pd… - Code: github.com/snap-stanford/bio… - Join the community: tinyurl.com/biomni-slackWith Amazing team and collaborators @StanfordAILab @StanfordMed @StanfordCancer @genentech @arcinstitute @UCSF @UW @PrincetonAInews @KexinHuang5 @serena2z @hcwww_ @YuanhaoQ @mintaylu @yusufroohani @RyanLi0802 @LinQiu0128 Gavin Junze Di Shruti Jennefer Xin Zhou @MWheelerMD Jon Bernstein @MengdiWang10 @PengHeAtlas @SnyderShot @lecong Aviv Regev