📰 Excited to share our new work on risk control in prediction! Multiple testing leads to practical calibration algorithms with PAC guarantees for any statistical error rate. Works with any model data distribution! arxiv.org/abs/2110.01052 #Statistics #MachineLearning

Announcing the Statistical Frameworks for Uncertainty in Agentic Systems workshop at ICML '26!

(1/5) Modern reasoning systems rely on test-time scaling: CoT, self-consistency, MCTS... But two challenges remain: 1️⃣ Confidence signals shift across tasks/prompts 2️⃣ Stopping decisions are typically static and heuristic We ask: Can we adapt confidence within each reasoning trajectory — while still preserving statistical guarantees? Calibrating LLM reasoning in test-time scaling is not new. But what if calibration itself could adapt online — at test time — to the specific reasoning trajectory of each instance? Our new paper proposes a Test-Time Training framework for calibrating generalizable LLM reasoning, enabling instance-level adaptation with distribution-level robustness. Paper: arxiv.org/abs/2604.01170

Today I'm sharing a preprint on conformal risk control for non-monotonic losses, a paper three years in the making. The key idea: validity of conformal can be reframed as a consequence of algorithmic stability. Therefore, any stable algorithm inherits a conformal guarantee. 🧵

Postdoc opportunity — If you do ML/stat/applied math/… and want to work at the frontier of biology , come join us! 🤖 🧬

Interested in pursuing #machinelearning, #appliedmathematics, #statistics, or #computationalresearch to work on biomedical problems at the @broadinstitute? Apply to become a @Schmidt_Center postdoctoral associate: broad.io/ewsc-postdoc

🎉 Our new machine learning challenge – Obesity ML Competition: Tackling Metabolic Diseases – is officially open! Register, watch our introduction videos and lecture series, and begin coding today: broad.io/MLC-2025 @broadinstitute @crunchDAO

Exciting research internship!

we're hiring a Ph.D. intern! join us @genentech in South San Francisco for a summer advancing ML & statistical approaches for clinical trial design & analysis 📉💊DMs are open, feel free to reach out! 🔗tinyurl.com/yc3hfndp

I wrote a review paper about statistical methods in generative AI; specifically, about using statistical tools along with genAI models for making AI more reliable, for evaluation, etc. See here: arxiv.org/abs/2509.07054! I have identified four main areas where statistical thinking can be helpful. These are just a subset of what is out there; other topics have been well-covered in other reviews. 1. Designing "statistical wrappers" around a model, for instance, changing behavior of a trained model (e.g., abstaining), where a score, e.g., an "unsafety score" is too high. The key connection to statistics is to use the quantiles of the loss (on a calibration set) to set the critical threshold, thus enabling conformal-type high probability guarantees. 2. Closely related, methods for uncertainty quantification, which enable the model to express uncertainty in an answer. A crucial component here is "calibration", whereby the uncertainty is required to reflect reality. 3. Statistical methods for AI evaluation: Specifically, tools for statistical inference (e.g., confidence intervals) on model performance. Exciting recent work proposes careful statistical models for leveraging a very small high-quality dataset, possibly combined with much larger low-quality datasets, for accurate evaluation. 4. Experiment design and interventions. Careful AI experiments to understand and steer models may require interventions such as modifying experimental settings in a controlled manner. This brings up connections to classical experimental design in statistics. This connection has largely remained implicit so far, and my review aims to make it more explicit; hoping that experimental design principles will become useful here. This review references the work of many, including @HamedSHassani @obastani @tatsu_hashimoto @yuekai_sun @CsabaSzepesvari @ml_angelopoulos @stats_stephen @yaniv_romano @yaringal @KilianQW @_onionesque their teams, and some work that I was also involved in. Hopefully, my review will be helpful to orient yourself in this exciting area. Nonetheless, since the area is rapidly expanding, it is possible that I missed important references. Please feel free to let me know of anything that I should add/change!

If you work at the intersection of CS and economics (or think your work is of interest to those who do!) consider submitting to the ESIF Economics and AI ML meeting this summer at Cornell: econometricsociety.org/regio…

(1/5) Beyond Next-Token Prediction, introducing Next Semantic Scale Prediction! Our @NeurIPSConf NeurIPS 2025 paper HDLM is out! Check out the new language modeling paradigm: Next Semantic Scale Prediction via Hierarchical Diffusion Language Models. It largely generalizes Masked Diffusion Models (MDM), and provides the progressively denoising capability for each token in the semantic level. Minimal computation overheads, much better results! arxiv: arxiv.org/abs/2510.08632 code: github.com/zhouc20/HDLM

Happy to share that our paper on how to obtain reliable statistical inferences from satellite-based maps is now published in Remote Sensing of Environment!

Today, NSF announced an add’l 500 NSF Graduate Research Fellowship Program awardees for the 2025-2026 cohort, bringing the total to approx 1,500. #NSFGRFP supports grad students as they pursue their dreams, build STEM skills, & become the next generation of innovators & leaders.

📢If you're interested in conformal prediction, algorithms w/predictions, robust stats & connections between them from a theory perspective, join us for a workshop at #COLT2025 in Lyon 🇫🇷 June 30! Submit a poster description by May 25, more here: vaidehi8913.github.io/predic…

Imagine a world without MIT.

Our paper notifications are out! Congratulations to the authors and look forward to an exciting lineup of discussions. Stay tuned for more details! #ICLR2025

🙌🎉Our 2025 recipient of the COPSS Presidents' Award, is Lester Mackey! This award is given annually to a young member of the statistical community in recognition of outstanding contributions to the profession of statistics.

📢 We are hiring a postdoc to work on remote sensing of soil carbon and land degradation! 🌱🗺️ The position will be hosted by the Earth Intelligence Lab & @mitenergy, with an earliest start date of April 2025. To apply: forms.gle/9iDJRX4nG7odXJLa9

What are prediction sets good for? It turns out just as calibration is the "right" way of quantifying uncertainty for risk-neutral (expectation maximizing) decision makers, prediction sets are the right way of quantifying uncertainty for risk-averse decision makers.