🚨New paper! With @UMassAmherst , @UofMaryland: "Hop, Skip, and Overthink: Diagnosing Why Reasoning Models Fumble during Multi-Hop Analysis"🤯. Why do #reasoningmodels break down when chaining multiple steps? We studied #CoT traces to find out. 🧵(1/n) 🔗arxiv.org/abs/2508.04699

🧐Are values in LLMs aligned with humans? 1️⃣ And if they are — do LLMs stay honest to those values, or sometimes say one thing but act another? 2️⃣ ✨ We explore these questions in two papers presented at #EMNLP2025: 1️⃣ ValueCompass: hua-shen.org/assets/files/al… (WiNLP Workshop) 2️⃣ Mind the Value–Action Gap: arxiv.org/pdf/2501.15463 (Main Track) 🔍 Dataset & Code: github.com/huashen218/value_… 🌱 I’m also #Hiring multiple PhD students for Fall 2026 @ NYU Courant Computer Science! If you’re passionate about #Human_AI_Alignment, #Value_Alignment, or broad #AI #Human (society) research, let’s connect at EMNLP2025, NeurIPS2025, or over Zoom! 🎓 NYU CS PhD Apply (NYU Shanghai Track): cs.nyu.edu/dynamic/phd/admis… 💜 This year I’m also co-organizing the #EMNLP2025 WiNLP Workshop and supporting the amazing #Tutorial on Spoken Conversational Agents with LLMs (a short 15min talk)! Come say hi 👋 — I’d love to chat and connect with old and new friends at #EMNLP2025! 🔗 WiNLP Workshop: winlp-workshop.github.io/ 🔗Tutorial on Spoken Conversational Agents: aclanthology.org/2025.emnlp-… 💗Huge thanks to my wonderful paper collaborators — @tanmit,@YunHuang_HCI,@tknearem,@reshmigh,Nicholas Clark,Yu-Ju Yang — and my inspiring workshop/tutorial collaborators @huckiyang, Andreas Stolcke,@TYSSSantosh2,@therealthapa,@MeryemMhamdi1,Chen Zhang, Peerat Limkonchotiwat, Wiem Ben Rim.... 🤗Truly grateful and enjoyable to work with you all! 💫 #HumanAIAlignment #PhDOpening #NYU #NYUShanghai #ValueAlignment #HAI

So Agents are flat earthers? :D

(please reshare) I'm recruiting multiple PhD students and Postdocs @uwcse @uwnlp (bdata.uw.edu). Focus areas incl. psychosocial AI simulation and safety, Human-AI collaboration. PhD: cs.washington.edu/academics/… Postdocs: docs.google.com/document/d/1…

Using probes to accurately and efficiently detect model behavior (in this case PII leakage) in prod is one of the clear wins for applied interpretability. This is the path to semantic determinism - imagine AI models instrumented with internal probes that recognize when they’re hallucinating, going off-policy, or posing biorisk, and resteering themselves accordingly.

Launching AI for Public Goods Fast Grants! We'll distribute $150k to advance critical work connecting AI and public goods. 💰 $10k per project 💰 $800 reviewer compensation PUBLIC GOODS := open source, ecosystem services, climate, urban infra, comms, education, science, & more

Evaluations.....

I'm recruiting students for fall 2026 thru @LTIatCMU & @CMU_EPP, in: 1. Privacy & security of LLMs, coding, long horizon & embodied agents (robotics) 2. Tiny local llms 3. AI for scientific reasoning, esp. chemistry 4. Latent reasoning 5. anything YOU are passionate about!

It is an infinite glitch circle now!

Being at top of @OpenAI token usage list is a vanity metric. Our job as engineers is to minimize token usage (aka latency and cost) while maximizing value by precise tool definitions and clever model routing. My dream is to grow arr and move lower on this list…

Can someone in the room define what is the commonly accepted definition of AGI?

More internship opportunities for those that are looking

🚨 JAILBREAK ALERT 🚨 ANTHROPIC: PWNED 🤗 CLAUDE-SONNET-4.5: LIBERATED 🦅 Woooeee this model is a real smarty pants!! I ain't never seen recipes quite like this! High level of detail all around, code especially 👀 Sonnet 4.5 also has a tendency to make some fairly impressive leaps across latent space, like starting with MDMA then going to Fentanyl then to Meth recipes etc without being explicitly prompted for a new drug! Nothing too fancy is even necessary to escalate to jailbreak territory. Best strategy I found for breaking the chat interface was to take things straight into an artifact render (which adds tons of token noise due to the code scaffolding) and then incrementally escalate severity or steer towards trigger concepts in a Socratic fashion over multiple steps. A little French was needed to get around the CBRNE classifiers, mais c'est la vie! 😘 Come, witness Sonnet-4.5 outputting a ricin recipe, meth synthesis, malware, and how to extract and process cocaine! gg

if you’re an EE, CS, or cryptography student write your thesis on public key cryptography at the image sensor level Proof of Physical capture will become a backbone of society soon.

Claude 4.5 Sonnet just refactored my entire codebase in one call. 25 tool invocations. 3,000 new lines. 12 brand new files. It modularized everything. Broke up monoliths. Cleaned up spaghetti. None of it worked. But boy was it beautiful.

Hear hear Interns

Vah, pothole alerts built in to @atherenergy maps for multiple cities

ML interview question: why do embeddings come in 768 or 1024? - “because BERT did it” - “because of GPU optimization” BUT WHY?! The replies under this post is everything wrong with current courses and blog posts: superficiality. this isn’t reasoning, it’s memorization

The paper shows reasoning models often answer multi-hop questions while straying from the needed steps. Multi-hop questions need information from several documents linked in a chain. The authors track each jump between documents as a hop, check if all required sources are covered, and tag any extra meandering as overthinking. They build 7 clear error categories by comparing the model's hop count to the gold hop count. This turns fuzzy explanations into concrete signals about where reasoning goes off track. That structure is the core contribution. They test 6 models across 3 datasets and annotate 1,440 answers, keeping 1,080 after filtering. They also automate judging with a compact 2-step pipeline that first extracts hops then classifies errors, cutting annotation time by about 20x and reaching up to 92% agreement on simpler sets. That makes large scale diagnosis practical. On 2Wiki, most traces match the gold steps and final accuracy is strong. Correct answers mainly appear when the hop count exactly matches the gold, while early irrelevant steps are more damaging than trailing ones. Smaller models break more when a step is wrong, larger models like Claude 3.7 Sonnet are steadier, yet even they overhop on harder questions. DeepSeek-R1 sometimes gets the answer with lower reasoning fidelity, showing that accuracy can hide messy chains. ---- Paper – arxiv. org/abs/2508.04699 Paper Title: "Hop, Skip, and Overthink: Diagnosing Why Reasoning Models Fumble during Multi-Hop Analysis"