Seeing a lot of bitter lesson/scale > specialized model tweets about this study, but short of OE/UTD intentionally buying weaker models, this result is more of a *harness* issue than a model one. MedQA score ranges across models were within 9 points (so less likely a base knowledge gap); the gap between gen vs clinical models widened on the real world clinical question answers, where the problems were mostly in omission and disorganized outputs rather than wrong facts.

Me teaching every surgeon I meet how to use ChatGPT for Clinicians / OpenEvidence.

Immensely proud.

Quick piece of advice for clinicians being offered equity to join a startup’s clinical or scientific advisory board: Its total BS when a founder says, “We’re offering you 10,000 shares. That’s 0.1% of the company, and since we just raised at a $100M valuation, that’s $100K of value.” That is not really how it works. Your common shares are not worth the same as the preferred shares investors just bought. There will likely be future dilution. And the paper value is not the same as actual economic value. Clinicians should evaluate these roles differently than typical startup advisors because you are not just giving advice. You are lending credibility, validation, and halo to the company’s care model. The real questions are: - Do you trust the founders and feel comfortable vouching for them? - Do you believe the problem they are solving matters? - Do you believe in the care model they are building? - Do you believe the product can actually improve care? If the answer is yes, the opportunity may still be worth it for the learning, relationships, and ecosystem connectivity. But if you want to evaluate the economics, ask the founder directly: how much dilution do you expect the common stock to take, and what would my advisor shares be worth pre-tax in a low, base, and high exit case?

Healthcare is one of the few sectors where policy has arguably *driven* tech innovation and diffusion rather than vice-versa. With that in mind, there are plenty of policy levers in both public health and medicine that are positive-sum! Reference pricing to promote service competition and better correlate price to quality, all payer rate setting (e.g., Maryland) to reduce admin complexity, lead removal from paint/water to reduce childhood lead toxicity and resulting healthcare costs, etc. Even tech-specific policies like Cures and TEFCA that improve interop will likely reduce admin overhead as CMS rolls out implementation details over the next few years. Until we see more policy movement and the implementation science (via academics or industry) to adopt and expand new system-level incentives and interventions, most newcos will, very reasonably, try to maximize adoption by operating within the tradeoffs of cost/quality/access rather than maximizing clinical value per dollar.

How long until this is a standard ask for health systems fielding agent vendors? cc @arieldora

If you’re starting an autonomy company today, public roads are the hardest place to begin. We sat down with @bsofman (CEO @BedrockRobotics) & @malharhar (Special Projects Head @AppliedInt) to discuss picking the right problem for physical AI.

AI has given venture capital a new way to repeat an old mistake: kingmaking. The pattern from 2021 is back: a category becomes "obvious," a top-tier firm anoints its winner, and everyone else acts like the decision is final. Sierra for support. Harvey for legal. Applied Compute for RL-as-a-service.

My latest in @Health_Affairs on the AI training data challenge health systems shouldn't ignore: Amidst AI labs moving towards model training on chat history with multi-year retention windows, ~2.6% of Claude & 5.7% of ChatGPT conversations contain clinical content, often as PHI within consumer tiers without BAAs. healthaffairs.org/content/fo…

AWS is down. Government is shut down. Epic is still up.

Love that AI labs are prioritizing biomedical research. Between Skills and MCP, the near-term future of computational biology might not necessarily be *better prediction models* but rather *better integration layers* that let general intelligence operate across the full research stack.

Excited for the imminent health system research/ops work on local 1) post-training (to improve hospital-specific clinical context), 2) benchmarking/uncertainty evaluation (to track population-specific accuracy/drift), and 3) continuous model monitoring and rapid on-prem patching (to minimize patient risk).