OpenEvidence is the most widely used AI-powered medical search, helping doctors access the world's knowledge at the moment it matters.
Joined November 2022
- Tweets 403
- Following 363
- Followers 34,616
- Likes 1,144
73 Photos and videos
Pinned Tweet
May 13
“We did the hardest thing in the history of American health care. We got the majority of American doctors to all voluntarily adopt a single technology platform.”
NBC News on how that happened, what U.S. physicians actually do with OpenEvidence, and how partnerships with NEJM, JAMA, NCCN, and Wiley make it possible.
22
52
221
147,872
OpenEvidence retweeted
Rigorous evaluation of medical AI is good for everyone, and we welcome it. Counter to a half-dozen independent studies from institutions such as the Mayo Clinic that were highly positive on OpenEvidence—a lone paper now purports to show that generalized AI beats specialized clinical AI (@UpToDate, @EvidenceOpen). The paper has a massive undisclosed conflict of interest and irredeemable methodological flaws.
Behind the scenes: The study authors run a competing in-house medical AI at their hospital, and asked OpenEvidence for an API to power it — including rights to build a "competing product" with OpenEvidence's own API. OpenEvidence declined. Then, this paper coincidentally appeared.
Point-by-point, looking closely at the datasets used in the study, the disingenuous and fatal flaws become immediately apparent 🧵.
16
54
247
60,490
Third: Public peer review records indicate that the two flawed datasets above were the only evaluations included in the initial submission. After peer reviewers pointed out that results “lack[ed] epistemic grounding,” the RCQ dataset was subsequently added.
The authors do not make the RCQ dataset public, nor do they provide even basic question characteristics or the process they used to create the dataset. They likewise exclude the selection process for the reviewers themselves.
Without more information it is challenging to speak in detail, but their obvious conflict of interest and inattention to basic data contamination concerns do not inspire confidence.
1
1
19
2,914
This paper and its benchmarks are clearly ill-equipped to judge clinical decision support tools or draw conclusions about specialized clinical AI.
So how should we actually evaluate these systems? Certainly not by boiling the answer down to a contaminated multiple choice quiz, or by measuring adherence to an arbitrary style guide written by one of the model creators.
Here are a few starting ideas: the right evaluation should match the realities and distribution of real-world use of clinical AI. It should engage holistically with real clinical use, end to end. And ultimately, it should measure real clinical impact.
Let's figure out the right way to do this in good faith.
1
29
2,840
First, contamination: Here’s what happens when you plug common benchmark questions into other LLMs–they have seen the questions–and answers!
1
3
21
3,489
Second, misrepresented metrics: HealthBench, created by OpenAI, scores responses largely based on arbitrary/subjective stylistic choices. Here is an example of OE scoring 20% "worse" because it didn’t use a specific email header.
1
2
25
3,362
OpenEvidence retweeted
40% of U.S. doctors are already using AI to make real-time clinical decisions. The catch? Most of these tools learned medicine from the open internet.
Stephen D. McLeod, MD, CEO of the American Academy of Ophthalmology, makes the case for something better: an AI built on eye care's own gold-standard guidance.
A clear-eyed take on where these tools help, where they fail, and what comes next.
Read: aao.org/eyenet/article/the-M…
#AAO #AIMedicine #EyeCare #EyeNetMag #Ophthalmology
1
4
18
2,264
OpenEvidence retweeted
@EvidenceOpen integrates @ASCO guidelines, figures, and flowcharts into specialty oncology model
cancerletter.com/in-brief/20…
1
4
982
Jun 10
RT @JReinerMD: The Open Evidence clinic encounter tool creates a note that’s written as if my scribe was me, but me with unlimited time. It…
8
OpenEvidence retweeted
We're proud to announce our partnership with @EvidenceOpen.
OpenEvidence is the most widely used AI-based clinical decision support tool across the United States. Under this partnership, the Academy will share its Preferred Practice Patterns® and other clinical guidelines with OpenEvidence for incorporation into its training libraries and report on output for quality and consistency.
We encourage our members to share their experiences with the OpenEvidence tool, including observations about performance and recommendations for improvements, using a feedback form that is available from the Academy’s OpenEvidence page.
Read: aao.org/openevidence
#Ophthalmology #EyeCare #MedicalAI #EvidenceBasedMedicine #AAO #OpenEvidence
4
18
2,012
Jun 9
Ophthalmology is a field where the right answer lives three layers down. Not “treat glaucoma,” but which target pressure, for which optic nerve, given which visual field trend, in a patient who can’t tolerate first-line drops.
Starting today, the American Academy of Ophthalmology’s Preferred Practice Patterns are integrated into OpenEvidence. Decades of subspecialty judgment from the world’s largest eye physician and surgeon association, now cited and searchable alongside peer-reviewed literature.
When an ophthalmologist asks a question at the point of care, the @aao_ophth authoritative guidance is right there in the answer, every claim cited back to the source.
2
1
62
623,039
OpenEvidence retweeted
Jun 9
AI is not replacing doctors.
It is helping us catch what might otherwise be missed and make better decisions, faster.
Tools like this are already saving and improving lives.
Jun 8
Last summer an ER doctor at Mount Sinai watched two med students and a resident pull up OpenEvidence mid-shift on a hard case. He assumed they were unusual. Then the health system found that a third of its 9,000 physicians were already using it.
That's how OpenEvidence spread. Doctors found it, tried it on real cases, and told the doctors next to them. The hospitals are now formalizing what their clinicians already do.
@SteveLohr piece in today’s New York Times follows that thread out to a small hospital in Alaska, where Dr. Barbara Creighton uses it on complex cases she’d otherwise send to a paid specialist consult. She calls it “like having a bunch of specialists in your pocket.”
The hype around medical AI usually skips the part doctors care about most, which is what the tool is for. Daniel Nadler summed it up in one line: “It’s not an oracle, it’s a tool. Knowledge and knowledge workers still matter.”
8
5
106
17,123
OpenEvidence retweeted
Jun 8
"It’s not an oracle, it’s a tool. . . . Knowledge and knowledge workers still matter."- Daniel Nadler, chief executive of OpenEvidence
nytimes.com/2026/06/08/busin…
1
2
6
2,055
Jun 8
Last summer an ER doctor at Mount Sinai watched two med students and a resident pull up OpenEvidence mid-shift on a hard case. He assumed they were unusual. Then the health system found that a third of its 9,000 physicians were already using it.
That's how OpenEvidence spread. Doctors found it, tried it on real cases, and told the doctors next to them. The hospitals are now formalizing what their clinicians already do.
@SteveLohr piece in today’s New York Times follows that thread out to a small hospital in Alaska, where Dr. Barbara Creighton uses it on complex cases she’d otherwise send to a paid specialist consult. She calls it “like having a bunch of specialists in your pocket.”
The hype around medical AI usually skips the part doctors care about most, which is what the tool is for. Daniel Nadler summed it up in one line: “It’s not an oracle, it’s a tool. Knowledge and knowledge workers still matter.”
5
20
160
802,856
OpenEvidence retweeted
👇 Drs. @AmiBhattMD & @ankeetbhatt explore the impact of large language models (LLM) on CV care, share a practical framework for navigating LLMs, highlight important components to engineer an effective clinical prompt, & more. bit.ly/3PMYapk
#ACCInnovation @EvidenceOpen
3
7
17
4,028
Jun 4
The ADA Standards of Care are now in OpenEvidence. See it live at @AmDiabetesAssn Scientific Sessions this weekend.
1
1
22
365,517
Jun 4
The second Cleveland Clinic AI Summit is on August 28, something to look forward to when summer ends.
Hosted by @jamecancerdoc, with speakers from Stanford, Google Health, Mayo, Harvard, and major health systems. Dr. Travis Zack, OpenEvidence CMO, will be speaking.
We are hosting the second @ClevelandClinic Artificial Intelligence Healthcare Conference for all Healthcare team members - from Medical Students to Executives - August 28th @ClevelandClinic. Please see the all star speakers list from @Google to @Stanford ccfcme.org/AISummit
3
23
3,307