OpenEvidence Beats ChatGPT on Drug Dosing, but Not Everyone Agrees

OpenEvidence answered long-acting antipsychotic dosing questions correctly 95% of the time in a new pharmacist-led study, more than double ChatGPT’s 40%. The medical AI platform now reaches more than 40% of America’s practicing physicians and recently logged a record 1 million clinical consultations in a single day. A separate, peer-reviewed benchmark tells a messier story.

Nature Medicine published findings weeks later showing general-purpose chatbots, including Google’s Gemini, beating OpenEvidence and other specialized clinical tools on broader medical tests. OpenEvidence’s public response to that paper turned into one of health care AI’s messiest fights this year.

OpenEvidence Now Reaches 4 in 10 U.S. Doctors

OpenEvidence built its reputation as a free search engine for clinicians, answering point-of-care questions with citations pulled from peer-reviewed literature. Full access requires verification against a doctor’s National Provider Identifier (NPI), the same number regulators use to track licensed prescribers.

The company added a tool that grades the strength of evidence behind every cited answer on July 10, 2026, calling it EvidenceGrade.

• 757,000+ verified clinicians use the platform regularly, according to OpenEvidence
• 200 million+ clinical consultations powered by the tool to date
• $12 billion valuation following a $250 million round in January, double the figure from three months earlier
• 20 million clinical conversations happening on the platform every month

Backers across OpenEvidence’s funding rounds over the past year include Thrive Capital, DST Global, Nvidia, Google’s venture arm GV and, notably, Mayo Clinic, a health system whose own physicians are among OpenEvidence’s potential users.

OpenEvidence’s own account of that record day, filed as a first for medical AI at this scale, said the tally counted only consultations with NPI-verified doctors, not casual or unverified use. The company is also an official partner of the JAMA Network, The New England Journal of Medicine, the National Comprehensive Cancer Network and Cochrane Systematic Reviews.

That scale has made OpenEvidence a fixture well beyond hospital attending physicians. Pharmacists, in particular, have folded it into a growing list of AI-assisted tasks.

Where Pharmacists Already Lean on AI

Pharmacists were using AI-driven tools before OpenEvidence existed, built into a range of tasks across hospital, retail and health-system settings. The applications go well beyond looking up a drug’s side-effect profile.

• Adverse drug reaction detection – scanning patient records and literature for emerging reaction signals
• Clinical decision support – surfacing relevant guidance at the moment a prescriber needs it
• Computerized prescriber order entry (CPOE) – flagging problems before a prescription is filled
• Dosing and interaction recommendations – suggesting doses and catching drug-to-drug interactions in real time
• Medication adherence monitoring – tracking whether patients are actually taking what they were prescribed
• Medication-error detection – catching mistakes before they reach a patient

Getting drug information right for a specific clinical decision is one narrower slice of that work, and it is where head-to-head testing between AI tools has actually happened. A search of published research turns up just two studies that measure OpenEvidence directly against its rivals on drug information accuracy.

OpenEvidence’s Dosing Accuracy More Than Doubles ChatGPT’s

The first study, led by researcher Ipema and colleagues, tested five AI chatbots, OpenEvidence, Clair, GlassHealth, DougallGPT and ChatGPT, against 30 questions pulled from real drug information requests.

Three pharmacists independently scored every response using CLEAR, a validated 25-point framework that measures completeness, lack of false information, evidence, appropriateness and relevance.

OpenEvidence posted the highest mean score, 17.82 out of 25. ChatGPT came second at 15.72.

A second study, from researchers Surbaugh and Moeller, tested four tools, ChatGPT, Microsoft Copilot Chat, Perplexity and OpenEvidence, on a narrower and riskier task: dosing recommendations for long-acting injectable antipsychotics, including what to do when a patient misses a scheduled dose. OpenEvidence got 95% of scenarios right, missing just one of the twenty tested. ChatGPT finished last at 40%.

Errors across every tool in both studies clustered in the same place: missed-dose scenarios, exactly the situation where a pharmacist has to reason through a delay rather than recite a standard schedule. Even OpenEvidence’s winning score in the Ipema study fell short of a basic bar. When the same 30 questions went to working drug information pharmacists, their answers outscored every chatbot tested, OpenEvidence included, by a statistically significant margin.

Is OpenEvidence More Accurate than ChatGPT?

The answer depends on which test is being run. OpenEvidence wins narrow, pharmacist-graded drug information contests against ChatGPT by wide margins, as the studies above show. But a benchmark published in Nature Medicine in June found the opposite pattern once questions move beyond pharmacy-specific tasks.

The study, titled “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks,” tested OpenEvidence and similar clinical tools against general-purpose models on broader medical knowledge questions. On MedQA, a standard bank of licensing-exam-style questions, Google’s Gemini scored 97.4% accuracy, ahead of 89.6% for OpenEvidence and 88.4% for the reference tool UpToDate. ChatGPT and Anthropic’s Claude also outscored the specialized clinical tools on the same benchmark, according to reporting on the findings.

A separate peer-reviewed comparison of OpenEvidence, ChatGPT and two other chatbots tested the tools against real clinical scenarios rather than exam-style questions, reaching a similarly mixed verdict. No single tool swept every category, and results shifted depending on the clinical subspecialty being tested.

The Rebuttal That Became a Twitter Firestorm

OpenEvidence did not respond to the Nature Medicine paper with a formal letter to the editor or a competing dataset. It ran a combative thread on X, alleging benchmark contamination, misrepresented metrics and an undisclosed conflict of interest, and it asked the journal to retract the study, apologize publicly and commission an independent review.

The company argued that public benchmarks like MedQA likely sit inside the training data of frontier general-purpose models already, inflating their scores through memorization rather than clinical reasoning. It also said the study’s authors ran a competing in-house medical AI tool and had previously sought, and been denied, access to OpenEvidence’s closed API.

Physicians on X were not impressed by the tone.

A twittorial? Ironically, you at @EvidenceOpen do not seem very ‘open’ to ‘evidence’ after all.

One physician posted that line as the backlash spread through healthcare social media in June, and it captured the broader reaction. Commentators called the episode a public relations blunder, noting that OpenEvidence still has zero peer-reviewed publications of its own, no published white paper, and a closed API it has declined to share with outside researchers.

What Concerns Do Physicians Still Raise?

Accuracy tops the list. One physician survey found 44% of respondents named accuracy and misinformation risk as their top concern with tools like OpenEvidence, 19% cited a lack of physician oversight or explainability, and 16% flagged legal or liability exposure. Those numbers predate the Nature Medicine dispute, which only sharpened the debate.

A separate pilot study posted to medRxiv tested whether OpenEvidence gives the same answer twice when asked complex subspecialty questions, a repeatability check that matters as much as raw accuracy once a tool sits inside a clinical workflow. Other researchers have separately flagged a lack of transparency in how OpenEvidence curates or excludes source articles, along with questions about how current its citations really are.

OpenEvidence Wants Pharmacists as Its Next Login

None of this has slowed OpenEvidence’s roadmap. The company’s stated 2026 priorities include deeper electronic health record integration through partners like Sutter Health and Epic, a clinical-trial-matching product called Open Vista built with Veeva, and a push into international, English-speaking markets it has not served before.

The company also wants to move past physicians entirely. Its next expansion targets nurses, advanced practice providers, pharmacists and medical students, groups that access the platform differently than attending physicians but face many of the same drug information questions the pharmacy studies above were built around. The American Diabetes Association’s December 2025 partnership with OpenEvidence, which feeds clinician feedback back into the group’s own treatment guidelines, is an early template for how a pharmacist-facing version might eventually work.

OpenEvidence’s rise mirrors a broader wave of capital chasing AI in medicine and drug development. Anthropic expanded its Claude Science workbench into drug discovery this year, and Insilico Medicine sealed a $600 million drug discovery deal with Takeda, a separate bet on AI-designed medicines. OpenEvidence’s bet is narrower: get inside the workflow of every clinician who touches a prescription, including the pharmacists and nurses who never write one.

A pharmacist-specific version has not been announced yet. Whatever OpenEvidence builds for that audience will land in front of a profession that just watched the tool win a narrow accuracy contest and lose a broader one in the same year.

Frequently Asked Questions

What Is OpenEvidence?

OpenEvidence is a free medical answer engine built specifically for verified clinicians, checking each user against their National Provider Identifier before granting full access. It draws on partnerships with the JAMA Network, The New England Journal of Medicine, the National Comprehensive Cancer Network and Cochrane Systematic Reviews to ground its answers in cited literature, setting it apart from general-purpose chatbots like ChatGPT.

Can Patients Use OpenEvidence Themselves?

Not in the way most consumer chatbots work. Full access requires National Provider Identifier verification, so the platform is built around licensed clinicians rather than the public. Most patients encounter its influence indirectly, through a doctor’s or pharmacist’s decisions, rather than by logging in themselves.

How Does OpenEvidence Make Money?

The platform is free for verified clinicians and earns revenue mainly through targeted pharmaceutical advertising placed alongside clinical answers. Private-market research firm Sacra estimates OpenEvidence has crossed a $100 million revenue run rate on the back of that ad model.

What Does EvidenceGrade Show Clinicians?

Launched July 10, 2026, EvidenceGrade is built on the GRADE framework, the same methodology behind Cochrane and World Health Organization guidance. It displays in real time how strong the published evidence is behind each cited answer, rather than presenting a single answer with no indication of how solid the underlying research is.

Will Pharmacists Get Their Own Version of OpenEvidence?

Nothing pharmacist-specific has launched yet, but OpenEvidence’s stated 2026 plans include expanding beyond physicians to a pool of roughly 5.2 million nurses, advanced practice providers, pharmacists and medical students. That would more than double the platform’s current user base if adoption comes anywhere close to what it achieved among doctors.

Disclaimer: This article is for informational purposes only and does not constitute medical advice. Decisions about drug dosing, prescribing or clinical care should always involve a licensed physician or pharmacist. Figures cited are accurate as of publication and are drawn from the studies, companies and researchers named above.