Microsoft says its AI is more accurate than doctors at diagnosis: what a peer-reviewed Science study showed, and what it did not

Short answer: on October 22, 2026, the journal Science published a peer-reviewed study of Microsoft's Diagnostic Orchestrator system. On archival medical cases from The New England Journal of Medicine, the system reached the correct diagnosis in 83% of cases, against 36% for doctors using reference materials and 22% without them. On emergency department cases from Boston, the result was 80% against 47% for doctors. Microsoft AI head Mustafa Suleyman tells The Verge the tool diagnoses "up to four times more accurately" than doctors, and calls it a step toward "medical superintelligence." At the same time, the company and the study's co-authors state plainly that the system was tested only on past cases from archives, and how it would perform with live patients in an actual hospital is still unknown. Below I go through the numbers in detail and what conclusions they do and don't support.
What was published and how the test was built
On October 22, 2026 (a Thursday), Microsoft published the results of testing its Diagnostic Orchestrator system in the journal Science. This is not a blog post or a press release, but a paper that went through peer review by independent scientists before publication in one of the world's most respected scientific journals.
Diagnostic Orchestrator is built like a panel of doctors: several AI agents reason through a patient's symptoms and medical history, decide what tests to order, and arrive at a diagnosis, following the same process doctors use. The version of the system in this study runs on OpenAI's GPT-5 model.
Microsoft compared the system against doctors on two sets of cases that it compiled itself into a benchmark called SDBench. The company says it will make this benchmark public.
What the numbers say and what they mean
Cases from The New England Journal of Medicine. On hundreds of cases drawn from this medical journal, Diagnostic Orchestrator reached the correct diagnosis 83% of the time. Generalist physicians using reference materials got the diagnosis right 36% of the time, and without references, 22%.
Emergency department cases. On admissions data from the Beth Israel Deaconess Medical Center in Boston, the system scored 80% accuracy against an average of 47% for human physicians. The study also found the system typically reached its diagnosis at a lower estimated cost than the doctors did.
This is where Suleyman's "up to four times more accurate" claim comes from: 83 divided by 22 (doctors without references) gives almost 3.8 times. Compared with doctors who used reference materials (36%), the gap is smaller, about 2.3 times. The interview claim doesn't specify which group of doctors is being compared, which is worth keeping in mind when reading the phrase "four times more accurate than doctors."
The researchers also say Diagnostic Orchestrator outperformed leading AI models that solved the same cases working alone, without the multi-agent setup. Diagnostic Orchestrator was first developed when the most advanced model available was GPT-4.1, though the version that took part in this study runs on the newer GPT-5. Even newer models such as GPT-6 Astra, Gemini 4 Argon, or Claude Opus 5.5 were not included in the peer-reviewed paper.
What the test did not check
This is the most important thing to remember about these numbers: Diagnostic Orchestrator was tested on archival cases already described in medical literature and databases, not on live patients in a working hospital. Microsoft itself says this plainly.
Suleyman describes the next step this way: "It needs clinical validation. That is a leap, but this is a major milestone along that journey." Dr. Dominic King, Microsoft's AI health lead and a co-author of the study, told The Verge that the company plans to begin testing the system in real hospital conditions together with the same Beth Israel Deaconess Medical Center. Neither King nor Suleyman gave a firm timeline, though King said it would likely "get underway in the next couple of months."
Microsoft also hasn't said anything concrete yet about commercializing the technology: no timeline, no price, and no description of how the tool would actually reach doctors and clinics.
Where this fits into Microsoft's plans
Diagnosis, according to King, is just one part of a broader Microsoft plan the company calls "medical superintelligence." In a blog post and an article in the journal BMJ Digital Health & AI, both published the same day, the company describes five more areas it believes AI will need to master. Among them: coordinating clinical care, making treatment decisions, and handling the human side of medicine.
Microsoft already has a notable presence in US healthcare through its digital tools, and health, according to the company, is one of the most common topics people ask its consumer assistant Copilot about. King stresses that rollout needs to be responsible, given the high regulatory requirements in medicine.
Suleyman himself sounds optimistic: "Next year we're going to see extremely impressive deployments of intelligent systems in production. I think that this is just going to change the way care is delivered in a big way." At the same time, he insists human doctors should and will always play a central role in patient care. He names easing doctors' "drudgery" as his goal, so they have more time for patients, while acknowledging that AI could end up giving doctors more work rather than less.
What to check if you own a clinic or a medical service
- Which exact set of cases the accuracy figure comes from. 83% and 80% belong to two different datasets (journal case reports and emergency-room records), not one single overall metric.
- Who does it: you or your medical director.
- How to verify: read the Science paper itself (not just a summary) before quoting a specific percentage to patients or partners.
- Whether the system was tested on cases similar to your patient profile. NEJM case reports and the records of one Boston hospital aren't necessarily similar to what your clinic deals with.
- Who does it: the clinic's chief physician or department head.
- How to verify: ask a vendor of a similar tool for accuracy data specific to your disease profile and patient age group, not a general percentage from a marketing document.
- What "clinical validation passed" means in a contract. Microsoft itself says validation hasn't been done yet: the system was tested only on archival cases.
- Who does it: the clinic's lawyer together with the chief physician.
- How to verify: before signing a contract with any diagnostic AI vendor, get a document stating exactly which patients and under what conditions the tool was tested, not just the archive.
- Who is liable for a misdiagnosis by the tool. The source doesn't answer this: the material only contains Suleyman's general statements about doctors' central role, not a description of how liability is actually assigned for this specific product.
- Who does it: the clinic's director and lawyer.
- How to verify: before signing, get a written explanation from the vendor and your own lawyer of how liability is split between the clinic, the doctor, and the AI vendor in case of a misdiagnosis, and don't rely on general phrases like "the decision stays with the doctor."
What this news does not support
- That the system is already working in hospitals. Testing in real clinical settings, according to Microsoft, is only being planned; the company gave no firm date, and King says only that they will likely start in the next couple of months.
- That "four times more accurate than doctors" is a single, rigorous scientific comparison. It's a calculation from an interview, based on comparing against the weaker of two groups of doctors (those without reference materials).
- That the system has regulatory approval for clinical use. The material contains no mention of registration as a medical device, in the US or anywhere else.
- Specific timelines or pricing for a commercial product. Microsoft only says commercialization "could be on the horizon."
Bottom line
On October 22, 2026, Microsoft published a peer-reviewed study of its Diagnostic Orchestrator system in the journal Science: on archival medical cases, it diagnosed more accurately than doctors, 83% against 36% and 22% on journal case reports, 80% against 47% on emergency-room cases. This is a result that went through independent scientific peer review, not just the company's own words. But the key caveat remains: the test ran on archival cases, not on live patients in a working hospital, and Microsoft itself admits this. If you're considering a similar tool for your clinic or medical service, the main thing is not to confuse accuracy on an archive with readiness to work with real people, and to demand data specific to your own patient profile from any vendor.
I work on AI agents and automation. If you would like to see my projects or discuss your own task, take a look at my portfolio.
Sources
- The Verge, October 22, 2026: Microsoft says it's finally cracked AI for healthcare. https://www.theverge.com/ai-artificial-intelligence/1016690/suleyman-microsoft-diagnostic-orchestrator-science-medical-superintelligence
- Science, October 22, 2026: Diagnostic Orchestrator paper (DOI 10.1126/science.aeh7930). http://www.science.org/doi/10.1126/science.aeh7930