Healthcare
Microsoft diagnostic AI outperforms generalist physicians in benchmark
A Science study reported that Microsoft’s Diagnostic Orchestrator outperformed generalist physicians on a controlled text-based diagnostic benchmark.
Microsoft’s Diagnostic Orchestrator reportedly achieved 83.2% accuracy on complex New England Journal of Medicine cases, compared with 36.1% for generalist physicians. On emergency cases, the system running OpenAI’s GPT-5 achieved 80.3% accuracy versus 46.8% for physicians. The system is designed to simulate a panel of physicians rather than operate as a single conventional chatbot. The reported study was published in Science on October 22. The findings point to a possible role for AI in diagnostic reasoning and support for difficult cases. However, the source emphasizes that the evaluation was a controlled, text-only benchmark comparing the system with individual generalists. That means the results do not by themselves demonstrate performance in hospitals, with real patients, imaging, physical examinations, or complete electronic health records. The comparison also does not establish that the tool should make unsupervised clinical decisions. The development is significant because it tests a general-purpose AI system against challenging published cases and reports a large performance gap in that benchmark. Clinical deployment, safety, workflow integration, and regulatory status remain separate questions not answered by the report.
Original source: Becker's Hospital Review