Construction & Engineering
AI models score as low as 2% on construction takeoff test
Togal.AI’s benchmark found large general-purpose AI models performed poorly on construction takeoffs from drawings, scoring between 2% and 34%.
A new benchmark from Togal.AI found that general-purpose AI models scored between 2% and 34% on construction takeoff tasks. The benchmark, called Construction’s Last Exam, contains 33 tasks requiring systems to interpret drawings, identify construction elements and produce measurements used for estimating and takeoffs. The tested models included API-accessible systems from OpenAI, Anthropic, Google, xAI and selected Chinese providers. Models received raw PDF drawings and could use built-in vision and coding capabilities, but not third-party tools or agentic workflows. Togal said its own machine-vision models were trained on tens of millions of construction plans. The results indicate that general-purpose models still have difficulty interpreting technical drawings accurately enough for a core estimating workflow. This matters because takeoffs influence pricing, procurement and project planning, and errors can produce costly downstream consequences. The benchmark also suggests that construction-specific training data and purpose-built systems may remain important even as general AI models improve.
Original source: Construction Briefing