When LLM judges agree, should we believe them?
THE SO WHAT
The question of when to trust consensus among LLM judges goes straight at the foundation of automated evals and agent oversight. If you're leaning on model-as-judge for QA, ranking, or safety checks, treat this as a research dependency — not a solved problem — and keep a human-in-the-loop for any high-stakes decisions.
READ THE SOURCE
MORE FROM THE WIRE
Applied AIAnthropic debuts Claude for Financial Advisors, with connectors to investment analytics and wealth-management tools from BlackRock, Addepar, Schwab, and others
Verticalized copilots wired directly into custodians and portfolio systems turn AI from a chat toy into a workflow surface that can move real money. If you run a data or tooling layer in wealth or banking, assume the new integration requirement is “works natively with the advisor’s AI front-end,” not just the CRM.
Applied AIChina dismisses AI ‘fearmongering’ as spy chief warns of threat to Communist party rule
When China’s public line downplays AI ‘fearmongering’ while its spy chief warns AI could threaten party rule, you’re seeing the gap between external narrative and internal risk posture. Multinationals building in or with China should treat AI policy as a national security lever first, industrial policy second.
Applied AIAI Bosses Risk Clash With Trump Over Innovation, Safety
If the White House treats AI as a centerpiece of its economic agenda, public safety advocacy from lab leaders becomes a political act, not just a technical one. Boards and policy teams should model scenarios where regulatory direction is set more by electoral calculus than by expert consensus.
Applied AISources: OpenAI bought Glass Imaging, which is developing AI-powered smartphone camera tech, in a deal valuing it at $300M+; it was valued at ~$100M last year
Paying $300M+ for AI-first camera tech is a bet that perception quality is now a core input to model performance and user experience, not a handset-side afterthought. If your product depends on real-world capture — images, video, sensor data — expect model vendors to move closer to that hardware and compress your room to differentiate there.