Updated

A large language model produced a hallucinated intelligence assessment about supposed Chinese nuclear components that nearly triggered a US military operation before human review caught the error. Researchers, including a GovAI scholar, warned that service members often do not understand the inherent uncertainty of LLM outputs, especially as the military accelerates its adoption of AI tools for intelligence and operations. The incident has raised alarms about the risks of integrating generative AI into high-stakes military decision-making without adequate safeguards or training on the technology's limitations. It comes amid a broader push by the Pentagon and allied agencies to deploy AI more widely, from intelligence analysis to autonomous drone targeting, intensifying debate over how much trust should be placed in unverified AI-generated outputs in life-or-death situations.