Abstract

Large Language Models (LLMs) often sound certain when they are wrong. With rapid adoption in critical sectors like healthcare, scientific discovery and research automation, where the consequences of errors can be substantial, reliable Uncertainty Quantification (UQ) becomes crucial for safe and trustworthy deployment. In this talk, we will first review the current landscape of UQ methods for LLMs and discuss the trade-offs between calibration, discrimination, and compute. We’ll then look at how generation temperature (the scalar that controls generative diversity) affects calibration at a semantic level. We’ll introduce a semantic calibration framework and show that simple post-hoc temperature scaling markedly improves meaning-level calibration and selective prediction across open- and closed-book question-answering tasks.