Abstract

Machine learning practitioners often face significant challenges in formally integrating their prior knowledge and beliefs into predictive models. Our goal is to build prediction models that can process numerical data and make probabilistic predictions, guided by natural language text which describes a user’s prior knowledge. Large Language Models (LLMs) provide a useful starting point for designing such a tool since they prove 1) an interface where users can incorporate expert insights in natural language and 2) an opportunity for leveraging latent problem-relevant knowledge encoded in LLMs that users may not have themselves. We show how LLMs can compute joint posterior predictive distributions over an arbitrary number of outputs that may be numeric or categorical in settings such as time series forecasting, multi-dimensional regression, black-box optimization, image modeling, and tabular data. Finally, we demonstrate the ability to usefully incorporate text into numerical predictions, showing how the text influences the predictive distribution and improves predictive performance.

References: LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language https://arxiv.org/pdf/2405.12856 JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs https://arxiv.org/pdf/2502.11877