AI and generalization · 2026 · Accepted at NeurIPS

What makes AI extrapolate?

Models that fit the same training data can behave very differently outside it. This paper examines how feature representations encode assumptions about the process that generated the data, and why those assumptions matter for extrapolation. Correct features alone are insufficient: the model must express the target, and training data must cover the relevant representation space.

Conference record Source version

Comparison of raw-space extrapolation with Fourier features that map training and unseen inputs onto the same circle. — For the periodic example, Fourier features map training and unseen inputs onto a shared circle. Extracted from Figure 1.
Aguilar et al., arXiv:2605.07483v2, Figure 1. Cropped from PDF page 2. Source · CC BY 4.0. Cropped the original PDF around the complete figure; rasterized without changing scientific content.
Open full-size figure

Accepted paper

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin

Advances in Neural Information Processing Systems (NeurIPS) · 2026 · Accepted for poster presentation

Summary based on arXiv:2605.07483v2, PDF pages 1, 2.