Multi-Task Bayesian In-Context Learning: Transformers Adapt to New Priors
The ICML 2026 paper reframes in‑context learning as approximate Bayesian inference, introduces explicit prior datasets as a context prefix for Transformers, and demonstrates through synthetic and real‑world experiments that this multi‑task approach closely matches Bayesian oracles while offering fast, controllable inference.
