How Adaptive Transfer Kernels Boost Low‑Resource Regression: IEEE TPAMI Insights

Recently, the top AI journal IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI, impact factor 24.314) accepted a new study from the Volcano Speech team titled Adaptive Transfer Kernel Learning for Transfer Gaussian Process Regression .

The work focuses on innovative theoretical research for applying transfer learning to low‑resource regression problems, with contributions including:

Providing a formal mathematical definition of transfer kernels.

Proposing three generalized forms of transfer kernels that subsume existing variants as special cases.

Introducing two improved transfer kernels—linear product kernel and polynomial product kernel—and demonstrating their effectiveness in transfer learning, linking transfer performance to domain relatedness and showing efficient performance gains.

Background

Gaussian Process (GP) regression is a fundamental Bayesian machine‑learning model widely used in engineering and statistics, but it requires large amounts of labeled data. Transfer GP leverages data from different domains via a transfer kernel to reduce labeling costs by modeling domain relatedness.

Although various transfer kernels have appeared in fields such as computational engineering, geostatistics, and NLP, a formal definition has been lacking. This paper first defines transfer kernels mathematically and summarizes three generalized forms.

Principles and Definitions

The core contribution is the formal definition of a transfer kernel, illustrated below:

Based on this definition, three generalized kernels are proposed: Chain Generalized Kernel , Sum Generalized Kernel , and Product Generalized Kernel , each handling domain information differently.

Existing widely used transfer kernels are special cases of the product generalized kernel:

The original product kernel uses a single scalar to model domain relatedness, which limits its ability to capture complex heterogeneous data. To address this, the paper proposes an improved linear product kernel:

This kernel combines two sets of base kernels linearly to distinguish intra‑domain and inter‑domain computations, with separate linear coefficients ensuring the inter‑domain coefficient never exceeds the intra‑domain one, satisfying the positive‑semi‑definite requirement.

The paper presents Theorem 1 guaranteeing semi‑definiteness when the linear coefficients obey the stated inequality, and derives corollaries describing when source and target domains are unrelated, partially related, or fully related, guiding the amount of transfer.

To capture non‑linear relationships, a polynomial product kernel is introduced:

Its structure can be visualized as deep networks of base kernels alternating linear and product layers, enabling richer domain modeling without additional learnable parameters in product layers.

Experimental Validation

The authors conduct extensive experiments demonstrating that the proposed kernels accurately learn domain relatedness and achieve lower root‑mean‑square error (RMSE) than baselines on low‑resource regression tasks, including time‑series extrapolation and four real‑world datasets where the method outperforms eight state‑of‑the‑art approaches.

The Volcano Speech team, part of ByteDance AI Lab Speech & Audio, continues to deliver high‑quality speech AI technologies across ByteDance products and offers these capabilities via Volcano Engine to external enterprises.