Heteroscedastic Gaussian Process Q-learning for Sample Efficient Experimentation
- Published
- Source
- arXiv
- Paper number
- 1492
- Field
- Machine Learning
- arXiv ID
- 2611.21249
Key points
- We learn the Q-function directly with a Gaussian process (GP), which avoids learning the dynamics
- If each Bellman target is treated as a noisy measurement of the Q-function, the learning rate that minimises the mean squared error is the Kalman gain, and applying it to every observed transition is equivalent to GP regression.
- Heteroscedastic Gaussian Process Q-learning (HGPQ) sets the noise variance of each target with a heteroscedastic GP, which learns how unreliable the targets are across the states and inputs.
- we prove a high-probability bound on the error of the value estimate, uniform over the explored region.
- Pooled over five simulated benchmarks, which include a chemical reactor and a bioprocess, HGPQ approaches the best return found by any method in fewer episodes than GP, kernel and deep RL baselines.
Paper links
External research summaries. These are not HDATF publications or measured product results.