NeurIPS · 2023

Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictions

Tongxin Li, Yiheng Lin, Shaolei Ren, Adam Wierman

Advances in Neural Information Processing Systems

Read paper BibTeX

Does the structure of advice improve robust decision making?

Q value predictions expose more information than an opaque policy recommendation. For a single trajectory MDP, the analysis characterizes consistency and robustness tradeoffs and shows how structured advice improves the guarantees available from a robust baseline.

Research themes

Connections

Algorithms guide learned decision makers

Q value advice and imperfect transition forecasts provide structured information for reinforcement learning. LEAD applies a related principle by supporting language agents with dueling bandit algorithms.

Open questions

Prediction quality as a decision resource

Which prediction errors actually matter for the decision being made?

Explore open directions

Cite this paper

Tongxin Li, Yiheng Lin, Shaolei Ren, Adam Wierman. Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictions. Advances in Neural Information Processing Systems, 2023. https://papers.nips.cc/paper_files/paper/2023/hash/8e806d3c56ed5f1dab85d601e13cbe38-Abstract-Conference.html

BibTeX
@inproceedings{tongxin-q-value-advice,
  title = {{Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictions}},
  author = {Tongxin Li and Yiheng Lin and Shaolei Ren and Adam Wierman},
  year = {2023},
  booktitle = {Advances in Neural Information Processing Systems},
  url = {https://papers.nips.cc/paper\_files/paper/2023/hash/8e806d3c56ed5f1dab85d601e13cbe38-Abstract-Conference.html}
}