Variance Optimization for Continuous-Time Markov Decision Processes

• 作者: Yaqing Fu
• 来源期刊: 统计学期刊(英文)
• 学科: 数学
• 关键词: CONTINUOUS-TIME Markov Decision Process Variance OPTIMALITY of AVERAGE REWARD Optimal POLICY of Variance POLICY ITERATION
• 年，卷（期）: 2019,（2）
• 页码范围: 181-195
• 页数: 15页
• 分类号: O1

摘要

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.