Reinforcement learning (RL) is concerned with learning to take actions to maximize rewards,by trial and error, in environments that can evolve in response to actions. AMarkov decision process (MDP) [6] is a popular framework to model decision making inRL environments.

In the MDP, starting from an initial observed state, an agent repeatedly(a) takes an action, (b) receives a reward, and (c) observes the next state of the MDP. The traditional objective in RL is a search goal - find a policy (a rule to select an actionfor each state) with high total reward using as few interactions with the environment aspossible, known as the sample complexity of RL problem [7].

This is, however, quite differentfrom the corresponding optimization goal, where the learner seeks to maximize thetotal reward earned from all its decisions, or equivalently, minimize the regret or shortfallin total reward compared to that of an optimal policy [8]. This objective is relevant inmany practical sequential decision-making settings in which every decision that is takencarries utility or value - recommendation systems (clicks by consumers translate into revenue),sequential investment and portfolio allocation (financial holdings make profits orlosses), dynamic resource allocation in communication systems scheduling decisions affectdata throughput), to name a few.

. .