Categories of reward functions.

The reward functions we consider can be classified into seven categories. First is ‘Objective only’, where the reward function depends only on the first component, w1. Similarly, ‘Expect only’ is the function that depends only on the second component, w2, and ‘Compare only’ is the function that depends solely on the third component, w3. Then, we have the functions that are a combination of two components—‘Objective+Expect’, ‘Objective+Compare’, and ‘Expect+Compare’. Finally, we have the reward function, ‘All’, that depends on all three components.