Midbrain dopamine neurons play important roles in learning, motivation and movements. It has long been thought that dopamine neurons broadcast a reward prediction error signal to drive learning to predict future outcomes. Recent studies have shown, however, that the signals sent by dopamine neurons are more diverse than previously thought. For instance, some dopamine neurons are activated by threatening stimuli but not by reward. It has been postulated that the activity of some dopamine neurons may be correlated with movement kinematics (e.g. speed) or the distance to a reward location (or motivational value) but not with reward prediction errors. These results have been taken as evidence that challenges the canonical view of dopamine signals based on reward prediction errors. In this talk, I will try to present a normative perspective on these diverse dopamine signals under the framework of the reinforcement learning theory.