Decision-making tasks in healthcare settings use methods that make a number of assumptions that we know are violated in clinical data. For example, clinicians do not always act optimally; clinicians are more or less aggressive in treating patients; clinicians have biases; and patients have (often unobserved) conditions that lead to differential response to interventions. In this talk, and following in Florence Nightingale’s path, I will walk through a handful of these violated assumptions and discuss statistical reinforcement learning and inverse reinforcement learning methods to address these violated assumptions. I will show on a number of scenarios, including sepsis treatment and electrolyte repletion, that these methods that have more flexible assumptions than existing methods lead to substantial improvements in decision-making tasks in clinical settings, reducing bias and leading to improved clinical outcomes.