Forecasting the course of an outbreak can inform public health planning and decision making. Accurate forecasts, however, are hampered by uncertainty about changes in human behaviour, pathogen genetics and environmental factors that are difficult to capture in real time. I will present a flexible framework based on Bayesian semi-mechanistic models that can be used to incorporate these uncertainties. I will show how we used this to produce regional forecasts during the West African Ebola epidemic of 2014-16, before discussing the applicability of our approach to a wider class of problems. I will conclude with an outlook on the potential for and challenges in using mathematical models to integrate different data sources for understanding and predicting the behaviour of infectious diseases.