Machine-learned interatomic potentials enable realistic finite temperature calculations of complex materials properties with first-principles accuracy. It is not yet clear, however, how accurately they describe anharmonic properties, which are crucial for predicting the lattice thermal conductivity and phase transitions in solids and, thus, shape their technological applications. In this talk I will discuss a recently developed on-the-fly learning technique based on molecular dynamics and Bayesian inference, and I will show how it can be employed in order to generate accurate force fields that are capable to predict thermodynamic properties. For the paradigmatic example of zirconia, an important transition metal oxide, I will show that this machine-learned potential correctly captures the temperature-induced phase transitions below the melting point. I will further showcase the predictive power of the potential by calculating the heat transport on the basis of Green-Kubo theory, which allows to account for anharmonic effects to all orders. Finally, I will introduce a ∆-machine learning approach that allows to train interatomic potentials from beyond-density functional theory calculations at a greatly reduced computational cost. The results demonstrate that on-the-fly machine-learned interatomic potentials offer a routine solution for highly accurate and efficient simulations of the thermodynamic properties of solid-state systems.