This paper proposes a novel finite-state Markov chain approximation method for Markov processes with continuous support. The method can be used for both uni- and multivariate processes, as well as non-stationary processes such as those with a life-cycle component.The method is based on minimizing the information loss between a misspecified approximating model (a Hidden Markov Model) and the true data-generating process. We prove that and find conditions under which this information loss can be made arbitrarily small if enough grid points are used. In contrast to existing methods, the method provides both an optimal grid and transition probability matrix. The method outperforms existing methods in several dimensions, including parsimoniousness. We compare the performance of our method to existing methods through the lens of an asset-pricing model, and a life-cycle consumption-savings model. We find the choice of the discretization method matters for the accuracy of the model solutions, the welfare costs of risk, and the amount of wealth inequality a life-cycle model can generate