Specialist species thrive under specific environmental conditions in narrow geographic ranges and are widely recognized as heavily threatened by climate deregulation. Many might rely on both their potential to adapt and to disperse towards a refugium to avoid extinction. It is thus crucial to understand the influence of environmental conditions on the unfolding process of adaptation. I will present a PDE model of the eco-evolutionary dynamics of a specialist species in a two-patch environment with moving optima. The transmission of the adaptive trait across generations is modelled by a non-linear, non-local operator of sexual reproduction. In an asymptotic regime of small variance, I justify that the local trait distributions are well approximatted by Gaussian distributions with fixed variances, which allows to report the analysis on the closed system of moments. Thanks to a separation of time scales between ecology and evolution, I next derive a limit system of moments and analyse its stationary states. In particular, I identify the critical environmental speed for persistence, which reflects how both the existence of a refugium and the cost of dispersal impact extinction patterns. Additionally, the analysis provides key insights regarding the path towards this refugium. I show that there exists a critical environmental speed above which the species crosses a tipping point, resulting into an abrupt habitat switch from its native patch to the refugium. When selection for local adaptation is strong, this habitat switch passes through an evolutionary ‘‘death valley’’ that can promote extinction for lower environmental speeds than the critical one.