The overlapping incidence of poverty and climate damages globally raises the question of whether and how poverty makes people more vulnerable to worsening weather shocks. To study this question, we overlay high-resolution, satellite-based drought and flood measures on household survey panel data from the randomized evaluation of a flagship anti-poverty graduation program in Bangladesh. The comparison of households that were equally poor before a randomly chosen group of them were given the graduation program reveals that the poorest bear the brunt of negative consumption impacts from unpredictable weather shocks. Those lifted out of poverty by the program do not reduce consumption and this protection is sustainable because it is achieved by diversifying labor activities rather than divesting assets. Programs that diversify income generating activities are thus a promising means of enhancing the climate resilience of the extreme poor and avoiding adding to their numbers in the future.