Our perception is often based on ambiguous and unstable sensory input. Yet, we exhibit a remarkable capacity for making successful perceptual decisions. How do our brains achieve our accurate and stable perception? I will show that the human brain exploits the temporal stability of the world to predict the future and facilitate perception in two distinct ways. In line with my findings in humans, I will show that the mouse visual cortex maintains long-term traces of individual past visual experiences, enabling the visual system to build up a statistical representation of the world to optimize the encoding of new information.