Balance of cortical excitation and inhibition (EI) is thought to be disrupted in several neuropsychiatric conditions, yet it is not clear how it is maintained in the healthy human brain. When EI balance is disturbed during learning and memory in animal models, it can be re-stabilised via formation of inhibitory replicas of newly-formed excitatory connections. I will talk about theoretical models that shows how such inhibitory re-balancing allows multiple memories to be stored in a stable fashion, and only released if EI balance is selectively disturbed. I will then (briefly) speak about some work using high field MRI and spectroscopy to obtain a macroscopic signature of memory processing. In this work we show evidence for selective inhibitory rebalancing of cortical memories in humans. By modulating the balance between excitation and inhibition using transcranial direct current stimulation, we show that the precise balance between excitation and inhibition can be selectively modulated to facilitate memory recall but is otherwise necessary to prevent interference between different memories. Finally, I will revisit the stabilising performance of inhibitory synaptic plasticity in recurrent cortical networks and introduce a class of cortical architectures with very strong and random excitatory recurrence that is stabilised by intricate, fine-tuned inhibition. I will show that excitation and inhibition in such networks dance with each other to transiently amplify specific activity states that can be used to reliably execute multidimensional movement patterns. The intriguing similarity to recent experimental observations along with tightly balanced excitation and inhibition, suggest inhibitory control of complex excitatory recurrence as a generic organisational principle in cortex.