Humans and animals are remarkable at attending to stimuli that predict rewards. While the underlying neural mechanisms are unknown, it has been shown that rewards influence plasticity of sensory representations in early sensory areas. Hence, top-down reward signals can modulate plasticity in local cortical microcircuits. However, synaptic changes require time, but rewards are usually limited in time. Because the two happen on different time scales, it is unclear how reward signals interact with long-term synaptic changes. We hypothesise that interneuron circuits, which recently emerged as key players during learning and memory, bridge the timescales. We investigate how temporary top-down modulation by rewards can interact with local excitatory and inhibitory plasticity to induce long-lasting changes in sensory circuitry. We construct rate-based and spiking models of layer 2/3 mouse visual cortex consisting of excitatory pyramidal neurons, and different interneuron populations, corresponding to somatostatin (SST)-positive, parvalbumin (PV)-positive and vasoactive intestinal peptide (VIP)-expressing interneuron types. We demonstrate how interneuron networks could store information about the rewarded stimulus to instruct (subsequent or more slowly emerging) long-term changes in excitatory connectivity in the absence of further reward.