Cognitive maps refer to internal representations of spatial or non-spatial relationships between physical locations, objects, people, and events in the world that afford behavioral flexibility. In my talk I will present a series of studies showing that the hippocampus (HC), entorhinal cortex (EC), and orbitofrontal/ventromedial prefrontal cortex (OFC/vmPFC) construct unitary cognitive maps of abstract relationships sampled piecemeal. We further show that novel direct inferences made over abstracted cognitive maps use a grid-like code in EC and medial PFC (mPFC), among other regions, when they are composed on the fly during decision making. These findings suggest that grid-like representations are used by the human brain to infer direct shortcuts, even in discrete, non-spatial decision problems. In a second study we examined the relationship between cognitive control and cognitive map geometry by conducting parallel analyses of fMRI data and hidden layers of a recurrent neural network (RNN) model trained to perform the same task. We find both 2D map-like representations in a HC, EC, and OFC network and simultaneous 1D orthogonal representations of only task-relevant dimensions, with irrelevant dimensions compressed, in a frontoparietal network, and the RNN, supporting representational stability for generalization and flexibility for current behavior, respectively. We further find that increasing control demands due to incongruence (conflicting responses) between current task-relevant and irrelevant dimensions produces warping along the context-invariant axis in subjective representations, and the degree of warping further accounts for individual differences in cognitive control. Finally, a third study decouples the abstract position in the cognitive map from its contents, and reveals highly flexible, context-dependent coding in the EC-HC-mPFC network, and an abstraction hierarchy amongst these regions, with EC showing the most abstract coding. Collectively, these studies show how task demands sculpt the subjective map’s representational geometry and how this geometry effectively balances context-invariant representations ideal for generalization with context-specific representations ideal for the particularities of the task at hand.