In daily life, we expend much mental effort e.g., for performing office work. Earlier work showed that humans have the tendency to avoid mental effort as well as other effort. However, it remained elusive whether humans avoid purely mental effort in changing environment. Further, it also remained unclear what behavioral and neural mechanisms underlie such learned avoidance and whether the same mechanisms worked regardless of cognitive-demand types. We addressed these issues by developing novel selection tasks, where the association between option and cognitive demand level changed over time, with two variations of mental arithmetic and spatial reasoning problems. As a result, most of participants avoided options of higher mental effort, and the demand of the second preceding trials affected their current choices. We fitted these avoiding participant’s choice data by several mathematical models. The probabilistic selection model outperformed the prediction error model in terms of the likelihood, but the sequential effect did not be explained by the probabilistic selection model but prediction error model. Therefore, we conducted model-based fMRI based on the prediction error model, and revealed that activity in the dorsomedial frontal cortex / dorsal anterior cingulate cortex, and anterior middle frontal gyrus was positively correlated with the trial-by-trial expected cost for the chosen option commonly across the different types of cognitive demand. Further, we identified brain areas correlating with cost prediction error at time of answering the problem, which partially overlapped with or were adjacent to the correlates of expected cost at time of choice cue in the dorsomedial frontal cortex. These results suggest that human avoid mental effort adaptively, and represent expected cost and cost prediction error irrespective of cognitive-demand types.