The electoral success of right-wing populist parties is often attributed to disaffection among certain voters. But while economic explanations for this disaffection are theoretically clear and quantifiable, explanations centered on cultural factors offer more vague accounts that are harder to evaluate empirically. We address this problem by theoretically distinguishing between five different ``storylines’‘ about the cultural origins of populism, and then test them using extensive data from Europe and the United States. Our analysis indicates that concerns about ethno-cultural change induced by immigration are central to understanding the populist vote; so, but to a lesser extent, is rural resentment. In contrast, explanations centered on communal disintegration, declining social status or an intergenerational values divide are pertinent in only specific cases. The analysis helps disentangle the cultural forces associated with the rise of populism and highlights the heterogeneous coalitions that form the populist base across different countries.