Recent advances in generative models are precipitating a crisis of AI-enabled misinformation, motivating further technical solutions to automate the moderation of facts. Yet within these systems, notions of “knowledge” and “facts” are either not rigorously defined, or follow a naive realist model, which excludes the possibility of nuanced and diverse viewpoints. In this talk, I will present an alternative model of knowledge as a material-discursive interaction between a knowing agent and the (virtual or physical) world, and discuss my in-progress thinking on its implications for the practice of building and evaluating generative AI systems and the limitations of knowledge in machines.