The human brain stores an extraordinary amount of knowledge about the world, supporting object recognition, language, reasoning, and abstract thought. What is the neural nature of this knowledge? Is semantic knowledge—such as “roses are red”—simply a trace of sensory experience, encoded in perceptual brain systems? How does the brain represent knowledge acquired through language alone, without direct perceptual grounding? I will present a series of studies from my laboratory that address these questions by examining semantic representations under radically different sensory and linguistic conditions. Using knowledge domains such as object color, we study congenitally blind individuals, color-blind individuals, typically developed macaques, and individuals who experienced early language deprivation. Across these populations, we ask what aspects of semantic knowledge depend on perception, what can arise independently of it, and how language reshapes neural representations. Together, the findings suggest that semantic knowledge in the human brain is supported by two distinct yet interacting coding systems, revealing how perceptual experience and language jointly contribute to the architecture of human knowledge.