Synesthesia is a neurological phenomenon in which stimulation of one sensory pathway automatically and involuntarily produces an experience in another. For some people, hearing a musical note may evoke a specific color or shape; for others, everyday sounds may trigger shifting patterns or textures.
Although often described as unusual, synesthesia is not rare. Current estimates suggest that roughly 3–5 percent of the population experience some form of it, with women slightly more likely to report synesthetic perceptions than men.
Research groups, including the University of Sussex Synesthesia Research Group, have helped expand public understanding of this condition by cataloging its variations and documenting how individuals describe their internal experiences.
Types of Synesthetic Experiences
Visual forms of synesthesia are among the most researched. Within these, clinicians and researchers often distinguish between “projective” and “associative” experiences.
Projective synesthesia refers to perceptions that appear as if externally located—visuals that seem to occupy space in front of the eyes.
Associative synesthesia describes experiences that occur internally, within the “mind’s eye,” without appearing to exist in the external environment.
While this distinction is helpful conceptually, it remains difficult to capture synesthetic experiences accurately. Historically, researchers have relied on verbal descriptions or collaborations with illustrators and animators to recreate participants’ accounts. Though valuable, these methods require interpretation, can be resource-intensive, and may distance the final product from the individual’s lived sensory experience.
VR as a New Tool for Direct Representation
To address these limitations, researchers have begun exploring virtual reality (VR) drawing as a more immediate way for synesthetes to illustrate their perceptual world. Building on previous work using VR tools to help autistic individuals communicate sensory experiences, the current project invited a small group of audio–visual synesthetes to use OpenBrush, a VR drawing platform. Participants were asked to visualize what they perceive when hearing common environmental sounds—such as birdsong, rainfall, or passing cars—in a three-dimensional virtual environment.
The VR medium allowed users to select animated brushes, move freely in digital space, and construct dynamic forms that more closely resembled their personal experiences. The process was repeated after a delay to assess consistency, an important consideration in synesthesia research.
Participant Responses and Early Observations
Although the sample size was small, responses were notable. Several participants described the sessions as the first time they had been able to accurately express their perceptions. Many asked for recordings of their VR drawings to share with others, stating that verbal explanations had previously felt incomplete.
The study also revealed striking individual differences in spatial orientation. Some participants drew their perceptions as if on a flat screen in front of them, while others created visuals beneath, around, or even behind themselves. These variations may offer new insights into distinctions between projective and associative synesthesia, particularly regarding how people locate sensory experiences in real or imagined space.
Implications for Research and Communication
VR offers more than a novel artistic medium—it shifts control from researcher to participant. Instead of relying on intermediaries to interpret descriptions, synesthetes can generate their own representations in real time, preserving nuance and complexity. This approach may help researchers better understand perceptual variability while giving individuals a clearer way to communicate their experiences to others.
Although these findings are preliminary, they highlight VR’s potential as a method for studying synesthesia and fostering more precise, embodied communication of sensory experiences.
Related topics
