Hearing Color Through A Cyborg

In the taxi line outside Raleigh-Durham International Airport this past May, Neil Harbisson was the only one with an antenna protruding from his head.

The self-proclaimed cyborg artist was en route to Moogfest, a festival of music and technology held in Durham, North Carolina. Harbisson was slated to perform a musical piece, enigmatically listed in the Moogfest program as “A Cyborg’s Synaesthetic Pedicure.” The show would feature his antenna—a device surgically implanted into his skull that enables him to sense color through sound.

“I hear different notes for different colors, so I ‘hear’ colors, from infrareds to ultraviolets,” Harbisson told the bemused Moogfest audience that afternoon.

Harbisson was born with achromatopsia, a rare form of colorblindness that causes him to see the world in grayscale. But since 2004, he’s experienced color as sound, first through a set of headphones, and shortly thereafter through subtle sonic vibrations transmitted through his skull.

Harbisson’s electronic third eye takes advantage of the fact that both colors and sounds can be represented as frequencies. The tip of Harbisson’s antenna registers light frequency, then transmits that information to a chip implanted in his skull. The chip transposes that light frequency down 39 octaves into the audible range and then vibrates, creating a color-specific tone inside the artist’s head. Harbisson says his current implant registers 360 separate color-tones, plus ones outside the range of normal human vision. Together, these notes make up a soundtrack inside his head that changes with every movement of his antenna.

On this particular afternoon, he set out to share a bit of that music with the Moogfest audience gathered inside Durham’s Arts Council PSI Theatre.

First, Harbisson’s artistic collaborator, the Catalan guitarist Pau Riba, got comfortable on a barstool onstage. Riba rested his conspicuously bare feet on the rung of the stool as Harbisson readied his own instrument: a series of bright nail polishes. What we were about to hear, Harbisson explained, was a “concert for nails, colors, and guitar.”

As Riba began to improvise on guitar, Harbisson steadied Riba’s naked foot, applying a coat of purple to the guitarist’s big toe. He then grasped the tip of his antenna, pointing it at the painted nail. A moment later, a note resounded from Harbisson’s laptop: a “D.” Additional nail polishes (and toes) fleshed out the improvisation, note by note.

(In a Q&A following the show, Harbisson explained the technical logistics behind the performance: His antenna, he said, is Bluetooth-enabled, allowing it to communicate with an audio program on his laptop that plays the desired tone once his electronic eye has detected it.)

Riba and Harbisson’s duet was easy on the ears—and that wasn’t by chance. Their harmony required more than one trip to the drug store, Harbisson admitted in the Q&A. “I had to find the nail polish that was tuned,” he said. “Most pop music and radio music, it’s just 12 notes, 12 colors all the time.” But because his antenna picks up hundreds of colors, the frequencies of the great majority fall somewhere in-between the 12 notes used in most Western music, leading to some unusual tones and cacophonic combinations. Finding the colors that hit the right notes for this performance entailed an extensive search. “It took us really a long time to find these shades,” he said. “They were very expensive, actually.”

A selection from Neil Harbisson and Pau Riba’s cyborg pedicure. Video by Annie Minoff

This afternoon’s whimsical pedicure was just one of many ways that Harbisson has attempted to share his unique perception of the world with audiences. In the past, he’s composed sound portraits of famous people, including Nicole Kidman and Al Gore, by transcribing the tones of their faces. He’s also created paintings based on the variations in pitches present in everything from pop songs (Justin Bieber’s “Baby”) to historic speeches (Martin Luther King’s “I Have a Dream”).

But sharing his singular worldview with audiences isn’t the essence of his art, Harbisson said. Instead, the “art” is designing his own sensory organs and perceptions of reality. “It’s like sculpting your brain or sculpting your body instead of a piece of stone,” he said.

As for Catalan guitarist Pau Riba (who’s uninterested in acquiring his own implants), collaborating with a cyborg musician has been a welcome opportunity to experiment. He described the Western musical scale using a Catalan expression Harbisson translated as “strict as church”—that is, inflexible. But now that color has entered his music? Said Riba, smiling, “I feel myself like a little child playing with toy instruments.”

*This article was updated on July 14, 2016. An earlier version misidentified the photographer who took the second image. He is Brian Livingstone, not Carlos Gonzales.