How You See With Your Brain

DESSA: I’m Dessa, and this is Science Friday.

Have you ever had that experience of trying to take a picture of a spectacular moon that’s glowing to fill up half of the sky, and then you look at the photo and the moon looks like a tiny dumb ping-pong ball, and you want to march into the Apple store and demand to know why this pocket-sized device fails to capture the wonder of the cosmos properly? Well, the majesty of that supermoon might be as much in your head as it is in the sky.

Your brain does a lot more than just receive data reports from your eyes. Seeing involves a lot of interpretation, of which you’re totally unaware. Vision is, well, complicated.

Neuroscientist Dr. Cheryl Olman is working to better understand how the brain processes visual information using sophisticated fMRI techniques, including studying the brains of people with schizophrenia. She’s a professor in the Psychology Department at the University of Minnesota, based in Minneapolis, Minnesota.

Dr. Olman, welcome to Science Friday.

CHERYL OLMAN: Thank you so much. What a pleasure to be here and what a pleasure to talk to you.

DESSA: OK, I know that at the core of your work is the idea that you see with your brain, not your eyes. Can you explain that?

CHERYL OLMAN: I can explain part of it. And there’s about 10,000 of us studying that question, and we’ll eventually get the full answer to that. So that quote comes from Dr. Bach-y-Rita, who was one of the early developers of sensory substitution devices for folks who had lost their vision. And I didn’t realize for the first 15, 20 years of my career how important that was to me.

I thought vision was something that I could describe as a one-directional process– photons hit your eyeballs, the eyeballs send that information to your brain, your brain starts by detecting some edges, and it turns those edges into shapes and into objects and scenes, and it’s just this feedforward one-directional process.

And no, it just isn’t. Like, that does happen. And there are a few milliseconds during which the input is the best way to describe what’s happening in your brain.

But almost instantly, your brain– it’s big, and it’s been around for a while and it’s been living in this world, and you have had so much experience with how the visual world works, you’re carrying around these amazing internal models of the world and you’re relying on them super heavily– so I’m standing there looking at a tree. And the tree I’m seeing is mostly the tree that was created by my brain, anchored to reality by this input that’s coming in through my eyes. But what I see– that giant moon that you saw– is your brain telling you a story– the truest story it can come up with, but it’s your brain’s work, not your eyes.

DESSA: It feels, in some ways, like you describe the visual system like a cast iron pan, like, it’s seasoned by the world. It’s one of one. And it’s as much about the treatment it’s received and its experiences as it is about the object itself.

CHERYL OLMAN: Absolutely. And everybody’s brain is doing that interpretation differently, which makes it challenging sometimes to do research.

DESSA: And you studied physics before you became a neuroscientist. Did that totally change your approach to understanding vision?

CHERYL OLMAN: Absolutely. I think it’s the physicist in me that wanted that one-directional process to make sense– just little building blocks, making something more and more complicated, until we got the answer. And I had to do an about-face, as I started realizing, when I would be studying people’s brains and I would– in some cases, show them nothing– and then I would analyze the responses. And I knew that there was nothing coming in through their eyes, and I would get these responses that looked like vision. And it was because they expected to see something or they predicted that something was going to be on that part of the image, when it wasn’t there.

And I had to really have a little mid-career crisis there. Like, oh, yeah. So I’m going to have to study the feedback processes. Your memory and your thinking brain and your past experiences are feeding back to the earliest stages of visual processing and shaping what you see.

DESSA: And I know that some of your research has been about people with schizophrenia, and specifically how optical illusions might appear differently to that population. Describe how that has fit into your research.

CHERYL OLMAN: And so the first thing I need to say is I’m not a clinical psychologist. And so as I work to understand how vision is different in folks who are experiencing psychosis, I’m also madly trying to learn for myself what it means to experience psychosis.

We’ve all heard the term “schizophrenia.” I’m starting to use the term “psychosis.” And that’s because we’re trying to move the field– much bigger than me– is trying to move away from specific diagnoses to observable attributes. Schizophrenia, if we know anything about it, it’s that, if you’ve met one person with schizophrenia, you’ve met one person who’s been offered a diagnosis of schizophrenia.

It’s different for everybody. And it’s a collection of things. Some people might experience stronger reality distortions. Other people might experience more negative symptoms, like a withdrawal from the world and a difficulty experiencing pleasure. And all of that gets lumped under the category of schizophrenia.

So people have been trying for years to come up with more concrete ways of understanding exactly what’s going on for individuals. And vision is just a really accessible, relatively understood, concrete feeling kind of system. 40, 50 years ago, people noticed that individuals with schizophrenia, when they looked at some really popular size illusions– size is notoriously difficult for us to figure out– we know that when something’s close to my face, it’s really big, and when something’s far away from me, it’s going to look small. But I know it’s not actually small. I know it’s way over there and that’s why it looks small.

So your brain’s always doing this calculation– how far away from me is that thing really? And so we can trick your brain. That’s what the illusions are doing. They’re tricking your brain into changing the context, to change how big something looks.

And then they show these illusions to some folks who had been offered a diagnosis of schizophrenia, and they didn’t respond to the illusion the same way. And that started a cottage industry of showing visual illusions to folks early on with the diagnosis of schizophrenia, and then, more recently, with a diagnosis that includes an aspect of psychosis, some kind of a reality distortion– so that also might be folks with bipolar disorder– and looking at these illusions and seeing, does that look as much bigger or as much smaller as I typically get from response from a control population?

DESSA: And does it?

CHERYL OLMAN: Yeah, it does and it doesn’t. And so a super hardworking grad student in my lab, Victor– he just finished his written preliminary exam, doing a meta-analysis of the literature, and he pulled out 40-something papers that had all asked that question– are people with schizophrenia more or less susceptible to these visual illusions? And the answer, frustratingly, is, it depends. Part of it depends on maybe your sample size just wasn’t big enough and you got this one answer, but it’s not going to replicate. And others, it depends on the details of the illusion.

And those size illusions turned out not to be particularly robust or reliable. But the family of illusions that we’ve studied the most for the last 10 years are about contrast. It’s like the intensity of an image. When there’s lots of whites and blacks, it’s a high contrast image and it looks really intense. So your brain does everything it can to just– OK, everything’s intense; I’m just going to turn down the knob so my neurons don’t have to work so hard.

So there’s these gain control mechanisms. And we’ve spent a lot of time trying to study those. And those seem to be reliably different in folks who are experiencing schizophrenia. Exactly how, we’re still working on that. But the way that the neurons, the brain cells, talk to each other and tell each other– just calm down, there’s a lot of this going on, you don’t need to do so much– those mechanisms appear to be different in folks with psychosis and in folks with schizophrenia.

DESSA: And is there any reason to believe that this differential response, potentially, to optical illusions might have anything to do with the kind of hallucinations that people who are experiencing psychosis report?

CHERYL OLMAN: Maybe. A lot of people are asking that question. And there’s good reasons to think it could. So if I’m looking at a messy, noisy environment, and my brain, instead of turning down the knob and saying, oh, that’s a mess, just calm down for a second– if instead, my brain’s like, whoa, there’s a mess; there must be something here– where we started talking about the visual system creating so much of your reality for you– it’s well equipped to take a mess and make meaning out of it– and so if you have different regulation of the inputs, you could easily see how a brain with a person who’s experiencing schizophrenia could make up some stuff to go with that noisy input. And we could call that a hallucination.

That’s a theory. And we don’t know if that’s true. But that’s the theory a lot of us are chasing.

DESSA: So is it sort of like a post hoc account for why I’m perceiving the things I am? Is that what you’re saying might be happening?

CHERYL OLMAN: Yes. Could you please come write my papers for me? Yes.

[LAUGHTER]

DESSA: I want to learn more about how you’re actually investigating this stuff. And I know that fMRI technology is involved. So before we delve into your methods, could you just provide us with a quick and dirty reminder of what is an fMRI machine?

CHERYL OLMAN: Hmm. This is my favorite topic. There’s MRI scanners in the hospitals all over the place. The only thing that makes an MRI scanner an fMRI scanner, a functional MRI scanner, is the way we run it and the fact that we’re showing you different stimuli while you’re lying in it. So it’s just a regular old MRI scanner, tricked out with a projector so I can show you pictures, and a button box that has no metal in it that I can hand to you while you’re lying in the scanner so you can hit buttons to tell me what you saw.

And then, you’re lying in the scanner, and the scanner is going beep, beep, beep, beep, beep, beep, beep, as I get my images of your brain. And as you’re doing my task, your very, very hungry brain– like 20% of the energy that we consume goes to feeding our brains; they’re just wildly demanding, and they have a lot of blood flow and they consume a lot of oxygen– and as you do different tasks, that blood flow and that oxygen gets shifted over to the part of your brain you need the most to do that task.

And as you do a harder or an easier task, I’ll see bigger or smaller changes in that blood flow. And so that’s what I’m measuring with fMRI. It’s just taking pictures of your brain, but it subtly changes in intensity as the blood flow changes and I can see which part of the brain is working harder on which part of my task.

DESSA: OK. So when one of your research participants goes in the scanner, what kind of tasks are you asking them to do?

CHERYL OLMAN: All right. So when I’m putting people in the scanners, I am trying to study how they parse their world. And so the stimuli that I use are incredibly boring. They’re close-up photographs of a haystack or the side of a basket. And then I’ll put these two textures next to each other. And so I want to know how your brain can tell that this image is a transition from one thing to another. And that particular transition invokes all of the mechanisms I want to study– that the brain uses to regulate its environment and make meaning out of its environment and find things.

And so you lie in the scanner with your little button box. And then I show you different combinations of textures at different intensities for 20 minutes straight, until you just can’t take it anymore. And then I process those data to see if the changes evoked by those boundaries, by those contrasts, the two textures, are different in the brains of folks who have schizophrenia or just control participants.

And so far, I’m seeing tiny little differences. They’re subtle. But I like this task. And I think it’s telling us something interesting about how different brains process textures.

DESSA: I will notice that there is an element of masochism in literally displaying haystacks as you look for a needle. [LAUGHS]

CHERYL OLMAN: Yes. Yes, it takes a certain kind of person to want to do this for a living, right?

DESSA: And is it difficult? Are there limitations to that method itself? How helpful is that fMRI data to you as you’re trying to isolate small differences or zone in on particular neuroanatomical regions of the brain? How good is the tool?

CHERYL OLMAN: Oh, it’s great and it’s horrible. I can see into people’s heads with millimeter precision. And there’s just nothing more magical than looking at somebody’s brain and actually being able to see it. So I know where I am. I know which part of your visual system I’m studying. I know which part of the visual field– the projector screen you’re looking at– I can tell exactly which part is evoking the responses I’m measuring. That’s great.

What’s horrible is, even with my millimeter resolution, I’m trying to study neurons. And neurons are like 10 microns. So individual neurons are like 1/100th of a millimeter. And in my resolution element– we call them voxels; they’re like volume pixels– because when I’m making images of your brain, it’s like I’m slicing it up and looking at one slice at a time. And each slice looks like a picture– so that would be a pixel– but it’s in volume, so we call them voxels. And each voxel contains something like 100,000 neurons.

And I want to know what each one is doing. But I have one data point. That voxel either got brighter or darker while I was doing my experiment. And so that summarizes the mass action of 100,000 neurons. And that’s what’s horrible about functional MRI. Which my colleagues would be very angry if they heard me say that. So I hope they’re not listening.

DESSA: It’s Science Friday, Cheryl. I think the tea just got spilled.

CHERYL OLMAN: Oops.

[LAUGHTER]

But it’s so frustrating. It’s 100,000 things, and I got to tell them what each one of them is doing from a single number. It’s like knowing your neighbor’s politics based on who your state representative is. It’s so coarse.

DESSA: If you’re just joining us, I’m talking about vision and perception with neuroscientist Dr. Cheryl Olman, from the University of Minnesota. This is Science Friday, from WNYC Studios.

I know that part of your work now is cooperating and collaborating and contributing to the Human Connectome Project, right, which has a goal of better understanding in a really high resolution how the human brain works. Can you explain that?

CHERYL OLMAN: Yes. And that’s where our hope lies. I have a single voxel that represents these 100,000 neurons, one data point to summarize so much. But I’m not the only person getting those data points. We have lots and lots of data. We do lots and lots of scans. And then there’s our group– there’s 20 other groups– and data sharing is just mandatory now. And so all put together, we’re going to be able to figure this out.

And what the Connectome Project is is this massive initiative– they got the name kind of copying it after the genome, which was this big push to do something nobody thought you could do 20, 30 years ago, and now they’ve sequenced the human genome. And so the Connectome is kind of like that.

In your brain, there’s like 100 billion neurons. And that’s not the problem. There’s 100 billion of them, and each one of them has 1,000 connections. So the complexity of the system is overwhelming. And so the complexity lies in the connectivity between neurons, not in the individual neurons themselves because how this one fires depends on how 1,000 other neurons fired at the same time.

That connectivity is the target of the Connectome, and mapping hundreds– thousands of people’s brains– using dozens of different perspectives. Because building up a big enough database, we can start answering some of these questions.

DESSA: And they’re starting with a more manageable project, right? Are they working on animal brains now, before tackling the human brain?

CHERYL OLMAN: Yes. There’s this BRAIN Initiative. And the idea of seeing every single neuron in a brain, it’s currently inconceivable for humans. Now, I have a small imagination. I bet in 20 years, you’re going to be interviewing somebody who’s like, yes, we just finished mapping every single neuron in the human brain.

That’s not feasible yet. It’s getting feasible for mice. They have much smaller brains, but they’re still plenty complex. And then not only figuring out the identity or the class of every neuron in the brain, but then starting to look at who’s connected to whom, and mapping out that connectivity. That is a conceivable goal maybe in the next decade. And then science grows faster than we can ever imagine. From the mouse, they’ll move to non-human primates, which have smaller but very complex brains, and then on to humans.

DESSA: It’s actually really exciting to hear you say that science moves faster than we can imagine because I think so many of us are accustomed to the cautions, right? We won’t know for three more– we can’t say anything for certain for five more– and it’s exciting to know that, in addition to all the double checking, there’s some awesome innovations happening as well.

Thank you so much for the conversation today, Dr. Olman. This has been the highlight of mine.

CHERYL OLMAN: My pleasure, indeed. And the highlight of my day, again, to talk about these favorite things of mine.

DESSA: Dr. Cheryl Olman, professor in the Psychology Department at the University of Minnesota, based in Minneapolis, Minnesota.

Copyright © 2023 Science Friday Initiative. All rights reserved. Science Friday transcripts are produced on a tight deadline by 3Play Media. Fidelity to the original aired/published audio or video file might vary, and text might be updated or amended in the future. For the authoritative record of Science Friday’s programming, please visit the original aired/published recording. For terms of use and more information, visit our policies pages at http://www.sciencefriday.com/about/policies/