193: The Pathologist's Eye: Exploring Templates and Perception with Claudia Mello-Thoms

Understanding Visual Perception in Medical Diagnoses: A Deep Dive with Claudia Mello-Thoms

In this episode, Christine continues the conversation with Claudia Mello-Thoms, an expert in visual perception as it relates to medical diagnosis. The discussion delves into the use of templates by experts to quickly recognize abnormalities in medical images, despite their large size and complexity. Claudia explains how perception operates at a subconscious level, often leading to perceptual errors in medical practices like radiology and pathology. The conversation underscores the need to understand the human element in diagnostics to reduce errors and improve accuracy. The episode also explores the difference between rapid (fast) and deliberative (slow) processing in the assessment of medical images, emphasizing the importance of fresh perspectives when diagnostic clarity is lacking.

00:00 Introduction to Visual Perception in Medical Diagnoses

00:40 Understanding Templates in Medical Imaging

01:34 Expert vs. Novice Diagnoses

02:13 The Role of Clinical History in Pathology

06:48 Fast vs. Slow Processing in Diagnoses

09:42 Perception and Cognition in Medical Imaging

10:31 Teaching and Learning in Medical Imaging

21:48 Challenges and Future Directions in Radiology

23:09 Conclusion and Final Thoughts

Transcript

Christine Ko: [00:00:00] This is the second part of my conversation with Claudia Mello-Thoms, someone who has been researching visual perception as related to medical diagnoses and the images that we make those diagnoses from. It's been a fascinating conversation, so if you haven't listened to last episode, please do. We covered how experts versus novices make diagnoses. The other thing we talked about was the fact that experts must be comparing what they're seeing with a template because visual recognition is so fast.

I am gonna pick up the conversation here where we talk about those templates.

A microscopic image that's scanned is a digital microscopic image. The image is huge. It's a lot of information. You said it's like the amount of information of the size of a football field.

Claudia Mello-Thoms: Yeah.

Christine Ko: That's intense. That's a lot.

Claudia Mello-Thoms: The size of the image is the size of a football field. 

Christine Ko: An expert is still comparing that, even though it's one image, rather than a stack [00:01:00] of images for a CT scan, the pathologist is looking at that sort of globally and somehow predicting what they're gonna see. 

Claudia Mello-Thoms: They're looking at the low resolution image, and they can detect things in the low resolution image that do not comply with what they expect it to be in that low resolution image. From there they decide where they are going to investigate further. 

Christine Ko: So the template then, I guess if I put my dermatopathologist hat on, is normal, like what I would expect if there was no diagnosis in the slide, and so then I'm immediately picking up on the stuff that shouldn't be there?

Claudia Mello-Thoms: I think that's an excellent question. In radiology, for sure, the template is [00:02:00] normal, but in pathology, very rarely I would think you would get a biopsy or something normal. You need to differentiate from other types of disease. So I think that what happens in pathology is when you look at this slide, you know the clinical history of the patient. So it's not like these slides drop in your computer and you know nothing about them. So you know what the clinical question is. And so based on that clinical question, you load your template for that particular type of disease that can answer the clinical question. That's my feeling for pathology. I have never proved it. I've never studied it, [00:03:00] but that's how I feel.

Christine Ko: Yeah, I don't think that can be right. Only in the sense that most of the dermatopathologists that I know, including me, we look at a slide without looking at the clinical information first.

Claudia Mello-Thoms: Oh, okay. Okay.

Christine Ko: You can look at the clinical information first, but we, my colleagues who do it that way, think that it's better not to be "biased" by the clinical information. And it's always somewhat validating that the diagnosis that I come to is correct if I look at the slide first and think, Oh, it's a cancer, and then I look at the requisition form and they're like, rule out that particular cancer. And then I think, Oh, okay, it matches. But of course sometimes I look at the slide without having looked at the clinical history. I don't know, or I might just think, Oh, maybe it's this; maybe I'm thinking, Oh, maybe it could be one, two, or three. And then I look at the clinical information [00:04:00] and sometimes I can narrow it down. Sometimes I can't. Or sometimes I'll think, Oh, I hadn't thought of that. So then, exactly what you just said, then maybe I compare with a template and when things click, I'm like, Oh, it is that. I just hadn't thought of that. So maybe sometimes your theory is right.

But maybe, for a dermatopathologist, a lot of times for us it's small biopsy. So it's just one slide, one image. It's not like one image after another, successive stack of images that are building up on each other. Sometimes we have 20 slides for one thing, so you know, then it would build on itself. But one image, maybe the template that I'm comparing to is 10, 20, 30, who knows what number of most common things, like an alphabet maybe. When we are reading English, that's my first language, I'm comparing, A, B, C, D, E, F, all the way to Z; and I know certain letters go together. So I may be saying this incorrectly, but I've heard [00:05:00] that everyone who reads their native language and does it pretty well, we're chunking the words and the phrases even, and speed readers chunk like a whole paragraph. And so if I'm reading a new book, I'm not gonna know what it says, of course, but my mind is predicting that whole paragraph if I become good at speed reading by oh, this letter next to this, and this word next to this word. The sentence is gonna be, that is a dog. Even though it could be that is a cat, but somehow, in a paragraph, I put it together 'cause maybe the last paragraph had dog three times. So I know it's gonna be dog. I can chunk even a whole page if I teach myself how to do it. So I was thinking that's maybe what we do in dermatopathology or other pathology. I have this sort of alphabet or database in my head of what could be there, like a basal cell cancer or an actinic keratosis or a rash of some kind. [00:06:00] So expertise involves creating that alphabet or database in my head that's a huge template in a way, but it doesn't seem huge, just like once we learn to read, it doesn't seem as complicated as it is to like a three or 5-year-old who's learning it and thinks it's really difficult. 

Claudia Mello-Thoms: Yeah. Yeah. I think I would like to highlight two things you said that I think are very important. One is to read without looking at the clinical history first. That's what I tell the radiologists over and over because it really biases. The other thing is some people proposed that there are two ways that you can read an image. One is what's called the fast process. It's when you immediately recognize what's going on, and that's your template comparison. [00:07:00] And one is the slow process, which is when you don't recognize what's going on. So you have to then build your diagnostic assessment piece by piece, right? Most experts rely on the fast process. So in your field of expertise, you might need to have X amount of templates so you can look at something and immediately say, Oh yeah, it's one of this, or No, I don't really know what it is, I have to go and further investigate. Likely, this is what happens in dermatopathology. It does fit with the observations that we had in the experiments we run, in that maybe the cases that we showed to the cytopathologist weren't so [00:08:00] challenging that they needed to build that diagnosis looking for little pieces because we showed the same cases to the cyto technicians, and we couldn't use something that was too complicated. So maybe if we had used more unusual, rare cases we would have observed the process you described. But for the dermatopathology cases that we showed to the resident, we did observe that the ones that got it correct knew where to go. So maybe those had already started building their template and could match the image to a diagnosis. The ones that couldn't get to the correct diagnosis had to go everywhere.

Christine Ko: Yeah. I've read, if I'm [00:09:00] about to pick up a cup, the muscles of my hand already know the movement that needs to be done. So it's like the muscles have already predicted what they're gonna do when it's in my mind that I'm gonna pick the cup up. There's almost this sort of instantaneous or pre prediction of what needs to happen for me to pick it up. And so it's the same thing? Is it or not? 

Claudia Mello-Thoms: It's a little bit different in the sense that the motor cortex is closer to our cognitive stages, which are the stages that we are consciously aware of. The detection of lesions is a perceptual process and perception occurs in the subconscious level. So you can't explain. Because being subconscious, it's [00:10:00] below the language layer, so the minute you start explaining, you are just making stuff up. You jumped up a level to cognition and you are retrospectively justifying how you detected that thing, because you have no idea how you detected that thing.

Christine Ko: That's fascinating. I didn't know that. Perception is subconscious. Okay. I'm gonna have to think about that for a long time. When I was in training as a resident, my first dermatopathology teacher, his name is Dr. Ronald Barr, would put up a slide and he would just know it. And me, as someone who had no templates looking at it, I'd be like, I don't know how he knows that. I don't really know what he's seeing. He's a great teacher, so he would explain to us, Okay, I'm seeing this and this, and so slowly over time I would learn to put it in context. I wasn't thinking of it that way, but I [00:11:00] guess I was building templates. Every once in a while, he would try to explain, but he couldn't really put it into words, and he would say, Christine, sometimes it's just gestalt. And it was so frustrating to me, not him, but the answer. 'Cause I was like, Why? Like, Why is it gestalt? But when I look at a beautiful sunset or something, if I try to put into words what I'm feeling, it's inadequate, for one thing, and seems fake sometimes, like you're saying, like I'm making something up and it's not really what I am seeing or feeling. All of that is because perception is subconscious? 

Claudia Mello-Thoms: When computer-aided detection started being developed in the early two thousands, they would ask radiologists how they detected this finding, and radiologists would [00:12:00] give them a list of features that they use to detect that finding. Except that they haven't used any of those. So the computer tried to use those features to detect similar findings, and the poor computer had no chance because it was trying to replicate what the radiologist said they were doing when in reality they weren't doing it. 

Christine Ko: Like being sabotaged. A lot of what you've said resonates with me because when I look at one slide for a patient, as I've said, my practice is to not look at the clinical first. I will look at it, but there's a concept of linear sequential unmasking, which I think is true, and I think that's what we're doing. Maybe without knowing that term, linear, sequential unmasking.

To not be biased at first and just see what I come up with and then of course, put it into the context of everything. If I pick up a slide and look [00:13:00] at it, if I know what it is right away and then it matches when I look at the clinical, That's great. If I don't immediately know what it is, a lot of the times, I won't end up knowing what it is. 

Claudia Mello-Thoms: One thing that we've shown in the past was that for mammography, if a radiologist didn't look at the mammogram and knew within five, 10 seconds, whether the case was normal or it had an abnormality, the best thing to do was to put the case aside, read a bunch of other cases, then come back to that case, what we called fresh eyes. Because sometimes for whatever reason, your mind is not connecting to the case, and so it's better to put [00:14:00] it aside, look at other things, and give the case a new chance. I don't know if that would work in your field, but in radiology it does work.

Christine Ko: It does work for us. One of my colleagues, Dr. Tim McCalmont, he calls it the set aside where he'll just set it aside and look at it later or the next morning with, as you said, fresh eyes. A fresh mind, fresh eyes.

This may seem like a silly question to you, but how do you know that perception is subconscious? 

Claudia Mello-Thoms: There have been a lot of studies that use eye tracking, like I have an eye tracking lab here in Iowa. So what we do is we unobtrusively monitor the radiologists' eyes as they are inspecting different types of medical images, and [00:15:00] sometimes we ask them about what they are doing, and you can observe a disconnect between what they're actually doing and their explanations. So you can tell the explanation always comes after the fact. It's like their brain has to jump up a level, go into cognition, into language, and try to explain what they did five seconds ago when perception is already way ahead. There's been so many studies to assess perception, not only in medical imaging, but also in cognitive psychology, and nobody has been able to reach perception because it's a subconscious process. We can't find a [00:16:00] way to access it in the brain. We found tons of ways of accessing cognition. After like 50 years of research, we still cannot access perception. 

Christine Ko: Wow. Okay. So when you were talking before about the fast processing and the slow processing, Daniel Kahneman termed it System 1 and System 2. Others say Type 1 versus Type 2 processing. Type 1 and System 1 being fast processing; Type 2, System 2 being slow processing. Not that slow processing is necessarily slow, like an hour. Neither of these systems or types of thinking really exist, but it's like a model for how people think. Metacognition, thinking about our thinking, 

When I first read that I did think, just as you said about looking at a medical image or some other image, that there is fast and slow processing. Meta perception, thinking about our/ my perception, and that there's [00:17:00] same kind of thing like System 1 or Type 1 and System 2, Type 2, fast versus slow perception. But you're saying, Not really the same thing, because i'm still jumping up from my subconscious to use my cognition to try to explain what's going on. But it's still after the fact. 

Claudia Mello-Thoms: Exactly. Metacognition is a well established process. It's watching yourself think, but there is no way you can watch yourself perceive. By the time you were trying to do that, you'd have to be in certain words, a level above of that you are trying to monitor. 

Christine Ko: Wow. I just.... Dermatopathology always made me think, Why aren't I seeing it the way someone else is? But I've learned over the years, we're not really looking at the same thing a lot of the time. With my [00:18:00] colleagues who also have a lot of experience, we'll similarly have the same diagnosis essentially pop up into our heads, virtually at the same time. So in that sense, I'm like, Oh, we are looking at the same thing. But when I'm teaching and I'm trying to explain what to look at for different diagnoses, I've started to say things like, You are saying that as your diagnosis 'cause you're just looking at this, I think, but that's really the wrong place to look. Not that it's wrong that you looked there. Like there's no right and wrong about that, but it's wrong in the sense that you're probably never really gonna get to the diagnosis by looking there.

Claudia Mello-Thoms: But I have one question for you. When one of your students comes up with the wrong diagnosis because they are looking in the wrong spot, do think it would make difference if instead of you saying, Oh, this is wrong [00:19:00] because you are looking here, and this area doesn't have the correct findings, but instead asking them, Why do you think the right diagnosis is here? Which features do you see? Waiting for them to explain and then taking them to the correct area and saying, What do you see here? 

Christine Ko: Yes. Yes to everything you just said. So I will do that. Not always. I mean, it depends on what's going on in the day. It goes along with your idea of a template because medical students especially I've noticed, will look at the ink. We put ink on the bottom of a biopsy to help orient it and to have that tissue processed in the right way. So there is ink. Ink, we've all seen ink, in the end, I realized, Oh, they're looking at the ink because that's familiar to them, but I'll have ignored it. It actually took me a while at first to understand [00:20:00] what was going on. Because I would see medical student, not every single one of 'em, but a lot of 'em would be like, What's that blue thing? And I'd be like, What blue thing? I don't see a blue thing.

They'd be like, That, that right there at the edge, you're right on it. And I'm like, Oh, that's the ink, and the ink. Has no, no way you're ever gonna get to the diagnosis by looking at the ink. But then I realized, oh, they're looking at that because that's familiar to them. And then residents early on, we will tell them, Okay, go like top down, so go from the top of the skin down to the base where the fat is. So a lot of times, I guess, those are the templates they're building. From the top to the bottom. And so they'll be like, Oh, but I see something right there in the top layer, we call it the stratum corneum. And I'll say, Yes, that's fine. You're right. That's there, but you need to look over here. Then they'll know the diagnosis. Wow, Claudia, that's just fascinating.

Claudia Mello-Thoms: Yeah. I love this field because there are so many unanswered [00:21:00] questions. To make an analogy with what you were saying about the medical students, we have them come and do a rotation in radiology. And they look at everything that is bright. Like in the breast, like if it's a dense breast, there will be lots of bright things, but that doesn't necessarily mean that will help you in making a diagnosis. Novices, they're attracted by highly salient objects. So in our case, it's everything that is bright. It's just a characteristic of the visual system that if doesn't have any other information, it goes for the high saliency.

Christine Ko: That's fascinating. So given that perception is subconscious, and we don't really know how we recognize things, do you have an opinion on, [00:22:00] for someone like me, the job is to interpret medical images using my perception; is there a way to optimize what I do? In terms of like not making error. 

Claudia Mello-Thoms: This is a million dollar question because in radiology, we know that 60% of unreported cancers in both breast cancer screening and lung cancer screening are perceptual errors. Using eye tracking, we can see that the radiology size looked there, but they never perceived the finding as being cancer. If we could understand why, we could significantly reduce the number of misses, but until [00:23:00] we understand perception better, I don't know how to reduce the number of errors.

Christine Ko: Okay. Do you have any final thoughts? 

Claudia Mello-Thoms: In radiology, after 50 years of improving the technologies, the error rates have not gone down. So people are finally beginning to think that we need to understand the radiologist because it's not just the new machines and the resolution and the whatever. It's the human observer that, needs to be understood, and how they interact with the technology. The most important thing right now is to really better understand the physician.

Christine Ko: Thank you so much, Claudia. A really [00:24:00] fun, fun, fun conversation. 

Claudia Mello-Thoms: Oh, I had fun too. Thank you.