Recommended
More Related Content
More from Herbert Klein
More from Herbert Klein (12)
Recently uploaded
Recently uploaded (20)
Medical imaging meets psychology of perception: optical illusions!
- 1. Medical imaging meets psychology of perception: Optical illusions! Herbert A. Klein, M.D., Ph.D., Clinical Professor of Radiology, University of Pittsburgh, retired Medical imaging Brain science The assistance of Stacey McKenzie and Mark Lenkner in preparing the video images is gratefully acknowledged.
- 5. Next I will show you a rotating 3D display called a MIP, from a 3D bone scan of a patient with a lesion in one sacroiliac joint (among a number of abnormalities). Then I will ask you whether it is the right or left sacroiliac joint. I would not strain your brain about this, just form an impression without enduring great anxiety over whether it is correct or not.
- 7. What is your impression as to the side of the sacroiliac lesion? 1. Right SI? 2. Left SI? 3. Can’t tell. Please make a note of your answer.
- 8. PET scan of brain in epilepsy
- 9. We can think in terms of 2 interfaces between nuclear medicine and brain science (psychology, psychiatry, neuroscience, etc.): 1. Nuclear medicine (and other brain imaging modalities like MRI) can be of use in studying the brain (like the epilepsy example). 2. Brain science (e.g. psychology) can contribute to our understanding of how we perceive nuclear medicine images. This talk will emphasize the second point.
- 10. Kinds of medical imaging. No Radiation Magnetic resonance imaging (MRI) Ultrasound Radiation X-rays, including computed tomography (CT or CAT scans) (Externally produced x-rays go through the patient to a film or detector.) Nuclear medicine scans (A radioactive substance is given, usually by injection, and rays come from within the body.)
- 12. Miller, JeroldW WallisandTomR. "Volume rendering in three-dimensional display of SPECT images." J. Nucl. Med. 1990.
- 15. The point is you can’t tell anterior from posterior, but you can tell laterals, because the clues are so strong. There is a right answer, which I will reveal later, in terms of the real patient’s real lesion. But, considering the images in isolation, there is no right answer. If you think the lesion is on the left, that is equivalent to saying that you perceive the image as rotating clockwise as seen from above. If you think it is on the right, that is equivalent to counterclockwise. There is no clue in the images you saw as to which way they are rotating.
- 21. Troje and McAdam: • “Observers see it spinning more often clockwise than counter-clockwise. Here, we show that this rotational bias is in fact due to the visual system's preference for viewpoints from above rather than from below.” (The angle was varied in the experiment, altering the bias.) • “Since it was first published, Kayahara’s silhouette illusion has travelled the Internet quite extensively, and it appears in the context of all sorts of weird theories and applications (for instance, as a ‘test’ to distinguish ‘right-brainers’ from ‘left-brainers’).”
- 22. www.echalk.co.uk
- 23. Front convex Back concave
- 24. Klopfer showed 2D videos of rotating masks and other objects and his results supported differential familiarity as an important explanatory factor. For example, the illusion was weaker with upside-down faces. We see things according to our experience—convex, not concave faces. Schneider et al showed static stereoscopic views to normal and schizophrenic subjects. Normals experienced the illusion of depth inversion of concave faces significantly more often than did schizophrenics.
- 25. Frith and Dolan, 1997. Philos. Trans. R. Soc. Lnd. BH Bil. Sci. 352:1221-1230: “Perception arises as a result of an interaction between sensory input and prior knowledge.”
- 26. Hypotheses (based on preliminary observations): 1. When there is no clue, as in standard MIP, interpretation is subject to biases, but we don’t know now what they are. 2. A given subject will be somewhat consistent as to his or her bias. (We call that low intra-subject variability.) 3. The biases may have correlations, for example, gender, handedness or ….. 4. Some variations could be tested, e.g. upside down images. 5. It gets down to the science of how the brain works.
- 28. What is your impression as to the side of the sacroiliac lesion? 1. Right SI? 2. Left SI? 3. Can’t tell.
- 31. Drew, Trafton, Melissa L-H. Võ, and Jeremy M. Wolfe. "The Invisible Gorilla Strikes Again: Sustained Inattentional Blindness in Expert Observers." Psychological science (2013).
- 32. Frith and Dolan, 1997. Philos.; Trans. R. Soc. Lnd. BH Bil. Sci. 352: 1221-1230 “Perception arises as a result of an interaction between sensory input and prior knowledge.”
- 33. Drew et al: “The message of the present set of results is that even this high level of expertise does not immunize individuals against inherent limitations of human attention and perception. Researchers should seek better understanding of these limits, so that medical and other man-made search tasks could be designed in ways that reduce the consequences of these limitations.”
- 34. Poll of 5 physicians in my department • All agreed that MIPs are helpful for showing cases to others, for example at conferences. • Most felt that they gave a helpful overview to guide interpretation. • No one thought that MIPs should be the main means of interpreting the study.
- 38. Summarizing possible cures for the MIP problem 1. Depth-weighting. 2. Stereoscopic viewing. 3. Etc.
- 39. Conclusion: The medical imaging illusion that I have shown you ought to be further investigated as a problem in the nature of biases in perception. What goes on in the brain to make us see things this way? The answers might improve our ability to deal with medical images.
- 40. “There are two kinds of people in the world, those who believe there are two kinds of people in the world and those who don't.” Robert Benchley, American humorist, 1889-1945
Editor's Notes
- My alternate title is “Medical imaging meets psychology of perception: Optical illusions.” It is about an interface that exists between medical imaging and brain science, specifically psychology, and illustrates the value of exchanging ideas between disciplines. I had a career in medical imaging, specifically nuclear medicine, in which I pursued many different interests, and finally this one, about which I have previously spoken to both psychology and nuclear medicine groups. I had been looking at a certain type of image that was puzzling to me and decided to look at it in greater depth from a psychological point of view. The talk includes some video displays and includes audience participation, like research subjects in a way, but painless and harmless. The talk is mostly psychology, but some about nuclear medicine imaging, and I’ll be glad to answer questions about that. Are there any physicians present? Any psychologists? Any technical people, like physicists or computer experts? Well, I hope there is something here for everyone.
- We live in a 3D world but are challenged to comprehend it with 2D retinas. When moving as in a car, the difference between the motion of close and distant objects helps us appreciate depth. Of course there are many other clues, a very important one being stereoscopic vision. The human body can be studied in 3D, as with so-called CT scans , MRI scans, PET and SPECT scans, and this is a particular challenge Various body structures are tightly assembled in complex arrangements, not like the woods near my old house in Pennsylvania.
- We generally have to make a painstaking inspection of 100’s of cross-sections like this, and we like anything that might make it easier and more agreeable to do a thorough and accurate job, while not creating difficulties due to bias or illusions. One of the ways that we help ourselves with this is called a MIP, which forms a major part of my talk, and which I will, of course, explain.
- For simplicity, my main examples will be from nuclear medicine bone images. Here is a two-dimensional bone scan, anterior and posterior, that is, front and back. We don’t have difficulty telling which side is which, because structures like the sternum or breast bone, are better seen on the anterior view and vertebrae, that is, the spine, on the posterior view. These are made using a detector, which shows best what is closest to it. So when it is in front of the patient, we see the breast bone well, but the signals from the spine are reduced by distance and the shadow of the tissues in front of it. Vice versa for the posterior view. This is the femur, the long bone in the leg. Now, by convention, we view the images as though looking at the patient. This femur has a stronger signal that its mate, meaning he has lesion, probably metastatic cancer, in the femur and we can say with confidence that it is the left femur. Note the sacroiliac joints, seen best on the posterior view. The next illustrative case will have a lesion in the sacroiliac joint.
- You should get a sense of 3D, which is the point, and you should see that one of the sacroiliac joints has a much more intense signal (that is, darker) than the other. Is it the left of the right?
- Now that was a 3D bone scan, specifically a PET scan, which I will explain, of a patient with cancer. Now I will say something that never occurred to me with my previous audiences, like a group of young psychology graduate students. In this group, today, I expect that there are folks who have had scans, of this or other types, for cancer, or have friends or loved ones who have. Here I am, finding these scans very fascinating (which, of course, happens in the case of research-oriented physicians), not only that, but even entertaining. I want to emphasize that this means no disrespect or lack of empathy with people with such diseases. In fact, if you are one of the many people who have had a scan for cancer, I want you to know that I am one too.
- We’ll come back to the 3D bone scan. For the moment, on the theme of the interface of medical imaging and brain science, here is a nuclear medicine image of the brain in a patient with epilepsy, also a MIP. If you were fortunate enough to attend the last talk, about epilepsy, this is relevant to that. This type of study helps in planning surgery for epilepsy.
- Now for 2 or 3 slides, I will get a little technical. So that will, I hope, appeal to those who are technically oriented. Everyone else can take a very short nap.
- No radiation is better for you if you can get away with it. Tests with radiation should be done as safely as possible and only when the benefit outweighs the risk.
- I’ll try not to be too, too technical, but I do want to explain some of the basis for nuclear medicine scans. We inject radioactive isotopes in people, and there are 2 kinds, both of which undergo radioactive decay, one kind sending out one ray at a time, the other 2 simultaneous rays, or more specifically, photons. The first kind can produce either a 2D image, like the regular bone scan that I showed you first, or 3D images called SPECT. Here’s an illustration for single photon. The photon comes out of the body, here, and hits the detector, here, and an image gets formed. If you ever got such a bone scan, the injected substance was, likely as not, called technetium-99m methylene diphosphonate. This diagram is for PET scan, showing 2 photons going out to detectors, in opposite directions, with images then formed. That kind of bone scan is less common, but was the most suitable for my talk, and the injected substance is fluorine-18 in the form of fluoride, like for your teeth. Incidentally, this is also not the most common kind of PET scan. So, again, the first kind can provide a 3-D scan sometimes, called SPECT, the second kind, PET, is always 3D.
- As you likely know, a digiital photograph has images composed of pixels, little squares. In 3D, we call the equivalent a voxel. A PET scan has millions of voxels, and can be thought of a cube full of voxels. To make the MIP, the computer looks at the cube from different angles, that is, rotating around, and always picks out the most intense voxel to place in the picture. Then, the pictures shown in sequence make the movie that we call a MIP. If we call “head to toe” the z-axis, we are looking at this cube exactly perpendicular to the z-axis. That sounds quite technical but it will be important later. I hope that didn’t put you to sleep. If it did, it’s time to wake up. I wanted you to know how we make our images, but I was also killing time before showing you the MIP again, and again ask you to decide whether the sacroiliac lesion is on the right or the left, but I will frame the question a little differently.
- Do a dance. So this is my nuclear medicine optical illusion. It might better be called an ambiguity.
- There’s no mistaking the right image for anything but a right lateral. The first image could be anterior or posterior. If you see it as an anterior, the lesion is in the right sacroiliac joint, if posterior, in the left sacroiliac joint, and that’s the problem. The anterior is the mirror image of the posterior, but you can’t tell which is which.
- With that in mind, I just want to show it to you again. Do you see my point? Anterior…posterior….just mirror images, and not necessarily what I said they were, maybe the opposite.
- I searched the literature for related psychological phenomena, that is, related to bias and illusion as they apply to rotating 3D images, not necessarily of a medical nature. The ballerina can be seen to be rotating either clockwise or counterclockwise. Note that, unlike my MIP, the vantage point is not perpendicular to the z-axis. This isn’t the only version that one can find. I won’t quiz you on this one, but most people see it as going clockwise.
- Just like the MIP, the silhouette can translate into a ballerina going clockwise or counterclockwise.
- There’s a special feature of the rotating ballerina, which I will use this picture of a theater to illustrate. Imagine she is going to come on stage here. You might see her from an angle as from the balcony, where this picture was taken, that is, from above, or from an angle as seen by the orchestra players, that is from below. A careful analysis of the dimensions in the rotating ballerina picture shows that it has to be one or the other, not straight on, like someone sitting here, say. It turns out here that clockwise rotation is equivalent to seeing the images at an angle several degrees above the perpendicular, but counterclockwise several degrees below. This is unlike the MIP, which is seen perpendicularly or straight on.
- This slide is from a research paper about the ballerina. So this research does not explain a bias when viewing the MIP. I haven’t seen the MIP illusion discussed as such in the literature, with the exception that the Wikipedia “Spinning Dancer” article shows a MIP of a PET scan.
- Here is another related illusion of a rotating image. I’m going to make this mask rotate, and see what happens. Most people see what we call depth inversion. After we start the video, we see the front side, where the nose is coming out toward you, which is to say it is convex. Then we move to the rear, where the nose is pointing away from you and we call it concave side…Watch what happens. Most people will see what we may call depth inversion where it again looks convex and there is a change in the direction of rotation. Of course, I’m not showing this to you in true 3D (which is to say stereoscopic), like the 3D movies you watch with glasses. However, the illusion occurs even with 3D versions.
- There is published research about this illusion too. First…Klopfer…Incredibly, it is more normal to have the illusion than not. Schneider et all…The schizophrenics perceived things more according to their true nature rather than being biased by experience. Learned familiarity with convex faces did not interfere with a literal perception of the concave faces. They had what psychologists call veridical perception
- The mask illusion may seem farther removed from the MIP illusion, but this brief statement of what can cause illusions may be applicable in various ways to the perception of medical images. How we see things is colored by what we are accustomed to in our past. Nobody has put forward a satisfactory explanation of the MIP illusion, to my knowledge.
- Time for more audience participation. Which side is the sacroiliac lesion on. Ready? (Show it twice.) Do you have an impression?
- It’s the same case, but with a difference. Something called depth weighting has been applied, and I’ll explain that. As a result, now you can tell anterior from posterior, and are more likely to see the display as rotating counterclockwise, which corresponds with the true story of the patient, whose lesion is on the right, not the left.
- To illustrate re anterior and posterior. Remember, I said that the original ambiguity occurred because the anterior and posterior images were the same, except for one being the mirror image of the other. This is a MIP, in which that situation has been changed. The anterior and posterior look different. Now you see the spine well in the posterior and the breastbone well in the anterior. Show that. Remember, in the first bone scan that I showed, the one in 2D, anterior and posterior looked different because of distance and the shadow of the tissues in between. In the modified MIP that I am showing you here, the computer has put in the shadows artificially, by the process called depth weighting. If you now see the sacroiliac lesion as being on the right, it is what is true for the patient, because you are now better able to perceive the movie as going counterclockwise rather than clockwise
- You might wonder, “Will a MIP cause the physician to make a mistake?” It should not, because his or her ultimate source of truth will be a cross-sectional image, here a so-called transaxial image (illustrate a slice) which follows the universal convention regarding anterior, posterior, right and left. This is anterior, this is posterior, this is left and this is right. So the abnormality is on the right. Not that physicians reading these scans can’t make mistakes! They can.
- To illustrate the fallibility of medical imaging specialists, I will now show a different kind of scan, computed tomography of the chest, commonly called a CT or CAT scan, specifically a transaxial cross section, which would be used to search for nodules in the lungs, which might be lung cancer. In a research project, when experienced radiologists were shown this scan for their interpretation, most of them looking for nodules failed to notice something, the gorilla at the upper right (actually the left lung).
- The problem, it was concluded, is that their minds were clouded because they were looking for nodules, not gorillas, something called inattentional blindness. Again, a gorilla in a CT lung scan is something they would never have seen. Gorillas don’t appear on lung CT scans! As I mentioned before, Frith and Dolan said… This seems a key point to keep in mind.
- In other words, lets try to find ways to ensure against errors.
- Standard PET, with activity in heart and liver, hence a more complicated case than the skeleton. You can tell which is left and right. But this is a non-depth-weighted MIP and still subject to some ambiguity. I see it as a case of situs inversus.