Understanding Real-World Visual Expertise in the Multidimensional Space

The Quest for Visual Expertise

The Three Minute Thesis (3MT) was definitely a fun experience. Folks at the Writing Studio at Vanderbilt University helped me to re-write and re-write the script that I felt so much pain during the process but so much pride afterwards. See my talk here. See my talk transcript below. See the code I use for my online experiments the similarity ratings task, the bird expertise test, and the bird identification task.

Given an X-ray image, Ms. Red, an expert radiologist, can diagnose Alzheimers from a single look. In contrast, Mr. Green, an aspiring medical student, still has to guess. This ability that Ms. Red has, the exceptional ability to make judgments with images, is called visual expertise. Beyond radiology, visual experts like Ms. Red play important roles in many domains in our society, such as airport baggage screening and Forensic fingerprint identification. My research goal is to understand such visual expertise so that researchers can design more effective training systems for aspiring students like Mr. Green to develop their visual expertise. Specifically, I study what experts and novices see & how that relates to their performance. To answer these questions about general visual expertise, I use bird identification ability as example to study, because bird identification is a commonly-used and accessible example of visual expertise, unlike radiology and forensics.

First, I study what bird experts and novices see in their mental space. In our mind’s eye, we see things as points located in a multidimensional space. For example, everyone has a unique mental space for puppies, with dimensions such as the puppy’s size, cuteness, and intelligence, In your puppy space, you might see your puppy as located lower than your neighbor’s on the dimension of size, but much higher on the dimension of cuteness. And your neighbor would certainly have a different puppy space. Your personal puppy space would influence the similarities you observe among different puppies, providing a good metric for me to identify the dimensions of your puppy space. Using this logic, I model and measure the experts’ and novices’ bird space by gathering their ratings of similarities among different birds. This will answer the question of “what they see”. Then I relate “what they see” to “how they perform”, in this case bird identification performance. This relationship between “what they see” and “how they perform” has significant implications. To be specific, if there are dimensions in the experts’ mental space that are critical for the efficient diagnoses using X-ray images, which novices like Mr. Green are not aware of, we can highlight those dimensions in their training to improve the training efficiency.