Understanding how the human visual system stores item shapes and how shape is eventually utilized to distinguish objects is a key objective of vision science. According to computer vision research, computational models based on the medial axis, commonly referred to as the “shape skeleton,” may be used to construct and then compare shape representations. Few research has examined how shape skeletons are encoded neurally, despite recent behavioral studies suggesting that humans also represent them.
Researchers from Carnegie Mellon, MIT, and Emory University investigated the idea that an object’s form is represented by the visual system using a skeleton structure. With the help of representational similarity analysis (RSA) and functional magnetic resonance imaging (fMRI), they discovered that a model of skeletal similarity adequately described the large unique variation in the response profiles of the visual cortex and the occipital lobe.
Shape skeletons are structural representations of an item based on its medial axis. Through internal symmetry axes, they offer a quantitative representation of the spatial organization of object outlines and component elements. Such a description may be used to identify objects across views and category exemplars, as well as to discern an object’s shape from noisy or insufficient contour information, according to computer vision research. In fact, adding a skeleton model to convolutional neural networks (CNNs) purchased off the shelf greatly enhances their performance on visual perception tests.
Similar to this, behavioral studies on people have revealed that subjects extract the skeleton of 2D objects even when boundary disturbances and fictitious outlines are present. Even after accounting for various models of vision, additional research has revealed that skeleton models are predictive of human object identification. As a result, shape skeletons may be crucial for object recognition and shape perception, however, their brain implications are still poorly understood.
We created a novel set of objects that allowed us to systematically vary object skeletons and directly measure skeletal coding. We then examined the unique contributions of skeletal information to neural responses across the visual hierarchy. More specifically, we used representational similarity analysis (RSA) to test whether a model of skeletal similarity predicted the response patterns in these regions while controlling for other models of visual similarity that do not represent the shape skeleton, but approximate other aspects of visual processing.
The discovery of skeletal processing in the visual cortex is compatible with research on human neuroimaging that demonstrates its participation in perceptual organization. In fact, the visual cortex is the first step of the visual hierarchy where symmetry structure has been decoded and has been repeatedly linked to the formation of shape perceptions. The occipital lobe has been demonstrated to be particularly sensitive to object-centered shape information and to be tolerant of some perspective alterations and boundary disturbances. The discovery of shape skeletons in this area is consistent with a function for skeletons in object recognition.
Our work highlights the unique role that shape skeletons play in the neural processing of objects. These findings not only enhance our understanding of how objects may be represented during visual processing, but they also shed light on the computations implemented in V3 and LO.
Skeletal representations of shape in the human visual cortex. Vladislav Ayzenberg, Frederik S.Kamps, Daniel D.Dilks, Stella F.Lourenco
Published: May 2021
DOI: 0.1016/j.neuropsychologia.2021.108092