Convolutional Neural Networks | How Do Artificial Networks See Like Humans?

The human visual cortex processes information in a distinct hierarchical manner. This system is organized into a series of stages, starting from the primary visual cortex (V1) and progressing to higher-level areas (V2, V4, and the inferotemporal cortex). In V1, neurons are specialized to detect simple features from visual input, such as lines, edges, and orientations. This foundational concept was famously demonstrated by researchers Hubel and Wiesel, who identified 'simple cells' responding to lines of a specific orientation and 'complex cells' responding to lines of a certain orientation regardless of their exact position. As the signal moves through the hierarchy to V2 and beyond, neurons combine the simple inputs from the previous stage to detect more complex shapes, like corners or basic textures. In the final stages, such as the inferotemporal (IT) cortex, neurons can respond to highly complex and specific objects, like faces or hands. This progressive assembly of features, from simple lines to complete objects, allows for an efficient and robust recognition system. Each level builds upon the last, abstracting the information into a more coherent and meaningful representation.

Convolutional Neural Networks (CNNs) are a class of deep learning models designed specifically to analyze visual imagery, and their architecture is directly inspired by the brain's visual cortex. A CNN consists of several types of layers, most notably convolutional layers, pooling layers, and fully-connected layers. The convolutional layers at the beginning of the network function like the V1 cortex, applying filters (also called kernels) to detect basic features like edges, corners, and colors in the input image. As the data passes to deeper convolutional layers, these simple features are combined to form more complex patterns, analogous to processing in V2 and V4. For example, a deeper layer might learn to recognize textures, circles, or squares by combining edge detections from earlier layers. The final fully-connected layers act like the IT cortex, taking the high-level features and classifying them into specific object categories. This layered, hierarchical structure allows CNNs to build a progressively more abstract and complex understanding of an image, directly mirroring the biological process of human vision.

In neuroscience, a neuron's receptive field is the specific region of the visual field where a stimulus will trigger that neuron to fire. For a simple cell in the V1 cortex, this might be a small area where it detects a vertical line. In CNNs, this concept is implemented through 'filters' or 'kernels' in the convolutional layers. Each filter is a small matrix of weights that slides over the input image, looking for a particular feature. The area of the image that a filter covers at any given moment is its effective receptive field. Just as a biological neuron is tuned to a specific stimulus within its receptive field, a CNN filter is designed to activate when it detects the specific pattern it is looking for (e.g., a horizontal edge) within the portion of the image it is analyzing.

The pooling layer in a CNN serves to reduce the spatial dimensions of the feature maps, making the network more efficient and resistant to small variations in the input. While there is no direct one-to-one anatomical equivalent, its function is analogous to the behavior of complex cells in the visual cortex. Complex cells, as identified by Hubel and Wiesel, respond to a specific feature (like a line orientation) within a larger receptive field, regardless of its precise location. Similarly, a max-pooling layer takes a small region of a feature map and outputs only the maximum value, effectively summarizing the presence of a feature in that region. This provides a degree of 'spatial invariance,' meaning the network can recognize an object even if it is shifted slightly, rotated, or scaled.

While CNNs are powerful tools inspired by the visual cortex, they are not a perfect model. There are significant differences. The human brain is vastly more energy-efficient and can learn to recognize new objects from very few examples, sometimes just one (a capability known as 'one-shot learning'), whereas CNNs typically require training on massive datasets. Furthermore, human vision is not a one-way, bottom-up process. The brain utilizes extensive top-down feedback, where higher-level knowledge and context influence perception in lower-level areas. For example, your expectation of seeing a face in a crowd helps you to spot one. Most standard CNNs are feedforward networks and lack these crucial feedback mechanisms. They also do not integrate vision with other senses or a lifetime of contextual knowledge in the seamless way the brain does. Therefore, CNNs represent a simplified but highly effective computational model of a subset of our visual processing capabilities.