What is salience?
Our sensory systems are drawn to anything that is conflicting or unusual within its surroundings. Salience is one of our prime survival mechanisms and it informs our conscious mind about changes in smell, taste, touch, sound and vision.
Whenever we are triggered by external stimuli, our attention is reactive and free of thought. Waking up in the middle of the night due to a loud noise from an unknown source. Feeling an uneven spot under your fingers. The smell from a local bakery. These are just a few examples of what we call a bottom-up saliency.
Our attention can also be guided consciously, such as when anticipating an action, e.g. reading, searching or looking both ways before crossing the street. This is what we call, top-down saliency.
Both types of saliency constantly work together to form our attention as humans and animals alike are unable to have more than one focus point at a time. The process of prioritizing different top-down and bottom-up influences is a challenge that our brain has to deal with our whole lives.
When it comes to color stimuli, our vision depends on 2 million nerve fibers that transfer around 10,000,000 bits of raw visual data from our eyes each second. Combined with a million more from the rest of our sensory systems, this sheer amount of data is too much to be handled in real time by our brain as our conscious mind can process no more than 50 at a time.
Our limited cognitive resources necessitate a method for determining the most relevant inputs and neglecting the rest within our field of view. But how does our vision decide where to put its attention?
Consciousness is frequently believed to be the brain's most prominent feature. The brief observations above, however, show that we rarely decide where to look, rather our visual cortex decides that for us. The majority of image processing appears to occur without conscious awareness and most of our reflexes appear to occur without our direct conscious control.
Whenever our attention is influenced by the saliency of an object, it is always reflexive in nature and happens instinctively, without much cognitive guidance.
It is believed that in order to guide these reflexive gaze shifts, the primary visual cortex builds a bottom-up saliency map from all the visual inputs. This “map” depicts how prominent the objects we see are in terms of saliency and we then directly shift our focus to the most salient area.
After our curiosity is satisfied we switch our attention to the second most salient item and so on. As we move in space and time, the environment changes and our attention is constantly attracted to the current most salient object in view.
There are several types of saliency maps created by our brain in order to process the variety of available features such as motion, scale, orientation, and most importantly for this article - color.
Following the evaluation of inputs from each feature map, our primary visual cortex combines them into a single, cohesive and intuitive saliency map. Our attention then gravitates towards the heaviest area on this map that breaks patterns and stands out.
Although such extensive processing results in a slight data delay, the unbelievable compression from millions of raw pixels to just a couple of prominent options is a superpower for our cognitive abilities. High salience levels allow us to analyse reality almost in real time.
On the other hand, low saliency images take us much more effort to scan, deconstruct and understand. Lost in low intensity inputs, our brain struggles to process this noise. Millions of indistinguishable bits of information that consume energy and time to be understood.
Whenever we are triggered by an external stimuli, our attention is reactive and we rarely decide where to look. Salience reflects how prominent the objects we see are. Based on distinct features such as scale, orientation and color our primary visual cortex builds a saliency map that guides our attention and helps us cope with the world.