2.1 Human Visual System (HVS)

Research on the HVS can be traced back to more than 100 years ago [1], and the basic physiological aspects of the HVS have been investigated and published in literature, such as [2–7]. The front end of the HVS, the human eye, is illustrated in Figure 2.1. The crystalline lens transfers the light from the visual field onto the back of the eye. The light entering the retina traverses several layers of neurons before reaching the light-sensitive layer of photoreceptors and being finally absorbed in the pigment layer. There are two kinds of photoreceptors in the retina, cones and rods, and at the centre of the retina is the fovea that has a concentration of cones, so the fovea has higher visual acuity than its periphery. Table 2.1 lists and explains the relevant major parts in the human eye as an easy reference for Figure 2.1. Note that we cannot show the neural layers and photoreceptors of the retina in Figure 2.1 due to the size limitation.

Table 2.1 A list and explanations of human eye and their abbreviations.

Figure 2.1 The cross section of the human eye [5]. Reproduced with permission from R Nave, ‘Cross section of the human eye’, Georgia State University, http://hyperphysics.phy-astr.gsu.edu/Hbase/vision/eye.html (accessed October 1, 2012)

The mid part of the HVS before the high visual cortex is shown in Figure 2.2. The visual pathways begin at optic nerve (ON) from the retina of both left and right eyes, and go to the optic chiasm (OC) where the nerve fibres from the nasal halves of the two eyes cross and enter the lateral geniculate nucleus (LGN). The nerve fibres from the temporal halves do not cross but enter the LGN on their respective side, that is right or left.

Figure 2.2 Visual pathways in the HVS [8]. Reproduced from J. R. Pomerantz, ‘Psychology351 ‘psychology of perception,’ Rice University, http://www.owlnet.rice.edu/~psyc351/Images/VisualPathways.jpg (accessed November 25, 2012).

From the LGN, nerve fibres go to the visual cortex via the optic radiation. This visual pathway is topographically organized [3]. Thus, a specific part of the retina maps to a particular region of the LGN, which is in turn linked to a cluster of cells of the visual cortex (V1–V4 areas). The primary visual cortex, called the V1 area, is the part of the visual cortex that receives signals from the LGN. Figure 2.2 shows the pathway of the HVS from the ON to the primary visual cortex V1. After that, the information flows in two main directions in the brain: towards the parietal lobe as the dorsal stream and the temporal lobe as the ventral stream. The dorsal stream starts at V1 and follows the path through V2, V3 and V5 (middle temporal or MT), finally projects to the partial cortex, which responds to spatial location and motion referred to as the where pathway or fast pathway. The ventral stream begins in V1 and passes through V2 and V4, ultimately reaching the inferior temporal lobe. This stream mainly processes the colour and shape of objects in visual field, which is referred to as the what pathway or slow pathway. The following subsection will discuss this in detail. The parts in the brain that control and guide eye movement, are the frontal eye field (FEF) and the middle eye field (MEF). Also, the part related to attention activity is the lateral intraparietal (LIP). Table 2.2 lists the main parts of the HVS pathway and their abbreviations.

Table 2.2 The main parts in HVS pathway and their abbreviations.

2.1.1 Information Separation

The HVS has specialized areas to perform particular processes, and different information is treated separately as detailed below.

Cones and rods are the two light sensors of the retina. During daylight (bright environment), the response of rods is saturated, so the main response comes from cones. However, in the night (dark environment), rods give the main response since the input to the cones is too weak to create the stimulus. Both cones and rods function at dawn and sunset. Responses of the two sensors fuse at a later stage.

The ganglion cells in the retina and cells in the fourth layer of V1 do not respond to diffused light but they are responsive to bright objects on a dark background and dark objects on a bright background [3, 9]. This implies that contours and edges can be easily detected there; in other words, retinal neurons have a mechanism of contrast computation.

Signal separation is further evident from the existence of two pathways (magnocellular and parvocellular) from the retina to V1 via the ON and the LGN. A broad division would be form and motion. However, there is far more segregation in the kind of visual information that the HVS processes. The larger path (related to the top four layers of the LGN) is the parvocellular pathway that transfers all colour, spatial detail and contrast information (i.e., the form) with thin nerve fibres that transfer information relatively slowly. The magnocellular pathway (with the other two layers of the LGN) transfers all transient and motion-related information, but it does not respond to chromatic information. Its fibres are thicker so as to allow faster information transfer.

There are three types of cones classified according to their spectral sensitivity in the retina: S-, M- and L-cones corresponding to blue, green and red, respectively. This is the basis of the trichromacy theory [10]. When light falls on the fovea where most of cones are housed, each of them picks up its relevant wavelength and becomes excited. The light in most cases is a combination of wavelengths so multiple types of cones are excited. In the LGN, the process that takes place for further colour interpretation is known as the opponent colour space [11]. In the top four layers of the LGN (the parvocellular pathway), there are two types of opponent cells: red-green (R-G) and yellow-blue (Y-B). After the LGN, the signal traverses to V1, which has colour perception centres called blobs [3]. The separation of an achromatic channel and two opponent-colour channels reduces redundancy among signals of different cells [12].

Visual cortices V1–V4 distinguish orientation, form, colour, motion and stereo information [13] while visual cortex V5 (MT) responds to motion. Cortical cells are responsive at distinct frequencies [14]. The fourth layer of cells in V1 receives signals from the LGN. There are simple cells with a neat subdivision of excitatory and inhibitory regions to detect critically orientated objects. After that, there is a layer of complex cells, which fire impulses while a critically orientated object is in the receptive field. These cells are sensitive to direction, corners, curvatures and sudden breaks in lines [3].

2.1.2 Eye Movement and Involved Brain Regions

As mentioned in Chapter 1, when a new target appears in the visual field as we look at a particular object, our eyes can move towards that target in a very short period. Such movement is called a saccade [15, 16]. While examining the same target, our eyes still make tiny movements, which are called microsaccades. The central subcortical structure involved in visually guided saccadic eye movement is the superior colliculus (SC) which resides on the roof of the mid-brain and is about the size of a pea, as shown in Figure 2.2.

The other involved areas are the FEF, the MEF, the parietal cortex and the visual cortex.

Saccadic eye movement is a complex phenomenon. First, it determines which objects are in the scene. This is carried out by many structures that include V1, V2, V4, LIP and the inferior temporal (IT) region. Second, one target needs to be chosen to examine. This decision again involves areas V1, V2, LIP, FEF and MEF. Third, a decision needs to be made about which objects not to look at, which involves V1, V2 and LIP. Fourth, the spatial location of the target is generated for the saccade, which involves V1, V2, SC and FEF. Lastly, a decision is made as to when to initiate the saccade within LIP and MEF.

The result of eye movement is that rays from the object fall at the fovea. Apart from saccadic eye movement, there are two other types of eye movements. Vergence movement occurs when we shift the view from a far object to a near one (convergence) and from a near object to a far one (divergence). Pursuit eye movement is used to track an object when it is still on the retina yet has a translational motion.

2.1.3 Visual Attention Processing in the Brain

The perceived environment always presents far more perceptual information than the HVS can effectively process. As a result, the HVS selects a part of it for analysis and then responds accordingly. Visual attention is selective awareness of the sensory world or selective responsiveness to stimuli [17, 18]. In this regard, the HVS characteristics are the consequence of evolution.

Visual attention is based upon location and visual features (a single feature such as colour or motion [19, 20] or multiple features [21, 22]). There are exogenous and endogenous cues that direct attention to a particular point [23]. The former refers to the external stimuli that lead to low-level reflexes for automatic focusing of attention at a particular location (i.e., bottom-up attention). The latter causes a voluntary shift in attention; for example, a subject is given an instruction to direct his/her attention to a location/object. This is associated with top-down attention, which is a high-level process in the higher areas of the brain such as the frontal lobes.

As already mentioned in Section 1.4.1, the visual attention process consists of three stages [24]: pre-attentive, attentive and post-attentive stages. In the pre-attention stage input information is processed by retina, LGN and primary visual area V1. Each cell of retina, LGN and V1 has a small representing area in the retina called the receptive field (RF), namely, each cell only receives the stimuli from its RF and the RF of ganglion cells in the retina its centre–surround opponent structure, which enables ganglion cells to detect edges in the RF. Various simple cells in V1, and even in the LGN, extract different features in the RF themselves, such as colour, orientation, motion and size, in parallel. The RF of complex cells and super complex cells is larger than that of simple cells; hence, the complex cells can extract more complex features such as shape, corners, curvatures and so on. In the pre-attention stage, features are only analysed but not integrated. At the attention stage, these features are integrated. The attention stage passes dorsal stream areas concerned with spatial localization and motion direction [25] and ventral stream areas concerned with the form of objects, which receive attention modulation [25]. In the post-attentive stage, a subject performs a further search among objects of the same group. Search efficiency in this stage will improve because the HVS has already attended to the presented objects and is now familiar with them.

As mentioned in Section 1.2.4, there are overt attention and covert attention. The former leads to actual saccades while the latter occurs when the eye remains fixated at a particular place but the attention is diverted. For overt attention, the pre-attentive processing is done in the visual cortex and some extra-striate areas. Then, the information is relayed to a network of cortical areas, including the superior frontal, the inferior parietal and the superior temporal regions [26], from which the information goes to the prefrontal cortex. For covert attention, the areas where the processing takes place after the visual cortex are the inferior temporal for recognition and identification of the visual areas and the posterior parietal cortex for spatial localization.

Decisions such as the target for a saccade, the distance of one target from another [27] and reaction time [28] all happen in the neurons of the FEF. Various features formed in the pre-attentive stage in the visual cortex converge at the FEF for the formation of the saliency map. Afterwards, there are specialized neurons in FEF to relay the information to the pons region in the brain stem to initiate eye movement to perform a saccade.