Most of our knowledge about human visual performance describes foveal vision with a well centered pupil. However, these studies of "on-axis" vision do not provide much insight into the effects of moving the retinal image or the pupil off-axis. In this chapter we define the inherent optical and neural axes of the human eye, and we examine the optical and visual effects of decentering either the retinal image or the entrance pupil. The optical axis of the eye is usually well centered on the eye's neural axis, which provides best image quality at the fovea. Decentration of the eye's entrance pupil can occur when using any visually-coupled electro-optical device with a defined exit pupil. This decentration can lead to reduced light sensitivity, reduced spatial resolution, reduced contrast sensitivity, color distortions, unpredictable changes in refractive error, wavelength-specific distortions of visual direction and wavelength-specific errors in the judgment of apparent depth and size. We then examine associated problem of a centered pupil and an off-axis object and conclude with a simple computational model that accounts for the majority of the effects on contrast sensitivity and spatial resolution.
Most of what we know about vision and the visual processes stems from experiments in which subjects fixate a target which is viewed through the eye's natural pupil. Although we implicitly consider this to be the typical visual environment, it represents a very specific set of viewing conditions in which the target and pupil are located on a particular axis called the visual axis. This chapter will examine the optical and visual consequences of displacing either the pupil or target from this axis. By examining the optical ramifications of off-axis vision, we reveal a more general model of human spatial vision that may be particularly pertinent to real world visual tasks performed with or without visually-coupled electro-optical systems.
In the human electro-optical imaging system, the quality of signal transmission is affected by the quality of the eye's optical system and the quality of the electrical (neural) image processing. Recent work in our laboratories 7, 50, 53, 55, 56, 67, 68, 71 has highlighted the substantial differences in both optical and neural image quality between on- and off-axis vision. Predictably, the quality of the optical image depends on the route taken by the light which forms the retinal image. The light path, and hence the optical image quality, varies with both target and pupil decentration. In addition to the off-axis decline in optical image quality, there are profound differences between on- and off-axis neural processing by the retina and brain and these are discussed in the following chapter (Modeling off-axis vision - II: the effect of spatial filtering and sampling by retinal neurons, by Thibos & Bradley).
The present chapter will define the optical axes of the human visual system and examine the optical properties and visual ramifications of off-axis vision. Since the neural processing of the eye's optical image exhibits an approximate radial symmetry about a high resolution fovea (see chapter by Thibos and Bradley), we can think of the fovea as defining a neural axis for the human eye. To begin the discussion, therefore, the location of the human eye's optical axes will be compared to that of the eye's inherent neural axis.
In order to take full advantage of the functional capability of the high resolution area within the neural array, optical image quality must be high at the fovea. For example, typical adult humans can easily resolve up to 50 c/deg. 11 and some falcons and eagles can resolve more than 100 c/deg. with their foveae. 25, 45 In humans, 50 c/deg. corresponds to a grating period of about 6.5 microns on the retina. Numerous studies of retinal image quality in human eyes 5, 11-13, 16, 28, 29, 61 have shown that, in a well focused eye, with an intermediate pupil size, spatial frequencies up to about 50 c/deg can be imaged on the fovea with sufficient contrast to be visible. 11
Three basic methods have been applied to the problem of measuring optical quality of the human eye. First, the double-pass method examines the quality of an aerial image formed by reflection back off of the retina of the image of a thin line or point of light. 12, 29, 43, 61 It is assumed that the retina acts as a plane diffuse reflector and that image degradation is the same for both passes. The single pass modulation transfer function (MTF) is taken as the square root of the modulus of the aerial image's Fourier Transform. In the second method, the eye's optical MTF is inferred by comparing contrast sensitivity measured under (1) normal viewing conditions in which contrast in the retinal image is attenuated by the eye's optics, and (2) interferometric conditions in which retinal image contrast is assumed to be unaffected by the eye's optics. The reduced contrast sensitivity observed with normal viewing is, therefore, attributed to optical attenuation, and the ratio of the two types of contrast sensitivity function is equivalent to the MTF. 11, 61 Both of these techniques produce similar results, but a recent comparison using the same subjects showed that the double-pass method tended to slightly underestimate optical quality, 62 probably due to an error in the first assumption (a plane cosine reflector). A third technique involving measurement of ray aberrations has been used to estimate the optical quality in the human eye. By measuring the monochromatic optical aberrations of the eye, the MTF can be calculated. Subjective 31 and objective 59 versions of an ingenious device called an aberroscope were developed for this purpose. A comparison with the previous two methods 62 suggests that the aberroscope estimates of the MTF are better by about a factor of two than those estimated with the double-pass and CS methods. Because the aberrascope method only measures the lower-order aberrations and does not evaluate the effects of higher order aberrations or scatter, it may over-estimate retinal image quality.
Figure 1: The average Contrast Sensitivity Functions of the two authors obtained using gratings generated on an oscilloscope and viewed through a 5mm pupil with those generated directly on the retina with an interferometer. Contrast sensitivity is the inverse of the minimum detectable contrast (contrast threshold). Although the retina can detect gratings with very high spatial frequencies (up to 120 c/deg) 64 , optical attenuation limits normal foveal vision to about 50 cycles per degree. The ratio of these two contrast sensitivity functions is the optical MTF of the human eye (see inset). 11 At the foveal resolution limit for natural view, retinal image contrast is reduced by optical factors to about 10% (CS natural view =1, CS interferometry = 10, ratio = 0.1).
Using the approach of Campbell and Green, 11 we have compared the authors' foveal contrast sensitivities obtained with interferometric and standard oscilloscope display technologies. Unlike the interferometrically generated gratings, retinal images of the oscilloscope gratings are degraded by the eye's optical aberrations and diffraction resulting in reduced contrast sensitivity (Figure 1). The ratio of the two contrast sensitivity functions provides a measure of the modulation transfer function of the human eye's optics (shown as an insert in Figure 1). This experiment was performed foveally with a well centered 5 mm natural pupil and well centered interference beams. The results show that, for example, at the normal high frequency cut-off (about 50 c/deg), the optics are transmitting about 10% of the object contrast to the retinal image. Similar results have been observed in earlier studies using this method. 11
Simple sphero-cylindrical corrections can usually provide myopes, hyperopes, and/or astigmats with a retinal image of sufficient quality to achieve high spatial resolution approaching the neural limit at the human fovea. 63 In addition, many visually-coupled viewing instruments (e.g. Night Vision Goggles: NVG) incorporate an adjustable ocular lens to control the exit vergence in a way similar to that of a spectacle lens, but most of these only correct spherical and not astigmatic refractive errors. With a suitable refractive correction most subjects can be provided with a retinal image of sufficient quality to take full advantage of the high quality neural fovea.
However, in order for the retinal image quality to achieve these high levels, the optical path within the eye must be well centered. For example, failures in centration can lead to reduced light sensitivity, reduced spatial resolution, reduced contrast sensitivity, color distortions, unpredictable changes in refractive error, wavelength-specific distortions of visual direction and wavelength-specific errors in the judgment of apparent depth and size. The next section will consider the optical and visual consequences of decentering the optical path within the human eye.
Whenever we view the world through a visually coupled device that has an exit pupil, there is potential for optical path changes in the eye. Clearly, when viewing the world through a simple aperture or pupil, the location of the aperture determines the path taken by light in the eye. Also, when using a device that projects an exit pupil into the eye's pupil plane, any movement of the device with respect to the head will introduce displacements in this exit pupil with respect to the eye. If the instrument exit pupil does not completely cover the eye's natural pupil, these displacements will change the path of light in the eye that is used to form the retinal image. In active environments with significant changes in head position and acceleration (such as a fighter pilot in a combat environment), there will be significant changes in the relative location of the eye and the exit pupil of a visually coupled device (such as a NVG). Also, because the normal visual field extends about 160 degrees horizontally and about 120 degrees vertically, images of peripheral targets are created using quite different optical paths compared to the foveal images, and predictably their optical properties are quite different.
Early optical studies of the human eye introduced the idea of an optical axis defined as the line which contains the centers of rotation of each of the optical elements of the eye. Aligning all of the optical elements within a multi-lens system is a typical optical strategy for optimizing image quality, and such a system will generally provide maximum optical image quality for targets placed along this axis. Because of the inhomogeneities in the quality of the neural processing (see chapter by Thibos and Bradley), optimal image quality at the fovea is visually most important. Therefore, early studies examined the location of the eye's optical axis relative to the eye's neural axis. That is, where does the eye's optical axis intersect the retina?
The eye's optical axis may be determined experimentally by identifying the location of a target which yields superposition of the reflected (catoptric) images, known as Purkinje images I through IV, from the four optical surfaces in the eye (front and back surface of the cornea, front and back surface of the lens, see Figure 2). If the optical axis of the human eye included the neural fovea, the Purkinje images would, to the experimenter, appear aligned with the fixation target. In general this alignment is only approximate for two reasons. First, it is rare that all four catoptric images perfectly superimpose for any target location, 37 which indicates that the different optical elements within the human eye are not perfectly aligned along a specific optical axis. Second, the axis that creates the best alignment generally does not intersect the fixation point, and therefore it does not intersect the neural fovea. 37 The horizontal angular separation (angle alpha) between the neural axis of the visual system (often referred to as the visual axis, see Figure 2) and the best approximation of an optical axis varies from +17 degrees temporal to -2 degrees nasal, and it appears to co-vary with refractive error 37 . Vertically, the visual and optical axes seem better aligned. 37 Although there are few contemporary studies of the eye's optical axis, 19 and the older data indicate significant inter-subject variability, many texts report that the optical axis is 5 degrees temporal to the visual axis. The visual and optical consequences of these naturally occurring misalignments have never been examined experimentally.
Figure 2: Schematic eye showing the principle optical surfaces, and the general anatomical structure of the eye. The optical axis (not shown) is defined as the line passing through or near the centers of rotation of the four optical surfaces of the eye. The achromatic axis follows the path of the chief nodal ray (passes through center of pupil and nodal points) and is the axis for which chromatic dispersion vanishes. The visual axis is defined as the axis connecting the fovea and the fixation target via the eye's nodal points.
A major limitation of this traditional formulation of the optical
axis of the human eye is that it does not take into account the
location of the eye's aperture formed by the iris. For this reason,
Thibos et al. 55 suggested the
path of the chief nodal ray as an alternative reference axis for the
eye. They reasoned that, for a given pupil location, each object
point in the visual field gives rise to a chief ray (i.e. a ray which
passes through the center of the pupil). Of the infinite number of
chief rays that are possible, only one is a nodal ray (i.e. enters
through the eye's anterior nodal point and exits in a parallel
direction from the posterior nodal point) and thus (by definition)
avoids the effects of refraction. Since this unique chief ray is
undeviated, it suffers no ray aberrations and thus the path of this
ray assumes special significance for vision. In particular, rays
traveling along this axis will be spared the effects of chromatic
dispersion and for this reason the axis was dubbed the "achromatic
axis" of the eye. 55 The magnitude of
angle 
between this achromatic axis and the visual axis (see Fig.
2) is of crucial importance for image quality of
polychromatic targets 55 on the fovea
and may also prove important for the analysis of off-axis,
monochromatic aberrations. In a sample of 5 subjects we found that
the pupil center was on average 0.14mm temporal to the visual axis,
and therefore, angle
was only 2 degrees. 55 The close
correspondence between the center of the natural pupil and the visual
axis indicates that the eye's pupil is located to provide best image
quality in the fovea.
All of the reference axes defined above fail to incorporate the directional properties of the retina, which has its own inherent optical axis. This factor is considered in the next section.
Stiles and Crawford in the 1930's discovered that the visual effectiveness of light entering the eye varied systematically with the path taken through the eye's optics 47 . The eye behaved as though the aperture was apodized and light entering at the center was considerably more effective than light entering through the edge of the pupil (Figure 3). Interestingly the location of maximum sensitivity can be moved. 1 This and other studies pointed to the photoreceptor optics, and not an apodizing filter in the optical path or pupil plane, as the source of the Stiles-Crawford effect (SCE).
Because the photoreceptor inner and outer segments are short thin cylinders with a higher refractive index than the surrounding tissue, they act as small optical fibers and exhibit typical waveguide properties such that light entering along the fiber axis will be totally internally reflected, and light entering from peripheral angles can escape into the surrounding tissue. 23 This optical property means that light entering on or near the receptor axis will pass through the entire length of the outer segment and therefore have an increased chance of isomerizing a photopigment molecule. Experiments show a large change in sensitivity to light entering the eye through different parts of the pupil. For example, rho values (Fig. 5) are typically around 0.05 mm-2, which means that sensitivity for light entering the pupil 4mm from the peak of the SCE is about 16% of the peak sensitivity. 2 Although the source of the SCE is the angular selectivity of individual photoreceptors in the human retina, it has been modeled as an optical apodization function in the pupil plane 38 .
The SCE has important ramifications for optical image quality and light sensitivity. First, if the eye's pupil is decentered with respect to the receptor axes, light entering the eye will have less visual impact. It will be similar to viewing the world through a neutral density filter. Fortunately, in most subjects, the pupil location and the receptor axes appear very closely aligned. This is inferred from the extensive data showing that the peak of the SCE is centered near to the pupil center (typically less than 1mm nasal of center). 2, 21, 58 This close correspondence is not surprising because one study has shown that the peak of the SCE will track the location of the pupil similar to the way that plants follow the direction of light 1 . This movement of the SCE is interpreted as a phototropic response of the individual photoreceptors, which should ensure a close correspondence between receptor axis and pupil location in all eyes. Therefore, in the normal eye, the pupil's location approximately maximizes the visual response. This optimal arrangement can, of course, be disrupted if a small artificial entrance pupil is introduced. For example, if the observer is using an optical device that has a small exit pupil, and this exit pupil is decentered with respect to the eye's normal entrance pupil, the light will effectively be reduced in intensity. It is also important to realize that some people have a naturally decentered SCE. 1, 2
Figure 3: Schematic representation of the Stiles Crawford Effect (SCE). A: Visual sensitivity () is dependent upon the radial distance between the point (x) at which light passes through the eye's pupil and the peak of the SCE. In the equation relating sensitivity to pupil location
= rho determines the steepness of this function. B and C: Although the origin of the SCE is retinal, it can be modeled with an apodizing filter (B) which behaves as a radially symmetric neutral density wedge in the pupil plane (C).
The potential for inadvertant decentration of an artificial pupil or instrument exit pupil can be largest when the natural pupil is most dilated. Dilation occurs at low light levels and is greatest for young observers. 66 For example, at 9 cd/m2 the pupil diameter of a 20 year old can be between 6 and 8 mm, and therefore light entering the eye at the pupil margin can be nearly an order of magnitude less effective than light entering through the pupil center. 2
The effect of pupil apodization on retinal image quality has been appreciated since some of the early work of Campbell who showed that the eye's depth of focus did not decrease with increasing pupil diameter by the amount predicted by geometrical optics. 8 Large pupils behaved as though they had a smaller diameter. Campbell was able to model the depth of focus data by substituting a uniform pupil with a radius of rc for the natural apodized pupil according to the empirically determined formula
 |
|
where rc is the radius of a uniform transmission pupil (no SCE) that would transmit the same amount of light as the larger (r) apodized pupil. In effect, Campbell showed that the apodization model of the retinal direction effect could predict the amount of light reaching the retina and the depth of focus of the human eye. He referred to this reduced pupil size as the "effective" pupil size.
Optical modeling of instrument and ocular optics indicates that the SCE or pupil apodization has little effect on well-focused image quality, 3, 4, 38, 42 but it has a substantial effect on the quality of defocused images. 42 Defocused optical image quality does not deteriorate as rapidly with increasing pupil size as might be expected if there were no apodization. 42 The protection against the effects of defocus provided by the SCE can be observed in the defocused contrast sensitivity (CS) data of Charman. 14 The small-pupil CS data are well fit by a uniform pupil model, but the defocused experimental data obtained with large pupils show substantially higher CS than predicted by a uniform pupil model. In addition to reducing the impact of image blur caused by uncorrected refractive errors, the SCE also reduces the impact of optical aberrations by attenuating the marginal rays which may be highly aberrated.
Finally, the SCE acts as an optical anchor, such that the effects of pupil decentrations on image location are effectively restrained. Pupil decentration will not lead to a shift in image position if the image is well focused (vertical dark bars in Figure 4). However, if the eye is defocused, lateral shifts in the pupil position produce lateral shifts in the defocused image position (shaded rectangles in Figure 4). Although the geometrical center of the eye's entrance pupil may be decentered, the effective pupil center (the central moment of the SCE-weighted decentered pupil) will be approximately anchored near the peak of the SCE (and hence near the center of the natural pupil). Therefore, problems of image misalignment that might be encountered in optical devices with exit pupils decentered from the eye's entrance pupil are partially ameliorated by the SCE. This interaction between decentered entrance pupil and the SCE and the impact on perceived visual direction has been modeled by Ye et al, 68 and it is discussed in detail in the next section.
Figure 4: Schematic showing the anchoring effect of the SCE. The top section of this figure shows ray tracing for two model eyes, one with a pupil centered on the visual axis (left) and one with a pupil decentered to the right of the visual axis (right). The lower section of this figure shows schematic retinal illuminance profiles (black for focused and gray for defocused) for retinal images of a point source viewed through a uniform pupil (middle section) and through an apodized pupil (lower section). Although the chief ray (geometric center of the ray bundle, and hence center of the blur circle) for a defocused image shifts with pupil decentration, the peak and central moment of the SCE-weighted blur circle remain fairly centered. See Fig. 2 for definition of visual axis.
Optical theory predicts that decentration of the entrance pupil can introduce a variety of optical aberrations, e.g. transverse chromatic aberration, coma, and astigmatism. Over the last 30 years, each one of these has been blamed for the visual effects of pupil decentration. Of the few studies of the visual impact of pupil decentration, an early study by Green 26 is the most striking. He measured contrast sensitivity (CS) for sinusoidal gratings presented on an oscilloscope as a function of the location of a small (2mm) artificial pupil. He found that decentration of the pupil led to large decreases in visual acuity (VA) and an even larger decline in mid- and high-frequency contrast sensitivity (Figure 5). For example, when the small artificial pupil was decentered 3 mm, Green observed a 0.5 log unit reduction in VA and a 2 log unit reduction in CS at 15 c/deg. He noticed no such decline in CS when decentering the beams of a laser interferometer. Since CS for monochromatic interference fringes is unaffected by defocus or any other aperture dependent aberrations, 11 Green attributed the loss in CS observed in the normal incoherent experiment to coma caused by off-axis viewing in an eye with spherical aberration. Earlier studies of the refractive properties of the human eye had shown that the refraction of the marginal optics was generally different than the central optics 32 indicating that the human eye exhibits off-axis aberrations similar to spherical aberration.
Figure 5: Data from Green 26 showing the effect of pupil decentration on foveal contrast sensitivity. These data were obtained by viewing a grating displayed on an oscilloscope with a P31 phosphor. Gratings were viewed through a small artificial pupil that was displaced from the point which provided best acuity. Note that the CS at 15 c/deg. has been reduced by about 2 log units.
Subsequent to Green's study, van Meeteren and Dunnewold 57 and Thibos 53 both argued the ocular chromatic aberration and not spherical aberration or coma were responsible for the reduction in contrast sensitivity and visual acuity with pupil decentration. Recently, off-axis astigmatism was proposed as a significant contributing factor 36 . Finally, Campbell, 10 and Campbell and Gregory 9 argued that reduced visual acuity for decentered ray-bundles could be explained by the anatomical properties of the photoreceptors. The next section will examine these various hypotheses.
The optical material of the human eye, like all refracting materials, exhibits chromatic dispersion. Obliquely incident rays will be spectrally dispersed, because short wavelengths are refracted more than long wavelengths. In an imaging system, this property can lead to wavelength dependent image planes (longitudinal or axial chromatic aberration), wavelength dependent image location within a given image plane (lateral or transverse chromatic aberration), and wavelength dependent image size, 50 all of which have been measured in the human eye. 30, 54, 71 Figure 6 shows a schematic representation of ocular longitudinal chromatic aberration (LCA) and transverse chromatic aberration (TCA).
Figure 6: A schematic diagram showing the effect of beam decentration on foveal TCA and LCA. The magnitude of foveal transverse chromatic aberration (TCA) is directly proportional to the decentration of the entrance pupil from the visual axis (see Figure 2) and it is related to the LCA by a simple formula. 55 TCA (radians)= LCA (diopters) * Pupil Displacement (meters). The visual axis is defined theoretically as the axis that connects the fixation point, the eye's nodal points, and the fovea. Operationally, this axis can be identified by locating the ray path from the fixation point to the fovea that creates no foveal TCA. 55 For a simple model eye with a single refracting surface, the ray that passes through the nodal point strikes the optical surface orthogonally and therefore is the unique ray that is not refracted and hence experiences no chromatic dispersion.
Experimental studies of ocular TCA and pupil decentration show that the retinal location of the short and long wavelength images can differ by as much as 40 arc minutes for an artificial pupil decentered 4mm from the visual axis. 55 Therefore, when a polychromatic target is viewed through a decentered pupil, the visible spectrum is smeared across the retina in the same meridian as pupil decentration. The spectral smearing of polychromatic images created by ocular TCA demodulates contrast of polychromatic images by introducing wavelength-dependent phase-shifts into the image. 53, 57 The theoretical effects of ocular TCA on retinal image contrast precisely predict the large decline in contrast sensitivity and visual acuity reported experimentally for polychromatic targets viewed through decentered pupils (Figure 7). 26, 53, 57, 70
Figure 7: This Figure compares two optical models of the effect of pupil decentration on retinal image quality. The solid lines are from Thibos 53 and they are calculated using a single surface water eye, the open circles are the data from van Meeteren and Dunnewold, 57 who used a wave optics model of the eye. Both models predict the same reduction in polychromatic (P31 phosphor) MTF with increasing pupil decentration (1, 2, and 3 mm from top to bottom). Interestingly, Thibos's model only includes TCA. The filled circles are the retinal contrast thresholds from Campbell and Green, 11 and the arrows show the measured reduction in acuity observed by Green 26 with an artificial pupil decentered 1, 2, and 3 mm from the visual axis (see Fig. 5). The cross-over points between the model MTF and the retinal contrast threshold function predict the maximum acuity possible for decentered pupils. The predictions provide an accurate fit of the acuity data observed by Green (abscissa value for arrow tips).
Using the contrast threshold function from Campbell and Green 11 and the predicted polychromatic MTFs for decentered pupils, 53 we can see that the highest detectable spatial frequencies imaged on the model eye retina for this polychromatic source correspond precisely with the measured fall-off in acuity seen in Green's data. The effectiveness of a TCA-based optical model to predict Green's data suggests that the reason Green observed no loss in contrast sensitivity for decentered interferometric gratings was not because they were created by interference on the retina, but because they were monochromatic. This conclusion is supported by results from our laboratories 6, 54 which show that interferometric monochromatic visual acuity is largely unaffected by beam decentration, but polychromatic interferometric acuity declines rapidly with increasing beam decentration (Figure 8). As predicted by the TCA models, the reduction in acuity and CS only occurs when the gratings were orthogonal to the axis of decentration (Figure 8).
Figure 8: Data showing the effect of beam entry point on interferometric visual acuity. Filled and open symbols show data for vertical and horizontal gratings, respectively. The left and right panel show luminance matched data for monochromatic and polychromatic visual acuity. The polychromatic source was an incandescent bulb. Beams were decentered horizontally from the visual axis.
In addition to the effects of TCA, some of the reduction in CS and VA is thought to originate in the retina. Due to the generally long axial length and narrow width of foveal photoreceptors, marginal rays in the pupil strike the retina obliquely and are thought to penetrate several photoreceptors as they pass through the retina (Figure 9). The idea of a "Retinal Direction Effect", originated by Campbell 9, 10 proposed meridional smearing of the image across several receptors to account for some of the loss in VA that accompanied pupil decentration. Interestingly, Campbell's model predicts a reduction in VA only for contours oriented orthogonal to the axis of pupil decentration, which is the same prediction made by TCA. In both cases (as shown schematically in Figure 9), a horizontally displaced pupil will lead to reduced contrast in the retinal images of vertical contours. However, this effect should be independent of spectral distribution of the light, and thus it probably accounts for the residual effects observed with monochromatic interferometric VA and CS.17 Interestingly, Green 26 observed no effect on CS with his decentered monochromatic interferometer, and we only observed a small effect on VA (see Figure 8). Both of these studies suggest that the "retinal direction effect" is quite small, and virtually all of the visual problems associated with beam decentration are attributable to TCA.
Interestingly, the detailed study by Chen and Makous 17 points out the inadequacy of the simple model proposed by Campbell and described in Figure 9. Their results show that high spatial frequency interferometric contrast sensitivity is reduced by oblique retinal incidence for gratings oriented either orthogonal to or parallel to the axis of decentration. This result suggests that the light that escapes from each cone and continues on to be absorbed by a subsequent receptor is reflected in many directions before it leaves the cone (see their figure 8). 17
Figure 9: A schematic diagram showing that obliquely incident rays pass through multiple photoreceptors and produce a meridional smearing of the image for contours oriented orthogonal to the axis of pupil decentration. In this example, oblique rays are from a horizontally displaced pupil, and while the horizontal contour is unaffected, the image of the vertical contour has been smeared across several receptors. Campbell proposed this model to explain a reduction in VA that occurred with pupil decentration.
In summary, there are five clear hypotheses to explain the reduction in contrast sensitivity and visual resolution that accompanies decentered pupils: 1. Reduced retinal illuminance because of the SCE; 2. Coma and spherical aberrations (proposed by Green); 3. Oblique rays spreading to several photoreceptors because of the axial length of the receptors (Campbell and Gregory); 4. TCA creating a meridional spectral "smearing" of the retinal image (van Meeteren, Dunnewold, and Thibos); 5. Off-axis astigmatism due to the oblique incidence of the ray bundle.
Both the theoretical modeling and the experimental data point to TCA as the most important factor, but it is hard to explain the reduction in VA reported by Campbell and Gregory 9, 10 and Chen and Makous 17 for monochromatic stimuli without postulating a retinal direction effect. Coma and astigmatism perhaps contribute a small amount, but because of the large depth of focus afforded by the small pupils used by Green 26 and by Marcos, 36 these aberrations probably had a small effect. But in practice, with generally much larger pupils, the coma and astigmatism would be expected to play a more significant role.
Fortunately, recent studies 46, 54 have shown that the center of the eye's pupil is usually close to the visual axis and therefore, under normal viewing conditions, TCA is small near the fovea. However, TCA grows linearly with eccentricity from the achromatic axis and the rate of change in TCA across the retina is determined by the axial location of the pupil. 71 The pupil location the human eye appears to minimize foveal TCA and thus optimize polychromatic image quality at the fovea.
In addition to contrast demodulating effects observed with polychromatic stimuli, there are other very pronounced effects of TCA observed when viewing colored targets. For example, it has been known for over 100 years that a simple optical model of TCA could account for the commonly observed color stereoscopy or chromostereopsis effect. When viewing equidistant, spatially adjacent objects of different colors (or wavelengths), most people report that the red objects appear closer than blue or green objects. Einthoven in 1885 22 realized that a horizontal interocular difference in monocular TCA created by unequal pupil decentrations (e.g. to the right in one eye and to the left in the other) would introduce a horizontal difference in the disparities of a long and a short wavelength target (Figure 10).
Figure 10: Schematic diagram showing how bi-temporal pupil decentrations can lead to an apparent difference in distance (d) because of the interocular differences in monocular TCA. In this example, when viewing a red and a blue target at the same distance, the red will be imaged more temporally on the retina than the blue. Perceived visual direction and depth is found by back-projecting the red and blue retinal images into object space through the nodal points (N). In this example, it becomes clear that the red target will appear closer than blue, and the converse would be true for bi-nasal pupils. Most observers report seeing red slight closer than blue and this probably reflects a bias in the population for the eye's pupil to be very slightly temporal relative to the visual axes.
We have tested Einthoven's hypothesis, and our experiments show that the color depth effect can be predicted precisely from the interocular differences in TCA. 67 The data in Figure 11a show the monocular TCA for the right and left eyes of a subject as an artificial pupil is moved temporally or nasally. The interocular difference in horizontal TCA predicted for binocular viewing is simply the vertical separation between these two data sets. In Fig. 11b we show the calculated and experimentally measured chromostereopsis. It is clear that Einthoven's model precisely predicts the observed color depth "illusion". However, as described in Fig. 4, this geometrical optics explanation must be modified by the SCE anchoring effect, 58 which has an increasing effect with increasing pupil size. 68 The angular horizontal disparity (the ordinate in Fig. 11b) can be used to calculate the linear depth error introduced by TCA using the following approximate equation for small angles:
 |
|
where 
is the angular disparity, 2a is the interpupillary distance,
D is the viewing distance, and d
is the linear depth interval. It is important to emphasize that these
predicted depth errors are not small, but extremely large. For
example, ten arc minutes of disparity will create a 45 cm depth error
when viewing a target at three meters. For spectrally pure stimuli
viewed through small pupils, this sort of huge depth error can
therefore be produced by an opposite lateral displacement of the
right and left eye's entrance pupil by as little as 1mm (see
Fig. 11b).
Figure 11: The left panel plots experimental data from one of the authors (AB) obtained with a small artificial pupil displaced nasally or temporally from the visual axis. For each location, ocular TCA was measured using a color vernier alignment procedure. 67 The two lines of the vernier target were 433nm and 622nm. The predicted chromostereopsis (solid line in the right panel) is simply the interocular difference between the right and left eye TCA data. The circles in the right panel are the experimentally measured chromostereopsis.
The color depth effect can be quite pronounced for broad spectrum stimuli. 65 For example, consider a red, green and blue phosphors on a typical TV monitor. Experimentally measured TCA for these stimuli are not as pronounced as observed between two monochromatic stimuli from the margins of the visible spectrum (e.g. Fig. 11), but large differences in apparent location of red and blue targets on a TV are produced when viewed through a displaced pupil (Fig. 12). This Figure provides an indication of the magnitude of the color misalignments that can be introduced with standard colored stimuli. As a worse-case example, the 6 arc minute of measured TCA produced between Red and Blue stimuli viewed through artificial pupils near the margins of the iris is approximately 100 times the typical vernier alignment threshold. This amount of TCA can introduce very significant errors into any alignment task that might involve two different colored targets. When viewed binocularly, this degree of asymmetric pupil decentration could introduce 12 arc minutes of horizontal disparity which, for an average person, corresponds to an error of about 2/3 meter in depth judgments at a 5 meter viewing distance.
Figure 12: Transverse chromatic aberration of the human eye for targets displayed on a video monitor using either the red, blue, or green phosphors. TCA was measured as the amount of offset of two line segments required to make them appear collinear when viewed monocularly through a displaced, artificial pupil (zero displacement corresponds to alignment of the pupil on the visual axis). The two lines in this vernier alignment task were either red and green (filled circles), green and blue (open circles) or red and blue (open squares).
The importance of ocular TCA for polychromatic image quality with
decentered pupils, and its significant impact on the perceived
location and depth of colored targets, has prompted us to develop a
computationally simple optical model of the eye that accurately
predicts the experimentally observed chromatic aberration. Our
single-surface model eye, dubbed the Chromatic Eye, 51
has an elliptically shaped refracting surface separating air from an
internal aqueous fluid which has slightly more chromatic dispersion
than water. The refractive index 
of this fluid is
 |
|
where a=1.31848, b=0.006662 and c=0.1292. Using this model, we were able to accurately model the experimentally observed effect of pupil decentration on foveal TCA in human eyes as well as the axial chromatic aberration. 51 We specifically employed an elliptical surface to avoid any spherical aberration. Subsequently we modified the model eye to include a degree of spherical aberration typical of human eyes. 52, 69 This was done by changing the cross-section of the refracting surface from an ellipse to a polynomial as defined by the 4th order equation
 |
|
This profile lies between a circle (which has too much spherical aberration) and an ellipse (which has too little spherical aberration). Refractive index was similar to the previous model,
 |
|
where a=1.320535, b=0.004685 and c=0.214102. This model, called the Indiana Eye, accurately predicts both the eye's chromatic and spherical aberrations .
Application of the single-surface model to the problem of ocular chromatic aberration revealed the optical equivalence between displacing a target and displacing the pupil. 55 In both cases the target is being displaced from the achromatic axis of the eye, but in one case the target moves and in the other case the axis moves. To see this, imagine the result of moving a target in Fig. 2 off the achromatic axis. As the target is moved farther into the periphery, the angle of incidence of the chief ray departs more and more from the normal, resulting in chromatic dispersion. If, on the other hand, the target remains fixed and the entrance pupil is displaced, the achromatic axis of the optical system shifts because the achromatic axis, by definition, goes through the center of the pupil. Shifting the achromatic axis away from the target again increases the obliquity of the incident ray bundle, causing increased chromatic dispersion of a polychromatic chief ray. To quantify this equivalence relationship, Thibos et al 55 showed that the angle of incidence of the chief ray (and hence the amount of ocular TCA) was the same for eccentric targets and for decentered pupils when
 |
|
where H is the decentration of the pupil and 
(in radians) is target eccentricity. The gain of this relationship is
controlled by the axial separation NP between the entrance
pupil and the nodal point.
The previous sections of this chapter have emphasized the impact of pupil decentration on foveal image quality. In this section we will examine the optical impact of off-axis targets. As noted above, many of the optical changes that accompany target decentration are equivalent, or at least qualitatively similar, to the optical effects of pupil decentration. In both cases the optical beam strikes the eye's optics with increasing obliquity and therefore the optical aberrations that accompany oblique rays (e.g. TCA, spherical aberration, coma, astigmatism) will affect retinal image quality. Despite this equivalence, the optical consequences of moving targets into the peripheral visual field are often dismissed as being unimportant since the spatial resolving power of the peripheral retina is so poor in comparison with foveal resolution. This dismissive attitude is changing, however, following the discovery of useful vision beyond the classical resolution limit in the periphery (see the chapter by Thibos & Bradley in this book) so it is worth reviewing the topic here.
There are few studies of the eye's optical system for peripheral targets, but early examination of peripheral refractive errors highlighted the significant differences between foveal and peripheral optics. For example, Ferree, Rand and Hardy 24 measured peripheral refractive error in 21 subjects out to 50 degrees in both the nasal and temporal fields. Their results, which have been replicated by others, 41 show that a very large amount of astigmatism develops in the far periphery (increasing to as much as 8 diopters at 50 degrees). As predicted by optical models, 20, 35, 60 most subjects had increasing peripheral astigmatism with increasing eccentricity with the tangential focus being myopic and the sagittal focus being hyperopic. In some subjects, both meridians were hyperopic, but again the sagittal meridian was more hyperopic. In general, real eyes tend to have much less peripheral astigmatism than spherical model eyes predict. 20, 35 This can be explained by the asphericity of the cornea and lens. 20 The location of the two foci (tangential and sagittal) will be influenced strongly by the shape of the eye-ball, which in turn is related to the spherical refractive error. This relationship was confirmed by Millodot, 39 who showed that, in more myopic eyes, both the tangential and sagittal meridians became more hyperopic than the fovea with increasing eccentricity This reflects the lager reduction in peripheral posterior nodal distance in axially myopic eyes.
The oblique incidence of the rays forming retinal images of peripheral targets should lead to increased TCA 55 but experimental verification of this prediction has proved difficult. Experiments in our labs 18 have shown that the detection of peripheral polychromatic interference fringes varies with their orientation in a systematic fashion. Polychromatic fringes are easiest to detect when they are radially oriented (i.e. their orientation is parallel with a line connecting the fovea with the center of the target) and they are most difficult to detect when oriented tangentially. This behavior can be accounted for quantitatively using the Chromatic Eye model described above. However, using ordinary targets viewed through natural pupils, Ogboso and Bedell 44 failed to observe TCA consistent with any optical predictions. Also, Zhang et al 71 , found only very small amounts of peripheral TCA. However, as predicted by a simple model, 72 the amount of peripheral TCA is amplified significantly if the natural pupil is substituted with an artificial pupil placed anterior to the eye (the typical location). 71 This should be a serious consideration in the design of any visually coupled instrument that creates an exit pupil.
Experimental measures of retinal image quality for peripheral targets have shown that image quality does deteriorate with increasing eccentricity. 33, 34, 43, 48 However, if peripheral refractive errors are corrected, image quality within the central 40 degrees (+/- 20 degrees) of the visual field remains uniformly high. 43, 48, 49
Although peripheral aberrations can be very large, their impact on peripheral vision depends upon the visual task and whether known refractive errors are corrected with spectacles or contact lenses. As noted above, the detection of high-frequency, polychromatic fringes is optically limited and this holds as well for natural viewing of polychromatic gratings. However, this optically-imposed cutoff frequency for contrast detection is far beyond the Nyquist limit for spatial resolution imposed by the relatively coarse sampling density of neurons in peripheral retina. 56 Consequently, the optical quality of the well-focused human eye is not a limiting factor for peripheral resolution tasks such as reading letters or identifying the orientation of sinusoidal grating targets viewed eccentrically. 27, 40 On the other hand, if refractive errors are left uncorrected, they may be large enough to limit resolution, and the effect will likely vary with target orientation because of the large off-axis astigmatism present in human eyes. 24, 39, 41
In addition to the increased off-axis optical aberrations that accompany target decentration, there are changes in retinal illuminance and distortions of the image. Fortunately, the combined effects of reduced pupil diameter and reduced posterior nodal distance with its decreased magnification (mm2 per deg2) may combine to provide almost constant retinal illuminance across all but the very far periphery, 15 but retinal illuminance will be lower in the very far periphery (>70 degrees).
The fortuitous alignment in most eyes of the visual axis, the pupil, and the SCE provides an approximate joint optical and neural axis for the human eye. Although some issues remain unresolved, it appears clear that deviations from this axis will lead to reduced optical quality of the retinal image, primarily due to transverse chromatic aberration and off-axis astigmatism. These aberrations will accompany both pupil and target decentration, but the visual ramifications of pupil decentration will generally be more pronounced because it will affect foveal vision.
In light of the above summary and the chapter on peripheral vision by Thibos and Bradley in this book, it becomes clear that optimal visual performance requires a well centered target and a well centered pupil. Target centration is usually well controlled by fixational eye movements that can maintain a small fixated target within the fovea and a large fixated target centered on the fovea. Also, when the eye's iris defines the entrance pupil for the eye, the entering optical beam will be well centered on the visual (foveal) axis. However, if an intervening visually coupled device employs a small exit pupil which is not centered on the visual axis, the optical problems summarized above would affect foveal image quality and hence foveal vision. Visually-coupled instruments that employ exit pupils larger than the natural pupil will be assured of a well centered optical path if the exit pupil completely covers the eye's entrance pupil. However, instrument decentration or relative movement between the head and instrument may lead to partial coverage, and hence the effective entrance pupil will be decentered. It is clear that any visually-coupled electro-optical device should employ an exit pupil large enough to ensure that it always covers the eye's entrance pupil.
The ideas presented in this paper have developed in parallel with the dissertation research of David Still, Xioaxioa Zhang and Ming Ye at Indiana University School of Optometry. Their work and ours has been supported by National Institutes of Health grants EY05109 and EY07638, and by a grant to the Indiana Institute for the Study of Human Capabilities provided by the USAFOSR.