1.

Introduction

Helmet-mounted displays (HMDs) provide pilots the ability to maintain a “head-up, eyes-out” awareness of the ambient scene during most flight maneuvers.^1,2 HMDs may provide symbology, other kinds of navigational aids such as synthetic imagery, and/or sensor imagery during day and night operations. For see-through displayed imagery to be legible and intelligible, it must be of sufficient contrast to stand out from the background, that is, to appear as an overlay upon the ambient scene. HMD imagery must be of sufficient luminance that when combined with the background (the background scene adds to the HMD imagery), it can be seen and interpreted with certain assurance that relevant visual information is communicated without visual ambiguity. As color coding is introduced into HMDs, it is important to define the daylight requirements that will yield easily distinguishable symbology colors against any ambient scene. Assigning symbology colors to represent essential aircraft, situational, or tactical information must be done with a clear understanding of the additive nature of see-through displays.

Given the additive nature of HMDs and head-up displays, two important questions regarding color discrimination arise. First, under what combination of contrast and/or background conditions results in a symbology color no longer resembling the hue of the displayed color (e.g., what conditions result in a red symbol no longer appearing as reddish in color). Second, under what combination of contrast and/or background conditions results in two symbology colors being mistaken for one another (e.g., what conditions result in red and green symbols appearing as the same color).

The findings presented here attempt to answer the second question. Symbology colors were evaluated as a function of luminance contrast based on the modeling of normal color vision observers as well as color deficient observers.^3,4 A color discrimination metric based on a prescribed CIELAB distance was used to define confusion thresholds for symbology as a function of background color and luminance contrast. The prescribed distance measure was determined based on curve fits to the primary author’s color matching observations. The confusion contrast thresholds were used to further address daylight luminance guidelines for see-through display systems. In the process of development of these guidelines, an error was discovered in Harding et al.¹ and Harding and Rash² previous calculations, and that error has been rectified here.

2.

Color Mixing of Symbology with Background

When two colors are added together in CIE $Yxy$ color space, the resulting color falls on a line connecting the two additive colors and the new colors luminance ($Y$) is equal to the sum of the two luminances. Since colors presented in a see-through HMD are additive with the ambient background, the chromaticity coordinates, $x$ and $y$, resulting from color mixing were derived from the summation of each contributing colors tristimulus values (i.e., $X_3=X_1+X_2$, $Y_3=Y_1+Y_2$, and $Z_3=Z_1+Z_2$), where $X=x·(Y÷y)$ and $Z=(Y÷y)·(1-x-y)$.⁵ Hence,

To evaluate color discrimination over a range of luminance contrasts, seven symbology colors representing the three primaries and three secondaries along with white (Table 1) were mixed with 14 background colors representing 0.1 grid intersections in the CIE $Yxy$ color space contained by the sRGB gamut (Fig. 1) plus D65 white.

Table 1

Seven symbology colors used in the color mixing modeling. Each color is described by its RGB (24-bit color mode with 8 bits per color) and CIE xyY values.

Fig. 1

Seven symbology (large black circles) and fourteen background colors (small white circles) used in the color mixing. The sRGB gamut is outlined by the dashed line triangle.

Figure 2 shows chromaticity coordinates for the seven symbology colors mixed with a background chromaticity of ($x=0.3$, $y=0.2$) as a function of Michelson contrast $[(L_{\text{max}}-L_{\text{min}})÷(L_{\text{max}}+L_{\text{min}})]$ ranging from 0.05 to 1.00 in 0.05 increments. The symbology colors all merge toward the background hue as contrast is reduced.

Fig. 2

Chromaticity coordinates of the seven symbology colors merging toward a background chromaticity of ($x=0.3$, $y=0.2$) as luminance contrast is reduced. For each symbology color, 20 points are plotted corresponding to a range in luminance contrast from 0.05 to 1.0 in 0.05 increments.

3.

Symbology Color Discrimination

Figure 3 shows the three-dimensional CIELAB distances measured between each of the seven symbology colors as a function of contrast for the chromaticity data shown in Fig. 2. CIELAB is more perceptually uniform than Yxy color space and is, therefore, more applicable for color discrimination modeling. After computing the $x$, $y$ chromaticity coordinates for mixed colors, three-dimensional CIE $L*a*b*$ distances were calculated. By holding luminance relatively constant, the difference between the three-dimensional $L*a*b*$ distances and the two-dimensional $a*b*$ distances were negligible. Also plotted in Fig. 3 is the confusion contrast threshold criteria of 13. The threshold was determined by least squares fitting of three-dimensional CIELAB distance criteria to the principal author’s perceptual thresholds (e.g., Fig. 5) for each combination of symbology color and background. To aid in the perceptual assessment, color charts were created for each of the 14 background colors. Figure 4 shows the color chart created for the (0.3, 0.2) background. Symbology plus background color mixing chips were created for the 20 contrast values ranging from 0.05 to 1.0.

Fig. 3

CIELAB distance measures for the data shown in Fig. 2. Six series distance measures are shown for each of the seven symbology colors. The color of the filled circle identifies the colors of the seven symbology colors. Also plotted is the confusion contrast threshold criteria of 13 (dotted lines). Some distances fell below a CIELAB distance of 1.0 and are not plotted in this figure.

Fig. 4

Color mixing charts for background set to chromaticity coordinates (0.3, 0.2). The top row of colors is set to the pure (i.e., not mixed) background color and the bottom row is set to each of the pure symbology colors from Table 1. The other rows show color mixing hues for a range of luminance contrasts from 0.05 to 0.95. Each color chip is representative of the corresponding data points shown in Fig. 2.

At a given contrast level, it is important to determine if the seven symbology colors can be distinguished from each other. In a see-through display, perceptual mistakes may have significant consequences; therefore, sufficient luminance contrast must exist to correctly distinguish all symbology hues from each other on a given background. Using a calibrated color monitor in a dimly lit room, the color mixing charts were evaluated visually to assess the contrast where the two color chips were quite similar in hue. The first author subjectively evaluated each of the 14 charts representing the 14 chosen backgrounds. He did not look for precise color matches, but rather for close similarities in hue where perceptual mistakes could surely be made, especially in situations where split second decisions must be made and symbology pairings are not available for comparisons. Using the color chart in Fig. 4, 40% confusion contrast thresholds were assigned to the Sym1/Sym3, Sym4/Sym7, Sym2/Sym5, and the Sym3/Sym6 pairings. A confusion contrast threshold of 45% was recorded for the Sym1/Sym6 pairing. All of the confusion contrasts thresholds for the (0.3, 0.2) background are shown in Table 2 and Fig. 5 shows the least squares fit to these data. The $R^2$ value shown in Fig. 5 was the highest found for any background as the average $R^2$ value was 0.67 with an average best fit CIELAB distance criterion of 13.3. $R^2$ is the correlation coefficient between the data points and the linear fit to the data (solid line in Fig. 5). The lowest $R^2$ values were for backgrounds (0.4, 0.5) and D65 and these backgrounds also had two of the lowest average perceptual confusion contrast thresholds. It is important to point out that threshold values were not interpolated, rather they were recorded as the highest contrast calculated at a CIELAB distance of 13. Often the threshold was 0.0%, which is clearly the result of the applied contrast increment spacing. With sufficiently small contrast increments, all thresholds would be $>0.0\%$.

Table 2

Confusion contrast thresholds for a background of (0.3, 0.2) determined by CIELAB distance measures (numbers in italics) and perceptual judgments (numbers in bold).

Fig. 5

Scatter plots showing the relationship between the primary author’s visual assessment of confusion contrast thresholds for the color mixing chart shown in Fig. 4 and confusion contrast thresholds based on a CIE $L*a*b*$ distance criterion of 13.0. Nine of the 21 comparisons were represented by a single point (5%, 5%), three at (5%, 0%), two at (40%, 40%), and two at (40%, 45%).

4.

Symbology Color Discrimination in Color Deficient Observers

In the general population, about 8% of males and $<1\%$ of females are color vision deficient (CVD). Of these, about 25% are dichromats and the remainder are anomalous trichromats characterized by different photopigments in cones that respond to medium and long wavelength light (i.e., red-green anomalous trichromats) or a defective S-cone (i.e., blue-yellow anomalous trichromats).⁶ Only about one in 1000 CVD individuals have a defective or an absent S-cone. Recently, Machado et al.³ developed a color vision model based on photoreceptor spectral absorption and color opponent processing at a later stage in the visual system. The model appears to offer advantages for evaluating different severities of color vision deficiencies based on a scale of zero to one. Zero, as in zero deficiencies, is representative of normal color vision, whereas a score of one represents a dichromat. Scores between 0 and 1 represent the severity of a diminished chromatic discrimination. For red-green color blindness, the model simply shifts the anomalous L- or M-cone photoreceptor absorption spectrum toward the normal M- or L-cone absorption spectrum. For each 10% reduction in the gap between the spectrums, it relates to a severity increase of 0.1. Thus a protanomalous score of 0.5 is representative of a shift of the L-cone photoreceptor absorption spectrum toward the M-cone spectrum to half of the normal separation. Fortunately, the authors provide a simpler method for calculating and visualizing the perception of CVD colors. In an appendix to Machados thesis,⁴ matrix coefficients are provided for calculating CVD RGB values for the entire range of CVD observers.

Matrix coefficients for protanomalous, deuteranomalous, and tritanomalous observers with severity indices of 0.1 to 1.0, in 0.1 increments, were applied to the RGB values for each of the 14 background color mixing charts. The color defective RGB values were then converted to CIELAB values and color differences for the color defectives calculated. Using the same CIELAB distance criterion of 13 permitted the calculation of confusion contrast thresholds for the entire range of CVD observers. In nearly all cases, CVD thresholds were higher than normal thresholds. Where CVD thresholds were lower, they were only 5% lower. Figure 6 shows average confusion contrast thresholds as a function of CVD severity. Each data point represents the average contrast for the 21 symbology comparisons across all 14 backgrounds. Hence, each point represents the average of 294 contrast calculations. At a severity score of zero (normal color vision), the average was 0.074 based on contrasts over the range of 0 to 0.75. The average CVD contrasts were all higher than the average normal observer contrasts. The largest increases in contrasts were observed in the protanomalous scores.

Fig. 6

Average confusion contrast thresholds for the 14 backgrounds as a function of CVD severity. Zero severity represents normal color vision and a score of 1.0 represents a dichromatic observer. Scores from 0.1 to 0.9 represent severity of color deficiency for anomalous trichromats.

Figure 7 shows confusion contrast thresholds for all symbology and background conditions for a normal observer and for a deuteranomalous observer with a severity score of 0.5. To show complete data sets, the tables by necessity become somewhat congested, but hopefully the color coding and organization will simplify understanding. Each symbology color (top row) is compared with all other symbology colors (second row) as a function of each of the 14 background colors. There is a 100% built-in redundancy in the tables, as each symbology color is compared against all other symbology colors in order to accurately reflect statistical averages across colors. For example, the confusion thresholds between Sym1 and Sym2 are shown in the first column under the Sym1 comparisons (overall column five) and the first column under the Sym2 comparisons (overall column eleven). The far right column shows the averages for each of the 14 background colors. The second row from the bottom shows the average thresholds for each of the symbology comparisons, whereas the bottom row shows the average thresholds, across six columns, for each of the seven symbology colors.

Fig. 7

Threshold confusion contrast percentages for a color normal observer and an anomalous deuteranope with a severity score of 0.5. The two tables represent symbology comparisons for each background color. The 14 background colors are shown in the fourth column. Each symbology color is represented in the top row along with its comparison with each of the other six symbology colors (second row from top). The rightmost column and the second row from the bottom show the mean percentages for each row and column, respectively. The bottom row shows the means across six columns representing the averages for each of the seven symbology colors.

Examining the color normal data, the greenish backgrounds [chromaticity coordinates: (0.3, 0.4); (0.3, 05); and (0.3, 0.6)] appear to provide little interference with symbology color discrimination with confusion threshold averages of 2%, 2%, and 0%, respectively. Likewise, the two yellow colors along with D65 [chromaticity coordinates: (0.4, 0.4), (0.4, 0.5), and (0.313, 0.329)] also appeared to provide little interference with discrimination with confusion averages of 5%, 1%, and 4%, respectively. With yellow, white, and off-white backgrounds, the symbology colors simply appear desaturated. The magenta and blue backgrounds provided the greatest interference [chromaticity coordinates: (0.2, 0.1), (0.2, 0.2), (0.3, 0.2), (0.4, 0.2), and (0.5, 0.3)] with average confusion thresholds of 17%, 10%, 18%, 15%, and 9%, respectively.

From a symbology color perspective, Sym1 (red) had the lowest confusion contrast average (5.48%) although it had a fairly high confusion rate of 16% with Sym6 (magenta). White symbology had the highest average confusion contrast threshold of 8.81% followed closely by yellow with 8.51%. The highest average confusion contrast thresholds were between Sym3 (blue) and Sym6 (magenta) with an average of 24% followed closely by Sym5 (cyan) and Sym7 (D65 white) with 22%.

Examining the confusion contrast thresholds for the moderate deuteranomaly in Fig. 7, the overall pattern appears similar to the color normal, albeit with higher thresholds. The three greenish backgrounds [chromaticity coordinates: (0.3, 0.4), (0.3, 05), and (0.3, 0.6)] had the lowest average thresholds of 5%, 3%, and 1%, respectively; the magenta and blue backgrounds had the highest thresholds ranging from 13% to 27%. The modeling shows that for moderate deuteranomaly, discriminating between yellow (Sym4) and green (Sym2) symbology would be extremely troublesome with an average confusion contrast of 50% across the 14 backgrounds. Overall, comparing the thresholds in Figs. 6 and 7 between the moderate deuteranomaly (0.5 severity) and the color normal observer, the deuteranomalous observer is 64% more likely to confuse symbology colors. Moreover, a moderate protanomalous observer is 100% more likely to confuse symbology colors (Fig. 6).

5.

Discussion

As color displays are integrated into military and civilian head- and helmet-mounted display systems, it is important to understand the perceptual benefits and limitations that color coding may provide. Here we examined color symbology discrimination as a function of luminance contrast.

Seven symbology colors were mixed with fourteen different background colors over the entire contrast range. Discrimination was based on distance measurements between symbology in CIELAB space.

5.6.

Luminance Guidelines for Color See-Through Displays

Harding et al.¹ and Harding and Rash² derived an envelope equation that described the highest probable amount of luminance complexity contained within a natural scene as a function of ambient luminance. Luminance complexity, the standard deviation of pixel luminances, was calculated for small patches of pixels (e.g., $10\times 10$) within the scene. Greater than 99% of the calculated percent standard deviations fell below the curve describing the envelope:

Eq. (1)

$$B_{\mathrm{SD}}=[-1.0\ln (L_B)+8.7]·100\%\text{over the range}1\text{to}6000\mathrm{fL},$$

where $B_{\mathrm{SD}}$ is the percent standard deviation of the background luminances and $L_B$ is the background ambient luminance. Equations (2) through (4) from Harding and Rash,² described minimum, average, and good Michelson contrast as a function of $B_{\mathrm{S}\mathrm{D}}$. The equations were derived from observer rankings, in a suprathreshold psychophysical experiment, of the quality of white symbology contrast overlaid over eight natural scenes, one artificial scene (high contrast and complexity), and one spatially uniform scene. The equations are shown below:

Eq. (2)

$$\text{minimum contrast}=0.06+0.58B_{\mathrm{SD}},$$

Eq. (3)

$$\text{average contrast}=0.18+0.58B_{\mathrm{SD}},$$

Eq. (4)

$$\text{good contrast}=0.36+0.58B_{\mathrm{SD}}.$$

${\mathrm{R}}^2$ values for Eqs. (2)–(4) were 87.8%, 89.3%, and 83.6%, respectively. With these equations, the authors plotted the display luminance requirements for symbology for the three contrast conditions. An error was made when plotting the luminance of the symbology ($L_{\text{sym}}$). Since $L_{\text{sym}}$ is equal to $L_{\text{max}}-L_{\text{min}}$, where $L_{\text{min}}$ is the luminance of the background or ambient scene, calculating Michelson contrast becomes $(L_{\text{max}}-L_{\text{min}})÷(L_{\text{max}}+L_{\text{min}})=L_{\text{sym}}÷(L_{\text{sym}}+2·L_{\text{min}})$. Unfortunately, in Fig. 6 of Harding et al.¹ and Fig. 8 of Harding and Rash,² the plotting was based on calculation of contrast using a faulty equation $[L_{\text{sym}}÷(L_{\text{sym}}+L_{\text{min}})]$; the denominator was missing the addition of a second $L_{\text{min}}$. $L_{\text{sym}}$ is replotted here (Fig. 8) based on Eqs. (1)–(4) with the following conditions. When calculating contrast using Eqs. (2)–(4) for low luminance and high spatial complexity as specified in Eq. (1), Michelson contrast values $>1.0$ are required, which clearly are not possible. Thus it is necessary to set an upper limit on Michaelson contrast. In the original research,^1,2 the average ${\mathrm{B}}_{\mathrm{SD}}$ for the eight natural scenes and one artificial background did not exceed 80%. Setting ${\mathrm{B}}_{\mathrm{SD}}$ to 80% in Eqs. (2)–(4) provides a contrast maximum of 0.524, 0.644, and 0.824 for minimum, average, and good contrast, respectively. Using these values, the three luminance curves in Fig. 8 all peak at an $L_B$ of 2700 fL. The linear rise in each curve from zero to 2700 fL is due to their respective maximum contrast limits. The three curves peaked at $L_{\text{sym}}$ luminances of 5931, 9744, and 25,180 fL for Eqs. (2)–(4) respectively. Of course, these values will be reduced based on optical densities of windscreens, visors, and HMD combiner lenses.

Fig. 8

Applying Eqs. (2)–(4) to the limit curve. The left vertical axis applies only to the limit curve. The linear rise in the luminance curves was due to BSD limits placed on each contrast condition (see text for details).

In several symbology pairings with particular backgrounds, extremely high-contrast confusion thresholds were found for both color normal and color deficient observers. An interesting question is whether or not the present data could possibly increase the luminance requirements generated by the three curves shown in Fig. 8. To answer this question, it requires an assessment of each backgrounds likely peak ambient luminance in the real world. Not having data that directly addresses background hue encounter rates for different geographical locations, seasons, and so forth, a 6000 fL peak could be used with background luminances adjusted for sRGB scaling. Using this approach, Fig. 9 shows the possible peak ambient luminance for each of the 14 background colors. Also plotted are the peak confusion contrast thresholds calculated for color normal and deuteranomalous observers (Fig. 7).

Fig. 9

Calculated ambient luminance (bar chart; luminance based on sRGB scaling with 6000 fL peak) for the 14 background colors. Background colors follow the order shown in Fig. 7 and each bar color is indicative of its chromaticity color. The dashed curves show the maximum confusion contrast thresholds (closed symbols = normal color vision, open symbols = deuteranomaly with 0.5 severity score).

Using these contrast and luminance data, $L_{\text{sym}}$ values can be calculated that would provide sufficient contrast to exceed the confusion contrast thresholds for each background. Figure 10 shows the $L_{\text{sym}}$ values calculated using the maximum confusion contrast thresholds. For the color normal observer, the highest $L_{\text{sym}}$ (15,617 fL) was for background (0.2, 0.1). $L_{\text{sym}}$ values for all other backgrounds were $<\mathrm{10,000}\mathrm{fL}$. For the deuteranomalous observer, the background at (0.5, 0.4) resulted in the highest luminance (31,612 fL). This high luminance for the CVD exceeds even the good contrast luminance figure calculated from the data of Harding et al.,^1,2 based on suprathreshold assessment of white symbology against natural scenes (Fig. 8).

Fig. 10

Maximum luminance requirements for the 14 background colors based on maximum confusion contrast ratios and luminance values shown in Fig. 9. Closed and open symbols are the same as in Fig. 9.

Based on the results presented here, symbology color confusion as perceived in daylight see-through display systems may be problematic, especially with certain background colors. The average and good contrast maximum luminance requirements [Fig. 8; peak luminances based on Eqs. (3) and (4)] exceeded the color confusion threshold luminance requirements, for color normal observers (Fig. 10), for all conditions with the exception of the luminance requirement for the background at (0.2, 0.1). The luminance for this background exceeded the 9744 fL average contrast requirement (Fig. 8). Compare this with the results for the deuteranomalous observer where nine backgrounds required greater luminance than the average contrast requirement and two exceeded the good contrast requirement. Because of these findings, increasing daylight luminance requirements, above and beyond the average, and good contrast luminance requirements presented here, may be necessary to accommodate CVD observers.