2.1.

Field and Crop Selection

Experiments were conducted in in-vitro conditions at Eco-Park, Tseung Kwan O, Hong Kong from Jan to Jun 2016 (Fig. 1). Lettuce (Lactuca sativa) was selected as the test crop because of its prevalence throughout Asia, its relative ease of growth, and its potential role as a high-yield food source.¹⁹ Lettuce was also selected for its ability to showcase the high spatial resolution of the proposed imaging system. This plant has a broad leaf area when viewed from above, with growers tending to maintain a distinct separation between individual lettuce heads.²⁰

Fig. 1

Satellite view of the region around Eco-Park in Hong Kong, with the red circle indicating the precise location of the lettuce plot. The inset shows the position of Eco-Park within Hong Kong. Photo credit: Google Earth.

Lettuce seedlings were germinated for 4 weeks and then planted into a $1.5\mathrm{m}\times 1\mathrm{m}$ sandy loam plot at Eco-Park, with a planting density of 9 plants per square meter. The lettuce grew under natural illumination, facing natural weather conditions in an outdoor environment, reproducing the conditions under which real crops grow. The lettuce was subjected to two conditions: optimal healthiness (8 plants) and suboptimal healthiness (11 plants). The former condition, which will be referred to as “healthy,” was achieved with an irrigation level of 100% (i.e., watering every 2 days), fertilized every 10 days with a peanut-based fertilizer containing nitrogen, protein, phosphorous, potassium, and calcium. The latter condition, which will be referred to as “unhealthy,” was achieved with an irrigation level of 50% (i.e., watering every 4 days) without any fertilizer. The first images were collected 120 days after initial planting.

2.2.

Camera Conversion and Optical Filter Selection

Lettuce reflectance measurements were obtained by converting a commercially available point-and-shoot digital camera (Canon PowerShot A2200: $1/2.3\text{-}\mathrm{inch}$ CCD sensor with $4320\times 3240\text{pixels}$) into a multispectral imager sensitive to both visible and NIR light. For agricultural health monitoring, red wavelengths (640 nm) are generally preferred because of (i) their sensitivity to changes in plant health, (ii) the stronger absorption of blue light by chlorophyll and anthocyanin pigments, and (iii) the scattering of blue light by the atmosphere (i.e., Rayleigh scattering), which tends to reduce the signal-to-noise ratio.²¹

However, even when equipped with a red-pass filter, the red channel of most image sensors is usually contaminated with both red (640 nm) and NIR (700 to 1000 nm) light. This can be resolved with multicamera systems in which one camera is assigned to receive only red light and another is assigned to receive only NIR light. However, for low-cost single-camera setups, such as the one proposed here, a blue-pass filter can be used to separate red light from NIR light. Blue-pass filters are particularly suited for low-altitude imaging because the crop-to-camera distance can be made sufficiently short that signal contamination due to atmospheric Rayleigh scattering becomes negligible. For these reasons, the camera’s built-in infrared-reflecting hot mirror was removed, and a Roscolux #2007 blue-pass filter was installed in its place to separate blue transmission (450 nm) from NIR transmission (700 to 1000 nm). As Fig. 2 shows, this optical filter has low transmittance in the red wavelengths (600 to 700 nm) but high transmittance in the NIR (700 to 1000 nm), and blue (450 to 500 nm) wavelengths. Equipped with these modifications, the red channel of the image sensor becomes an NIR channel, which will be used in Sec. 2.5 to compute various vegetation indices.

Fig. 2

Transmittance of the Roscolux #2007 blue-pass filter, as measured in situ using a USB-650 VIS-NIR Red Tide spectrometer connected to SpectraSuite, both produced by Ocean Optics. A tungsten-filament lamp was used as the white-light source for reference illumination.

2.3.

White Balance Calibration

Most digital cameras have white balance settings that are optimized for red, green, and blue (RGB) inputs.²² With the hot mirror removed and the Roscolux #2007 blue-pass filter installed, the blue channel now receives both blue and NIR light. To compensate for this, a blue calibration image was taken as the reference of the white balance, so that the camera digitally diminishes the blue channel intensities while increasing the red channel intensities. Calibration of the white balance of the camera was performed with in-situ lighting conditions.

2.4.

Image Sampling

Spectral images of lettuce were taken under natural illumination, with the camera 0.5 m away from the plant leaf surface. The white balance was recalibrated with each measurement using a blue reference image, as per Sec. 2.3. The compact nature of this imaging system makes it ideal for top-down sweeps with low-altitude UAVs or side-on panoramas conducted on foot or on autonomous vehicles. To assess the sensitivity of this system, both top-down and side-on views were captured of each plant, as shown in Fig. 3. The side-on images were taken with the camera horizontal to the lettuce. The top-down images, representing typical UAV-based measurements, were taken with the camera tilted at 30 deg from the vertical.

Fig. 3

Diagram of the experimental setup showing two different camera angles.

2.5.

Selection of Vegetation Indices

As explained in Sec. 2.2, the proposed imaging system cannot distinguish between red and NIR light. This restricted our choice of possible candidates for the optimal vegetation index to those that use either blue or green light, as well as NIR light. Four candidate indices were selected from the literature, as shown in Table 1. Typical reflectance spectra for healthy and unhealthy lettuce are shown in Fig. 4. It can be seen that the blue reflectance changes little between health statuses, but that the NIR reflectance changes markedly. One of the most widely used vegetation indices, the NDVI, has in its numerator the intensity difference ($\mathrm{NIR}-\mathrm{RED}$). Therefore, the NDVI would be large for healthy lettuce but small for unhealthy lettuce, making it a particularly sensitive indicator of plant health.²³ By contrast, the blue-normalized difference vegetation index (BNDVI) uses blue light and the green-normalized difference vegetation index (GNDVI) uses green light, which means that neither would be as sensitive as the NDVI. Nevertheless, the NDVI cannot be used here because it requires a pure red channel, which is unavailable from the proposed blue-green imaging system. Consequently, only vegetation indices that use blue or green light were tested (see Table 1).

Table 1

Vegetation indices selected for evaluation on lettuce.

Fig. 4

Typical reflectance spectra for healthy and unhealthy lettuce (Lactuca sativa).¹⁰

2.6.

Image Processing

Figure 5 shows the steps in the image processing procedure. The first step was to automatically crop the lettuce from its background so that the calculated vegetation index pertained only to the lettuce itself [Figs. 5(i) → 5(ii)]. Next, the images were processed through a MATLAB script that calculated the vegetation index of each pixel and produced a spatially averaged index for the whole image [Figs. 5(ii) → 5(iii)]. These steps were then repeated for each of the four vegetation indices listed in Table 1.

Fig. 5

Steps in the image processing procedure: (i) → (ii) cropping of the lettuce plant and (ii) → (iii) computation of the vegetation index. In this example, BWDRVI values are shown on the colorbar to the right.

2.7.

Data Analysis: Supervised and Unsupervised Clustering

The vegetation indices computed in Sec. 2.6 were analyzed for plant health using an unsupervised EM clustering algorithm and a Gaussian mixture model.²⁷ This iteratively analyses the input data to find two distinct clusters blindly from a pooled set of healthy and unhealthy plant data.²⁸ With the assumption that the data obeys a normal distribution, Gaussian fits were assigned to the two clusters. The data were used as a training set for the EM Gaussian mixture model. This method is a generalizable procedure that accepts processed images of any given plant and returns a probability of it being healthy, provided that a training set for this particular species exists.²⁸ This is a process that eliminates bias and offers quantitative measures of plant health.⁹

The vegetation indices were passed into fitgmdist (a built-in function in MATLAB) to produce two Gaussian curves without any knowledge of the health status of each index value. The two curves enable probabilistic assignment of any index value to a cluster, as well as providing a measure of the contrast between healthy and unhealthy lettuce. For evaluation of the clustering quality, the healthy and unhealthy data for each vegetation index were also used in a supervised cluster analysis to manually create two Gaussian curves, providing a known baseline reference against which the unsupervised clustering results could be compared.

In summary, our objective is to determine the most accurate vegetation index for discerning optimally healthy lettuce from suboptimally healthy lettuce using the multispectral images captured with our low-cost camera system (Sec. 2.2). As will be discussed in Sec. 3, by selecting the index that provides the sharpest contrast between healthy and unhealthy lettuce, we were able to create an unsupervised EM Gaussian model for objective diagnosis of lettuce health.

3.1.

Comparison of Vegetation Indices

Comparing the unsupervised and supervised clusters (specifically, their means and variances) and the errors between them provides an objective indicator of the most accurate vegetation index to be paired with the proposed imaging system. This comparison is shown in Figs. 6Fig. 7Fig. 8–9. Of the four indices tested, the blue wide-dynamic range vegetation index (BWDRVI) is the most capable of discerning healthy from unhealthy lettuce, as indicated by (i) the close match between the unsupervised and supervised clustering curves, at both camera angles [Figs. 7(a) and 7(b) versus Figs. 7(c) and 7(d)]; and (ii) the distinct separation between the Gaussian peaks of the healthy and unhealthy lettuce [Figs. 7(a) and 7(c) versus Figs. 7(b) and 7(d)].

Fig. 6

Supervised and unsupervised clustering of BNDVI values. (a and b) The supervised clustering, (c and d) the unsupervised clustering, and (e and f) a histogram of bootstrapped means. The left column is for the side-on view, and the right column is for the top-down view.

Fig. 7

Supervised and unsupervised clustering of BWDRVI values. (a and b) The supervised clustering, (c and d) the unsupervised clustering, and (e and f) a histogram of bootstrapped means. The left column is for the side-on view, and the right column is for the top-down view.

Fig. 8

Supervised and unsupervised clustering of green chlorophyll index values. (a and b) The supervised clustering, (c and d) the unsupervised clustering, and (e and f) a histogram of bootstrapped means. The left column is for the side-on view, and the right column is for the top-down view.

Fig. 9

Supervised and unsupervised clustering of GNDVI values. (a and b) The supervised clustering, (c and d) the unsupervised clustering, and (e and f) a histogram of bootstrapped means. The left column is for the side-on view, and the right column is for the top-down view.

To further demonstrate the superiority of BWDRVI, we define an accuracy index (AI) that quantifies the ability of a vegetation index to correctly identify any lettuce data point as belonging to either a healthy or an unhealthy cluster:

For each of the four vegetation indices (Table 1), AI values were calculated by classifying the data as belonging to either healthy or unhealthy clusters on the basis of the maximum posterior probability.²⁹ In this way, an AI value was generated for each vegetation index, as shown in Fig. 10. It can be seen that images taken from the top-down perspective and processed through BWDRVI produce the most distinct clusters and can therefore be used to determine, with the highest accuracy, the health status of lettuce. Furthermore, when mixing top-down and side-on camera angles, BWDRVI still emerges as the most accurate vegetation index, which is consistent with the observations seen in Fig. 7.

Fig. 10

Accuracy index [Eq. (1)] for the four vegetation indices listed in Table 1, at two different camera angles (side-on and top-down views).

3.2.

Bootstrap Resampling

A bootstrap resampling analysis was performed with 20,000 resamples in order to gain further statistical confidence on the results. Subfigures (e) and (f) of Figs. 6–9 show the histograms of the means of the bootstrapped resamples. The close correspondence of the histograms in Figs. 7(e) and 7(f) to their respective Gaussian curves in Figs. 7(c) and 7(d) confirms that BWDRVI is indeed well suited for unsupervised cluster analysis of lettuce health.