1.

Introduction

Diffuse reflectance spectroscopy, where a broadband source illuminates a sample and the reflected spectrum is measured, is a well-trusted method for noninvasively characterizing biological tissues.^1,2 The diffuse reflectance signal is a function of the wavelength-dependent bulk optical properties of the tissue. With proper modeling, it is possible to estimate the concentration of specific chromophores in the tissue. The scattering properties, which also affect the diffuse reflectance signal, can also be estimated.

In the visible and near-infrared (NIR) wavelength ranges, the main absorbers in tissue are water, oxy- and deoxyhemoglobin, and melanin. Assessing the concentrations of these chromophores can be used to identify pathological changes in tissues. Such measurements have been shown to identify lesions in several tissues, including breast,^3,4 skin,⁵ oral cavity,⁶ bladder,⁷ esophagus,⁸ and the uterine cervix.^9,10 Reflectance mode spectroscopy has also be used to characterize other chromophores in tissue, such as bilirubin when trying to assess jaundice in neonates.¹¹ The main disadvantage of spectroscopy is that it is based on point measurements that average over a large area. Thus, a visual follow-up is often needed, particularly when clinical workflow requires selecting sites to biopsy.

The challenges posed by point measurements can be overcome by spectral imaging, in which images of the tissue of interest are captured at different wavelengths, resulting in a two-dimensional (2-D) array of spectral measurements. Each pixel has an associated spectrum that can be analyzed in search of pathological abnormalities, which allows clinicians to identify suspicious regions in the tissue for further examination.^12–14 Multispectral imaging has been primarily used in dermatological applications, to monitor portwine stain treatment,¹³ burns,¹⁵ and diabetic foot ulcers.¹⁶

Despite the advantages of multi- and hyperspectral imaging in obtaining clinically relevant information, few commercial systems that can assess oxygen saturation in images exist, and those that are FDA-cleared cost upward of $40,000 and are often difficult to use.^17–19 In contrast to these expensive, cart-filled systems from 15 years ago,^20,21 the revolution in consumer electronics has made low-cost cameras and LEDs ubiquitous, bringing down the costs significantly. Several groups and companies have attempted to build a low-cost multispectral imaging system that can quantify scattering and absorption in the tissue,^22–24 including some based on LED illumination at various wavelengths,^25–28 in which spectral data were validated against spectral measurements from existing systems. However, a full end-to-end analysis that shows both the ability to reproduce a broad spectrum in an accurate way and yields reasonable in vivo scattering and absorption measurements with low-cost hardware has been elusive, and there have been questions about the overall feasibility of using low-cost hardware to make clinically relevant optical measurements.

In this work, we present the end-to-end functionality of a low-cost multispectral imaging system that captures 13 high-resolution images at different illumination wavelengths. The system is based on an area-scan camera and low-cost LEDs, such that the bill of materials is less than $1000. Fitting algorithms that analyzed the spectral data estimated the amount of each absorber (oxy- and deoxyhemoglobin, water, and melanin), as well as the overall scattering properties of the tissue. To validate our measurements, we performed a series of standard experiments used to characterize spectral systems. First, we compared spectral data measured with our system to white-light spectrometer readings of the 24 different color tiles from a color calibration card. We then captured in vivo images of skin from healthy volunteers, and reconstructed whole spectra using a lookup table based on Monte Carlo simulations. Finally, we captured images of a normal skin before and after erythema, demonstrating the ability of the measurements to noninvasively assess a change in tissue properties. Our results show that building a low-cost imaging system that can measure scattering and absorption is indeed feasible and that with proper calibration and modeling, such a system can be adapted to a specific clinical application.

2.

Theory

A diffuse reflectance measurement on skin can be described by Ref. 29:

which are used to calculate the absorption coefficient:¹³

The absorption coefficients of oxy- and deoxyhemoglobin in whole blood [${\mu }_{a.\mathrm{oxy}}(\lambda )$ and ${\mu }_{a.\mathrm{deoxy}}(\lambda )$] are from Prahl.³¹ The absorption coefficient of water is from Hale and Querry,³² converted from the reported imaginary refractive index $n^{″}$ into ${\mu }_{a.\mathrm{water}}=4\pi n^{″}/\lambda [{\mathrm{cm}}^{-1}]$. The reported absorption coefficient of the interior of a cutaneous melanosome³⁰ is described as ${\mu }_{a.\mathrm{melanosome}}(\lambda )=(679.16{\mathrm{cm}}^{-1}){(\lambda /500\mathrm{nm})}^{-3.33}$.

The scattering properties of any tissue type are given as:

which are used to calculate the reduced scattering coefficient:¹³

This ${\mu }_s^{\prime }$ is related to the scattering coefficient ${\mu }_s=\frac{{\mu }_s^{\prime }}{1-g}$. The values ${\mu }_a,{\mu }_s$, and $g$ are used in a light transport model (Sec. 3.7).

3.

Materials and Methods

Altogether, three sets of experiments were performed. To characterize the performance of the system, we captured spectral image data of different tiles from a color calibration card and compared them with point spectroscopic measurements. We then captured spectral images of skin (forearm) from eight healthy volunteers and reconstructed spectra that best fit the data. Finally, we captured spectral images of skin from three healthy volunteers before and after inducing erythema by placing a warm wet towel on the skin. In our analyses, we used a GPU version of the multilayer Monte Carlo model (CUDAMCML³³) to estimate tissue composition, showing how local changes in optical properties of tissue layers can be determined from the system.

3.1.

System Design

A schematic of our imaging system is shown in Fig. 1(a). Pictures of the entire system and optical head are shown in Figs. 1(b) and 1(c), respectively. Briefly, the imaging system can be subdivided into an illumination module, a detection module, and a tablet computer that controlled illumination and imaging by the system.

Fig. 1

(a) Imaging system block diagram, (b) full system setup with tablet, and (c) optical head.

In the illumination module, spectral illumination was provided by a set of LEDs across the visible and NIR regions mounted on a printed circuit board (PCB). Visible LEDs were LUXEON Z from Lumileds, NIR LEDs were LX1 from LED ENGIN. A list of the LED central wavelength and bandwidth [full width at half maximum (FWHM)] is shown in Table 1. Electrical current to drive the LEDs was provided by a simple low-side field effect transistor (FET) switch with current limited by a series resistor. As each LED has a unique voltage drop, resistor values were selected to match each LED. A microcontroller developer board (Arduino Nano) was used to interface the LED drive board to the computer system. Beam uniformity across LEDs was achieved using LEDs with a wide illumination angle without any focusing optics in the illumination path.³⁴ In addition to spectral illumination, there was also a white LED (Cree XLamp) used to illuminate the tissue when the spectral LEDs were turned off (i.e., in preview mode), providing a view of the tissue to position the system properly; a white LED was also integrated into the multispectral PCB but not used in the analyses. This preview mode white LED was mounted separately, with its own PCB. A linear polarizer was installed in front of all the LEDs (Edmund Optics).

Table 1

Comparison of central wavelength (and FWHM) of LEDs reported by a vendor to spectrometer measurements. Note the white LED (a) was not used in the analyses described in Sec. 3.6.

LED name (internal)	Reported by the vendor [nm]	Measured by spectrometer [nm]
UV1	395 (11)	403 (12)
UV2	425 (14)	426 (14)
Royal blue	447 (20)	453 (20)
Blue	470 (20)	477 (27)
Cyan	505 (30)	505 (31)
Green	530 (30)	526 (38)
Lime	568 (100)	542 (89)
Amber	590 (80)	600 (18)
Red1	627 (20)	637 (16)
Red2	655 (20)	663 (19)
IR1	740 (20)	732 (38)
IR2	850 (30)	855 (34)
IR3	940 (40)	948 (66)
Whitea	N.A.	N.A.

The detection module consisted of a low cost monochrome camera (Basler puA2500–$14\mu \mathrm{m}$) coupled mechanically to a 25-mm machine vision lens (Basler) and a linear polarizer (Midwest Optics). The camera polarizer was oriented orthogonally to the illumination module polarizer to eliminate glare. The camera and lens were selected such that $\sim 4.7\mathrm{cm}$ was imaged along the camera’s short axis at an appropriate working distance of 30 cm.

The entire system was controlled by a tablet computer (Acer) that runs custom application software, which allows the operator to preview the image scene, perform image capturing sequences, and store the data both locally and at a remote location for subsequent analyses. The bill of materials of the entire system, including the computer, was under $1000.

3.3.

Reflectance Spectroscopy Measurements

To validate our multispectral imaging setup, we mounted reference samples against a 3-cm diameter aperture and took measurements both using the spectral camera and using a white-light spectrometer (Flame-S 2000 spectrometer, Ocean Optics). Spectrometer-captured spectra (Fig. 2) were used for comparison. Complete spectra were captured with a 4.75 W Tungsten Halogen bulb white-light illumination light source (Ocean Optics). Light diffusely reflected from the sample was collected by a $400\mu \mathrm{m}$ multimode fiber, placed at normal incidence at a distance of 2 cm from the target. The light source was placed at a distance of 8 cm and at an angle of 30 deg from the normal such that the sample was illuminated obliquely, with the specular reflection pointing away from the probe. The probe only acquired the central part of the aperture to minimize effects of lateral light leakage, with an acquisition time of 300 mS.

Fig. 2

Spectrometer system for generating ground truth spectral data.

4.

Results

4.1.

Multispectral Imaging System Validation

To validate the spectral camera system, spectral data were collected from the color calibration card using both the multispectral imaging system and the white-light spectrometer. A picture of the color calibration card is shown in Fig. 3 (top), and the comparison between the two spectral methods for each of the 24-color squares is shown in Fig. 3 (bottom), with the red curve representing the spectrometer, and the blue dots representing measurements from individual LEDs. As can be seen, there is a good agreement between the two systems on each of the 24 samples. We took this agreement as an indication that our LED spectral camera yields reliable spectral data.

Fig. 3

Photo of Colorchecker classic card (top), with a comparison of spectral camera to a spectrometer (bottom).

4.2.

Multispectral Imaging of Healthy Skin

Using our multispectral imaging system, we imaged skin on the forearms of eight healthy subjects and fit the data to a theoretical model. Figure 4 compares LED data (green dots), the best fit spectrum (blue line), and the dot product of the best fit spectrum and the LEDs’ spectral shape (blue exes), to account for the finite LED bandwidth following the method described in Sec. 3.7. Good overall agreement can be seen point-wise between the spectral data and the best fit curve for the eight subjects (Fig. 4). Moreover, the estimated parameter concentrations are within normal physiological levels (Table 2). In assessing the performance of individual LEDs, the Lime LED (568 nm) and to a lesser degree the IR3 LED (940 nm) seem to be consistently higher than the fitted curves while maintaining a good match with the pointwise fit. We believe that this is due to the 89- and 66-nm FWHM of the LEDs (Table 1) and the spectral shape of the tissue, which begins to dramatically increase partway through the LED’s spectrum.

Fig. 4

Spectral camera measurements (green dots) versus estimated fit spectra (blue line) and the product of the LED spectrum and the estimated fit spectra (blue exes) for eight volunteers participants.

Table 2

Estimated average tissue parameters for eight volunteers (with standard deviation). For completeness, values of μa and μs at 500 nm are also shown, determined from Eqs. (4) and (5).

Layer	B	S	W	M	μs500′	f	bMie	g	μa500	μs500
Epidermis	0 (—)	0.75 (—)	0.75 (—)	0.03 (0.003)	40.00 (—)	0 (—)	1.00 (—)	0.90 (—)	20.38	400.0
Basal layer	0 (—)	0.75 (—)	0.75 (—)	0.122 (0.01)	40.00 (—)	0 (—)	1.00 (—)	0.90 (—)	82.86	400.0
Papillary dermis	0.0005 (—)	0.55 (—)	0.65 (—)	0 (—)	30.00 (—)	0.62 (—)	0.91 (—)	0.90 (—)	0.11	300.0
Dermal venous plexus	0.048 (0.028)	0.819 (0.066)	0.75 (—)	0 (—)	10.00 (—)	0.62 (—)	0.91 (—)	0.90 (—)	5.38	100.0
Reticular dermis	0.0005 (—)	0.55 (—)	0.65 (—)	0 (—)	30.00 (—)	0.62 (—)	0.91 (—)	0.90 (—)	0.11	300.0
Subcutaneous tissue	0.05 (—)	0.55 (—)	0.75 (—)	0 (—)	10.00 (—)	0.60 (—)	0.90 (—)	0.90 (—)	5.60	100.0

4.3.

Multispectral Imaging with Erythema

To determine if our spectral camera setup can identify changes in optical properties, we also captured additional image sets from three healthy volunteers before and after inducing erythema. The tissue properties corresponding to the spectral data were then determined from a Monte Carlo lookup table, assuming the tissue as a six-layer model (Sec. 3.7). The spectral data (dots), as well as reconstructed curves for the best fit lines (solid line) as captured before (blue) and after (red) erythema, are shown in Fig. 5.

Fig. 5

Spectral camera measurements (points) versus estimated spectral fit spectrum (line) for normal skin (blue, dots and solid line) versus induced erythema (red, exes and dashed line).

In analyzing spectral data from this experiment, we kept most of the parameters in the various layers fixed, and only varied the blood content and oxygen saturation in the superficial plexus layer, as well as the melanin content in the epidermal and basal layers (the ratio of which was fixed at 1:3).³⁵ Numerical estimates for the fixed parameters in each layer are shown in Table 3. Plots of the sum of errors as a function of blood content at various oxygen saturation and melanin values are shown in Fig. 6. It can be seen that the blood content in the plexus layer is more than doubled as a result of inducing erythema. Additionally, the similarities between the curves representing different oxygen saturation levels show that our model and system are less sensitive to changes in oxygen saturation than to changes in blood content. Considering we have a few LEDs in the 600- to 700-nm range, which were identified to be problematic in Sec. 4.1, this result makes intuitive sense. Additionally, we also see that there is a coupling between the optimal blood content as a function of melanin concentration. Because both melanin and oxy- and deoxyhemoglobin absorb more at lower wavelengths,² these observations also make intuitive sense.

Table 3

Base tissue parameters used to construct the tissue lookup table used in both experiments. Superscript v represents parameters that were varied. For completeness, both μa and μs at 500 nm are also given, calculated from Eqs. (4) and (5).

Layer	B	S	W	M	μs500′	f	bMie	g	μa500	μs500
Epidermis	0	0.75	0.75	${0.03}^v$	40	0	1	0.9	20.38	400.0
Basal layer	0	0.75	0.75	${0.12}^v$	40	0	1	0.9	81.50	400.0
Papillary dermis	0.0005	0.55	0.65	0	30	0.62	0.91	0.9	0.56	300.0
Dermal venous plexus	${0.15}^v$	${0.9}^v$	0.75	0	10	0.62	0.91	0.9	16.81	100.0
Reticular dermis	0.0005	0.55	0.65	0	30	0.62	0.91	0.9	0.06	300.0
Subcutaneous tissue	0.05	0.55	0.75	0	10	0.6	0.9	0.9	5.60	100.0

Fig. 6

Measurement of fitting error as a function of blood content, for different oxygen saturation levels (curves) and melanin values (a)–(l). Normal skin (blue) versus induced erythema (red). Separate plots are shown for each of the three volunteers. The vertical lines represent in each curve show where the error function was at a minimum.

5.

Discussion

The results presented here show the end-to-end characterization of a low-cost multispectral imaging system built with a $1000 bill of materials. The system was based on a PCB with 13 LEDs and an area scan camera that are controlled by a tablet. Altogether, we ran three sets of experiments to provide evidence that this system captures useful spectral data and allows for estimation of tissue chromophore content. The first experiment validated that the spectral data generated by our system yield comparable measurements to spectra measured with a spectrometer by comparing data from a color calibration card. Next, we used the system to capture image sets of the forearm of eight volunteers and fitted a tissue model (via a Monte Carlo lookup table) to the readings, demonstrating that the fitted values were within physiological ranges. Finally, we showed that our setup is capable of assessing a clear change in tissue composition (blood content of the superficial plexus layer) in spectral images captured before and after inducing erythema in three volunteers. Taken together, these results suggest that our portable, low-cost system is capable of reliably measuring spectral differences in tissue layers.

For several years, it has been known that spectral imaging systems do need to not be large or expensive, and indeed, multiple groups have built such systems.^{22,24,27,36,37} However, to our knowledge, these are the first end-to-end analyses that show it is possible to estimate tissue scattering and absorption properties with a low-cost system and that expensive hardware is not necessary for making clinically relevant measurements. Until now, validation of low-cost spectral analysis systems has focused primarily on the hardware, with less emphasis on the analysis and measuring optical properties. The tissue parameters measured by our system—blood content, oxygen saturation, and scattering from different tissue layers—are fundamental descriptors of tissue that provide a snapshot of its physiology on the micron scale. They provide information that clinicians would love to access but have not been able to, outside of a research setting at large university hospitals. Indeed, most systems that make such measurements that are in clinical use today are built on a cart.^20,21,38 The small-form factor, light weight, and low power nature of our system allow it to be portable, carried by physicians who move between patients beds or between clinics. If successfully translated and productized, it can have a profound effect on clinical care.

The big unknown question with our setup is how does it compare with more expensive spectral imaging systems, specifically in terms of the ability to make scattering and absorption measurements. Here, Fig. 3 shows that reflectance from the LEDs matched very well with the spectrometer. Errors of $\sim 10\%$ reflectance were only found on one LED (590 nm) on three of the 24 squares, representing $<1\%$ of the total measurements. Figures 4–6 show that even with inexpensive components, it is still possible to build a system capable of generating data that enables quantitative measurements of tissue scattering and absorption. The main compromise in our system was in the time required to prepare the data analysis. The time it took to iteratively run simulations to analyze a single spectral measurement was over two days using a modern GPU. This is because the process involved running tens of thousands of CUDAMCML simulations that took time and drive space. Comparing these data with spectra we measured took only a few seconds. The Monte Carlo lookup table method, while computationally expensive in our experiments, can be optimized to run much more efficiently using methods derived from artificial intelligence (AI),^13,39 which will ultimately allow deployment on computers with less computational power. AI is particularly promising in this respect, as it utilizes a large Monte Carlo lookup table for training, yet the resulting classifier is not very large.

The main challenge with getting meaningful information out of the spectral measurements made by our setup is fitting it to the right model of light transport in tissue. Earlier attempts at fitting experimental data to a model were often based on a diffusion theory approximation.¹² Although diffusion theory is straightforward for a semi-infinite medium, diffusion math gets complicated when there are multiple layers to the tissue. Because skin is a multilayer tissue, fitting algorithms based on diffusion theory can only be trusted so-far, and Monte Carlo approaches are preferred. We saw this with our spectral data (data not shown). Specifically, within the context of experiments like the one shown in Fig. 5, the analysis becomes challenging: inducing erythema on a semi-infinite tissue, the average increase in blood content to the entire tissue is not very large. However, within the context of a layered tissue model, it becomes very apparent that increasing (doubling) the blood content in the superficial plexus layer causes the change in spectral diffuse reflectance measured by the spectral camera. Thus, Monte Carlo-based approaches are required to make meaningful measurements using our system.

In assessing the accuracy of the in vivo analysis, Fig. 6 shows that the model and system are much more sensitive to changes in blood content and melanin content than to oxygen saturation. The reason for this is because the number and quality (stability, FWHM) of the specific LEDs needed to estimate oxygen saturation was limited. There were larger variations in output from the relevant LEDs relative to the other LEDs in the setup, specifically the Amber (590 nm). This can be seen in the color calibration data where this LED has a lower reflectance than measured by the spectrometer in the white, neutral 8, and neutral 6.5 squares but not the others (Fig. 3). The other important LED for this measurement, lime (568 nm), has a FWHM of 89 nm, and this large bandwidth exerted a non-negligible influence from nearby wavelengths on the measured reflectance, which reduced our sensitivity to measuring changes affecting that part of the spectrum.

In addition to challenges caused by the hardware, there are also some inherent challenges in working with a multilayer Monte Carlo model, as the fitting analysis becomes an underdetermined problem, with more unknowns than measurements. Such a problem requires making assumptions on tissue layers when oftentimes there is insufficient information in the literature. For example, differences in scattering properties between the papillary and reticular dermis significantly lowered our fitting error for one of the volunteers (data not shown), but making such educated guesses is not trivial, as there are not many reliable measurements of scattering by different dermal layers. Similarly, variations in the water content across the various layers have not been properly assessed. Rigorous studies of the actual values of many of these tissue parameters are necessary to improve measurement accuracy.

An alternative approach for assessing blood content and oxygen saturation in tissues is spatial frequency-domain imaging (SFDI)⁴⁰ and its derivatives. SFDI holds two key advantages to unmodulated imaging (e.g., continuous wave) approaches: the ability to measure the absorption and scattering separately from each other and being able to control for the depth in which the measurement is made using the modulation frequency. However, the SFDI systems built to date are based primarily on red and NIR light and are thus less suitable for layered tissues, such as skin or the cervix, in which the layer of interest is superficial and spans only a couple of hundred microns in depth. Indeed, higher spatial frequencies and shorter wavelengths that have been demonstrated to date in vivo are required for making such measurements accurately. Our system uses shorter wavelength light (purple, blue, and green) that is more suitable for assessing changes in layered tissues. Moreover, while there have been recent advances in making low-cost SFDI devices (and variants),^41,42 to our knowledge there has not yet been a noncontact, low-cost, small-form factor device validated in a clinical settings. Thus, while both methods make similar measurements and rely on similar light transport models for data analysis, they are suited for different clinical applications.

It is important to note that Monte Carlo models are specific to a particular tissue (i.e., skin), even though they can be easily adapted for alternative geometries of additional tissues. This is another issue to consider when designing such a system. Our system was designed for cervical imaging using a colposcopy imaging geometry. A system like the one described here was deployed in a cervical cancer clinic to look for cervical dysplasia.⁴³ An exploratory trial has already began.

Additional future work will critically assess the combination of LEDs used for the spectral illumination to maximize performance at reduced cost. The current implementation is limited by assumptions made in the fitting method, which employs a limited set of tissue parameters. More extensive Monte Carlo methods will be implemented to better optimize the method, as a larger lookup table can provide finer estimates of tissue parameters.

6.

Conclusion

We presented an end-to-end analysis of a low-cost multispectral imaging system, from design to development and validation, and measuring optical properties of tissue in vivo. Our results show that images captured with a low-cost spectral camera setup yield comparable fitting results as measurements captured by a spectrometer. Such measurements were made on 24 calibration samples, showing overlap in measured diffuse reflectance. We then demonstrated that it is possible to measure optical properties of the imaged tissue using a multilayer Monte Carlo model. Finally, using our system and analysis, we were able to show physical changes between normal forearm skin and erythema. We consider the experiments presented here as a proof of principle that it is possible to capture meaningful spectral measurements using a low-cost spectral camera set up. We believe our method holds considerable promise for clinical utility if implemented correctly.