2.1.

Simulation

Fluorophores were modeled as omnidirectional point light sources to perform simulations of fluorescence lensless imaging. As shown in Appendix, a point light source at height, $Z$, above the imaging plane casts light of intensity, $I$, at a distance, $X$, from the point on the plane closest to the light source

The distribution of fluorophores in a cell or microsphere was modeled by adding the effect of thousands of omnidirectional light sources evenly distributed throughout a unit sphere. The positions of the light sources were determined by superimposing a 3-D grid on top of the sphere. The grid size was chosen so that 10 points were included between the center of the sphere and the edge. A light source was added at each integer point on this grid that fell within the sphere for a total of 4160 points. A MATLAB (MathWorks) script was developed to calculate the illumination pattern for different separations between the sphere and the imager.

2.2.

Imager Preparation

Commercial imagers are almost universally shipped with cover glass protecting the sensor from moisture, dust, and mechanical interactions. In both conventional imaging applications and lensless shadow imaging, this glass has only minor effects on imaging quality, but for LFI the minimum sample-sensor separation of approximately 1 mm imposed by this cover glass significantly decreases the performance. The cover glass can be attached to the imaging sensor either indirectly by first mounting the sensor in a carrier and then attaching the glass to the carrier or directly by first connecting the glass to the wafer and then cutting the wafer into individual sensors.²⁰ It is very challenging to remove the cover glass without damaging the sensor when it is attached directly.

In this work, a variety of commercial imager chips were examined and the packages used in two commercial web cameras (Logitech QuickCam Communicate MP and Logitech QuickCam Pro 9000, New York, California) were determined to be the most easily removed. In these chips, the cover glass is glued to a plastic package that surrounds the imager. The cover glass was removed by first carefully scoring the plastic package along with the edge of the glass using a scalpel and then using the scalpel to lever the glass out.

After removing the cover glass, the imager was tested to ensure that it continues to function as designed, but placing a liquid sample directly on the imager resulted in immediate failure. The imager was protected by encapsulating the bond pads and bond wires with cyanoacrylate glue (Scotch Super Glue). The imager was held at an angle so that the glue would flow toward the edge and the glue was dispensed from a syringe (BD Insulin Syringe Ultra-Fine 30 Gauge). The imager was then allowed to cure in an open area to minimize redeposition of glue on the imager surface.

2.3.

Phantom Preparation

In order to keep fluorescent microspheres at fixed locations above the imager for shadow and fluorescence imaging, simple phantoms were fabricated using 12 μm fluorescent microspheres (Spherotech FP-10045-2) and agarose (Bioworld Agarose Low Melt Temperature). The microspheres were diluted $1000:1$ in deionized water to form a working solution. A 3% solution of the agarose was prepared following the manufacturer’s instructions to mix 90 mg of agarose powder with 3 mL of deionized water. The mixture was then placed in the microwave at high power for 30 s to dissolve the agarose. The melted agarose solution was then kept in a bath of boiling water until used. About 35 μl of the agarose solution was pipetted on top of the imager. About 2 μl of the microsphere solution was then pipetted into the agarose solution as close to the imager as possible. LFI was performed to identify if microspheres had settled. If not, the imager was placed under a heater for approximately 15 s to allow the microspheres to settle. If microspheres had settled, the imager was placed in a $-20°\mathrm{C}$ freezer for 1 min to set.

2.4.

Shadow Imaging

The established technique of shadow imaging was used to calibrate the new LFI technique.¹² The height of cells can be determined by using light sources mounted at different angles above the imager. As shown in Fig. 1(a), when the sample is illuminated by a source directly above the imager, a shadow is cast by each microsphere directly below its location. Shifting the light source shifts the shadow. The height of the sample, $Z$, can be calculated using the right triangles that are formed as depicted in Fig. 1(b), where $X$ is the movement of the sample’s shadow due to the movement of the light source, $H$ is the separation between the light source and the sensor, and “$L$” is the lateral displacement of the light source.

Fig. 1

Diagrams demonstrating shadow imaging. (a) The position of the shadow moves as the light source moves. (b) The height of the sample can be calculated using two shadows cast by the sample from different light sources using the similar triangles shown here.

Rather than moving a single light source, an illuminator was built that contained five LEDs (Lite-On White LED LTW-670DS). Each LED was independently controlled by a microcontroller (Microchip 18F4550) that communicated with a personal computer over the universal serial bus. The LEDs were mounted on a sheet metal box formed in the shape of a trapezoid as shown in Fig. 2. Initial experiments showed that it was difficult to accurately determine the position of the LEDs in a bowl shape and that using a rectangular enclosure resulted in little illumination from the LEDs at the sides reaching the sample because they are pointed in the wrong direction. The trapezoid shape offered a simple geometry while pointing the LEDs close enough to the sample to ensure strong illumination. Images were captured with the CMOS imager for each illumination setting with 5.3 ms exposure time, low gain, and high contrast settings.

Fig. 2

A trapezoid shaped box was built out of sheet metal to position light emitting diodes for illuminating the sample.

2.5.

Lensless Fluorescence Imaging

The novel LFI method is divided into two parts: exciting the fluorophores and measuring the light they emit. LFI does not use a filter to block the excitation light from hitting the sensor, so it is important to excite with light that the sensor does not measure. In this work, a handheld 254-nm light (Ultraviolet Products, LLC UVGL-55, Upland, California) was used because CMOS sensors are insensitive to this wavelength. The bulb also emits low levels of visible light, so these wavelengths were filtered out using a bandpass UV filter (Omega Optical, 250BP30, Brattleboro, Vermont). This illuminator is then placed as close to the imager as possible using a styrofoam stand.

Emitted photons were captured using a CMOS imager (Quickcam Communicate MP) prepared as described above. In order to capture the relatively faint fluorescence signal, the exposure was set to the maximum value of 1 s, gain was turned up to high, and contrast was turned down to medium.

2.6.

Image Processing

The images from shadow and fluorescence imaging were processed in MATLAB to generate the calibration graph. This processing took place in two stages. First, the fluorescence image was processed to detect microsphere location and fluorescence shape. Then, the shadow images of the same sample were processed to determine the height of each microsphere.

The first step while analyzing the fluorescence image was to determine the approximate center of each microsphere. This process is shown for the simulated data from two microspheres in Fig. 3. First, the image was thresholded to create a black and white image as shown in Fig. 3(a). Noise in the original image can occasionally create a single pixel that is not connected with the other pixels representing a microsphere, so the thresholded image was then dilated by expanding each white pixel by two pixels in every direction to form the image in Fig. 3(b). Each connected region of white pixels was then given a region number shown as different colors in Fig. 3(c). The center of each region, marked with an “$X$” in Fig. 3(d), was calculated by averaging the coordinates of all the pixels in that region. This location was used as an initial estimate of the center of the microsphere, but it was often inaccurate because the thresholding process was very sensitive to noise.

Fig. 3

The approximate location of the center of each fluorescent microsphere is located in four steps. (a) Raw data are thresholded. (b) The image is dilated so that each sphere is represented by a single contiguous region. (c) Each region is numbered. (d) The center of each region is identified by averaging the pixel values in that region.

The estimate of the microsphere location was improved and the width of the fluorescence was determined by performing a Gaussian fit on the image. Two-dimensional (2-D) Gaussian images were generated based on Eq. (1) for each value of $\sigma$ between 0.1 and 30 in steps of 0.1

The fitting procedure searched a space five pixels on each side of the estimated center value. At each location, the cross correlation was calculated between the measured image and each of the 300 different Gaussian images. An example of the result of these correlations is shown in Fig. 4(a). The correlations and Gaussian parameter $\sigma$, called the width of the fluorescence, were recorded for the best fit at each location. Then, the best correlation across all of the locations was determined as shown in Fig. 4(b). The accurate center location and width of the fluorescence signal were recorded for each microsphere.

Fig. 4

An improved estimate of sphere location and fluorescence width is determined by finding the best Gaussian fit. (a) For each location, the best fit versus Gaussian width is determined. (b) The best fit at each location is compared to find the best location and width.

Locating microspheres in shadow images were slightly more complex than locating the microspheres in fluorescence images, because all the microspheres in the sample appeared in the shadow image while only those microspheres closest to the imager appeared in the fluorescence image. The larger number of microspheres made it much more likely that two microspheres would appear close to each other. A method was developed for identifying cases where multiple microspheres could lead to incorrect location determination. First, a candidate microsphere location was determined. A window was defined covering 15 pixels in each direction of the microsphere location determined by fluorescence imaging.

Then, a test described here was performed to determine if the widow contained only one microsphere as in Fig. 5(a) or multiple microspheres as in Fig. 5(b). A potential center was determined by finding the point with the best correlation to a $3\times 3$ block. This location is marked by an “$X$” in Fig. 5(c). The single microsphere and multiple microsphere cases were distinguished by first calculating a threshold half way between the minimum and maximum values in the window. Then, the pixels within a $5\times 5$ box around the identified microsphere were set to zero. If less than five of the remaining pixels were above the threshold, as shown in Fig. 5(d), the location was considered valid. If more than five of the remaining pixels were above the threshold, then the window likely contained more than one microsphere and the location was considered invalid.

Fig. 5

Shadow images are processed to distinguish between a single microsphere (a) and multiple microspheres (b). For each single microsphere, the center is identified as shown in (c) by finding the best fit to a $3\times 3$ block. The validity of this fit is then checked by counting the number of dark pixels outside a $5\times 5$ grid surrounding the center (d).

If the microsphere had valid locations in all five images, the height of the microsphere, $Z$, was then calculated using Eq. (3) based on the similar triangles shown in Fig. 1(b):

Four heights are calculated for each microsphere based on the five shadow images created. The values of these heights are then averaged to determine the height of the microsphere.

2.7.

LFI of Flowing Samples

LFI was performed on two types of flowing samples: fluorescent microspheres and cells stained with quantum dots. About 1 μl of the same microspheres used for the calibration was mixed with 500 μl of DI water for this experiment. The cell sample was prepared using RAW 264.7 cells from American type culture collection, Manassas, Virginia and quantum dots from Invitrogen, Grand Island, New York (Q25011MP). The cells were grown on a fibronectin treated glass slide placed inside a humid chamber. The cells were left in the incubator until they covered 80% of the base of the slide. The protocol recommended by Invitrogen was followed to stain the cells except incubation time of cells with the staining solution was increased to 3 h to improve the absorption of quantum dots. The stained cells were then rinsed with fresh media and trypsinized to detach the cells. The detached cells were mixed with the media and loaded in a syringe for the experiment.

A simple microfluidic device was fabricated directly on the CMOS imager. First, a design was developed in Autocad. Then, the design was cut out of double-sided tape (Adhesive Research, ARC are 92712, Glen Rock, Pennsylvania) using a laser cutter (Epilog 10000). A PDMS slab was made by mixing the base with the curing agent at a $10:1$ ratio (Sylgard 184). The mixture was poured in a cell culture dish and placed on the hot plate for a few minutes to remove bubbles and left at room temperature over night to cure. A small $4\mathrm{cm}\times 3\mathrm{cm}$ cube was cut after the PDMS had hardened and ports were punched using a 22 gauge blunt needle. One side of the tape was stuck to the PDMS aligned with the ports and the other side was stuck to the imager. Once the imager set up was ready, PTFE tubing of size 30 from Cole-Parmer, Vernon Hills, Illinois (Part No. EW-06417-11) was inserted into the ports. The input port was attached to a syringe (Easy touch insulin syringe U-100) loaded with the sample. The output port fed a microcentrifuge tube. During imaging, the sample from the insulin syringe was injected into the chamber by a syringe pump (New Era Pump Systems, Inc. NE-1002×, Framingdale, New York) at the rate of $5\mu \text{l\hspace{0.17em}}/\min$.

3.1.

Simulation

LFI was simulated for unit spheres at different heights above the imaging sensor as described in Sec. 2. Each simulation produced a 2-D matrix with each value representing the intensity measured by a single pixel of an imaging sensor. The intensities are plotted for a single horizontal line passing through the center of the microsphere image in Fig. 6 to allow for the comparison between different sphere-sensor separations.

Fig. 6

Lensless fluorescence imaging simulations produced two-dimensional images of microspheres. A single row through the center of each image is presented in this figure to make it possible to compare different sphere-sensor separations. (a) A plot of intensity versus radial distance emphasizes the declining peak with increasing separation. (b) Plotting the same data normalized for the same peak emphasizes the increasing width for increasing separation.

Figure 6(a) is a plot of the fluorescence intensity for four different sphere-sensor separations. As the separation increases, the peak intensity decreases. Beyond four or five times the sphere’s radius, the peak becomes indistinguishable from the background. For typical cells with a radius of 5 to 10 μm, this distance is 25 to 50 μm.

Figure 6(b) is a plot of the same curves as Fig. 6(a) but with all the curves normalized to the same maximum value. This format emphasizes the change in shape of the fluorescence curves with the separation. Each of these curves is approximately Gaussian in shape with spheres with greater separation leading to wider curves.

3.2.

Lensless Fluorescence Imaging

LFI was performed on 226 microspheres in nine samples. Figure 7 shows the basic stages of image processing that were applied to each sample. The same techniques that were applied to simulated data in Fig. 3 were applied here to real data. Figure 8 summarizes the results for four typical microspheres. For each microsphere, the captured image is shown on the left and the plot on the right shows a cross-section view of the captured data and of the Gaussian fit. Moving from Figs. 8(a)–8(d), the intensity decreases and fluorescence width increases. Gaussian curves fit the data well until the broadest peak where noise plays a larger role. The curve fit in Fig. 8(d) highlights the fact that the Gaussian fits were performed on the 2-D data so they may not be the best fit for the one-dimensional data plotted in the figure.

Fig. 7

The lensless fluorescence image from one to nine samples used in this work shows a large number of fluorescent spheres. (a) The raw data are quite faint with varying intensities. (b) Thresholding makes it easier to identify each sphere. (c) The image is dilated to ensure that each sphere is represented by a single region. (d) The numbering of the regions is denoted by different colors for each region.

Fig. 8

Data for individual fluorescence microspheres shows a good match to the Gaussian model. Four fluorescence blobs are shown that span the width of measured images from very narrow (a) to very wide (d). Both the two-dimensional and one-dimensional intensity plot are shown for each microsphere. (a) Very narrow microsphere, (b) narrow microsphere, (c) wide microsphere, and (d) very wide microsphere.

3.3.

Shadow Imaging

Figure 9 shows that the microsphere shadows are clearly visible when the sample is illuminated from above. The images in Figs. 9(a)–9(e) are the same subsection of shadow images taken from five different illumination angles from the right to the left with Fig. 9(c) taken with illumination from directly above. The four spheres that appear in the five images are labeled in Fig. 9(f). Their paths are tracked across the five images in Fig. 9(g). The greater movement of the microspheres denoted by a star and a square imply that they are higher than the microspheres denoted by a circle and a diamond.

Fig. 9

The same subset of five shadow images taken of a single sample are shown in (a)–(e). The four spheres labeled in (f) are found in all five shadow images. The movement of each of these four spheres across the five images is shown in (g) to demonstrate the difference in how much different spheres move based on their distance from the sensor.

Of the 226 spheres identified by LFI, only 54 made it through the filtering process described in Sec. 2 to ensure that accurate shadow locations could be calculated for all five images. Their heights are presented as a histogram in Fig. 10. There are a large number of microspheres at 5 μm because these microspheres are resting on the surface of the imager chip approximately 5 μm above the actual sensor elements. No microspheres above 35 μm were recorded because they could not be identified in the lensless fluorescence image.

Fig. 10

This histogram demonstrates that the spheres were evenly distributed between 5 and 35 μm. Higher microspheres were excluded because they were not visible in the lensless fluorescence image.

3.4.

3-D Fluorescence Shadow Correlation

The correlation between 3-D fluorescence images and microsphere heights was determined by plotting the fluorescence width versus the sphere height as shown in Fig. 11. The data is labeled with the run number in which each microsphere was measured to show the repeatability of this method. A line of best fit for the data is described by the equation

Fig. 11

The relationship between fluorescence diameter and sample separation is linear as shown by this data collected over nine runs and plotted with a best fit line.

Over the range of sphere heights from 5 to 35 μm, the maximum error between the best-fit line and the actual measurement was 7 μm and the standard deviation of the error was 2.6 μm.

3.5.

Lensless Fluorescence Imaging of Flowing Samples

Cells could not survive the procedure used to suspend microspheres in agarose, so LFI of cells was performed with the cells in liquid suspension. In the first experiment, detached cells stained with quantum dots were mixed with fresh media and placed on the imager and allowed to evaporate. The sample was excited with a UV source and images were captured as the water evaporated. As shown in Fig. 12, despite the more complicated geometry of the cell compared to microspheres, the lensless fluorescence image is still well modeled by a Gaussian curve.

Fig. 12

Data from individual cells. (a) Cell with high sample-sensor separation. (b) Cell with relatively low sample-sensor separation.

To demonstrate that the technique works with a flowing sample, the imager with the microfluidic device was employed. The sample was pumped into the microfluidic channel and excited with a UV source. A video of the sample flowing through the chambers was captured. Figure 13 contains frames from the video demonstrating movement of microspheres [Figs. 13(a)–13(c)] and cells [Figs. 13(d)–13(f)].

Fig. 13

Frames highlighting the movement of sample inside a lensless on-sensor microfluidic device (a)–(c) movement of microspheres (d)–(f) movement of quantum dot labeled RAW cells.