1.

Introduction

Asthma and chronic obstructive pulmonary disease (COPD) are the most prevalent obstructive airway diseases, affecting tens of millions of people in the United States alone.¹ The most common way to diagnose the diseases and reassess the progression of the diseases is spirometry, which measures how much a patient inhales and how fast he/she exhales. In a spirometry test, the patient is instructed to exhale rapidly and forcefully into a mouthpiece connected to a spirometer that measures breath flow rate and volume. To ensure that all the air is inhaled into the spirometer for accurate flow measurement, the patient is also instructed to wear a nose clip, which leads to complaints of discomfort. For good hygiene, a disposal mouthpiece is used for each spirometry test. The needs of the spirometer, nose clip, and mouthpiece contribute to factors that prevent widespread use of spirometry at home. To overcome this barrier, here we describe an imaging-based noncontact spirometry using a webcam.

There are increasing efforts in developing noncontact respiratory monitoring methods. Depending on the monitoring principles, these methods can be divided into three categories: thermal, photoplethysmography (PPG), and body movement detections. The first category measures the air temperature change associated with exhaled breath near the mouth and nose regions of a subject using an infrared imaging system.^2,3 The temperature change can be also detected via the pyroelectric effect.⁴

The second category extracts the respiratory signal embedded in the PPG signals. PPG measures the change of light absorption or reflection induced by the change of blood volume with each pulse. The movement of the thoracic cavity affects the blood flow during breathing, which leads to a modulation in the PPG signal by the respiratory activity.⁵ Several PPG signal processing methods, including independent component analysis,⁶ principal component analysis,⁵ digital filtering,⁷ and variable frequency complex demodulation⁸ have been proposed to remove noise in PPG and extract respiration-induced modulation in PPG.⁹

The third category detects the subtle chest movement induced by breathing with different technologies, such as frequency-modulated radar wave^10,11 and ultrawideband impulse radio radar.¹² Several optical imaging-based methods have been introduced to monitor respiratory activities. For example, our group¹³ developed a differential method to track the shoulder movement associated with breathing. Reyes et al.¹⁴ measured the intensity change of the chest wall. Lin et al. applied an optical flow algorithm to detect respiratory activities.¹⁵ These methods used low-cost cameras. More sophisticated three-dimensional imaging with multicamera and projector-camera setups were used to track chest surface deformation during breathing.^16,17

The works summarized above demonstrated noncontact monitoring of respiratory activities, but to date a noncontact spirometry has not been reported. To achieve this capability, one must be able to accurately and quickly measure respiratory cycles over an extremely wide flow rate range and validate the results with real subjects. In this paper, we describe an imaging-based method to perform spirometry, which can track the flow-volume (spirometry) curve and extract important parameters, including forced expiratory volume in the first second (FEV1), forced vital capacity (FVC), and peak expiratory flow rate (PEF). These parameters are critical for the diagnosis and management of asthma and COPD. We compare and calibrate the imaging-based noncontact spirometry with the simultaneously performed traditional spirometry and validate the results with a pilot study including 16 subjects.

2.

Method

The imaging-based spirometry first captures the image of a subject’s upper body, including face and shoulders, during inhalation and exhalation with a webcam (Fig. 1). Then, it analyzes the shoulder movement and determines the spirometry exhalation rate versus exhalation volume curve, FEV1, FVC, and PEF. To validate and calibrate the method, this work includes a simultaneous spirometry test of each subject with a traditional spirometer (gold standard reference) [Fig. 1(a)]. The data from the imaging-based and traditional spirometry tests are compared and used to construct a calibration curve for the subject. From the calibration curve, the exhalation volume is determined from the shoulder movement, and the corresponding exhalation rate is obtained from the time derivative of the exhalation volume.

Fig. 1

Principle of noncontact spirometry. (a) Experiment setup. (b) Video recording. (c) and (d) Video analysis. (e) Signal analysis. (f) Flow-volume (spirometry) curve.

2.3.

Signal Analysis

To calibrate the shoulder movement, we counted the number of pixels of a certain feature in the image frame and related it to the actual physical length of the feature. The conversion factor $\alpha$ is defined as

Eq. (4)

$$\alpha =\frac{\text{feature}\mathrm{\backslash \_}\text{length}(\mathrm{mm})}{\text{feature}\mathrm{\backslash \_}\text{pixel}\mathrm{\backslash \_}\text{number}(\text{pixel})}.$$

In the $i$’th frame, the vertical component of the feature points, representing the shoulder position pos, due to respiration, is given by

Eq. (5)

$$\mathrm{pos}(i)=\alpha \frac{1}{n}\sum _{j=1}^n{y}_j(i),$$

where $y_j(i)$ is the vertical component of the $j$’th point in the $i$’th frame, $n$ is the total frame number of the video.

To correlate the shoulder displacement (change in position) with the spirometer reading, a calibration curve was built for each subject, which converted the shoulder displacement to breathing volume in forced spirometry (Fig. 2). The exhaled volume data from three randomly selected forced breathing cycles of each subject were fitted with a fifth-order polynomial. Note that the sampling rate of spirometer ($>100\mathrm{Hz}$) was higher than the web camera (30 Hz). To match the spirometer reading and camera output, we down sampled volume data from the spirometer for close comparison with the measurement of our optical system. The flow rate was determined from time derivative of the breathing volume.

Fig. 2

Calibration curves (a) to (p) for different subjects. The dotted lines are obtained from three forced breathing cycles, and the blue lines are fifth-order polynomial fitting curves. The sum of squares due to error (SSE) indicators are of the fitting quality (see text for more details).

3.

Results

3.2.

Flow-Volume Curves and Vital Respiratory Parameters

Using the calibration curves obtained above, we converted the remaining three breathing cycles into flow-volume curves (blue lines) for each subject. This flow-volume is one of the most important spirometry test results that is provided by most commercial spirometers (Fig. 3). For comparison, the simultaneously recorded curves from the commercial spirometer are also shown in Fig. 3 (red lines). Frechet distance (FD) and Pearson product-moment correlation coefficient (Pearson’s $r$) were used to measure the similarity between the imaging-based spirometry curves and the gold standard spirometry curves. Given two curves $f:[a,b]\to V$ and $g:[a,b]\to V$, FD is defined as

Eq. (8)

$$F(f,g)=\underset{\alpha ,\beta t\in [\mathrm{0,1}]}{\inf \max }\{d(f[\alpha (t)],g[\beta (t)])\},$$

where $\alpha$ (resp. $\beta$) is an arbitrary continuous nondecreasing function from [0,1] onto $[a,b]$ (resp. $[a^{\prime },b^{\prime }]$).

Fig. 3

Flow-volume curves (a) to (p) for different subjects. The red curves are from the gold standard spirometer, and blue curves are from the imaging-based method. $r$ is a Pearson product-moment correlation coefficient, and “FD” measures the distance between the calibrated curve and spirometer result.

Pearson’s $r$ measures the linear dependence between the imaging-based results and the gold standard spirometry results. As shown in Fig. 3, the Pearson’s $r$ values ranged from 0.89 to 0.99, and FD was found to range from 0.76 to 3.32, indicating good agreements between the imaging-based and traditional spirometer curves for most subjects.

Vital parameters, FEV1, FVC, and PEF, were calculated from the imaging-based spirometry curves (see Table 1 and Fig. 4.). Figure 4 shows that the results measured using the imaging-based method are in excellent agreement with those by the gold standard spirometer. To quantify the agreement, the Pearson $r$, root-mean-square-error (RMSE), relative error, and paired sample $t$-test were analyzed. The Pearson $r$ values for FEV1, FVC, and PEF are 0.95, 0.98, and 0.97, respectively, indicating good linear correlation between the imaging-based and traditional spirometers. The RMSE values of FEV1, FVC, and PEF are 0.27, 0.18, and 0.56 respectively, indicating small differences between the two methods. The average errors of FEV1, FVC, and PEF are 8.5%, 6.9%, and 7.7%, respectively. In the paired $t$-test, FEV1 and FVC show the pairwise difference between the imaging-based spirometry and the gold standard spirometry with a mean of zero at the 5% significance level, which is consistent with the RMSE values.

Table 1

FEV1, FVC, and PEF values for three forced breathing cycles (mean±σ).

Subject	FEV1 (L)		FVC (L)		PEF (L/s)
	Estimated	Gold	Estimated	Gold	Estimated	Gold
1	$3.69\pm 0.06$	$3.43\pm 0.04$	$4.34\pm 0.21$	$4.63\pm 0.06$	$9.07\pm 0.32$	$8.42\pm 0.31$
2	$2.46\pm 0.12$	$2.50\pm 0.01$	$2.55\pm 0.12$	$2.59\pm 0.04$	$5.03\pm 0.65$	$4.86\pm 0.07$
3	$3.11\pm 0.12$	$3.09\pm 0.04$	$3.25\pm 0.08$	$3.12\pm 0.06$	$6.55\pm 0.57$	$6.86\pm 0.11$
4	$2.14\pm 0.33$	$2.38\pm 0.03$	$2.42\pm 0.07$	$2.69\pm 0.04$	$5.10\pm 0.57$	$4.80\pm 0.03$
5	$3.67\pm 0.14$	$3.84\pm 0.14$	$3.98\pm 0.07$	$4.03\pm 0.03$	$8.16\pm 0.40$	$7.80\pm 0.12$
6	$2.59\pm 0.19$	$2.83\pm 0.06$	$3.14\pm 0.22$	$3.14\pm 0.03$	$3.01\pm 0.20$	$3.07\pm 0.16$
7	$2.62\pm 0.28$	$3.04\pm 0.01$	$3.32\pm 0.12$	$3.31\pm 0.03$	$5.01\pm 0.13$	$5.14\pm 0.13$
8	$3.37\pm 0.09$	$3.73\pm 0.02$	$3.80\pm 0.09$	$3.95\pm 0.15$	$9.71\pm 0.36$	$9.40\pm 0.30$
9	$4.46\pm 0.10$	$4.47\pm 0.16$	$4.86\pm 0.22$	$4.85\pm 0.04$	$10.46\pm 0.34$	$10.16\pm 0.95$
10	$3.73\pm 0.12$	$3.30\pm 0.02$	$4.12\pm 0.16$	$4.11\pm 0.07$	$7.30\pm 1.07$	$7.02\pm 0.21$
11	$2.74\pm 0.05$	$2.75\pm 0.07$	$3.01\pm 0.09$	$2.85\pm 0.02$	$5.90\pm 0.55$	$5.81\pm 0.11$
12	$2.76\pm 0.09$	$2.86\pm 0.06$	$3.80\pm 0.26$	$3.71\pm 0.04$	$5.67\pm 0.49$	$6.00\pm 0.44$
13	$3.78\pm 0.08$	$3.81\pm 0.02$	$4.21\pm 0.18$	$4.14\pm 0.02$	$8.67\pm 0.32$	$8.40\pm 0.17$
14	$4.88\pm 0.12$	$4.46\pm 0.05$	$5.07\pm 0.10$	$5.24\pm 0.01$	$7.55\pm 0.49$	$7.60\pm 0.37$
15	$3.07\pm 0.13$	$3.28\pm 0.10$	$3.23\pm 0.10$	$3.57\pm 0.01$	$8.06\pm 0.18$	$7.49\pm 0.38$
16	$4.54\pm 0.24$	$4.52\pm 0.05$	$4.90\pm 0.07$	$5.10\pm 0.01$	$8.92\pm 0.08$	$8.93\pm 0.45$

Fig. 4

Comparison of (a) FEV1, (b) FVC, and (c) PEF obtained with the imaging-based and gold standard spirometry for different subjects. The values are averaged over three breathing cycles, and the error bars are standard deviations of the measurements.

4.

Discussion

Respiration induces craniocaudal, anteroposterior displacements, and cross-sectional area changes of rib cage and abdomen,²¹ which has been studied by analyzing the three-dimensional movements of the body.²² In this work, we have shown that accurate spirometry can be obtained from the shoulder displacement alone without complex three-dimensional chest movement measurement. This finding is consistent with a previous study that shows good correlation between vertical body movement measured by magnetic resonance imaging and spirometry.²³ To convert the displacement into volume, if we could simply model the lung as a cube or sphere, a third-degree polynomial fitting would suffice. However, the human lung is a complicated system, in which bronchi and alveoli have different shapes. Empirically, we found that a fifth-degree polynomial could fit the breathing volume well. Although a higher degree could be even more accurate, it brings up the over-fitting issue. In a future study, we could explore a general calibration to make the system easier for home use.

In the imaging-based spirometry, accurate tracking of the shoulder displacement is critical. In this work, we used the KLT tracking algorithm, which requires good contrast of the shoulder image. This means that the subject’s clothes must have a substantially different color from the background color. We found that this requirement was not difficult to meet for all the sixteen subjects who wore different clothes with different colors and patterns (Fig. 5). In addition to KLT, other algorithms^24–26 may also be used to track motions.

Fig. 5

Snapshots of the videos in the dataset. The subjects wore clothes with various colors and patterns.

Subjects with different genders, ages, body mass indices, and heights were included in this small pilot test. We found that height is moderately correlated with FVC. The Pearson correlation between height and FVC is 0.69. The correlation between body mass index and FVC is 0.50, which showed weak correlation. For the two gender groups, we found a significant difference ($p<0.004$) between male ($4.63\pm 0.69\mathrm{L}$, $\text{mean}\pm \mathrm{SD}$) and female subjects ($3.39\pm 0.72\mathrm{L}$, $\text{mean}\pm \mathrm{SD}$) in FVC. More subjects with diverse profiles will be needed to validate these observations.

The spirometry curves and extracted respiratory parameters provide key information about asthma, COPD, and other respiratory conditions. For example, a concave upward pattern is often observed in asthma or COPD patients.²⁷ Certain distinct patterns are linked to the obstructing lesions in the upper airway and variable intrathoracic obstructions. However, all the subjects enrolled in our current study are healthy adults. In future study, we will include subjects with respiratory diseases. Another direction of further improvement of the current work is to use a faster camera. The current work used a camera with a frame rate of 30 fps, and the faster camera will improve the temporal resolution and is expected to lead to more accurate measurement of the exhalation rate.

5.

Conclusion

We have demonstrated an imaging-based spirometer for accurate forced spirometry tests with a webcam. We validated the technology with a pilot study including sixteen subjects and compared the data with simultaneously performed spirometry tests with a gold standard commercial spirometer. The spirometry curves and key respiratory parameters, including FEV1, FVC, and PEF, from the imaging-based approach are in excellent agreement with those by the gold standard technology. The imaging-based spirometer does not require acquiring a spirometer, using a mouthpiece, or wearing a nose clip, which will lower the cost and improve the user experience, thus contributing to the diagnosis and management of the large and growing asthma and COPD populations.