1.

Introduction

Temperature is closely related to human life, agricultural production, and social economy, and it affects all aspects of life. Studies showed that the increase in temperature reduced crop yields.^1,2 Doi et al.³ presented that temperature change had an impact on deep-sea biodiversity. Dottori et al.⁴ proposed river floods with warming would increase human and economic loss. Temperature change also increased building energy consumption.⁵ The most serious issue is that temperature change would have an impact on the spread of diseases and endanger people’s lives.^6,7 Accurate prediction of temperature is significant to protect people’s lives and property and maintain stable economic development. However, temperature prediction is very challenging due to various uncertain relevant factors.

A series of forecasting methods including conventional methods and machine learning methods were proposed to predict temperature. Wang et al.⁸ proposed an improved support vector machine (SVM) to predict the daily minimum temperature; Babu et al.⁹ used different autoregressive integral moving average (ARIMA) models to predict the average global temperature; Jallal et al.¹⁰ proposed an artificial neural network (ANN) with delayed exogenous input to forecast air temperature on a half-hour scale. With the appearance of recurrent neural network (RNN), more and more methods based on RNN are used to solve the problem of temperature prediction. The long short-term memory (LSTM) network is one of the most popular methods. Li et al.¹¹ provided temperature prediction results every half an hour using stacked LSTM network. Qi and Guo¹² predicted the next hour’s temperature using the LSTM network by fully considering the historic temperature and meteorological condition. Huang et al.¹³ proposed multistep temperature prediction model using the LSTM network based on temperature data of surrounding cities. Sadeque and Bui¹⁴ provided cascaded LSTM network for weather forecasting that can outperform some of the existing well-known models. Joanna et al.¹⁵ presented an outdoor air-temperature time-series prediction model for a multifamily building using ANNs and obtained outstanding prediction results by selecting the best combination of predictors and the optimal number of neurons in a hidden layer. Wang et al.¹⁶ proposed the development and evaluation of a new algorithm based on pattern approximate matching to predict the temperature of five cities in China. Yu et al.¹⁷ presented an air temperature forecasting framework based on graph attention network and the gated recurrent unit, which overcame the flaw of the conventional graph network and achieved the best performance. Hrachya et al.¹⁸ implemented a weather prediction technique based on machine learning to improve the hourly air temperature prediction for up to 24 h. Toni et al.¹⁹ combined LSTM and prophet model to forecast 5-year daily air temperatures in Bandung; the results showed that the combination of two networks performed well for the prediction of low temperature and high temperature.

For time series data prediction, time series decomposition methods have an inspiring effect on improving the accuracy of time series data forecasting. Zhang²⁰ achieved foreign exchange rate forecasting using the combined model of empirical mode decomposition (EMD) and LSTM network. Jin et al.²¹ suggested a vegetable price forecasting model using seasonal and trend decomposition using loess (STL) and LSTM network. Wang and Lou²² proposed a hydrological time series forecast model based on wavelet denoising and ARIMA-LSTM that can be well adapted to the hydrological time series forecast and has the best forecast effect. Duan et al.²³ forecasted base station traffic using STL-LSTM networks, which have better performance compared with the other algorithms. Huo et al.²⁴ solved the problem of long-term span traffic prediction using STL and LSTM model. Yin et al.²⁵ updated the STL-LSTM model using an attention mechanism to achieve high accuracy of vegetable price forecasting. Chen et al.²⁶ forecasted the short-term metro ridership using the STL-LSTM model and proved it can achieve high accuracy.

In this paper, we propose a prediction model combining the STL method and bidirectional long short-term memory (Bi-LSTM) neural network to achieve the prediction of daily average temperature. Because temperature is affected by a variety of uncertain factors, it is difficult to obtain satisfactory results in temperature prediction directly using deep learning models. The time series decomposition method can be used to decompose periodic time series into trend components, seasonal components, and residual components. In a general periodic time series, the trend component generally represents the low-frequency variation, whereas the seasonal component represents the periodic variation. For temperature data, the seasonal component represents the periodic fluctuation of temperature with seasonal changes. The trend component represents changes in temperature that are influenced by other factors, such as increase in carbon dioxide. This paper first uses the STL method to decompose temperature data into trend component, seasonal component, and remainder component and then inputs the decomposition components and the original data into the two-layer Bi-LSTM neural network for training. The final output of the network is the predicted temperature.

The structure of the paper is as follows: Sec. 1 is the introduction; Sec. 2 introduces the study area; Sec. 3 is the related works, including the STL method, LSTM model, and Bi-LSTM model; Sec. 4 shows the structure of the proposed model; Sec. 5 is the experimental results; Sec. 6 is the conclusion.

2.

Study Area

China is located in eastern Asia and on the western coast of the Pacific Ocean. It has a vast territory with a total land area of $\sim 9.6\text{million}{\mathrm{km}}^2$. The terrain of China is higher in the west and lower in the east and is distributed in a stepped manner. The combination of temperature and precipitation is diverse, forming a diverse climate. There are a total of 34 provincial-level administrative units, including 23 provinces, 5 autonomous regions, 4 municipalities, and 2 special administrative regions.

3.

Materials and Methods

We introduce the materials and related methods including seasonal and trend decomposition using loess, LSTM model, and Bi-LSTM model.

3.2.

Methods

3.2.1.

Seasonal and trend decomposition using loess

STL is a time series decomposition method proposed by Cleveland et al.,²⁹ which can decompose a time series into trend, seasonal, and remainder components based on loess. Suppose there is a temperature time series $X_v$, STL can decompose $X_v$ into three addictive components: trend component ($T_v$), seasonal component ($S_v$), and remainder component ($R_v$), which can be expressed as follows:

Eq. (1)

$$X_v=T_v+S_v+R_v.$$

Suppose $x_i$ and $y_i$ are measurements of an independent and dependent variables for $i=1$ to $n$. $g(x)$ is a smoothing of $y$ given $x$ that can be computed for any value $x$ along the scale of the independent variable. That is, loess is defined everywhere and not just at the $x_i$; as we shall see, this is an important feature that in STL will allow us to deal with missing values and detrend the seasonal component in a straightforward way. That is, loess can be used to smooth $y$ as a function of any number of independent variables, but for STL, only the case of one independent variable is needed. $g(x)$ is computed as follows: given a positive integer $q$, when $q\le n$, the $q$ values of the $x_i$ that are closest to $x$ are selected and each is given a neighborhood weight based on its distance from $x$. Let ${\lambda }_q(x)$ be the distance of the $q$’th farthest $x_i$ from $x$. Let $W$ be the tricube weight function:

Eq. (2)

$$W(u)=\{\begin{array}{ll}{(1-u^3)}^3& \text{for}0\le u<1\\ 0& \text{for}u\ge 1\end{array}.$$

The neighborhood weight for any $x_i$ is

Eq. (3)

$$v_i(x)=W\left(\frac{|x_i-x|}{{\lambda }_q(x)}\right).$$

The next step is to fit a polynomial of degree $d$ to the data with weight $v_i(x)$ at $(x_i,y_i)$. The value of the locally fitted polynomial at $x$ is $g(x)$.

STL consists of two procedures: one is an inner loop and the other is an outer loop. The inner loop is nested inside the outer loop. The inner loop is used to update the seasonal component and trend component; the process of $k$’th epoch is as follows:

• (1) Detrending. Calculate a detrended series by subtracting trend series from original series: $X_v-T_v^{(k)}$.

• (2) Cycle-subseries smoothing. Each cycle-subseries obtained from step (1) is smoothed by loess and the result is the preliminary seasonal series $C_t^{(k+1)}$, consisting of $N+2\times$ frequency values that range from $v=-\text{frequency}+1$ to $N$ + frequency, in which $N$ is the length of data.

• (3) Low-pass filtering of smoothed cycle-subseries. The preliminary seasonal series of step (2) is processed by a low-pass filter consisting of moving average of length frequency and the remainder trend series $L_t^{(k+1)}$ is obtained by loess.

• (4) Detrending of smoothed cycle-subseries. Calculate seasonal series $S_t^{(k+1)}=C_t^{(k+1)}-L_t^{(k+1)}$.

• (5) Deseasonalizing. Get deseasonalized series: $Y_t-S_t^{(k+1)}$.

• (6) Trend smoothing. The deseasonalized series of step (5) is smoothed by loess to obtain the trend component $T_t^{(k+1)}$.

The outer loop starts when the inner loop reaches the accuracy requirement. The remainder component $R_t^{(k+1)}$ is computed in the outer loop using the estimated trend and seasonal components:

Eq. (4)

$$R_t^{(k+1)}=Y_t-T_t^{(k+1)}-S_t^{(k+1)}.$$

3.2.2.

LSTM model

The RNN is designed to process sequence information, such as speech recognition and machine translation. The disadvantage of this method is long-term dependencies which will lead to gradient disappearance. The emergence of LSTM fix this problem. Different from RNN, LSTM proposed by Hochreiter and Schmidhuber³⁰ adds structure of forget gate, input gate, and update gate to forget and update information to the cell state.

The structure of LSTM is shown in Fig. 1. The first step of LSTM is to determine what information of cell state needs to discard, which is handled by forget gate using a sigmoid function:

Eq. (5)

$$f_t=\sigma (W_f·[h_{t-1},x_t]+b_f),$$

where $x_t$ is the input of current step, $h_{t-1}$ is the hidden state of the previous step, $W_f$ is the learnable weight of the forget gate, $b_f$ is the learnable bias, $\sigma$ is the sigmoid function, and $f_t$ is the output of the forget gate.

Fig. 1

Structure of LSTM model.

The input gate determines what new information obtained from the current input $x_t$ can be saved in the current cell state $C_t$. This process has three steps. At first, determine what information to update from the current input $x_t$ using a sigmoid function. Then, obtain a candidate vector ${\tilde{C}}_t$ using a tanh function. Finally, update current cell state $C_t$ in terms of cell state of previous step $C_{t-1}$ and candidate vector ${\tilde{C}}_t$. The operations are as follows:

Eq. (6)

$$i_t=\sigma (W_i·[h_{t-1},x_t]+b_i),$$

Eq. (7)

$${\tilde{C}}_t=\tan h(W_C·[h_{t-1},x_t]+b_C),$$

Eq. (8)

$$C_t=f_t*C_{t-1}+i_t*{\tilde{C}}_t.$$

The output gate determines the output state of current step:

Eq. (9)

$$o_t=\sigma (W_o·[h_{t-1},x_t]+b_o),$$

Eq. (10)

$$h_t=o_t*\tan h(C_t),$$

3.2.3.

Bi-LSTM model

Bi-LSTM is based on LSTM and combines the information of the input sequence in both the forward and backward directions, which can better capture the two-way semantic dependence. The structure of Bi-LSTM is shown in Fig. 2. The LSTM unit has limitations, and it is able to make predictions using past data but not future data. Bi-LSTM overcomes the limitations of LSTM, it consists of two different LSTM hidden layers with opposite output directions. Under this structure, both backward and forward information can be utilized in the output layer. Bi-LSTM has advantages that LSTM does not have, so we choose the Bi-LSTM network as the temperature prediction network.

Fig. 2

Structure of Bi-LSTM model.

4.

Proposed Model

To improve the accuracy of temperature prediction, we built a temperature prediction model combining STL and Bi-LSTM named parallel STL-Bi-LSTM neural network.

As shown in Fig. 3, the proposed model mainly has five steps: at first, original temperature data are decomposed into three components: trend component, seasonal component, and remainder component using the STL method. Then, original temperature data, trend component, seasonal component, and remainder component are input into two-layer Bi-LSTM neural network. The three-part decomposed data obtain one predicted value through the two-layer Bi-LSTM network and the fully connected layer, respectively, then adding the three predicted values to obtain one predicted value of the decomposed data. The origin temperature data obtain one predicted value through the two-layer Bi-LSTM network and a fully connected layer. The two predicted values are merged to obtain a feature containing two predicted values. The final predicted value is obtained by adding the two values through a fully connected layer using learnable weights.

Fig. 3

Structure of parallel STL-Bi-LSTM model.

5.

Experiment

The experiment has three parts: first, the original temperature data are processed to input into the proposed model. Second, we determine the optimal parameters of the proposed model based on the performance of different parameters on the same testing data. Third, the proposed model is evaluated on testing data and the prediction results are compared with other prediction models.

5.1.

Data Preparation

The temperature data need to be processed before inputting into the neural network. For the input of the proposed model, the original temperature data and the trend component, seasonal component, and remainder component need to be converted to input and output pairs for training. The process is as follows: decompose original data using the STL method into three components: trend component, seasonal component, and remainder component. Figure 4 shows an example of STL decomposition results. Frequency is the number of observations in each period, or cycle, of the seasonal component. The trend component in different years is different, because the temperature in different years is different and the decomposed trend data will also be different. The seasonal component obtained by STL decomposition is variation in the data at or near the seasonal frequency.²⁹ After using the curve connection, it looks like a continuous waveform, but the actual meaning is still regularly changing data. The seasonal component is the regular data of the decomposition period, which mainly analyzes the regular change of the irregularly changing data. Then set time_step, which denotes how much data are used to predict the next value. Use a sliding window of length time_step to traverse the data to generate the training set, and select the value of time_step + 1 as the corresponding predicted value. Generate the input and output pairs of original data and the trend, seasonal, and remainder components for the proposed model.

Fig. 4

Decomposition results of temperature data when the frequency is 2. The first line is original temperature series, other lines are trend component, seasonal component, and residual component.

5.4.

Model Evaluation

The training set and testing set are separated from temperature data of 34 cities. Additional testing data of four regions are added to test the robustness of the proposed model. We tested the model using the testing set and the temperature data of four regions and calculated the average RMSE and MAE to compare with other prediction models.

The proposed model can be divided into two parts according to the input: the input part using STL decomposition components (part 1) and the original temperature data input part (part 2). To prove that the proposed model combines the advantages of the two parts to improve the prediction effect, we used the same parameters to train the model using the STL decomposition components input and the model using the original data input.

It is shown in Tables 3 and 4 that the proposed model achieves good prediction results for the temperature in different regions, and the proposed model is better than the other two comparative models in the prediction results. Compared with the model that uses the original temperature data, the proposed model greatly improves the prediction accuracy. At the same time, the high accuracy of the model that uses STL decomposition components input also shows that the combination of the time series decomposition method can improve the accuracy of the model.

Table 3

Comparison of RMSE (°F) on models of different input.

Model	Testing set	North region	South region	Northwest region	Tibetan region
Our	0.108	0.127	0.069	0.139	0.103
Part 1	0.155	0.150	0.152	0.181	0.149
Part 2	4.230	4.8	3.475	4.967	3.409

Note: Bold values represent that the proposed model has the smallest RMSE on testing set than two part models.

Table 4

Comparison of MAE (°F) on models of different input.

Model	Testing set	North region	South region	Northwest region	Tibetan region
Our	0.089	0.103	0.080	0.115	0.085
Part 1	0.128	0.112	0.080	0.136	0.117
Part 2	3.208	3.627	2.622	3.387	2.613

Note: Bold values represent that the proposed model has the smallest MAE on testing set than two part models.

The proposed model was compared with other time series prediction models composed of time series decomposition methods combined with LSTM neural network. At the same time, the proposed model was compared with the linear regression model (LR) and the nonlinear regression model SVM, in which SVM uses temperature data and temperature data combined with STL decomposition data as input. Finally, we compare our model with the Bi-LSTM model used by Liang et al. for atmospheric temperature prediction.³¹

The comparison results are shown in Tables 5Table 6–7. The average of RMSE and MAE of the proposed model, STL-LSTM, and EMD-LSTM are 0.11 and 0.09, 0.35 and 0.27, 2.73 and 2.07, respectively. We can find that the RMSE and MAE of the proposed model are significantly lower than the comparison networks STL-LSTM and EMD-LSTM, indicating that the proposed model improves the accuracy of temperature prediction and can predict temperature more accurately. From the comparison results, the neural network-based methods outperform the regression-based methods in temperature prediction. Table 7 shows that the proposed model has the highest coefficient of determination, proving that the proposed model has outstanding performance and successfully realize the city temperature prediction of China. Figure 5 shows the visualization of predicted and actual temperature of Jixi using the proposed model, which also indicates that the proposed model can obtain outstanding prediction results.

Table 5

Comparison of RMSE (°F) on different prediction models.

Model	Testing set	North region	South region	Northwest region	Tibetan region
Our	0.108	0.127	0.096	0.139	0.103
STL-LSTM	0.338	0.359	0.323	0.403	0.306
EMD-LSTM	2.357	3.492	2.245	2.918	2.626
LR	4.343	4.82	3.483	5.141	3.496
SVM	4.276	4.882	3.424	5.095	3.541
STL-SVM	0.323	0.757	0.114	0.548	0.217
Bi-LSTM	4.0537	4.6882	3.4058	4.7769	3.3438

Note: Bold values represent that our model has the smallest RMSE among all comparison models.

Table 6

Comparison of MAE (°F) on different prediction models.

Model	Testing set	North region	South region	Northwest region	Tibetan region
Our	0.089	0.103	0.08	0.115	0.085
STL-LSTM	0.262	0.279	0.26	0.31	0.252
EMD-LSTM	1.804	2.576	1.569	2.333	2.086
LR	3.222	3.650	2.572	3.872	2.664
SVM	3.111	3.608	2.475	3.782	2.667
STL-SVM	0.116	0.216	0.071	0.25	0.108
Bi-LSTM	3.0815	3.5447	2.5454	3.6915	2.5445

Note: Bold values represent that our model has the smallest MAE among all comparison models.

Table 7

Comparison of R2 on different prediction models.

Model	Testing set	North region	South region	Northwest region	Tibetan region
Our	0.9999	0.9999	0.9999	0.9999	0.9999
STL-LSTM	0.9996	0.9994	0.9991	0.9997	0.9996
EMD-LSTM	0.9857	0.9577	0.9656	0.8995	0.9459
LR	0.9551	0.9574	09477	0.9595	0.9668
SVM	0.9564	0.9563	0.9495	0.9602	0.9659
STL-SVM	0.9998	0.9989	0.9999	0.9995	0.9998
Bi-LSTM	0.9411	0.9410	0.9133	0.9590	0.9511

Note: Bold values represent that our model has the largest R2 among all comparison models.

Fig. 5

Visualization of predicted temperature and actual temperature.

6.

Conclusion

This paper proposed a parallel STL-Bi-LSTM model, which combined STL and Bi-LSTM model to realize the prediction of city daily average temperature. Experiments showed that the prediction model combined with the STL can greatly improve the prediction accuracy. Moreover, the prediction loss of the proposed model is smaller than that of STL-LSTM and EMD-LSTM, which proved that the proposed model is very suitable for temperature prediction. The prediction model proposed in this paper can theoretically be used for temperature prediction in other countries, provided that sufficient temperature data of the country is used for network training, and the optimal network weights are obtained to realize the temperature prediction of the country. In this study, we only predicted temperature using decomposition components and original temperature series, more impacting factors will be considered into the model to improve the prediction accuracy in future studies.