1.

INTRODUCTION

Machine learning (ML) becomes a popular approach in industry applications¹. However feature extraction is still a time-consuming step in the applied machine learning projects. It is heavily relied on data analyzer’s experience and domain knowledge, which hinder ML’s broader expansion to non-experts.

Automated machine learning (AutoML) is an exploration to ease the process of building ML models by automating commonly used steps, such as feature extraction & preprocessing, model selection, and hyperparameters tuning². For example, Featuretools³ is a Python library for automatically engineering features from relational and transactional data. The library introduces the concept called Deep Feature Synthesis (DFS). For multiple datasets with relationships defined among them such as parent-child based on primary keys (or unique identifiers columns), DFS can create new features based on certain calculations, such as summation, count, mean, mode, standard deviation, and so on. Such a data schema based feature extraction approach can be extended to industry domains by considering causal relationships.

To leverage causal relationship in feature extraction, a formalized description model is a prerequisite. In industrial Fault Detection and Isolation (FDI), several graphical models have been proposed to describe the causal relationship of dynamics system. Ould Bouamama et al.⁴ divide them into 3 categories, (1) structural graph based ARR (Analytical Redundancy Relation) including bond graph, bipartite graph, linear graph, etc. (2) qualitative graph based methods such as causal graph, functional graph, signed directed graph, etc. (3) causal probabilistic model including Bayesian network and dynamic Bayesian network. The interest of using graphical methods arises from its easy construction, and ability to obtain main properties (monitorability, diagnosability, sensor placement, observability, controllability), which can be provided before industrial design. Sztyber et al.⁵ introduced a new causal graph called as GP (Graph of a Process). GP can be constructed on the basis of process diagrams and expert knowledge. Vertices of the graph represent process variables, disturbances, and control signals. Directed edges depict relations between variables.

This paper aims to explore the feasibility of automatic feature extraction in in industrial or manufacturing domain, Since in industrial domain, system dynamics diagram, piping & instrumentation diagram (P&ID) and other system diagrams contain rich “structural relationship” between raw variables implicitly. If properly captured and harvested, many useful features can be extracted automatically to ease machine learning process. The paper is structured as follows. A system dynamic diagram (SDG)⁶ approach is discussed in Section 2. The extraction algorithms of typical model structures in SDG, including anchor variable, path, branch, and closed loop, are presented in Section 3. Automatic feature extraction policy to different model structures is discussed in Section 4.

2.

SYSTEM DYNAMICS MODELLING

To be consist with previous work, the symbolic representation by Sztyber et al. is adopted with minimal extensions, as shown in Figure 1. The first 3 symbols are the same as the work of Sztyber et. al. That is, measurements are marked with black circles, control signals with white circles with a black dot, and unmeasured state variables are marked with white circles. Besides these 3 nodes, 3 node types and 2 linkage types are defined. To clearly model the branch and assemble structure, a virtual node is added by a circle with dash line. This is useful in many industrial processes. For example, in the adaptive water feed system, the water of 3 waterfeed subsystems comes from the same main water pipe, the gas generated from each subsystem are feeded into the same steam pipe to the turbine generator. Group node is used to denote that multiple variables are co-located, such as pressure and flow of water main pipe. To identify the control objective, objective symbolic is added. Linkage are divided into 2 types as physical linkage and logic linkage. Logic linkage is used to represent the designed control logic, and physical linkages is used to model the interaction in the physical world. Since the purpose of this paper is not just about fault diagnosis, fault node type in Sztyber is ignored here. Group Node and Branch/Joint Point are categorized as virtual node, since they are not physically existed but a logic node to represent system structure. The type of other 4 nodes are called as basic node.

Figure 1.

Symbol representation of modeling elements.

An adaptive waterfeed subsystem in power plant is used as an example in this paper to illustrate the conceptions and algorithms. The system receives water from another subsystem by a pump. Feed water flow rate is adjusted to the plant’s target power output by controlling pump’s speed. The water is fed into 3 steam generator branches. Each branch’s water flow rate is controlled by valve’s openness. The steam from 3 branches is merged into a main gas pipe to drive steam turbine for power generation. The expected water level in steam generator is set according to target power output. The expected water level is automatically maintained by adjusting related valve openness. The P&ID diagram is shown as Figure 2a⁷. According to previous description, the control policy diagram can be shown as Figure 2b.

Figure 2.

Schematic diagram of adaptive waterfeed system.

The system dynamics diagram of the water feed subway is shown in Figure 3a.

Figure 3.

System dynamics diagram and it merging process with control policy. Loop 2 and Loop 3 are similar to Loop 1. To reduce diagram complexity, they are omitted and denoted as cloud shape.

The control policy can be merged with original SDG of the system. The merge algorithm is shown in below. The process can be illustrated as Figure 3.

After the merge, we obtain a directed diagram with vertex and linkage. Here are some annotations for the convenience of the discussion hereafter.

3.

TYPICAL STRUCTURE IDENTIFICATION

With the merged system dynamics diagram, 4 types of model structures can be identified. Different model structures have different feature extraction policy, which will be discussed in next section.

3.2

Path and branch identification

A path is a directed physical linkage between nodes, which starts from a basic node, and end with a virtual node, or a basic node without outbound physical linkage. The identification algorithm is shown below.

The paths in Figure 3 are shown in Figure 4. If two paths has the same previous node and the same ending node, they are called as parallel branches.

Figure 4.

Path identification illustration.

3.3

Control loop identification

Under a fault condition, target variable sometimes can still achieve expected configure value by a closed control loop. But a control variable usually exhibits abnormal behavior. Thus control loop structure is also important to feature extraction. The identification algorithm is shown below.

The control loop identification process of Figure 3 can be illustrated as Figure 5.

Figure 5.

Control loop identification illustration.

4.

FEATURE EXTRACTION POLICY

For the variable with the same unit (for example, centigrade degree), PCA/PLS/ICA and other linear/nonlinear feature extraction methods are more meaning than those operator cross different physical units.

For different variables in the same path structure, if they are in the same physical units, differences between them are also meaningful. For example, pressure drop after passing a valve, temperature drop with a pipeline. For variable with different units, grammar based genetic algorithm can be used⁸. Grammar defines the possible combination.

For the branch structure, the pair comparison or allocation ratio are often meaningful. For example, if the flow of one pipe network branch is much lower than others, there might exist pipe blockage.

For the loop structure, transient behavior of objective variables or value distribution of control variables are important feature in anomaly detection.

Anchor variable clustering means different working conditions. Cluster information can be used as categorical features in data analytics.

5.

CONCLUDING REMARKS

In this paper, a system dynamics diagram (SDG) based feature extraction approach is proposed. SDG is adopted to capture the relationship between variables. Merge algorithm of P&ID and control policy diagram is presented. SDG structure identification algorithms are introduced, including anchor variable, single path, parallel branch, and closed-loop. For different SDG structures, different feature extraction operators can be configured. The approach is illustrated with a feedwater flow control system. Such a formalized graph structure based automated machine learning can significantly reduce feature extraction efforts. There are several extension directions. Firstly, SDG in this paper is given by domain experts. A data aided approach¹⁰ will be useful when there is no clear mechanism relationship existed. Secondly, SDG model simplification and consistency checking algorithms are necessary in a complex SDG situation. Thirdly, the feasibility of a predictive model¹¹ can be deduced according to SDG structures. All of these works will enrich the synergy of data-driven approach and domain knowledge driven approach¹² in industrial data analytics.