1.

Introduction

Ink and wash paintings (IWP) are distinguished from Western art in that it is executed with the Chinese brush, Chinese ink, mineral, and vegetable pigments. In particular, many IWP artists used monochromic ink to paint and sometimes did not even use gradually changing tones. Painting was regarded as an integrated form of art with calligraphy, for which color is usually not used at all. As a result, IWPs primarily rely on the use of brush strokes to represent the style of an artist. Therefore, art work such as IWPs need special image analysis methods. For instance, a significant genre of IWPs is Shan-Shui paintings, in which mountains, stones, trees, rivers, waterfalls, and small pagodas are prominent, as shown in Fig. 1. As seen, there is little to compare among these paintings in terms of image content. An important factor that art historians often evaluate when analyzing and comparing paintings is the characteristic strokes used by artists. In Western arts, many impressionism masters formed their styles by special strokes, such as the swirling strokes of Van Gogh and the dots of Seurat. Synergy can also be found in Chinese arts, where Huang Gongwang, an artist of the Yuan Dynasty, is one of the representatives renowned for creating a way of painting mountains using calligraphic style brush strokes.

Fig. 1

Illustration of typical Shan-Shui painting by Huang Gongwang.

Existing work reported in the literature for Chinese artist classification is typified by image processing approaches as reported by Li and Wang and Jiang et al.^1,2 Li and Wang¹ developed a general classification framework for Chinese paintings, in which a mixture of two-dimensional multi-resolution hidden Markov models (MHMM) is designed as statistical modeling to characterize the properties of strokes from different artists. After training the models, the methods can be applied for categorization of images. In the study, the Daubechies four wavelet coefficients are considered as the content representation for each image. The algorithm has been evaluated using a test data set with 276 paintings from five different Chinese artists. In their research, only the luminance component of pixels is used to extract a number of local features for classification. When five artists are included inside the data set, they reported a precision rate of 62%. The best precision rate reported is around 80% when the classification problem is scaled down to have only two artists. Jiang et al.² introduced an algorithm which tries to categorize traditional Chinese paintings into Gongbi (traditional Chinese realistic painting) and Xieyi (freehand style) using low-level features and a support vector machine (SVM) classifier.

On the other hand, research on image processing approaches for classification of Western arts has been quite active for the past decade.^3–8 Shen³ recently reported some work on classifying Western arts, in which both global and local features are used to include color, texture, and shape. While color layout is used to describe the global features, small image patches filtered by Gabor filters are used as the local features. Both global and local features are fed into a neural network to complete the classification by combining the classification score for global and local features, respectively. As such image processing approach relies on color elements to complete the classification, it is normally not applicable to the problem of IWP classification since most IWPs do not have color information. Johnson and Hendriks⁴ surveyed the brush analysis and artist identification techniques. Melzer et al.⁶ explored how to apply different type of brush strokes to classify portrait miniatures. Both a model based and a semi-parametric neural network approaches are used as the detector. Sablatnig et al.⁷ studied using “structural signature” based on brush strokes for solving the artist classification problem. A hierarchically structured classification scheme is developed to separate the classification into multiple forms of low-level features, including color, shape, and structure of brush strokes. Xiao et al.⁸ studied the machine learning approach, such as SVM, to achieve excellent classification results for input images and videos.

In this paper, we explore a style-based approach to recognize Chinese artists via automatic classification of IWPs by following the similar principle that local and global features are extracted to describe the artistic styles and drive a number of neural networks arranged in parallel for completion of the IWP classification into the authoring artist. To integrate the consideration of both global features and local features, an entropy balanced fusion scheme is also described to reduce the negative effect of IWP content. Due to the nature that most IWPs do not have color components, we propose to use the grayscale histogram as the major technique to construct style descriptors in characterizing the distribution of the strokes.

The rest of the paper is organized into three sections. Section 2 describes our proposed algorithm for IWP classification, and Sec. 3 reports experimental results and evaluations. Finally, Sec. 4 provides concluding remarks.

2.

Proposed Algorithm

Throughout its long history, IWP has carried its own particular perceptual style that only brush strokes provide reliable information to identify artists.¹ In this paper, we devote to extract various painting styles from art work in order to automatically classify them into their artists.

Specifically, Chinese artists mainly rely on different strength in shaping the lines or contours with multi-level distribution of heavy and thick or swift and thin strokes to express different appearance and emotions inside their IWPs. Inspired by many reported image processing techniques, we carried out some empirical studies by extracting simple gray-level histograms^9,10 out of the entire image as its global feature and out of a local region as a local feature. We found that grayscale histograms remain effective in describing the distributions of stroke-based information, which show good potential in automatic classification of IWPs. Figure 2 shows that IWPs by five different artists (Xu, Wu, Huang, Zheng, and Liu), have widely different histograms marked by various colors, in which Fig. 2(a) indicates the difference in global features, and Fig. 2(b) reveals the difference in local features. As illustrated in Fig. 2(a), for example, Xu and Liu reach sharp peaks at intensity levels around 230 and 250, respectively, which verify that their paintings are characterized by a lot of marginal lines. In addition, Xu reaches the other vertex at a low intensity level of about 10, indicating that Xu integrated firm and bold brush strokes with the precise contour of objects in dark ink. Meanwhile, this is also related to his habit of painting in that he used to apply freehand techniques in certain areas with highlights, such as a horse’s head, neck, abdomen, legs, and hooves with heavy and thick strokes. In contrast, Huang is characterized by a gentle distribution and finely dispersed curves which are identical with his stroke styles. From observation, it is also seen that Huang always applied flat controlled strokes, broad-range bold ink wash and calligraphic style to a graphic formula—rock forms filled with straight, parallel, “hemp-fiber” texture strokes and layers of horizontal dots.

Fig. 2

Empirical study upon IWP style comparison across different artists. (a) IWP differences indicated by the global feature; (b) IWP differences indicated by the local feature.

Figure 2(b) shows that the curves of intensity distribution by both Wu and Zheng have two obvious wave peaks, which typically verify their styles, i.e., subjects in their brush are thin, clear, forceful, and steep. At the same time, thick and light lines occupy more than half of the space. For example, Zheng has two wave peaks at intensity levels of 50 and 150, respectively, which indicates that tight structure is seldom in his paintings and his technique involved both dry-brush and wet-brush strokes, including forceful and thick strokes as well as swift and thin strokes. In particular, the small leaves of bamboo are characterized by using vertical strokes with heavy wash, while the big pieces use horizontal strokes with light wash. In addition, Liu reaches his wave peak at intensity levels of 180, which is also consistent with his artistic style that a monochromatic under-painting is drawn out in complete detail is overlaid with swift and thin washes with various combinations.

Based on the above analysis of our empirical studies, we propose a style-based arts classification algorithm as described in Fig. 3, where both global features and local features are extracted from IWPs to drive an entropy balanced neural network classifier to complete the classification of IWPs. In the proposed algorithm, global visual features contain image characteristic from the coarser level including the appearance related to different types of paintings such as human objects, landscape and flower and bird, the ink blot power from the global perspective and different styles for various artists in different period. The local features, on the other hand, focus on the local block of an image and hence pay more attention to the detailed painting style, typically described by different brush techniques while using ink. Therefore, global and local visual features extraction play complementary roles in the proposed algorithm, although both of them contain various discriminative powers with respect to artists’ identification.

Fig. 3

Systematic illustration of the proposed algorithm.

To advance an effective algorithm to distinguish artists according to their multifarious stroke styles, we propose a simple preprocessing step to convert input IWP images into grayscale ones, ensuring that color information is excluded from the classification process as color is not available in IWPs.

Following the preprocessing as illustrated in Fig. 3, both global features and local features are extracted to describe the style of artists inside their IWPs for their automated identification, where the construction of features are mainly based on grayscale histograms.¹⁰

Let $\{R_1,R_2,\dots ,R_i,\dots ,R_T\}$ be a subdivision of the intensity scale [0, 255], a global feature can be extracted as follows:

For local features, the basic principle is to find the most representative local region, where a grayscale histogram can be constructed to characterize the details of brush strokes, including the strength, the distribution, the shape and the contour, etc., as observed and analyzed in the previous section. Considering the balance between the efficiency and effectiveness in local feature extraction, we design the local region to be a square block of $64\times 64\mathrm{pixels}$, which is sufficient to contain structural attribute of an image yet maintains a reasonable computing cost. In addition, the entire process of local feature extraction should be considered in terms of two aspects: (1) to detect the most representative stroke; and (2) to locate the most representative local region. For the first aspect, we propose to apply edge detection technique to detect all strokes inside the IWP, and for the second aspect, we apply a window function to filter all detected edges by moving the window along a raft-scanning order to cover the entire area of the IWP image. Details of the operations are given below:

Given the grayscale image $G(x,y)$, we extract all brush strokes inside the corresponding IWP via:

To detect the most representative local region, we examine a block of $64\times 64$ via the following operation:

Inside Eq. (4), $\omega (i,j)$ is a window function which is defined as

The most representative block, ${\text{Block}}_L(i,j)$, is determined via

Essentially, the local region detection combined with Sobel edge detection is designed to follow the principle that: (1) edges detected indicate the position of the brush strokes; (2) the selected local region ${\text{Block}}_L$ contains the most densely populated edges, and hence contains the most details of the artistic style (brush strokes). Such design achieves a number of advantages, which can be highlighted as: (1) Since the selection of ${\text{Block}}_L$ is dependent on the density and the distribution of edge pixels, the value of sensitivity threshold $\tau$ inside Eq. (3) would not have any effect upon the local feature extraction; (2) local features can be directly extracted from the grayscale image and its local histogram without considering the shape or contour of each stroke, which could be very complicated from the edge information alone.

Figure 4 illustrates the local feature extraction process by using a random selection of the five artists, where Fig. 4(a) shows the results of edge detection and identified representative local block, Fig. 4(b) describes the corresponding desirable local regions indicated in grayscale images, Fig. 4(c) reveals the result of local feature identification on the selection of the five artists using Eq. (6), and Fig. 4(d) illustrates the histograms of the desirable region which represents the local feature.

Fig. 4

Illustration of local feature extraction corresponding to a random selection of different artists: (a) the local blocks with the highest density of detected edges; (b) the corresponding desirable local regions indicated in gray-level images; (c) magnified illustration of the desirable local region given in (b); and (d) histograms of (c) which represent the local style descriptors.

The final stage of our proposed algorithm is to use a back-propagation neural network, which is a direct adoption of the standard BP NN unit, to complete the recognition of traditional Chinese artists by classification of the global feature and local feature, respectively. To integrate the decisions made from the global feature based classification and the local feature based classification, we propose an entropy balanced fusion scheme as described below.

Let $\xi (x)\in [{\alpha }_1,{\alpha }_2\dots {\alpha }_M]$ be an auto-classification function, ${\alpha }_j$ represents the Chinese artist to be recognized and $M$ represents the total number of artists. We calculate the entropy, $H(G)$ and $H(L)$, for global feature and the local feature from their individual histograms, respectively. The final classification results are derived via the following fusion process:

3.

Experimental Evaluations

To evaluate the proposed algorithm, we prepared an IWP database with collections of 200 IWPs authored by five famous traditional Chinese artists. A summary description of such a database is given in Table 1, in which 50% of the IWPs are used as the training data set and the rest is used as the test data set, and no overlap between the training set and the test set is allowed throughout the experiments reported in this section. Figure 5 illustrates a group of representative examples to show a general flavor of the IWPs used for the evaluation of the proposed algorithm. As seen, the collected IWPs among different artists not only illustrate a common feature in terms of their presentation style in describing objects like mountains, horses, bamboos, and natural scenes, but also illustrate that each artist has his own artistic style in drawing natural objects even with the similar content. In addition, it can be seen that the size of the paintings varies across different artists, which is typical in the form of traditional Chinese paintings to suit different purposes. Our proposed algorithm, however, will not be affected since the feature extraction process is independent from the painting sizes. While the content of all IWPs is not exactly the same, the principal content elements remain similar, such as flowers, animal object, trees, and leaves etc. As a matter of fact, some artists do share the same object as seen with horses at the top row and bottom row, as well as tree leaves across different artists in Fig. 5. From top to bottom, each row of the samples corresponds to one artist to follow the order of Xu Beihong (1895 to 1953), Wu Changshuo (1844 to 1927), Huang Gongwang (1269 to 1354), Zheng Banqiao (1693 to 1765), and Liu Danzhai (1931 to 2011), representing a typical group of most influential Chinese traditional artists across a wide range of historical periods from 13th to the 21st century. In addition, the painting samples shown in Fig. 5 also show that these artists have their own styles in formulating strokes and drawing lines to construct similar objects, providing sufficient spaces for research on visual information processing to digitally describe their style features.

Table 1

Description of ink and wash paintings (IWP) collection database established for the experiments.

Fig. 5

Representative samples of the data set for experimental purposes.

Following the spirit of image retrieval,^11,12 we adopt precision and recall rates to measure the performance of our proposed algorithm. To provide a comprehensive evaluation of the proposed algorithm with regards to the research problem, we designed the experiments as such that two important factors are considered and compared. The first factor is the number of artists considered for recognition, in which three cases are designed, including recognition of individual artist among two randomly selected artists, four randomly selected artists and all five artists. The second important factor is to evaluate the effectiveness of style feature descriptions proposed in the paper, which can also be organized in three cases, including global feature, local feature and their entropy balanced fusion. Table 2 summarizes all the experimental results in consideration of both factors and all three cases, where three different algorithms are assessed for comparative analysis purposes, which include the proposed, MHMM,¹ and C4.5 decision tree method.¹³

Table 2

Summary of experimental results.

In consideration of the first factor, the results summarized in Table 2 indicate that the number of artists does not have significant influence upon the recognition results. As an indicative example, the average precision rates for all the three cases are {90, 87, 83} for the proposed classification algorithm (entropy-balanced), where the maximum average difference across all the cases is actually less than 5%. Similar conclusions can be established for all other cases. This indicates that the proposed algorithm does have a certain level of robustness in terms of artist recognition by their authored IWPs.

In consideration of the second factor, the results summarized in Table 2 indicate that the global style descriptor generally performs more effective than local style descriptor, respectively, suggesting that consideration of style description over the whole IWP tends to be more discriminating than that of local details. However, such a conclusion is not unanimous since there exist a number of cases where the local feature based recognition is better than that of global features. These include both precision and recall for case 1 and some artists in case 2 and case 3. Such results present a helpful argument to support the principle adopted for the proposed algorithm that histogram-based description of IWP styles for different artists is a powerful approach. While global feature description focus on general distribution and configuration of all strokes applied for such paintings, the local features proposed concentrate on details of individual strokes in reflection of uniqueness embedded inside each individual artistic style. Further, the results in Table 2 also illustrate that the proposed entropy-balanced fusion scheme unanimously outperforms all other cases, proving that the proposed fusion algorithm successfully draws on the strength of both global features and local features in describing different artistic styles, leading to more effective recognition results.

Comparing with the MHMM method¹ and C4.5 decision tree methods,¹³ the proposed method delivers considerably better results on average of precision and recall in all the three cases. As seen, when the number of artists is increased, the classification results achieved by the proposed actually reduces slower than that of the other two benchmarks.

4.

Conclusions

As one of the significant cultural heritages, IWP paintings have always played a crucial role in modern Chinese life and been applied for many different purposes, mainly regarded as valuable treasures for collections. In this paper, we described an algorithm for automatic recognition of traditional Chinese artists via classification of their IWPs. Inspired by the traditional Chinese art historians in examining the art piece and judging whether it is genuine or a fake, we proposed to characterize the painting style of brush strokes via global features and local features. While global feature provides descriptions of brush strokes on the scale of the whole IWP region, local features focus on a small region to describe details of the brush stroke, including its strength, distribution, shape, and contour. To integrate the classification results, we further proposed an entropy-balanced fusion scheme to stifle the influence of content or information and strengthen the style description and characterization in reaching the final decision. Experimental results reveal that the proposed algorithm outperforms the existing representative benchmarks, including MHMM¹ and decision tree C4.5 based techniques,¹³ providing excellent potential to be further developed into a useful tool for management and retrieval of traditional Chinese IWP arts.

While our approach illustrates some initial progress toward computerized IWP classification via image processing approaches, how to interpret the strokes into artistic style descriptors and digital features remain a challenging research issue. In this regard, out approach actually relies on the statistics technique to characterize the stroke features and complete their artistic style description. One of the possible directions for further research can be identified as digital translation of the stroke formulation process via their physical structures and details, such as their shape, contour, depth and trajectories etc., to provide more accurate digital description and hence producing higher differentiating power for computerized classification of IWPs via machine learning techniques.