Review of Computer Vision Applications in Fabric Recognition and Color Analysis

FAN Mingzhu(范明珠), XIN Binjie(辛斌杰), ZHU Runhu(朱润虎), DENG Na(邓 娜)*

1 School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201600, China2 School of Textiles and Fashion, Shanghai University of Engineering Science, Shanghai 201600, China

Abstract: Detecting various parameters of woven fabrics is one of the important methods to evaluate the quality of fabrics. In the early stage of industrial development, fabrics were mainly relied on manual to determine the quality, which was inefficient and unstable, so intelligent inspection is a popular development trend today. In recent years, computer vision technology has been widely used in the fields of fabric density measurement, color analysis, and weave pattern recognition. Based on the above three aspects, the advanced research progress of global researchers is reviewed in this paper and the shortcomings of current research and possible research directions in the future are analyzed. Computer vision technology is not only objective evaluation, but also has the advantages of accuracy and efficiency, and has a good development prospect in the field of textiles.

Key words: woven fabric; computer vision; density measurement; color analysis; weave pattern; deep learning; yarn location

Introduction

The textile industry is an indispensable and important part of the global economy. The basic structure of the fabric plays a key role in the mechanical properties of the fabric and the appearance of clothing. Therefore, the research on the detection of fabric parameters has always been a hot spot. The development of the textile industry towards automation and intelligence is the general trend. Computer vision has a very broad application prospect in the textile industry.

The traditional method of detecting fabric quality is through manual observation on the factory assembly line, which has the disadvantages of long detection time and low efficiency. It is also easy to be affected by the proficiency of inspectors and personal subjective feelings. In view of this situation, researchers began to apply computer vision technology to the field of textile inspection, trying to use it to identify fabric textures automatically. In 1986, Akiyamaetal.[1]first used the optical principle to classify weave points by analyzing the fabric diffraction pattern. On this basis, the relationship between the diffraction pattern and the weave pattern is established. Since then, computer vision technology has been widely used in the field of fabric recognition.

The woven fabric is interlaced by warp and weft yarns vertically. The cross state of the warp yarn on the weft yarn is defined as warp weave point, and vice versa is the weft weave point. The weave pattern is usually composed of plain, twill and satin, which are the three most common patterns. The recognition of fabric texture depends on the analysis and processing of fabric structure and color features. The method with high accuracy and generalization is a very challenging task. This paper summarizes from three aspects: density measurement of fabrics, color analysis of fabrics, and recognition of weave pattern.

1 Density Measurement of Fabrics

Fabric density refers to the number of yarns per unit length, which can be divided into warp density and weft yarn density. The methods of measuring fabric density are divided into three categories. The first method is the frequency domain method, but it only has a very accurate detection effect on pure color fabric[2-3]. The typical algorithm is that after preprocessing of the pure color fabric image, the fast Fourier transform can find the first two brightest points on the abscissa and ordinate in the frequency domain diagram to calculate the number of warp and weft yarns[4-6]. Wavelet transform, co-occurrence matrix[7-8]and gray projection can also be used to obtain the density of pure color fabrics[9-10]. Zeng[11]also proposed a Hough transform fast detection algorithm based on multi-threshold analysis to correct the inclination of the filtered fabric image, and then obtained the sub-image containing warp and weft yarn direction information with an adaptive orthogonal wavelet transform to calculate the warp and weft yarn densities of the two reconstituted fabrics. Jingetal.[12]used a similar method and the detection accuracy rate reached 99.92%.

The second method is mainly for multi-color fabrics. There is no regularity to find in the frequency domain which is more than two-color interference[13]. Researchers chose to use gray projection in the spatial domain[14]. This algorithm was to project the fabric image in horizontal and vertical directions respectively based on the characteristics that the pixel value of yarn gap would change to eliminate the interference of color and obtain the yarn position information. Aldemiretal.[15]proposed a filtering method integrating linear and nonlinear that using anisotropic diffusion filtering to avoid the common blur and positioning error to enhance the edge region between adjacent yarns of fabrics. Gabor filter was often used to correct the inclined gray image and enhance the edge information. On this basis, the gray projection method was used to detect the fabric density of which the accuracy was more than 90%. The third method is deep learning which has been a popular algorithm in recent years. Preparing a large fabric dataset for training and testing is the first step of the experiment[16]and innovations to convolutional neural networks(CNNs) are usually improved on the framework. Although deep learning could obtain high accuracy and robustness, there are several problems. (1) It is difficult to collect many high-quality fabric samples. (2) The adaptability of convolution network to different sample detection needs to be improved. (3) The running time of the algorithm is expected to be further shortened. Table 1 shows the results and accuracy comparison of typical algorithms for fabric density measurement in recent years.

Table 1 Typical algorithms for fabric density measurement

2 Color Analysis of Fabrics

2.1 Printed fabric

Color segmentation played an indispensable role in the production and analysis of printed fabrics. It was an important step before subsequent dye dyeing and plate forming. Researchers have done a lot of work on the color segmentation of printed fabrics. Generally speaking, it could be divided into two categories: methods based on unsupervised clustering[22-25]and supervised clustering[26-28].

The idea of the unsupervised clustering method was to divide the color of printed fabrics in different color spaces[29], but the robustness of the algorithm was not high. A typical example of unsupervised clustering is fuzzy C-means (FCM)[30-31]. Generally, the segmentation effect in LAB color space is better than that in red-green-blue(RGB) and hue-saturation-value(HSV) color spaces, especially when multi-color is similar and the saturation values are different.

Supervised clustering combined neural networks with genetic algorithms, histogram segmentation, or principal component analysis (PCA) to segment colors[32]. Although it could obtain high accuracy, this method needed to collect and train a large data set, which consumed a lot of time and energy.

For complex pattern segmentation like the camouflage fabric, Zhuetal.[33]used the mean shift clustering algorithm to segment the image into color blocks of different colors and then used neural network regression to realize the transformation of the image from RGB color space to LAB color space. In addition, the normal distribution method was used to analyze and measure the camouflage color, and the camouflage pattern segmentation was realized. This method overcame the difficulty of color measurement caused by the diversity of fabric texture and pattern. Compared with the color measurement results of the data color spectrophotometer, the average color difference obtained by normal distribution analysis was no more than 2.

2.2 Yarn-dyed fabric

Many color segmentation methods were for the analysis of printed fabrics and pure color fabrics. There was less research on yarn-dyed fabrics. In addition, clustering division in RGB color space had a good classification effect for fabric images with obvious changes in brightness, while the classification effect was not good for fabric images with relatively similar brightness values and obvious changes in hue values. Further, the classification effect that clustering in Hue-Saturation-Lightness(HSL) color space was poor for fabric with similar brightness value and hue value and obvious changes in saturation value[34].

There are basically two methods for color analysis of yarn-dyed fabrics introduced in this section. (1) Without segmenting a single yarn, the fabric image was preprocessed and combined with a neural network or clustering to separate the color. (2) The yarn of multi-color fabric was located so that the color of a single yarn could be clustered and analyzed.

Fabric color separation without yarn segmentation usually used filtering combined with co-occurrence matrix[35], color histogram[36-37], and others to preprocess and extract features[38]. In the phase of color analysis, genetic algorithm[39], back propagation neural network (BP neural network) and traditional clustering methods[40-45]were used for recognition. Although the overall analysis accuracy of the color for yarn-dyed fabrics by this technical route was good, it did not locate the color of a single yarn in combination with the characteristics of yarn-dyed fabrics.

Another method is to separate the yarn first, which can analyze the color of the single yarn to the whole fabric image. The focus of image preprocessing was to enhance the contrast between yarn and gap and tilt correction[46-49]to lay a good foundation for positioning yarn. Like the key methods in the above density measurement, this step used the spatial domain or frequency domain method to locate yarns and then used the clustering method such as FCM[42, 49]and X-means[50]to analyze the color.

3 Recognition of Weave Pattern

An indispensable part of fabric structures is to classify the type of weave points to obtain the weave pattern. At present, many research ideas in this field basically include two stages: feature extraction and weave point classification. Before these two steps, the fabric image is preprocessed by image denoising and enhancement to better extract features. The research on fabric weave and pattern recognition is reviewed in this part.

3.1 Feature extraction

The prerequisite for classifying warp and weft points is to ensure the quality of their feature extraction. For woven fabrics, the object of feature extraction is divided into a single weave point and a small weave repeated region.

The premise of feature extraction for a single weave point is that the accuracy of yarn positioning should be high. In order to minimize the interference of color and other factors, the fabric image would be preprocessed to eliminate sharp noise and enhance the contrast of the image, which could make better use of Fourier transform and gray projection for yarn positioning. Then, frequency domain or spatial domain methods were used to extract features to lay a good foundation for the next classification stage.

For the feature extraction of weave repeated region, frequency-domain methods such as Fourier transform and Gabor transform were used to obtain the regularity of structural changes to find the extreme point of the spectrum diagram, so as to determine its change characteristics and draw the weave pattern. The matrix method in the spatial domain included the gray level co-occurrence matrix. Although the 14 texture features extracted by its description of direction and distance had good discrimination ability, the calculation was very time-consuming. Therefore, many researchers reduced the amount of calculation by reducing the gray level of the image, or by adjusting the distance and direction scales.

The comparison of recognition accuracy using different feature extraction algorithms and the same classification method was that Salemetal.[51]tested and compared Gabor’s combination of wavelet transform, the local binary mode operator method, and the gray level co-occurrence matrix method (GLCM) to extract feature parameters. On this basis, support vector machine (SVM) was used to classify features. Among them, the accuracy of the Gabor wavelet and GLCM fusion method is 98%, which is greater than the accuracy 97.2% of the GLCM, but the running time of the fusion method is a little longer. In 2014, Jingetal.[52]also used GLCM and Gabor wavelet to extract features, combining principal component analysis (PCA) to process texture feature vectors to obtain the minimum redundancy and maximum principal component feature vectors. Subsequent classification results showed that, compared with the GLCM method and the Gabor wavelet method, the classification accuracy of the fusion of the two feature vectors reached 95%. In 2022, Murthyetal.[53]established a fabric texture classification system based on GLCM features on inferred texture units. In the TILDA database, the average classification accuracy of this framework is 96.94%.

In the method of using the space domain, the more representative one was to use the gray projection method to locate the yarn. In 2014, Xiaoetal.[54]used the gray projection method to segment the intersection of warp and weft yarns, combining the gradient histogram information to obtain the texture characteristics of the intersection to judge the type of the points. Later in 2015, Kuoetal.[55]used the same method to improve the characteristics of warp and weft yarns by horizontal projection and vertical projection respectively and drew an organization chart. In 2018, Xiaoetal.[56]located and segmented the weft and warp yarns based on the gray projection of the 2 D space domain. In the feature extraction process, histogram of oriented gridient (HOG) was used to extract significant invariant features to obtain the texture features of the woven fabric. Finally, the FCM is used to classify the warp and weft points.

Now more researchers are focusing on deep learning, and the feature extraction results related to this technology will be introduced together with fabric recognition and classification in the next section.

3.2 Recognition and classification

Texture classification is not only a key step in the field of computer vision and pattern recognition, but also the basis of image segmentation, object recognition and other visual tasks. Fabric pattern recognition is practically equivalent to fabric texture classification[57].

The classification of weave points is a binary classification. The commonly used methods were FCM, SVM and a class of neural network algorithms. The weave pattern generated after weave point recognition which could be classified into plain, twill, satin, or other types. Recently, many scholars used deep learning method to solve the problem of feature extraction and classification, which was the focus of current research. The research group of the author[58]is about to publish the research on fabric texture analysis based on images collected under composite light sources. On the transmission image, gray projection is used to locate the yarn, and the position information is matched to the corresponding reflection image. The feature extraction and classification of organization points are based on reflection images. The improved FCM-K-nearest neighbor(FCM-KNN) algorithm is used in the classification algorithm, and the classification results are effectively obtained. The flow chart of the fabric texture recognition system and the FCM-KNN algorithm are shown in Figs.1 and 2, respectively.

Fig. 1 Flow chart of the fabric texture recognition system

Fig. 2 Flow chart of the FCM-KNN algorithm

At present, foreign scholars and domestic universities have conducted a lot of research on identifying fabric structure. Table 2 shows the accuracy of typical methods.

Table 2 Accuracy of typical methods

SVM is not only insensitive to outliers, but also easy to grasp key samples, and has good robustness characteristics. Lietal.[61]used SVM as a classifier to recognize fabric patterns, which could automatically and accurately classify plain, twill and satin fabrics. Texture classification using SVM as a classifier has been performed with high accuracy[62-63]. SVM has also achieved good results in fabric defect detection[64-66]. Panetal.[67]used the FCM to divide the warp and weft points and then used the BP neural network to recognize the fabric pattern which had a certain fault tolerance. Kuoetal.[68]used the FCM to do the same thing. Fasahatetal.[69]used wavelet transform and clustering methods to preprocess yarn-dyed fabric images, combining with genetic algorithms to optimize the obtained range and detect yarn positions. The average accuracy reached 96.20%. In high-density texture fabric images, the average accuracy was also greater than 94.72%. Xiaoetal.[70]proposed a new method for recognition based on transform invariant low-rank texture (TILT) and HOG-FCM，which provided a good idea to solve the skew problem. In the current research on this part, large-scale neural networks are very popular. PNN has strong real-time, fault tolerance, and nonlinear classification capabilities. Jingetal.[52]chose PNN to classify fabrics accurately and effectively, with an accuracy of over 95.00%.

Fig. 3 Structure of the MTMSnet

Hussainetal.[71]proposed a deep learning model for classification and recognition of woven fabrics based on data enhancement and transfer learning methods, using residual network to extract and classify fabric texture features end-to-end automatically. Experimental results showed that the model had good robustness and could reach the most advanced accuracy even when the physical properties of the fabric changes. The application of CNN in fabric texture analysis has been very popular in recent two years. Here are some relatively outstanding studies. Mengetal.[60]established a fabric data set by using a portable wireless device, which contained a large number of fabric types and structure parameters. A new multi-task and multi-scale CNN shown in Fig. 3 was proposed to predict the position map of yarns and floats, and the basic weave cycle of the fabric was obtained by combining with the map and intersections. Rangkutietal.[72]sent the preprocessed image to the CNN network for image segmentation, feature extraction, and classification. The recognition and classification of traditional cloth patterns reached the highest accuracy of 99.7%. The model proposed by Huangetal.[73]integrated two advanced architectures, Deeplab-V3+ and Ghost-Net, and realized fabric texture based on lightweight networks. Raufetal.[74]used Res-Net to classify plain, twill and satin in real time. With the acceleration of the training speed, the depth CNN classifier achieved the best accuracy of 96.15% in identifying woven fabrics.

4 Conclusions

Traditional fabric pattern recognition and color analysis methods rely on people’s subjective consciousness to judge which is not only time-consuming and inefficient but also influenced by the inspector’s knowledge and experience. Compared with traditional subjective methods, computer vision technology has the characteristics of fast, accurate, and reliable quality. Currently, the most widely used deep learning methods have achieved high performance, but there are still some limitations. For example, the supervision of data sets will consume a lot of time, and there is still much room for improvement in the recognition accuracy of double-layer fabrics or more complex fabrics. The future research direction is possible: (1) building a new unsupervised model; (2) exploring new end-to-end methods; (3) establishing data sets containing a large number of fabric image parameters for analysis.