Abstract:As a color consistence model derived from human visual system research, Retinex has been widely used in processing uneven illumination and color shift images. This paper first introduces the fundamental principle and subsequent development of Retinex theory. Then according to the current research status, the Retinex model is divided into four types: Path-based model, PDE (Partial differential equations) model, variational model and center-surround model, and each type of model is reviewed. Finally, the paper summarizes the advantages and disadvantages of the four Retinex models, and introduces the typical application of Retinex in image enhancement, as well as the future development direction.

Abstract:When the total sky imager (TSI) is used to observe the sky, the structural characteristics of the device will make the collected cloud image information incomplete, which affects the analysis of images. In order to deal with the problems, including the wrong order due to the sharp decrease to zero of the confidence level, the discontinuity of image and the large complexity of time for traversal searching the matching block in the process of repairing ground-based cloud image by the Criminisi algorithm, we propose a ground-based cloud image inpainting method based on the improved Criminisi algorithm in this paper. The calculation formula of priority is improved, and the unique red-blue ratio feature of the ground-based cloud map is introduced as a confidence term, so that the pixel block with more information has higher priority. In the process of searching for the matching block, the searching area is selected based on heuristic information in order to avoid the blocks far away from the block to be repaired and those with low correlation, which effectively shortens the searching time and reduces the time complexity of the algorithm. Experimental results show that the improved Criminisi algorithm has better image restoration effect, can reduce the time complexity and improve the image inpainting efficiency.

Abstract:In order to improve the accuracy and running speed of segmentation of building remote sensing images, a threshold segmentation method based on the 2-D Tsallis cross entropy image threshold selection using chaotic cuckoo search optimization is proposed. Firstly, the formula of 2-D Tsallis cross entropy threshold selection based on the histogram is derived. Next, in order to improve the convergence rate, logistic chaotic map is applied to the cuckoo search algorithm. Finally, the proposed chaotic cuckoo search algorithm is utilized for precise optimization of thresholds based on the 2-D Tsallis cross entropy, so as to realize the threshold segmentation of building remote sensing images with optimal threshold. A large number of experiments show that, compared with 2-D reciprocal cross entropy thresholding method, 2-D Tsallis entropy thresholding method, 2-D Tsallis gray entropy thresholding method based on chaotic particle swarm optimization and so on, the objects in the images segmented by the proposed method are more accurate, the details are more explicit, in addition, its running time is shorter.

Abstract:Portable imaging devices are ubiquitous in everyday life. However, as the hand jitter or the fast moving objects in the scene during shooting process, the captured image or video is often blurred, causing important details loss. In order to restore the blurred video and image to a clear state, we combine the recent research hotspots—Generative adversarial network, and propose a novel end-to-end bidirectional time-domain feature flow blind motion deblurring algorithm. The algorithm makes full use of the feature information of spatio-temporal continuity constraint to establish a bidirectional transmission channel of time-domain features between the adjacent frames. The multi-stage autoencoder deblurring network structure and the parallel coding and hybrid decoding fusion solution can fuse the multi-channel content information of a frame triplet and restore a clearer frame for a video. Experimental results show that the proposed algorithm is superior to the existing advanced algorithms on the traditional image quality evaluation indexes, i.e., peak signal to noise ratio (PSNR) and structural similarity (SSIM), and visual quality within acceptable time cost.

Abstract:When dealing with low illumination images, the traditional Retinex algorithm can improve the image recognition, but there are some shortcomings, such as "halo artifacts" and the lack of image details. In this paper, a new image enhancement algorithm, which combines the guided filtering image hierarchical processing with multi-scale Retinex algorithm, is adopted. Firstly, in the HSI color space, the original image is divided into detail image and basic image by using the guide filter algorithm. Then, gain coefficients are constructed for the two separated image layers, which are respectively enhanced and reconstructed to obtain a new brightness image. Finally, the new brightness image is restored in the RGB color space to output the final image with higher brightness and better restoration. Experimental results show that the algorithm makes the edges and details of the image more prominent, and can eliminate the "halo artifact" phenomenon. Moreover, the objective evaluation index has also been greatly improved.

Abstract:The training of deep convolutional neural networks becomes more and more difficult and its performance is degraded with the increase of the number of convolution layers to solve the problem. A facial expression recognition method is presented based on deep residual network. The method uses building blocks for residual learning to improve the training and optimization process of the deep convolutional neural network model and reduce the time cost of the model convergence. In addition, to improve the generalization ability of the network model, a hybrid dataset for training network model is made up of the expression image samples which are selected from the KDEF and CK+ expression datasets. The comparative experiment was conducted with 10-fold cross validation method on the hybrid dataset. In term of expression recognition accuracy, we compared the residual networks with residual learning and the conventional convolution neural networks without residual learning and demonstrated the effect of network depth on the recognition accuracy. The average recognition accuracy of 90.79% is achieved as a 74-layer deep residual network is adopted. The experimental results show that the deep convolutional neural network constructed with building blocks for residual learning can solve the contradiction between the network depth and the model convergence, and can improve the accuracy of expression recognition.

Abstract:Nowadays, there are many facial expression datasets for expression research. But, these datasets are small amounts of images and little expression information, which limits the expression research. This paper introduces the construction of the facial expression dataset in the wild (FELW) and the test case. The FELW dataset includes many facial expression images, and the images of different age, race and gender are collected from the Internet. Each image includes two labels—the state of facial part label and the expression label, that labeled by the appropriate method. And the author imported Kappa consistency check to improve the recognition rate of the FELW dataset. The author used traditional method and deep learning method to experiment and analysis on this dataset. Compared with other public facial expression datasets, the FELW dataset has much more amounts of images and more varieties of expression, and contains two labels to help to the expression research.

Abstract:In view of the fact that the existing 3D databases have fewer behavioral categories, few interactions with scenes, and single and fixed perspectives, this paper provides a large-scale human body complex behavior database DMV action3D based on RGB-D cameras, from two fixed perspectives and a mobile robot records human behavior from a dynamic perspective. There are 31 different behavioral classes in the database, including daily behaviors, interaction behaviors, and abnormal behaviors angles. Validated database collected more than 620 behavioral videos, about 600 000 frames of color images and depth images, to provide robots with optimal viewing. In order to verify the reliability and practicability of the data set, this paper adopts four methods for human behavior recognition, which are HOG3D features extracted by CRFasRNN method based on the information features of customs nodes, CNN and conditional random field (CRF), and then adopts SVM method for human behavior recognition. Spatial and temporal characteristics are extracted based on the three-dimensional convolutional network (C3D) and the 3D dense connection residual network, and the motion tags are predicted by softmax layer. The results show that DMV action3D human behavior database is characterized by a variety of scenes and complicated movements, and the difficulty of recognition is greatly increased. The DMV action 3D database has great advantages for studying human behavior in real environments, and provides a better resource for serving robots to recognize human behavior in real environments.

Abstract:Vehicle type recognition has become critical in intelligent transportation systems. The existing technology about vehicle type recognition is difficult to balance the recognition accuracy and recognition speed. Aiming at the problem of vehicle type recognition in the highway environment, a layered broad model combining the shallow feature layer with the broad feature layer is proposed, which can recognize the vehicle in real time. Firstly, the combination of color space conversion and multi-channel HOG algorithm is used to reduce the influence of illumination environment and realize the feature extraction of vehicle image. Combined with PCA dimension reduction technology, the computational complexity is reduced. Then sparse representation and nonlinear mapping of image features reduce correlation between image features. Finally, the ridge regression learning algorithm is used to train the extracted sample features, and the weight coefficient between the sample features and the sample tags is obtained to realize the recognition of the vehicle type. Experimental results on the BIT-Vehicle database show that the recognition accuracy of the proposed method is 96.69%, and the recognition speed is as high as 70.3 fps. The proposed algorithm can effectively enhance the feature expression ability and improve the vehicle type recognition accuracy, and ensure the real-time performance, which is superior to other algorithms in recognition accuracy and speed.

Abstract:This paper proposes a sign language recognition method based on color-depth videos and complex linear dynamic system (CLDS), which ensures that the time series modeling data can strictly correspond to the original data and accurately characterize the sign language features. Thus the classification precision is improved significantly. The depth videos are used to compensate the missing information of RGB videos, and the motion boundary histogram (MBH) features are extracted from the sign language videos to obtain the feature matrix of each behavior. The feature matrixes are modelled using CLDS method with output of the descriptor M=(A, C) which can uniquely represent the sign language video. Then the similarities between the models are calculated utilizing the subspace angles and the improved KNN algorithm is presented to achieve the final classification result. Experiments on the Chinese sign language dataset (CSL) show that the proposed sign language recognition approach has higher precision than many existing methods.

Abstract:As a new technology, extreme learning machine (ELM) has good generalization performance in regression and classification. Weighted fuzzy local information C-means (WFLICM) uses point coordinate distance and the local pixel coefficient of variation to mark the impact factor of each point to the middle point, improving the robustness of fuzzy C-means cluster algorithm. Based on ELM and improving WFLICM, new kernel weighted fuzzy local information C-means based on ELM (ELM-NKWFLICM) is proposed. The method is based on ELM feature mapping technique, mapping the original data to the high-dimensional ELM hidden space through the ELM feature mapping technique, and then is clustered by the new kernel weighted fuzzy local information c-means (NKWFLICM) of the improved new kernel local spatial information. Experimental results show that the proposed algorithm has better robustness than the WFLICM algorithm and retains the original image details well. The algorithm is more efficient in dealing with complex nonlinear data, and overcomes the sensitivity of fuzzy clustering algorithm to fuzzy exponents.

Abstract:In order to get photon counting fusion images with better effect, this paper proposes a fusion algorithm based on improved regional energy fusion rules. The photon counting images are obtained by the multi-pixel photon counter (MPPC) single detector in different illumination conditions, and the high-frequency and the low-frequency parts are obtained after wavelet transform of photon counting image. In the high-frequency part of the two source images, the energy of the corresponding region, neighborhood mean square, matching degree and threshold deviation are calculated. And the threshold is determined by pixel values and mean values of regional image. If the matching degree is greater than or equal to the threshold value, the weighted regional energy fusion method is used to solve it. Otherwise, the enhanced pixel value of source image with larger energy in local region is selected as pixel value in the corresponding fusion image. Using improved regional energy fusion rules for image fusion, it is proved that details of photon counting fusion image are clearer and target is easier to recognize，and the values of information entropy, average gradient and spatial frequency are about 20%, 25%, and 30% higher than those of regional energy fusion rules.

Abstract:Aiming at the problem that object tracking with single image feature under complex circumstances has low accuracy and poor robustness, a correlation filtering object tracking algorithm based on multi-feature fusion is proposed. Firstly, histogram of oriented gradient (HOG) features, color histogram features and convolutional features are respectively extracted from the target and background regions, and a fixed-coefficient fusion strategy is adopted to combine the feature response maps of HOG features and color histogram features. Then the fused response map and the convolutional features response map are fused by adaptive weighted fusion strategy，and the scale estimation algorithm is used to solve the problem of target scale changes. Finally, the sparse model update strategy is used to update the model. The proposed algorithm is evaluated on OTB-2013 dataset and compared with state-of-the-arts object tracking algorithms. Extensive experimental results show that our method significantly improves the performance in median distance precision and median overlap precision compared to the optimal algorithm. The accuracy and robustness of the proposed algorithm are superior to those of other algorithms in complex scenarios because of the effective use of HOG, color histogram and convolutional features.

Abstract:As an important digital image manipulation technique, image cloning can naturally and smoothly embed the cloned region in the source image into the specified region in the target image. In traditional image cloning technique, image information can be obtained by analyzing the gradient domains and solving Poisson equations. Its shortcomings of great algorithm complexity and large memory consumption limit the real application of traditional algorithm in high-resolution image. An improved mean-value coordinates algorithm is proposed in this paper, which transforms image cloning into efficient image interpolation procedure easy to implement. Moreover, the multi-scale and GPGPU based parallel computation technique is applied to further improve the overall operation efficiency of the proposed algorithm. Experimental results show that the proposed algorithm can realize the real-time cloning for the 1 mega pixel image region.

Abstract:Infrared and visible image fusion is designed to generate a fused image that provides a more complete description of the scene. In this paper, a novel multi-scale hybrid image decomposition algorithm is proposed, which can effectively extract the small-scale texture detail information of the visible image and the large-scale edge information of the infrared image. The large-scale edge image of the infrared image is used to be segmented and construct the fused weights, which can not only inject the multi-scale infrared spectrum into the visible image effectively, but also preserve the details of the scenes in the visible image. Experimental results show that the proposed algorithm can obtain state-of-the-art performance both in subjective and objective evaluation.

Abstract:Visual attention mechanism has been commonly used in state-of-the-art fine-grained classification methods in recent years. However, most attention-based image classification systems only apply single-layer or part-specified attention feature, with simple multiplication-based attention applying method, which limits the information provided by the attention. This paper presents a multi-channel visual attention based fine-grained image classification system. Multi-channel attention features are extracted from the image and applied to low-level features, with subtraction of mean values corresponding to each layer of attention for high-order representation, making the model an end-to-end optimizable deep neural network architecture. On multiple commonly used fine-grained classification datasets, the presented method outperforms state-of-the-art methods with a large margin.

Abstract:The feature extraction of leaf serration plays a crucial role in studying the plant gene relationship. To overcome the drawbacks of current algorithms, a novel leaf feature extraction algorithm based on bounded variation is proposed. Thereby, multi-parameters such as serration numbers, serration depth, and serration width can be obtained. These parameters provide an important basis for the subsequent gene analysis. Firstly, the boundary coordinates of a leaf are obtained based on image preprocessing. Secondly, we calculate the bounded variation between the adjacent pixels of the boundary. As such the initial values of the corner points can be obtained. Thirdly, the values of serration depth are computed for error compensation, and the coordinates of corner points (top and bottom) are obtained. Finally, all kinds of features, including serration numbers, serration depth, and serration width, are calculated. A big aspen dataset (12 000 pieces) collected from real world are used for algorithm validation. The results show that the estimation accuracy of serration numbers achieves 86.3% and all the parameters shown above can be batch extracted.

Abstract:The technique of portrait segmentation plays an important role in human face recognition, such as 3D human body reconstruction, motion capture and other practical applications, and its reliability directly affects the effect of subsequent processing. Based on the target segmentation algorithm of label propagation theory, an improved portrait segmentation algorithm is proposed. Firstly, this paper introduces the fuzzy set theory to improve the ability of complex background image segmentation. Secondly, this paper uses the super-pixel over-segmentation to preprocess, and uses the over-segmentation result to optimize the similarity definition for improving the smoothness and reliability of the segmented contour. Experimental results show that, compared with the original label propagation algorithm, the presented algorithm has higher segmentation accuracy, and the segmentation contour with higher smoothness is retained.

Abstract:In order to solve the problem that it is difficult to accurately control the resources of dumb devices in China Mobile Resource Database, taking the optical splitter as the entry point and the image recognition angle as the breakthrough, the time delay integration (TDI)-likely imaging analysis algorithm for processing outside plate(OSP)is proposed. Firstly, the uploaded images are filtered to ensure the quality of the image data. Then the HSV color space is used to extract the port position and outline, and the image contours are used to filter the noise and the distribution direction of the splitter port. Finally, this paper uses the TDI-likely imaging algorithm to analyze the port occupancy, outputs the port of the optical splitter and gives the split ratio and the occupied port number of the optical splitter, thus improving the data quality of the splitter resource by the resource data platform. Results show that the algorithm runs fast and the port analysis is accurate, so the algorithm has a high application prospect.

Abstract:The spoke position of a wheel needs to be adjusted before the wheel-flaw detection by X-Ray machine. A method based on image processing is proposed for initial angle measurement of the spoke. Firstly, the Hough transformation is used to remove the interference information outside the wheel-circle, and then a sector sample of detection area is set on the wheel pattern image according to the number of the spokes. Finally, the tested sector detection area on the wheel is rotated, and the angle is calculated by comparing the gray scale. Experimental results show that the method can measure the initial angle of the spoke, which meets the industrial requirements.