Multimodal emotion classification method based on multilevel deep convolution neural network in social networks

Abstract

In the tasks of image emotion classification, the prior information that affects image emotion expression is explored, and a new multi-level deep convolutional neural network framework is proposed.The framework comprehensively considers global and local perspectives, and introduces the prior information that affects the image emotion, and learns the emotional expression of the image from five levels successively, including original image, salient subject, color, local information of original imageand local information of color. The experimental results indicate that the salient subject is the most important among the prior information that affects image emotion. Besides, in the open emotional image gallery of big order and small order of magnitude, the classification accuracy of this framework is higher than the existing traditional manual characteristic methods and deep learning method, its average classification accuracy rate is 2.8% higher than the optimal method, and in particular, the emotional category “disgust” increases by 15%, which effectively breaks through the bottleneck of the current image emotion classification.