Image recognition is an important field in artificial intelligence, it makes use of the computer to conduct image processing, analysis and understanding to recognize a variety of different objects. And it uses a series of enhancement and reconstruction methods to effectively improve the image quality. Traditional deep Convolutional Neural Network (DCNN) not only improves the recognition accuracy, but also reduces the recognition speed. How to improve the speed while maintaining the accuracy has become an important direction in image recognition. In this paper, we propose a modified Residual network based on multi-scale segmentation for aerobics motion image recognition. The new residual network has the characteristics of shorter network length and faster recognition speed. First, it reduces the length of the network and gets a new residual network with seven layers. Then, combining with the multi-scale segmentation method, an image recognition residual network is obtained. Finally, experiments on the CIFAR10 dataset, the results show that the proposed new motion image recognition method has better recognition accuracy and faster recognition speed.