Traditional motion recognition methods can extract global features, but ignore the local features. And the obscured motion cannot be recognized. Therefore, this paper proposes a modified Histogram of oriented gradients (HOG) combining speeded up robust features (SURF) for sports motion feature extraction and recognition. This new method can fully extract the local and global features of the sports motion recognition. The new algorithm first adopts background subtraction to obtain the motion region. Direction controllable filter can effectively describe the motion edge features. The HOG feature is improved by introducing direction controllable filter to enhance the local edge information. At the same time, the K-means clustering is performed on SURF to obtain the word bag model. Finally, the fused motion features are input to support vector machine (SVM) to classify and recognize the motion features. We make comparison with the state-of-the-art methods on KTH, UCF Sports and SBU Kinect Interaction data sets. The results show that the recognition accuracy of the proposed algorithm is greatly improved.