Action recognition is prevalent in the field of intelligent image processing, and feature extraction is the key of action recognition. The number of features extracted by the improved dense trajectory algorithm (iDT) is huge and occupies a large amount of hardware storage space. Aiming to address this problem, this paper proposes to reduce the number of features by using trajectory deletion, feature clustering and salient feature extraction, while improving the accuracy of action recognition. Trajectory deletion is the deletion of trajectories with little or no information. Feature clustering is to cluster features of each action, and the cluster centers are used to represent the action. Salient feature extraction is to extract salient features within the same action category, and the salient features are used to train the codebook. In order to verify the effect of the algorithm, experiment is carried out in KTH and UCFSports datasets. After improvement, the features in KTH are reduced by 80.80% and the accuracy is increased by 0.93%. In UCFSports, the features are reduced by 79.68% and the accuracy is increased by 4.26%.