Feature selection is an important part of data preprocessing. Feature selection algorithms that use mutual information as evaluation can effectively handle different types of data, so it has been widely used. However, the potential relationship between relevance and redundancy in the evaluation criteria is often ignored, so that effective feature subsets cannot be selected. Optimize the evaluation criteria of the mutual information feature selection algorithm and propose a mutual information feature selection algorithm based on dynamic penalty factors (Dynamic Penalty Factor Mutual Information Feature Selection Algorithm, DPMFS). The penalty factor is dynamically calculated with different selected features, so as to achieve a relative balance between relevance and redundancy, and effectively play the synergy between relevance and redundancy, and select a suitable feature subset. Experimental results verify that the DPMFS algorithm can effectively improve the classification accuracy of the feature selection algorithm. Compared with the traditional chi-square, MIM and MIFS feature selection algorithms, the average classification accuracy of the random forest classifier for the six standard datasets is increased by 3.73%, 3.51% and 2.44%, respectively.