In the face recognition community, the research of sparse representation has seen a recent surge of interest. Even though the images are with varying expression and illumination, as well as occlusion, most of the algorithms still have a good recognition effect. However, when the test and training images contain both the changes of illumination and expression, the traditional sparse representation algorithm often performs the wrong face recognition. In sparse representation, the 1  -norm was used to define the fidelity of sparse coding. In fact, the fidelity terms (sparse coefficients) can represent the testing samples as a sparse linear combination of the training dictionary, and hence they have a very important influence on the final classification. In this paper, we propose a simple and effective face recognition algorithm, in which the sparse coefficients and original residuals are fused effectively. The useful information of sparse coefficients can be fully reflected in the residuals. Hence the new residual values, that are obtained, can improve the fidelity of residuals. We exploit the fusion nature of sparse coefficients to redefine the computing method of residuals, and then perform classification. We conduct several experiments on publicly available database to verify the efficacy of the proposed approach and corroborate our claims.