RFID label image is often polluted by noise, which reduces the efficiency of feature recognition of RFID label geometric distribution and affects the subsequent optimization. Aiming at it, this paper highlights a denoising algorithm for RFID label image based on singular spectrum analysis of phase space reconstruction. First, RFID label image is transformed into some sequence signals. Second, the improved Cao algorithm and SSA algorithm is proposed to denoise image, moreover, the processed sequence signal is reconverted to get the denoising image. In the paper, the rules of choosing the percentage to eliminate eigenvectors in SSA and how to superimpose the grayscale matrix are stipulated to improve the denoising efficiency. Finally, this paper compares the proposed algorithm with three denoising algorithms on different RFID label images to verify the effectiveness of this algorithm. The PSNR of the proposed algorithm are at least 0.3 dB higher than other denoising algorithms and the SSIM of the proposed algorithm are basically similar as NLM. Specially in the RFID semi-physical simulation experimental platform, the average of the SSIM of the square labels is 0.002 higher than NLM. The experiments show that the proposed algorithm in this paper is superior to the current common algorithms.