Based on the basic principle of Mean Shift algorithm, this paper proposes an improved target detection and tracking method based on mixture gauss model and Mean Shift algorithm, aiming at the complex background problems such as occlusion, shadow, illumination change, etc. The method uses the color feature in YCbCr color space as the target feature, uses the weighted operation of background and target to eliminate the interference of environmental noise, and highlights the effective information of the target itself. In the process of tracking, the target template is constantly updated to keep as consistent as possible with the target state, so as to achieve accurate, real-time and stable tracking of the target in the video stream. The experimental results show that the improved algorithm can effectively reduce the number of iterations and has a good tracking effect.