Recently, moving object detection (MOD) in UAV (Unmanned Aerial. Vehicle) videos has been widely used in many fields. However, different objects and different algorithms often result in different detection accuracy. SSD (Single Shot MultiBox Detector) series and YOLO (You Only Look Once) version 5 are two popular object detection model, and their performance are always evaluated and compared with other improved method for optimizing detection accuracy. In this paper, an improved YOLO_v5 detection algorithm was proposed to further improve the detection accuracy. It adopted a cascaded inter-frame verification mechanism which is based on the neural network and uses spatial information and integrates object speed and direction as well to improve the detection accuracy of moving objects. To evaluate its performance, the open UAV video data from Stanford University was used to test the algorithms, and three types of moving objects were analyzed. The experimental results demonstrate that the proposed MOD method can improve the detection accuracy of small moving objects, which have a good application value, and can lay a foundation for subsequent related studies.