Cancer has become one of the greatest threats for human life. Doctors can get original images of the sick organs with the assistance of medical technology. However, manual interpretation of the original images is time consuming and labor consuming. Nowadays, the intelligent detection of tumor cell images is commonly adopted in cancer diagnosis. In this paper, we propose Imporved Selective Search (ISS) algorithm and CTCNet based on Circulating Tumor Cell (CTC) technology to improve the cancer images’ detection efficiency. CTCs can be collected by a sampling needle with EpCAM antibody which can specifically bind to tumor cells. After fluorescent staining process, images obtained from sampling needle will be processed by the ISS algorithm for candidate region preselection. All of the eligible areas are evaluated with a self-designed neural network called CTCNet, resulting in efficient recognition of circulating tumor cells. During this process, all algorithms are accelerated through the GPU and NPU hardware platform, which further improves the detection speed of the system. In the experiment, we first verify the efficiency of proposed ISS algorithm by compared with Original Selective Search (OSS), we found that the number of candidate boxes reduced from 549 to 16 and the time consuming reduced by 0.3s after adopting the ISS algorithm. In order to evaluate the performance of the proposed 7-layer CTCNet, we compared CTCNet with SVM, BP neural network, AlexNet and VGGNet by using the dataset of 12312 samples from 30 patients, among them, there were 12 patients with early cancer and 18 patients with advanced cancer. And we got the highest recognition accuracy of 97.95% on CTCNet, which even beyond the VGGNet with deeper layers. In contrast to other combinations, we detect CTC in diverse clinical CTC images, the joint application of ISS algorithm and CTCNet achieves an outstanding system performance with accuracy up to 94.03% in the whole view images. Meanwhile, because of the application of a lightweight network in different hardware acceleration platform, the detection time of the CTC image in a single view can be less than 12s.