Traffic light recognition (TLR) is an important component of vehicular vision system for an autonomous vehicle, which has been extensively studied. However, existing algorithms cannot guarantee real time and effectiveness at the same time. Thus we propose an automatic traffic light recognition method based on prior knowledge and optimized threshold segmentation. The method uses the differential GPS (DGPS) and designed labeling software to obtain the prior knowledge of traffic lights. Then an optimized threshold segmentation combining some custom constraints for traffic light detection is introduced. Finally, a tracking algorithm with the prior knowledge for logic validation is combined, to get more accurate light detection results. Experimental results show that our proposed method can achieve the detection rate of 99.4%, with average 31ms processing time per image.