For stereo vision systems, the accuracy of parameters calibration has a great influence on measurement and positioning precision in the later process. However, most existing calibration methods own shortcomings of low precision and high computation complexity, which can hardly be applied to occasions with strict measurement requirements. In this paper, we present an accurate camera calibration method for binocular systems based on geometric features of vanishing points formed by two sets of parallel lines. Based on the central symmetry, the position of distortion center is firstly calculated out. Then we treat it as the initial value and utilize genetic algorithm to optimize distortion parameters by minimizing straight degrees of corrected lines, which greatly reduces the algorithm computational complexity. After calibrating internal parameters with the proposed method, external parameters are figured out on the basis of geometric properties of orthogonal vanishing points. Finally, geometry constraint between epipolar lines is adopted for optimizing structure parameters of the binocular system by minimizing matching errors under different views. It significantly improves the algorithm robustness and re-duces noise interference. Experimental results show that the proposed method with impressive precision and stability can exactly finish the parameters calibration for stereo systems.