This paper describes a real-time pedestrian detection processor of Field Programmable Gate Arrays (FPGA) using a novel structure aided by E-HOG IP. This structure proposes a three stages detection to reduce the amount of calculation and improve the processing speed. Compared to the traditional methods, in the first stage, the Sobel-step can select windows of interest before feature extraction and classification. In the second stage, the uniform-LBP (Local Binary Pattern), which is implemented cell by cell, has approximately half-dimension of HOG and simultaneous SVM (Support Vector Machine) to decrease total computation. In the third stage, improved HOG (Histogram of Oriented Gradient), called E-HOG (Efficient-HOG), is suitable to be realized in hardware. E-HOG and SVM are adopted to improve the detection rate. To evaluate the performance, the proposed structure is implemented into an FPGA while the E-HOG is manufactured as one common IP. The results indicate that this processor can detect pedestrians with 48 fps for VGA resolution under 25Mhz in low FPGA series and satisfy the demand of real-time.