An obstacle perception system for intelligent vehicle is proposed. The proposed system combines the stereo version technique and the deep learning network model, and is applied to obstacle perception tasks in complex environment. In this pa-per, we provide a complete system design project, which in-cludes the hardware parameters, software framework, algorithm principle, and optimization method. In addition, special experi-ments are designed to demonstrate that the performance of the proposed system meets the requirements of actual application. The experiment results show that the proposed system is valid to both standard obstacles and non-standard obstacles, and suitable for different weather and lighting conditions in complex environment. It announces that the proposed system is flexible and robust to the intelligent vehicle.