Aiming at the problem of multi-scale pedestrian detection, a pedestrian detection algorithm based on fusion feature pyramid networks (FPN) and faster R-CNN (Faster region convolutional neural network) is proposed. Firstly, FPN and region proposal networks (RPN) are fused. Secondly, FPN and Fast R-CNN are fused. Finally, the pedestrian detection algorithm with fusion FPN and Faster R-CNN is trained and tested on Caltech dataset, KITTI dataset, and ETC dataset, respectively. The mAP (mean Average Precision) of this algorithm reaches 69.72%, 69.76% and 89.74% on Caltech dataset, KITTI dataset, and ETC dataset, respectively. Compared with Faster R-CNN, this algorithm not only improves the pedestrian detection accuracy, but also obtains satisfactory detection effect on the problem of multi-scale pedestrian detection.