Infrared and visible image fusion has been a hot issue in the field of image processing. By fusing the infrared image and visible image of a same scene, the thermal radiation information of the infrared image and the texture information of the visible image can be retained in one image. However, the image fusion of infrared and visible images will lead to unclear image texture and loss of deep details. The conventional multi-scale transform methods will lead to the fuzzy edge detail of fused image. To solve these problems, an image fusion algorithm based on edge preserving filter combined with convolutional neural network is proposed. First, the source images are decomposed into two-scale images by rolling guidance filter. Second, data normalization is used for generating weights of base layers fusion. Third, the activity weight maps of detail layers are generated by VGGnet. Finally, different fusion strategies with different scales are used for image reconstruction. Compared with the existing representative methods, the proposed model performs well in both subjective and objective evaluation, especially in information entropy and edge detail preservation.