The main task of image fusion is to extract the advantage information from the source images. Multiscale transform (MST) is commonly used in image fusion, but it is difficult to integrate detail information and structural information from source images. Therefore, it is important to integrate the information in each image patch group. Different from MST, edge preserving technology obtains more edge information than MST by extracting spatial information. Based on this superior feature, we propose an infrared and visible image fusion method via rolling guidance filter (RGF) and weight map. First, source images are decomposed into multiscale detail layers and structural base layers by RGF; second, to avoid the loss of local information, the modified Laplacian energy weight matrix is used as detail weight maps to merging the detail layers; then to reduce the loss of contours of edge objects, a structural weight maps are used to merging the base layers; finally, these two different weight maps are applied to reconstruct the fused image. Simulation results indicate that our fusion approach performs better in merging detail information and structural information than some classic MST fusion algorithms and the latest fusion methods.