According to the issues of single-scale image feature extraction, small receptive field, and cannot highlighting salient features in existing deep learning based image fusion algorithms, this paper proposes a multi-scale dilated convolution network with attention mechanism for multi-focus image fusion. First, a multi-scale dilated convolution block (MDB) is proposed. The MDB with different dilation rates can provide different receptive fields, and consequently it can extract the multi-scale features. Moreover, the attention mechanism is introduced into the MDB, which can adaptively select the salient features and improve the performance further. The proposed fusion network consists of three parts, including feature extraction, feature fusion, and image reconstruction. Specifically, the feature extraction part is composed of several MDBs. The experimental results demonstrate that the proposed method is competitive to some existing deep learning based methods. Some ablation studies also verify that the MDB can enhance the ability of feature extraction and improve the image fusion quality.