Details

Autor(en) / Beteiligte

• Yang, Zhijia
• Gao, Kun
• Mao, Yuxuan
• Zhang, Yanzheng
• Zhang, Xiaodian
• Hu, Zibo
• Wang, Junwei
• Wang, Hong
• Li, Shuzhong

Titel

SADFusion: A multi-scale infrared and visible image fusion method based on salient-aware and domain-specific

Ist Teil von

• Infrared physics & technology, 2023-12, Vol.135, p.104925, Article 104925

Ort / Verlag

Elsevier B.V

Erscheinungsjahr

2023

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• Image fusion intends to generate an informative image with maximally possible features and details of the source images. However, the existing deep-learning fusion methods are rarely considering the discrepancy of visible and infrared (IVIF) modalities, besides losing control of balance in preserving the texture details and thermal target, owing to the identical structure and parameters of multi-modality feature extractors’ designing and weak domain guidance of network training. In this paper, a novel multi-scale fusion network based on the salient-aware and domain-specific methods is proposed, termed as SADFusion. To boost the multi-modality feature extracting performance of our method, the proposed network adopts dual encoding with short connection and multi-scale structure. Moreover, the domain-specific framework and corresponding training strategy are designed to achieve the identical encoding for different image modalities. Moreover, multi-scale attention fusion modules (MSFAF modules) are proposed to effectively fuse the extracted complementary features at every scale. Finally, we construct the specific salient-aware loss to guide the model to trade off the preserving necessary information, by utilizing salient modality features as pixel-to-pixel intensity and gradient maps. Experiments based on the public datasets demonstrate the superiority of our method over the state-of-art fusion methods, which particularly highlights the targets and retains the effective information. •A multi-scale IR/VI fusion method, based on salient-aware and domain-specific, is proposed.•The Domain-specific encoding framework is employed to ameliorate domain discrepancy.•The salient-aware loss is devised to preserve the significant features in the source images.•Our proposed method performs better than existing methods.