Details

Autor(en) / Beteiligte

• Huang, Wenfeng
• Liao, Xiangyun
• Chen, Hao
• Hu, Ying
• Jia, Wenjing
• Wang, Qiong

Titel

Deep local-to-global feature learning for medical image super-resolution

Ist Teil von

• Computerized medical imaging and graphics, 2024-07, Vol.115, p.102374, Article 102374

Ort / Verlag

United States: Elsevier Ltd

Erscheinungsjahr

2024

Quelle

MEDLINE

Beschreibungen/Notizen

• Medical images play a vital role in medical analysis by providing crucial information about patients’ pathological conditions. However, the quality of these images can be compromised by many factors, such as limited resolution of the instruments, artifacts caused by movements, and the complexity of the scanned areas. As a result, low-resolution (LR) images cannot provide sufficient information for diagnosis. To address this issue, researchers have attempted to apply image super-resolution (SR) techniques to restore the high-resolution (HR) images from their LR counterparts. However, these techniques are designed for generic images, and thus suffer from many challenges unique to medical images. An obvious one is the diversity of the scanned objects; for example, the organs, tissues, and vessels typically appear in different sizes and shapes, and are thus hard to restore with standard convolution neural networks (CNNs). In this paper, we develop a dynamic-local learning framework to capture the details of these diverse areas, consisting of deformable convolutions with adjustable kernel shapes. Moreover, the global information between the tissues and organs is vital for medical diagnosis. To preserve global information, we propose pixel–pixel and patch–patch global learning using a non-local mechanism and a vision transformer (ViT), respectively. The result is a novel CNN-ViT neural network with Local-to-Global feature learning for medical image SR, referred to as LGSR, which can accurately restore both local details and global information. We evaluate our method on six public datasets and one large-scale private dataset, which include five different types of medical images (i.e., Ultrasound, OCT, Endoscope, CT, and MRI images). Experiments show that the proposed method achieves superior PSNR/SSIM and visual performance than the state of the arts with competitive computational costs, measured in network parameters, runtime, and FLOPs. What is more, the experiment conducted on OCT image segmentation for the downstream task demonstrates a significantly positive performance effect of LGSR. •We proposed a novel CNN-ViT model, LGSR, for multi-modality medical image SR.•We defined a local-to-global learning framework to restore information thoroughly.•LGSR costs fewer parameters than CNN SOTAs, and is faster than CNN-ViT SOTAs.