Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration

Hengjian Yu; Xiangrong Zhou; Huiyan Jiang; Hongjian Kang; Zhiguo Wang; Takeshi Hara; Hiroshi Fujita

doi:10.1117/12.2512698

15 March 2019 Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration

Hengjian Yu, Xiangrong Zhou, Huiyan Jiang, Hongjian Kang, Zhiguo Wang, Takeshi Hara, Hiroshi Fujita

Proceedings Volume 10953, Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging; 109531X (2019) https://doi.org/10.1117/12.2512698
Event: SPIE Medical Imaging, 2019, San Diego, California, United States

Abstract

This paper proposes a novel method to learn a 3D non-rigid deformation for automatic image registration between Positron Emission Tomography (PET) and Computed Tomography (CT) scans obtained from the same patient. There are two modules in the proposed scheme including (1) low-resolution displacement vector field (LR-DVF) estimator, which uses a 3D deep convolutional network (ConvNet) to directly estimate the voxel-wise displacement (a 3D vector field) between PET/CT images, and (2) 3D spatial transformer and re-sampler, which warps the PET images to match the anatomical structures in the CT images using the estimated 3D vector field. The parameters of the ConvNet are learned from a number of PET/CT image pairs via an unsupervised learning method. The Normalized Cross Correlation (NCC) between PET/CT images is used as the similarity metric to guide an end-to-end learning process with a constraint (regular term) to preserve the smoothness of the 3D deformations. A dataset with 170 PET/CT scans is used in experiments based on 10-fold cross-validation, where a total of 22,338 3D patches are sampled from the dataset. In each fold, 3D patches from 153 patients (90%) are used for training the parameters, while the remaining whole-body voxels from 17 patients (10%) are used for testing the performance of the image registration. The experimental results demonstrate that the image registration accuracy (the mean value of NCCs) is increased from 0.402 (the initial situation) to 0.567 on PET/CT scans using the proposed scheme. We also compare the performance of our scheme with previous work (DIRNet) and the advantage of our scheme is confirmed via the promising results.

Citation Download Citation

Hengjian Yu, Xiangrong Zhou, Huiyan Jiang, Hongjian Kang, Zhiguo Wang, Takeshi Hara, and Hiroshi Fujita "Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration", Proc. SPIE 10953, Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging, 109531X (15 March 2019); https://doi.org/10.1117/12.2512698

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available