Multi-scale transformer 3D plane recovery

Fei Ren; Qingling Chang; Xinglin Liu; Yan Cui

doi:10.1117/12.2680197

27 June 2023 Multi-scale transformer 3D plane recovery

Fei Ren, Qingling Chang, Xinglin Liu, Yan Cui

Proceedings Volume 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022); 1270529 (2023) https://doi.org/10.1117/12.2680197
Event: Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 2022, Nanjing, China

Abstract

The 3D plane recovery and reconstruction method of a single image aim to identify plane instance regions and estimate plane parameters and is widely used in technologies such as AR/VR(Augmented Reality/Virtual Reality). In recent years, significant progress has been achieved in single-image 3D plane recovery approaches based on deep learning, however much of the research has focused on overall plane segmentation performance rather than the accuracy of small-scale plane segmentation. Given the lack of an accurate definition of the segmentation of small-scale plane regions in existing methods, we propose a novel multi-scale transformer-based plane recognition and recovery model, which can accurately identify the edges of small-scale plane regions. The network branches composed of the model are used to detect plane and non-planar areas respectively. Different input feature scales make the two network branches have different global feature extraction capabilities. Finally, the two branches are strengthened to recognize plane areas in the same scene through mutual information loss. The consistency makes the two branch networks have the ability to share parameters. The experimental results in the Scannet and NYU V2 datasets show that the model can accurately identify small-sized plane areas, and the detection and recovery accuracy reaches the state-of-the-art effect.

Citation Download Citation

Fei Ren, Qingling Chang, Xinglin Liu, and Yan Cui "Multi-scale transformer 3D plane recovery", Proc. SPIE 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 1270529 (27 June 2023); https://doi.org/10.1117/12.2680197

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Transformers

Image segmentation

Feature extraction

Machine learning

Surface plasmons

3D image reconstruction

Image restoration

Show All Keywords

Keywords/Phrases

Search In:

Publication Years