Citation: Wei, H.; Tang, L.; Wang, W.;
Zhang, J. Home Environment
Augmented Reality System Based on
3D Reconstruction of a Single
Furniture Picture. Sensors 2022, 22,
4020. https://doi.org/10.3390/
s22114020
Academic Editors: Zhihan Lv, Kai Xu
and Zhigeng Pan
Received: 18 April 2022
Accepted: 20 May 2022
Published: 26 May 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Home Environment Augmented Reality System Based on 3D
Reconstruction of a Single Furniture Picture
Hongtao Wei, Lei Tang *, Wenshuo Wang and Jiaming Zhang *
College of Information Engineering, Wuhan University of Technology, Wuhan 430070, China;
weiht@whut.edu.cn (H.W.); wws1049722105054@whut.edu.cn (W.W.)
* Correspondence: 1336631096@whut.edu.cn (L.T.); zjiaming@whut.edu.cn (J.Z.)
Abstract:
With the popularization of the concept of “metaverse”, Augmented Reality (AR) technology
is slowly being applied to people’s daily life as its underlying technology support. In recent years,
rapid 3D reconstruction of interior furniture to meet AR shopping needs has become a new method.
In this paper, a virtual home environment system is designed and the related core technologies
in the system are studied. Background removal and instance segmentation are performed for
furniture images containing complex backgrounds, and a Bayesian Classifier and GrabCut (BCGC)
algorithm is proposed to improve on the traditional foreground background separation technique.
The reconstruction part takes the classical occupancy network reconstruction algorithm as the network
basis and proposes a precise occupancy network (PONet) algorithm, which can reconstruct the
structural details of furniture images, and the model accuracy is improved. Because the traditional
3D registration model is prone to the problems of model position shift and inaccurate matching
with the scene, the AKAZE-based tracking registration algorithm is improved, and a Multiple
Filtering-AKAZE (MF-AKAZE) based on AKAZE is proposed to remove the matching points. The
matching accuracy is increased by improving the RANSAC filtering mis-matching algorithm based
on further screening of the matching results. Finally, the system is verified to realize the function
of the AR visualization furniture model, which can better complete the reconstruction as well as
registration effect.
Keywords: 3D reconstruction; instance segmentation; 3D registration; furniture objects
1. Introduction
With the impact of the global new crown epidemic, shopping without leaving home is
encouraged by the state to reduce the spread of the epidemic brought about by the gathering
of people. Online shopping for small items usually does not require consideration of size,
and returns and exchanges are easier to carry, but online shopping for medium to large
furniture products can have certain drawbacks. Traditional furniture procurement is
mainly based on subjective feelings to infer the placement of furniture in the room and
placement effect; this approach will cause the purchase of furniture back in the actual
placement effect and the size of the house and furniture color with the incongruity. The
return of furniture products is quite time consuming and laborious, so the current furniture
procurement model is mostly offline procurement. Offline furniture city furniture format
is relatively fixed and cannot provide users in the shopping site favorite personalized
furniture products, according to the website furniture picture to get the same model for
real-time home experience product suitability of the new way to call out.
Augmented Reality (AR) technology, as an important technical support for the under-
lying technology of “metaverse”, displays virtual models or information in the real world
for interaction, which can usually assist users to display and work in reality [
1
]. With the
increase in the computing power of electronic products, AR technology is used in all aspects
of people’s lives. Viewing a single image of furniture on a shopping website and creating a
Sensors 2022, 22, 4020. https://doi.org/10.3390/s22114020 https://www.mdpi.com/journal/sensors
