Theia: Vision-based camera localization and drift-free BIM alignment on indoor construction sites
- Project title: Vision-based camera localization and drift-free BIM alignment on indoor construction sites
- Project acronym: Theia
- Funding: SnT Partnership
- Entity: Gamma AR
- Project starting date: Jan. 2023
- Project duration: 36
- Project role: PI
Abstract:
GAMMA AR has developed a solution for efficient and detailed construction site monitoring and documentation. The solution provides the user equipped with a hand-held tablet with a virtual reality interface that seamlessly connects the real construction site with BIM. One of the existing challenges is the alignment between the real construction site and BIM, employing the sensors available on the hand-held tablet, i.e. usually only an RGB camera and an IMU.GAMMA AR has developed a solution that requires the user to provide an initial alignment through their user interface and then it tracks the motion of the tablet to have a continuous alignment on BIM. Nevertheless, despite having proved to be a practical solution that provides a smooth user experience, the alignment deteriorates over time.
The Automation and Robotics Research Group (ARG) of the SnT-UL has developed a solution for the alignment of the construction site and BIM for a quadruped robot equipped with a LIDAR sensor. This solution, called LIDAR-based iS-Graph (which stands for Informed Situational Graph) consists of (1) our LIDAR-based S-Graph, which simultaneously builds in real-time a map of the construction site and estimates the pose of the robot, and (2) a matching and merging algorithm of the S-Graph with the A-Graph (Architectural Graph) generated from the BIM. This solution does not require any fiducial marker or user input, although if existing, the alignment convergence is speeded up. This solution does not have any drift over time that may cause misalignments. Additionally, the ARG-SnT-UL has developed a basic vision-based S-Graph (i.e. RGB and inertial RGB), but it has never been connected to BIM.
The main goal of Theia is the development of a solution for real-time vision-based Simultaneous Localization and Mapping (SLAM) of a hand-held tablet on real construction sites (i.e. as-built), with drift-free alignment with existing Building Information Management (BIM) data (i.e. as-planned) that upgrades the current solution of GAMMA AR. The solution will be based on our novel Situational Graphs (S-Graphs), and will extend and combine our Informed S-Graphs and our Vision-based S-Graphs.
This overall goal is broken down into three scientific objectives:
- Vision-based global localization of the hand-held device within BIM (i.e. global alignment between the real construction site and the BIM) when the main structural elements (e.g. walls) are built and there are no major deviations from the BIM.
- Vision-based individual alignment between the main elements of the construction site and BIM, when there are no major deviations.
- Vision-based global and individual alignment between the construction site and its main elements and BIM in the presence of deviations.
Related Research: