DeepX possesses the software technology and development team required for consistent development of autonomous construction machinery systems. The primary components of development can be categorized into four areas: Robotics, Control, Perception, and Simulation. Here, we will introduce representative technologies.
ROS-based robotic system
In DeepX, we prioritize the robustness and extensibility of our systems and are developing a robotic system based on ROS2.
A user-friendly UI for non-engineers and on-site personnel.
In DeepX, for projects that are close to on-site verification and deployment, we anticipate that the autonomous driving system will be used by non-engineers and on-site personnel. Therefore, we design and develop the system’s UI to be intuitive. Furthermore, after development, we refine the UI based on various feedback obtained during its trial use.
In the case below, we developed a UI that allows for the autonomous driving operation to be specified and executed by simply “designating the excavation and dumping areas with a mouse and clicking the execute button”.
Path Planning and Tracking
Construction sites are dynamic environments where conditions change continuously. At DeepX, to address that we developed excavator path planning and tracking technology that can adapt to a wide variety of machines and terrains.
Path Planning in Simulation
Path Tracking at the Site
Our all-purpose path planning technology covers everything from excavation to soil transport and dumping. It combines expert knowledge from skilled operators and realtime optimization to maximize the excavated soil volume for any terrain shape. For soil transport, our state-of-the-art motion planning algorithms navigate around obstacles while maintaining smooth movement.
Path tracking deals with the ever-changing dynamics of hydraulic systems, unpredictable terrain, sensor faults, and system instability. Our controller algorithms are robust against these uncertainties, thanks to advanced modeling and extensive simulator-based optimizations.
This technology has proven its versatility and robustness across diverse construction sites and machines through countless and extensive on-site deployments.
At DeepX, ensuring the safety of the construction site is paramount. We have developed collision avoidance technology for both our automation system and remote-controlled construction machines.
Collision Avoidance for Automation System
Collision Avoidance for Remote Control
Our automation system features active collision avoidance, allowing it to safely operate near remote-controlled machines and stop when necessary.
For remote-controlled machines, we’ve created an intuitive collision avoidance systems. This technology filters out unsafe operator commands, maintaining a safe distance from obstacles while minimally altering the operator’s inputs.
This functionality is powered by high-performance real-time algorithms and advanced predictive models, which anticipate future distances to obstacles. Our collision model accounts for all obstacles, including walls, buckets, and other moving machines.
Object detection using point cloud data
At DeepX, when autonomously operating construction machinery, the system often uses point cloud data obtained from Lidar as input data to detect objects that the system should recognize, such as dump trucks and earth buckets, and achieve real-time detection.
In the example below, we developed a system that processes point cloud data from Lidar, allowing it to understand in real-time the relative position, posture, shape, and loading status of the dump truck as seen from a hydraulic excavator. Particularly, to accommodate various conditions such as different positions, postures, and soil shapes, we utilized deep neural networks in our algorithm development.
Map Generation Using Point Cloud Data
At DeepX, we often use point cloud data obtained from Lidar as input data to recognize and visualize the surrounding environment of construction machinery on-site. This allows us to achieve real-time recognition and visualization.
In the example below, we developed a system that processes point cloud data from multiple Lidars and sensor data related to the construction machinery in real-time, allowing us to understand the surrounding environment of the caisson excavators and the real-time position and posture of the construction machinery itself.
Generation of point cloud data including Lidar’s scanning pattern.
At DeepX, we often utilize simulation data for the development, training, and validation of object detection models using point cloud data. The actual data from Lidar often reflects the effects of a unique scanning pattern. We’ve found that simply randomly sampling 3D data of target objects in simulations is insufficient.
In the following case, we replicated the specific scanning patterns of Lidar within the simulation. This allowed us to use the simulated Lidar data for the development and verification of models and systems intended for real-world applications.
On-site Demonstration of Continuous Autonomous Operation of Excavators
DeepX has developed an autonomous operation system for hydraulic excavators. In April 2023, in collaboration with Fujita Corporation, DeepX successfully demonstrated the autonomous operation of a continuous cycle of excavation and dump loading as a single operation at a construction site.
Developing Sway-Free Crane Automation Technology
In 2022, DeepX, in collaboration with Tadano Ltd., developed and announced a crane automation technology that carries out a three-axis lifting operation, which uses rotation, boom movement, and winching simultaneously, while keeping the load sway to a minimum.
On-site Demonstration of a Real-Time 3D Visualization System with Digital Twins
DeepX has developed a system called 'GeoViz', which visualizes the site in real-time three dimensions in a digital twin format. In 2022, together with Oriental Shiraishi Corporation, DeepX demonstrated this system at a site using the pneumatic caisson method.
Automated Operation of Hydraulic Shovels – A Solution to the Shortage of Operators
To automate various civil engineering works at construction sites, we collaborate with Fujita Co., Ltd. a major general constructor. We aim to automate the operations of a hydraulic excavator, which is a general-purpose heavy machine. The automation of the excavation work with a hydraulic excavator has been realised by utilising the image recognition technology with the state-of-the-art Deep Learning and Reinforcement Learning technology.