DeepX possesses the software technology and development team required for consistent development of autonomous construction machinery systems. The primary components of development can be categorized into four areas: Robotics, Control, Perception, and Simulation. Here, we will introduce representative technologies.

 

Robotics

ROS-based robotic system

In DeepX, we prioritize the robustness and extensibility of our systems and are developing a robotic system based on ROS2.

A user-friendly UI for non-engineers and on-site personnel.

In DeepX, for projects that are close to on-site verification and deployment, we anticipate that the autonomous driving system will be used by non-engineers and on-site personnel. Therefore, we design and develop the system’s UI to be intuitive. Furthermore, after development, we refine the UI based on various feedback obtained during its trial use.
In the case below, we developed a UI that allows for the autonomous driving operation to be specified and executed by simply “designating the excavation and dumping areas with a mouse and clicking the execute button”.

 

Control

Path Planning and Tracking

Construction sites are dynamic environments where conditions change continuously. At DeepX, to address that we developed excavator path planning and tracking technology that can adapt to a wide variety of machines and terrains.

Path Planning in Simulation

Path Tracking at the Site

Our all-purpose path planning technology covers everything from excavation to soil transport and dumping. It combines expert knowledge from skilled operators and realtime optimization to maximize the excavated soil volume for any terrain shape. For soil transport, our state-of-the-art motion planning algorithms navigate around obstacles while maintaining smooth movement.
Path tracking deals with the ever-changing dynamics of hydraulic systems, unpredictable terrain, sensor faults, and system instability. Our controller algorithms are robust against these uncertainties, thanks to advanced modeling and extensive simulator-based optimizations.
This technology has proven its versatility and robustness across diverse construction sites and machines through countless and extensive on-site deployments.

Collision prevention

At DeepX, ensuring the safety of the construction site is paramount. We have developed collision avoidance technology for both our automation system and remote-controlled construction machines.

Collision Avoidance for Automation System

Collision Avoidance for Remote Control

Our automation system features active collision avoidance, allowing it to safely operate near remote-controlled machines and stop when necessary.
For remote-controlled machines, we’ve created an intuitive collision avoidance systems. This technology filters out unsafe operator commands, maintaining a safe distance from obstacles while minimally altering the operator’s inputs.
This functionality is powered by high-performance real-time algorithms and advanced predictive models, which anticipate future distances to obstacles. Our collision model accounts for all obstacles, including walls, buckets, and other moving machines.

 

Perception

Object detection using point cloud data

At DeepX, when autonomously operating construction machinery, the system often uses point cloud data obtained from Lidar as input data to detect objects that the system should recognize, such as dump trucks and earth buckets, and achieve real-time detection.
In the example below, we developed a system that processes point cloud data from Lidar, allowing it to understand in real-time the relative position, posture, shape, and loading status of the dump truck as seen from a hydraulic excavator. Particularly, to accommodate various conditions such as different positions, postures, and soil shapes, we utilized deep neural networks in our algorithm development.

Map Generation Using Point Cloud Data

At DeepX, we often use point cloud data obtained from Lidar as input data to recognize and visualize the surrounding environment of construction machinery on-site. This allows us to achieve real-time recognition and visualization.
In the example below, we developed a system that processes point cloud data from multiple Lidars and sensor data related to the construction machinery in real-time, allowing us to understand the surrounding environment of the caisson excavators and the real-time position and posture of the construction machinery itself.

 

Simulation

Generation of point cloud data including Lidar’s scanning pattern.

At DeepX, we often utilize simulation data for the development, training, and validation of object detection models using point cloud data. The actual data from Lidar often reflects the effects of a unique scanning pattern. We’ve found that simply randomly sampling 3D data of target objects in simulations is insufficient.
In the following case, we replicated the specific scanning patterns of Lidar within the simulation. This allowed us to use the simulated Lidar data for the development and verification of models and systems intended for real-world applications.