In July 2023, DeepX, in collaboration with Oriental Shiraishi Corporation, developed an automatic driving system for construction machinery, specifically for ‘caisson excavators’ used in pneumatic caisson construction. Through demonstrative experiments, it was confirmed that multiple caisson excavators can be automated simultaneously and that this system meets the performance requirements necessary for on-site implementation.
（This article introduces the expanded initiatives based on what was demonstrated in ‘On-site Demonstration of a Real-Time 3D Visualization System with Digital Twins‘.）
 

About Pneumatic caisson method

The term “Pneumatic caisson method” refers to a technique where ‘pneumatic’ means air and ‘caisson’ means box.
It is widely used for the foundations of bridges and buildings, as well as for sewage pumping stations, underground adjustment tanks, shield tunnel shafts, and the main structures of subways and road tunnels.
A reinforced concrete box (body) is constructed above ground, a workspace is set up underneath it, and excavation machinery and earth buckets for soil removal are installed.
By repeating the process of excavation, soil removal, and sinking of the body, it is advanced vertically.
A characteristic of this method is that compressed air, commensurate with the groundwater pressure, is introduced to prevent groundwater from entering the workspace.
Due to the high air pressure in the workspace, remote construction is the standard.
The caisson excavator targeted in this demonstration refers to the excavation machinery used in the pneumatic caisson method.

 

Challenges of remote construction and the significance of autonomous construction machinery

In recent years, in the pneumatic caisson method, operators have been performing construction by remotely operating the caisson shovel based on the information from the camera for remote operation. However, there are challenges such as it being difficult for the site supervisor to accurately grasp the overall situation inside the workspace, and the operators not being able to grasp the distance to the ground, resulting in decreased work efficiency. At the end of the day, workers are still required to enter the workspace and measure directly. It is difficult to accurately grasp the overall situation inside the workspace at a glance, and it requires skilled expertise to operate the machinery while accurately understanding the sense of distance to the ground and the shape.

The actual situation at the site seen in the camera footage used by the operator.

In the construction industry in Japan, compared to other industries, the aging rate is high, and the shortage of workers is a significant issue now and for the future. In recent years, there has been a strong push to alleviate the labor shortage in the construction industry by promoting labor-saving and automation measures. Within this context, the autonomous operation of construction machinery is expected to contribute to labor-saving and safety improvements, potentially alleviating and resolving significant challenges in this industry. Our initiative aims to solve such challenges by promoting the ICT integration and automation of construction in the pneumatic caisson method.
 

Demonstrated that multiple construction machines can be operated automatically in a stable and continuous manner, meeting the performance requirements for on-site introduction.

In this demonstration, we confirmed that the developed construction machinery autonomous driving system meets the performance requirements for on-site introduction.
We took on primarily the development and provision of software, as well as integration work on site using these software. For the software, we developed a ROS2-based robot system that forms the basis, along with GUI, algorithms for perception, and a simulator for development verification. Oriental Shiraishi handled other demonstration sites and hardware arrangements.

The autonomous driving system for the caisson excavators that we developed this time can stably and continuously operate the caisson excavators at a level suitable for on-site introduction, automating the excavation of the ground and the transportation of the excavated soil.
Below, we introduce the representative functions of this system and the development organization.

A. Main functions of the developed autonomous driving system

A-1. Real-time visualization system in digital-twin

In real-time, this system perceives the position and posture of the two caisson excavators, as well as the shape and height of the ground and can visualize them in three dimensions in a digital twin format. It can also automatically calculate one of the most important indicator, the opening ratio, that shows the unique and critical work situation in pneumatic caisson construction. As a result, on-site workers and supervisors can immediately get an overview of the current situation at any time, improving work efficiency and management efficiency.

A-2. Collision prevention

This system has a feature that prevents collision between the caisson excavators. Specifically, if the two caisson excavators get too close, an alert will be issued on the screen, and if they get even closer, a warning will be issued and the operation will stop. This can prevent collisions between machinery not only during the automatic operation of multiple machines, but also when automatic and manual operations are mixed, improving the efficiency and safety of the entire operation.

A-3. UI that makes it easy to specify work details

Users can initiate the automated operation simply by specifying the excavation area (where to dig) and soil removal area (where to transport the excavated soil) for each excavator on the GUI of this system, which visualizes the terrain and construction machinery in real time, and then clicking the start button. After starting the automatic operation, the system 1) automatically calculates the trajectory for moving the machinery, considering the shape of the ground, 2) the machinery moves automatically according to the calculated trajectory, and 3) the system can automatically carry out a series of operations such as excavation and soil removal in the specified areas.

A-4. Simultaneous automatic operation of multiple machines

After the start of automatic operation, users basically only need to monitor the behavior of the machinery, so it is possible to operate multiple machines by one person, which is expected to have labor-saving effects. Also, it is possible to have continuous automatic operation during times when work is usually not done (for example, at night), which may potentially shorten the overall construction period.

A-5. Stable automatic operation and seamless manual operation under conditions not suitable for automatic operation

In addition to a real-time environmental perception mechanism, this system allows for excavation operations that take into account the shape and hardness of the terrain. Therefore, it is a robust system that can cope with changes in the environment within a certain range, enabling stable automatic operation even at construction sites with many variations and fluctuations in conditions. In case of encountering irregular situations that cannot be dealt with above, it is assumed that the system will switch to manual operation, improve the situation by hand, and then switch back to automatic operation. To accommodate such operations, this system can easily switch from automatic operation to manual operation and vice versa.

Furthermore, Mr. Negishi from the Technical Department of Oriental Shiraishi, a user of the construction machinery automatic driving system developed this time, has given us the evaluation, “The system’s user interface is easy to see, and automatic operation can be executed with intuitive and simple operations. Even a single operator can keep multiple construction machines moving automatically without stopping, which is very convenient. In the future, we would like to challenge ourselves to expand the range of work we can handle while promoting the introduction to the site, and to move more construction machines at the same time“.

B. Development Organization

In this demonstration, a solution team closely collaborated with customers to facilitate requirements definition and verification, while a software engineering team developed the visualization system. The software development and verification were carried out through the collaboration of these teams. Notably, the software engineering team comprised members from various nationalities, forming a global team.

A scene where a member from Oriental Shiraishi explain the history of the Pneumatic caisson method and the importance of autonomous driving.

The members of the solution team and the software engineering team involved in this project.

 

Pneumatic caisson method and Remote, Automated, and Autonomous operations

The pneumatic caisson method was the first construction method to be remotely executed in Japan.
Because the air pressure in the work space is high and remote construction is standard in many cases, sensors and actuators have been digitalized, providing a basis for automation.
Also, since people do not usually enter the work area, the safety requirements for autonomous operation are lower compared to general earthworks, and there are fewer restrictions from Industrial Safety and Health Act, which prohibit people from entering the working radius of industrial machinery.
Lastly, the work primarily involves repeated excavation and transportation of the earth.
Therefore, this method is considered to have favorable conditions for demonstrating and advancing automation and autonomous technology.

Future outlook

We plan to introduce the construction machinery automatic driving system developed this time into the field within this fiscal year and start its actual operation. We will continue to forge ahead in pioneering the practical application of autonomous operation of construction machinery.