In 2022, DeepX, in collaboration with Tadano Ltd. (hereafter referred to as Tadano), developed and announced a crane automation technology that carries out a three-axis lifting operation, which uses rotation, boom movement, and winching simultaneously, while keeping the load sway to a minimum.

Crane operation and the difficulty of its control

A crane is a piece of construction machinery engaged in lifting operations at various sites across a wide range of industries, including but not limited to construction and civil engineering.
Cranes have operating axes such as rotation, undulation, and winches, and are widely used to transport heavy loads from one point to another. Crane operators must simultaneously operate multiple levers, such as rotation, vertical motion, and winch, while visually checking the position of the load.
In particular, since the load is suspended by a wire, it undergoes pendulum motion. The phenomenon of the load swinging (“load swing”) is extremely dangerous because the load is heavy. If a person comes into contact with a swinging load, it can lead to a serious accident. Also, because the wire is long, it takes a long time for the load swing to naturally subside once it starts. In most cases, there are people waiting at the destination of the load, who perform a task called “unhooking,” which involves removing the load from the crane’s hook.
Therefore, operating while suppressing load swing is essential from the perspectives of safety and productivity.

The degree of load swing allowed in crane operation is extremely small compared to the length of the wire.

The disturbances that make crane operation more challenging than it appears

Furthermore, crane operation is affected by disturbances such as mechanical deflection and wind. Due to the heaviness of the load, a phenomenon occurs where the arm or rod section (known as the “boom”) supporting the wire and the frame supporting the boom bends. The degree of bending is not uniform and varies depending on the load and posture. Under certain conditions, the tip of the boom can bend more than a meter, with a load. Operators must consider the degree of bending, which changes from moment to moment during operation, while moving the position of the load to the desired location.

On top of that, the behavior of the load changes due to the effect of the wind, behaving differently than in windless conditions. At construction sites, the wind may or may not blow depending on the day, location, and height. Sometimes the wind blows in a certain direction, and sometimes it suddenly blows in a random direction. Operators need to be flexible in responding to unexpected sudden wind movements of the load, while grasping the overall trend of the effects of the wind.

To cope with these disturbances, skilled expertise is required. It is said that mastery can take three to five years, and in some cases, as long as ten years.

Examples of factors that make crane operation difficult.

The challenge of the industry and the significance of operation support and autonomous driving functions

On the other hand, in Japan, the number of construction workers is decreasing due to the decrease in the working-age population caused by the declining birthrate and aging population. The decrease in skilled operators who can freely operate construction machinery is also a major challenge. The mastery of crane operation often requires more years than the operation of other construction machinery, making this challenge even more serious.

This initiative aims to reduce the burden of crane operation and improve safety through assistive and automation functions. Specifically, we are conducting research and development with the aim of improving on-site safety by simplifying, facilitating, automating, and making mobile crane operations more autonomous using robotic systems that perform real-time information processing, including AI.

Problems associated with cranes and expected benefits of automation.

The Developed Crane Automation Technology

In this initiative, we developed crane automation technology that carries out a three-axis operation, a lifting operation that simultaneously uses rotation, undulation, and winch, while keeping the load swing to a minimum.

This technology, after performing the hooking operation (the task of hooking the load onto the hook at the end of the wire) and the ground-cutting operation (the operation of lifting the load from the ground), allows you to specify the rotation angle, working radius, and lifting height that correspond to the target coordinates where you want to carry the load. By running the autonomous driving program, it enables the automatic transportation of the load from any point in space to the specified target coordinates, while keeping the load swing to a minimum. Currently, the developed automation software is rated as having operational performance equivalent to that of an intermediate operator.

The condition of the crane and load during autonomous operation as seen from the seat and the end of the boom.

DeepX has developed the automation system based on ROS2

DeepX was primarily responsible for the development and provision of software. For the software, we developed control algorithms and simulators for development and verification, in addition to the base robot system. Tadano handled the arrangement of other test environments and hardware.

A. The main software we developed

A-1. Robotic system

We have developed a robot system based on ROS2. In this project, we are looking ahead to the improvement and extension of this technology and the development of subsequent autonomous driving systems, and thus, we have particularly focused on the robustness and extensibility of the system in our development.

A-2. Simulator

We developed a simulator, based on Gazebo, that reproduces the deflection of the crane, hydraulic actuators, and other disturbance elements. Additionally, our robot system based on ROS2 can be connected to the developed simulator, allowing for detailed and quick development verification of the entire robot system through the simulator.

A-2. Control algorithm

We developed a control algorithm using an approach that uses a simulator to optimize and tune the control model. A feature of this algorithm is that it does not uniquely identify and design the behavior of the control target system, but learns and optimizes the control algorithm based on simulation data and the design values of the actual machine, enabling the safe and accurate movement of the load to any position while suppressing load sway.

Technical approach.

B. Development organization

In this development, a solution team closely collaborated with customers to facilitate requirements definition and verification, while a software engineering team developed the automation system. The software development and verification were carried out through the collaboration of these teams. Notably, the software engineering team comprised members from various nationalities, forming a global team.

Future outlook

We intend to work on further advancements in technology aimed at automating more complex crane operations, specifically targeting mobile cranes. Additionally, we aim to implement this technology in various cranes across a wide range of industries. We will continue to pioneer the frontier of construction machinery autonomous driving for practical implementation in the future.