Since 2017, DeepX has been working with Ishida Co., Ltd. to develop a robot that can serve specific amounts of pasta.
Ishida is a machine manufacturer that primarily makes and sells commercial equipment for weighing and packaging food. The company’s world-first “combination weighing” system holds an 80% share of the food production line machine market. A variety of foods that people may come across on a daily basis, such as convenience store lunch boxes and side dishes, packaged vegetables, and bagged potato chips, are weighed and packaged by Ishida machines.
Ishida is engaged in the development of automating operations related to food production. However, the processes involved in weighing and dispensing foods that are soft with a deformable shape are difficult to automate with existing technology, and most of this work is still done manually. As the labor shortage in the food processing industry continues and is expected to worsen in the future, improving productivity is becoming an urgent need for growth in the industry.
That is what started the development project for a “pasta-serving robot,” combining Ishida’s hardware technology with DeepX’s AI technology. What sort of approach is being taken? What is rewarding about engaging in this sort of development?
In this interview at the R&D facility, Ishida hardware engineer, Shuichi Maeda, and DeepX AI engineer, Akito Mori, discuss their experience.
Conventional automatic weighing technology has limitations when grasping deformable food
Mori (DeepX): Before they began collaboration with DeepX, Ishida was developing a pasta weighing machine in-house.
Maeda (Ishida): That’s right. Ishida anticipated this situation over 20 years ago, and has been taking on the challenge to automate in-house over the past few years. But I felt there was a limit to what Ishida’s technology could do. Conventional automated pick-and-place robots can only handle movement within certain rules; for example, only repeating a specific action like “moving an arm to a predetermined position and grasping the item.”
However, even if the machine moves according to the rules, the shapes of deformable objects will vary every time. This is especially true with pasta sold at convenience stores; it’s very difficult to pick up a specific amount due to various factors such as it being somewhat sticky and soft because long noodles can easily get tangled or because pasta is boiled differently depending on the menu. I felt that just improving the hardware would not be enough.
Mori (DeepX): However, if we can succeed in picking up a set amount of pasta, we will pretty much overcome the issue with deformable food.
Aiming to perfect the AI algorithm by repeatedly weighing and correcting errors
Mori (DeepX): There are two types of AI that DeepX is currently developing for Ishida. One confirms the condition of the pasta with 3D image recognition and determines where to position the robot arm to grasp the specified amount. The other adjusts the grip when picking up the pasta based on the resistance that the arm perceives from the pasta.
These two sets of AI are used to move the robot, put the pasta in a container, weigh it, and do it all over again if there is an error. Currently, we are calibrating the algorithm by trial and error by repeating this process.
Maeda (Ishida): When we decided to adopt AI technology, we thought this would be very fascinating. Yet at the same time, there was also
Mori (DeepX): Ishida has exceled a making stand-alone machinerythe robot they showed us in the beginning was not something that could be controlled by a computer. In order to operate the robot with AI, we first requested the development of hardware that could be computer controlled.
Proceeding with development in anticipation of on-site introduction
Maeda (Ishida): Since I’m usually in charge of mechanical design, based on my past experience, I designed an actuator (device) that moves the hand. I then showed it to Mori to get his feedback.
Rather than just communicating his requests one-sidedly, Mori tried to work with my perspective as much as possible. I wanted to make something simple and versatile that could be introduced into various fields. This is why we didn’t want to make it too complicated.
Mori (DeepX): Since the ultimate focus of the project is introducing the AI-equipped robot to the production line, I felt that considering those points was necessary for development. For example, we needed to have a system that could reduce installation costs to some extent when it is finally commercialized.
At DeepX, we strive to understand the customer’s thinking and move towards the goal while seeking a realistic solution. Consequently, I spoke to Maeda about what can be achieved with the introduction of AI, and also what AI actually is in the first place.
Automation technology that is attracting interest in the field
Maeda (Ishida): So, over the course of about a month, we modified the hardware so that it could be controlled by a computer. After that, we started experimenting with pasta.
Mori (DeepX): We went to the factory to film the movements that employees made when dispensing the food, and I myself actually tried picking up pasta by hand. In order to meet the pickup requirements that we considered, I consulted Maeda about having the hardware improved. Currently we are going through the PDCA cycle (Plan, Do, Check, Act) at fairly fast pace.
Maeda (Ishida): About a year after the start of the project, we managed to proceed with development to the point where we could do testing at the factory.
Mori (DeepX): Since we were able to get relatively good results from our in-house experiments, we felt pretty confident about this testing. However, the pasta that was boiled at the actual factory was in a completely different condition from what we had been using in our experiments, and we couldn’t achieve the results of the in-house experiments at all. We made improvements to both the software and hardware . Maeda also took the first train out and made a round trip from the site to Tokyo to obtain the parts we needed.
Maeda (Ishida): To tell you the truth, I was kind of in a panic (laughs). But as it turned out, we were able to After that, testing was conducted over four days, and during that time, people working at the factory really gave us their encouragement, and I could sense the excitement there about automation.
Visiting the factory leads to new discoveries
Mori (DeepX): I could see the foods I usually buy in a store being made right there at the factory. Moreover, I could do experiments while watching how that food was being produced. Visiting the factory enabled me to discover many things.
Maeda (Ishida): at the factory about the proper movement for the person grasping the pasta, and what sort of movement is . Going to the worksite will give you a more realistic sense of what is being developed.
Mori (DeepX): It’s not enough just to make a robot that can accurately weigh food. There are many points that must be taken into consideration in terms of hardware, such as a design that is easy to wash so that bacteria does not spread.
Thinking through what’s necessary to implement AI in society
Maeda (Ishida): At this point, we’ve only just cleared one hurdle. The next stage will entail thinking about the methods and countermeasures that will help prevent errors when we introduce the AI in the factory. Our company’s know-how can help us manage the typical errors that may occur in hardware, but those involved in AI are a big unknown. I’d like to work with Mori in preventing anticipated problems.
Mori (DeepX): Implementation is never easy. In order to implement AI in society, it’s not just a matter of creating AI. You also need to think about the steps involved in connecting it to society and you have to gather the knowledge necessary for that.
Maeda (Ishida): Over the past year, I’ve really been impressed with DeepX as they have collected data and repeated experiments, and as their work has started to pay off.
Take the arm movement, for example. The multi-joint arm makes complex twisting and turning motion possible. But, as I mentioned earlier, what Ishida is aiming for is simplicity. I wanted to reduce the number of joints. That’s why I thought smooth movement would be difficult.
But when we actually introduced the AI, it moves the arm very smoothly. Since the AI’s program is based on human movement, the robot moves like an articulated robot even when there are few joints. I think that DeepX was able to achieve this because they were very careful in figuring out how to approach even slight movements. I really sensed the significance of joint development.
Mori (DeepX): The key aspect of deep learning is what to have the computer learn. There are countless possibilities before you, but it’s important to consider “What is the necessary data?” and “How will we obtain that data, and what sort of algorithm would be the most practical to apply?” For that reason, you always have to think and be sure about what the real challenge is. By doing that, you can proceed efficiently with development and, consequently, come close to fulfilling the project.
Implementation of AI in these sorts of applications is not a goal that can be easily achieved, but thanks to the combination of wisdom and know-how from Maeda and his colleagues at Ishida, as well as members of our company, we’re working together to overcome challenges one by one.
Senior Engineer, DeepX, Inc.
Mori joined a major enterprise system integrator after graduating from a master’s program at the University of Tokyo. He worked as a data analyst and a project manager constructing systems which utilize AI technology in the new business development division for about six years. Sharing the vision of DeepX, he joined the company in 2018 with the desire to take on the challenge of automatic control for machinery – something which had not yet been achieved in the world. Leveraging his experience in business development, project promotion, and program implementation, Mori is engaged in a wide range of operations at DeepX.
Mechanical Engineer, Ishida Co., Ltd.
Maeda joined Ishida Co., Ltd after graduating from a master’s program at his university’s faculty of science and engineering. Assigned to the development technology department, he was involved in the development of new products such as measuring instruments and labelers for the food processing industry for about 10 years. In order to promote the automation of work related to food production in response to the labor shortage trend, he moved to a newly established department in 2017, and is now mainly engaged in grasping-related development work.
Ishida Co., Ltd.https://www.ishida.co.jp/
Ishida Co., Ltd. is an equipment manufacturer that supports safety for “the world’s food,” covering everything from production areas to food manufacturing and processing, logistics, and retail, with a focus on manufacturing and sales of commercial weighing and packing machines. The company currently operates in more than 100 countries around the world.