Revolutionising Agriculture Robotics with 3D Deep Learning

As far as physics goes, we live in a world of three dimensions—the x, y and z of everyday coordinates. For millennia, natural selection has given humans the ability to see things in height, length and width. Our vision allows us to connect with our surroundings and complete daily tasks efficiently and safely. From hunting to gathering to food production, we continue to meet our basic needs because of this ability. 

As one of the world’s oldest professions, the agriculture industry still depends on workers’ vision to perform a range of farming activities: planting, cutting, pruning and harvesting to name a few. However, the quality of labour and the shortage of workers affects the productivity, profitability and scalability of food production. Ushering in a new age of agriculture, we are finding more and more ways to replicate the human visual system to automate and robotise even the most complex crop work activities. 

Robovision has been a pioneer in 3D deep learning, a technological revolution for building state-of-the-art agriculture robotics. Combining multiple camera 3D vision technology and deep learning techniques, we are developing a new generation of agricultural robots together with our OEM partners. It’s a whole new range of agriculture robotics that can plant, harvest, prune and keep learning. 

When it comes to complex plant manipulations, there’s no way around 3D. For a robot arm to know where to cut a rose, traditional 2D deep learning should be able to tell leaves from branches or segments ready-to-harvest flowers, but to catch the twig at just the right angle, you need that third dimension.

Figure 1: A rose cutter with an integrated vision AI system developed by Robovision

By combining multiple RGB cameras possibly together with depth sensors, Robovision creates a 3D representation of reality. To get the image just right, we calibrate the cameras so that each one is perfectly aligned with the others. Not by getting out a ruler, but by using a scalable multi-camera calibration software. Next, we process or analyse the 2D images and the compiled 3D model using our deep learning technique. That’s how we build a robust pipeline for robotic farming. 

Figure 2: 3D reconstruction of objects is an area of expertise that Robovision has built

With a configuration this precise, almost anything in plant phenotyping or horticulture robotics becomes possible. Together with our OEM partner – ISO Group, Robovision built a rose cutter in which plants are isolated in space and lit just the way we want them, before being assessed by our computer vision in a highly controlled environment. 

At the other end of the spectrum, there is a de-leafing robot that moves through a greenhouse. The robot looks at a wall of tightly packed greenery and knows exactly which crops to harvest, based on their shape, colour and so on. A much more chaotic, challenging environment, yet one we know how to handle.

Figure 3: Tulip bulb planters using 3D deep-learning 

Next to the setup, calibration is vital to tackle the richness of nature in all its colours, types and shapes. But the reason we have over 600 machines running live all over the world isn’t just about our software’s versatility. It’s equally the merit of our plant team. They’re the ones who screen each particular agricultural robot concept. They listen to each partner’s plans and wishes, scan the gear they already have in place and make sure we close that 10% gap we mentioned.

Yes, AI can do impressive things. But tweaking the software so the robot does exactly what the market needs is necessary, if you’re looking to excel. That’s why, with over 12 years of experience in Agtech, our team almost knows just as much about agriculture, crops and plants as we do about deep learning.

Figure 4: Our expert team has over 12 years of experience in Agtech

In horticulture robotics, it’s hard to come to a scalable product. There’s no lack of examples where a robot is designed and developed in a R&D greenhouse, and performs its task with high precision. However, when taking the robot out of its controlled demo environment and accuracy plummets, farmers are left frustrated because their investment isn’t paying off. 

“When building ready-to-deploy robots in agriculture, the last 10% is 90% of the work. It’s about ensuring a robot works out in the field, not just in the demo or R&D setting. Our approach is to deliver robust algorithms plus an AI platform that enables end-users to adapt the algorithms to their specific needs, and thus making these robot concepts scalable.”

Rob Gielen, Sales Manager Agri & Food at Robovision

In the Robovision AI approach, we enable farmers to take matters into their own hands. Through the Robovision AI platform, farmers and operators can capture data, upload it, label it and thus retrain their model themselves. In other words: a robot that was taught to cut a random variety of cucumbers right when they’re at their juiciest, can be taught to do the same, at the same accuracy, with cucumbers of another variety and/or morphology. Thanks to our modular algorithms, AI becomes scalable while fitting right in your palm, no code needed.

Figure 5: Robovision Platform makes it easy to apply 3D deep learning without a background in engineering or AI