With potential to increase both safety and quality aspects in our daily use of and interaction with vehicles, autonomous vehicles are currently a major trend in the automotive industry. The initial focus up to now has been on autonomous driving of passenger cars, like platooning and queueing assistance etc. There have also been initial tests with systems of construction equipment that perform autonomous asphalt spreading and gravel loading etc. A further step to extend and improve the service we experience today, might be to combine vehicles and peripheral support devices to join autonomous driving with autonomous loading and unloading of goods. In the future, an autonomous electrified distribution truck might for example work together with support devices to enable autonomous loading and unloading of goods to and from our doorstep just hours after we ordered a pick-up or delivery service online.

The Robot based Autonomous Refuse handling (ROAR) project is a first attempt to demonstrate such an autonomous combination. An operator driven refuse collection truck is equipped with autonomous support devices to fetch, empty, and put back refuse bins in a predefined area.

The physical demonstrator in the ROAR project constitutes one truck and four support devices. When the truck has stopped in an area, a camera-equipped quadcopter is launched from the truck roof to search for bins and store their positions in the system. As bin positions become available in the system, an autonomously moving robot is sent out from the truck to fetch the first bin. The system’s path planner calculates the path to the bin as an array of waypoints. The planner calculates paths based on a pre-existing map of the area. Upon following the waypoints, the robot is intelligent enough to avoid obstacles that are not on the map. To accomplish this detection, the robot is equipped with a LiDAR and ultrasonic sensors.

After reaching the last waypoint, the robot changes from navigation to pick-up mode. By exploiting the LiDAR and a front facing camera, the exact position and orientation of the bin can be detected. The robot aligns itself so that the bin can be picked up.
After the pick-up, the planner provides the robot with a new path back to the truck. After the last waypoint, the robot aligns with the lift at the rear of the truck. The lift is set at a pre-defined angle, so that the robot can move up to the lift and hook the bin onto it. During the emptying of the bin, the lift system monitors the area around the lift with a camera to assure that no person is in the way for the lift. If so, the lift movement is paused until the area is clear.

An emptied bin is picked up by the robot and returned to its initial position, once again based on a path from the planner. When reaching the initial bin position, the bin is put down. The robot can thereafter move to the next bin to be emptied, and the emptying procedure is repeated.

When there are no more bins to empty, the robot moves back to the truck and aligns itself with the lift. Similar to a bin, the robot is hooked on to the lift and the overall procedure is completed. The truck can thus be started and be driven to the next area.
The coordination of the truck and the support devices is based on a discrete event system model. This model abstracts the overall emptying procedure into a finite number of states and transitions. The states capture distinguishable aspects of the system, such as for example the positions of the devices and empty/full states of the bins. The transitions model start and completion of the various operations that the devices can perform. All steps in the above description of the emptying procedure can be modeled by such operations.

The investment in the discrete event model carries a number of attractive properties. During the development phase, the model can be derived using formal methods. Verification as well as synthesis (iterative verification) is then employed to refine an initial model to satisfy specifications on the system.

Moreover, the development of the actual execution of an operation can be separated from the coordination of the operation. As an example, consider the operation modeling that the robot navigates along a path. From an execution point of view, the operation must assure that given a path the robot eventually ends up at the last waypoint without colliding with any obstacle. From a coordination point of view, the operation must only be enabled when there is a path present in the system and the robot is positioned close to the initial waypoint.

The model contains two types of operations; operations that model the nominal behavior, and operations that model foreseen non-nominal behavior. The recovery operations in the second group can for example describe what the system can do when the robot cannot find a bin at the end of a path, or how to re-hook an incorrectly placed bin on the truck lift.

The discrete event model can also be exploited to handle more severe recovery situations, after unforeseen errors. As part of the development, the restart states in the system are calculated from the model. Upon recovery to simplify the resynchronization between the control system and the physical system, the operator sets the active state of the control system to such a restart state and modifies the physical system accordingly. By recovering from a restart state, it is guaranteed that the system can eventually finish an ongoing emptying procedure.

The truck and the support devices are connected using the Robot Operating System (ROS). ROS is an operating system-like robotics middleware that among other things enables message-passing between components defined in the system. Two types of messages are used in the ROAR project. The first type is messages related to starting and completion of operations. An operation start message is triggered from a user interface and is translated into a method call in the support device executing that operation. Under nominal conditions, this support device will eventually respond with a message saying that the operation has been completed. Both messages will update the current active state of the control system.

The second type is messages related to transferring data. Data transfer can be both internally within the programs connected to a support device and externally between support devices. An example of external data transfer is a path that is created in the path planner and then transferred to the robot.

During execution, the discrete event model is hosted on a web server. Interaction with the model is facilitated by the server’s API. Operator interaction is accomplished through a web based user interface. By enabling a web-based interface an operator can access the model using any device connected to the system’s network. This can for example be a computer in the truck cabin or a touchpad strapped to the operator’s forearm.

At the other end, ROS is also connected to the API. As pointed out before, this connection enables that operations started by the operator through the user interface are translated into method calls in the appropriate support device. Completion of the operation execution is translated into a post-request in the API. This will update the discrete event model to capture that the operation has been completed.
The physical demonstrator in the ROAR project is limited to a single robot for the bin handling. A next step could be to include more bin handling robots. For the specific field of application with refuse handling, more bin handling robots could enable higher efficiency in the emptying procedure. Many robots might also permit that the noisy truck can be parked further away from the bins, and thus cause less disturbance where people live. Today this is to be avoided because a distant truck will force the operator to walk too long.

From a more general point of view, coordination of multiple autonomous devices is an open research question. The two extremes are that the coordination is either performed from one central unit to which all devices are connected, or that the devices are intelligent enough to solve the coordination internally among them in a distributed manner. The two major coordinating challenges to handle is distribution of tasks between the devices and distribution of space where the devices can operate. The overall goal is thus so accomplish all tasks in some optimal way assuring that no devices are physically blocked in the operating area.

The productification of this overall control and coordination between one truck and several autonomous support devices is an interesting challenge. Imagine a future scenario where a haulage contractor company orders a new system. The truck is perhaps ordered from company A, with heavy-duty equipment from company B. The equipment is complemented with support devices from company C and company D. To operate properly, the system should also use services from the cloud, provided by some companies E and F. To further add to the equation, it is likely that operators are also in the loop to cope with unforeseen situations, complex item handling and parts of the decision making.

All in all, this text has only cracked open the door for what will come after the autonomous driving of passenger cars that we see today. There are still many mountains to climb and standards to agree upon before other areas than “just” the driving becomes automated. The outcome of the ROAR project is thus only a small step on a long journey a head.

The ROAR project is initiated and lead by Volvo Group. Chalmers University of Technology, Mälardalen University. Pennsylvania State University take part in the project as being Preferred Academic Partners to Volvo Group. The intention from Volvo Group is that students through bachelor and master theses should perform most of the development.

Article provided by
Patrik Bergagård, PhD, ROAR Project Leader
Martin Fabian, Professor, Automation
IFAC Technical Committee 1.3: Discrete Event and Hybrid Systems