The example problem – clean up this room!

In the course of this book, we will be using a single problem set that I feel most people can relate to easily, while still representing a real challenge for the most seasoned roboticist.  We will be using AI and robotics techniques to pick up toys in my upstairs game room after my grandchildren have visited.  That sound you just heard was the gasp from the professional robotics engineers and researchers in the audience. Why is this a tough problem, and why is it ideal for this book? 

This problem is a close analog to the problem Amazon has in picking items off of shelves and putting them in a box to send to you.  For the last several years, Amazon has sponsored the Amazon Robotics Challenge where they invited teams to try and pick items off shelves and put them into a box for cash prizes. They thought the program difficult enough to invite teams from around the world.  The contest was won in 2017 by a team from Australia. 

Let’s discuss the problem and break it down a bit.  Later, in Chapter 2, we will do a full task analysis, use cases, and storyboards to develop our approach, but we can start here with some general observations.

Robotics designers first start with the environment – where does the robot work? We divide environments into two categories: structured and unstructured.  A structured environment, such as the playing field for a first robotics competition, an assembly line, or lab bench, has everything in an organized space.  You have heard the saying A place for everything and everything in its place—that is a structured environment. Another way to think about it, is that we know in advance where everything is or is going to be.  We know what color things are, where they are placed in space, and what shape they are.  A name for this type of information is a prior knowledge – things we know in advance. Having advanced knowledge of the environment in robotics is sometimes absolutely essential. Assembly line robots are expecting parts to arrive in exactly the position and orientation to be grasped and placed into position.   In other words, we have arranged the world to suit the robot. 

In the world of our game room, this is simply not an option.  If I could get my grandchildren to put their toys in exactly the same spot each time, then we would not need a robot for this task.  We have a set of objects that is fairly fixed – we only have so many toys for them to play with.  We occasionally add things or lose toys, or something falls down the stairs, but the toys are a elements of a set of fixed objects.  What they are not is positioned or oriented in any particular manner – they are just where they were left when the kids finished playing with them and went home.  We also have a fixed set of furniture, but some parts move – the footstool or chairs can be moved around.  This is an unstructured environment, where the robot and the software have to adapt, not the toys or furniture.

The problem is to have the robot drive around the room, and pick up toys.  Let's break this task down into a series of steps:

1. We want the user to interact with the robot by talking to it.  We want the robot to understand what we want it to do, which is to say, what our intent is for the commands we are giving it. 
2. Once commanded to start,  the robot will have to identify an object as being a toy, and not a wall, a piece of furniture, or a door.
3. The robot must avoid hazards, the most important being the stairs going down to the first floor.  Robots have a particular problem with negative obstacles (dropoffs, curbs, cliffs, stairs, and so on), and that is exactly what we have here. 
4. Once the robot finds a toy, it has to determine how to pick the toy up with its robot arm.  Can it grasp the object directly, or must it scoop the item up, or push it along?  We expect that the robot will try different ways to pick up toys and may need several trial and error attempts. 
5. Once the toy is acquired by the robot arm, the robot needs to carry the toy to a toy box.  The robot must recognize the toy box in the room, remember where it is for repeat trips, and then position itself to place the toy in the box. Again, more than one attempt may be required. 
6. After the toy is dropped off, the robot returns to patrolling the room looking for more toys.  At some point, hopefully, all of the toys are retrieved.   It may have to ask us, the human, if the room is acceptable, or if it needs to continue cleaning.

What will we be learning from this problem?  We will be using this backdrop to examine a variety of AI techniques and tools.   The purpose of the book is to teach you how to develop AI solutions with robots.  It is the process and the approach that is the critical information here, not the problem and not the robot I developed so that we have something to take pictures of for the book.  We will be demonstrating techniques for making a moving machine that can learn and adapt to its environment.  I would expect that you will pick and choose which chapters to read and in which order according to your interests and you needs, and as such, each of the chapters will be standalone lessons. 

The first three chapters are foundation material that support all of the rest of the book by setting up the problem and providing a firm framework to attach all of the rest of the material.