SCL Tutorial : Bot to Bot System

scl.git/applications-py/alexa_reader/
scl.git/applications-linux/scl_redis_sim/
scl.git/applications-linux/scl_redis_ctrl/
scl.git/applications-linux/scl_redis_visualizer/

Goal : Specify robots actions using natural language

This tutorial aims to introduce you to the Bot to Bot system, which maps natural language to robot motion. We will program a simple verbal bite interpreter for free space motions, and learn how to integrate it with a task controller for a robot.

Step 0 : Compile all SCL libraries

As a reminder, please follow the instructions here : SCL installation.

Step 1 : Get the robot configuration files

Here, we will focus on using a simulated Kinova Jaco 6 robot. You may get the robot specification files here : Kinova Jaco 6 files .
Or if you prefer code:

 scl@unicorn:~/Code/scl.git$ wget http://web.stanford.edu/~smenon/scl/KinovaJaco.zip
 scl@unicorn:~/Code/scl.git$ unzip KinovaJaco
 scl@unicorn:~/Code/scl.git$ cp -rf KinovaJaco specs/

Step 2 : Run the robot simulator

Now, we'll run the robot simulator. First, start the physics engine by going to the terminal and typing:

 scl@unicorn:~/Code/scl.git$ cd applications-linux/scl_redis_sim
 scl@unicorn:~/Code/scl.git/applications-linux/scl_redis_sim$ sh make_rel.sh
 scl@unicorn:~/Code/scl.git/applications-linux/scl_redis_sim$ ./scl_redis_sim ../../specs/KinovaJaco/KinovaCfg.xml kinovajaco6

This will start a physics simulation of the kinovajaco6 robot. The configuration file has a small value for gravity (0,0,-0.1 m/s2) just so you can observe what the simulation does.
To see the simulation, open a new terminal and type:

 scl@unicorn:~/Code/scl.git$ cd applications-linux/scl_redis_visualizer
 scl@unicorn:~/Code/scl.git/applications-linux/scl_redis_visualizer$ sh make_rel.sh
 scl@unicorn:~/Code/scl.git/applications-linux/scl_redis_visualizer$ ./scl_redis_visualizer ../../specs/KinovaJaco/KinovaCfg.xml kinovajaco6

At this point, you should see a window with a robot falling under gravity. Great! It's time to start the controller.

Step 3 : Control the robot

Now that we have a simulated robot up and running, we'll use an SCL task controller to control it.

 scl@unicorn:~/Code/scl.git$ cd applications-linux/scl_redis_ctrl
 scl@unicorn:~/Code/scl.git/applications-linux/scl_redis_ctrl$ sh make_rel.sh
 scl@unicorn:~/Code/scl.git/applications-linux/scl_redis_ctrl$ ./scl_redis_ctrl ../../specs/KinovaJaco/KinovaCfg.xml kinovajaco6 opc hand

At this point, the controller will start computing torque control commands and send them to a redis database.

To trigger the controller, we have to do set one enable flag in the redis database. To do so, open a terminal and type:

 scl@unicorn:~/Code/scl.git$ redis-cli
 127.0.0.1:6379> set scl::robot::kinovajaco6::fgc_command_enabled 1

Great! You'll notice that the Kinova will stop moving and hold position. You may now control it's hand by specifying simple text commands through redis.

 127.0.0.1:6379> get scl::robot::kinovajaco6::traj::xgoal
 "0.145454 0.508317 -0.253682 "
 127.0.0.1:6379> set scl::robot::kinovajaco6::traj::xgoal "0.145454 0.508317 0.253682"
 OK
 127.0.0.1:6379>

Notice, we took the xgoal position that the robot was holding itself at and moved it up along the z-axis. As a result, the robot moved its hand up. Sweet! So we can control the robot by specifying text keys in a transactional database. The amazing part of this system is that it is programming language agnostic. So from now on, we'll stop with C++ code and move to Python code (or your programming languages of choice)

Step 4 : Run a simple Verbal Bite Interpreter in Python

At this point, I would encourage you to quickly go through the Bot to Bot paper if you haven't already done so. The Bot to Bot presentation might help as well.

To refresh your memory, a verbal bite interpreter is a program that takes as an input a text command. The text commands, of course, come from somewhere in the natural language processing pieline. It then maps these to a mathematical operation that completely specifies a trajectory.

We provide a small template application. Please note that the actual bot to bot system outlines many capabilities that we won't consider. Our goal is simply to write a short script that takes text as an input and produces motions as an output.

For this, open the following file:

 scl@unicorn:~/Code/scl.git$ cd applications-py/alexa_reader
 scl@unicorn:~/Code/scl.git/applications-py/alexa_reader$ gedit alexa_reader.py

In line 41, change the ip address to your own ip address (localhost, or 127.0.0.1). And then run the interpreter:

 scl@unicorn:~/Code/scl.git/applications-py/alexa_reader$ python alexa_reader.py

Now that you have an interpreter, you can simply feed it text input by setting the appropriate key in redis. Ideally, this would be an asynchronous push from something at a level above in the stack. But just to highlight the modularity of the system, we'll directly splice into the level and edit the redis key:

 127.0.0.1:6379> set alexa::from '{"commands":"right"}'
  127.0.0.1:6379> set alexa::from '{"commands":"front front"}'

And you'll find the robot moving the way it is supposed to.

Step 5 : Edit the Verbal Bite Interpreter or write your own

Note that you essentially have direct control of the robot's hand. And you can write your own routines to create arbitrary motions at the hand. Feel free to modify the basic pythong program and create your own.

Have fun!

References and Links

Operational Space Control Math Tutorial: 3-dof and 6-dof chain robots.