Ph.D. Thesis

Title: Communication in Robot Learning: Usage of the Combined Task and Social Channel for Action Learning
Committee:
Advisor:Prof. Dr. Mohamed Chetouani Sorbonne Université, France
Reviewers: Prof. Dr. Ginevra Castellano Uppsala Universitet, Sweden
Prof. Dr. Patrick Henaff Université de Lorraine, France
Examiners: Prof. Dr. Francisco S. Melo Instituto Superior Técnico, Portugal
Dr. Malika Auvray Sorbonne Université, France
Dr. Mehdi Khamassi Sorbonne Université, France

Human behavior allows for actions that combine task and social aspects in one action. Human-Robot Interaction (HRI) has not yet sufficiently addressed these combined actions. In my thesis I focus on actions that combine task and social actions. We are interested in how humans use these type of actions in HRI settings, how teaching influences this behavior and how these actions can be used to to augment robot learning.

Overview

In order to position my research, I developed a communication model that is able to describe teaching situation and their influence on the usage of the task and social channel. For the details please refer to my thesis, in particular Chapter 6.

Focusing on the human side I addressed the question How are humans teaching a sensorimotor task to a robot? by conducting and evaluating a HRI experiment. I showed that when humans teach a robot how to solve a sensorimotor task, they communicate additional information via their actions as opposed when they just solve the task. For more details please see the corresponding publication.

Focusing on the robotic side I addressed the question How can we integrate a social component (specifically goal communication) into robot learning? by developing a reinforcement learning framework that integrates reasoning of a potential observer as feedback. I used the framework to compare different approaches implementing the reasoning process of the observer and show that they achieve higher legibility values than a classical reinforcement learning baseline. For more details please see the corresponding publication.

Contributions
The contributions can be summarized as:
  1. Proposition of a communication model

  2. Insights on human behavior and perception of sensorimotor actions in the context of HRI

  3. RL framework integrating reasoning about a potential observer

User Study on Human Teaching Behavior Towards Robots in a Sensorimotor Task

In this study we were interested in the way how humans use the available communication channel to teach how to solve a task to a robot. We had two hypotheses:

  1. Humans modify their behavior when teaching how to solve a sensorimotor task in comparison to solving it.
  2. Humans perceive the use of negative demonstrations as informative.

According to our hypothesis the model for the solving condition, respectively for the teaching condition will look as follows:

In order to test our hypotheses we designed and conducted a study consisting of two experiments:

  1. In the first experiment we asked participants:
    • First to solve a sensorimotor task
    • Second to teach how to solve it to a robot
  2. In the second experiment we asked new participants to rate the informativeness of the demonstrations on a Likert-Scale

1. Experiment

The first experiment was conducted with 42 participants. An example environment is shown below:

The task is solved on a tablet with a pen and consists of going from the start zone to the end zone. Walls can't be passed, unsafe zones can, but should not be passed. Negative demonstrations pass unsafe zones.

The experiment includes a Solving- and a Teaching-phase, in order to show that there is a difference between solving and teaching. In the Solving-phase, the participants were just asked to solve the task correctly. In the Teaching-phase, the participants were asked to give positive and negative demonstrations.

2. Experiment

In the second experiment we asked 72 new participants to rate the collected demonstrations. The participants were asked to rate the statement: "The demonstrator deviates from the simplest way of solving to convey to you other information about the task" on a 5-point-Likert scale from strongly disagree to strongly agree.

Results

The ratings were clustered as non-informative, neutral and informative. Each demonstration was classified according to a majority voting between all participants that gave a rating on a particular demonstration, demonstrations that did not have a majority for neither non-informative nor informative were counted as not-clear.

For the solving condition we have the relative numbers:

Solving-condition, relative numbers (in %)
non-informative not-clear informative
positive demonstrations 84.43 8.12 7.45

For the teaching condition we have the relative numbers:

Teaching-condition, relative numbers (in %)
non-informative not-clear informative
all demonstrations 50.58 14.75 34.67
positive demonstrations 71.57 11.1 17.33
negative demonstrations 22.37 19.66 57.97

In the set of positive demonstrations, the relative portion of informative demonstrations in the Teaching-phase (17.33%) is higher than in the Solving-phase (7.43%). This difference between the Solving- and Teaching-phase is significant considering the columns for non-informative, not-clear and informative, χ²(2, N=1780)=39.52, p<0.01, as well when only considering the columns for non-informative and informative, χ²(1, N=1600)=34.42, p<0.01.

Conclusion

In this study we showed that there is a difference between people solving and teaching a sensorimotor task to a robot confirming our first hypothesis. The novelty of this work is that we show that people perceive a significant higher portion of negative than positive demonstrations as informative. Further, ~58% of the negative demonstrations are informative, indicating that our second hypothesis is also verified.

Remotely controlled Lego Wall-E

Not too scientific and not a big project either - a lot of fun nevertheless! I customized a LEGO Wall-E robot set with two motors that can be controlled remotely. The two motors can be controlled independently. See the result in the video below. Wouldn't it be fantastic to add more motors to get functional grippers?

More to come

In the days and weeks to come, I'll add more descriptions of past work. In particularly, a focus on my papers, my Master's thesis and the project on the bipedal walker Fabi.