Scribd
Upload
34 views

Reinforcement Learning and Transfer Learning: Simulation-Robot System For Object-Handling

This document discusses a project exploring transfer learning and reinforcement learning techniques for vision-based robotic controllers to allow robots to rapidly adapt to new environments. The project aims to develop efficient reinforcement learning algorithms for object handling tasks using self-learning robots, and explore how to transfer knowledge learned in simulations to real robots to minimize computational costs and improve results. It will involve contextualizing relevant techniques, testing in classical applications, evaluating new approaches, and comparing results to classic methods.

Uploaded by

Expedito Mello
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
34 views

Reinforcement Learning and Transfer Learning: Simulation-Robot System For Object-Handling

This document discusses a project exploring transfer learning and reinforcement learning techniques for vision-based robotic controllers to allow robots to rapidly adapt to new environments. The project aims to develop efficient reinforcement learning algorithms for object handling tasks using self-learning robots, and explore how to transfer knowledge learned in simulations to real robots to minimize computational costs and improve results. It will involve contextualizing relevant techniques, testing in classical applications, evaluating new approaches, and comparing results to classic methods.

Uploaded by

Expedito Mello
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Title

Reinforcement Learning and Transfer Learning: Simulation-Robot System for


Object-Handling

Summary

To allow robots to be able to adapt rapidly to new environments they are


being faced with, the following project explores a transfer learning system to vision-
based robotic controllers, making use of reinforcement learning techniques.

Introduction

The self-learning capacity of robotic controllers is vastly explored, by


academic literature and by the business market alike. Notwithstanding, the majority
of researches consists on the learning of aforementioned controllers, utilizing still
non-processed data - such as the values of each pixel on a visor – with no previous
knowledge.
It is known, however that data processed by different controllers, tasks or by
simulation is not reused in other robots. Therefore, it is vital to investigate the
algorithms to reuse of said data so it is possible to not only avoid expenditures, but
also to save computational time, and enhance a controller’s performance when
being adapted to a new function.

Contents

Self-learning robots have been achieving high accuracy and repeatability in


specific assignments. Withal, with the intention of being more practical, and having
the capacity of being handled in common places (such as households), they need
to swiftly adapt to situations in which they will be utilized; which immensely contrasts
with the actual lengthy span of time and heavy data requirement needed for the
above acclimation. In the field of robotics, there is a bigger obstacle due to external

1
elements, e.g.: energy used, and/or maintenance of components. Considering, then,
the mishaps, recently-done analyses seek to make use of similar and previously
processes data so that the optimal decision can be made more nimbly.
An important paradigm for robots to develop an acceptable control over their
own motors, which will be approached by this project is the Reinforcement Learning
– a topic in the field of Machine Learning – that enables computer programs and
robots to fulfill a certain duty through trial and error. Different tasks can currently be
taught, like: walking, folding shirts, handling objects, among others.
Another sphere of study explored is Transfer Learning. This area of research
is based on the utilization of the expertise obtained after the optimization of a task
to improve another, understanding the second to be of a related domain. Such
proposal supplies an array of benefits when compared to old methods: the reduction
of the processing (that would otherwise make it unfeasible) being the main one.
The linking of these two groups of techniques, used in similar ways in articles
referenced ([1], [2], [3] e [4]), will be implemented to achieve the proposed scientific
investigations. The first one is developing efficient algorithms of Reinforcement
Learning, optimizing the object-handling assignments through the means of self-
learning with the use of vision-based robots (based on activities made in
backgrounds such as the Amazon Picking Challenge). The second inquiry will be
exploring different possibilities for the transference of simulation-acquired
knowledge to an actual robot, with the intention of minimizing the computational cost
in order to learn such task, and, finally, yield better results.

Methodology and Research Plans

To achieve the proposed goals, this research will perform four steps of
scientific development:

1. Contextualization:
This step will be responsible for keeping scientific coherence, revising the
academic literature, so that it is possible to elaborate a tools and knowledge map
that are in accord with what could and should be used during the development of
the project. Some examples follow: classical techniques of Reinforcement Learning,

2
different architectures of Neural Networks, Neural Networks applied to
Reinforcement Learning and optimizing Evolutionary Algorithms.
Estimated duration: 3 months.

2. Tests in classical applications and building of the development environment:


In this step, simulations and actual environments of classical experiments
shall be used, such as the inverted pendulum, with the intention of investigating the
first results. This phase is also intertwined with the implementation of a vast library
for the simulation of a controller and the communication with the robot. At its end,
the goal will be to have a basis of results with simple experiments, a framework for
reinforcement learning, a task simulator that will be performed by the robot and a
connection component based on the ROS control interfaces, an operational system,
between robot and computer.
Estimated duration: 10 months.

3. Test phase:
Having the environment built, all possible and viable approaches found will
be used, that are within the span of time allotted for the project, and that appear to
yield promising results. It will be possible, then, to test new methods, such as the
Deep Neural Networks, to evaluate the obtained algorithms.
Estimated duration: 6 months.

4. Final analyses phase:


Having ended the building of a strong literary basis, an environment for
simulation and new approaches, the accuracy-level tests between the new methods
and the classic ways will be performed and compared. With promising final results,
the implementation of the present project will be an important step towards the
development of new technologies to further enhance everyday life.
Estimated duration: 5 months.

You might also like