RSJ2011 OSS Robotics and Tools OpenHRI Intro
1 like623 views
OpenHRI is an open source software for building human robot interaction systems using standard components and interfaces. It integrates audio, speech recognition, synthesis and dialogue management components. OpenHRI aims to reduce the difficulty of development through easy installation, support for multiple languages, and automated documentation. Testing involves inverting the perceptual system to require native speakers. Deployment relies on package repositories for Linux and in-house scripts for Windows. The next challenge is sustainability of the project.
![NEDO IRTS common interface
specification and OpenHRI
PortAudio, PulseAudio speex Julius
(Octet) (Octet)
Audio Device Audio Filters Speech Recognizer
NLTK (XML)
(under development)
Lexical Processors
(Any) SEAT
OpenJTalk,
[matsusaka2009]
Festival (XML)
Soar
Speech Dialogue Manager
Synthesizer (XML)
(Any)
(Any)](https://image.slidesharecdn.com/rsj2011-oss-robotics-matsusaka-ppt-110908210214-phpapp02/85/RSJ2011-OSS-Robotics-and-Tools-OpenHRI-Intro-6-320.jpg)
![Deployment
For Ubuntu system:
- Heavily rely on service of launchpad.net.
Upload the source package ->
1) Compile the binaries for all the architectures.
2) Construct and host the debian repository.
For Windows system:
- In-house package update script.
- Heavily rely on full time programmer working for
this project.
- [Work-in-progress] Automatic compilation and
packaging.](https://image.slidesharecdn.com/rsj2011-oss-robotics-matsusaka-ppt-110908210214-phpapp02/85/RSJ2011-OSS-Robotics-and-Tools-OpenHRI-Intro-17-320.jpg)
RSJ2011 OSS Robotics and Tools OpenHRI Intro
- 1. Developing OpenHRI -Open Source Software Components for Human Robot Interaction- Yosuke Matsusaka National Institute of Advanced Industrial Science and Technology (AIST)
- 3. Background • Industrial Robot – Expert Operator – Specified Action • Personal/Service Robot – Naïve User＝Operator – Various Situations and Tasks • "Communication" is the key function.
- 4. Why Development of Communication Function is Difficult? Reason: Requires Wide Range of Knowledge -Audio Signal Processing -Lexical Analysis -Speech Recognition / Synthesis -Dialogue, etc… As a result…: High development effort. (When we develop from scratch…) Low learning curve. (Because we are not the professional in developing communication systems…)
- 5. OpenHRI •Reference implementation of common interface specification defined in NEDO Intelligent Robot Technology Software Project (NEDO-IRTS). •Integrate wide range of open source software based on RT-Middleware specification.
- 6. NEDO IRTS common interface specification and OpenHRI PortAudio, PulseAudio speex Julius (Octet) (Octet) Audio Device Audio Filters Speech Recognizer NLTK (XML) (under development) Lexical Processors (Any) SEAT OpenJTalk, [matsusaka2009] Festival (XML) Soar Speech Dialogue Manager Synthesizer (XML) (Any) (Any)
- 7. Application Examples HRP-series: Humanoid platform for general tasks. RH-1: TAIZO: Office service robot. Rehabilitation exercise Guide service, Control demonstration robot. TVs, etc... 17 exercise + greet. OpenHRP: Virtual simulator SmartPalTM: Object handling. Navigaion. openhrp-demo.m4v
- 8. Comparison to Previous Researches Auditory Complete Complete Support for Components Communication Robotic System Dialog System Manager HARK ◯ △ × × (Julius-HARK) ROS+HA ◯ ◯ ◯ × RK (Pocket Sphinx, (100 > Flite) components) Galatea × ◯ × × (Julius, GalateaTalk) OpenHRI △ ◯ ◯ ◯ (less support for (Julius, OpenJTalk, (200 > mic. array) Festival) components)
- 9. Feature: Very Easy to Install On Ubuntu (3 commands): $ sudo apt-add-repository ppa:openhri/ppa $ sudo apt-get update $ sudo apt-get install openhriaudio openhrivoice seatsat -> Install all the dependencies. On Windows (integrated installer): http://openhri.net/getinstaller.php -> Hit "Yes" on all the dialogs.
- 10. Feature: Standard Oriented Grammar and Tools W3C-SRGS <?xml version="1.0" encoding="UTF-8" ?> <grammar> <rule id="greet"> <one-of> <item>hello</item> <item>good afternoon</item> <item>good evening</item> <item>good bye</item> <item>bye</item> </one-of> </rule>
- 11. Feature: Standard Oriented Grammar and Tools W3C-SRGS $ srgstojulius sample.grxml | juliustographviz | dot
- 12. Feature: Multi-Lingual Voice Recognition and Synthesis Currently Support: - Japanese (OpenJTalk, Julius runkit) - English (Festival, Julius acoustic model by voxforge) Experimental Support: - German (MARY, Julius acoustic model by voxforge) Planned: - Spanish, Korean, ...
- 14. Multi-Lingual Testing Normal system composition: AudioIn VoiceRecog DialogMgmt VoiceSynth AudioOut - Require native speakers to test each language. Make it inverse! Sentence Result VoiceSynth (AudioFilter) VoiceRecog Producer Validator ※ Thanks to component architecture. We only have to modify component connections do this.
- 15. Document Automation We use SPHINX and rtdoc (included in rtshell) to automatically generate component documents. Component I18N Component Source Documents (Document Embedded) (reStructuredText) rtdoc gettext + poedit SPHINX Component Binary Integrated I18N (I18N Document Document Embedded) (html, pdf)
- 17. Deployment For Ubuntu system: - Heavily rely on service of launchpad.net. Upload the source package -> 1) Compile the binaries for all the architectures. 2) Construct and host the debian repository. For Windows system: - In-house package update script. - Heavily rely on full time programmer working for this project. - [Work-in-progress] Automatic compilation and packaging.
- 18. Summary Overview: - OpenHRI is a complete, easy to install components to construct human robot interaction system. - Common interface is standard oriented and shared with commercial software companies. Development Techniques: - Test is done by inversing the perceptual system. - Document automaton by SPHINX, rtdoc. - Deployment by launchpad, update script. Next challenge and open question: "How can we make this sustainable?"
- 19. NEDO’s Strategy to Spread RT-Middleware Industry -2009 2010- More Industrial Developers performance (Commercial components) Switch High initial cost Open Source Co Developers mm on (FLOSS components) Inte rfa ce Less initial cost Application Developers