By

A. Srinivasa Rao
N.V.V.Jayaram Swaroop
2nd C.S.I.T.
KakinadaInstituteEngineering &Technology
Abstract

• This paper presents how Java technology is being implemented in Robots, which can see,
hear, speak and move.
• Java technology offers an array of API tailor made to the needs of the Robotics realm.
• The Java speech API lets to build command and control recognizers, dictation systems and
speech synthesizers.
• The Java media framework is used to receive and process virtual images.
• Gives the details of the implementation of java technology for the future in mars rover2009
Introduction
Robotics is playing an ever greater role in our world -- from industrial assembly 'droids performing the
most mundane of tasks, to search-and-rescue robots saving lives in the depths of collapsed buildings, to
interplanetary exploration robots probing the vastness of space. Such robots increasingly take on tasks
that are too boring, costly, or dangerous for human beings to perform.

But to develop robotic systems that are efficient and cost effective, we need a technology infrastructure
that is both robust and future-proof. In a world of increasingly diverse devices and operating systems,
with wildly differing hardware capacities and designs, it's essential that a robotics infrastructure be
device-neutral and platform-neutral. After all, in the world of space exploration, the contractor for a
given system can change at the stroke of a politician's pen.

When early explorers of the moon experienced a breakdown in their equipment, their technical manuals
reportedly advised them -- should all other diagnostic techniques fail -- to "kick with lunar boot." But
for a robotic Mars rover sitting on an alien planet 141 million miles from Earth, there's no one there to
do the kicking. A red planet robot in search of alien life can't afford the blue screen of death.

Enter Java Technology

The ability to hear, see, speaks, and move is key to the human ability to manipulate the world. But to
perform such human-like tasks, robotics systems must be able to emulate these functions via software.
And when it comes to enabling robust, device-neutral, platform-neutral systems, Java technology
virtually wrote the book.

Simon Ritter is a Technology Evangelist at Sun Microsystems. He speaks on many different aspects of
Java technology, but is a particular champion of Java technology within the world of robotics. He has
developed a Robotics Software Development Kit, and regularly gives compelling demonstrations of
robotics systems using Java technology. Among these demonstrations are small, resource-constrained
robots that use the LEGO Mind storms RCX "brick." Ritter provides detailed directions (both hardware
and software) for building, programming, and deploying his various LEGO robots.

And lest anyone think that Java technology-powered robotics is unavailable outside the domain of
enterprise developers, robotics speech synthesis engines are available for approximately $30, webcams
can be had for well under $100, and the LEGO Mind storms robot can be purchased for approximately
$200!
 Speech Recognition
The Java Speech API (JSAPI) provides a simple yet powerful interface into speech systems -- including
speech recognition and speech synthesis. "The Java Speech API doesn't actually provide the speech
recognition and speech synthesis systems.It provides the programmatic interface into them."

There are several low-cost facilities that use the Java Speech API. "It's been around for a few years now,
and IBM has been very good to implement the Java Speech API -- which we can download from their
Alpha Works site for free." And the Via Voice engine is very reasonably priced. For under $30,
developers can be up and ready to build speech-enabled Java applications.

Key to JSAPI's speech recognition functionality is the command-and-control recognizer. "You specify
certain words you want the system to recognize,and indicate that we want to have an event sent to
someone when those words are recognized." The Java Speech API's Grammar Format allows to define
words that the system should recognize. "It provides us with a cross-platform way of controlling the
speech recognizer" .

Sample code to create a robotics voice listener using a grammar format file

import javax.speech.*;
import javax.speech.recognition.*;

// Create recognizer and allocate resources

Recognizer recognizer = Central.createRecognizer(null);
recognizer.allocate();

/* Add engine listener - to notify when speech engine

* is stopped, started, etc.
*/
recognizer.addEngineListener(engineListener);

// Read-in grammar file

File gf = new FileReader(grammarFile);
RuleGrammar rules = recognizer.loadJSGF(gf);

/* Add result listener. VoiceListener then called when a

*particular grammar word is recognized. The associated
*event tag is passed to it
*/
rules.addResultListener(new VoiceListener());

// Tell recognizer to commit changes

recognizer.commitChanges();
// Request focus of microphone away for other apps.
recognizer.requestFocus();
// Start listening
recognizer.resume();
 // Remember to catch exceptions

Once a JSAPI result listener is defined, any recognized words will create an event that causes the defined result
listener method (VoiceListener) to be called. The method can then access the passed event tag, and determine the
recognized command.

Sample code to receive and process a speech command

import javax.speech.*;
import javax.speech.recognition.*;

// Called when a given grammar command is recognized

public class VoiceListener
extends ResultAdaptor {
public void resultAccepted(ResultEvent re) {
FinalRuleResult result = (FinalRuleResult)re.getSource();
// Get and examine grammar tag
String[] tags = result.getTags();
System.out.println("First tag was :" + tags[0]);
}
}

Speech Synthesis
On the flip side of the voice functionality coin lies speech synthesis. Here, the Java Speech API Markup
Language enhances the functionality and sophistication of spoken text. "We can select a particular voice
".depending upon the underlying system is supporting the API. We can also add context information in
terms of how to pronounce things by some estimates, only 10% of the information conveyed in a verbal
exchange comes from the actual words being spoken. So in order to make synthesized speech as subtle
and information-rich as that of human beings, it's important to include elements of pitch, volume, and
range -- prosody values -- as well as emphasis. "A lot of information comes from pitch, value, and range,"
says Ritter. "If we want the computer to sound more realistic, we need to be able to emulate those
characteristics."

Sample Java Speech API speech synthesis code

import javax.speech.*;
import javax.speech.synthesis.*;
 public void say(String words) {
// Create synthesizer
Synthesizer s = Central.createSynthesizer(null);
// Allocate resources
s.allocate();
// Speak string
s.speak(words, null);

The Vision Thing

For human beings, our most crucial ability for navigating the physical world is vision. In fact, by some
estimates, half the human brain is required to support visual processing! So naturally this is a key facility
for many robotic systems. But here again, the hardware for implementing such functionality is
surprisingly affordable. "USB webcams are really quite inexpensive now" .

There are two basic techniques for accessing visual data within Java technology-based robotic systems. A
quick and easy technique uses a Java technology/TWAIN interface. "TWAIN began as an interface
standard for connecting scanners to PCs”. “But it's been extended to work with things like webcams. A
group out of Slovakia has now created a Java/TWAIN interface, and if you are a non-commercial
organization, we can download it for free. That allows us to take an image from a TWAIN device and
create a Java AWT image from it -- which we can then display and manipulate."

But while simple to use, the TWAIN interface presents an on-screen user interface, leaving many
parameters of the image inaccessible programmatically. Plus, TWAIN is primarily designed for getting
single images. "When we use TWAIN for a demonstration, we would get about one frame, every one-and-
a-half seconds. That's a bit on the slow side if we’re trying to do real-time processing."

Java Technology Powered Mobile Robots -- the LEGO Mindstorms Robotics Invention System
For all those who might assume that mobile robotics is not something the average Java programmer can readily
explore, we have to experience the LEGO Mind storms Robotics Invention System. "LEGO came up with the Mind
storms system about three years ago," says Ritter. "It was developed in conjunction with MIT."
At the heart of the system is the programmable RCX "brick," a small
computer contained within a yellow LEGO brick. The brick consists of
a Hitachi 8-bit processor (16 MHz), 16 Kb of ROM, 32 Kb of RAM, 3
sensor inputs, 3 motor outputs, a 5-character LCD display, and an
infrared serial data communications port. The brick is a small and
extremely resource-constrained computing device -- particularly by
today's desktop standards of GHz processors and hundreds of Mb of
memory. LEGO RCX "Brick

The three outputs of the brick can be connected to motors and other devices, and the three inputs can be
connected to such varied sensor devices as light, touch, rotation, and even heat. The system can process over
1000 commands a second, and features a fully multitasking operating system (allowing up to ten
simultaneous tasks).

The brick was initially designed by LEGO to be programmed via a PC-based system that allows the visual
assembly of on-screen functional components. This component-driven system then generates a completed
program that can be downloaded into the brick.

But what opened the brick up to whole new vistas of innovation and functionality was the development of
the open source leJOS environment. The creators of leJOS have managed to squeeze an actual Java Runtime
Environment (including multi-threading) into 14 KB on the brick. But leJOS is obviously not a complete
implementation of the Java platform."Having the AWT on the brick doesn't make much sense." Due to
memory constraints, leJOS also lacks garbage collection, but the hooks are there for future implementations.

With leJOS, Java developers now have an inexpensive (yet multi-threaded) robotics platform available to
them. The basic kit starts at only about $200.

In order to accommodate future versions of the brick, the system was designed to allow for the easy loading
of new LEGO firmware. But that also makes it very simple to replace the firmware with the leJOS
environment. The first step toward enabling the brick to run Java programs is to load leJOS.

To actually execute the program, we simply push the brick's "start" button. But with a five-character LCD
display as the only output device, debugging programs can sometimes be challenging!

There are currently three versions of the RCX brick. Versions 1.0 and 1.5 are essentially the same, and
employ an infrared serial communications link to connect the PC to the brick (2400-baud). With the leJOS
environment, the Java Communications API can be used for communicating with the brick.

The RCX version 2.0 uses a USB communications port. And while leJOS can be installed onto the system,
there is no current support for the Java USB API (JSR-080). LeJOS now supports the USB connection on
 both Windows and Linux and includes a communications API that provides the same functionality as the
JavaComm API.

Innovation on Parade

The exciting innovations that have been created using Java technology-enabled LEGO robots is truly
impressive. In May of 2001, the Java technology evangelists at Sun staged a competition at Bay Area
universities to see who would come up with the most innovative LEGO brick robot. UC Santa Cruz turned
in the "SlugBot," a robot that played one-finger melodies on a small keyboard. Stanford produced the
"MazeBot," which solved a JSP (JavaServer Pages) generated maze. And UC Berkeley developed the
winning entry, the "PaperBot," a robot that selected and sorted paper by color.

UC Santa Cruz's Music UC Berkeley's Paper

Playing SlugBot Sorting PaperBot

: Stanford's Maze Solving

MazeBot
Robots, Ante Up!
Ritter demonstrated several of his LEGO robots at the 2002 Sun Network Conference in San Francisco.
One of them, a LEGO brick on wheels, was programmed to detect and follow an irregular black trail on a
field of white. "One thread detects events," says Ritter, "and the other thread controls the motors and
figure out the directions."

Another of Ritter's "LEGO-bots" acts as a command-driven/speech synthesizer-enabled robot blackjack

dealer. The system recognizes commands, vocalizes activities, and visually interprets dealt hands.
 With the limited processing constraints of the LEGO brick,
for many of Ritter's demonstration robots, it's not feasible to
have all of the functionality located entirely on the brick. On
the other hand, the brick is being utilized as far more than
just a PC-controlled set of motors. "Obviously, the vision
side of things is done on the PC" .
Simon Ritter's Demonstration
LEGO Robot And the same is true of the blackjack dealer robot. While the
speech synthesis and visual processing is being handled on the PC, the brick is handling the movement of
all the motors, figuring out where the card is in the machine, and so forth.

The Future
And most recently, Sun Labs' James Gosling has been consulting with scientists and engineers at NASA and
JPL, with the aim of incorporating Java technology into upcoming Mars rover missions. The red planet is
over twenty light-minutes from Earth. And as a result, even at the speed of light (and radio waves), it takes
over twenty minutes to get a message to a planetary rover, and then another twenty minutes to confirm the
receipt (and implementation) of that message. Such distances don't make for effective real-time control of a
vehicle, so such planetary exploration robots must make many decisions on their own. The next generation
of the software for the Mars rover was originally designed and built using C++, but Gosling is working with
NASA and JPL to create a Java version of the code that will be sent with the rover on the 2009 Mars
mission.

Mars Rover Mars

Surface of Mars
Conclusion
"The Java Speech API is a very nice, low-cost way of adding voice control, voice recognition, and
voice synthesis to the applications," says Ritter. "And the Java Media Framework API is a very
useful and very powerful set of technologies that allow you to get information from a webcam and
do processing on it. And then you have leJOS, which is a great open source project offering the
power and simplicity of Java technology to control robots."

When it comes to Java technology and robotics, seemingly even the sky is no longer the limit.

References
Java Speech API Home Page
Java Speech Grammar Format Specification
Java Speech API Markup Language Specification
Java Speech API JSR-113
Java Communications API
Java Media Framework API 2.1.1
Java/TWAIN Interface
Java 2D API Home Page
Java Advanced Imaging API Home Page
IBM's Speech for Java (Implementation of the Java Speech API)
IBM ViaVoice
Java/TWAIN Interface
LEGO Mindstorms Home Page
leJOS Home Page
leJOS Programming Books
LEGO Mindstorms Books:

The Unofficial Guide to LEGO MINDSTORMS Robots

LEGO MINDSTORMS Programming: Unleash the Power of the Java Platform
Programming Lego Mindstorms with Java
Machine Vision Algorithms in Java Technology and Applications