Mtrx 5700 : Experimental Robotics

Introduction

Dr. Viorela Ila

 Course Objectives
• The objective of the course is to provide students with the
essential skills necessary to develop robotic systems for
practical applications.

• Robotics is “the coolest mechatronics playground!”

 Administrative Details
• Lecturers : Viorela Ila, Yiduo Wang, Stefan Williams, Suraj Bijjahalli
• Lecture Time : Tuesdays 9-11
• Tutorials : Fridays, 9-12 or 2-5
• Lectures, Assignments and Previous Exams will be available on Canvas
site
• Contact Details :
– E-mail : [email protected], [email protected],
[email protected] or [email protected]
– ED STEM is the main communication channel!
 Tutors
Jack and Jessie

Jack Naylor Jesse Morris

[email protected] [email protected]
Course Outline
 Room Week Date Topics Lecturer Assignments
LT 1050 Tue 21 Feb 2023 Lecture Introduction to robotics. Viorela A1 - released
MXLAB 1 Fri 24 Feb 2023 Lab Lab Setup
LT 1050 Tue 28 Feb 2023 Lecture Robot Kinematics and Dynamics Vio/Stefan
2
MXLAB Fri 03 Mar 2023 Lab Kinematics and Dynamics
LT 1050 Tue 07 Mar 2023 Lecture Sensors, Measurements and Perception Yiduo
3
MXLAB Fri 10 Mar 2023 Lab Kinematics and Dynamics - Demo A1 - due (2 weeks)
LT 1050 Tue 14 Mar 2023 Lecture Robot Vision and Vision Processing Yiduo A2 - released
4
MXLAB Fri 17 Mar 2023 Lab Perception
LT 1050 Tue 21 Mar 2023 Lecture Robot Learning Suraj
5
MXLAB Fri 24 Mar 2023 Lab Perception
LT 1050 Tue 28 Mar 2023 Lecture Navigation and Localization Viorela A3 - released
6
MXLAB Fri 31 Mar 2023 Lab Perception and SLAM A2 - due (3 weeks)
LT 1050 7 Tue 04 Apr 2023 Lecture Estimation and data fusion Viorela Final Project -released

Fri 07 Apr – Fri 14 Apr 2023 Easter Break

LT 1050 Tue 18 Apr 2023 Lecture Estimation and SLAM Viorela
MXLAB 8 Fri 21 Apr 2023 Lab Lab Robot Perception and SLAM
LT 1050 Tue 25 Apr 2023 Lecture ANZAC DAY
9
MXLAB Fri 28 Apr 2023 Lab Lab - Robot Perception and SLAM - Demo
LT 1050 Tue 02 May 2023 Lecture Major Project - Pitch ALL
10
MXLAB Fri 05 May 2023 Lab Major Project A3 - due (3 weeks ++)
LT 1050 Tue 09 May 2023 Lecture Obstacle Avoidance and Path Planning Viorela
11
MXLAB Fri 12 May 2023 Lab Major Project
LT 1050 Tue 16 May 2023 Lecture Robot Archirectures / Case Study Viorela
12
MXLAB Fri 19 May 2023 Lab Major Project
LT 1050 Tue 16 May 2023 Lecture Robot Archirectures / Case Study Vio/ Guest
12
MXLAB Fri 19 May 2023 Lab Major Project - Demo
 Recommended Texts
Manipulator Kinematics and Dynamics
• John J. Craig, Introduction to Robotics: Mechanics and Control, 3rd Edition, Prentice-Hall, 2003
• Lorenzo Sciavicco, Bruno Siciliano, Modelling and Control of Robot Manipulators (Advanced
Textbooks in Control and Signal Processing), Springer 2000
• Mark W. Spong, M. Vidyasagar, Robot Dynamics and Control, Wiley, 1989
Computer Vision
• Ballard and Brown, Computer Vision, Prentice Hall, 1982
• David A. Forsyth and Jean Ponce, Computer Vision -- A Modern Approach, Prentice Hall, 2002
Machine Learning
• Christopher Bishop, Pattern Recognition and Machine Learning, Springer , 2006
• Stuart J. Russell and Peter Norvig, Artificial Intelligence, A Modern Approach, 2nd Edition,
Prentice Hall, 2002
Mobile Robotics
• Sebastian Thrun, Dieter Fox and Wolfram Burgard, Probabilistic Robotics, The MIT Press, 2005
• Greg Dudek and Michael Jenkin, Computational Principles of Mobile Robotics, Cambridge
University Press, 2000
• Roland Siegwart, Illah R. Nourbakhsh, Davide Scaramuzza, Introduction to Autonomous Mobile
Robots (Intelligent Robotics and Autonomous Agents), 2nd edition. ETH Zurich, MIT Press
 Assessment
• Introductory Labs (40%)
– Assignment 1 - Kinematics Lab: Due Week 3 (10%)
– Assignment 2 - Perception Lab: Due Week 6 (15%)
– Assignment 3 - Navigation Lab: Due Week 10 (15%)
• Major Project Presentation and Report (30%): Due STUVAC
• Exam (30%) – In Person, supervised
 Learning Outcomes
Following completion of this UoS students will:
• Be familiar with sensor technologies relevant to robotic systems
• Understand conventions used in robot kinematics and dynamics
• Understand the dynamics of mobile robotic systems and how they are modelled
• Have implemented navigation, sensing and control algorithms on a practical robotic system
• Apply a systematic approach to the design process for robotic systems
• Understand the practical application of robotic systems in applications such as manufacturing,
automobile systems and assembly systems
• Develop the capacity to think creatively and independently about new design problems
• Undertake independent research and analysis and to think creatively about engineering
problems
 Learning Outcomes
• Theory: Knowledge-based learning
– Learning basic concepts and mathematical models for mobile robot locomotion,
kinematics, motion control, path planning, navigation, sensing and perception (vision),
learning, localization, mapping, SLAM.

• Practice: hardware lab and project based learning

Not recommended
– Program intelligent behaviours on a physical robotic platforms:

• TurtleBot-burger
• UR5
• Tello
• Baxter

• Learning methods: attending lectures, attending tutorials, doing labs and project, prepare
labs and project demo and report.
 Labs are to be done in groups
• 4 students per group (some groups of 5 permitted if needed):
– We are back in the lab!
– The assignments can be implemented on simulated environment and/or we will provide data to
test your code.
– Translate your project to real hardware (Ask tutors and staff how to access the hardware)
– When working in the lab, respect lab rules
• Labs and Project work and report are group assignments! Work in the lab
collaboratively and write the reports as collaborative papers.

• The Exam is an individual task!!! We will carefully check for plagiarism

 Working in Groups
• Establish communication channels (in person, zoom, discord, slack)
• Projects will be challenging, divide the tasks properly, share responsibilities
• Plan your work on time!
• Keep record of the progress and results
• Establish code style rules
• Backup your code!
 What is a Robot?

• Robot (a Slavic word for worker) was first

introduced in 1921 in a play, Rossum's
Universal Robots, by the Czech playwright,
Karel Čapek.

• A traditional definition of a robot is a

programmable multi-function manipulator
designed to move material, parts, or
specialized devices through variable
programmed motions for the performance of a
variety of tasks.
 • Wikipedia: a robot is a mechanical intelligent agent
which can perform a wide range of physical tasks on its
own, or with guidance.

• In practice a robot is usually an electro-mechanical

machine performing sufficiently complex tasks, guided
by a computer and/or electronic programming.
What is a • Robots can be autonomous (intelligent) or semi-
Robot? autonomous (programmable and controllable).

• It is a mechanical physical device capable of

completing intelligent tasks based on a robust system
of actuators and sensors that both drive the
mechanical device and sense its state and
environment.
 Robotics is a multi-disciplinary field
Mechanical Engineering
• Design of the mechanism, kinematics and dynamics of the system.
Electrical/Electronic Engineering, information engineering
• Design of the actuator and communication system.
Computer Science and engineering
• Computer Vision and Artificial Intelligence.
Systems Engineering
• System analysis and integration.
• Signal conditioning and Control.
 Components of a Robot

Component Discipline
Manipulator/Rover, End effector Mechanical design
Sensors/Power system Electrical Design
Perception/Control/Intelligence Artificial intelligence, Computer Vision
Software, logic Computing Systems
Communication, interconnectivity Electronic Engineering; Tele
communication
 Applications
• Industrial robots (manipulator)

• Mobile robots (land, underwater, aerial)

• Service robots (medical, household, personal)

From Industrial Robots…
To the future robots.

Slide 19
Australian Centre for
Field Robotics
 Australian Centre for Field Robotics

• More than twenty years ago the University of Sydney’s

Australian Centre for Field Robotics was established to
undertake innovative research and training in the autonomous
systems field, to engage with industry in the commercial
development of autonomous systems and to deliver the
benefits of these systems to society
 Australian Centre for Field Robotics
• An engineering research institute at the University of Sydney
• Research staff
– 10 academics
– 40 research fellows
– 40 PhD students
– 30 software, mech/aero, electrical/electronic staff
• One of the largest field robotics and intelligent systems group in
the world
• Dedicated to the scientific advancement and industry uptake of
autonomous robots and intelligent systems for outdoor
operations.
• Annual budget of $13M with 80% sourced directly from industry
projects
Government and Industry Support:
 Intelligent Transport Systems
• Active areas of research
– High integrity navigation, planning and control
– Modeling cars and pederstrians intent
– Collision prediction under uncertainty
– Increasing situational awareness
• Application Areas
– Stevedoring
– Autonomous driving
– Mining (RTCMA)
– Special forces training
• iMove Cooperative Research Centre
– A $100M initiative to study future transportation systems
https://www.youtube.com/watch?v=t8l-oOInbeA
https://www.youtube.com/watch?v=9l_EbGLItyY
https://www.youtube.com/watch?v=Li9eWpLGFiU
 Aerospace and Defence
• Fundamental contributions in high integrity navigation, data
fusion and control
• Applications in
– Broad-area surveillance
– Search and rescue
– Air defence
• Partners
– DST-Group
– BAE Systems, ST Aerospace
– US Air Force, Ministry of Defence UK, US Office of Naval Research
– Department of Agriculture, Fisheries, and Forestry
– Land and Water Australia
– Australian Plague Locust Commission
– Meat and Livestock Australia, QLD Biosecurity
 Autonomous UAV Docking

https://www.youtube.com/watch?v=punZ7ONvcMg

Wilson, D. B., Göktogan, A. H.,

& Sukkarieh, S. “Guidance and
Navigation for UAV Airborne
Docking”., Robotics: Science
and Systems, 2015 (winner Best
Paper)
 Logistics (Aerospace)
• Partnership with Qantas to develop next generation flight planning and
logistics systems
• Applying fundamental research methods to
– Develop a transformational platform for fuel optimisation and operational
efficiencies
– Flight planning
– Fuel optimisation using real-time weather information
– Predictive maintenance
• Qantas spend $4B on fuel – 0.5% fuel savings on mainline operations
 Marine Robotics
120° E 150° E

• Navigation and mapping: SLAM using both visual and Scott Reef (2009, 2011)

acoustic data, visualisation

GBR Ribbon Reefs (2007)
15° S
GBR Noggin Reef (2007)
GBR Lagoon Seagrass GBR Myrmiddon Reef (2011, 2012)
(2011,2012, 2013) GBR Viper Reef (2007)
GBR Hydrographers Passage (2007
GBR Inshore Reefs (2013)
Ningaloo (2007, 2012)

• Machine Learning: automated processing of large

WA Abrolhos (2010, Moreton Island (2010, 2011,
2011, 2012, 2013, 2014) 2012, 2013)

30° S WA Jurien (2010, 2011, Solitary Islands (2012)

2012, 2013)
WA Rottnest (2010, Port Stephens (2010, 2012, 201
2011, 2012, 2013) SA Whyalla (2008)
Sydney (2012)

volumes of data
SA Sir Joseph Banks MPA (2008)
Batemans Bay (2010, 2012, 2014)
Legend

coral < 50000

kelp 50000 − 100000
coral/kelp Flinders CMR (2011, 2013)
seagrass/algae 100000 − 500000 Freycinet (2009, 2010, 2011,
sponge 2012, 2013, 2014)
rocky reef > 500000 Maatsuyker Island (2015) Tasman Peninsula (2008, 2009, 2010,
canyon Huon (2009, 2010, 2012, 2014) 2011, 2012, 2014)

• Novel sensing: high-resolution stereo imaging,

45° S

hyperspectral, lightfield imaging

• Planning: Optimisation of survey design to maximise April 2010 April 2011

expected information gain

• Change detection: multi-year change in marine
habitats April 2012 April 2013

• Applications: Ecology, archaeology and geoscience

research
 EV Nautillus (Med),
URI/OET/ACFR, 2010-2012

Deepwater Horizon, Pavlopetri,

WHOI/ACFR, 2010 Nottingham/ACFR, Fukushima,
2010/2011 UTokyo/ACFR,
NOAA, Sicily, 2014
Umich/Nottingham/ACFR, RPM/ACFR, Antikythera, Okinawa Reefs,
2015 2011/2013 WHOI/Argo/ACFR, U Tokyo, Southampton, Artificial Hydrothermal,
Methane hydrates, ACFR, 2017 UTokyo/ACFR, 2014
2014/2015
WHOI/ACFR,
2011/2013 EV Nautillus (Caribbean), Hawaiian Reefs,
URI/OET/ACFR, ACFR/SOI/WHOI/MIT/URI/UMich,
Liquid Jungle, 2013/2014 Hawaii, 2018
ACFR/WHOI, Lizard Island, St
Panama, 2007 Andrews/UMacQ/
ACFR, 2013-
Ecology 2015
Scott Reef,
SOI/WHOI/URI/UH/AC
FR, 2015
Archaeology

Geoscience
IMOS AUV
Facility
2007-2017
 Logistics of Multivehicle Operations

Glider (WHOI) Logitics (All) AUVs (Umich/Usyd)

Imaging Float (URI) USV (USyd)

IMOS AUV Facility – Archive of 5M seafloor images available online
Viorela Ila x
x xx
x
Engineering Degree
• Technical University of Cluj-Napoca, Romania (Oct. 2000)
PhD in Information Technology
x
• University of Girona, Spain (Nov. 2005)

Industry
• R&D Engineer at Icnita S.L., Girona, Spain (May 2005 – April 2006)

Research
• Post-doctoral research at IRI, Barcelona, Spain (May 2006 – April 2009)
• Post-doctoral research at Georgia Tech, Atlanta, USA (May 2009 – Sept. 2010)
• Research scientist at LAAS-CNRS, Toulouse, France (Oct. 2010 – Dec. 2011)
• Research scientist at Brno University of Technology, Brno, CZ (Feb. 2012 – Jan.2015)
• ACRV research fellow, ANU, Canberra, Australia (Feb. 2015 – Dec.2018)
• The University of Sydney (Jan. 2019 – Present)
40
 University of Girona, Spain (UdG)

Underwater Robotics Image Mosaic Hardware Implementation

Institut de Robòtica i Informàtica industrial, Spain (IRI)

50

40

30

y(m)
20

10

20 15 10 5 0 5 10 15 20 25 30
Urban Robotics x(m)
Simultaneous Localization and Mapping (SLAM)
41
Georgia Tech, USA

Graphs Matrices Bayes Tree

LAAS, Toulouse, France and BOT project

x
x 2
2
1
0

x
x 1
x
1 1
2
0 x 3
1
1
Multirobot SLAM Underwater Positioning System 42
 Brno University of Technology, Czech Republic

block matrices acceleration

SLAM ++ High Performance Nonlinear Least Squares Library
ACRV – Australian National University

Active Mapping Dynamic Mapping

Jigsaw: Multi-Modal Data Management for Digital Film Production

44
Application to Underwater 3D Reconstruction
Dynamic Scenes
International Developments in Robotics
 Automating
Surgical Subtasks
Biomimetic Robots

Slide 55
Many More Robot Applications

Slide 57
MTRX5700: Major Project
MTRX5700: Major Project
 MTRX5700: Major Project
• https://www.youtube.com/channel/UCD-
V1Qf4ep_NDZRIK22sDUA
 ROS (Robot Operating System)
• Developed at Willow Garage (CA, USA) Originated at Stanford AI Lab
(Sachin Chitta and Radu Bogdan Rusu)

• Implements many robot algorithms

• ROS is a distributed framework of processes (a.k.a. Nodes) that

enables executables to be individually designed and loosely coupled at
runtime.

• BSD license (open source) support code reuse in robotics research and
development.

• Now ubiquitous in practice and easy to integrate with other robot

software frameworks.
ROS:
 What is ROS?

ROS (Robot Operating System)

 ROS Package System

1. Packages contain a bunch of executable programs that use ROS

2. Stacks contain a bunch of related packages: For example everything ROS related with turtlebots.
3. Repository is the server a bunch of stacks are stored
4. Universe is everything ROS related
 Packages
• Software in ROS is organized in packages
• A package is a folder that contains your code, build files, launch files, etc.
• Can contain any number of nodes (individual executables)
• ‘package.xml' – lists the ROS dependencies & system dependencies
• Should only contain code that is related to the nodes stored in the package

• Examples:
– turtlebot_bringup – initialise turtlebots
– gmapping – ROS SLAM package
– quad_velocity_control – custom package designed by to control the velocity of
a quadrotor
How ROS Packages are formatted

src / scripts launch

Contains all source code.
Contains all ROS launch
This includes C++ .cpp
files
and Python .py files

includes package.xml
Contains all header files
(eg. .h files for C++). CMakeLists.txt
 Nodes
• Single-purposed executable programs
– e.g. sensor driver(s), actuator driver(s), mapper, planner, UI, etc.
• Modular design
– Individually compiled, executed, and managed
• Written with the use of a ROS client library
• Nodes communicate with each other using topics & services.
• Executed with:
$ rosrun <package name> <node name> [<arg:=value>]
 Messages
• Nodes communicate by passing around messages
• A data structure with typed fields
• Many pre-defined messages exist
• Custom messages can be made in a text file (.msg extension)
• Message can also contain other messages, as part of the
message
• For example, geometry_msgs/Pose.msg
 Ways to communicate
• Topic
– A topic is a messaging method that sends data to all nodes requiring
information, as soon as a node updates that message.
• Service
– A service is a special messaging method, that will give information when a node
calls for the information.
– Services can also perform other roles:
• carry out remote computation
• trigger functionality / behavior
 Launch file
• Used to execute multiple nodes at once, or even other launch files
• Can put parameters on ROS system
– eg. address to camera data
• Written in XML
• File extension is ‘.launch’
• Executed with:
$ roslaunch <package name> <file.launch> [<arg:=value>]
 Robots that are using ROS
Full list at http://wiki.ros.org/Robots

etc.
 Conclusions
• Robotic systems are playing an increasingly important, and
diverse, role in our society
• The study of robotics involves an integration of a number of
different areas including hardware, electronic and software
• Lots of exciting applications are driving developments in this
field