Intelligent mobile Robotics & Perception SystemsIntelligent mobile Robotics & Perception Systems

Editor's Notes

• #2: Today we gonna see the intelligent mobile robotic and perception System .
• #3: In the overview of this PPT . We will start with simple introduction and definition to Robot hardware (for move and perception) then the Environment representation then ……
• #4: The definition of robotics may not be very clear till today, but they are indeed our future. Not everyone is so interested in the prospect of a radical change that robots are about to bring, but we are embracing the technology and working out how to best collaborate with it. Robots are not new – in fact, we’ve been seeing them since the ever popular Terminator movies as evil assassin robots.  Artificial Intelligence (AI), on the other hand, is the next generation of robotics that involves intelligent machines that work and react like humans. It is often called as machine intelligence and has been around for quite some time now.
• #5: Robots are programmable machines specifically programmed to carry out a complex series of tasks without any human intervention. robots are becoming more capable and more diverse than ever. Robots are often characterized by their capabilities in performing dull to dangerous tasks, easily and without needing humans to perform them.
• #6: Most people would think robots and artificial intelligence (AI) are one and the same, but they are very different terms associated with different fields. Robots are hardware and AI is software. In technical terms, robots are machines designed to execute one or more simple to complex tasks automatically with utmost speed and precision, whereas AI is like a computer program that typically demonstrates some of the behaviors associated with human intelligence like learning, planning, reasoning, knowledge sharing, problem solving, and more.
• #7: Artificial Intelligence and Machine Learning in the Industrial Robotics Application Artificial intelligence (AI) and machine learning capabilities have been quickly making their way into industrial robotics technology. In the never-ending quest to improve productivity, manufacturers are looking to improve the inflexible capabilities of standard industrial robots. The merging of robotics and AI technology has several consequences, and early adopters of these new robotic systems are reaping the benefits. The technology, while relatively new, is widely available and impacts manufacturing processes in a number of ways.
• #8: And for robots types we have 3 types :
• #9: For the hybrid robots it’s contain both manipulators and mobile robots
• #10: Now going to the robot hardware what exactly the things making the robot act and see like human
• #11: 1- As the man have receptors the robot are equipped with effectors & Actuators. * * * and all of that called sensors
• #12: now more about effectors
• #13: 1- And an actuator is the actual mechanism that enables the effector to execute an action
• #14: 1-So the sensors is a device that detects and responds to some type of input from the physical environment. 2- Examples of sensors:
• #15: We already explain what is sensors .because nowadays The robot are equipped with sensors Passive sensors Active sensors
• #16: This is an example of true Passive sensors
• #17: And This is an example of Active sensors who Send energy into the environment, like sonars.
• #18: And for each mobile robot who have sensor and some of them also have some physical mechanism to make it able to move . Those are the most used vehicle designs in robotic
• #19: If we talked about move we must know the degrees of freedom about this kind of movement . And As a general meaning it is How many parameters needed to specify something?
• #20: Degree of freedom counts one for each independent direction of movement. 6 degrees of freedom are required to place an object at a particular orientation.
• #21: Degree of freedom is
• #23: Those some real example for mobile robots This for ‘Car’- steer and drive
• #24: And this contain many wheels each one can drive
• #25: This is very famous kind of wheels for robots today
• #26: 1-And now to the important chapter of the robot locomotion 2-Robot locomotion is the collective name for the various methods that robots use to transport themselves from place to place. -tires robots are typically quite energy efficient and simple to control. However, other forms of locomotion may be more appropriate for a number of reasons, for example traversing rough terrain, as well as moving and interacting in human environments.. -A major goal in this field is in developing capabilities for robots to autonomously decide how, when, and where to move. However, coordinating a large number of robot joints for even simple matters, like negotiating stairs, is difficult. Types of locomotion 1.1 Walking 1.2 Bipedal walking 1.3 Running 1.4 Rolling 1.5 Hopping 1.6 Metachronal motion 1.7 Slithering 1.8 Swimming 1.9 Brachiating 1.10 Hybrid
• #27: Just as example of walking robot
• #29: 1-The evolution of robotic sensors Historically, robotic sensors have become richer and richer We will see those exemples And the reasons for this advancement is the : Commodization of consumer electronics More computation available to process the data
• #33: And then The SmartTer Platform are more modern then all the others with very hight revolution in camera with life video streaming and sonsors And also the localization and estimation
• #34: Multimodal detection and tracking system Using deep learning algorithm to obtain The ability to detect people in real-world environments is crucial for a wide variety of applications including video surveillance and intelligent driver assistance systems. The detection of pedestrians is the next logical step after the development of a successful navigation and obstacle avoidance algorithm for urban environments. However, pedestrians are particularly difficult to detect because of their high variability in appearance due to clothing, illumination and the fact that the shape characteristics depend on the viewpoint And Using deep learning algorithm we can detect those and giving the meaning for the picture toked by the robot sensor
• #35: This is the most 3 kind of data used in Detection and tracking displayed Camera Prjlaser Laser
• #36: As more information about sensors outline we already said there is Optical encoders Heading sensors Compass Gyroscopes Accelerometer IMU GPS Range sensors Sonar Laser Structured light Vision (next lectures)
• #40: Vicon and Optitrack Is System of several cameras that track the position of reflective markers Indoor motion capture systems like VICON and Optitrack can be used to provide position and attitude data for vehicle state estimation, or to serve as ground-truth for analysis. The motion capture data can be used to update the local position estimate relative to the local origin. Heading (yaw) from the motion capture system can also be optionally integrated by the attitude estimator.
• #41: After the locomotion the perception is also important for robotics, -Perception is a process to interpret, acquire, select and then organize the sensory information that is captured from the real world. Also perception is understood as a system that provide the robot with the ability to perceive, comprehend, and reason about the surrounding environment. - For example: Human beings have sensory receptors such as touch, taste, smell. So, the information received from these receptors is transmitted to human brain to organize the received information. According to this information, action is taken by interacting with the environment to manipulate and navigate the objects. Perception and action are very important concepts in the field of Robotics. The following slides will show more details about it .
• #42: So . Perception technically is hard!
• #43: And Dr pinker said In robotics, the easy problems are hard and the hard problems are easy
• #44: This is how the process going on Firstly we need to collect data information in the data raw using vision, smell and the other sensors Then the navigation : the robot begin the treatment of this data to give a meaning to it . The features ( corners , lines , colours ) After that the interaction also objects recognize and then begin the servicing or reasoning to recognize the situations or places
• #45: The key components of a perception system are essentially sensory data processing, data representation (environment modeling), and ML-based algorithms. Since strong AI is still far from being achieved in real-world robotics applications, this chapter is about weak AI, i.e., standard machine learning approaches . And for this figure It’s The Key modules of a typical robotic perception system: The sensory data processing (focusing here on visual and range perception); data representations specific for the tasks at hand; algorithms for data analysis and interpretation (using AI/ML methods); and planning and execution of actions for robot-environment interaction. Robotic perception is conclusive for a robot to make decisions, plan, and operate in real-world environments, by means of numerous functionalities and operations from occupancy grid mapping to object detection.
• #46: Nowadays, most of robotic perception systems use machine learning (ML) techniques, ranging from classical to deep-learning approaches
• #47: 1-Machine learning for robotic perception can be in the form of unsupervised learning, or supervised classifiers using handcrafted features, or deep-learning neural networks . 2-Regardless of the ML approach considered, data from sensor(s) are the key ingredient in robotic perception. Data can come from a single or multiple sensors, usually mounted onboard the robot, but can also come from the infrastructure or from another robot .
• #48: Sensor-based environment representation/mapping is a very important part of a robotic perception system. Mapping here encompasses both the acquisition of a metric model and its semantic interpretation, and is therefore a synonym of environment/scene representation
• #49: Robot perception functions, like localization and navigation, are dependent on the environment where the robot operates. Essentially, a robot is designed to operate in two categories of environments: indoors or outdoors. Therefore, different assumptions can be incorporated in the mapping (representation) and perception systems considering indoor or outdoor environments. -Localization involves one question: Where is the robot now? Or, robo-localization , where am I, keeping in mind that "here" is relative to some landmark (usually the point of origin or the destination) and that you are never lost if you don't care where you are. -Navigation comprises everything a robot needs to get from point A to point B as efficient as possible without bumping into furniture, walls or people.
• #50: Among the numerous approaches used in environment representation for mobile robotics, and for autonomous robotic-vehicles, the most influential approach is the occupancy grid mapping
• #51: After collecting the data from environment now we need to choose model such as mapping; 2d-3d grids for the Environment representation step
• #52: This 2D mapping is still used in many mobile platforms due to its efficiency, probabilistic framework, and fast implementation. Although many approaches use 2D-based representations to model the real world, presently 2.5D and 3D representation models are becoming more common. The main reasons for using higher dimensional representations are essentially twofold: robots are demanded to navigate and make decisions in higher complex environments where 2D representations are insufficient; (2) current 3D sensor technologies are affordable and reliable, and therefore 3D environment representations became attainable.
• #53: Robotic mapping Mapping is the problem of integrating the information gathered with the robot’s sensors into a given representation. It can intuitively be described by the question “What does the world look like?” Central aspects in mapping are the representation of the environment and the interpretation of sensor data .
• #54: This semantic mapping process uses ML at various levels, e.g., reasoning on volumetric occupancy and occlusions, or identifying, describing, and matching optimally the local regions from different time-stamps/models . not only higher level interpretations. However, in the majority of applications, the primary role of environment mapping is to model data from exteroceptive sensors, mounted onboard the robot, in order to enable reasoning and inference regarding the real-world environment where the robot operates.
• #55: Machine learning is the science of pattern recognition and computational learning theory in artificial intelligence. Machine learning focuses on the construction and study of algorithms that learn from and make predictions on data. In order to ensure more robust robot perception in light of unpredictable, dynamic, and noisy real- world scenarios, scientists pair machine learning approaches with biologically-inspired sensor systems.
• #56: Now we will see some of project still on studies Like
• #57: The Strands project is formed by six universities and two industrial partners. The aim object of the project is to develop the next generation of intelligent mobile robots, capable of operating alongside humans for extended periods of time. While research into mobile robotic technology has been very active over the last few years , robotic systems that can operate robustly, for extended periods of time, in human-populated environments remain a rarity. Strands aims to fill this gap and to provide robots that are intelligent,
• #58: This figure shows a high level overview of the Strands system : the mobile robot navigates autonomously between a number of predefined waypoints. A task scheduling mechanism dictates when the robot should visit which waypoints, depending on the tasks the robot has to accomplish on any given day. The perception system consists, at the lowest level, of a module which builds local metric maps at the waypoints visited by the robot. These local maps are updated over time, as the robot revisits the same locations in the environment, and they are further used to segment out the dynamic objects from the static scene .
• #59: The AUTOCITS (https://www.autocits.eu/) project will carry out a comprehensive assessment of cooperative systems and autonomous driving by deploying real-world Pilots, and will study and review regulations related to automated and autonomous driving. AUTOCITS, cofinanced by the European Union through the Connecting Europe Facility (CEF) Program, aims to facilitate the deployment of autonomous vehicles in European roads, and to use connected/cooperative intelligent transport systems (C-ITS) services to share information between autonomous vehicles and infrastructure, by means of V2V and V2I communication technology, to improve safety and to facilitate the coexistence of autonomous cars in real-world traffic conditions. The AUTOCITS Pilots, involving connected and autonomous vehicles (including autonomous shuttles, i.e., low-speed robot-vehicles), will be deployed in three major European cities in “the Atlantic Corridor of the European Network”: Lisbon (Portugal), Madrid (Spain), and Paris (France).
• #60: So just how capable is current perception and AI, and how close did/can it get to human-level performance? DR zeliski in his introductory book to computer vision said that traditional vision struggled to reach the performance of a 2-year old child, but today’s CNNs reach super-human classification performance on restricted domains The recent surge and interest in deep-learning methods for perception has greatly improved performance in a variety of tasks such as object detection, recognition, semantic segmentation, etc. One of the main reasons for these advancements is that working on perception systems lends itself easily to offline experimentation on publicly available datasets, and comparison to other methods via standard benchmarks and competitions. *** the end ***** Machine learning (ML) and deep learning (DL), the latter has been one of the most used keywords in some conferences in robotics recently, are consolidated topics embraced by the robotics community nowadays. While one can interpret the filters of CNNs as Gabor filters and assume to be analogous to functions of the visual cortex, currently, deep learning is a purely nonsymbolic approach to AI/ML, and thus not expected to produce “strong” AI/ML. However, even at the current level, its usefulness is undeniable, and perhaps, the most eloquent example comes from the world of autonomous driving which brings together the robotics and the computer vision community. A number of other robotics-related products are starting to be commercially available for increasingly complex tasks such as visual question and answering systems, video captioning and activity recognition, large-scale human detection and tracking in videos, or anomaly detection in images for factory automation.
• #61: So just how capable is current perception and AI, and how close did/can it get to human-level performance? DR zeliski in his introductory book to computer vision said that traditional vision struggled to reach the performance of a 2-year old child, but today’s CNNs reach super-human classification performance on restricted domains The recent surge and interest in deep-learning methods for perception has greatly improved performance in a variety of tasks such as object detection, recognition, semantic segmentation, etc. One of the main reasons for these advancements is that working on perception systems lends itself easily to offline experimentation on publicly available datasets, and comparison to other methods via standard benchmarks and competitions. *** the end ***** Machine learning (ML) and deep learning (DL), the latter has been one of the most used keywords in some conferences in robotics recently, are consolidated topics embraced by the robotics community nowadays. While one can interpret the filters of CNNs as Gabor filters and assume to be analogous to functions of the visual cortex, currently, deep learning is a purely non symbolic approach to AI/ML, and thus not expected to produce “strong” AI/ML. However, even at the current level, its usefulness is undeniable, and perhaps, the most eloquent example comes from the world of autonomous driving which brings together the robotics and the computer vision community. A number of other robotics-related products are starting to be commercially available for increasingly complex tasks such as visual question and answering systems, video captioning and activity recognition, large-scale human detection and tracking in videos, or anomaly detection in images for factory automation.