CIS 849: Autonomous Robot Vision Instructor: Christopher Rasmussen

CIS 849: Autonomous Robot Vision Instructor: Christopher Rasmussen

CIS 849: Autonomous Robot Vision Instructor: Christopher Rasmussen Course web page: September 5, 2002 Purpose of this Course To provide an introduction to the

uses of visual sensing for mobile robotic tasks, and a survey of the mathematical and algorithmic problems that recur in its application What are Autonomous Robots? Mobile machines with power, sensing, and computing on-board

Environments Land (on and under) Water (ditto) Air

Space ??? What Can/Will Robots Do? Near-term: What People Want Tool analogy Never too far from human intervention, whether physically or via tele-operation Narrow tasks, limited skills

3-D: Dirty, Dangerous, and Dull jobs What Can/Will Robots Do? Task Areas Industry Transportation & Surveillance Search & Science Service What Might Robots Do?

Long-term: What They Want Mechanical animal analogy may become appropriate Science fiction paradigm On their own Self-directed generalists Industry Ground coverage Harvesting, lawn-mowing (CMU)

Snow removal Mine detection Inspection of other topologies MAKRO MAKRO (Fraunhofer): Sewer pipes CIMP (CMU): Aircraft skin CIMP

CMU Demeter Transportation & Surveillance: Ground Indoors Clodbusters (Penn) Many others Highways, city streets

VaMoRs/VaMP (UBM) NAVLAB/RALPH (CMU) StereoDrive (Berkeley) VaMoRs Off-road Ranger (CMU) Demo III (NIST, et al.)

Ranger Penn Clodbuster Obstacle avoidance with omnidirectional camera UBM VaMoRs Detecting a ditch with stereo, then stopping

Transportation & Surveillance: Air Fixed wing (UBM, Florida) Helicopters (CMU, Berkeley, USC, Linkoping) Blimp (IST, Penn) UBM autonomous landing aircraft

Florida MAV USC Avatar Landing on target (mostly) Search & Science Urban Search & Rescue Debris, stairs Combination of autonomy

& tele-operation Dante II Hazardous data collection Dante II (CMU) Sojourner (NASA) Narval (IST) Narval

Sojourner USF at the WTC courtesy of CRASAR Urbot & Packbot reconnoiter surrounding structures

Service Grace (CMU, Swarthmore, et al.): Attended AI conference Register, interact with other participants Navigate halls, ride elevator Guides Polly (MIT): AI lab Minerva (CMU): Museum

Personal assistants Nursebot (CMU): Eldercare Robotic wheelchairs Grace CMU Minerva In the Smithsonian

What Skills Do Robots Need? Identification: What/who is that? Object detection, recognition Movement: How do I move safely? Obstacle avoidance, homing Manipulation: How do I change that?

Interacting with objects/environment Navigation: Where am I? Mapping, localization Why Vision? Pluses Rich stream of complex information about the environment Primary human sense

Good cameras are fairly cheap Passive stealthy Minuses Line of sight only Passive Dependent on ambient illumination Arent There Other Important Senses? Yes

The rest of the human big five (hearing, touch, taste, smell) Temperature, acceleration, GPS, etc. Active sensing: Sonar, ladar, radar But Mathematically, many other sensing problems have close visual correlates The Vision Problem

How to infer salient properties of 3-D world from time-varying 2-D image projection Computer Vision Outline Image formation Image processing Motion & Estimation Classification

Outline: Image Formation 3-D geometry Physics of light Camera properties Focal length Distortion Sampling issues Spatial Temporal

Outline: Image Processing Filtering Edge Color

Shape Texture Feature detection Pattern comparison Outline: Motion & Estimation Computing temporal image change Magnitude

Direction Fitting parameters to data Static Dynamic (e.g., tracking) Applications Motion Compensation Structure from Motion

Outline: Classification Categorization Assignment to known groups Clustering Inference of group existence from data Special case: Segmentation Visual Skills: Identification

Recognizing face/body/structure: Who/what do I see? Use shape, color, pattern, other static attributes to distinguish from background, other hypotheses Gesture/activity: What is it doing? From low-level motion detection & tracking to categorizing high-level temporal patterns

Feedback between static and dynamic Minerva Face Detection Finding people to interact with Penn MARS project Blimp, Clodbusters

Airborne, color-based tracking Visual Skills: Movement Steering, foot placement or landing spot for entire vehicle MAKRO sewer shape pattern Demeter region

boundary detection Florida Micro Air Vehicle (MAV) Horizon detection for self-stabilization UBM Lane & vehicle tracking (with radar)

Visual Skills: Manipulation Moving other things Grasping: Door opener (KTH) Pushing, digging, cranes KTH robot & typical handle Clodbusters push a box

cooperatively Visual Skills: Navigation Building a map [show 3D.avi] Localization/place recognition Where are you in the map? Laser-based wall map (CMU) Minervas ceiling map

Course Prerequisites Strong background in/comfort with: Linear algebra Multi-variable calculus Statistics, probability Ability to program in: C/C++, Matlab, or equivalent Course Details

First 1/3 of classes: Computer vision review by professor Last 2/3 of classes: Paper presentations, discussions led by students One major programming project Grading

10%: 30%: 10%: 50%: Two small programming assignments Two oral paper presentations + write-ups Class participation Project

Readings All readings will be available online as PDF files Textbook: Selected chapters from prepublication draft of Computer Vision: A Modern Approach, by D. Forsyth and J. Ponce Web page has other online vision resources Papers: Recent conference and journal articles spanning a range of robot types, tasks, and visual algorithms

Presentations Each student will submit short written analyses of two papers, get feedback, then present them orally Non-presenting students should read papers ahead of time and have some questions prepared. I will have questions, too :) Project

Opportunity to implement, test, or extend a robot-related visual algorithm Project proposal due in October; discuss with me beforehand Data I will provide canned data, or gather your own We will have a small wheeled robot to use for algorithms requiring live feedback Due Wednesday, November 27 (just before

Thanksgiving break) More questions? Everything should be on the web page: or e-mail me at [email protected]

Recently Viewed Presentations

  • Rate Update -

    Rate Update -

    For demographic elements with no alternative data source or Social Determinate identifier, AHCCCS will create an online portal to be accessed directly by behavioral health providers for the collection of the data elements for members. Collected data will then be...
  • Energy Resources - Lower Dauphin School District

    Energy Resources - Lower Dauphin School District

    are energy-rich chemical compounds that contain carbon and hydrogen atoms. During combustion, the carbon and hydrogen combine with oxygen in the air to form carbon dioxide and water. This process releases energy in the forms of heat and light.
  • Transforming the Future with PSO Data Intended Outcomes

    Transforming the Future with PSO Data Intended Outcomes

    ODE Expectations. Teachers will have access to the PSO data results. 100 % of districts with leavers will complete their federal PSO collection. PSO collection is the required Data Collection and Reporting for the settlement
  • Indiana Department of Veterans Affairs

    Indiana Department of Veterans Affairs

    Ethics Training. All Veteran service officers and employees of that office should complete ethics training every year. This is many times not required this often, but in our line of work it is best to stay sharp and not become...
  • Regular Structures

    Regular Structures

    Reversible Logic Synthesis with Garbage Bits Lecture 6 Marek Perkowski Background The 3 * 3 Toffoli gate is described by these equations: P = A, Q = B, R = AB C, Toffoli gate is an example of two-through gates,...
  • Design of Genetical Genomics Studies Which Use Two-Color ...

    Design of Genetical Genomics Studies Which Use Two-Color ...

    Statistical Analysis and Design of Experiments for Large Data Sets Steven Gilmour School of Mathematical Sciences Centre for Statistics Introduction I will discuss microarrays, but there are many other possible biological applications Microarray experiments provide a measure of gene activity...
  • Chapter 22: Rise to World Power - Blacklick Valley School ...

    Chapter 22: Rise to World Power - Blacklick Valley School ...

    Japan ignored the Open Door Policy, which led to war with Russia and conflict with the United States. Clash over Manchuria (lots of natural resources) ... Policing the Western Hemisphere. Main idea: Using its economic and military power, the US...
  • Review


    randnum = Math.floor(Math.random() * imgArray.length) what happens when we click on the image with the id "pic2 "? <script type = "text/javascript">function ChangeIt(idholder){