CIS 849: Autonomous Robot Vision Instructor: Christopher Rasmussen
CIS 849: Autonomous Robot Vision Instructor: Christopher Rasmussen Course web page: www.cis.udel.edu/~cer/arv September 5, 2002 Purpose of this Course To provide an introduction to the
uses of visual sensing for mobile robotic tasks, and a survey of the mathematical and algorithmic problems that recur in its application What are Autonomous Robots? Mobile machines with power, sensing, and computing on-board
Environments Land (on and under) Water (ditto) Air
Space ??? What Can/Will Robots Do? Near-term: What People Want Tool analogy Never too far from human intervention, whether physically or via tele-operation Narrow tasks, limited skills
3-D: Dirty, Dangerous, and Dull jobs What Can/Will Robots Do? Task Areas Industry Transportation & Surveillance Search & Science Service What Might Robots Do?
Long-term: What They Want Mechanical animal analogy may become appropriate Science fiction paradigm On their own Self-directed generalists Industry Ground coverage Harvesting, lawn-mowing (CMU)
Snow removal Mine detection Inspection of other topologies MAKRO MAKRO (Fraunhofer): Sewer pipes CIMP (CMU): Aircraft skin CIMP
CMU Demeter Transportation & Surveillance: Ground Indoors Clodbusters (Penn) Many others Highways, city streets
VaMoRs/VaMP (UBM) NAVLAB/RALPH (CMU) StereoDrive (Berkeley) VaMoRs Off-road Ranger (CMU) Demo III (NIST, et al.)
Ranger Penn Clodbuster Obstacle avoidance with omnidirectional camera UBM VaMoRs Detecting a ditch with stereo, then stopping
Florida MAV USC Avatar Landing on target (mostly) Search & Science Urban Search & Rescue Debris, stairs Combination of autonomy
& tele-operation Dante II Hazardous data collection Dante II (CMU) Sojourner (NASA) Narval (IST) Narval
Sojourner USF at the WTC courtesy of CRASAR Urbot & Packbot reconnoiter surrounding structures
Service Grace (CMU, Swarthmore, et al.): Attended AI conference Register, interact with other participants Navigate halls, ride elevator Guides Polly (MIT): AI lab Minerva (CMU): Museum
Personal assistants Nursebot (CMU): Eldercare Robotic wheelchairs Grace CMU Minerva In the Smithsonian
What Skills Do Robots Need? Identification: What/who is that? Object detection, recognition Movement: How do I move safely? Obstacle avoidance, homing Manipulation: How do I change that?
Interacting with objects/environment Navigation: Where am I? Mapping, localization Why Vision? Pluses Rich stream of complex information about the environment Primary human sense
Good cameras are fairly cheap Passive stealthy Minuses Line of sight only Passive Dependent on ambient illumination Arent There Other Important Senses? Yes
The rest of the human big five (hearing, touch, taste, smell) Temperature, acceleration, GPS, etc. Active sensing: Sonar, ladar, radar But Mathematically, many other sensing problems have close visual correlates The Vision Problem
How to infer salient properties of 3-D world from time-varying 2-D image projection Computer Vision Outline Image formation Image processing Motion & Estimation Classification
Outline: Image Formation 3-D geometry Physics of light Camera properties Focal length Distortion Sampling issues Spatial Temporal
Direction Fitting parameters to data Static Dynamic (e.g., tracking) Applications Motion Compensation Structure from Motion
Outline: Classification Categorization Assignment to known groups Clustering Inference of group existence from data Special case: Segmentation Visual Skills: Identification
Recognizing face/body/structure: Who/what do I see? Use shape, color, pattern, other static attributes to distinguish from background, other hypotheses Gesture/activity: What is it doing? From low-level motion detection & tracking to categorizing high-level temporal patterns
Feedback between static and dynamic Minerva Face Detection Finding people to interact with Penn MARS project Blimp, Clodbusters
Airborne, color-based tracking Visual Skills: Movement Steering, foot placement or landing spot for entire vehicle MAKRO sewer shape pattern Demeter region
boundary detection Florida Micro Air Vehicle (MAV) Horizon detection for self-stabilization UBM Lane & vehicle tracking (with radar)
Visual Skills: Manipulation Moving other things Grasping: Door opener (KTH) Pushing, digging, cranes KTH robot & typical handle Clodbusters push a box
cooperatively Visual Skills: Navigation Building a map [show 3D.avi] Localization/place recognition Where are you in the map? Laser-based wall map (CMU) Minervas ceiling map
Course Prerequisites Strong background in/comfort with: Linear algebra Multi-variable calculus Statistics, probability Ability to program in: C/C++, Matlab, or equivalent Course Details
First 1/3 of classes: Computer vision review by professor Last 2/3 of classes: Paper presentations, discussions led by students One major programming project Grading
10%: 30%: 10%: 50%: Two small programming assignments Two oral paper presentations + write-ups Class participation Project
Readings All readings will be available online as PDF files Textbook: Selected chapters from prepublication draft of Computer Vision: A Modern Approach, by D. Forsyth and J. Ponce Web page has other online vision resources Papers: Recent conference and journal articles spanning a range of robot types, tasks, and visual algorithms
Presentations Each student will submit short written analyses of two papers, get feedback, then present them orally Non-presenting students should read papers ahead of time and have some questions prepared. I will have questions, too :) Project
Opportunity to implement, test, or extend a robot-related visual algorithm Project proposal due in October; discuss with me beforehand Data I will provide canned data, or gather your own We will have a small wheeled robot to use for algorithms requiring live feedback Due Wednesday, November 27 (just before
Thanksgiving break) More questions? Everything should be on the web page: www.cis.udel.edu/~cer/arv or e-mail me at [email protected]
For demographic elements with no alternative data source or Social Determinate identifier, AHCCCS will create an online portal to be accessed directly by behavioral health providers for the collection of the data elements for members. Collected data will then be...
are energy-rich chemical compounds that contain carbon and hydrogen atoms. During combustion, the carbon and hydrogen combine with oxygen in the air to form carbon dioxide and water. This process releases energy in the forms of heat and light.
ODE Expectations. Teachers will have access to the PSO data results. 100 % of districts with leavers will complete their federal PSO collection. PSO collection is the required Data Collection and Reporting for the settlement
Ethics Training. All Veteran service officers and employees of that office should complete ethics training every year. This is many times not required this often, but in our line of work it is best to stay sharp and not become...
Reversible Logic Synthesis with Garbage Bits Lecture 6 Marek Perkowski Background The 3 * 3 Toffoli gate is described by these equations: P = A, Q = B, R = AB C, Toffoli gate is an example of two-through gates,...
Statistical Analysis and Design of Experiments for Large Data Sets Steven Gilmour School of Mathematical Sciences Centre for Statistics Introduction I will discuss microarrays, but there are many other possible biological applications Microarray experiments provide a measure of gene activity...
Japan ignored the Open Door Policy, which led to war with Russia and conflict with the United States. Clash over Manchuria (lots of natural resources) ... Policing the Western Hemisphere. Main idea: Using its economic and military power, the US...