Real-time Image Processing System - Ryerson University

Real-time Image Processing System - Ryerson University

Real-time Image Processing System Peter Chun Dr. Lev Kirischian Dr. Vadim Geurkov Overview Data Acquisition device (CMOS digital camera) Image Processor & hardware controllers (FPGA) Communication VGA interface Interface (DAC) (USB, FireWire, Parallel Port)

Memory (SRAM) 20-02-09 Peter Chun 2 Schematic 20-02-09 Peter Chun 3 Topics on the table Controllers on FPGA Digital Camera (OV5017 and M4088) SRAM (IDT71V416: 256K X 16-bit) VGA (DAC board THS8134) 20-02-09

Peter Chun 4 Topics on the table (continue) Real-time Image Processing system Step by Step Guide : How to construct oneStep1 : VGA verification Step2 : SRAM WRITE/READ verification using verified VGA Step3: Digital camera data acquisition on verified VGA using verified SRAM 1. Static one-frame 2. Continuous video capture 3. Video capture with pixel update 20-02-09 Peter Chun 5 VGA

Hardware Interface DB15 Number of pins you need: 6 Pin Descriptions 20-02-09 Hsync: Horizontal Sync Vsync: Vertical Sync R: red G: green B: blue GND: ground Peter Chun 6

Hardware Interface: DB15 5 4 10 15 20-02-09 3 9 14 2 8 13 1 7

12 PIN Number Description 1 RED 2 GREEN 3 BLUE 4 RESERVED 5 GND 6 RED RETURN 7 GREEEN RETURN 8 BLUE RETURN 9 N/C 10 GND 11 GND(COLOR) 12 N/C 13 HSYNC 14 VSYNC 15 N/C 6 11 Peter Chun

7 VGA (continue) VGA arrangement (0,0) (0, 0) Horizontal Synchronization Vertical Synchronization (640,480) (640, 480) 20-02-09 Peter Chun 8 Timing Requirements

Horizontal D Red, Green, Blue Horizontal Sync (Hsync) C B 20-02-09 E A PARAMERTES A B C

D E TIME 31.77u s 3.77us 1.89us 25.17us 0.94us Peter Chun 9 Timing Requirements Vertical R

Red, Green, Blue Horizontal Sync (Hsync) 20-02-09 Q S P O PARAMERTES O P Q R

S TIME 16.67ms 64us 1.02ms 15.25ms 0.35ms Peter Chun 10 VGA (continue) Example 20MHz 20-02-09

Peter Chun 11 SRAM Signals Control signals we, oe, cs, address(length..0), bhe, ble Data signals Data(depth..0) NOTE: depth is how many bits can you access at one time. length is how many of data can you store, which is equivalent to 2length. 20-02-09 Peter Chun 12 SRAM (continue) read

20-02-09 Peter Chun 13 SRAM (continue) write 20-02-09 Peter Chun 14 SRAM (continue) Example 40MHz 20-02-09 Peter Chun 15

Digital Camera Hardware Overview Amplifier for Gain Control Row Optical Sensor Array A/D converte r Digital Pixel Process Column

Analog region hsync 20-02-09 vsync pixel Digital pixel region pclk Peter Chun data[max..0] 16 Digital Camera (continue) Reference Signals

Hsync indicates the duration of active horizontal pixel insertion. It remains high during valid line and reset while not valid. Vsync represents the duration of active frame. When valid, it remains low and asserts high briefly to indicate the start of new frame. Pclk establishes the availability of the pixels. Either at the rising or falling edge of pclk, the pixel data is ready to be outputted. 20-02-09 Peter Chun 17 Digital Camera (continue) SetUp Exposure control, AGC (auto gain

control), Gamma correction, Frame rate, resolution control Method Memory Access I2C protocol 20-02-09 Peter Chun 18 Digital Camera (continue) Memory Access =100n s =100n s =50ns =50ns

=50ns =20ns 20-02-09 1 2 3 initial ready done 1 Peter Chun initial 2

3 ready done 19 SetUp Example 1 initial 2 ready 3 done 20-02-09 case y is -- Initial state after resets

when initial => if(loop_num = num_mul) then loop_num <= 0; y<=HWCTL_initial; else loop_num <= loop_num + 1; end if; -- Horizontal Window size control when HWCTL_initial => y<=HWCTL_ready; when HWCTL_ready => if(loop_num = num_mul) then loop_num <= 0; y<=HWCTL_done; else loop_num <= loop_num + 1; end if; when HWCTL_done => y<=VWCTL_initial; 1 initial

2 ready 3 done Peter Chun process(y) begin -- initial and reset values oeb <= '0'; csb <= '1'; web <= '1'; a <= "0000"; cam_data <= "00000000"; if y = initial then oeb <= '0'; csb <= '1'; web <= '1'; elsif y = HWCTL_initial then a <= "0110"; cam_data <= "00000000"; oeb <= '1';

csb <= '0'; web <= '0'; elsif y = HWCTL_ready then a <= "0110"; cam_data <= "00000000"; oeb <= '1'; csb <= '0'; web <= '0'; elsif y = HWCTL_done then oeb <= '0'; csb <= '0'; web <= '1'; 20 Digital Design Tips Multiple processes For reference signals (pclk, vsync, href) Filtering unstable states of the signals 20-02-09 Peter Chun

21 Digital Design Tips (continues) Address space division Address setup time exceeds synchronous clock events Long integer or std logic vector need attention Flip image Use decrement of address space 20-02-09 Peter Chun 22 Digital Camera (continue) Pixel Timing 20-02-09

Peter Chun 23 Digital Camera (continue) Horizontal Timing 20-02-09 Peter Chun 24 Digital Camera (continue) Vertical Timing 20-02-09 Peter Chun 25

Design evolution Frame based One memory Pixel based One memory Internal memory 20-02-09 Peter Chun 26 Examples 40MHz 66MHz 20-02-09 Peter Chun 27

Edge Detection Gradient based 20-02-09 Peter Chun 28 Edge Detection (continue) An edge pixel is described using two important features Edge strength, which is equal to the magnitude of the gradient Edge direction, which is equal to the angle of the gradient. 20-02-09 Peter Chun 29

Edge Detection (continue) Roberts cross Operator 385 pixels Max(|d-a|, |b-c|) ,where 288 pixels 20-02-09 a b c d Peter Chun 30 Roberts Cross Operation

385 pixels a1 b1 1 a2 b2 d1 e1 f1 x1 y1 z1

c2 d2 e2 f2 x2 y2 z2 -1 0 288 pixels or

0 1 -1 0 A 20-02-09 c1 0 B C D E F

Peter Chun X Y Z 31 Example Real-time image processing system with edge detection 20-02-09 Peter Chun 32

Recently Viewed Presentations