Fast Motion Estimation Algorithms

Master:

Prof. Hashemi
By:

Amir Shahrokhi
1

Progress
What

has been done:

Download Papers about ME Specially Block Matching Survey Papers and Their References

What

will be done:

Simulate Some of Proposed Algorithms Using MATLAB Comparison of different algorithms

Outline
Fast

Motion Estimation Algorithms:

Introduction Block Matching Algorithms

Full

Search Method 2-D Logarithmic Search 3-Step Search Conjugate Direction Search Diamond Search

Introduction
Temporal

correlation between successive frames An easy way to remove this redundancy:

Frame Replenishment:
A

threshold for difference in pixel intensity from previous frame If less than threshold Send nothing If more than threshold Send position & intensity difference Advantage: Easy to implement, Low complexity Disadvantage: Low compression when rapid changes In low bit-rates may cause scene cuts Better way: Motion Compensated Coding

Introduction (Continue)
Motion

Compensated Coding

Frame changes are considered as object movements so:

Estimate

motion of current frame respect to previous Calculate difference between motion compensated & current frames Code motion vectors and differences as this frame To have better compression we need to good motion estimation

Different algorithms for motion estimation

Block

Matching (Our Discussion) Pel Recursive Technique Optical Flow

5

Outline
Fast

Motion Estimation Algorithms:

Introduction Block Matching Algorithms

Full

Search Method 2-D Logarithmic Search 3-Step Search Conjugate Direction Search Diamond Search

Block Matching
Divide current frame to small rectangular blocks Motion of each block is assumed to be uniform Find the best match for each block in previous frame Calculate motion vector (MV) between current block and its counterpart in previous frame Typical size for blocks: 16x16 pixels Maximum movement: w: typically 8, 16 or 32 Matching Criteria:

Mean Absolute Error (MAE) Mean Square Error (MSE) Sum of the Squared Error (SSE)

MAE is preferred due to its simplicity

Block Matching
Search

Window (in previous frame)

Rectangle with the same coordinate as current block in current frame, extended by w pixels in each directions
q+2w w p+2w q

w
w

Block Matching
Full

Search

All candidates within search window are examined (2w+1)2 positions should be examined Advantage: Good accuracy, Finds best match Disadvantage: Large amount of computation: (2w+1)2 matches, 16x16 MAE for each match Impractical for real-time applications In order to avoid this complexity, we should reduce search positions Fast Block Matching Algorithms

Block Matching (Fast Algorithms)

Improvements
Li,

to Full Search

Successive Elimination Algorithm (SEA)

Salari Using SAD as matching criterion Idea: excluding many search positions while finding still best match Let f(x,y) be the intensity of each pixel in current block and f(x,y) be intensity of each pixel in a current search position, and (x,y) be the counterpart of (x,y), so from triangular inequality: |S f(x,y) S f(x,y)| <= S |f(x,y) f(x,y)| = SAD So difference between sum of intensity values can be used as a lower bound for SAD value. This bound is used to exclude search points Cost is to calculate sum of intensities, search points differ from each other in 2 rows or columns and they have a great overlap Sum of intensities can be calculated efficiently
10

Block Matching (Fast Algorithms)

2-D Logarithmic Search

Jain and Jain Examine central point & its four surroundings Distance from center: w/2 Find best match If best match is in center or on boundaries, half distance from center Examine five new points centering previous best When distance is 1, use all 9 matches, find best. Stop
2 1

2 4

1 4 4

4
4

2
4 3

4
4

11

Block Matching (Fast Algorithms)

Three-Step Search (3SS)

Koga et al 9 Points: Central point & its 8 surroundings Distance: w/2 Find the best match Use previous best as center Half distance, select 8 new Repeat algorithm 3 times Examines 25 points Assumes a uniform distribution of MVs
1 1 1

1 2 2 3 3 3 3 2 3 1 3 3 3 2 2 2 2 2

12

Block Matching (Fast Algorithms)

Improvements

to 3SS

New Three-Step Search (N3SS)

Li, Zeng, Liou Experiments show that most of best matches are near center More points in center: 8 neighbors of center of search area (33 Search points) More robust & Smaller error in motion compensation

Four-Step Search (4SS)

Po, Ma The idea is same as N3SS, a hybrid of 3SS and 2DL, (For w=8) Central point and 8 surrounding with distance = 2 If best match on center, distance = 1 select 8 new points and end else select best match as new center with previous distance (2) If following step repeated 3 times, distance=1, select 8 new points and end 2 to 4 steps 17-27 search points, Less search points on average Better MSE than 3SS & less search points than N3SS, Better prediction
13

Block Matching (Fast Algorithms)

Conjugate Direction Search

Srinivasan, Rao Start from center point, find best match between this point & its 2 horizontal neighbors Search 1 point in the direction of best match & Examine 4 If no change in best match in horizontal, repeat for vertical When no change in vertical, select the point as best match worst case 2w+3 points
7 6 5 3 5 2 1 1 1

14

Block Matching (Fast Algorithms)

Improvement

to CDS

Fast One-Step Search (FOSS)

Srinivasan, Ramachandran 5 macro-blocks : center of search area and 4 points between center and 4 borders of search area Calculate number of search points for these macro-blocks in eight directions Direction with least number of search points is used as optimal for CDS This is done for first P-frame after I-frame This direction is unique for all P-frames in a GOP Reduces search points by 2, with the cost of 0.4% overhead of overall compute Better direction select and faster than CDS

15

Block Matching (Fast Algorithms)

Diamond Search (DS)

Zhu, Ma Start with 9 points creating a large diamond with center of search window in center Find best match, repeat with this best match as center If best match is in center, use 5 points creating small diamond, find best match, this is target Better performance than 3SS Less complex than N3SS and 4SS, with same performance
16

Block Matching (Fast Algorithms)

DS Example
2 1 1 1 1 1 1 1 1 2 4 1 3 4 2 4 3 3 4 3 3

17

References

FPGA Implementation of a Novel, Fast Motion Estimation Algorithm for Real-Time Video Compression, FPGA 2001, CA. USA, S. Ramachandran and S. Srinivasan, Feb. 2001 Image & Video Compression for Multimedia Engineering, Y.Q. Shi and H. Sun, 2000 A New Diamond Search Algorithm for Fast Block-Matching Motion Estimation, IEEE Trans. Image Processing, S. Zhu and K. K. Ma, Feb. 2000 A Novel Four-Step Search Algorithm for Fast Block Motion Estimation, IEEE Trans. Circuits System, Video Technology, L. M. Po and W. C. Ma, June 1996 Successive Elimination Algorithm for Motion Estimation W. Li and E. Salari IEEE Trans. , Jan. 1995 A New Three-Step Search Algorithm for Block Motion Estimation, IEEE Trans. Circuits System, Video Technology, R. Li, B. Zeng, and M.L. Liou, Aug. 1994 Predictive Coding Based on Efficient Motion Estimation, IEEE Trans. on communications, R. Srinivasan, K.R. Rao, Aug. 1985 Motion Compensated Inter-Frame Coding for Video-Conferencing, T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, Proc. NTC81, Nov. 1981 Displacement Measurement and its Applications, IEEE Trans. on communications, J.R. Jain and A.K Jain, Dec. 1981
18

Any Question?

19