Overview: Motion-Compensated Coding

Overview: motion-compensated coding
n Motion-compensated prediction
n Motion-compensated hybrid coding
n Power spectral density of the motion-compensated
prediction error
n Rate-distortion analysis
n Loop filter
n Motion compensation with sub-pel accuracy
Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 1
Motion-compensated prediction
previous frame
stationary ∆t
background
current frame
x
time t
y
moving
object
 dx 
„Displacement vector“ d  shifted
 y object
Prediction for the luminance signal S(x,y,t) within the moving object:
Sˆ(x, y, t) = S(x − d x , y − d y , t − ∆t)

1
Motion-compensated prediction: example
Previous frame Current frame
Current frame with Motion-compensated

displacement vectors Prediction error
Motion-compensated hybrid coder
Control „side information“
Video Channel
signal
Intraframe
S
e DCT coder
- Intraframe
Decoder
Motion
compensated
predictor Decoder
2
Motion-compensated hybrid decoder
Channel
reconstructed
Intraframe video signal
Decoder
Motion
compensated
predictor
side information
Model for performance analysis of an

MCP hybrid coder
luminance
signal S R-D optimal
e
intraframe
- encoder
intraframe
decoder
e‘
motion s‘
compensated
predictor
displacement estimate
 dx   ∆ x 
true displacement d  + ∆  displacement error
 y  y
3
Analysis of the motion-compensated prediction error
Motion-compensated signal
c(x ) = s(x − ∆ x ) − n( x )
Prediction error
Previous e(x ) = s(x ) − c (x )

frame = s(x ) − s(x − ∆ x ) + n(x )
x
c(x) s(x)
Current
frame
Displacement d x Displacement error ∆ x
Analysis of m.c. prediction error (cont.)
n Motion-compensated prediction error
e(x) = s(x)−c(x)= s(x)−s(x−∆x) +n(x) = (δ(x)−δ(x − ∆x ))∗s(x)+n(x)

n Power spectrum of prediction error, assuming constant displacement
error ∆ x, statistical independence of s and n
Φ ee (ω) = Φ ss (ω)(1 − e − jω∆ x )(1 − e jω∆ x )+ Φnn (ω )
( )
= 2Φ ss (ω ) 1− Re {e − jω∆ x } + Φnn (ω )
n Random displacement error ∆ x , statistically independent from s, n

{ ( )
Φ ee (ω) = E 2Φ ss (ω) 1 − Re{e − jω∆ x } + Φ nn (ω )}
( {
= 2Φ ss (ω ) 1− Re E{e jω∆ x
−
}})+ Φ (ω ) nn
= 2Φ ss (ω )(1− Re {P(ω )})+ Φ nn (ω)
4
Analysis of m.c. prediction error (cont.)
What is P( ω )? P (ω ) = E {e − }
jω ∆ x
n
= ∫ {
p ∆ x (∆ )e − jω ∆ d∆ = F p ∆ x (∆ x ) }
−∞
Fourier
Fourier transform
transform of
of the
the displacement
displacement error
error pdf!
pdf!
n Same as characteristic function of displacement error, except for sign

n Extension to 2-d
( )
Φ ee (ω x ,ω y ) = 2Φ ss (ω x ,ω y ) 1 − Re{P(ω x ,ω y )} + Φ nn (ω x ,ω y )
Fourier transform of the noise spectrum

displacement error pdf
power spectrum of
p(∆ x , ∆ y )
luminance signal
Power spectrum of motion-compensated

prediction error
π
0
frequency ω x
5
R-D function for MCP with integer-pixel accuracy
Minimum bit-rate for given SNR
n (∆ x , ∆ y ) T assumed
uniformly distributed
between
1
∆x = ± pel
2
1
∆ y = ± line ~0.7 bpp
2
n Gaussian signal model 3
−
 ω x2 + ω 2y  2
Φ ss (ω x , ω y ) = A 1 + 
 ω 02 
n Typical parameters for CIF
resolution (352 x 288
pixels)
Required accuracy of motion compensation
n p(∆ x , ∆ y ) isotropic
Gaussian pdf with variance σ
2
3
−
 ω x2 +ω 2y  2
Φss (ω x ,ω y ) = A1+
ω02 
n

n Typical parameters for CIF
resolution (352 x 288 pixels)
n Minimum bit-rate for
SNR = 30 dB
6
Model of MCP hybrid coder with loop filter
e R-D optimal
intraframe
luminance - encoder
signal S
intraframe
decoder
spatial
filter
F
motion
compensated
predictor
displacement estimate
 dx   ∆ x 
true displacement d  + ∆  displacement error
 y  y
Motion-compensated prediction error with loop filter
Motion-compensated signal Prediction error

c(x ) = s(x − ∆ x ) − n( x ) e(x) = s(x) − f (x)∗ c(x)
= s(x) − f (x)∗ s(x − ∆ x ) + f (x) ∗n(x)
Previous
frame Impulse response
of loop filter
x
c(x) s(x)
Current
frame
Displacement d x Displacement error ∆ x
7
Spatial power spectrum of m.c. prediction error
with loop filter
Φee(Λ) = Φss(Λ)(1+ | F(Λ)|2 −2Re{F(Λ)P(Λ)})+Φnn(Λ)| F(Λ)|2
P(Λ) 2 -D Fourier transform of displacement error pdf

F(Λ) 2 -D Fourier transform of f (x, y)
Φ uu spatial spectral power density of signal u
Λ vector of spatial frequencies (ω x ,ω y )
n(x, y) noise
Optimum loop filter
n Wiener filter minimizes prediction error variance
∗ Φ ss ( Λ )
Fopt ( Λ) = P
12(Λ
3) ⋅ Φss4
1 Λ4
(4 )2+4 Φ nn4(4
Λ
3)
accounts for accuracy of accounts for noise

motion estimation
n To determine Wiener filter from measurements:

cross spectrum between s(x,y) and
the motion-compensated signal
Φ sc (Λ) c(x, y) = r(x − dˆ x , y − dˆ y )
Fopt (Λ) =
Φ cc (Λ)
8
Required accuracy of motion
compensation with loop filter
n p(∆ x , ∆ y ) isotropic Gaussian pdf with variance σ 2
n Minimum bit-rate for SNR = 30 dB
~0.8 bpp
Experimental evaluation of
sub-pixel motion compensation
n ITU-R 601 TV signals, 13.5 MHz sampling rate, interlaced,
blockwise motion compensation with blocksize16x16
Zoom Voiture
9
Influence of noise on the
performance of MCP
Motion Compensation Performance in

H.263
500
Bit Rate [kbps]
400
Intra mode
300 only
Integer-pel
accuracy
200
1/2-pel
100 accuracy
0
24 30 36
Simulation details: PSNR [dB]
Foreman, QCIF, SKIP=2
Q=4,5,7,10,15,25
10
Summary: motion-compensated coding
n Motion-compensated prediction exploits similarity of
successive frames of a video sequence.
n Hybrid coder combines motion compensation and spatial
2D coding.
n Power spectral density of motion-compensated prediction
error is flat.
n Loop filter improves the prediction.
n Maximum gain by motion compensation with integer-pel
accuracy ~0.8 bit/sample.
n Sub-pixel accuracy of motion compensation can improve
prediction
n “Noise” in the image signal limits the accuracy of motion
compensation.
11

Overview: Motion-Compensated Coding

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Overview: Motion-Compensated Coding

Uploaded by

Copyright:

Available Formats

Overview: motion-compensated coding

Sˆ(x, y, t) = S(x − d x , y − d y , t − ∆t)

Current frame with Motion-compensated

Motion-compensated hybrid coder

Control „side information“

Model for performance analysis of an

Previous e(x ) = s(x ) − c (x )

Analysis of m.c. prediction error (cont.)

n Motion-compensated prediction error

e(x) = s(x)−c(x)= s(x)−s(x−∆x) +n(x) = (δ(x)−δ(x − ∆x ))∗s(x)+n(x)

n Random displacement error ∆ x , statistically independent from s, n

= 2Φ ss (ω )(1− Re {P(ω )})+ Φ nn (ω)

n Same as characteristic function of displacement error, except for sign

Fourier transform of the noise spectrum

Power spectrum of motion-compensated

Required accuracy of motion compensation

Motion-compensated prediction error with loop filter

Motion-compensated signal Prediction error

Φee(Λ) = Φss(Λ)(1+ | F(Λ)|2 −2Re{F(Λ)P(Λ)})+Φnn(Λ)| F(Λ)|2

P(Λ) 2 -D Fourier transform of displacement error pdf

Optimum loop filter

n Wiener filter minimizes prediction error variance

accounts for accuracy of accounts for noise

n To determine Wiener filter from measurements:

Motion Compensation Performance in

You might also like