Video summarization.

An Approach to summarize video automatically

Table of Contents
I. Problem statement ................................................................................................ 2
Brief
II. Motivation .............................................................................................................. 2

2/4/2017
III. Challenges ............................................................................................................. 3
IV. Implementation ..................................................................................................... 3
V. Evaluation .............................................................................................................. 5
VI. Contact Information .............................................................................................. 6
Problem statement

Brief

We need to summarize video into smaller video which contain the most
important frames and shots, we need this shoots to be representative and
cover diversity of the video.

We put a constraint that video summary duration must be within 15% of

duration of the original video.

We hope this approach will work on different kinds of videos.

Motivation
Video has rapidly become one of the most common sources of visual
information. For example it takes over 85 years to watch all videos on
Youtube, So automatic video summarization is a key tool to help human
users browse video data to overcome problems like slow bandwidth and
problem that user himself might have enough time to watch video so he can
watch summary to get the needed insight.

Also found ways now are built especially for special type of videos like
summarizer for football matches only or summarizer for drama movies we
hope here that our desired approach will work for any kind of video or on
any criteria.

Our approach will depend mainly on training neural network using pervious
users experience

VIDEO SUMMARIZATION. JUNE 2016 2

Challenges

1- Finding dataset which cover different kind of videos.

2- Dealing with large videos in terms of sampling process.

3- Dealing with variables size of videos.

4- Training time when dealing with large videos.

5- The inter-dependency across video frames is complex and highly

inhomogeneous.

6- Finding pre-trained models to extract features from the image.

Implementation
Project is divided mainly into three parts

A- Training the model.

1- Sampling video into frame 2FPS(2 frames per seconds() using

python code with - library- which help in dealing with videos

2- Given set of frames for each video we need for each frame set of
features vector of length 1024- using pre trained convolution
network produced by Google(GoogleNet) using tensor flow frame
work with Python .

VIDEO SUMMARIZATION. JUNE 2016 3

 3- Build our recurrent network with two hidden layer and number of
units changes dynamically depending on number frames in the
video using keras frame work with Python .

- Xi represents vector feature of frame i in the video

- Yi represents importance of the frame.

4- Train the model using different datasets

B- Evaluate model.
Evaluate model using part unseen of data.

VIDEO SUMMARIZATION. JUNE 2016 4

 C- Production model.

Make script which integrate different parts of the system. Script will
take video and produce suitable summary for it.

1- It samples video
2- Get features
3- Use the network to predict values of y
4- Use knapsack Algorithm to produce most important frames with
respect to 15%.
5- Display the video.

Evaluation

Evaluate model using part unseen of data.

We use mainly two datasets Summe and Tvsum

Settings of tests Results

Dataset training and validations Testing Our results

80% SumMe 20% SumMe 41.78.1
SumMe

80% TVSum 20% TVSum 47.508475.16

TVSum

VIDEO SUMMARIZATION. JUNE 2016 5

Contact Information

Name ID
Nourane Ihab Bechir 75
Mina sameh 74
Maged milad
Omar Mahmoud Elsherif 46

VIDEO SUMMARIZATION. JUNE 2016 6