Watch a folder for new files and convert them

with bash script

For a recent web app project I needed a solution to automatically convert uploaded files with ffmpeg.
This post describes my solution so Ill still know what I did 4 weeks from now.
The client uploads folders, subfolders and files to a specific folder via FTP. Lets say this folder sits at
/data/upload/. Every *.mp4 file inside these folders and subfolders needs to be encoded with
ffmpeg. This will result in three different files per uploaded file, each with a different bit rate. This
should happen automatically and only for new uploaded files.

Watch a folder for uploaded files

I watch a folder with the app iwatch. Install iwatch via sudo apt-get install iwatch.
During install it may ask you to specify your configuration for Postfix. This is outside the scope of this
post. You can run iwatch as a daemon and specify a configuration file:
1

iwatch -d -f /etc/iwatch/iwatch_encoder.xml

The xml file looks like this:

1
2
3
4
5
6
7
8
9
10
11

<?xml version="1.0" ?>

<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >

<config>
<guard email="root@localhost" name="IWatch"/>
<watchlist>
<title>Neue Video-Daten erhalten. Encoding starten.</title>
<contactpoint email="foo@bar.baz" name="Administrator"/>
<path type="recursive" alert="off" exec="find %f -iname '*.mp4' -execdir /data/upload/
</watchlist>
</config>

Line 9 is the crucial part.

Lets dissect it:
type="recursive" - This makes sure to watch the folder recursively. So you dont miss
files inside subfolders.
alert="off" - You can get emails every time something happens in /data/uploads. I chose
not to.
exec="find %f -iname '*.mp4' -execdir
/data/upload/encode_files.sh {} \;" - The part inside the quotation marks is the
command I want to execute once a file is uploaded. It will be executed for every file uploaded.
events="close_write" - I watch only this event. It means to fire the command once a
file is closed after write. This is essential for large (video) files. Otherwise the command
triggers while the file is still uploaded.
<path>/data/upload</path> - This is simply the path you want to watch.

Trigger a command for uploaded files

find %f -iname '*.mp4' -execdir /data/upload/encode_files.sh
{} \; This is bash magic happening. Took me quite some time to find out.
%f is the full path of the filename that gets an event. (via iwatch documentation).
What the command does:
Find every mp4-file (*.mp4) inside directory %f and execute a certain command with this file.
execdir - run the command inside the directory where you found the mp4-file.
/data/upload/encode_files.sh {} \; - run the bash script encode_files.sh
and pass the mp4-file as first parameter so I can access the filename inside the script.

The encoding script

1
2
3
4
5
6
7
8
9
10
11

f=$(basename "$1")
if ! [ -f "/data/streamfiles/$f" ]; then
ffmpeg -i "$1" -vcodec libx264 -b:v 1000k -ab 96k -ar 48k -f mp4 -strict experimental "/
fi

if ! [ -f "/data/streamfiles/128_$f" ]; then
ffmpeg -i "$1" -vcodec libx264 -b:v 250k -ab 96k -ar 48k -f mp4 -strict experimental "/d
fi
if ! [ -f "/data/streamfiles/250_$f" ]; then
ffmpeg -i "$1" -vcodec libx264 -b:v 500k -ab 96k -ar 48k -f mp4 -strict experimental "/d
fi

f=$(basename "$1") - Take the first parameter and strip it of its path. This gets you the
bare filename (my_uploaded_file.mp4 instead of
/data/uploaded/subfolder/my_uploaded_file.mp4)
if ! [ -f "/data/streamfiles/$f" ]; then - If the file doesnt exist at the
target destination run the following command
ffmpeg -i "$1" -vcodec libx264 -b:v 1000k -ab 96k -ar 48k -f mp4
-strict experimental "/data/streamfiles/$f" - This is a bunch of ffmpeg
gibberish, also outside of the scope of this post. Ffmpeg seems overwhelming at first but is quite
well documented.