You are on page 1of 4

1.

Exercise
VU Multi-modal Information Retrieval, Summer 2008
Get acquainted with Matlab. Develop solutions for the following problems: - Read all images from a directory (for loop!) and convert all Jpegs (if statement!) to the PNG format. - Image processing: resizing, cropping, etc. - Audio processing: increase/decrease loudness, cropping, etc. - Convert colour images to greyscale images - Apply a Gaussian filter to images - Read, transform and write the frames of AVI videos - Print images on the screen - Plot histograms, waveforms, spectrograms. - Create a GUI in GUIDE (type doc guide in matlab) that shows two images specified by the user Feel free to experiment! Use samples from the media test database (do not submit these!). Submission deadline is 15.4.2008.

Please make sure, that your code can be run without modifying the Matlab path, i.e. use only relative paths and put all source files in one directory. The program must be started by calling a Matlab script named lab[1-4].m Use the TUWEL forums for questions concerning the assignments. Active participation in TUWEL is encouraged and will be honoured. Please submit Matlab scripts and documentation as zip file via TUWEL (label your files according to this schema: MMIR_08_Matrikelnummer_Exercise_[1-4] ). Assessment can be found after submission in TUWEL. mmir_lab@ims.tuwien.ac.at, www.ims.tuwien.ac.at

2. Exercise
VU Multi-modal Information Retrieval, Summer 2008
Extract one feature each from audio and video content. If possible, the visual feature should be invariant against rotation and scaling. Furthermore, both features should be as robust against white noise as possible. The audio feature should additionally be invariant to the sample rate. Please do not implement a simple colour histogram or a unitary transformation. For example, design a feature based on motion activity (optical flow) or a cepstrum feature. Select a folder containing video/audio files with the Matlab function uigetdir, call the respective feature extraction function for each file, and save the feature data to a file with the feature s name and the extension .mat . The result are two MAT-files, one containing the computed features for the audio files and one for the video files.

Submission deadline is 29.4.2008.

Please make sure, that your code can be run without modifying the Matlab path, i.e. use only relative paths and put all source files in one directory. The program must be started by calling a Matlab script named lab[1-4].m Use the TUWEL forums for questions concerning the assignments. Active participation in TUWEL is encouraged and will be honoured. Please submit Matlab scripts and documentation as zip file via TUWEL (label your files according to this schema: MMIR_08_Matrikelnummer_Exercise_[1-4] ). Assessment can be found after submission in TUWEL. mmir_lab@ims.tuwien.ac.at, www.ims.tuwien.ac.at

3. Exercise
VU Multi-modal Information Retrieval, Summer 2008
Implement a query engine for your two features. Choose appropriate distance measures for similarity measurement. Apply the vector space model, a Bayesian classifier or some other probabilistic model. Implement the query engine as a Matlab function (Input: query example, Output: ranked similar media). Assess the quality of your feature and your query engine by recall and precision (based on a self-defined ground truth (classes)). Recall and precision are defined as follows: Recall = retrieved relevant media / relevant media Precision = retrieved relevant media / retrieved media Implement a Matlab function that selects an audio/video file with uigetfile, and outputs a figure with the most similar media in the data base. Additionally visualise the results in a recall/precision graph. The query engine should use the pre-computed features from Exercise 2 (no repeated feature computation necessary). Submission deadline is 20.5.2008. Please make sure, that your code can be run without modifying the Matlab path, i.e. use only relative paths and put all source files in one directory. The program must be started by calling a Matlab script named lab[1-4].m Use the TUWEL forums for questions concerning the assignments. Active participation in TUWEL is encouraged and will be honoured. Please submit Matlab scripts and documentation as zip file via TUWEL (label your files according to this schema: MMIR_08_Matrikelnummer_Exercise_[1-4] ). Assessment can be found after submission in TUWEL. mmir_lab@ims.tuwien.ac.at, www.ims.tuwien.ac.at

4. Exercise
VU Multi-modal Information Retrieval, Summer 2008
Extend your query engine by iterative refinement by relevance feedback. Implement a Matlab function that takes a vector of media files and relevance information as input (positive/negative examples). Employ a classifier for the identification of further positives. Select an appropriate kernel function. The process is as follows: Select a media file with uigetfile, find the N best matches (as in Exercise 3), and present the results in a simple GUI. The GUI (see GUIDE example in Exercise 1) should enable the user to mark images of the result set as positive or negative. From the positive and negative labelled images iteratively (from run to run) build a ground truth with two classes (relevant, not relevant). Use this ground truth to train a classifier and present the refined results in the GUI. An example SVM implementation for Matlab can be found under http://sourceforge.net/projects/svm/ (Installation: 1. Extract to toolbox directory , 2. Add to Matlab path, 3. Learn application by "help Contents") Submission deadline is 17.6.2008. Please make sure, that your code can be run without modifying the Matlab path, i.e. use only relative paths and put all source files in one directory. The program must be started by calling a Matlab script named lab[1-4].m Use the TUWEL forums for questions concerning the assignments. Active participation in TUWEL is encouraged and will be honoured. Please submit Matlab scripts and documentation as zip file via TUWEL (label your files according to this schema: MMIR_08_Matrikelnummer_Exercise_[1-4] ). Assessment can be found after submission in TUWEL. mmir_lab@ims.tuwien.ac.at, www.ims.tuwien.ac.at

You might also like