Java Media Framework Introduction

The Java Media Framework (JMF) is a recent API for Java dealing with real-time multimedia presentation and effects processing JMF handles time-!ased media" media which changes with respect to time #$amples of this are video from a television source" audio from a raw-audio format file and animations The !eta JMF % & specification will !e used for this report" as the' currentl' reflect the features that will appear in the final version

Stages
The JMF architecture is organi(ed into three stages)

*uring the input stage" data is read from a source and passed in !uffers to the processing stage The input stage ma' consist of reading data from a local capture device (such as a we!cam or T+ capture card)" a file on disk or stream from the network The processing stage consists of a num!er of codecs and effects designed to modif' the data stream to one suita!le for output These codecs ma' perform functions such as compressing or decompressing the audio to a different format" adding a watermark of some kind" cleaning up noise or appl'ing an effect to the stream (such as echo to the audio) ,nce the processing stage has applied its transformations to the stream" it passes the information to the output stage The output stage ma' take the stream and pass it to a file on disk" output it to the local video displa' or transmit it over the network For e$ample" a JMF s'stem ma' read input from a T+ capture card from the local s'stem capturing input from a +-. in the input stage It ma' then pass it to the processing stage to add a watermark in the corner of each frame and finall' !roadcast it over the local Intranet in the output stage

Component Architecture
JMF is !uilt around a component architecture The compenents are organi(ed into a num!er of main categories)

Media handlers *ata sources -odecs/#ffects .enderers Mu$/*emu$es

Media Handlers
Media0andlers are registered for each t'pe of file that JMF must !e a!le to handle To support new file formats" a new Media0andler can !e created

Data Sources
A *ata1ource handler manages source streams from various inputs These can !e for network protocols" such as http or ftp" or for simple input from disk

Codecs/E ects
-odecs and #ffects are components that take an input stream" appl' a transformation to it and output it -odecs ma' have different input and output formats" while #ffects are simple transformations of a single input format to an output stream of the same format

!enderers
A renderer is similar to a -odec" !ut the final output is somewhere other than another stream A +ideo.enderer outputs the final data to the screen" !ut another kind of renderer could output to different hardware" such as a T+ out card

Mu"/Demu"es
Multiple$ers and *emultiple$ers are used to com!ine multiple streams into a single stream or vice-versa" respectivel' The' are useful for creating and reading a package of audio and video for saving to disk as a single file" or transmitting over a network

#resenting Data

The Java Media Framework provides a num!er of pre-!uilt classes that handle the reading" processing and displa' of data 2sing the Pla'er" media can easil' !e incorporated into an' graphical application (A3T or 1wing) The Processor allows 'ou to control the encoding or decoding process at a finer level than the Pla'er" such as adding a custom codec or effect !etween the input and output stages

$sing the #la%er

The Pla'er class is an eas' wa' to em!ed multimedia in an application It handles the setup of the file handler" video and audio decoders" and media renderers automaticall' It is possi!l' to em!ed the Pla'er in a 1wing application" !ut care must !e taken as it is a heav'-weight component (it won4t clip if another component is placed in front of it)
import import import import java.applet.*; java.awt.*; java.net.*; javax.media.*;

public class PlayerApplet extends Applet { Player player = null; public void init() { setLayout( new orderLayout() ); !trin" media#ile = "etParameter( $#%L&$ ); try { '(L media'(L = new '(L( "et)ocument ase()* media#ile ); player = +ana"er.create(eali,edPlayer( media'(L ); i- (player."et.isual/omponent() 0= null) add($/enter$* player."et.isual/omponent()); i- (player."et/ontrolPanel/omponent() 0= null) add($!out1$* player."et/ontrolPanel/omponent()); 2 catc1 (&xception e) { !ystem.err.println( $3ot exception $ 4 e ); 2 2 public void start() { player.start(); 2 public void stop() { player.stop(); player.deallocate(); 2 public void destroy() { player.close(); 2

In this case" we are using the static create.eali(edPla'er() function of the Manager class to create the Pla'er o!5ect This ensures that the visual and control panel components are

created !efore it gets added to the window !' !locking until then It is also possi!le to create an unreali(ed pla'er and implement the -ontroller#vent0andler interface The window then waits for the controller2pdate event to fire and adds the components to the la'out as the' are reali(ed)
public sync1roni,ed void controller'pdate( /ontroller&vent event ) { i- ( event instanceo- (eali,e/omplete&vent ) { /omponent comp; i- ( (comp = player."et.isual/omponent()) 0= null ) add ( $/enter$* comp ); i- ( (comp = player."et/ontrolPanel/omponent()) 0= null ) add ( $!out1$* comp ); validate(); 2 2

2sing a simple applet tag" a multimedia stream can easil' !e em!edded in a we!page)
5APPL&6 /7)&=PlayerApplet 8%)69=:;< 9&%396=:<<= 5PA(A+ >A+&=#%L& .AL'&=$spar?le;.mpe"$= 5@APPL&6=

This will create an applet with an em!edded MP#6 video stream)

This can also !e used to em!ed multimedia content in an 0TM7 file" as shown !elow Previousl'" !rowser-specific plugins were re8uired

$sing the #la%er with Swing

The Pla'er can !e easil' used in a 1wing application as well The following code creates a 1wing-!ased T+ capture program with the video output displa'ed in the entire window)
import import import import import import javax.media.*; javax.swin".*; java.awt.*; java.net.*; java.awt.event.*; javax.swin".event.*;

public class A+#6est extends A#rame { Player Bplayer; A+#6est() { add8indowListener( new 8indowAdapter() { public void window/losin"( 8indow&vent e ) { Bplayer.stop(); Bplayer.deallocate(); Bplayer.close();

!ystem.exit( < );

2); set&xtent( <* <* :;<* ;C< ); APanel panel = (APanel)"et/ontentPane(); panel.setLayout( new orderLayout() ); !trin" media#ile = $v-wD@@E$; try { +ediaLocator mlr = new +ediaLocator( media#ile ); Bplayer = +ana"er.create(eali,edPlayer( mlr ); i- (Bplayer."et.isual/omponent() 0= null) panel.add($/enter$* Bplayer."et.isual/omponent()); i- (Bplayer."et/ontrolPanel/omponent() 0= null) panel.add($!out1$* Bplayer."et/ontrolPanel/omponent()); 2 catc1 (&xception e) { !ystem.err.println( $3ot exception $ 4 e ); 2 2 public static void main(!trin"FG ar"s) { A+#6est jm-6est = new A+#6est(); jm-6est.s1ow(); 2 2

Capturing !eal&time Data

+ideo and audio data can !e captured in real-time from input sources and streamed to files on the local files'stem

Capturing Audio
To capture audio" the specified sampling fre8uenc'" sample si(e and num!er of channels must !e specified JMF will attempt to locate an' devices which will support this format and return a list of all that match
/apture)evice%n-o di = null; .ector deviceList = /apture)evice+ana"er."et)eviceList( new Audio#ormat( $linear$* HHE<<* EC* ; ) ); i- ( deviceList.si,e() = < ) di = (/apture)evice%n-o)deviceList.-irst&lement(); Processor p = +ana"er.create(eali,edProcessor(di."etLocator()); )ata!ource source = p."et)ata7utput();

The source o!5ect returned from the Processor can then !e turned into a Pla'er o!5ect !' calling Manager createPla'er() To capture it to an audio file instead" a *ata1ink can take the data instead)
)ata!in? sin?; +ediaLocator dest = new +ediaLocator($-ileD@@output.wav$); try { sin? = +ana"er.create)ata!in?(source* dest); sin?.open(); sin?.start();

2 catc1 (&xception e) { 2

The com!ined source a!ove will take input from the first matching audio device (usuall' a microphone) and stream it to a wave file on the local files'stem

Capturing 'ideo
-apturing video is identical to capturing audio Most video sources have an accompan'ing audio track that must !e encoded as well" so we must create a compound destination file
#ormat -ormatsFG = new #ormatF;G; -ormatsF<G = new Audio#ormat( $linear$* HHE<<* EC* ; ); -ormatsFEG = new .ideo#ormat( $cvid $); @@ /inepa? video compressor Processor p; try { p = +ana"er.create(eali,edProcessor( new Processor+odel( -ormats* null ) ); 2 catc1 ( &xception e ) { 2 )ata!ource source = p."et)ata7utput(); +ediaLocator dest = new +ediaLocator( $-ileD@@output.mov$ ); )ata!in? -ilewriter = null; try { -ilewriter = +ana"er.create)ata!in?( source* dest ); -ilewriter.open(); -ilewriter.start(); 2 catc1 ( &xception e ) { 2 p.start();

This source will create a 9uicktime-format file called :output mov: with an audio track encoded raw and a video track encoded with the -inepak compressor

Conclusions
JMF is a highl' fle$i!le multimedia architecture that shows a lot of promise In the future" hopefull' 1un will work on making it more sta!le as well as on documenting the framework and providing more e$ample code The support for +ideo For 3indows (+F3) makes it a good contender for future multimedia applications In its current state" it is usa!le" !ut the lack of information makes it difficult to create a comple$ program

E"ercises

; %

-reate a JMF-!ased teleconferencing s'stem for two people to communicate over 2se .TP as the network communication protocol -reate a 1wing-!ased T+ watcher program The program should allow the user to select the video device to use for capture

Issues

-an JMF help propel Java into the field of multimedia displa' and editing4 3ill 1un develop a 7inu$-native :performance pack: for JMF" even though 7inu$ competes with 1olaris4 3hat changes need to !e made to the current !eta version of JMF to make it more usa!le when it !ecomes a release4

!e erences

Java Media Framework % & start page) http)//5ava sun com/products/5avamedia/5mf/% &/ JMF % & Programmer<s guide) http)//5ava sun com/products/5avamedia/5mf/% &/5mf%&-&=-guide pdf