Processing videos in Java

06 Mar 2009

In this post, I would like to share with the visitors on how to

How to load videos into Java using JMF
How to extract frames from a video file like avi,mpg,etc using Java Media Framework.

The class I developed for this purpose is called VideoUtility which is similar to the Image Processing utility I wrote in Java. In order to process Videos in Java, you need to use Java Media Framework, which can be obtained from the Sun Website and should be installed. Note that the JMF sometimes can be installed as a platform dependant package which makes it more efficient than the platform-independent version. Anyway once you download the JMF library, add it to the application classpath. Refer to the code shown below. I appreciate if you keep my copyright intact and if you find proper use of this class, please let me know.

        1: import java.awt.Image;

       2: import java.io.File;

       3: import java.io.FileNotFoundException;

       4: import java.io.IOException;

       5: import java.util.ArrayList;

       6: import javax.media.Buffer;

       7: import javax.media.Manager;

       8: import javax.media.MediaLocator;

       9: import javax.media.NoPlayerException;

import javax.media.Player;

import javax.media.Time;

import javax.media.control.FrameGrabbingControl;

import javax.media.control.FramePositioningControl;

import javax.media.format.VideoFormat;

import javax.media.util.BufferToImage;

/** @author Krishna Vangapandu **/

public class VideoUtility

{

    @SuppressWarnings("deprecation") 

/** * videoFile - path to the video File. */ 

public static Player getPlayer(String videoFile) 

    throws NoPlayerException, IOException

    {

        File f = new File(videoFile);

        if (!f.exists()) throw new FileNotFoundException("File doesnt exist");

        MediaLocator ml = new MediaLocator(f.toURL());

        Player player = Manager.createPlayer(ml);

        player.realize();

        while (player.getState() != Player.Realized);

        return player;

    }

    public static float getFrameRate(Player player)

    {

        return (float)noOfFrames(player)/(float)player.getDuration().getSeconds();

    }

    public static int noOfFrames(Player player)

    {

        FramePositioningControl fpc = (FramePositioningControl)player.getControl("javax.media.control.FramePositioningControl");

        Time duration = player.getDuration();

        int i = fpc.mapTimeToFrame(duration);

        if (i != FramePositioningControl.FRAME_UNKNOWN) return i;

        else return -1;

    }

    /** * * @param player - the player from which you want to get the image 

    * @param frameNumber - the framenumber you want to extract 

    * @return Image at the current frame position */ 

  public static Image getImageOfCurrentFrame(Player player, int frameNumber)

    {

        FramePositioningControl fpc = (FramePositioningControl) player .getControl("javax.media.control.FramePositioningControl");

        FrameGrabbingControl fgc = (FrameGrabbingControl) player .getControl("javax.media.control.FrameGrabbingControl");

        return getImageOfCurrentFrame(fpc, fgc, frameNumber);

    }

  

    public static Image getImageOfCurrentFrame(FramePositioningControl fpc, FrameGrabbingControl fgc, int frameNumber)

    {

        fpc.seek(frameNumber);

        Buffer frameBuffer = fgc.grabFrame();

        BufferToImage bti = new BufferToImage((VideoFormat) frameBuffer .getFormat());

        return bti.createImage(frameBuffer);

    }

  

    public static FramePositioningControl getFPC(Player player)

    {

        FramePositioningControl fpc = (FramePositioningControl) player .getControl("javax.media.control.FramePositioningControl");

        return fpc;

    }

  

    public static FrameGrabbingControl getFGC(Player player)

    {

        FrameGrabbingControl fgc = (FrameGrabbingControl) player .getControl("javax.media.control.FrameGrabbingControl");

        return fgc;

    }

  

    public static ArrayList<Image> getAllImages(Player player)

    {

        ArrayList<Image> imageSeq = new ArrayList<Image>();

        int numberOfFrames = noOfFrames(player);

        FramePositioningControl fpc = getFPC(player);

        FrameGrabbingControl fgc = getFGC(player);

        for (int i = 0;i <= numberOfFrames;i++)

        {

            Image img = getImageOfCurrentFrame(fpc, fgc, i);

            if(img!=null) imageSeq.add(img);

        }

        return imageSeq;

    }

  

    public static ArrayList<Image> getAllImages(String fileName) throws NoPlayerException,IOException

    {

        Player player = getPlayer(fileName);

        ArrayList<Image> img = getAllImages(player);

        player.close();

        return img;

    }

}

The main method of interest here is the "getAllImages()" method which returns all the frames as an arraylist of Images. For the specified filename it obtains a Player that is used to retrieve all the images. The reason I posted this article is that I find quiet a few visits to my blog to my article on Image Processing. I am not sure on how useful that post was but I thought may be there is someone out there who might need to get started to extract frames from videos. I put in a lot of effort and looked up a lot of material to actually come up with some code that makes sense.

Musings of a Software Engineer v2.0

Processing videos in Java