Video keyframes

From EChase
Jump to: navigation, search

ORF apparently have a highly-sophisticated in-house keyframing system, but unfortunately we can't use that so we have to generate our own keyframes. We can do this in one of two ways: the first is to simply take frames at regular intervals throughout the video, and the second is to extract only the frames that are "interesting", i.e., that occur at some scene change etc.

Each has its advantages and disadvantages: regular intervals are very easy to calculate but don't guarantee that the keyframes reflect the scene changes in the video, whilst scene-change detection is both harder to do and also means that we have to find some way to record the timing of the keyframes so they can be properly displayed in the user interface.

In the end I decided to use a combination of the two, selecting every 10 seconds to begin with and then only the "interesting" frames out of those, recursively. This recursion means that it's possible to limit the number of keyframes to a sensible number: as the majority of the ORF footage is fairly static and not highly-edited (no fast cuts, etc), it's not useful to have a large number of keyframes that show the same thing being presented to the user, so similar keyframes are removed. I'd have preferred to just do the scene-detection way, but although the AVI format is indexed and so usefully provides keyframe information, I couldn't find anyway of recording the time at which the keyframe occurs in any reusable way (quite aside from figuring out how to perpetuate it into the metadata so the UI could use it).

KEYFRAMES.bat[edit]

@echo off

:: Keyframe extractor
::
:: Uses mencoder to extract jpgs of keyframes from a video. 
::
:: Usage: keyframes <infile> <outdir> <keycount> [<pass no>] [<original filename>]

setlocal

:: Create output dir
if not exist %~f2 mkdir %~f2

:: Create temp batch file to strip zeros - used later for renaming keyframes
if not exist zerostrip.bat (
echo @echo off > zerostrip.bat
echo set num=%%~n1 >> zerostrip.bat
echo :ZEROLOOP >> zerostrip.bat
echo set num=%%num:*0=%% >> zerostrip.bat
echo if "%%num:~0,1%%" == "0" goto ZEROLOOP >> zerostrip.bat
echo set num=%%num: =%%>> zerostrip.bat
echo rename %%1 "%%2 %%num%%.jpg" >> zerostrip.bat
)

set /a passno=%4+1

set ORIGINAL=%5

if "%ORIGINAL%" == "" set ORIGINAL=%~n1

:: This skips by 10secs for each 'frame' when using 25fps video. If using 24fps change to 240.
set framestep=250

:: If this is not the first pass then only bother with keyframes in video
if not %passno% == 1 set framestep=I

:: Extract keyframes from input video
echo Extracting keyframes from input video... (pass %passno%)
mplayer %1 -speed 100 -vf framestep=%framestep% -vo jpeg:outdir=pass%passno% -nosound >nul 2>nul

: Get number of keyframes and store in environment variable
DIR/A-D/B/-P pass%passno% %1 >setcnt1.tmp
ECHO SET cnt=%%2>----------.bat
FIND /C /V "" setcnt1.tmp >setcnt1.bat
FOR %%c in (call del) do %%c setcnt1.bat
FOR %%f in (setcnt1.tmp ----------.bat) do DEL %%f

set /a count=%cnt%-1

ECHO %count% keyframes extracted.

:: If no of keyframes is greater than desired number, do extraction again on joined keyframes
if %count% GTR %3 (
	echo Number exceeds %3, looping...

	:: Create temp movie of keyframes
	cd pass%passno%
	mencoder "mf://*.jpg" -mf type=jpeg:w=320:h=240 -o "../output%passno%.mp4" -ovc lavc -lavcopts vcodec=mpeg4 >nul 2>nul

	:: Return to original directory
	cd ..

	:: Recurse on temp movie
	call keyframes output%passno%.mp4 %2 %3 %passno% %ORIGINAL%

	:: Delete temp movie
	del output%passno%.mp4
) else (
	:: Rename all keyframes to include original filename
	cd pass%passno%

	for /f "usebackq delims=" %%i in (`dir /b *.jpg`) do call ..\zerostrip %%i %ORIGINAL%
	
	for /f "usebackq delims=" %%i in (`dir /b *.jpg`) do copy "%%i" "%~f2\%%i"

	cd ..

	:: Delete all passes
	for /f "usebackq delims=" %%i in (`dir /b pass*`) do @rmdir /s /q %%i
)

:: Delete temp zerostrip.bat
if exist zerostrip.bat del zerostrip.bat

:END

Despite it's length, the above script should be fairly easy to understand, thanks to the comments. What might not be entirely clear is how to run it. Basically, the script expects three parameters: the video to generate the keyframes for, the name of the output directory, and the number of keyframes to generate. The other two parameters are optional and are used during recursion, so are only really meant for use by the script itself, not by users.

Example usage:

keyframes collections/orf/EasternEurope1.avi keyframes 12

This will output a set of up to 12 JPEG files (less if the video isn't long enough to warrant 12 frames, i.e, it's less than 120 secs) in the directory 'keyframes' with the filename 'EasternEurope1 X.jpg', where 'X' is the number of that keyframe.

One last thing to note is that this script as-is assumes that the input video is 25fps - which the content from ORF so far is. If working with some other framerate then the %framestep% value will need changing, so for 24fps it would be 240 instead of 250. This is also how you'd change the interval of the keyframes: if you wanted keyframes to be at 20 second intervals then you'd change it to 500 for a 25fps video.

There are probably better ways to do this, such as the method described on the videotranscoding wiki, which I discovered after I'd written this script.


Back to video tips and tricks