Time to select a projet, profiling ffmpeg2theora

Time has come to choose a project for the DPS915 CUDA programming course.

After looking online, a project that caught my eyes was the ffmpeg2theora

ffmpeg2theora is built on top of the ffmpeg project and its goal is to provide a command line interface to convert videos to the theora format wrapped in an Ogg container

My idea for the project is to add GPU optimization support to the converter, specifically using the CUDA api for Nvidia graphics cards.
At the moment it is not clear how or if it’s even possible to do that since the converter itself has a lot of dependencies and talking with some developers in the #vorbis channel I was told that the optimizations would have to be done in the libtheora and a big chunk of the library it’s already written in assembly for performance reasons.

So for now I’m trying to gather as much information as possible.

Here is a list of some resources relevant to the research

• FFmpeg
• Theora
• Vorbis
• Libav
• MPEG
• WebM
• Matroska
• Ogg
• Entropy Encoding
• MPlayer
• Transcoding
• libavcodec
• libkate

To get an idea of the project I decided to build it and play around with the converter

As I would expect to build ffmpeg2theora from source requires a bunch of dependencies.
The developers created two scripts that make the process easy.
One script clones the latest stable release of ffmpeg from their git repository and builds it and the other does the same thing but for libkate

Besides installing ffmpeg and libkate I also needed to install

• libvorbis
• libogg

On Ubuntu I also had to install

• yasm
• gawk

The Build system

For the build system they use SCons
SCons is a software construction tool implemented in Python, it is a replacement for the famous make.

I have to say that at first I was kind of skeptic, but after reading their user docs and hacking around some scripts I fell in love immediately.
SCons doesn’t try to solve all the problems in the world, but they take a very pragmatic approach towards build tools and have some info to back it up.

Here is the SCons script used in the ffmpeg2theora projet:

# SCons build specification
# vi:si:et:sw=2:sts=2:ts=2
from glob import glob
import os

import SCons

def version():
    f = os.popen("./version.sh")
    version = f.read().strip()
    f.close()
    return version

pkg_version="0.29"

pkg_name="ffmpeg2theora"

scons_version=(1,2,0)

try:
    EnsureSConsVersion(*scons_version)
except TypeError:
    print 'SCons %d.%d.%d or greater is required, but you have an older version' % scons_version
    Exit(2)

opts = Variables()
opts.AddVariables(
  BoolVariable('static', 'Set to 1 for static linking', 0),
  BoolVariable('debug', 'Set to 1 to enable debugging', 0),
  BoolVariable('build_ffmpeg', 'Set to 1 to build local copy of ffmpeg', 0),
  ('prefix', 'install files in', '/usr/local'),
  ('bindir', 'user executables', 'PREFIX/bin'),
  ('mandir', 'man documentation', 'PREFIX/man'),
  ('destdir', 'extra install time prefix', ''),
  ('APPEND_CCFLAGS', 'Additional C/C++ compiler flags'),
  ('APPEND_LINKFLAGS', 'Additional linker flags'),
  BoolVariable('libkate', 'enable libkate support', 1),
  BoolVariable('crossmingw', 'Set to 1 for crosscompile with mingw', 0)
)
env = Environment(options = opts)
Help(opts.GenerateHelpText(env))

pkg_flags="--cflags --libs"
if env['static']:
  pkg_flags+=" --static"
  env.Append(LINKFLAGS=["-static"])

if env['crossmingw']:
    env.Tool('crossmingw', toolpath = ['scons-tools'])

prefix = env['prefix']
if env['destdir']:
  if prefix.startswith('/'): prefix = prefix[1:]
  prefix = os.path.join(env['destdir'], prefix)
man_dir = env['mandir'].replace('PREFIX', prefix)
bin_dir = env['bindir'].replace('PREFIX', prefix)

env.Append(CPPPATH=['.'])
env.Append(CCFLAGS=[
  '-DPACKAGE_VERSION=\\"%s\\"' % pkg_version,
  '-DPACKAGE_STRING=\\"%s-%s\\"' % (pkg_name, pkg_version),
  '-DPACKAGE=\\"%s\\"' % pkg_name,
  '-D_FILE_OFFSET_BITS=64'
])

env.Append(CCFLAGS = Split('$APPEND_CCFLAGS'))
env.Append(LINKFLAGS = Split('$APPEND_LINKFLAGS'))

if env['debug'] and env['CC'] == 'gcc':
  env.Append(CCFLAGS=["-g", "-O2", "-Wall"])

if GetOption("help"):
    Return()

def ParsePKGConfig(env, name):
  if os.environ.get('PKG_CONFIG_PATH', ''):
    action = 'PKG_CONFIG_PATH=%s pkg-config %s "%s"' % (os.environ['PKG_CONFIG_PATH'], pkg_flags, name)
  else:
    action = 'pkg-config %s "%s"' % (pkg_flags, name)
  return env.ParseConfig(action)

def TryAction(action):
    import os
    ret = os.system(action)
    if ret == 0:
        return (1, '')
    return (0, '')

def CheckPKGConfig(context, version):
  context.Message( 'Checking for pkg-config... ' )
  ret = TryAction('pkg-config --atleast-pkgconfig-version=%s' % version)[0]
  context.Result( ret )
  return ret

def CheckPKG(context, name):
  context.Message( 'Checking for %s... ' % name )
  if os.environ.get('PKG_CONFIG_PATH', ''):
    action = 'PKG_CONFIG_PATH=%s pkg-config --exists "%s"' % (os.environ['PKG_CONFIG_PATH'], name)
  else:
    action = 'pkg-config --exists "%s"' % name
  ret = TryAction(action)[0]
  context.Result( ret )
  return ret

env.PrependENVPath ('PATH', os.environ['PATH'])

conf = Configure(env, custom_tests = {
  'CheckPKGConfig' : CheckPKGConfig,
  'CheckPKG' : CheckPKG,
})

if env["build_ffmpeg"]:
  if env.GetOption('clean'):
    TryAction("cd ffmpeg;make distclean")
  else:
    TryAction("./build_ffmpeg.sh")

if not env.GetOption('clean'):
  pkgconfig_version='0.15.0'
  if not conf.CheckPKGConfig(pkgconfig_version):
     print 'pkg-config >= %s not found.' % pkgconfig_version
     Exit(1)

  if not conf.CheckPKG("ogg >= 1.1"):
    print 'ogg >= 1.1 missing'
    Exit(1)

  if not conf.CheckPKG("vorbis"):
    print 'vorbis missing'
    Exit(1)

  if not conf.CheckPKG("vorbisenc"):
    print 'vorbisenc missing'
    Exit(1)

  if not conf.CheckPKG("theoraenc >= 1.1.0"):
    print 'theoraenc >= 1.1.0 missing'
    Exit(1)

  XIPH_LIBS="ogg >= 1.1 vorbis vorbisenc theoraenc >= 1.1.0"

  if not conf.CheckPKG(XIPH_LIBS):
    print 'some xiph libs are missing, ffmpeg2theora depends on %s' % XIPH_LIBS
    Exit(1)
  ParsePKGConfig(env, XIPH_LIBS)

  FFMPEG_LIBS=[
      "libavdevice",
      "libavformat",
      "libavfilter",
      "libavcodec >= 52.30.0",
      "libpostproc",
      "libswscale",
      "libswresample",
      "libavutil",
  ]
  if os.path.exists("./ffmpeg"):
    pkg_path = list(set(map(os.path.dirname, glob('./ffmpeg/*/*.pc'))))
    pkg_path.append(os.environ.get('PKG_CONFIG_PATH', ''))
    os.environ['PKG_CONFIG_PATH'] = ':'.join(pkg_path)
    env.Append(CCFLAGS=[
      '-Iffmpeg'
    ])

  if not conf.CheckPKG(' '.join(FFMPEG_LIBS)):
    print """
        Could not find %s.
        You can install it via
         sudo apt-get install %s
        or update PKG_CONFIG_PATH to point to ffmpeg's source folder
        or run ./get_ffmpeg.sh (for more information see INSTALL)
    """ %(" ".join(FFMPEG_LIBS), " ".join(["%s-dev"%l.split()[0] for l in FFMPEG_LIBS]))
    Exit(1)

  for lib in FFMPEG_LIBS:
      ParsePKGConfig(env, lib)

  if conf.CheckCHeader('libavformat/framehook.h'):
      env.Append(CCFLAGS=[
        '-DHAVE_FRAMEHOOK'
      ])

  KATE_LIBS="oggkate"
  if env['libkate']:
    if os.path.exists("./libkate/misc/pkgconfig"):
      os.environ['PKG_CONFIG_PATH'] = "./libkate/misc/pkgconfig:" + os.environ.get('PKG_CONFIG_PATH', '')
    if os.path.exists("./libkate/pkg/pkgconfig"):
      os.environ['PKG_CONFIG_PATH'] = "./libkate/pkg/pkgconfig:" + os.environ.get('PKG_CONFIG_PATH', '')
    if conf.CheckPKG(KATE_LIBS):
      ParsePKGConfig(env, KATE_LIBS)
      env.Append(CCFLAGS=['-DHAVE_KATE', '-DHAVE_OGGKATE'])
    else:
      print """
          Could not find libkate. Subtitles support will be disabled.
          You can also run ./get_libkate.sh (for more information see INSTALL)
          or update PKG_CONFIG_PATH to point to libkate's source folder
      """

  if conf.CheckCHeader('iconv.h'):
      env.Append(CCFLAGS=[
        '-DHAVE_ICONV'
      ])
      if conf.CheckLib('iconv'):
          env.Append(LIBS=['iconv'])

  if env['crossmingw']:
      env.Append(CCFLAGS=['-Wl,-subsystem,windows'])
      env.Append(LIBS=['m'])
  elif env['static']:
      env.Append(LIBS=['m', 'dl'])

# Flags for profiling
env.Append(CCFLAGS=['-pg'])
env.Append(CCFLAGS=['-g'])
env.Append(CCFLAGS=['-DDEBUG'])
env.Append(LINKFLAGS=['-pg'])

env = conf.Finish()

# ffmpeg2theora
ffmpeg2theora = env.Clone()
ffmpeg2theora_sources = glob('src/*.c')
ffmpeg2theora.Program('ffmpeg2theora', ffmpeg2theora_sources)

ffmpeg2theora.Install(bin_dir, 'ffmpeg2theora')
ffmpeg2theora.Install(man_dir + "/man1", 'ffmpeg2theora.1')
ffmpeg2theora.Alias('install', prefix)

The script just set some configurations for the build and checks for some dependencies.
I added some extra flags because I wanted to generate a profile of the application

env.Append(CCFLAGS=['-pg'])
env.Append(CCFLAGS=['-g'])
env.Append(CCFLAGS=['-DDEBUG'])
env.Append(LINKFLAGS=['-pg'])

Summarizing the steps to build ffmpeg2theora:

Download the source code

Run:

sudo ./get_ffmpeg.sh
sudo ./get_libkate.sh
sudo scons
sudo scons install

**If you need to install any other dependencies the configure scripts will output to the terminal
**On the mac I had some problems in running “sudo scons”, the pkg-config path would get corrupted and the build would fail, by loggin in the shell as root and sourcing the environment variables of my profile solved the problem (I didn’t have this issue on Ubuntu)
**If you don’t run the get_ffmpeg script as root the libraries won’t be installed in the system and the build will fail during the linking stage

Profiling

Next step was to generate a profile of the program and see which area of the application was consuming most of the CPU time.
I used the Instruments Timer Profiler to create a profile of the application.
I have previously blogged about how to use the Instruments Timer Profiler on the mac

instruments -t "/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/Resources/templates/Time Profiler.tracetemplate" ./ffmpeg2theora myvideo.mp4

and the profile information was generated:

Looking at the profile gave me a better idea of how the converter works but I still need to run the converter with a larger video to see where the heavy processing takes place.

What’s next?

This is a very intimidating projet considering that I’m not very familiar in video encoding and CUDA programming, but what better way to learn something than by doing it?
I still remember taking the Topics in OpenSource Development last year with David Humphrey here at Seneca college and how we started hacking Firefox. At the beginning it was very hard and overwelming, but after a while the beast didn’t look as scary as before. That just proved to me that as long you put the time into something you will get the results no matter what. In the end hard works does pay off, indeed.

With that being said, I’m a little scare about diving into an area that I don’t much about and trying to implement something new, but at the same time I welcome the challenge and I will try to learn as much as I can during the process.
Video Processing and GPU Programming are two topics that interest me so I’m sure it will be a lot of fun