Home > Advanced Linux-related things, Linux > MP3 to Video using GStreamer visualizations

MP3 to Video using GStreamer visualizations

January 25th, 2011

VLC showing a sparkly shiny visualization
Everyone loves music visualization, but not all apps support it in a sensible way. Maybe you want to shuffle a random assortment of video and audio files in a player that doesn’t handle that well (VLC!), or not at all (mplayer!). Or maybe you want to upload something to youtube, with gorgeous HD visualizations instead of that lame static cover art image?

The few google results on the topic that weren’t spam suggested screencapping software. Yeah, that’s great… until you have more than two files.

Once again, everyone’s favourite multimedia swiss army knife – GStreamer – steps up to the plate.

Here’s an example of encoding an MP3 to an H.264 .mkv file using the gorgeous goom visualizer (requires the mp3 and x264 plugins for gstreamer):


gst-launch filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mp3parse ! matroskamux name=mux ! filesink location="output.mkv" stream. ! queue ! mp3parse ! mad ! audioconvert ! queue ! goom ! ffmpegcolorspace ! video/x-raw-yuv,width=1280,height=720 ! x264enc ! mux.

It’s beautiful – and the video is pretty sweet as well.

It’s worth noting that this approach does not re-encode the MP3, like some less awesome approaches would do (causing loss of quality). It simply muxes it together with the visualizer’s video stream. x264 even seems to distribute itself well across cores.

No, wait, what? MP3 and H.264? Of course, I meant Vorbis and Theora! Let me rephrase:


gst-launch filesrc location=input.ogg ! queue ! tee name=stream ! queue ! oggdemux ! vorbisparse ! oggmux name=mux ! filesink location="output.ogg" stream. ! queue ! oggdemux ! vorbisdec ! audioconvert ! queue ! goom ! ffmpegcolorspace ! video/x-raw-yuv,width=1920,height=1080 ! theoraenc ! mux.

The same goodness applies, except for the parallelism. If you have a multicore CPU, there’s massive speedup to be had through simple shell script based multithreading. (Why full HD this time? VLC on Windows crashes on 720p Theora!)

And there you have it. A simple, hack-free, modular and flexible way of encoding visualization videos for MP3 and Ogg Vorbis files. Thanks, GStreamer!

Advanced Linux-related things, Linux , ,

  1. Vidar
    January 25th, 2011 at 23:28 | #1

    If you’re creating your own pipeline and GStreamer gets stuck during preload, try adding more queue elements.

    If you get errors like “/GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data flow error”, read further down in the error output. If it says “reason not-negotiated (-4)”, you’re probably missing some suitable converters like audioconvert or ffmpegcolorspace.

  2. November 18th, 2012 at 01:33 | #2

    updated the mp3 version for 1.0

    gst-launch-1.0 filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mpegaudioparse ! matroskamux name=mux ! filesink location=\output.mkv\ stream. ! queue ! mpegaudioparse ! mad ! audioconvert ! queue ! goom ! videoconvert ! video/x-raw,forma=yuv,width=1280,height=720 ! x264enc ! mux.

  3. November 18th, 2012 at 11:50 | #3

    so much for my editing skills corrected below:

    gst-launch-1.0 filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mpegaudioparse ! matroskamux name=mux ! filesink location=\output.mkv\ stream. ! queue ! mpegaudioparse ! mad ! audioconvert ! queue ! goom ! videoconvert ! video/x-raw,width=1280,height=720 ! x264enc ! mux.

  4. frente69
    July 18th, 2013 at 02:13 | #4

    Wow! Works great. I have been trying to figure this out for ages! I have also been playing with some of the other visualisations(libvisual) but they seem to do weird things. I encoded a song that was 4:00 minutes long but the end result reported it was 6:07 with the last two minutes causing most media players to pause. odd.

  5. Vidar
    July 18th, 2013 at 18:48 | #5

    @frente69
    Odd indeed. Could it be a sync or frame rate issue? If nothing else, I guess there’s ffmpeg to crop it :/

  1. No trackbacks yet.

Ignore the following captcha and enter 'bunnies' in the field instead.