Automated Avid, ProRes & LW format conversions - auto file renaming tools

notrons

I'm looking for people who are interested in automated .mts file conversions to ProRes and Avid DNxHD 4:2:2 formats, or for import into Lightworks, to test my new DSLR Post Prep Utility Windows application. The program is designed to automate the process of creating either and or both HQ and proxy conversions on demand, automatically by watched folder, or can be scheduled for off-hours. It uses the ffmpeg engine for all of the conversions (which run in their own minimized windows). When in watched directory mode it can also be minimized to the system tray where it continues to provide notifications and watch for new files. The following conversions are currently supported;

HQ Formats - 4:2:2

Uncompressed 4:2:2 10bit (working with LW)
ProRes 4:2:2 YUV 10bit 176Mbs HQ
DNxHD 4:2:2 YUV 10bit 176Mbs HQ
MPEG4 Wrapper for Lightworks
MPEG-2 4:2:2 I-Frame w/ M701 tag (for Lightworks only)
MPEG-2 4:2:2 I-Frame (w/out M701 tag) 



Proxy formats

ProRes 4:2:2 YUV 10bit 36Mbs
DNxHD 4:2:2 YUV 10bit 36Mbs 1080P 24
DNxHD 4:2:2 YUV 10bit 60Mbs 720P 24
DNxHD 4:2:2 YUV 10bit 145Mbs 720P 60
DNxHD 4:2:2 YUV 10bit 115Mbs 720P 50

DNxHD and ProRes HQ conversions seem nearly identical, but I think ProRes is a little better quality as a proxy format. DNxHD renders much faster than ProRes. I have a beefy workstation with Matrox hardware and it can handle all of the usable formats pretty easily but I would love to hear about how they work out for others. Please give it a try and let me know about the experience!

Check out http://hdcinematics.com/NewWeb/tools.html for more info. Also available are tools that add custom prefixes and timestamps to files on a directory-wide basis - to help keep clips uniquely named and organized (ExtensionCord). You might want to grab the Matrox and Avid codecs while you're there and download a copy of HD-MINI to audition clips on demand.

With this utility you can select a different flavor of HQ codec from proxy codec and have both conversions done automatically and off-hours. If you use ExtensionCord to add a prefix and (file creation) timestamp to your files before converting, you can edit with the proxies and then offline / link to the HQ clips of exactly the same name at render time.

I hope you can find some use for these ...; )

Cheers!

kavadni

It look very useful, does it include timecode information in the output files?

Would it be difficult to allow user to set the timecode? .. haha oops just read handy other tools :)

Thankyou very much

rigs

Thanks for this. I use Avid and will give it a go.

kavadni

I am on Avid too, if can have (virtually one click) timecode matched offline and online copies in DNxHD .. voila!

Ebacherville

Vegas Pro 10, drag and drop the MTS to the timeline, then start editing.

Iv'e worked in Premier and Final Cut and Vegas, Vegas just seems to do work best for my work flows with many sources of video, (I'm inhouse media specialist for a medium sized company) when you deal with every pro format under the sun from Prores to beta tape, and every consumer format imaginable. Vegas saves more more time and hassle than any other program. I do also use some free utilities for some conversions of odd formats. I run Vegas so much my mac is bootcamped and in Windows more than in OSX.

One of the free windows utilities I use all the time is Any Media Converter free edition ( http://www.any-video-converter.com/products/for_video_free/ ) but rarely do I need to convert formats for editing. Any media converter I use manly for batch conversions for say, converting to mp4 and webm for website video embeds etc.. that little utility just rocks for taking hi-res stuff and downsizing it and compressing it for whatever needs you may have. Total customization on dimensions and bitrates etc.

notrons

@Ebacherville - I think that most workstations are powerful enough to also drop .mts files onto a Premiere Pro timeline and slice & dice 'till the wee hours but this program (and approach) is basically designed for those wishing to convert the original 4:2:0 8bit files to 4:2:2 10bit for color correction, grading and other post-processing fun. I personally have found that using the files with increased color space will provide greater latitude and hold up better in post, and have seen enough discussion to help bear this out. It is great to know that Vegas is a viable alternative to Adobe and Apple products and I'm sure that those researching NLE's will appreciate the info! Perhaps you can find some use for one of the tools I have created someday ; )

@videohq - yeah, I know exactly what you are asking and that was actually my original idea. It didn't take long for me to realize that the development time for such a project would be more than I can currently commit - and maybe more than makes practical sense - it's hard to judge demand. If you are an Adobe Media Encoder user you can easily automate Matrox conversions using a watched folder, (see my website). I went with ffmpeg as it is well established and tested.

I did a lot of testing between codecs. Matrox wins for speed and quality and is lightning fast with Matrox MX02 hardware - it also does very well without the Matrox hardware. Normally I convert to Matrox uncompressed 10bit 4:2:2 and can edit, color grade, etc. with the HQ footage. If the footage is especially toothy I will convert to I-Frame intermediate to edit and composite with then link to uncompressed for rendering. Visually it is hard to tell any difference. The Avid codec comes in a close second for speed and quality and ProRes is great quality but considerably slower. In the end the Matrox, Avid and ProRes HQ formats are very hard to tell apart - and they all handle very similarly. If you are not running Media Encoder you should give the DNxHD Avid codec a try...

notrons

@videohq - that's a good piece of info - let me do some research, I might be able to oblige with mpeg I-Frame down the road...

notrons

@videohq - try this slight variation of your command-line under Linux

ffmpeg -i [input.mts] -vcodec mpeg2video -pix_fmt yuv422p -qscale 1 -qmin 1 -intra -acodec pcm_s16le [output.m2v]

...only differences are audio bit order and output to .m2v. MediaTab identifies the .mov version as MPEG-4 Quicktime while the .m2v reports MPEG-2. If this conversion works for you I will add it to the available codecs... if not I'll keep digging ; )

Roberto

Since you're in development/consultancy mode here, can I add this to the wish-list? I'd like a GUI to specifically address the FFMPEG library via the FFMBC customised set:

see http://www.personal-view.com/talks/discussion/2169/using-ffmbc-ffmpeg-for-broadcast-for-avchd-rewrapping-conversion#Item_15

And use a notebook for editing; render farm for crunching. (Just a personal project thing)

I'd prefer a Linux GUI but will go with whatever OS; even WINE if it works. Like others, I'm looking at Lightworks.

notrons

@Roberto et al. - after doing some more research and testing I found ffmbc conversions more flexible and predictable so have changed the render engine to ffmbc and limited the codecs that I found to be most consistent. If you have downloaded a version of DSLR Post Prep already, please uninstall and re-download the new version - it is better behaved ; )

As far as the next development goes, I think I'll sit back a bit and watch the discussion to figure the best way to go. Linux GUI? That's an interesting concept ; ) If there are enough Linux users interested I can certainly write a menu-driven application or perhaps a Tcl/TK GUI that will run on just about any Linux (or Mac or even Windows) system - but a dedicated GUI is out of the question without standardizing on one flavor and compiler version of Linux.

@videohq - I snagged a copy of Lightworks and have yet to be able to come up with a usable .mts conversion using either ffmpeg or ffmbc - even using their command-line recommendations. More research to do...

[edit] I should mention that an upgrade to the latest version of Quicktime was required to get LW to ingest my files. After the QT upgrade they worked just fine...

notrons

@videohq - yep, I got that too. I was able to get .mts files into an MPEG4 wrapper and imported right into LW. I did let LW recode which was pretty quick. I didn't try to edit but got the feeling it would be sluggish.

I added the wrapped MPEG4 format to the HQ codecs list so you can automate the process - running multi-threaded it is very fast. If you could re-download from the original link, uninstall and re-install I would appreciate any test feedback you could provide. In the meantime I will look into coming up with a bit more economical intermediate codec...

[edit] After rechecking on my test machine I also find that the MPEG-2 I-Frame format files work perfectly in Lightworks - but do not play in VLC or WMP. They also work just fine in Premiere Pro CS5.

Thanks again for the feedback! We'll get this thing ironed out yet! ; )

Rambo

Notrons, not working for me i have FFmpeg installed ( from http://ffmpeg.zeranoe.com/builds/) on a window7 laptap and your utility but your program doesn't display the ffmpeg version on startup ?? I tried installing the ffmpeg version linked on your website and windows says compatible with this version of windows.

Update, i see you're now using ffmbc in your program, does that negate the need to have ffmpeg installed?? Thanks

Rambo

Ok i've downloaded the 32bit version of ffmbc and swapped it out (the one in your program is 64 hence the reason for the incompatibility message ) DSLR Post Prep Utility loads, displays the ffmbc version, i load the mts files, set a render directory, press convert and the program tries to write the files to the origin directory instead of the destination but nothing is converted. Any thoughts.?

Roberto

@notrons

"......so have changed the render engine to ffmbc and limited the codecs that I found to be most consistent"

Wow, Xmas so soon? Thanks!

notrons

@Rambo - good catch. I forgot I was developing with a 64-bit version!

I have created a alternate 32-bit version that is ready-for-download. If you go back to

http://hdcinematics.com/tools/DSLRPostPrep.html

and re-enter your email address it will send you the links to both versions...

pc_bel

@notrons

For me proxy files are to much quality. When I add transitions, effects and color correction my computer becomes choppy. Better editing if they had options with less bitrate and may be options for less size (pixels). I run a Windows 7 64bit, i7 2600 3.4Ghz, 12 Gb RAM, WD Caviar Green HDs.

Anyway great job!!!! Thank you very much!!!!

notrons

@pc_bel - I will definitely look into some more economical codecs - thanks for the good word! ; )

YOSS

@notrons -Sorry for the ignorance... but I'm having trouble running "DSLR Post Prep" It doesn't covert any file to the output directory. Do I need to also install "ffmpeg " How do I do that?

notrons

@Rambo - I think it would be best to uninstall the version you have now, resubmit your email address on the website and get the link to the 32bit version. I have tested it with the 32bit version of ffmbc that it ships with and it has been consistent. I do not have a 32bit system to test on however so bear with me! If you still have troubles let me know! Sorry for any confusion...

@YOSS - a couple of things to look for. First, if you are running a 32bit version of Windows you will need the new version of DSLR PPU for 32bit. Re-visit http://hdcinematics.com/tools/DSLRPostPrep.html and resubmit your email address to receive the new links. As @Rambo had mentioned, I neglected the fact that I had shipped the original version with a 64bit build of ffmbc (the conversion engine) which will not work on 32bit Windows.

When you start PPU you should see the following message in the Log Window;

FFmbc is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License.

FFmbc is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with FFmbc; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

...if you do not see the message, and you are certain that you have a version appropriate to your operating system please let me know and I can provide additional troubleshooting info. I will also be working up that information on the website as I compile it.

YOSS

@notrons I'm running W7 64Bit. I'm seeing the ffmbc message at the beginning.

I already received the mail with both links. I downloaded the first one (DSLRPostPrepSetup.msi) as I supposed was 64 bit version (as my OS). The file was around 4.4mb and strangely auto-installed on the x86 Program Files directory.

I later tried the other file (DSLRPostPrepSetup32.msi), also around 4.4m, which also installed on the x86 directory. I supposed it should also work as any other 32 bit software installed on my 64 bit OS.

I got the same response with both installers after hitting convert; "The output directory is empty" ... it never does the conversion, the message "converted" appears almost instantaneously.

...see attached screen capture from the interface

Thanks!

notrons

@YOSS - wow, it looks like the conversion happened - but the GUI really has no way of knowing that the ffmbc conversion totally succeeded. Did it actually give you a message box with the error "The output directory is empty"?

The conversion work actually takes place in a DOS window that starts up minimized and disappears after each file is processed - you won't see anything conversion-wise happen in the Log Window, and with the prores codec it may take quite some time to convert so watch for the minimized DOS window... (Hope that makes sense! ; ) Please run it again and watch for the minimized DOS session - even if ffmbc fails it will still start, but it does close pretty quickly - you will see it on the taskbar though. Please let me know what you find. Also if you could try with different codecs that would further help me pin down the problem.

The other thought is permissions on your D:\ drive - make sure you have user rights there (although if you created the folder you probably have the rights ; ) Otherwise I am at a bit of a loss. If you are willing to help troubleshoot I will figure out a way to get you going...

[edit] One other thing - I see you are using the original filename "00003.mts" - if you already have a file in your target directory that has the same name, but with extension .avi, .mov, or .m2v, the program assumes that you have already converted that file and it will not convert.

Rambo

Notrons, what's happening is the program isn't always exporting to the output directory shown in the output box so it defaults to the import directory. I had this issue as well. Therefore the files don't get converted.

YOSS

@notrons "The output directory is empty" -no message like that.. it's just me saying it.

-As soon as I hit convert I see a very fast shake on the task bar (like something opening and closing very fast), but never see the dos window or the prompt icon on the task bar.

-I've tried different codecs & no success.

-I've all the permissions on mi D partition.

-I think it might be a problem with the 32 / 64 bit installer since both of them auto installed on the x86 Program Files directory. -& you said you haven't tested the 32 bit one.. (attached properties from both installers)

notrons

@Rambo, @YOSS - thanks for the details - I'll continue to dig. @Rambo, how did you cure the issue? By re-selecting a target directory?

YOSS

My output directory is empty.

notrons

@YOSS - I think I found the problem. One thing the program does not like is converting into the same parent directory as the originals. I see your original files are in "D:\GH2 Tests\rocket v3b" and your conversion target directory is "D:\GH2 Tests\MOV". This won't work - instead select a target directory outside of the parent, i.e. "D:\Converted" and it should work.

Also, since ffmbc is a DOS-based program it does not like spaces in directory names. Instead of "GH2 Tests" try "GH2_Tests" instead. I just tested and it will not work with spaces...

This point I will address in the next version - currently you cannot type in directory paths directly, you must double-click the text box and use the directory picker to select the target dirs.

I think if you address these issues it will work. I am just now starting to realize how much I take for granted ; ) Thanks for your feedback - please let me know how it goes...

Howdy, Stranger!

Categories

Tags in Topic

Top Posters