Personal View site logo
Make sure to join PV on Telegram or Facebook! Perfect to keep up with community on your smartphone.
Please, support PV!
It allows to keep PV going, with more focus towards AI, but keeping be one of the few truly independent places.
GH2 - 2.5fps AVCHD 24h Timebuster/TimeHBusteR 2.0 settings - The day is not over!
  • Timelapses can be amazing, beautiful, strange, provocative. And FUN!

    The day I discovered the GH2 could do 2fps I got so excited, I loaded up a full battery, put the camera on the balcony and aimed it at the shopping mall door. I got over 3+1/2h of Cinema 24p footage (@2fps) and after speeding it up 15x (to 300fps) in VirtualDub I almost couldn't believe my luck. This is what I got and I have yet to grade it properly:

    Since that event, I wondered about the best way to patch up the camera to capture them. MJPEG was the obvious candidate. It's a Full Intra codec and it can be setup to capture just 2fps. That means absolutely no wasted frames unlike AVCHD, so MJPEG must be great, but there was a caveat: even with stock settings there was a capture time limit that was well below AVCHD. How is that possible since it's only capturing 2fps, the bitrates have got to be frugal? Well, it happens that there is a 2GB filesize limit within MJPEG so even after taking Driftwoods Quantum 100-50's MJPEG settings and cutting the quality to take half the bitrate, you will end with less than one hour of footage. Compare that to AVCHD and you see what got me started.

    Then there was the IQ issue. I stumped into this: http://sonnati.wordpress.com/2010/10/19/h-264-for-image-compression/ which made me start another full set of tests (http://www.personal-view.com/talks/discussion/2111/gh2-mjpeg-resolution-findings-and-myth-busting/p1) in which I found that unlike most crazy MJPEG resolutions that only seem to be 720p upscaled, the one used on Driftwoods patch was the best that ATM can be extracted from MJPEG and that even rivals with AVCHD. That test ended up on a tie, so in my book AVCHD wins for flexibility and recording time.

    At the same time these tests were taking place, I fought a lot with the stock settings trying to make a patch that used GOP 13 and yes, that's the frequency at which you see major block changes when you set the shutter speed to 1/2s. This GOP13 would insure excellent quality as it makes the result Full Intra once you trash the needed frames. But the frames and limits and buffers and modes were giving me all kinds of speed messages Hell. By that time I also discovered that the null B frames that encoded no changes at 2fps were much larger than P-Frames which also appeared a lot more frequently than I wished they would. Null B-Frames were 2.5x bigger than P and when there was movement, the P-Frames would take as much space (and sometimes even more) than I frames, so I had to do something about it to make it more efficient. Meanwhile Nick @Driftwood and @LPowell were kind enough to share their knowledge and let me in on a few tricks. These allowed me to suppress B-Frames altogether and starve P-Frames to death.

    So here I was armed with a bazooka and a missile and I couldn't even fire a pistol. Then a few days ago @Balazer came out with his Cake patch featuring a completely VBR approach through a Constant Quantizer as if he couldn't care less about bitrates, limits or buffers or all the stuff that was putting me down. I tried it a 2fps and I couldn't believe how well it worked. And since the frame rate I'm using is ridiculous when compared to normal video modes, there are no speed/buffer/spanning issues even if you have the slowest SD card in the business. In this case, this VBR aproach is a no brainer and the cherry on top is that you get Constant Quality at Quantizer 22 which is very very good IQ!

    So, hands to work, bake the Cake, mix Driftwood's MPEG settings, apply the P/B-Frame tricks and it was working on the third try. :O

    It could have been a lot faster as I tried to best the 60i mode and leave all other modes alone. It ended up being the wrong choice because the 60i GOP isn't sticking and probably the Interlacing is making too big P-Frames around the I-Frames. In the end I managed to use tweak 24p for timelapses and the frame usage when there is no movement is a beautiful set of I-Frame spikes with all else almost null. 60i is now GOP13 on I/P frames exclusively and the P sizes look generous, so the starved P-Frames on 24p don't seem to affect it. 720p modes seem also unaffected. In the end this patch tries to give you the best 2fps and still leave you with 2 very workable modes without repatching: 1080i60 and 720p.

    USE IT AND ABUSE IT. DON'T forget to weed out the starved frames in post. AfterEffects instructions here: http://www.personal-view.com/talks/discussion/comment/49325#Comment_49325 and freeware workflow is here: http://www.personal-view.com/talks/discussion/comment/39079#Comment_39079

    Thank you @balazer for baking Cake.

    Thank you @drifwood for the MJPEG settings and no B-Frame trick.

    Thank you @lpowell for telling me how to starve the P-Frames.

    Thank you @cbrandin for StreamParser 2.4 which gave me wonderful new numbers.

    .

    VERSION HISTORY :

    1 - TimeBuster 1 - First Release

    • Based on Cake v1. It uses GOP13, a Constant Quantizer VBR approach without B-Frames, minimum P-Frame size and quality.

    2 - TimeBuster 1.1 - with 2 significant changes:

    • Now based on @balazer 's Cake 1.1, it has switched to use the Rate Control mechanism from @LPowell 's FlowMotion 100.

    • 720p mode was broken due to use of a very low quantizer (12) which lead to skipped frames in 720p. It now uses QP=20 on both 720 & 1080 modes which will have an observable effect in both bitrate & quality even though it still gives you 24+ hours on 64GB.

    3 - TimeBuster 1.2 - with 1 significant change for a fully featured 720p mode:

    • Since @balazer 's Cake 1.1 is now heavily based on @LPowell 's FlowMotion 100, I decided to copy LPowells 720p settings since they will probably play better with the other modes. The only difference on 720p is a constant QP which had to be lowered yet from 20 to 21 because it still dropped frames on torture charts and/or sometimes present speed issues at very high ISO (>3200) on Class 10 SDs.

    4 - TimeBuster 1.3 - with a lot of significant changes:

    • Ported to PTTools 3.64d

    • Savings on P-Frames is now accomplished by using a Scaling Table that is full of 0xFFFF for 1080p modes.

    • Quantizer boundaries were removed since they are now much more dependent on the new Quantizer Tables.

    • Redesigned for merging, these definitions try to be as independent as possible from their re-packaging definitions (in this case FlowMotion 1.1) and are published in two flavors: Timebuster 1.3 Base (for merging) and Timebuster 1.3 FloMo (merged with FlowMotion 1.1)

    NOTE1: Timebuster Base 1.3 was merge tested with @LPowell 's FlowMotion 1.1, @Balazer 's Cake 1.2 and Nick @Driftwood 's Quantum X (v2) with minimal impact on all modes except for 1080p. I recommend FlowMotion or Cake for the simple reason that the 1080p24 Frame Limit is set to a much higher value in these settings. In fact, if you care for nothing other than the 24p timelapse IQ and don't mind impacting some of the other modes, you should set it to 10,000,000.

    NOTE2: Merging Timebuster Base 1.3 with other settings is easiest done by loading Timebuster Base 1.3 first and then Alt+Clicking your choice of merger.

    5 - Timebuster is now comprised of 2 main variants:

    a) 24h Timebuster 2.0 - with the following changes (from TimeBuster 1.3):

    • The GOP for 24p is now married to 2.5fps. The bitrate hit is a little above 20% but the recording length can be easily compensated by raising the QP. As a benefit it's still very close to 2fps and it's the closest to a 360º shutter we can get on 24p.

    b) 24h TimeHBusteR 2.0 (NEW, uses HBR 30)

    • Similar philosophy to 24h Timebuster 2.0 Base but as the name suggests, uses HBR mode instead of 24p. There is another difference, P-Frame quality is fully preserved as this kind of frames is used for one of the interlaced slices on Key Frames.

    • GOP married to 1/2.5 on HBR 30p (NTSC) and to 1/5s on HBR 25p (PAL). The big benefit is that both are using a 360º shutter. Uncheck "1080i50 and 1080p24 GOP Size=5" before merging if you don't plan to use Shutter Speed (SS) =1/5s and want to better preserve the 24p mode and HBR 25.

    Both variants are now:

    • Using B-Frames. There is a 6% penalty in bitrate usage but this was necessary to make sure the QP was respected. I believe the gain from setting the QP at will is just too great to ignore, since not everyone wants to record for 24h straight. QPs as low as QP=16 were tested and theoretically it could go as low as QP=12 @ISO160 when merged with FlowMotion 1.11 on regular tripod mounted footage.

    • GOP married to 2.5fps. The bitrate hit is a little above 20% but the recording length can be easily compensated by raising the QP. The reason behind this was getting as close as possible to 360º.

    • User definable GOP - GOP can be set at will according to intended SS, use the attached "GOP-SS-table.png" to look up the right value. Defaults settings are SS=2.5fps and GOP 1080i=10 for 24h Timebuster 2.0 and SS=2.5/5fps and GOP 1080i50/i60=12/5 for 24h TimeHBusteR 2.0.

    • User definable QP - QP can be set at will according to desired IQ/recording length. I've gathered quite an amount of information so that in the future I can make an Excel tool to estimate the best QP from Card Size/Footage Snapshot/Desired ISO/Recording Length. Default settings are Q24/23 for 24h Timebuster/TimeHBusteR.

    As before, there is the Base Release to be merged at wish with popular settings (load Base first in PTTools and then "ALT+Click" your choice of merger) and there is also the FlowMotion Release (already merged with FlowMotion 1.11). I've merge tested against @lpowell 's FlowMotion 1.11, @balazer 's Cake 95, and @bkmcwd 's GOP3ZILLA and no undesirable effects were observed. The key is using a VBR settings definition with a very high "1080p24 Frame Limit", especially if you are going for a low QP (<22).

    (edit) If you want to merge with Nick @driftwood 's settings, I recommend going for Cluster v1 as shines in being both a long GOP set and also having the biggest "1080p24 Frame Limit" of all the sets I know.

    6 - Updated Release of Timebuster/timeHBusteR 2.02 FlowMotion - WARNING HBR version IS BROKEN at 24p

    • (EDIT) This turned out to be a rushed release. Several users have reported problems with timeHBusteR (the HBR variant) It looks like FlowMotion 2.0(+) is highly dependent on GOP length (trough heavy use of GOP Tables, GOP Opt Tables and GOPx2 parameters). I'm investigating other possibilities right now so watch this space for further news. In the meantime I recommend you stick to timeHBusteR /Timebuster v2.0.

    Since FlowMotion 2.02 looks very stable now, I'm releasing my pre-merged versions of Timebuster/timeHBusteR. There is nothing new regarding timelapsing except for FlowMotion's 2.02 changes.

    • 24h Timebuster 2.02 FlowMotion which is 24h Timebuster 2.0 Base merged with FlowMotion 2.02

    • 24h timeHBusteR 2.02 FlowMotion which is 24h timeHBusteR 2.0 Base merged with FlowMotion 2.02

    7 - Updated Release of Timebuster/timeHBusteR 2.02 FlowMotion variants - 24p is now fixed on timeHBusteR. There was a note telling you should remove the "1080i50 and 1080p24 GOP Size=5" for better preserving 24p but no one seemed to read it and it was causing more trouble (to most of those who wanted 24p preserved) than it was worth (for those who wanted more timelapsing versatility.

    NOTE: The mergeable timeHBusteR wasn't updated! Just the the FlowMotion variant. I ASSUME that if you are using this base for merging that YOU KNOW WHAT YOU ARE DOING, and read the DOC.

    The day is not over. :)

    setj.zip
    952B
    TimeBuster 1.1.zip
    1K
    TimeBuster 1.2.zip
    1K
    Timebuster 1.3.zip
    1K
    Timebuster Base 1.3.zip
    876B
    GOP-SS-table.png
    867 x 439 - 51K
    24h Timebuster 2.0.zip
    2K
    24h TimeHBusteR 2.0.zip
    3K
    Timebuster Settings Documentation.zip
    29K
    24h (Timebuster + timeHBusteR) FlowMotion 2.02.zip
    3K
    Timebuster FlowMotion 2.02alpha.zip
    3K
  • 323 Replies sorted by
  • Here's what it looks like on 24p @2fps on static footage. Almost as good as 2fps MJPEG would look. And the only impairing I observed (on 60i) was the long GOPs, but with good P-Frame size. The beauty of the patch is that you can now easily trade IQ for recording length with very little changes to Cake (as I did). Probably all you need is to balance the frame limits with the QP.

    This is how 24H looks on a very detailed and colored chart. Average Total Bitrate = 7.012.276. On real life and detailed footage but with less texture, I got under 5Mbps (on 24H).

    On a 64GB SD this translates into 29 hours.

    THE DAY IS NOT OVER.

    2fps 24P.png
    1726 x 841 - 824K
    30fps 60i.png
    1726 x 841 - 159K
  • This is awesome! naf SAID!

  • @duartix, are you in lisbon?

  • Impressive stuff!

  • @eyefi:

    Yes, busted!

    Don't tell me you are the guy with the girl... :)

  • @duartix, ok I don't tell you. keep up the good work. Maybe we'll talk soon.

  • @duartix does it mean you would have created the perfect patch to combine timelapse at 2fps (which would give factor 13x as a base) with slow motion at 60i transferred to 24p (which would give factor 1/2.5 as a base) and all at 1080p.

    If you can combine it like I did it as shown below it would be an awesome timebuster with extreme possibilites of dynamic range...

  • @tida : Yes, well to be honest, keeping 60i was an accident. I was going for an all round patch trying to preserve 24p for regular shooting and dedicating just 60i for timelapses. But as I wrote before, something I didn't foresee made me reconsider: 60 is "i", which means that at 2fps when the footage changes are recorded by the camera from t=#0 they will be spread across 2 fields at frames #30 and #31 and to encode these changes properly in Full Intra, you would need GOP1 which defeats the purpose of getting long record times.

    In the end, I'm glad this happened because this is a timelapse dedicated patch and this way you can preserve the modes that work on extreme timing ends. 720p60 is preserved, 2fps @1080 has great recording times and IQ, and you still have 1080i60 with very good quality and resolution. I've tried to set 60i GOP length to GOP1 but as I said before, it doesn't stick, and the camera goes with 13 in that mode. (edit) Stupid me, (I was testing PAL, that's why it didn't look like it sticked) (/edit) It's not too bad but it's not ideal either, so if someone has an idea on how to change it, please let me know as I'd like to try it!

    It's all about choices, as much as I'd like to do an all round patch, the focus here is on timelapses and if you really need 24p at the same time, well then you might as well use another SD card with a better 24p patch on it. Provided you have 2 cards, you can change the patches on the field as many times as you need, all it takes is 3min at a time and a full battery (two things that timelapsers usually can spare :D ). Timelapses can be done with amazing quality in the smallest & cheapest cards, so I think the card is not an issue.

  • @duartix A couple questions:

    Does this patch actually record @ 2fps in AVCHD? or does the camera still record 24fps and you have to remove all of the duplicated frames? I couldn't find in the pTool settings where you switched 24fps ---> 2fps.

    Do you still have to use a 1/2 second shutter, Manual mode + manual focus?

    Could you write a simple workflow from start to finish (camera settings to post) about the timelapse video you've posted?

    This is very exciting, thanks for you time and efforts!

    ~J~

  • @Hallvalla : Yes, you still need to put the camera into MF so that you can then slow the shutter speed manually to 2fps. The camera will still record 24fps, the thing is that most of those frames will be subjectively NULL (they still occupy some minimal space) as you see in the first StreamParser capture. When there is significant movement however, some of the P-Frames surrounding the I-Frames will occupy some noticeable space.

    I'll try to capture something today and post it here. I'll try to post the workflow too.

  • @duartix : How you get the 2fps in camera? Sorry but I don't find the way.

  • @duartix

    Real approach to this must be ability to change encoder so it'll skip the frames and you must be able to specify how many you want to skip. It'll also require to completely turn off audio.

  • @rafa As it was said just before, you need to be on Manual Movie mode, and set the camera to MF. Only then can you reduce the shutter speed below 1/25(30)

    @Vitaliy_Kiselev : That would be really great as P-Frames sometimes still get hungry and audio also can account for up to 5% at the moment, any chances those would be possible?

    Until then, there is a saying here in Portugal that goes like this: "He who doesn't have a dog, hunts with a cat."

  • @Hallvalla:

    1 - Camera top Dial must be set to M Movie, video mode must be set at 24p (cinema) and focus to MF, with the rear dial set shutter speed to 1/2s (pressing the rear dial against the camera changes control from A to S).

    2 - Record.

    3 - Copy MTS file to disk .

    4 - Open VirtualDub (make sure you have the DirectShow filter installed http://forum.videohelp.com/threads/286901-VirtualDub-Now-Has-a-Directshow-Driver-Plugin )

    5 - Go to File Open (make sure that in "File of type" you select "DirectShow input driver") and open the file.

    6 - On Video menu select Frame Rate. There are two options on panel Frame Rate Conversion:

    a) Decimate by 12 (null frames will be deleted but you won't get the speed effect)

    b) Convert to 300fps (accelerated)

    7 - On Video menu configure the Compression codec.

    8 - Save file as...

    This is the easiest workflow I can imagine (on my video I applied some basic filters (menu Video | Filters) and used the Blend option to blend a B&W Fade Out and Fade In.

    The guaranteed better way to do it (to make sure you are only selecting I-Frames) should be with AviSynth and this post should give you an idea: http://www.personal-view.com/talks/discussion/1559/avchd-1080p-time-lapse/p1

    There are also blending options on After Effects but I know very little about it.

  • @duartix Thanks. Hmmm, just found out VirtualDub runs on Mac OS X through Wine (like pTools). Will give it a shot.

  • TimeBuster 1.1 is out, featuring 2 significant changes:

    • Since this is based on @balazer 's Cake 1.1 it has switched to using the Rate Control mechanism from @LPowell 's FlowMotion 100.

    • 720p mode was broken due to use of a very low quantizer (12) which lead to skipped frames in 720p. It now uses QP=20 on both 720 & 1080 modes which will have an observable effect in both bitrate & quality even though it still gives you 24+ hours on 64GB.

    The day is not over. :)

    TimeBuster 1.1.zip
    1K
  • @duartix thanks, looking forward to use it. If one would like to use higher shutter speed than 1/2s would the patch keep it's performance by just lowering the GOP. Would the calculation in below mentioned macro fit?

    http://www.personal-view.com/talks/discussion/1559/avchd-1080p-time-lapse#Item_61

    StepSize1 = abs(ShutterSpeed_s*FrameRate_fps)+1 //For your Patch StepSize1 == GOP13 at 1/2s

  • Let's see what are the most desirable SSs:

    • SS=1/2s then you get: 1/2*24+1=GOP13

    • SS=1/4s then you get: 1/4*24+1= GOP7

    • SS=1/6s then you get: 1/6*24+1= GOP5

    • SS=1/8s then you get: 1/8*24+1= GOP4

    • SS=1/12 then you get: 1/12*24+1 = GOP3

    (EDIT) THIS LOGIC ISN'T WORKING. Look further in this thread: http://www.personal-view.com/talks/discussion/comment/39686#Comment_39686

    Apparently the formula guarantees that you'll have a new I-Frame that is pretty much in synch with your later frame sampling rate and that the rest are tiny null P-Frames to be discarded. If I were in your shoes, I wouldn't go for other funny SSs like 1/5 or 1/10 since 1/4 and 1/8(12) are very close and will work even without scripting tools.

    In Timebuster 1.1, I've lowered the QP from 22 to 20 so we get a little better IQ but still get around 24h. In principle the patch will keep it's IQ performance but it will pay for that in bitrate and recording length. All other things equal a 1/4 SS will spend 13/7x the bitrate when compared to 1/2s, so you'll get just short of 13h of footage on 64GB. At 1/8s it's well over 7h. Do take into account that this will always vary with footage content and mostly with ISO.

    Having said that, this is a game of just matching GOP with SS. The beauty of recording footage at lower SS is that it's much less demanding on the Camera CPU, on the codec and on the storage card. If you have recording length to spare you can even be adventurous and lower the value of "Quantizer for 1080 modes", but remember that if you lower that QP by 6, you'll end up with approximately 2x bigger frames and then you might already be hitting the ceiling on the 1080p24 Frame Limit.

    Q20 even though it is the default stock setting, it is very good quality already IMHO and if the stock GH2 wasn't so limited in buffers, frame limits and bitrates, it would be spitting out so much quality we wouldn't be using any hacks...

  • Lowering the QP by six doubles the precision of the coefficients. It doesn't necessarily mean the coded frame sizes will double.

  • OK, thanks for the clarification. I'll perform some tests to compute the impact.

  • @duartix what could be the reason that I find with your patch that especially under stress the 13st-1 frame answers with high bitrate.

  • I believe it's a common thing among all patches I tried. I don't know if it's the camera normal behavior saving up bitrate to build up a buffer for rainy days, but I know there's a parameter called "Initial quantizer". I suggest you lower it from 20 to let's say 18 and see what happens.

  • @duartix I know there was a big discussion with @balazer about GOP 12 or GOP 13 for 1/2 sec at 24p but is that really 100% sure to use GOP 13. Why has Panasonic used the standard parameter GOP 12. For 25p I understand that it must be GOP 13 but... maybe I should try your patch with GOP 12 and look how it behaves under stress. What is your practical experience...

    13/23.976 = 1/1.844 > 1/2s == go, but why not the next....

    12/23.976 = 1/1.998 > 1/2s == go

    12/25 = 1/2.083 < 1/2s == no go

    13/25 = 1/1.923 > 1/2s == go

    @Vitaliy_Kiselev is there already something which you know about standard GOP settings for 25p from the new firmware.

    In a GOP12 at 24p case I would have to implement for my macro

    StepSize1 = round(ShutterSpeed_s*FrameRate_fps) == GOP

    which sounds more logic for me...

  • @tida : Logic has nothing to do with it... :) Otherwise your simple and logical formula should work!

    At first my thought was that the parameter AVCHD GOP Length in PTTool was counting the I-Frame twice unlike the standard definition of GOP: http://en.wikipedia.org/wiki/Group_of_pictures but that's not the issue, the parameter conforms with the definition. The issue here is that the camera @2fps spits out the frame changes once every 13 frames and that's why we're using GOP13 to stay in synch (and I've just loaded a 30min MTS file in StreamParser and confirmed that it still stays in synch in the end). If you use GOP12 you'll see a big P-Frame travelling away from the I-Frames and passing by an I-Frame every 12 GOPs.

    Now this leaves me weary about the other shutter speeds and I'll have get to check their GOP structure.

    BTW, Panasonic chose GOP12 most probably to maintain a consistent I[BBP]+ GOP structure.

  • @duartix thanks for your explanation, you have done a very good job. It sounds like a proof for me. One explanation could be that 50i and 24p has to share same GOP size.