Sharing a most spectacular Failure.

I thought I'd share it because there are a few things to learn from the analysis and because it was really really close to working. My first approach for TimeHBusteR was the same as Timebuster 1.3: Marrying the GOP to shutter speed, getting rid of B-Frames and trashing P-Frames. All in all it looked pretty simple:

1080i60 GOP Size=12

Encoder setting 1 1080i/p=2

1080i Scaling B=0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF

The IQ wasn't without all sorts of issues, reduced resolution, horizontal blocking and there were these very small but noticeable and very annoying interlacing artifacts. And it was a spectacular failure because only today I understood how close it came to working but also how it can never work...

So what's happening here? First since this is an interlaced stream, there is something unusual: Only one of the slices of the I-Frames is encoded as I (#12/1), the other is encoded as P(#12/2). Now the P-Frames are real trash as expected, but in an unexpected twist, the P-Slice that's accompanying the I-Slice (EDIT) should be really good with a fixed 22 QP. This should give the frames of interest for timelapsing good IQ, right? I'm on the beach right now, but then it hits me when I look at it on StreamEye...Half the picture is a P-Slice and most of that is either skipped blocks or inter blocks, predicted from... yes, from a trashed P-Frame. So I die on the beach. :( And so should interlaced!!!

(update) Probably the P-Slice is using the Scaler Table for P-Frames and that is a more likely reason for all the IQ issues I mentioned before...

P.S. Today I made some slow but steady progress, thanks @driftwood .

(edit file attached)

IQ should should be 20. then lower Q accordingly.

yeah always start from there.

@duartix 20 is the default for IQ, 720 QP, and 1080 QP, so if you enter it manually, you should see no change in behavior.

No success...

It's not affecting HBR nor 24p.

QP is still reported as 22 for P-Frames and 20-24 for I. :(

Can't see what else I need to change...

@LPowell : OK, it makes sense, so how can I "convince" it into thinking it has enough bitrate (because it does have, and much more than it needs). Does it involve any other settings than the ones I changed? Is it something as simple as raising the related modes bitrate? Like Video Bitrate FSH/FH or 24H/L?

Unfortunately this is where I usually start copying other people's settings and adapting to my needs. :(

Ironically, reliability which is probably the biggest PITA, is the least of my worries.

Now, I've raised Video Bitrate 24H/L/FSH/SH/FH/H, Video Buffer, Video Buffer 24p, 1080p24 FB2, 1080p24 Frame Limit, 1080p24 High Top/Bottom Setting, 1080i Top/Bottom Setting, Other Modes Top/Bottom Setting and...

NADA! StreamParser is still reporting QP=23... WTF??? This is what the setings look like.

Well, it took me a good part of the weekend to figure out what the problem is and why the QP doesn't stick... Surprisingly it's related to GOP.

Since I can't get my simple settings to work (there's an immediate freeze in the video recording as if the camera was doing WB bracketing in video) so I tried a lot of other people's VBR settings on Q18. They all work fine at Q18 but I'm trying to change the 24p GOP to 5 (347º shutter on 24p and 360º on HBR25 at 1/5s) and 1080i GOP to 12 (360º shutter on HBR30) and that's where the problems start. :(

I tried @balazer 's Cake 95 and as soon as I set 24p GOP=5 or 1080i GOP=12, the QP doesn't stick anymore and goes on vacation to a constant 22.

I tried @bkmcwd 's natural_v1.11. Setting 1080i GOP=12 makes the QP on HBR30fps go up to 27-28 on I and 29-30 on B. However when setting the 24p GOP=5 it does respects the QP in 24p.

Finally I tried @lpowell 's FlowMotion 1.11 and these are the only settings that respect the QP in both 24p and HBR30fps.

Does anyone have an idea what the problem could be? I mean, I'm using a 1/2.5 shutter speed in these tests. It's not full intra, there's a lot of temporal redundancy for either B or P or both to do their efficiency magic, so bitrate and buffers shouldn't be challenged... Why doesn't the QP work as it should?

That would be a plausible reason @Lpowell , but I hadn't yet done my stuff on the Scaling tables. It was just QP=18 and GOP=5|12.

Well, I'll be trying that now, I'll see what happens. As a matter of fact I'll have to weight using B-Frames as a bitrate saviour since I cannot trash P-Frames because of the dual I/P nature of the keyframes in 1080i.

Cheers @balazer!

No I haven't, it's against my religion... ;)

But I already know what's breaking it and it's not nice. It's not GOP either after all (well at least not alone)... I set Encoder Setting 1 1080i/p = 2 on @lpowell 's FlowMotion to supress B-Frames and the QP blew up! Surely it couldn't be B-Frames! But it was! I took Cake 95, set it to use B-Frames by Encoder Setting 1 1080i/p = 3 and QP obligingly respected my autoritah.

Why? I have no idea. The bitrate wasn't being taxed, so I suppose the buffers should also be healthy... Anyway, due to the interlaced nature of the keyframes encoding, I'm condemned to use good P-Frames in HBR, so this time it will be the B-frames' turn to be trashed.

Releasing 24h Timebuster 2.0 and 24h TimeHBusteR 2.0!

Due to some respectable user demand suggesting to keep timelapse on 24p but because my motion fluidity tests suggested otherwise (using HBR modes), 24h Timebuster 2.0 is now comprised of 2 main variants:

a) 24h Timebuster 2.0 - with the following changes (from TimeBuster 1.3):

• The GOP for 24p is now married to 2.5fps. The bitrate hit is a little above 20% but the recording length can be easily compensated by raising the QP. As a benefit it's still very close to 2fps and it's the closest to a 360º shutter we can get on 24p.

b) 24h TimeHBusteR 2.0 (NEW, uses HBR 30)

• Similar philosophy to 24h Timebuster 2.0 Base but as the name suggests, uses HBR mode instead of 24p. There is another difference, P-Frame quality is fully preserved as this kind of frames is used for one of the interlaced slices on Key Frames.

• GOP married to 1/2.5 on HBR 30p (NTSC) and to 1/5s on HBR 25p (PAL). The big benefit is that both are using a 360º shutter. Uncheck "1080i50 and 1080p24 GOP Size=5" before merging if you don't plan to use Shutter Speed (SS) =1/5s and want to better preserve the merger's 24p mode and HBR 25.

Both variants are now:

• Using B-Frames. There is a 6% penalty in bitrate usage but this was necessary to make sure the QP was respected. I believe the gain from setting the QP at will is just too great to ignore, since not everyone wants to record for 24h straight. QPs as low as QP=16 were tested and theoretically it could go as low as QP=12 @ISO160 when merged with FlowMotion 1.11 on regular tripod mounted footage.

• GOP married to 2.5fps. The bitrate hit is a little above 20% but the recording length can be easily compensated by raising the QP. The reason behind this was getting as close as possible to 360º.

• User definable GOP - GOP can be set at will according to intended shutter speed (SS), use the attached "GOP-SS-table.png" to look up the right value. Defaults settings are SS=2.5fps and GOP 1080i=10 for 24h Timebuster 2.0 and SS=2.5/5fps and GOP 1080i50/i60=12/5 for 24h TimeHBusteR 2.0.

• User definable QP - QP can be set at will according to desired IQ/recording length. I've gathered quite an amount of information so that in the future I can make an Excel tool to estimate the best QP from Card Size/Footage Snapshot/Desired ISO/Recording Length. Default settings are Q24/23 for 24h Timebuster/TimeHBusteR.

As before, there is the Base Release to be merged at wish with popular settings (load Base first in PTTools and then "ALT+Click" your choice of merger) and there is also the FlowMotion Release (already merged with FlowMotion 1.11). I've merge tested against @lpowell 's FlowMotion 1.11, @balazer 's Cake 95, and @bkmcwd 's GOP3ZILLA and no undesirable effects were observed. The key is using a VBR settings definition with a very high "1080p24 Frame Limit", especially if you are going for a low QP (<22). If you want to merge with Nick @driftwood 's settings, I advise to go for the Sedna Q20 Pack.

The day is not over. :)

Thanks Duartix for all the hard work and time you spend in making these new Time(H)Buster patches! I will give them both a try and then choose my favorite. When I have a nice TimeLapse made I will show it in this topic. Maybe other can do the same. -Cheers-