What happens if the I frames have the same amount of data with a GOP = 3 as they do with a higher GOP? The only difference is that there are more I frames with GOP = 3 and no P frames. Naturally the total bit rate would then have to increase. However, that is what appears to be happening. With a GOP = 3 I was hitting 80+ mb/sec. With it lower it was below 60 mb/sec.
@mpgxsvcd "However, I simply don't believe it produces a more film like look."
Here's where we'll have to agree to disagree in a friendly way. I can't prove it because it's a 'look' that we're discussing. I saw "Another Planet" in the theater the other day, it was shot on video with some long GOP interframe codec (I'll find out which cam) and they did a 35MM blowup. It was fine in static shots but the second anything moved the motion was different and there was substantially more blurring than there would have been with 35mm because the same frame by frame info just simply isn't there to transfer frame by frame to 35mm. My theory is that at 3GOP in our case (or best case AVC INTRA one of my favrorite acquisition codecs) there are more I frames per second that get you closer to the same frame by frame representation of the motion that you have in film. I think the longer the GOP the less chance you have at properly representing the motion, forget detail for now. I think an audience has a much firmer grasp of what motion looks like to them on film than they do of detail. Afterall we watched films for years on VHS with a 240 line resolution so there was no detail but motion representation was consistent. Now is there a huge difference between 3 and 6 GOP? No. But to my eye 3 'looks' more filmic to me. But that's entirely subjective. And that's what great about Vitaliy's patch, we can set the GOP (within limits) to what we want.
EDIT: From interview with "Another Planet" director -- "Mike Cahill: Sure. I shot with a Sony EX3 – which is an HD camera that shoots in 1080p." The EX3 uses an Mpeg-2 long gop codec. Go see the film projected if you can and see how crappy the motion looks - I swear it's due to the long GOP. Then watch Danny Boyle's "28 Days Later" on Blu-Ray. That was shot on a Canon XL1 in DV 25P - an intraframe codec. The detail is less but the motion is a helluva lot more filmic. I shot some second unit with that pal XL1 for a couple years and the motion on the 35mm blowup of those shots (though less detail) always looked great and filmic.
I really think this is the "scientific test" you're looking for. Hell, even watch "Paranormal Activity" (long GOP codec) on Blu-ray and then watch "28 days Later" Forget resolution, just focus on motion. You will SEE the difference, I guarantee it.
That's good, except for one thing - you could get a buffer overrun if the actual bitrate is much higher that the one you put in PTool. Hmmm... maybe there is a fix for this.
I haven't got a buffer overrun with the latest Kae GOP settings. Those appear to be stable for the high settings. They aren't stable for the low settings but that point is minor. I still think that GOP = 3 might produce a preferred picture because it simply is pumping out such a high bit rate with so many I frames at what is a relatively low frame rate(24p). All of those things are good for not only motion estimation but also detail.
I just want to see some evidence that proves it. I am trying to get that now. However, I don't have the means to do a scientific test at this moment.
Here is a good example of what I am seeing at 32 mb/sec and 25 mb/sec with a GOP of 3.
The first stream parser is from 32 mb/sec High settings. It starts off with a nice cadence and then it drops to lower bit rate values for the rest of the clip.
The second stream parser is from 25 mb/sec Low settings. It stays consistent throughout the whole clip. With Kae's recent 65 mb/sec settings I see the same consistency throughout the clip. However, I am not sure if the increase in total bit rate is enough to account for the extra I frames.
For some reason I can't upload the pictures right now. It just times out.
The next release of PTool will probably fix these issues. We have found a couple of things that are probably causing this.
The difference between the two settings may be because of buffering. The Gh2 codec is smart enough to slow itself down when it runs out of buffer space.
Is the reason ptools is capped at 65m for 24H due to some hardware limitation of the cam? Or is it conceivable that you and Vitaliy can hack this to let us go even higher assuming B frames would encode and behave efficiently?
Please use proper topic (encoder related), this one is more about low GOP settings. For most topic readers I think all this encoder talk make no sense.
Here is an interesting development. Kae your low settings do not exhibit the low bit rate first GOP that the high 1080p setting does. It also works for GOP = 12 as well. This means that you can get greater than 32 mb/sec with no adverse affects by using the low settings. I have not determined whether it is better with GOP = 3 or GOP = 12 yet though. At least it doesn't mess up the first frames.
When the encoder reaches its bitrate limit, P-frames can be degraded in an unnatural manner that can never occur in I-frames. If the encoder is forced to skip over certain P-frame macroblocks (because it lacks sufficient bitrate to encode them), macroblocks from the previous frame may appear instead, mixed in with properly encoded P-frame macroblocks. This can cause a motley effect where certain sections of a moving object appear to remain stationary, or update less frequently than other sections.
In such bitrate-starved conditions, a 3-frame GOP (which replaces all P-frames with I-frames) may turn out to degrade in a more natural-looking manner. With I-frames, the encoder cannot skip any macroblocks, it must encode everything from scratch. At worst, it may be forced to produce muddy macroblocks, but it will not reuse stale sections of previous frames.
The end goal can't be how high bitrate we can squeeze. At least that's what the short GOP patch seems. But the reason why high bitrate becomes critical to short GOP is that I frames might choke the bandwidth for B frames. So high bitrate is just a mean to stabilize short GOP. That's all.
Intuitively when entire scene is changing fast, B and P frame optimizations make no sense. More likely to lead weird artifacts. That's where short GOP shines... I guess.
We focus on maximum number of I-frames per second. But it seems... we should focus on minimum number of B-frames per second. Basically same thing, but B-frame's behavior needs to be analyzed. @cbrandin's quantization study seems intriguing. Tame the B-frames!!!
I just did some quick and dirty tests but I kept everything consistent. A GOP of 3 with Kae's settings is definitely superior to a GOP of 12 for 1080p. The bit rate for the I and the B frames is identical for both the only difference is that you replace I frames for the P frames and the B frames stay the same.
However, the I frame and B frame data rate was significantly higher for a GOP = 6. In fact the P frame data rate for GOP = 6 is actually very close to the data rate of the I frames for a GOP = 3. Visually the two files look the same. And their total file sizes were almost identical.
I will redo the test with more scientific procedures but this might finally show us some clear answers as to what is happening.
@mpgxsvcd 'the I frame and B frame data rate was significantly higher for a GOP = 6'
This is very interesting. What exactly did you shoot and with the motion how could you keep it 'consistent'? Have you tried this with heavy camera motion? Obviously harder to replicate than with motion in front of a stable camera but worth a shot. For motion, I still prefer the 'look' of 3 over the 6 (and I've shot with 6 a lot) but it's a tiny difference. I wonder if motion can be captured more accurately with 8 I frames and 16 B frames per second (3 GOP) or 4 I frames 4 P Frames and 16 B frames per second (6 GOP)?
Nice stuff. But Vimeo just can't encode motion worth a damn, I gave up on them. Everything I try to encode with heavy panning or motion stutters like your stuff so I don't even post it. For kicks try uploading to You Tube. Their 1080P doesn't quite look as clean as Vimeo's 720P but the motion doesn't stutter.
Edit: just downloaded your original and the stutter isn't there. Heavy noise though, did you shoot at 2500-3200iso? And there are some edit glitches. What nle did you use, how did you ingest it and what codec did you write the mov file with? You have to be very careful at all these stages.
For kicks did a quick a low GOP work flow test to see if I could get results that were visually indistinguishable motion-wise when transcoded to blu-ray from the original clip's extreme 3 GOP settings. I started with a heavy camera and subject motion 3GOP test clip using the 14-140 with auto shutter (shutter's probably working at 80-100) iso 640, F4, dynamic mode, settings at detente, no ois and ingested it into the Avid using the DNXHD175x 10 bit intraframe codec (the best they offer) and then graded it using the stock Magic Bullet Blockbuster preset (Tony Scott bleach bypass look). I exported using a quicktime reference mov (no transcode from the Avid) into TMPGenc Xpress and encoded using their BluRay 32k preset BUT I set it to VBR 2 pass and 3 GOP. The blu-ray transcode (123 megs) is half the size of the original clip and is visually indistinguishable from it (especially the motion - I went through it frame by frame) - no extra blurring. Add some grain and I think anyone would have a hard time telling this from 35mm shot with a fast shutter.
Can you direct me to your latest 65 Mbps GOP 3 patch? It seems you have several versions on this thread and I'm not sure if I have the latest. I downloaded a patch today but my GH2 wouldn't play back the clips. But no problem playing back on the computer after importing into FCP7. Still I'd prefer to use one of your 65 Mbps GOP 3 patches that can also be played back in the GH2. Thanks very much in advance.
Yeah, I would love to be able to delete them all as some have iterations aren't optimum. What I use everyday is the latest and is attached. 24L records funny so don't use it. 24H is fine with the ever present 10 frame glitch in front. The other modes are at 12 GOP for stability's sake. 6 GOP on those modes will work but I have stability issues with them sometime. The 1080i/50 is always iffy as the GOP is locked to 3 because of the 24p GOP setting - so if you have to shoot 1080i/50 be forewarned.
Why do not you test GOP1 with gh2? It must be more stable than gh1 GOP1. At gh1 it works when you set max bitrate and overall bitrate to 85000000 - 12000000. Or this numbers are not possible with gh2?
65m at 3 is max. At 1 it's too much trouble (very unstable) and the film look of 1 isn't that much different, if at all, so I see no point. The 3 GOP with one I frame and two B frames is really, really nice and good enough for anything I'd ever want to do. What I would rather have is 4:2:2 but so far that's a pipe dream unless Vitaliy can hack the hdmi out to make it work like sdi on the af100.
The reason I ask is that I am researching buffering issues. I'm not so interested in which is better, I'm just trying to identify buffer issues vs. GOP structure. Problems could manifest themselves as stability issues, or quality issues when shooting detailed scenes.