I have seen a lot of really good testing going on here. Some of the interpretations of results, however, seem a little off to me - or at least a little simplistic. Although testing things like high bitrates and low GOP are clearly valuable, interpreting results based on maximizing just one thing doesn’t effectively address the issue of how each of these affect a usable final outcome. No one thing will produce the results we want, and some of the things that need to be considered seem to never even come up. Some of the issues I see:

High bitrate means good results. Not necessarily. Once the bitrate has been set high enough to outperform other factors in the camera (e.g. the sensor), there is no need to go farther.

High AQ settings produce noise. Not true – high AQ simply reproduces noise more faithfully.

Setting noise reduction in the camera is bad. Not necessarily true. If you are shooting at high ISO’s more noise will be produced. The codec has to work hard reproducing this noise, and you’ll end up just crushing the blacks or doing noise reduction in post anyway. Applying some noise reduction in camera under these circumstances can be very helpful because the NR is applied before video is encoded and you are not consuming codec bandwidth reproducing noise.

The camera should always be set with Contrast at -2. Not true, the camera’s contrast should be set so the histogram shows the entire range being used. If you always set Contrast to -2 low contrast scenes will have poor gradations because you have set the dynamic range artificially low thus limiting the number of bits being used for the image.

In camera saturation should always be set to -2. Not true, for the same reason given above.

In camera sharpness should always be set to -2. Absolutely correct! Sharpening produces fake detail which the codec will have to work hard to encode. Sharpening is always done better in post.

Bigger P and B frames are always good. Not true – with static scenes B and P frames serve no purpose insofar as image quality is concerned because they primarily encode changes to the image. If things are correct, static scenes will produce small P and B frames.

Bigger P and B frames are always bad. Also not true – with highly dynamic scenes P and B frames get bigger because lots is changing.

Smaller I frames coupled with big P frames is bad. No, it’s perfectly normal behavior because with highly dynamic scenes because there is usually some motion blurring – which reduces I frame size – combined with big changes from frame to frame – which increases the size of P and B frames.

Individual frame quality can be improved without raising AQ. Mostly not true. The exception to this is when the codec runs out of bandwidth before completing a frame, resulting in image degradation in lower parts of the frame (macroblocking). The quality of motion encoding, on the other hand, benefits greatly from bitrate increases irrespective of increases in AQ.

Bigger I frames means better individual frame image quality. True.

Bigger P and B frames means better individual frame image quality. Only if there is motion, otherwise big P and B frames do little to contribute to individual frame IQ.

Bigger P and B frames means better motion encoding. When they appear in high motion scenes, absolutely true.

Improvements to image quality are readily visible. Absolutely not true. Some of the most important improvements only become visible in post, when color grading, etc…

To put all this together; consider a test where high bitrates coupled with AQ 4 produces an unacceptable amount of noise at ISO 3200. Moreover, the stream is exhibiting artifacts, such as odd looking pulsing in the frame sizes. What is happening is that the codec is being overly stresses trying to encode an inordinate amount of noise. Raising the bitrate won’t help; that just stresses the codec even more. Lowering AQ will seem to help – but it isn’t really doing anything except lowering the quality, which would be better achieved by just lowering the bitrate. The solution here is to apply a little in camera noise reduction. If you just lower AQ, all gradients and image subtleties will be degraded, whereas NR is only applied to the darker parts of the image near the noise floor.

Of course, all this assumes that you will be doing some post processing. I assume most people interested in advantages brought about by the hack do that.

Chris