Personal View site logo
Official Low GOP topic, series 5
  • 751 Replies sorted by
  • With Cluster v2f, we have a unique opportunity for a comparison of 720p B-frame encoding quality. Cluster v2f uses the 720p B-frame quantization Scaling Table that I designed for Flow Motion v2, but with one crucial change to the luma table DC coefficient:

    Flow Motion v2: 720p Scaling B=0x707, 0xB0B, 0xB0B, 0xF0F, 0xF0F, 0xF0F, 0x1515, 0x1A1A, 0x1A1A, 0x1515, 0x1A1A, 0x1C1C, 0x1A1A, 0x1C1C, 0x1C1C, 0x1C1C, 0x705, 0x907, 0x907, 0xD0A, 0xA08, 0xD0A, 0x130F, 0x100D, 0x100D, 0x130F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x705, 0x907, 0x907, 0xD0A, 0xA08, 0xD0A, 0x130F, 0x100D, 0x100D, 0x130F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F

    Cluster v2f: 720p Scaling B=0x303, 0xB0B, 0xB0B, 0xF0F, 0xF0F, 0xF0F, 0x1515, 0x1A1A, 0x1A1A, 0x1515, 0x1A1A, 0x1C1C, 0x1A1A, 0x1C1C, 0x1C1C, 0x1C1C, 0x705, 0x907, 0x907, 0xD0A, 0xA08, 0xD0A, 0x130F, 0x100D, 0x100D, 0x130F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x705, 0x907, 0x907, 0xD0A, 0xA08, 0xD0A, 0x130F, 0x100D, 0x100D, 0x130F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F, 0x7F7F

    Cluster v2f substitutes a luma DC value of 3 for FM2's luma DC value of 7. The effect of this change is to use much finer quantization quality for the DC component in B-frames than for the higher resolution details in each macroblock. However, there is another scaling factor that affects quantization as well: the macroblock QP value selected by the encoder for each frame.

    After shooting 720p60 SH mode sample videos using Cluster v2f and Flow Motion v2, I used Stream Parser to examine the range of QP values used in each frame. In the Cluster v2f video, the average QP value was about 34, while in the FM2 video, QP was about 28. The encoder takes these values to index into a table of Q step-sizes, which it then uses to scale up the values found in the quantization Scaling Tables. For the Cluster v2f video, the average Q step-size was about 32.5, while for the FM2 video, it was about 16.25. In simple terms, that means that the values in the Cluster v2f table were scaled to be about twice as coarse as those in the Flow Motion v2 table.

    If we scale the 720p B-frame tables proportionally by those QP step-sizes, the effective scaling tables look like this, arranged in 4x4 matrix format:

    Cluster v2f

    06, 22, 30, 42

    22, 30, 52, 52

    30, 52, 56, 56

    42, 52, 56, 56

    Flow Motion v2

    07, 11, 15, 21

    11, 15, 26, 26

    15, 26, 28, 28

    21, 26, 28, 28

    After this relative scaling, the DC values at the start of each table are nearly the same, while the Cluster v2f values for macroblock details are twice as coarse as those used in Flow Motion v2. To illustrate the effect this has in practice, I cropped 400% frame grabs from my 720p sample videos:

    Cluster v2f 720p60 SH mode: image

    Flow Motion v2 720p60 SH mode: image

    To put these magnified images in context, here's the uncropped frame grab: image

    Cluster Creek Bed 4x.jpg
    1024 x 512 - 149K
    Flow Motion v2 Creek Bed 4x.jpg
    1024 x 512 - 168K
    Cluster Creek Bed.jpg
    1024 x 512 - 240K
  • Whoops, thats the wrong version of Cluster v2e/f. Apologies about leaving in a test version of 720p60 when I was comparing my matrices with Lee's. I'll upload the correct cluster matrix (a derivative of my Sedna matrix and used in v2a) version under newly titled v2g. You'll notice the matrices should be as v2a though I have edited i frames.

    I was wondering why I was getting problems today. The other side of the coin is that Cluster is not aimed at comparing itself with lower GOPs, it is an experiment in saving memory for long recordings whilst retaining decent quality over stock. I want to make that clear first. However, it performs rather well in 24p beating Flowmo - and HBR is only just behind running at less memory. And Ive still to see an all conquering 720p setting from anyone including my v2 versions - I must say, Flowmo performs excellent at 720p - well done to Lee. Cluster v3 has a bit of work to go.

    Driftwood Cluster v2g - sete.zip
    1K
  • Anyway v2 series is now obsolete. Here's first of the v3s.

    Driftwood Cluster v3 - With Sedna adapted matrix - setj.zip
    2K
  • @driftwood

    I am sorry to be unable to perform help of your 6GOP setting. BTW, your setting which is not Intra becomes reference material especially for me. Thank you always! :-)

    @LPowell

    "I believe this is crucial for achieving P and B-frame image quality comparable to I-frames."

    I take care similarly. How much do you think the tolerance level in the range of DC value is?

  • Guys

    Which setting gives the highest quality for the lowest bitrate? I am going to India for a month and want to shoot lots of great footage but I don't want to carry a trillion SD cards. I shoot most stuff with Sedna AQ1 nowadays (with slightly lower frame limits) but for my India trip I want to go safe and have a patch that performs well in all modes with a good storage/duration ratio. Sanity? Cake? Flowmotion?

  • @driftwood @LPowell

    I feed back the result of my usual test to you.

    Each left is shot illuminating fine textiles with a halogen lamp. Each right is shot the moving Stray's chart. http://www.personal-view.com/talks/discussion/comment/51720#Comment_51720

    Both FM2.01 and Cluster will have too high the bit rate, and will stop during recording. I think that it will probably stop also at the wood which I am always testing with my DIY steadicam.

    I think that 60p not stopping is not realized unless it is adjusted so that 55M may not be exceeded at least.

    In addition, I think that the GOP table of HBR30p of FM2.01 also has the necessity for adjustment to a slight degree.

    *All the film modes are "cinema[-2.-2.+1.-2]".

    Cluster v3_60p_stop.jpg
    2433 x 1395 - 2M
    FM2.01_60p_stop.JPG
    2430 x 1393 - 2M
    FM2.01_HBR30p_x_ISO1600.JPG
    1216 x 1393 - 941K
    GOP3ZILLA_MAX_V2B7_60p.jpg
    2435 x 1394 - 2M
  • @sohus Try Cluster v3 as the matrix is an edited Sedna.

  • @driftwood

    "However, it [Cluster] performs rather well in 24p beating Flowmo....."

    In what respect? Thanks.

  • @bkmcwd - "How much do you think the tolerance level in the range of DC value is?"

    The DC value reported by Stream Parser is the peak value (i.e. lowest) used to encode macroblocks in each frame. If the difference in DC values between P/B-frames and the previous I-frame is more than 2, it indicates that I-frame quantization quality is not being maintained in P and B-frames. This is most important when shooting low-light or fast-action, because in those conditions, many areas in P and B-frames will actually be encoded as Intra macroblocks, just as in I-frames. For consistent motion picture quality, all Intra macroblocks should be recorded at the same average quantization quality.

  • ...and DC values will change according to changes in picture detail.

    @jrd detail.

  • Can anyone tell what would be the latest and greatest patch for documentary film work in general? I have been using the classic 176mb Intra, but I would like something a bit smaller in size. It would be great to have something ~100mb/s size, spanning of course, and will work with in camera playback, mostly 24p, but occasional slow motion use of 60fps, I have 95/mb SD cards. I am doing a doc film that aims to improve diet and fitness levels for the elderly. Also, what would I need to do to shoot in 2:35:1? I would like to still use my same lenses if possible. Any help will be highly appreciated. I don't believe LA7200 will fit on CP.2 lenses.

  • @lebowski1998 - "It would be great to have something ~100mb/s size, spanning of course, and will work with in camera playback, mostly 24p, but occasional slow motion use of 60fps..."

    http://www.personal-view.com./talks/discussion/3337/gh2-flow-motion-v2.01-update-failsafe-hbr-720p-modes/p1

  • 176mb Mysteron Burst Mode is the highest bitrate hack that can be used right?

  • yep, which is decent enough... we've had it higher but unstable.

  • @LPowell @driftwood

    Thanks always! :-)

  • Hey Guys, So I am really confused. There are soooo many hacks out there and I just dont know witch one of all those I should use. I tryed some of them but I never knew if this was a old one and if there would be better ones. I am making Music Video Clips and short films. So I really wanna have this sharp cinematic look (for example RED Scarlet style). Can you tell me witch is the best hack for my work? Also visit my youtube page (http://www.youtube.com/nitramdre) to see what work i do and what hack would suit me.

    PROBLEM, HELP! When I made my latest short film "No Mercy" (

    ) I found some really wierd pixel stuff on my footage. Here is the sort film I've done. Now, if you pause at 2:48 and look at his hair you will see what i mean. It got worse because of Youtube but still on my stock footage it look kinda the same. So weird. Can't figure out what it is and the weird thing is you dont see it always. Some shots are fine and some of those have this "pixel" problem. You can also look at the image down below. I think you can see it better there. On the second one you can see all the Pixel Squares better.

    I use the SanDisk 30mb/s 16GB. Is that maby the problem? I'm thinking of buying a SanDisk 95mb/s 64GB. I heard that was the best SD card out there on the market. I would really appriciate your help guys. Thanx alot!

    Greets Martin

    Screenshot.png
    1911 x 1071 - 3M
    Screenshot_1.png
    1910 x 1075 - 3M
  • @towi Im going to test gop3hq soon any news about 720p50. anyone compare gop3hq, flow mo2,02, cluster 2,3, 4 and cake 2,3 - stability, spanning and quality in 25p HBR and 720P50? Thank you everyone for heard work.

  • This may interest some of you PTool tinkerers.

    My goal was to produce an intra 24p/VMM patch free of redundant and/or ineffectual settings given the constraints of a 45MB/s card. This is just an exercise, not an attempt to achieve absolute image quality; it's possible that for these moderate bit rates, higher GOP settings can beat intra. Caveats out of the way, here's what I have:

    A Little Light Intraspection (ALLI) v1.1

    24p and VMM ONLY

    138 Mbps 24H, 95 Mbps 24L spanning, VMM H unstable, VMM L spanning.

    ALLI24H 24H ALLI24L 24L

    ALLI changes only six parameters:

    • Video bitrate 24H & 24L: 150000000 and 88000000.
    • 1080p24 Frame Limit: 5750000. Version 1.0 didn't use a frame limit, which caused the video buffer to run down.
    • 1080p24 GOP Size: 1 & Encoder setting 1 1080i/p: 1. The former is apparent. Without the latter, the video is useless -- choppy frames skipping all over the place. I don't understand its function, but it is necessary.
    • 1080p24 Scaling Fallback. Without altering this scaling table, 24L mode will oscillate in and out of fallback mode to hit the target 24L bit rate, making the bit rate graph look like a series of plateaus. The default fallback table is extremely coarse, so DC jumps to around 20. Simply replacing the fallback table with the default Scaling I table allows the encoder to hit the target bit rate while maintaining the same DC.

    Version 1.0 had a set quantizer value, but the encoder does a better job of choosing QP depending on scene complexity.

    ALLI is not without problems. Attached is a frame grab of my death chart (actually a death CRT) shot with the 14-42 kit lens at 42mm, f/7.1, 1/80, ISO 640. The jump in QP presents itself as mosquito noise, which is highly annoying. At closer examination, this mosquito noise looks like discontinuities between adjacent blocks. I've tried correcting this with the deblocking tables, but I'm out of my element here, so my efforts were fruitless. If you have insight into deblocking, please share.

    So if you enjoy tinkering, this may be of some use. I'd love to hear if this is stable for others with a different variety of SD cards, or especially if you can improve it.image

    deathcrt.png
    1920 x 1080 - 4M
    ALLI.zip
    524B
    ALLI1.1.zip
    520B
  • I set myself a challenge to see if I actually could beat intra with a longer GOP patch, again forgoing redundant and/or ineffectual settings. The answer is depends, but there is definitely an efficiency gain in scenes without much movement. Here are the results:

    Simple GOP4 v1.0

    24P and VMM ONLY

    133 Mbps 24H, 95 Mbps 24L spanning, VMM H unstable, VMM L spanning. image24H image24L

    Simple GOP4 changes only five settings:

    • Video bitrate 24H & 24L: 150000000 and 88000000.
    • 1080p24 GOP Size: 4 & Encoder setting 1 1080i/p: 1. The former is apparent. Without the latter, the video is useless -- choppy frames skipping all over the place. I don't understand its function, but it is necessary.
    • 1080p24 Scaling Fallback. Without altering this scaling table, 24L mode will oscillate in and out of fallback mode to hit the target 24L bit rate, making the bit rate graph look like a series of plateaus. The fallback table is simply a copy of the default 1080p24 I table.

    On my static death CRT, Simple GOP4 has a more consistent QP across the frame than ALLI due to the larger I frames. On a heavy motion test, SGOP4 runs into the same bandwidth limitations of ALLI and turns, essentially, into intra.

    I'm pretty happy with Simple GOP4, and I'd be glad if anyone finds this information useful or has suggestions.

    SimpleGOP4.zip
    498B
  • @Ashun Good testing. A few points. Your Intra 138M setting will fail on the buffer/run down because its not enough bitrate.

    "Encoder setting 1 =" controls the frame type usage in the GOP.

    Encoder 1080* (*or 720) = 1 (equals stable Intra only) when GOP is set to = 1. Set to 2 or 3 and you will notice erratic Intra if the GOP setting is set to 1. Its a bloody nice effect though ;-)

    Subsequently,

    Encoder 1080* (*or 720) = 2 will only use i and p frames in your GOP settings (So long as GOP >= 2).

    Encoder 1080* (*or 720) = 3 will make use of i, p and/or i and b frames in your GOP setting (So long as GOP >= 3).

    Hope that helps.

  • It does. Thank you very much.

  • If you look way back at the patches thread, youll notice I made 3GOP variations utilising P or B frames in the the original Driftwood GOP3ILLAs which pushed B and P frames to the level of i frames.

  • To gurus: does anyone know why there are almost no GOP2 settings? It was found useless or inefficient? I'm dreaming about 24P GOP2 with up to 1180kb I-frame and quite large P-frame. Does such setting exist? Thanks!

  • @Ashun The lower 24L Stream Parser chart you uploaded above shows the encoder being forced into a type of Fallback Mode, starting at frame 44. When this occurs, the encoder's bitrate allocation mechanism fails to estimate I-frame bitrate requirements correctly, and produces clamped I-frames with degraded image quality. This type of dysfunctional behavior is likely the result of your attempt to alter the 1080p24 Scaling Fallback Table to suppress the undulating fallback pattern you observed in 24L mode.

    @kovtun66 In its B-frame mode, the AVCHD encoder has a built-in preference for using three-frame patterns, in the form of IBB and PBB sequences. In practice, it's been difficult to produce consistent encoding behavior when the GOP-length is not evenly divisible by 3.

  • @Ashun How to spot fallback. Check with a stream analyser (like Elecard Stream Analyser) where and if it goes into fallback matrices. Or check the i frames in Streameye or similar quantisation proggies for deep analysis. Chris Brandon's Streamparser is excellent for an overview of Qp.