It would be interesting to see how the 10 bit file handles a very heavy grade compared to the 8bit file. Also, how much highlight recovery is possible on the 10 bit file compared to 8 bit.

I dont know if its the way you saved them, or just the way the GH4 handles it, but there is ZERO advantage here with the 10 bit vs 8 bit files. I put the same extremely ridiculous grades on these images to stretch them to the limit, and they all broke apart and showed banding and compression patterns in the exact same way. What gives? I figured the 10 bit file would have more highlight and shadow data retention, but it doesnt appear to at all from your files.

It's not so much about the 10bit verses 8bit differences, it's more about the difference in color space that you get when recording to an atomos device. The move from 4:2:0 to 4:2:2 color space is the big plus here, and is admirably demonstrated by this piece on Vimeo.

Watch it from about 25min 45sec..

I'm going to be shooting UHD internally, and 1080p 8bit 422 on the ninja blade at the same time. Prores LT is getting on for twice the approved bit rate for UK Broadcast, so for me that's a good compromise between quality and disk space. I'll keep the 4k UHD files as a backup and in case there's any serious re-sizing required.

The banding is not caused by the color compression (because there's a lot of color values actually between a strip of band even in 8 bit) it is actually due to the codec compression. This is why we never render with lossy compression codec in VFX even if the image is 16 bit, it'll still have the banding effect. To go further with the maths, what happens when your codec compress an image, is that it's going to re-organize all pixels of the images to make it looks "smarter". Let's say you have a perfect blue sky and a character moving in your picture with eventually some landscape at the bottom of the image. The algorithm will actually take all the blue pixels of the sky (since there values and X,Y position are very close) and arrange them: Imagine that all the sky pixels are put from the 0,0 value (bottom left of the screen) and then expands from value by adding X until they are all organized. That would be lossless BUT codec compression do smarter thing, they actually recognize every pixel that has the same value on the image and put them in the exact same pixel area. There are further algorithms responsible for your banding, especially some which recognize the position from one pixel towards another and if close enough and values almost match they'll make a smart re-arrangement (no matter the bit depth, it's actually a value of sensibility) provoking banding when the player uncompressed because the codec thought it was smart to average the values between 10 pixels since they are "almost" identical and close one to another. All of this is done to save as much memory as possible, but I can guarantee to you that a 16 bit OpenEXR with a very high compression ratio that makes loose some image actually looks much worse than my GH2 poor 8bit AVCHD codec.

If I ever got the time I'll try to show you how you can completely get rid of this effect even with very bad banding. it's actually not that hard, it requires to make a Matte of the part of the image that has banding (since it's usually a primary color it won't be very hard) add grain, apply a small gaussain blur to the grain to smooth out and then do a noise removal to flatter the texture making it looks very smooth. All that can be done within after effects.

You can add lossless images here in PNG format. do that if you want to show us quicker.

@StudioDCCreative Could you maybe add an image of the 8 bit 4:2:0 (preferrably same resolution as the others)? Id like to compare if its not too much trouble.

@joethepro here you go:

https://www.dropbox.com/s/8oest5ir9ksctbl/GH4%201080p%20-%20Internal%20vs.%20External.zip?dl=0

Simultaneously recorded with Ninja Star externally (ProRes 422HQ setting) and internal FHD 200M All-I, 24P, ISO 400, 0-255 levels, Portrait profile, with nearly everything -5 and a -3 to highlights.

Again, a bit surprised at what appears to be a gamma difference between HDMI out and the internal recording, but the detail is definitely still there on both sides, both in the highlights and in the shadows.

Anyways, hope this helps you out.

Good question. Presuming you're importing to an NLE, how does the software map those values?Also, are you importing to rec709 and not RGB color space?

If you're working with a QuickTime based NLE (like avid for instance) there is a distinct gamma shift between importing in RGB and Rec709 color space. Hence my question. Can I ask what scales you refer to when you mention 'digital level 12', and 'linear mapping from 0-1024'? These are new references for me.

On further research it appears there may indeed be some kind of behind the scenes conversion happening with the ProRes. I'll talk to Atomos and see what the NJS is coding the ProRes as.

@Mrbill Thanks for the link. I used to edit (long time ago) on Avid, so I'm aware of what you're talking about - which is why I was pretty quick to suggest it wasn't the issue - mostly because FCPX doesn't choose a colour space on media import. However, you raised a valid point which got me wondering what colour space FCPX does indeed use and whether it might be different for H.264 vs. ProRes media, and now I've got some digging to do!

Also, just to clear something up, let's say we WERE using Avid. My point is really that, as long as I didn't import one clip with RGB and another with 709, and the two clips used the same colour space when being coded, there shouldn't be a difference between them unless the difference is actually written into the digital levels of the clip itself (obviously it will be yanked around slightly for compression, etc., but not enough to cause a global gamma shift on the file).

That said, I think we're both suggesting effectively the same thing: somewhere in the processing path there is a difference for the externally recorded file and the internally recorded one which looks suspiciously like an unanticipated colour space conversion a la 709 vs. sRGB (in fact, the only effective difference between 709 and sRGB IS gamma, the primaries are identical). My theory is that either it happens in the GH4 before HDMI output or in the Ninja Star as part of the ProRes workflow/conversion, although my research suggests it might be being done in FCPX as a result of either missing or incorrect metadata in the ProRes file (or, oppositely, an assumption about H.264 which is not the same as the ProRes).

I think your suggestion was that it was, from your experience, quite possible that it was in the NLE, although I believe in the case of FCPX that if it is being done there, it's because of something not right in the file triggering this. Either way, thanks for raising the question as now we have a lot more data to start working towards a resolution! And I hope I'm not misunderstanding you!

If anyone else has any real data regarding this particular situation, or a way to figure out the metadata in the ProRes file to see what might be causing the issue, let us know!

@caveport: good points. My GH4 is set 0-255 (and always has been since the day I got it). I've double-checked that again to make sure that hasn't changed. I don't have the ability to check the HDMI output values numerically, but I should say that to my eye at least this doesn't look like that's the issue - you can easily just check the TIFF files and slap the inverse correction on one or the other and if they match perfectly, you'd have your answer.

As for the external recorder, it has zero options other than pulldown conversion (if necessary, in this case it's not) and ProRes 422 HQ, standard, or LT as the codec.

What do you recommending "checking the files" with? I can poke at the originals now for some information, if you can point me in the direction of a suitable Mac tool to do that.