4K Downscaling progress topic

crowbar

All of you are talking above my head with technical matters related to this topic. However, the topic interests me greatly because I just had a terrible experience dropping 4K into a 1080p timeline in that my 4 year old, slow computer hated it! Could only apply limited effects, no playing the edit (should have used proxy media) and then I had to wait about 15 hours for the 3 minute video to render and export! Prior to this I did several short projects with GH4 UHD 4K but prior to import into NLE, I converted the clips to ProRes422 (HQ) 1080p using Firecoresoft converter and the edit/grading/rendering etc. all went much smoother and faster. I would appreciate feedback on this software and opinions on it's conversion quality. I have some short clips on dropbox which can be downloaded from the provided link. Plan to upgrade my computer to the latest and best by the end of the year but in the meantime would certainly like to know if my Firecoresoft workflow is adequate or are your solutions as discussed above significantly superior? The 3 clips include: GH4 4K straight from camera.MOV, GH4 4K to 4K ProRes422HQ by 5DtoRGB & 4K H.264 straight from camera downscaled in NLE to 1080p .mov, GH4 4K to ProRes422HQ 1080p by Firecoresoft.mov dropbox.com/sh/lz60bcxu9fsqb7i/AABsm8PTiiBhWIFn_H618qCga

karl

Ok, I think I found a better way for scaling than the one I proposed above: Instead of downscaling luma alone and re-fitting with unscaled U and V planes, I now upscale U and V (with "nearest neighbor" algorithm, so just doubling the pixels in both directions), merge those upscaled U and V with the unscaled luma, then convert to yuv444p10, then downscale the whole into 1920x1080 yuv444p10.

This seems to retain both the chroma details plus it gains the 10-bit accuracy for the luma channel.

The following script uses ProRes4444 as output format that allow for 10 bits per channel YUV:

ffmpeg -i "$1" \
  -filter_complex 'extractplanes=y+u+v[y][u][v]; [u] scale=w=3840:h=2160:flags=print_info+neighbor+bitexact [us]; [v] scale=w=3840:h=2160:flags=print_info+neighbor+bitexact [vs]; [y][us][vs]mergeplanes=0x001020:yuv444p,format=pix_fmts=yuv444p10le,scale=w=1920:h=1080:flags=print_info+bicubic+full_chroma_inp+full_chroma_int' \
  -sws_dither none \
  -q 0 -quant_mat hq \
  -c:v prores_ks -profile:v 4 \
  -c:a copy \
  -c:s copy \
  -map 0 \
  "$1_2k_ProRes4444.mov"

Find attached an enhanced 4k test image plus the downscaled 2k version of it (but with only 8bits per channel due to JPEG restrictions).

And yes, the 2k output JPEG is bigger than the 4k input PNG, because the JPEG was stored lossless.

karl

BTW: Please notice that you need to be very careful if you want to pixel-peep the results of the downscaling: Many video players will use by default some yuv420 hardware video overlay - thus removing the color details you are looking for as they play.

For Linux users, it's a good idea to use e.g. "mplayer -vo x11 ..." for replay - the X11 driver, unlike the default video overlays, uses ordinary RGB pixels with full resolution.

And BTW: Some web browsers cannot even display my 2k_output.jpg attached above correctly (this JPEG uses yuv444) - Firefox for example seems to loose the chroma detail. Better download the image and use an external viewer... probably with some zoom... ;-)

karl

@heradicattor: I have no experiece with the lut3d filter or its side effects.

Azo

@karl

Wow impressive work. Have you though about creating an app or plugin possibly for Mac that you could sell? I would buy it if it was a reasonable price. I am sure there are a lot of Mac users that would be interested in purchasing something like this.

mrbill

Way above my head! I'd be very happy to pay for a PC version with a nice, simple GUI

radikalfilm

@mrbill How much would you pay exactly? :) I could use some cash fast...

Silvercat

@carl: I'm trying to write a script-preset working with anothergui. My problems i that (on a PC) I have an error: No such filter: 'extractplanes=y+u+v[y][u][v];' Error configuring filter

I'm tryng from a cmd now. What I'm wrong?

hellrazor

@karl: This is great work! I will try today afternoon as soona s I get home! I am currently working on denoising while converting h264 to prores by using ffmpeg. Do you know a filter to reduce the banding that the denoise introduces especially around the h264 artifacts, or might it be too early in the process? -filter:v hqdn3d=6.0:6.0:6.0:6.0

maxr

@RRRR Vitaliy's plugin 4 PS was Sverlov's C3C New version is faster and has sharpness slider. Depending on which use is intended, I'll go with that, free fast batch engine xnconvert or if direct from a raw, iridient blackman8 ( R-L Deconcolution sharp + downscaling) for very smooth details ,-)
http://www.personal-view.com/talks/discussion/7141/high-quality-image-size-reduction-c3c-image-size/p1

@karl many arrigatos 4 all the excelent work =)
from your last settings this is what worked 4 me (0 skills), GH4 file, from mac terminal:

ffmpeg -i INPUTFILE -filter_complex 'extractplanes=y+u+v[y][u][v]; [u] scale=w=4096:h=2160:flags=print_info+neighbor+bitexact [us]; [v] scale=w=4096:h=2160:flags=print_info+neighbor+bitexact [vs]; [y][us][vs]mergeplanes=0x001020:yuv444p,format=pix_fmts=yuv444p10le,scale=w=2048:h=1080:flags=print_info+bicubic+full_chroma_inp+full_chroma_int' \ -sws_dither none \ -q 0 -quant_mat hq \ -c:v prores_ks -profile:v 4 \ -c:a copy \ -c:s copy \ -map 0 \ OUTPUTFILE

which is same without first \ and different input/output file size :P

karl

@Azo: I don't own a Mac and don't intend to buy one... Apple and their "let us think for you so you don't have to"-philosophy is kind of the opposite of what I like about computers ;-) - but you can sure use ffmpeg for free even on Macs, as @maxr showed above.

@mrbill: I'm sure you can use the command line version after some minimal practice... and you'll see that using the command line and scripts will even be much more convenient when converting a whole batch of files routinely.

@willifan: You might have entered that "extractplanes" filter where only simple, linear filters are supported? Since extractplanes has multiple outputs, it can only be used within the "-filter_complex" kind of filter graph.

@hellrazor: You asked the hqdn3d filter to perform a strong denoising, and that's what it does - if you see banding afterwards, you might just try less agressive parameters to hqdn3d. Or try other denoise filters that ffmpeg contains, such as "dctdnoiz" or "owdenoise". Are you sure you see artefacts from h.264? Do you use too-low bandwidths for recording? If that is really the case, maybe an additional "pp=ac" filter before denoising could help a little.

@maxr: It's good to see how easily portable the ffmpeg invocation is.

heradicattor

Reference of pp=ac at http://guru.multimedia.cx/category/postprocessing/

fancydancer

Having problems with the last two scripts posted here.

[AVFilterGraph @ 03a46a00] No such filter: 'extractplanes=y+u+v[y][u][v];' Error configuring filters.

Any help appreciated.

Got it to work with this script,

ffmpeg -i "P1000731.MOV" -filter_complex "extractplanes=y+u+v[y][u][v]; [u] scale=w=3840:h=2160:flags=print_info+neighbor+bitexact [us]; [v] scale=w=3840:h=2160:flags=print_info+neighbor+bitexact [vs]; [y][us][vs]mergeplanes=0x001020:yuv444p,format=pix_fmts=yuv444p10le,scale=w=1920:h=1080:flags=print_info+bicubic+full_chroma_inp+full_chroma_int" -sws_dither none -q 0 -quant_mat hq -c:v prores_ks -profile:v 4 -c:a copy -c:d copy "OUTPUT.mov"

Silvercat

@karl My problem with filter is solved, it was only a shell problem (I'm using windows and cmd). I must change ' with " and it work. But I have some other problem: 1) with "-map 0" option conversion don't start, the error is "Data stream encoding not supported yet (only streamcopy)". I'm using the last ffmpeg build for windows. Without option -map 0 conversion start and work. 2) ... but I'm unable to have the same TC, 3) and the file converted is darker that the original, despite I use .mp4 or .mov for input.

I wrote the script for AnotherGUI (easy for batch conversion), it work with the same limitations.

Silvercat

This morning I done a lot of test.
At the end, the conversion without "-filter_complex" work for timecode stream copy, and also for gamma.
This code (for PC) produce a correct results about Timecode and gamma:

"C:\Program Files\ffmpeg\bin\ffmpeg.exe" -i "INPUT.MOV" -vf scale=1920:1080 -sws_dither none -q 0 -quant_mat hq -c:v prores_ks -profile:v 4 -c:a copy -c:d copy "OUTPUT.mov"

But if I use "-filter_complex....." instead the simple "-vf scale=1920:1080" the video will be a lot darker, and I lost the TC.
The code partially working for pc is the follow:

"C:\Program Files\ffmpeg\bin\ffmpeg.exe" -i "INPUT.MOV" -filter_complex "extractplanes=y+u+v[y][u][v]; [u] scale=w=3840:h=2160:flags=print_info+neighbor+bitexact [us]; [v] scale=w=3840:h=2160:flags=print_info+neighbor+bitexact [vs]; [y][us][vs]mergeplanes=0x001020:yuv444p,format=pix_fmts=yuv444p10le,scale=w=1920:h=1080:flags=print_info+bicubic+full_chroma_inp+full_chroma_int" -sws_dither none -q 0 -quant_mat hq -c:v prores_ks -profile:v 4 -c:a copy -c:d copy "OUTPUT.mov"

Please note the double quote instead the single before and after the "-filter_complex.." command, on a PC it's a must...
I'm not so expert with ffmpeg, so, please help me...

tonalt

Is there really huge advantage downscaling the footage in advance vs. let e.g. DaVinci Resolve to do it?

4k is so nice for re-framing, e.g. do 50% zoom to person's face.

RRRR

@tonalt I reckon Resolve should have good scaling.. However, if you want to process a batch of clips and deliver in 1080p, I'm sure you can improve results. Mind the gh4 outputs an 8bit 4k, so the outset is a wee bit different than footage from an EPIC, for instance. Then there are also footage specific stuff you could apply in / before scaling. Maybe slightly soften a bunch of shots with really intense amounts of details to avoid moire or aliasing... I don't know, that's what we need to find out! For me personally, 1080p is what I need for most projects. Cropping a 4k frame would be the exception, rather than the rule (as I try to shoot exactly what I want).

However, I will jump onto using Resolve now with the latest edition. So I'll get to compare results sooner rather than later.

Edit: Feel free to post your scaling results with stuff done in Resolve, as it's also relevant to the thread!

karl

@willyfan: "-map 0" basically tells ffmpeg to put all streams from the input file also into the outpuf file. In my script, I specified the encoder to use for video streams from the input (-c:v prores_ks), and I specified that audio and subtitles should just be copied into the output file.

The error message you got tells us that your input file contains also a "data" stream (timecodes, I guess), and we did not specify an encoder for this, and ffmpeg has no idea what encoder to use for this "data" stream. So try to specify "copy" as the codec for data, that will probably do it: Add "-c:d copy" after the "-c:s copy". And just to be safe, if there's also an "attachment" stream in your input, add also "-c:t copy".

Regarding the "darker image": I guess this is just a consequence of the conversion from 8bit to 10bit per channel - the way your player interprets the values inside those 10bits is probably different from what ffmpeg thinks is a "neutral" conversion.

Unluckily, I cannot reproduce this effect: When I replay a ProRes4444 file created with the script above (using mplayer under Linux), it looks just as bright as the original.

Do you get the "looks darker" effect with different kinds of players? Can you try VLC?

You could also try to use another output format (like "xxx.mp4") and see whether the result then looks "darker" for you.

Or you can try to use the "dst_range" parameter in the last scale filter to map the values into a different range of values that your software interprets suitably "light".

karl

@tonalt: If there's nothing serious keeping you from using 4k for editing/rendering, it's probably the (quality-wise) best method to do so, and to only derive a 2k version from your final 4k rendering.

The downscaling of 4k recordings to 2k intermediates for editing, even when done as above to retain maximum details/accuracy, is certainly only second best to work with the original 4k files as much as possible.

karl

@maxr: BTW: If you want to use the same script for scaling bot 3840 and 4096 pixel wide videos, you can use expression like "w=iw*2" instead of "w=4096" in the first two scale filter parameters and "w=iw/2" instead of "w=1920" in the last scale filter parameters.

Silvercat

The problem with "darker" image happen with Adobe premiere, but also with virtualdub and vlc. It seems that white is correct, but black is a lot darker, with lost details in the dark.
May be that the problem is from GH4 setting, I'm using 0-255 setting, may be is better to use 16-255 setting. i will try tomorrow.
Yes, mp4 and mov files from GH4 have a quicktime timecode stream. If I convert without complex_filter ffmpeg copy the timecode stream, but I'm unable to have the timecode in my final file, I will try with your advice.

Silvercat

I'm going insane ...
@karl: The good news is that with 16-255 setting instead the 0-255 setting (done in the GH4 menu) the luminance of the clip now is correct after the conversion. May be there is a bug in ffmpeg, as it does not properly recognize the clip (adobe premiere do it) but only when we use -filtercomplex because if I convert only with scale it work.
The new problem is in the colours: now I have correct luminance but the image appare to be more "reddish" and saturated
About time code, the timecode stream is missing from destination file (checked with mediainfo) despite the "-c:d copy" and "-c:t copy" option. the strange is that doing the conversion with the normal "scale" filter the stream is copied correctly, all problems seems to be in the "-filter_complex" chain. Do you have a real Gh4 file, for test?

maxr

@karl SUPER!!! =)

Silvercat

I tried a bit different approach to scale, as using extract plane and merge plane I have some difference from input and output. In this code, I scale all size to double (8K!!) with neighbor algorithm, and after downscale for a 4 factor with bicubic.

This is the code:
C:\Program Files\ffmpeg\bin\ffmpeg.exe" -i "INPUT.MOV" -filter_complex "format=pix_fmts=yuv444p10le,scale=w=iw2:h=ih2:flags=print_info+neighbor+full_chroma_inp+full_chroma_int [a]; [a]format=pix_fmts=yuv444p10le,scale=w=iw/4:h=ih/4:flags=print_info+bicubic+full_chroma_inp+full_chroma_int" -sws_dither none -q 0 -quant_mat hq -c:v prores_ks -profile:v 4 -c:a copy -c:s copy -c:d copy -c:t copy "OUTPUT.mov"

I checked it with the 4K png image, and I can see the 1X1 pixel pattern in the bottom box, and a solid brown in the top box (using quicktime player). Color and luminosity don't change anymore, despite I use 0-255, 16-255 or 16-235 settings on the GH4.
The only problem I'm unable to find a solution is keep the timecode stream. The problem is for sure the "filter_complex" because if I scale directly with the usual "-vf" statement the timecode stream is copied.

Silvercat

Ok, I discovered also the color problem on the karl script.
1) We must set the Gh4 to 16-235 recording
2) We must add the option "in_range=full" for U and V scale.
So, I have a working script:
ffmpeg -i "$1" \ -filter_complex 'extractplanes=y+u+v[y][u][v]; [u] scale=w=3840:h=2160:in_range=full:flags=print_info+neighbor+bitexact [us]; [v] scale=w=3840:h=2160:in_range=full:flags=print_info+neighbor+bitexact [vs]; [y][us][vs]mergeplanes=0x001020:yuv444p,format=pix_fmts=yuv444p10le,scale=w=1920:h=1080:flags=print_info+bicubic+full_chroma_inp+full_chroma_int' \ -sws_dither none \ -q 0 -quant_mat hq \ -c:v prores_ks -profile:v 4 \ -c:a copy \ -c:s copy \ -c:d copy \ -c:t copy \ "$1_2k_ProRes4444.mov"
Or, for PC:
"C:\Program Files\ffmpeg\bin\ffmpeg.exe" -i "input.MOV" -y -filter_complex "extractplanes=y+u+v[y][u][v]; [u] scale=w=3840:h=2160:in_range=full:flags=print_info+neighbor+bitexact [us]; [v] scale=w=3840:h=2160:in_range=full:flags=print_info+neighbor+bitexact [vs]; [y][us] [vs]mergeplanes=0x001020:yuv444p,format=pix_fmts=yuv444p10le,scale=w=1920:h=1080:flags=print_info+bicubic+full_chroma_inp+full_chroma_int" -sws_dither none -q 0 - quant_mat hq -c:v prores_ks -profile:v 4 -c:a copy -c:s copy -c:d copy -c:t copy "output.mov"

the timecode problem still exist, but now I do not know how to move forward.

Howdy, Stranger!

Categories

Tags in Topic

Top Posters