Personal View site logo
Make sure to join PV on Telegram or Facebook! Perfect to keep up with community on your smartphone.
Please, support PV!
It allows to keep PV going, with more focus towards AI, but keeping be one of the few truly independent places.
Checker Board HDR Sensor Idea
  • apefosapefos

    I think the checkerboard design for hdr sensor can be better to avoid aliasing and moire and more easy for image reconstruction in missing pixels in each exposure (dark/bright) compared to alternate lines (simpler algorithm):

    Maybe this can be good for 4K hdr video in 3840 x 2160 pixels sensor, and much better for 1080p because in 3840 x 2160 pixels sensor each four pixels will be one pixel after downsize, so the debayer and interpolation will be very good. this design using 7680 x 4320 pixels will be amazing for 4k hdr, because each four pixels will be one pixel after downsize.

    3840 x 2160 will be great for 1080p hdr (8.240.400 pixels, easy to do in M43)

    7680 x 4320 will be great for 4k hdr (33.177.600 pixels, easy to do in sony full frame mirrorless)

    If you look following the diagonal, you will perceive there are two pixels besides an empty pixel for red, green and blue, same diagonal direction for red and blue and other diagonal direction for green. this makes the interpolation more easy and better to do when comparing to the alternate line pairs design.

    checkerboard_hdr_sensor.jpg
    1280 x 720 - 152K
    checkerboard_hdr_sensor_small.jpg
    320 x 180 - 42K
  • 10 Replies sorted by
  • apefosapefos

    Now the same idea, but with RGBW pixels instead of just RGB:

    Maybe the RGBW can be better for low light but I think the RGB can be better for image reconstruction and interpolation in each exposure.

    checkerboard_hdr_sensor_rgbw.jpg
    1280 x 720 - 170K
    checkerboard_hdr_sensor_rgbw_small.jpg
    320 x 180 - 42K
  • Vitaliy_KiselevVitaliy_Kiselev

    Why do you think it is better? I mean from interpolation math pov?

  • apefosapefos

    I did another look at the draws and I perceived that both designs RGB and RGBW can have a good interpolation.

    When looking the pixels in the diagonal direction (in both X directions), there is always two pixels of same color surrounding a missing pixel, so it is possible to reconstruct the missing pixels in both RGBW or RGB with same precision. Colors can alternate position in reconstructed pixels array, but I think this is not a problem, because the debayering will happen after the interpolation, so the debayering algorithm can consider this.

    I just have a feeling that having more green pixels near to each other maybe can make the debayering and interpolation to be better but this is just intuition, I have no idea about moire or color fringing using RGBW.

    No matter RGBW or RGB, I strongly believe that the checkerboard design can be much better than the two alternate lines design. The distance from pixels to get information from is much closer to each other in the checkerboard.

    Also I think they can use the interpolation from surrounding pixels together with the idea of multiply or divide the pixles values from one exposure to get the values for the other exposure, when different exposures are done with different isos, as I said in the A7000 rumors topic.

  • Ralph_BRalph_B

    A simpler approach might be to take the second green photosite of a normal Bayer pattern and under expose it. This would provide the luminance information for the extreme highlights. The color information for the extreme highlights would be added in from the normally exposed photosites. It won't be 100% accurate, but probably good enough to get the job done, since the eye is far more sensitive to luminance than color.

  • apefosapefos

    This is interesting idea, but maybe the red and blue pixels will be completely washed out in 255 value in highlights so there will be no way to recover color from them...

    These are three another ideas for the pixels design:

    two are more close to the nikon honeycomb (maybe there is no need to put darker/lighter filters over the pixels because the pixels size being different are enough for different exposures):

    Probably this design is even better than the checkerboard, because there is no need for interpolate and recreate missing pixels in different exposures, just an specific debayer algorithm must be created.

    Another approach is the different exposures happens in different layers, instead of having the sensor divided for different exposures, there will be pixels in a layer below the main exposure, similar to the sigma foveon sensor.

    pixels.jpg
    640 x 480 - 16K
    subpixels.jpg
    322 x 289 - 9K
    layers.jpg
    462 x 280 - 15K
  • apefosapefos

    well... there are many ways to do it... and the manufacturers probably are developing and experimenting this from some time ago... waiting the correct moment to turn this into products...

  • Vitaliy_KiselevVitaliy_Kiselev

    @apefos

    Actual solution can be to have DRAM buffer on chip (as Sony did) and use two times readout in HDR mode. In such case you will have info do repair blown out pixels.

  • apefosapefos

    I think Arri Alexa and RED HDRX also does this two readouts from the sensor.

  • Vitaliy_KiselevVitaliy_Kiselev

    I think Arri Alexa and RED HDRX also does this two readouts from the sensor.

    may be, but most probably they double fps and do not do it during same exposition. As it requires specially designed sensor.

  • apefosapefos

    the two readouts are done this way: the pixel starts to get excited by light and generate electrons, then some amount of electrons are read to get the information. But the pixel does not stop get excited by light, then a second amount of electrons are generated, the second read consider all electrons since the begining of light.

    The frame rate does not change, but there are two shutter speeds (electronic shutter read) for the exposures. A short time readout for highlights and a longer time readout for shadows. The longer read is the shot read + some extra time. The beginning of read is the same for both short and long readouts, the short readout stops first, the long readout keep going a little more time.

    This is not the best hdr because it can generate time differences more perceived in motion situations.

    If I would chose the best hdr, I think the dual layer sensor or each pixel with a subpixel can be the best ones, but they need to create new sensors which does not exist today. the checkerboard is not that bad, more easy to do with sensors we already have today just applying different isos in checkerboard design.

Add a Comment