Highlight recovery

Joanna · January 10, 2020, 8:45am

Actually, that was not highlight recovery, it was just a case of “dimming” the specular highlight, which has no detail anyway.

My “system” of exposure control comes from many years experience with large format film cameras, shooting B&W negatives and trying to get 14 stops of exposure range down to 8 stops for printing in a darkroom.

The same principles of “dynamic range compression” apply to digital sensors, in order to get the best out of them. You should never need to recover highlights in a digital image - because there is simply no detail to recover.

The problem for a lot of digital photographers is that they don’t realise that a RAW image is intrinsically flat in tonality and very little headroom for highlights, but it does have a load of shadow detail available. Take my camera (Nikon D810) - it has 14 stops dynamic range at 100 ISO but… that range is split very unevenly, with 12 stops below a measured exposure and only 2 stops above.

The trick is to carefully analyse the capabilities of the sensor in your camera and shoot according to the results. When I shoot colour transparency film, with Fuji Velvia 100 film, I am strictly limited to 5 stops range (3 stops under and 2 stops over). For landscape, it’s a case of either using physical graduated filters on the bright areas or not bothering to take the shot.

When your camera takes a RAW image, it also embeds a jpeg preview in the file and it is this that you sometimes catch a glimpse of when DxO first opens an image. However, if you have set DxO to a default preset of no corrections, you can then see an image that appears to be under-exposed. This is not the truth. Because the file is a 12-14 bit image, it contains a great deal more detail than is immediately obvious without any corrections - most of which is in the shadow areas.

Take a look at a high dynamic range image in FRV and switch on the shadow detail boost, then try the highlight detail boost - this is where you discover whether or not you have detail that is recoverable. If any of the “warning lights” appear, you will not be able to recover that area. Anything that is not marked as out of range is a valid detailed image.

I know the temptation of letting the camera do all the work with things like automatic exposure but, as I found out to my cost, it cannot always be relied on and if I find myself with a high range subject, the first thing I do is to switch to manual exposure mode and spot metering.

Once you have ascertained the range of your sensor, it is a simple matter of setting the exposure compensation dial to the “above measured” range of your sensor, which is usually between +1⅓ and +2 stops and taking your exposure reading from the brightest part of the scene. In your case, that would be the white plumage on a bird. As long as you don’t change the direction you are shooting in too much, this exposure will be valid for most other shots taken at the same time. If you were to change from shooting with the light to shooting against it, then you would have to re-meter of course.

Your images may well end up looking dark when you open them in DxO but this is simply because you have not yet “compressed” the range into the limited 8 stops the monitor displays. It is not under-exposed!

No it doesn’t; see my last paragraph. Also, whichever software you use, the same principles apply. I have half a dozen apps that I show to students for them to compare and they all show the same results before making any adjustments. The difficulty is getting used to how the different software does its thing.

With a high range image, I always start with Smart Lighting, which does some very clever stuff to reduce the range and spread the histogram. Sometimes I need more than others, sometimes less, but it is a good starting point. I very rarely use exposure correction. Just as with wet printing negatives, it is more about selective contrast enhancement than it is about controlling exposure. I highly recommend the highlight and shadow contrast adjustments that you get with FilmPack.

But, first things first - get to know what your sensor’s range and start to use manual exposure mode more

OXiDant · January 10, 2020, 9:12am

Good stuff you explained.
I second the filmpack contrast sliders.
That grey issue when recovering highlight is with the contrastslider highlight very good to control.

My lazy amateur set up.
1 my camera has idyn, it uses 1/3 2/3 3/3 stops ev underexposure measured from exposure reading and highlight detection. It also flatten contrast curve but only for soocjpeg.
So i practise the camera sees highlight which expand over the dynamic range of the sensor and start’s to underexpose in 3 steps. Effecting also my raw file… And only when i need it.
2 smart lighting is always on at 25% not more and i use the boxes to rase detailing in shadows and “spotlight” faces and objects and place one on the brightest spot in the image.
This sets my “dynamic range”
3 then i use centre weighted exposure compensation to give me the global exposure i want.
(smart lighting is keeping shadow and highlight inside the set bounderies when i lift or lower exposure comp.)
4 use both highlight sliders equaly in tone and contrast to adjust highlight spots, more contrast bring in detail but also risk of spotting, smearing and less contrast is smooting the bright area.
Same for midtones and shadows.
5 microcontrast, slider or clearview? Depends on the image. Sometimes i use controlpoints for selective clearview adding to add selective detailing due microcontrast.
6 tone curve, works great as blackpoint retrieval. I have a image( amfibi in water) around here were clearview doesn’t cut it and the dehaze isn’t enough. With tonecurve and clearview i get a good result.

And the clipping blinkies sun and moon are saturation warnings not only lumination warning so when a color is saturated but stil in details ok it stil goes flashing. (not something i would want and histogram is showing the srgb colorspace not the raw data which can be distracting.)
There is a strange thing in histogram when 255 is hit it shows a 0 -7 after that random.

Final, contrast and saturation and vibrance are my " punch" pop tools and i am playing with some profiles made for my camera but the generic one is less pushed so the more saturated images with more tone curve are often hit the clipping warnings.

Peter

mrihooper1 · January 10, 2020, 9:19am

Hi m9k,
Ha! guess we will have to wait that little bit longer for some simple highlight recovery slider.
"I don’t get this and it’s not in my workflow to ‘’first lower exposure to recover highlights and then mess with I don’t know what to recover the rest of the image’’.
Me too.

mrihooper1 · January 10, 2020, 9:46am

Hello Joanna,

Thanks for the lecture, your point on no headroom is of course correct. Don’t think I was implying otherwise.
My point is that short of underexposing, I’m not going to get from PL any highlight recovery. FRV shows plenty of highlight recovery available.
No warning lights.
If I turn off Smart Light, I am certainly getting a much flatter, and therefore manageble file.
My life is not geared around spending lots of time rescueing the impossible images. If I can recover the necessary detail in the highlights, bring out the best of the dark feather detail, in Lr, then it seems to me, that is where I’ll continue to work.

My lighting setup is pretty easy. Bright sun if I can get it, Cloudy if I have to, and go home once the shutter speed drops off the handholdable. Life is too short to polish bad apples.

All good.

mrihooper1 · January 10, 2020, 9:51am

Hi Peter,

Thanks for the hints on using the micro. I’m quite new to PL, and wanted to get my head around the issues I face with my images. When PL gets it right, it superb. Its the differences, I see when I encounter some washed highlights that is frustrating me.

If I run the nef through RawPower, I’m about to export a tiff with plenty of recovered detail.
I was hoping to find a simple solution in PL make a preset and live happily ever after.
But then people have been known to call me Peter Pan.
All good.

platypus · January 10, 2020, 10:29am

Check this post for similar posts: My first converted images are muddy and soft

OXiDant · January 10, 2020, 10:33am

When i am home i will search for some threads i posted about highlights exposure detailing and such.
I a kind of a techy and sink my teeth in a problem experimenting and ive hit some walls in the histograms readouts regarding the clipping warnings, how the prime does work with the automated “magic wants” . The behaviour under the hood so to speak.
But the filmpack extra’s does help alot in controling your rawfile.
( iam not use or used lightroom so don’t know if it’s better.)

Back to highlight exposure and such i have some examples who shows for you to get the max out PL.
(need to gather/collect them so need some time.)

Peter

platypus · January 10, 2020, 11:48am

Highlight recovery… sounds like an oxymoron. In most cases, we do not recover highlights but fix whatever the processing/presets have messed up…

I often find that the DxO Standard preset produces images that show blown highlights, while data in the raw file are perfectly okay as seen in RawDigger.

Changing colour rendition can greatly improve things. Try the setting with gamma 2.2 and see what you can get back, unless the highlights are grossly overexposed, in this case, highlights are simply dead!

geno · January 10, 2020, 11:52am

Absolutely, I was really upset about the standard setting, too. Messed up skin tones, way too hard contrast and punchy colours. The first thing I did in PL was to create a “flat” preset.

freixas · January 11, 2020, 8:18pm

Ok, here’s a chance to test Adobe (or whatever) against PhotoLab. I find many photographers are unwilling to share RAW images, so I will donate a RAW file for people to play with. It has clearly blown highlights, but it also has a lot of highlight detail near the blown out area. How well can your program recover this detail?

I have PL3 and Adobe Camera RAW (ACR) CS6, so that’s what I will contribute.

The original RAW is on my Google Drive at:

NOTE: If you arrive at this thread some weeks from now, the file may no longer be there.
NOTE: I didn’t pick this because it’s a pretty image. I picked it because there are details in the misty horizon that are tough to bring out.

On PL3, my default view enables default lens corrections, vignetting, noise reduction and chromatic aberrations. The rendering is Generic: Neutral colors, neutral tonality v2. Everything else is disabled.

On ACR, the default sharpening and lens correction is enabled. Everything else is at its default value.

I’ll start with six reduced images. For each program, I produced one image at default value, one image where I just moved the highlights level to -100 and one where I performed multiple adjustments. My goal was only to see how much highlight detail I could recover near the horizon. I didn’t spend a ton of time with the adjustments—that would seem to miss the point. I did spent more time on ACR than I did on PL3, because it appeared less successful.

First, the three PL3 version:

Now, the three ACR versions:

Finally, I selected a small area from all the images above and created a composite:

I see a lot of people saying that LR is doing a better job. Perhaps those of you with LR will give this a try. I wouldn’t object if someone also tried Capture One (or anything else). Ideally, the foreground shouldn’t turn totally black.

I thought of creating a new post for this exercise, but since this thread is titled “Highlight Recovery” and since it is a response to people here claiming that LR can recover highlights easier and better than PL, it seems appropriate here.

geno · January 11, 2020, 9:58pm

I have got a file, too. In Lightroom it is just a highlight reduction of -22 or so. In PL…well, not that easy, it takes way more (too much) time. I know, a sunstar, again
Lightroom 6.14:

DSC_0104.NEF (22,5 MB)

platypus · January 11, 2020, 10:00pm

According to RawDigger, the image has a fair amount of really blown highlights. They are gone, nothing to do unless you want to fake something.

Let’s also consider, that an image like this might profit from the dreamy quality of the light and that you might want to reduce structure and clarity instead of introducing some weird lighting as in the third ACR example. I’d consider the upper fourth of the DxO Multiple version acceptable and I’d try to get rid of some of the halo effect at the edge of the foreground hill.

freixas · January 11, 2020, 10:45pm

I looked at this in RawDigger. I understand that there are areas that are fully blown. I don’t expect any details to be recovered from those areas. The areas of interest are just below the blown out areas.

I think the esthetics are irrelevant for this exercise. I’m certainly not interested. Just see what highlights you can recover.

Joanna · January 11, 2020, 11:38pm

Surely, it’s simple. If Raw Digger says there is nothing to recover, then why bother?

It’s like me trying to get something out of a B&W negative that is 100% clear - no amount of wishing, hoping or dreaming, let alone trying various software can ever recover anything from those areas.

Once again, there is no way you are ever going to recover anything from an image of the sun - it simply defies the laws of physics.

freixas · January 12, 2020, 1:09am

What I actually said is to ignore the area that is blown. The area below that is not blown. Recover the highlights there. Look at the composite crops if this is still unclear to you.

Either you can make the distant hills pop out or you can’t. PL3 can. ACR has more trouble. LR…well, until someone gives it a go, it’s an open question.

It’s not really clear what you want to recover. I’m guessing you want the beams radiating from the sun to be a bit more defined. That’s about the only place where there are highlights that are just at the edge of being recoverable.

You are correct that if this is the only thing you care about, LR appears to do a better job with one slider. But the detail elsewhere in the sky doesn’t look very good to me. I gave it a try, though. For best results, overlay my image over yours in whatever tool you have and turn the layer off and on. In my viewer, the sun beams look close to the same in level of detail, but the sky detail overall is clearer.

Because what I was trying to do was match what you were trying for, it did involve way too many local adjustments. I would not normally have bothered trying to fix the beams.

In the image I submitted, entire mountains are hidden in the highlight details, which is why I think it’s a more useful test. How about giving my image a go?

JimScott · January 12, 2020, 1:35am

It sounds like You are trying to avoid “sunstars” - if I understand this correctly then shoot with a larger aperture - such as f/11 or below.

geno · January 12, 2020, 6:42am

@JimScott, actually I like sunstars and try to balance between them and too much lens flare (which pretty often doesn’t work out, as you can see…).
In LR the transition between the center and the rays is smooth, in PL I get artifacts - or I have to really spend time on the transition.

geno · January 12, 2020, 7:27am

This is the point - in LR I am done in less than an minute while PL takes quite some time.

Edit: @freixas, yes, your version contains way more details and sure looks better overall. Thanks for your edit - would you share the dop file?

Pieloe · January 12, 2020, 8:34am

I don’t understand the request
For you, the brown color on the sun stars is a good answer !?
I tried PL with and without
SmartLighting
ClearView
Microcontraste
HighLight Fine contrast (FilmPack)
I have good results and never seen this ugly Lr look.
Pascal

Joanna · January 12, 2020, 8:42am

I know I might seem like a bit of a bore but, as a very techie type photographer, coming from a film background, I am finding it quite incredible that folks are still trying to discuss highlight recovery by discussing images that contain irrecoverable highlights.

Let’s take having the sun in the image for starters.

If you wanted to take a photo of the sun, clearly defined as a circle, you are going to need an ND filter of around 16 stops. Since the best DSLRs only have a range of 14 stops, this means that you are trying to image something that is beyond the range of any sensor yet made and, if you were to attempt to get a well-defined circular sun, there is zero chance that you would render anything else in the image as anything other than black.

To put 16 stops in some kind of context, it might seem like a small number but, in fact, it is on a logarithmic scale and is the equivalent of something being 65,536 times as bright as something else.

So, you are trying to create an image that contains an area that is around 65,000 times as bright as the area of sky next to it. Your eye can’t survive looking at it for any reasonable period of time without going blind. The sensor can’t cope with being exposed to it for to long without pixels getting irretrievably burnt out. And you are expecting an image editing app to be able to recover detail?

It’s not just the transition between the sun and the sky next to it, you also have to consider diffraction and flare caused by such a bright light passing through the lens.

Allowing such a bright light to pass directly through the lens will induce diffraction. What you are seeing with “sunstars” is simply the light diffracting off the blades of the diaphragm; two rays for every blade in the diaphragm. Increase the size of the aperture and the rays will get less but diffraction through the more circular opening will now create a less distinct haze, which is going to be many times more impenetrable than any distancing haze.

Then there’s the issue of flare. If a point source of light hits the interior of a lens, it’s going to start bouncing around off any imperfections in the glass and junctions between lens elements and each other or its mounting inside the lens body. In fact, even indirect contra-jour lighting is going to cause a haze over and above anything in the distance. Whether it’s just distinct internal reflections or more general flare from light coming towards the camera, all this adds up to a lack of detail and contrast in those bright areas that can spread to the rest of the image as well.

To avoid flare, and subsequent loss of detail and contrast, when the sun is outside of the image, you can use a compendium style lens hood :
1-m
… which can be precisely adjusted to shield the lens from the direct light hitting the inside of the lens.

For those times when the sun is visible in the image, you are going to have to accept that anything you do to “improve” the image is going to be limited by the laws of physics and that, in some circumstances, there isn’t an app in the world that will achieve perfection.