Speed Up Exports that Use Prime Noise Reduction?

Pascal, that’s good news that a new display engine is on its way. It will help all of us who regularly develop moderate to large numbers of photos in DxO PhotoLab. Very glad to hear it. Any idea on when the new display engine will be released or put into beta?

This information is public. This new engine is used in the full screen and Local adjustments modes.
Pascal

I don’t have 4K screens to test with (Windows). At 2K slider processing (preview) changes are pretty much instantaneous. (real time). De-Noise is enable 100% of the time, Lens profiles too. Smart Lighting as well. My only lag is batch export. Just a matter of me building a new machine if I stay at 2K.

You are shooting the Canon 6D Mark II? For the record, that’s a 26 MP camera with 6240 × 4160 pixels, in file size about equivalent to the A7 III on which I have much lower lag than on the 5DS R. Could you test with some Canon 5DS R files?

I’ve uploaded a few here. 5DS R files are too big for the direct upload to feedback.dxo.com – it looks like DxO really don’t like files over 50 MB, not in PhotoLab not on the forum. These files are ISO 6400 so they should give PhotoLab a good workout. You can download all three at once by clicking Télécharger le dossier in the upper right hand corner.

@mwsilvers I believe you had asked about some 5DS R files? There they are.

1 Like

I use a 5DS and yes PL struggles (Windows version). Using prime, local adjustments and most changes are slow. The others I use are Sony a600 and Canon 7D2 both are 1/2 the file size I know but changes are very much faster and exports are on a different level. 5DS corrections even in windows are sluggish and if I have a adjustment I am copying over 10-15 images solitaire comes in handy for time filling as it does for exporting! If you delete 5DS you have to allow a full time for a load before deleting or all you get us “unknown error” have have to repeat it. I find you just have to expect to take a long time to deal with them which I can see some one with lots of images might not what to have to allow for. I expect it stems from PL being developed before such large files became so much more widely used.

1 Like

Thanks John for the field report. It’s both a relief and a worry to know that 5DS/5DSR (and probably D850, A7Rx files too) are very slow on the current Windows engine. It behooves DxO to improve the processing engine (probably mainly by adding a proxy front end version for adjustments along with GPU acceleration, exports will continue to be slow, it’s a lot of data to move. Adobe found GPU acceleration should not be used for pixel perfect final exports) before adding more modules like DAM solution.

If PhotoLab is cripplingly slow with the current cameras (64 to 100 MB are on their way apparently, not that I’m lining up for it), how do DxO expect professional photographers to even consider PhotoLab? Well apart from rescue software for very high ISO images. Most pros avoid those kind of images in any case. My first question to DxO management would be: Is PhotoLab software for amateur users with low to medium resolution cameras only?

Export is less an issue Jon, I think, than fluidity of interface when editing. If there’s a big rush job, HQ Fast noise reduction is more than adequate: my export times drop from close to two minutes per file to twenty to thirty seconds. I sometimes prefer HQ Fast in fact, as it looks more like film grain, whereas Prime can create this sort of artificially clean look, particularly in large doses. I usually set Prime at about 12 which strikes a balance between reduced noise and a natural appearance.

Keep in mind that for export, DxO won’t be able to use GPU acceleration as there are fine errors in colour and detail which come up. Not an issue for preview, but not what you want for final output. These errors are why Adobe has problems with GPU acceleration for final output in Premiere.

In short, I’m not worried about the long export times but I’m really worried about how long it takes to go through a session with larger files due to the delay on the sliders. Delays during editing discourage experimentation and fine tuning, making PhotoLab less attractive to work with.

1 Like

Yeah, I hear you and agree that fluidity of the interface is a BIGGER issue, but I’d still love to see some speed improvements to the exporting with PRIME.

  • Jon

Hi Alec!

Based on the images You provided I believe it is more likely the 4K screen res (readout) that is causing the problems… using one of the images You provided “8M8A3121-C1.CR2” (78.3 MB) as the basis for my supposition.

I am using a fully loaded mid-2011 iMac (High Sierra) 27" 2K monitor; PhotoLab (1.2.2); Nikon D70 (6 Mp); D300 (12 Mp); Sony DX100 series (20 Mp) as touchstones in the following comparisons.

My hardware is definitely not state-of-the-art which I believe lends more credence to my initial supposition.

There is no significant difference between the on-screen updates of the D70; D300; RX100 images or your image on my system. They occur for the most part near instantaneously - even when using “Lens Sharpness” or “Prime Noise Reduction”. “Local adjustments” ditto.

The “loupe” image for “Prime Nose Reduction” has exactly the same pixel dimensions for any of the different camera resolutions mentioned above. So the loupe updating should not be a factor vis-a-vis image resolution. The same amount of pixels being processed.

The 100% screen proxy updates (accurately) at the same time as the loupe what ever the image resolution.

Below is a “screen recording” example of PL’s “loupe” from your 78.3 MB image - as I move from selected area to selected area (which takes a couple of seconds when I move the mouse and click a new selection area). Note the only time the “circular arrow” update icon shows up is on one of the noisier samples (last one - “White Shorts”) - not even 2 seconds. Otherwise updates are nearly instantaneous.

Just below this video is a screen capture of the original “White Shorts” image (with noise).


“White Shorts” screen capture (w/noise)
14%20PM

Hi,

Thanks to everyone for taking the time to respond.

First I feel really lucky that I’m on a PC instead of a Mac. My 28 seconds isn’t bad considering what you’re all seeing. I have a 4K monitor and I haven’t noticed significant problems with the sliders even though I turn on Prime early in my workflow and I never turn off Lens Sharpness. I notice a minor degradation if I’m editing and exporting at the same time but not enough to stop doing it. The sliders aren’t as smooth as I’d like but it’s not a problem.

Doubling my export throughput with more processing cores would be great. I’d love to do it but I’m just not ready for that cash outlay. The only thing I can consider now are cheaper upgrades, probably a more powerful display card and/or SSD.

My next step is to see if I can borrow a more powerful display card that will work with my 300W PSU and see what happens. If that helps then I’ll upgrade to a better card, maybe a GTX 1050 Ti. Re the SSD I’m not sure what to think. When I’m exporting or editing I see next to no drive activity, either watching the light or the MS Task Manager, so it seems unintuitive that an SSD would speed up PhotoLab. But even if an SSD didn’t help PhotoLab that much it would still make the machine much more usable because it speeds up so many other things.

If I can install a borrowed or bought display card or SSD I’ll report back on my results.

Thanks,
Mike

1 Like

I have better performance on Mac with Radeon cards (even an old 6750 in my 2011 MBP 17" i7) than with Nvidia. PhotoLab is apparently running with OpenCL which sometimes performs better on Radeon cards. Even an nVidia GTX 980 doesn’t seem very spritely on my six core Mac Pro. This may be platform specific – it would be great if DxO offered some more detailed information about which cards run well and which ones less well. This is all they tell us:

Windows: OpenCl 1.2-capable graphic card with 1GB of video memory to handle OpenCL acceleration
Mac: Graphics card with 512 MB of video memory to handle GPU acceleration

I’m disappointed to see the latest versions require at least 10.12 on Mac. My main working environment is OS X 10.11 and I don’t plan to change that as 10.11 is very stable and headache free. I do keep one computer (my main video and photo computer) with 10.14 for video application (FCPX specifically). Not supporting older versions of OS X will alienate many photographers who would prefer to work on their pictures than spend timing fighting their OS.

There’s no new technology in PhotoLab which requires the latest OS. Apple does make it hard for developers to support older OS by deliberately making them the latest xCode unable to build for older OS X. The workaround is either to use an older xCode for the main build (older versions will run on newer OS of course) or to do two builds. FastRawViewer manages to do this.

Hello Alec,

Thank you for the files. I have no problem with your images in terms of customizing (I mean the preview/thumbnails update when applying different corrections and the sliders behavior is smooth). Though my machine is even less powerful than most of you have (PC):
Processor: Intel® Core™ i5-3330 CPU @ 3.00GHz (4 CPUs), ~3.2GHz
Memory: 16384MB RAM
But processing with PRIME takes ~ 1.5-2 mins per image. And I agree we still need improvements on the performance.

John, let me draw @Benoit and @wolf attention to your problem. In the meantime could you remind us your PC configuration?

Thank you,
Regards,
Svetlana G.

My set up is i7-4900MQ CPU @ 2.80GHz 16 GB memory

mostly running on Intel® HD Graphics 4600 with NVIDIA GeForce GT 750M

2 SSD,s 500Mb and 1Tb
I use a Dell U2433 at 98%Adobe RGB

Its not a major problem, just slow with large files, what concerns me is I only do a few images at a time, 10-20 at most. Many users I know will have many more and it must be off putting to have such a slow processing (my son does event photography so may have into the 1000 plus images. He does a basic processing on them (mostly low light so uses RAW) and then runs it on them all. He did some on my laptop and found PL good but far to slow for this and I am sure many others will be in the same position). I suspect the problem is the relevant programming was done at a time of smaller files and lower screen output was the norm
For me the largest pain is the slow lag when using local adjustments, its faster to close the local adjustments and see what the result is and then reopen and make further changes if needed. PRIME I just rate images that needed it and select them at the end and add PRIME to them. Its a bit of a pain if I go back to them as they are then much slower to load again, but if I expect that I just select them again and remove PRIME!

1 Like

Hello,

Generally speaking, we know that PRIME is slow. We are constantly researching ways to accelerate it (long-term users might recall that it was way slower when introduced back in the days of OpticsPro 9), but it’s not easy, and I acknowledge that we’re not satisfied yet.

Still, I can recall some bits of information about PRIME:

  1. Because of its slowness, PRIME is only applied during image export and for the magnifier in the Noise Reduction palette. You cannot see its effect in the main Preview window. In turn, enabling it or not should have no impact on the lag between changing a cursor value and seing the updated image in the main Preview window, neither for local nor for global adjustments. Collapsing or hiding the Noise Reduction palette might speed up the interface a very little bit.

  2. PRIME’s speed (during export and for the magnifier in the Noise Reduction palette) mainly depends on the CPU. If PRIME matters to you, choose a high clock speed and many cores. For exporting a single image (or a very small number of images), clock speed is most important, for exporting many images, cores are more important. Desktop CPUs are generally faster than mobile CPUs, even if both are labelled “i5” or “i7”.

On my PC (i7-6700K @ 4GHz), exporting @John7’s three images from the 5DS with PRIME enabled took 4 min 20 sec in total, which seems plausible given the huge number of pixels.

4 Likes

Wolf I agree over the export, my main concern is the slowness of local adjustments. Just done done some 7D2 and its not much faster in updating the image than with the much bigger 5DS files. With exports you can play a game, make a drink, go something else, but editing you are siting there waiting for the result to see if its right, worked (I fined local adjustments a bit unrelatable as to what they will do and often have to redo them(or even take them off).

1 Like

Thanks for the tips. That’s really good information for PhotoLab users and much appreciated. I will hide the Noise Panel as well as not enable it to the very end. For other users coming across this thread, really, really Prime Noise should only be switched on at the very end of changes. If a photographer needs to preview an image with noise suppressed HQ Fast is good enough and it’s even good enough for production (certain images look better). HQ Fast is much better than C1 noise reduction and somewhat better than Lightroom noise reduction.

Could you explain how and where GPU acceleration affects performance for Windows and Mac to help Mike, John and I decide how to configure our machines please? I do have another Radeon RX580 with which I could replace my GTX 980 if it would make a big difference in performance. I haven’t just substituted and tested as it would require an OS upgrade to do so.

Interesting, i was playing with essential free version before i bought the elite and i found the HQFast (only option) in default mode/setting not the greatest compared with my other raw converter (Silkypix) in defaultsetting. And a DNG of DxO op10? into second raw converter wasn’t really “great” in noise reduction, so i used Define2 to smooth the noise further. (Didn’t used much of the sliders and started testing Silkypix plus Dfine2 against Primenoise.) Then i started to read in about the control sliders. (default luminance 40 in prime and Define2 (also in " automode" when exporting a tiff vs 80+ luminance prime directly jpeg is around the same. (conclusion some practise in controlling the sliders in HQ and Prime does help. :wink: )

Question for @wolf:
I am sure this is asked earlier but out of curiosity:
When Prime is selected will HQfast be used for preview purposes realtime on screen when noisereduction is enabled?

i know prime is activated and applied in export only and in the small preview window in the menutoolbar in the magnifier, so do we need to switch between HQ fast and Prime to see the noise reduction on screen before exporting? Or is also HQ fast noisereduction not realtime in preview done?

It looks HQ Fast on screen to me when Prime Noise Reduction is enabled. The only place we see Prime is in that little window. If there’s a different answer to that question I’d be interested to know if there’s a third noise reduction system (preview). PhotoLab prides itself on almost real time accurate preview (many photographers are very annoyed at how poor the C1 preview is) so it would be strange if it wasn’t full HQ Fast shown now.

When magnification is >= 75%, HQ (Fast) noise reduction is indeed used in the Preview window, independently of whether PRIME or HQ (Fast) is selected.
When magnification is < 75% or while a slider is moved, we apply a different raw conversion (including a simpler noise reduction algorithm) to speed up the display. This aims at delivering results similar to HQ (Fast), but it’s not strictly equal.
I strongly recommend to visualize the image at >= 75% to have meaningful feedback on denoising sliders when using HQ (Fast). And you must use the magnifier in the Noise Reduction palette to have meaningful feedback on denoising sliders when using PRIME.

About the GPU, I can give some insight on what we do today. But please note that these details are subject to change at any moment, as our processing evolves. Currently, the Mac version of PhotoLab 2 does not use the GPU much. The Windows version makes better use of the GPU, but mainly during image export (and it does not accelerate PRIME). So, as of today, to get fast slider updates, the best thing you can do is get a fast CPU.

4 Likes

Thanks for the clear answer. So as Uncoy guest it has three denoise algorithm’s.
1- Full image view algoritm which is almost the same quality as HQ and used for viewing your image wile editing things like exposure and such. under the 75% zoom rate
2- HQ denoise view above 75% zoom view., at this stage HQ is applied on the image and viewed on screen and give immidate apply on the image and thus on preview wile changing the sliders .
(So when you are in PRIME mode and go in 75%/above HQ is applying the slider corrections you do in the PRIME TAB to give you a visualisation of what PRIME will do by export? or only when HQ tab is active the realtime preview is active for HQ denoise control sliders?
(Same as CA-correction, microcontrast/sharpening >75%.)
3- PRIME when you hit export.

(When HQ is working and adapt to what you are applying in the sliders when you are in PRIME-mode at 75% that will help to see some preview about expected outcome.)

Last question:
It (denoise in full image preview) aims to similair results as HQ in full image view so when you can toggle between HQ (algorithm 2) and algorithm 1 you won’t see any (big)change between the full image and -75%? (I ask this because by CA-correction it’s showes fairly obvious. Big difference between full image and zoomed til CA -correction kick’s in. or is HQ needed in 4K screens for a good preview of denoising?

P.S.
( because when screen’s go bigger and 4K is more default screen, the overal full view editing wil be done more and more which will need full view applying corrections to see a image which looks like the exported one in preview.)

Interesting al this balancing between working speed and accuracy. The FF- file sizes are getting bigger by higher Pixelcount so the CPU and Graphiccards needs to be utilized to the max which can be resulting in over heated mounts and chiphousing. (i have my CPU limited to 90% because when i use Vidcoder for big files it stays at max CPUcore-usage for hours and you don’t want to burn your chipset to mush with hours of overheating.I tested the auto coolingfan software of windows 10 and it’s not enough to keep the temp safe inside the deltatemp for healty live of the cores in the CPU and a limitation of 90% does keep it inside the delta T four hours of full stress :grinning:.)
So if DxO can stress the chipset and heatup the chips (IC’s) due the full useage of CPU and GPU a small temperature monitoring of those IC’s would be good to keep the CPUcore healty.
Edit: Piriform Speccy does give you all temps even of each core and the workload.

(hm i think i will test how much DxO in export mode is pushing my PC to the borders of it’s capability’s just to see if it’s hit the 90% limitation, if not you still have some headroom :grinning:.)