Speed Up Exports that Use Prime Noise Reduction?

(Svetlana G.) #21

Hello Alec,

Thank you for the files. I have no problem with your images in terms of customizing (I mean the preview/thumbnails update when applying different corrections and the sliders behavior is smooth). Though my machine is even less powerful than most of you have (PC):
Processor: Intel® Core™ i5-3330 CPU @ 3.00GHz (4 CPUs), ~3.2GHz
Memory: 16384MB RAM
But processing with PRIME takes ~ 1.5-2 mins per image. And I agree we still need improvements on the performance.

John, let me draw @Benoit and @wolf attention to your problem. In the meantime could you remind us your PC configuration?

Thank you,
Regards,
Svetlana G.

(John Barrett) #22

My set up is i7-4900MQ CPU @ 2.80GHz 16 GB memory

mostly running on Intel® HD Graphics 4600 with NVIDIA GeForce GT 750M

2 SSD,s 500Mb and 1Tb
I use a Dell U2433 at 98%Adobe RGB

Its not a major problem, just slow with large files, what concerns me is I only do a few images at a time, 10-20 at most. Many users I know will have many more and it must be off putting to have such a slow processing (my son does event photography so may have into the 1000 plus images. He does a basic processing on them (mostly low light so uses RAW) and then runs it on them all. He did some on my laptop and found PL good but far to slow for this and I am sure many others will be in the same position). I suspect the problem is the relevant programming was done at a time of smaller files and lower screen output was the norm
For me the largest pain is the slow lag when using local adjustments, its faster to close the local adjustments and see what the result is and then reopen and make further changes if needed. PRIME I just rate images that needed it and select them at the end and add PRIME to them. Its a bit of a pain if I go back to them as they are then much slower to load again, but if I expect that I just select them again and remove PRIME!

1 Like
(Wolf) #23

Hello,

Generally speaking, we know that PRIME is slow. We are constantly researching ways to accelerate it (long-term users might recall that it was way slower when introduced back in the days of OpticsPro 9), but it’s not easy, and I acknowledge that we’re not satisfied yet.

Still, I can recall some bits of information about PRIME:

  1. Because of its slowness, PRIME is only applied during image export and for the magnifier in the Noise Reduction palette. You cannot see its effect in the main Preview window. In turn, enabling it or not should have no impact on the lag between changing a cursor value and seing the updated image in the main Preview window, neither for local nor for global adjustments. Collapsing or hiding the Noise Reduction palette might speed up the interface a very little bit.

  2. PRIME’s speed (during export and for the magnifier in the Noise Reduction palette) mainly depends on the CPU. If PRIME matters to you, choose a high clock speed and many cores. For exporting a single image (or a very small number of images), clock speed is most important, for exporting many images, cores are more important. Desktop CPUs are generally faster than mobile CPUs, even if both are labelled “i5” or “i7”.

On my PC (i7-6700K @ 4GHz), exporting @John7’s three images from the 5DS with PRIME enabled took 4 min 20 sec in total, which seems plausible given the huge number of pixels.

4 Likes
(John Barrett) #24

Wolf I agree over the export, my main concern is the slowness of local adjustments. Just done done some 7D2 and its not much faster in updating the image than with the much bigger 5DS files. With exports you can play a game, make a drink, go something else, but editing you are siting there waiting for the result to see if its right, worked (I fined local adjustments a bit unrelatable as to what they will do and often have to redo them(or even take them off).

1 Like
(Alec Kinnear) #25

Thanks for the tips. That’s really good information for PhotoLab users and much appreciated. I will hide the Noise Panel as well as not enable it to the very end. For other users coming across this thread, really, really Prime Noise should only be switched on at the very end of changes. If a photographer needs to preview an image with noise suppressed HQ Fast is good enough and it’s even good enough for production (certain images look better). HQ Fast is much better than C1 noise reduction and somewhat better than Lightroom noise reduction.

Could you explain how and where GPU acceleration affects performance for Windows and Mac to help Mike, John and I decide how to configure our machines please? I do have another Radeon RX580 with which I could replace my GTX 980 if it would make a big difference in performance. I haven’t just substituted and tested as it would require an OS upgrade to do so.

(Peter) #26

Interesting, i was playing with essential free version before i bought the elite and i found the HQFast (only option) in default mode/setting not the greatest compared with my other raw converter (Silkypix) in defaultsetting. And a DNG of DxO op10? into second raw converter wasn’t really “great” in noise reduction, so i used Define2 to smooth the noise further. (Didn’t used much of the sliders and started testing Silkypix plus Dfine2 against Primenoise.) Then i started to read in about the control sliders. (default luminance 40 in prime and Define2 (also in " automode" when exporting a tiff vs 80+ luminance prime directly jpeg is around the same. (conclusion some practise in controlling the sliders in HQ and Prime does help. :wink: )

Question for @wolf:
I am sure this is asked earlier but out of curiosity:
When Prime is selected will HQfast be used for preview purposes realtime on screen when noisereduction is enabled?

i know prime is activated and applied in export only and in the small preview window in the menutoolbar in the magnifier, so do we need to switch between HQ fast and Prime to see the noise reduction on screen before exporting? Or is also HQ fast noisereduction not realtime in preview done?

(Alec Kinnear) #27

It looks HQ Fast on screen to me when Prime Noise Reduction is enabled. The only place we see Prime is in that little window. If there’s a different answer to that question I’d be interested to know if there’s a third noise reduction system (preview). PhotoLab prides itself on almost real time accurate preview (many photographers are very annoyed at how poor the C1 preview is) so it would be strange if it wasn’t full HQ Fast shown now.

(Wolf) #28

When magnification is >= 75%, HQ (Fast) noise reduction is indeed used in the Preview window, independently of whether PRIME or HQ (Fast) is selected.
When magnification is < 75% or while a slider is moved, we apply a different raw conversion (including a simpler noise reduction algorithm) to speed up the display. This aims at delivering results similar to HQ (Fast), but it’s not strictly equal.
I strongly recommend to visualize the image at >= 75% to have meaningful feedback on denoising sliders when using HQ (Fast). And you must use the magnifier in the Noise Reduction palette to have meaningful feedback on denoising sliders when using PRIME.

About the GPU, I can give some insight on what we do today. But please note that these details are subject to change at any moment, as our processing evolves. Currently, the Mac version of PhotoLab 2 does not use the GPU much. The Windows version makes better use of the GPU, but mainly during image export (and it does not accelerate PRIME). So, as of today, to get fast slider updates, the best thing you can do is get a fast CPU.

3 Likes
(Peter) #29

Thanks for the clear answer. So as Uncoy guest it has three denoise algorithm’s.
1- Full image view algoritm which is almost the same quality as HQ and used for viewing your image wile editing things like exposure and such. under the 75% zoom rate
2- HQ denoise view above 75% zoom view., at this stage HQ is applied on the image and viewed on screen and give immidate apply on the image and thus on preview wile changing the sliders .
(So when you are in PRIME mode and go in 75%/above HQ is applying the slider corrections you do in the PRIME TAB to give you a visualisation of what PRIME will do by export? or only when HQ tab is active the realtime preview is active for HQ denoise control sliders?
(Same as CA-correction, microcontrast/sharpening >75%.)
3- PRIME when you hit export.

(When HQ is working and adapt to what you are applying in the sliders when you are in PRIME-mode at 75% that will help to see some preview about expected outcome.)

Last question:
It (denoise in full image preview) aims to similair results as HQ in full image view so when you can toggle between HQ (algorithm 2) and algorithm 1 you won’t see any (big)change between the full image and -75%? (I ask this because by CA-correction it’s showes fairly obvious. Big difference between full image and zoomed til CA -correction kick’s in. or is HQ needed in 4K screens for a good preview of denoising?

P.S.
( because when screen’s go bigger and 4K is more default screen, the overal full view editing wil be done more and more which will need full view applying corrections to see a image which looks like the exported one in preview.)

Interesting al this balancing between working speed and accuracy. The FF- file sizes are getting bigger by higher Pixelcount so the CPU and Graphiccards needs to be utilized to the max which can be resulting in over heated mounts and chiphousing. (i have my CPU limited to 90% because when i use Vidcoder for big files it stays at max CPUcore-usage for hours and you don’t want to burn your chipset to mush with hours of overheating.I tested the auto coolingfan software of windows 10 and it’s not enough to keep the temp safe inside the deltatemp for healty live of the cores in the CPU and a limitation of 90% does keep it inside the delta T four hours of full stress :grinning:.)
So if DxO can stress the chipset and heatup the chips (IC’s) due the full useage of CPU and GPU a small temperature monitoring of those IC’s would be good to keep the CPUcore healty.
Edit: Piriform Speccy does give you all temps even of each core and the workload.

(hm i think i will test how much DxO in export mode is pushing my PC to the borders of it’s capability’s just to see if it’s hit the 90% limitation, if not you still have some headroom :grinning:.)

(Mark) #30

I’m running on a somewhat slower windows 10 machine. I always export just one image at a time. My much smaller Canon 7D Mark II raw files average 25-30 mb. Exporting them with PRIME selected into 100%, 300 ppi jpegs takes on average 25-30 seconds each. So while processing large numbers of much higher resolution raw files may be disappointingly slow for others, in my situation the performance is very acceptable. In fact, PhotoLab performance in general is excellent on my machine, which I’m assuming is also related to the smaller raw file size.

Since you also brought up the subject of viewing at greater than 75%, I want to once again bring up my pet peeve with PhotoLab, the inability to see the effects of sharpening below 75%. This can be a real time-consuming pain, and makes it much more difficult to see the overall effect of sharpening on the final image. It also makes micro contrast adjustments more difficult since the combination of sharpening and micro contrast can sometimes cause a result that looks over-sharpened. If you could work on, or consider lowering the view of sharpening to 50% that would be extremely helpful. Practically all your competitors show sharpening at lower levels of magnification. I understand that there may be performance considerations, but I hope it’s in your long term plans

Mark

(Mark) #31

How slow are your 7D Mark II local adjustments? While not instantaneous or even fast, it generally does not take that long for the image to reflesh. It takes anywhere from 5-10 seconds to maybe 25 seconds to refresh. And how long do you are 7D Mark II exports take with and without PRIME?

Mark

(Peter) #32

Plus one; a dedicated button to activate/deactivate this view below 75% would be helpfull to keep preformens up wen needed.

1 Like
(John Barrett) #33

The problem is having to try changes to see if they work, then having to go back to redo if they are not right, it all adds up. But the tr=ests I just did were
Local adjustment refresh about 10-25sec

Saving 12 sec with prim 40sec

With a 5DS a local adjustment 30sec (couldn’t be kept as didn’t work so would have needed to be redone!)

Saving 39sec with prime 2m10sec

The increases in prime saving is greatly increasing as you increase the size of file, the other are more or less linked to file size. As file sizes are increasing this is a potential problem as I don’t understand why there should be a very large slowing as you jump from the 25 to 50Mb between the 2 cameras.

But that for me is less of a problem, its still the lag between adjustments and seeing there results that the working problem, exporting I just do something else.

1 Like
(Pathal) #34

I concur to @OXiDant that any modification of 75% level must be optional.

(Alec Kinnear) #35

Mark and @Pathal, we are asking DxO for faster previews and workflow and at the same time you are both pushing them for more accurate previews. The two directions are more or less incompatible. When working on sharpening or noise reduction, the only view which is relevant is 100%. What would help performance is if PhotoLab would first calculate just the onscreen pixels at 100% view rather than the full screen.

The reason performance is so poor right now is that most of the time PhotoLab is building a pixel perfect full image and then displaying just the part we need. For lower than 75% resolution apparently, there is already a preview proxy (without visible sharpening) mainly for colour. I’m delighted to hear that and will take advantage of it to do more of my work at Fit to Screen (hopefully PhotoLab will be a bit faster).

John Barrett, you wrote:

With a 5DS a local adjustment 30sec (couldn’t be kept as didn’t work so would have needed to be redone!)

Are you doing your local corrections with both Lens Sharpening and Prime Noise turned on? If so, I highly recommend turning then off and adding them only at the end. If you have to turn one on, make sure it’s just Lens Sharpening (I can see cases where one wants to see the sharpened image before making local adjustments).

You’re right, the lag between making an adjustment and seeing the result is really an issue. We need improved fast preview processing from PhotoLab (with a checkbox on/off). In the meantime, PhotoLab also needs us to follow a sensible workflow. If photographers turn on all the processor heavy controls early in our digital darkroom workflow, PhotoLab will be very slow.

Concur on the export with Prime Noise not being an issue. Final exports with really high quality noise reduction take all the time they take. Final exports cannot be accelerated with GPU or preview proxy versions. Here’s a discussion of some of the quality issues with GPU acceleration. It’s not what you want to be using to create master files.

HQ Fast is there for event photographers who have to get pictures out quickly. Even HQ Fast export must be full CPU pixel perfect processing. There’s few photographers who want to export glitchy masters, even if it saves time.

1 Like
(Peter) #36

Not my text quoted under my name but hence we all can get both.
Present time we we have no settings in display to personalize our needs.
What we need are display settings which we can toggle to need.

  • quick fast view,: No noise correction, no CA correction, no sharpening, no any thing which slows down the preview buildup.
  • present mode with 75% threshold for som parts.
  • Full rendered preview on full image. Included noise reduction, sharpening, microcontrast, clearview, CA correction. The hole bunch.

:grinning:

(Pathal) #37

@uncoy,

Alec, I think that you “preview” too fast my post.

When I say

it means that if DxO decide to propose full denoising when viewing at zooming below 75%, it must be optional for the user, otherwise I will wait still more for cursor action.

Of course if they solve at the same time the performance issue to give a zoom at 100% with all denoising instantly, ok.
But now we seems to be far from that target.
SO we need to keep the possibility to work at 74% without to be too much impacted by high resolution denoise.

(Alec Kinnear) #38

Thanks Pathal. It seems we are in agreement. This perfect preview is lovely but it makes it impossible for DxO to offer real time sliders. There has to be a compromise somewhere. Like you I’m fine with Fit to Screen (or anything less than 100%) being an accelerated preview. If I want to see pixel perfect, I can either switch to 100% or do a quick export. There could also be a switch between accelerated preview and pixel perfect right there on the screen at all times.

#39

Hi,

I just replaced the HD on my laptop with an SSD to get better overall performance and I was not disappointed. The time to start, wake after sleep, and load applications is way faster (I’d say 3x to 10x faster, though didn’t test to be sure) than before. It actually makes the machine pleasant to use, which it wasn’t before because it was so slow.

After the change I tested to see what the effect of the SSD was on DxO PL export time using Prime NR. The laptop is a HP Laptop 17-g161us (pretty underpowered and old) running Windows 10. I processed 10 raw files from my Canon 5D3. I made sure no other apps were running and everything was the same between the 2 tests. With the HD the time was 18:57 and with the SSD the time was 17:34.

To me this isn’t a surprise. The drive light didn’t come on much when exporting on the HD so it seemed likely that the SSD wouldn’t help much.

Mike