Preferences - Advanced/Performance/Processing Images

DxO PhotoLab macOS topics
1

I am trying to understand exactly what this preference does. I had assumed it controlled the number of CPU cores used by the application. But, the User Guide simply says that this is a way to control the impact of the DxO application on the system - you can reduce the load (if you want to run other apps at the same time) by reducing the number of images processed in parallel.

I have been watching what is going on under different settings. I have tested 8 images, 4 images, and 1 image options. In each case, the app uses all the CPU cores. I have an (Intel) iMac Pro with a Xeon W 8-core CPU and a MBP M1 Max with 10 cores (two efficiency, 8 performance). I benchmarked these two systems with the 8/4/1 image settings. Both systems are fastest when processing 4 images. I have a set of 43 RAW images I use for testing - a mixture of Sony A7RM3 and M4, some Olympus E-M5III, and various (older) Canon Digital Rebel images. Here are the timings:

iMac Pro
8 5 minutes, 18 seconds
4 5 minutes, 2 seconds
1 6 minutes, 41 seconds

MBP M1 Max
8 4 minutes, 30 seconds
4 4 minutes, 22 seconds
1 5 minutes, 57 seconds

On both systems, even with 8 images selected, DxO PL5 doesn’t come close to loading these systems anywhere close to 100% - so I doubt this adjustment has any significant effect to the performance of other apps running in parallel - macOS is going to handle the multi-processing. (At least for my hardware - maybe it has more effect for lower end hardware.)

Since I usually run large batches of raw images, I start the process and then leave to have a coffee - I usually don’t have anything else running. I thought perhaps running more images in parallel would help performance.

The other odd thing I have noticed is that this processing is single-threaded somewhere - if you watch the export window as the processing works - with preferences set to (say) 8 images in parallel - images processing and output occurs one image at a time. I had expected that the images would output in batches of 8 images at a time. That doesn’t happen.

I am not at all unhappy with this process, and the speed of processing - I just want to know more about what it actually happening (under the covers!).

David

2

Are you sure you’re checking the GPU usage too?

3

Yes. I am using iStat Menus and I can see both CPU and GPU use. It does use a lot of memory - each thread uses about 2GB of memory.

4

Not familiar with that. I’m on windows and I know the default GPU usage doesn’t show compute loads. It only shows 3D loads. So it doesn’t show up by default. If you select the compute load the GPU is heavily utilized.

That’s windows. Mac may be different.

5

Either I don’t understand or there’s an inconsistency here.

Export is done by several xpccor processes, not by PhotoLab process itself. Number of parallel exports will decide how many of these xpccor processes get created. More processes will lead to increased RAM usage but will be better to hide disk latency. Nowadays with SSDs between 2 and 4 parallel exports is enough. If you’re low on RAM (let’s say 8GB) you might want 1-2 exports at a time.

Regarding PL showing one image processed at a time, even if your settings are 8 parallel exports, all 8 processed images won’t be ready exactly at the same time. Maybe that’s what you’ve noticed.

One last point to note, if you have DeepPRIME enabled in your exports, this algorithm cannot be parallelized, even among several processes, so it might be the bottleneck in your exports.

6

… because of (exclusively) using the GPU ??

7

Yes, I am using Deep Prime (always). I haven’t tried the export without Deep Prime.

CPU usage is highly variable - it will jump to about 70-80% load for a few seconds, then drop down to around 10%. On the M1 Macs it is hard to separate GPU load since the system uses unified memory (no separate GPU RAW) but on my Intel iMac Pro, I can see GPU usage. I could push preferences up to 12 images (the maximum allowed), but given what I see now, I doubt if that would improve performance. Both my systems have 32GB of RAM so the heavy memory usage is not a problem (memory use is higher on the MBP M1 Max due to the unified memory usage).

8

Because one instance of DeepPRIME already fully uses your GPU or Neural Engine. Adding more instances of DeepPRIME in parallel only brings instabilities. Also we do not support using multiple devices (for instance GPU + ANE) although it is theoretically possible.

If the CPU is waiting for your GPU or ANE to finish DeepPRIME processing, then this is expected.

M1 Mac should use ANE by default, not your GPU. And I don’t know if you can monitor ANE usage. Note that memory usage is distinct from the use of the compute units from CPU/GPU/ANE. So deducing which one is used only based on memory usage is not so reliable.

2 Likes
9

iStat Menus has a CPU and GPU monitor (but I am not sure how accurate the GPU measurement is on the M1 SOC). It really doesn’t show much GPU activity - so, as you say, the Apple Neural Engine is being used instead - though I don’t know of any way to monitor that.

It is interesting that the MBP M1 Max is faster than my iMac Pro - DxO has an opportunity to further improve performance by avoiding Rosetta.

Thanks for the information - this was mostly curiosity - I am happy with performance now (on both systems). Since Pure Raw is supposed to be faster now on the M1 Macs I wonder if that improvement will make its way to PL5 soon.

10

PL 5 already contains (since its release) the speed improvements that were brought to PR 2.

11

Indeed. Processing 60 images with DeppPRIME on my MBA M1 has sped up quite a bit:

  • DPL4: 15 min. 5s
  • DPL5: 3 min. 36 s
12

I just ran the export again on the iMac Pro with the noise reduction set to High Quality (not Deep Prime). In that case, it does pin the CPU at close to 100%. But, with Deep Prime turned on, I see a much lower CPU load - the work is being done by the ANE (which I can’t track).

13

Hi Lucas,
at some point it could be interesting, or to have the most powerful and fastest « export » or to be able to « export » and « live view » changes in the editor at the same time (for example DeepPrime in nearly real time for the displayed photo in the editor).
I am sure as the technology evolves we will be able to have this confort one day :smiley:

14

@Lucas Thanks very much for your explanations. I’m new to DxO and it is very encouraging to see DxO staff contributing!

15

Getting a little of topic, I’m actually surprised at how well the iMac Pro stands up to the latest M1 Max chip? You can buy an iMac Pro for a similar price to a Mac Studio these days and have a free 27" retina display thrown in? How is the speed difference between the two for other cpu / gpu heavy tasks? Am I mad for even considering an iMac Pro in 2022?

I think I’ll still go the Studio route for future proofing, more updated connectivity etc. but an interesting quandary.

16

The iMac Pro is the best Mac I have ever owned. I bought an Apple “refurbished” model and saved about $500 - they are probably much less expensive now. The thing about this system is that its performance is almost effortless - the fans hardly ever spin up to be noticeable. (The exception was when I was running Win 10 (MS FS2020) in a bootcamp partition.). The monitor is, of course, beautiful - essentially the same as the new Apple display. It even has a better camera than the other iMac 27 models. My system is three years old now and Apple Care has run out - so I will probably move to the new Studio system and display, but I am in no hurry. Apple support will likely continue for three years, but you don’t know for sure. It you like the combined cpu and display, I think it is still a good buy.

(Versus the MBP16 M1 Max, for normal processing, I don’t really notice a difference - but a lot of software hasn’t been converted to native M1 code yet - as that progresses, the M1 systems will get faster. And, the MBP16 sits in my lap, and doesn’t get warm at all - my last MBP16 (Intel) got too hot to hold when doing work.)

1 Like
17

You’re not making this decision any easier.

Mint pre-owned, $2,299 from a reputable dealer like OWC with 90 day warranty. If I went Mac Studio, what I don’t have a good handle on is how much I would need to spend on a 27" 4k monitor to be similar quality or better than the iMac display (the Apple Studio display is overpriced imho), one with wider color gamut maybe.

Sorry everybody for turning this into a gear thread :roll_eyes:

18

I took a look at the Apple Refurbished website. You can get a basic iMac Pro model (like mine) for $3250. That includes the one year Apple guarantee (and you can buy Apple Care if you want three year coverage). Models with 2 TB SSDs, and the more capable Xenon CPUs are available for more. One thing to keep in mind is that these systems are set up for multiprocessing - the high-end 27" iMacs were actually faster for smaller loads. But I know when you load those systems up, you hear the fans (the iMac Pro has a much better cooling system). But if you are going to batch process lots of images (using Deep Prime) - and the images are 61mPix images (Sony A7RM4) like mine are, then the iMac Pro is going to perform better (and be a lot quieter). If you buy the (entry-level) Studio computer and you want a monitor remember that the iMac Pro has a 5K monitor (like the new Studio monitor). I don’t know about you, but even with my old eyes, I can tell the difference.

I did some quick calculations and I can get a (base) Studio and the new Studio monitor for about the same price as I paid for the iMac Pro three years ago. (I will likely bump the SSD to 2 TB and add memory - which will increase the cost.). It is pretty clear that the Ultra doesn’t add anything for image processing - it seems to be designed for video processing. So I am thinking the new Studio and the Studio Monitor is actually a good deal - for Apple, at least. I am going to wait until availability spreads to other dealers because I can get a discount that way. (B&H Camera, for example, will discount the sales tax - which is 8.3% here in Las Vegas.)

1 Like