While having Optimize Storage enabled do some edits of any raw photo in folders shared to iCloud like Desktop or Documents.
Trigger the Optimize Storage feature (e. g. by reaching the capacity of the disk) so the big files are offloaded. Icon of a cloud next to the raw file indicates that the file was offloaded to the iCloud.
Open the folder with the raw files in PhotoLab
Unlike other programs and finder, PhotoLab does not see those raw files offloaded to iCloud and deletes all their edits (including .dop sidecar files that are not offloaded to the iCloud, because they are small)
The edits are lost even after re-downloading the files (by clicking the icon of the cloud)
Expected behavior:
Must have: PhotoLab does not delete any edits or .dop without user’s confirmation
Nice to have: PhotoLab sees files offloaded to iCloud like MacOS does
Example how files automatically offloaded by MacOS to iCloud look – indicated by the Cloud icon like:
macOS never decides where files go, that is something that you decide
RAW files are never edited, PhotoLab creates DOP sidecar files alongside the RAW file
How do you mean you re-downloaded them? They are automatically synchronised to and from the iCloud folder. You only need to work on the file in the iCloud folder.
Why are you putting RAW image files on iCloud anyway? Do you like paying for the storage they take up, when you could have them on an external SSD which would allow PhotoLab to work much faster?
If you have only opened an image but not changed anything, there will be no sidecar.
Your Mac didn’t decide anything. Can you show a screenshot of your iCloud configuration in Preferences?
I just copied a RAW file into a folder on my iCloud, opened it in PL6, edited it, closed PL6 and the sidecar file was next to the file in the iCloud folder. It reopened in PL6 without any problem.
MacOS automatically decides if to upload files from folders like Desktop to iCloud to save disk-space. It’s a built-in function called “Store in iCloud”
It decides automatically, without any user intervention when and what files to offload.
Such files have an icon of cloud next to them. This is how it looks:
An edited raw file - in my case a .NEF - is opened and edited by PL6 a sidecar .dop is created.
If PL6 is shutdown, the .NEF removed from that folder, leaving only the .dop and PL6 is launched once again.
PL6 will upon launch discover the that no :NEF exists and then directly delete the .dop side car.
The .dop is not placed in the trash - it’s deleted without confirmation nor trace.
MacOS offloaded files to iCloud as described above.
I opened Photolab to find out it does not see such files. And as @Required discovered, Photolab probably removed the .dop files.
I re-downloaded the files, but edits are gone.
Forever? No caches? No traces?
This is a serious bug. Why (the hell) is Photolab removing .dop files at any time? This is a timebomb in so many situations other then this when moving files around.
This is not a PhotoLab bug, it is down to your not having configured iCloud, including disk permissions, incorrectly. i will investigate further tomorrow.
But I will say that you should not be using the Desktop for storing more than a minimal amount of data, otherwise it slows the system down. And you definitely not be trying to run PhotoLab on files stored in the iCloud - the speed penalty and network load will cause all sorts of problems
It’s default behavior of MacOS. I did not configure it and I did not change any permissions. I only saved raws to Desktop.
Storing files on Desktop does not slow down anything. It’s the same partition of the disk, so there is not even a technical reason for that. It’s just shadowed in the iCloud.
I use the Desktop to sync between devices and it works perfectly and fast.
No apps other than Photolab have any problems with that and Photolab does not warn not to do it either.
If Photolab erases files and forgets edit by its own intention under the default MacOS configuration, it’s a severe bug.
The question remains if there is any way to restore the edits?
Edits are stored in the database and remain there indefinitely or until the database is deleted - under normal circumstances.
When the file is removed from the desktop while DPL is running, DPL removes all entries that concern the deleted file.
When the file is removed from the desktop while DPL is NOT running, DPL does not know anything about it and then, the data remains in the database…but is disassociated with the removed file.
If .dop fies are present and moved to iCloud too, downloading the files AND the .dop files should bring back the edits.
Can’t verify this right now, I’m not near my computer.
The .dop files were not moved to the iCloud, because MacOS moves only big files there (to save space). Photolab removed them when it did not see the raw.
By what you wrote it rather seems that if PhotoLab deleted the .dop files, it probably destroyed the database records as well, right?
So the only hope would be if they are cached somewhere(?)
Hope will not help, changing your way of working will.
You could disable automatic file handling for the Desktop.
You could choose a different folder instead.
I always disable automatic drive space savings and don’t even backup items on the Desktop with time machine. This has saved me lots of trouble and drive space on the TM volume.
Unfortunately, you did, more than likely unwittingly. There is a screen that shows during setup or upgrade that asks you to check a box to enable this, but it is checked by default, so some folks accept it as the “normal” behaviour.
In fact, it is a marketing ruse by Apple to get you to fill up you free iCloud allowance and then have to start paying for more space.
If you don’t see files that are supposedly on your desktop from within PhotoLab, it will be because they have not been downloaded from the iCloud, as shown by your screenshot with the little cloud icon showing.
As @platypus indicates and I have mentioned, simply don’t use iCloud Desktop and Documents for editing photos - it will only fill up your iCloud allowance and cost you money. It is not just PhotoLab that gets confused, here is a good video that explains things, using Adobe Bridge as an example…
Sadly, it happened within a day, so there was no TM backup yet:/
I found how to disable the feature, but:
a) I’m searching if there is a way to recover the edits that were already deleted by the Photolab
b) it can happen to anybody else in the future. This bug is not documented anywhere AFAIK.
Great. However you name it, it would be great to solve it so it won’t happen to anybody else at much larger scale than me.
The simplest hotfix would be just not to delete the .dop files. Or ask before deletion if there is any mysterious reason for that.
Lightroom does not delete .xmp files either.
Did you watch the video. Did you download the files from the iCloud before editing them? If not, PhotoLab will not see them or behave properly. This is not the first time this has been brought up and is due to Apple convincing users to use this option to gain more income from their iCloud service.
Unless any and all files that you want to work on are downloaded from iCloud (no cloud icon), this is never going to work as the file has to be on a physical disk to be valid for many apps, not just PhotoLab.
This only came about because you used your Desktop for storing images instead of the more normal Pictures folder. As I mentioned, using the Desktop as a working folder does slow down the computer as it has to keep on updating visually if anything changes - something that is not true for ordinary folders that are not normally visible whilst working in an app.
Since this has been mentioned before and no action has been taken, I don’t see DxO doing anything about this soon. What you are doing is unusual and requires special handling, no matter what the app.
Unlike some other apps, PhotoLab is not built to run from iCloud drives, as is witnessed by your own finding the files do not show up in the library.
Folders marked as iCloud in finder don’t truly exist on your computer and are not always automatically synced to/from the cloud but can also be subject to removal if macOS deems it necessary.
Don’t forget the RAW file is never edited, so iCloud doesn’t know it has changed, which could trigger it being removed from the physical disk, which, in turn, will trigger PhotoLab to think that it has disappeared, thus removing the sidecar file.
The Feature Request could read:
Handle database integrity that can be compromised by Apple-specific drive optimisation of iCloud functionality to prevent issues like this: (add your original post text here)
We dont need to provide a solution, we provide a request and its reason.
No, but I was/am a programmer writing software for macOS, iOS and iPadOS and know how this stuff works. Cloud storage has changed its behaviour over the years and requires very careful handling from within an app.
It was primarily invented for Apple’s own apps and tightly integrated into their “document” model. This means that everything is normally read and written from one integrated file bundle, registered with the system. PhotoLab can deal with various image file formats, none of which are registered with the Apple bundle handling mechanism. Added to which, the sidecar files are not directly connected to the image file, apart from via the PL database.
As a developer, I can see all sorts of difficulties in trying to integrate the ever moving target which is cloud programming, especially with an app that deals with things like sidecar files as well as a main file type.
You also state that you use the cloud desktop to sync with multiple devices. Using PhotoLab from multiple machines, without iCloud, can be pretty tricky on its own.
In short, do yourself a favour and change to storing your images in something like the Pictures folder that was designated for that purpose. If you need to access them from more than one machine, enable sharing.
