Win 10 DxPL 'Removes' the wrong files!

,

@DxO_Support-Team On Win 10 DxPL Deletes the wrong files after using a “Rename folder” command and re-using the original directory name!

This problem has been reported in (Unwanted Virtual Copies and Moving DxPL edited images (DOPs) between System - Revisited - #36 by BHAYT) but has been placed here to highlight the issue for other users.

Summary:- While testing synchronising image edits & metadata changes between two systems I started to run tests with embedded and sidecar metadata after some very successful tests just using DxPL to set the metadata!

As a result of my mistakes I deleted files and encountered problems that will be reported later, so I changed my tactics and renamed a directory using the ‘Rename folder’ command to “remove” the original folder and then copied the same images to the original folder, i.e. I copied a complete folder back in place and re-discovered it in DxPL.

When I reverted to my old strategy of deleting images within DxPL using the ‘Remove’ command I was surprised to discover the images were actually deleted from the (old) renamed directory!

I have tested this issue with Windows PL5.5.0, PL6.0 and PL6.0.1 and it is present in all 3 releases!

@DxO_Support-Team Bug Report - DxPL ‘Removes’ “wrong images”

After an appalling set of tests, all the problems were started by my own silly mistakes and I will write up the issues tomorrow (hopefully that will be today).

As described above, while conducting tests I had to keep clearing data because of my errors and the occassional DxPL “glitch” so I stopped deleting images and started renaming the directory but also deleted images as well and here is the error I encountered.

This is the procedure to reproduce the error and a single image will do

  1. My test directory contains 4 images, 2JPG and 2 RAW, 4 DOPs and 2 xmp sidecar files.

  2. I copy them to a test location and navigate to that in DxPL (AS(OFF)) with DOP load etc. set as default (ON)!.

  3. Use ‘Rename folder’ in DxPL to change the name of the directory to “{name}-Renamed”.

  4. Now copy the original set of images, DOPs and sidecar files to (as) the original directory so that there are two directories containing the same sets of files, “{name}-Renamed” and the newly reconstructed “{name}”

  5. Select the 4 images in the “{name}” directory and ‘Remove’ in DxPL

  6. DxPL deletes the images from “{name}-Renamed” but shows “{name}” as being empty but “{name}-Renamed” also shows as empty and is indeed empty!

  7. Only a Restart will show the true state of “{name}”, i.e. images intact but the “{name}-Renamed” images have all been deleted from the disk!!

  8. DxPL is getting its wires (database references) crossed or something like that!!!

@Musashi we need simple changes to DxPL implemented alongside the more heavyweight features, e.g. @platypus has been asking for DB, file and folder management options for almost 2 years, as have I. @Solivac asked for similar features in Folder create, rename, move, copy and delete inside PL.

How difficult is it to add an option to ‘Remove entries’ from the database but actually leave them on disk! Sadly the ‘Remove’ command name has already been used, I would have suggested that the current command should have been named as ‘Delete’ and what I am requesting should have been the ‘Remove’!

But whatever command name is chosen (‘Remove from database’) we need it and have done for some time, it is essentially the current ‘Remove’ but simply doesn’t remove the data physically from disk, how hard is that to implement, i.e. the current ‘Remove’ but stop short of the !?

In my case I was working between two systems and so System A’s copy was intact and the original data had been captured as “BaseLine” with copies on both system so I had three backup copies but in other circumstances irreplaceable data could have been lost!

The ‘Remove’:-

The original directory after the ‘Remove’ in PL6.0:-

The renamed directory:-

The Original directory after a restart:-

My concern was that this “bug” might impact the procedure I wrote for avoiding any problem with copying from laptop to desktop PC and back again by

  1. In DxPL change the name of the directory to be replaced to {name}-OLD
  2. Copy the replacement images either to {name}-NEW and then to {name} or just directly to {name}
  3. Discover the {name} directory in DxPL
  4. When happy then delete the {name}-OLD directory

but with this “bug” if the copy to the original directory i.e. to {name} needs to to be deleted, if executed via DxPL the contents of {name}-OLD would be deleted instead of the contents of {name}! This can be avoided by deleting outside DxPL!

Is this a regression back to a problem in PL3/4?

@Sparky2006 I don’t know, was that problem supposed to be fixed or has it been there all along and is still there now - well remembered!

Just tested on PL4 and it was certainly there in that release, i.e. present on PL4.3.5, PL5.5.0 and PL6.0.1 and PL3.3.0 and OpticsPro 11.4.2 @DxO_Support-Team!!

PS. By my reckoning it has been there since at least 2016 or thereabouts, just as well it isn’t a serious bug then!

There have been a lot of file management bugs. I got them to sort out dops not always being deleted when images were deleted and when saving to a new named folder PL would often creat a new folder to save to and the named folder you created. It sounds like the file handling modual was a mess and has only been part sorted rather than fully redone.

@John7 File management and/or database references.

It is an unusual error because I changed the name of the directory after a failed test, before effectively replacing it with an exact copy, rather than deleting directly the first time because of concern over putting the same data right over the deleted data!?

I then deleted the (replaced) contents when that test failed, but DxPL chose the wrong items to delete!

@DxO_Support-Team, @Sparky2006 and @John7 so I repeated the test using PL6.0.1 taking snapshots of the database structures, ‘Folders’, ‘Sources’ and ‘Items’ and the “reason” is “simple” but as for the cause that is down to DxO.

  1. Reloaded my empty database backup copy to clear everything out so that reviewing the database would be easy.
  2. Navigated to the directory of 4 images and reviewed the database containing the various elements of the Directories in the ‘Folders’ structure

& ‘Sources’ pointing to Id = 11

with 4 images in the ‘Items’ structure

  1. Renamed the structure and the ‘Folders’ structure showed the name change but still ‘Sources’ reference Id = 11 (as it should be)

  1. Added the originally named structure to the directory structure, came in as Id = 12

  2. Checked the ‘Sources’ and ‘Items’ but still only the original 4 records in the tables!?


    and ‘Sources’ still pointing to Id=11

  3. So where are the additional 4 ‘Items’ and ‘Sources’ entries that should be present and pointing at Id = 12 in the ‘Files’ structure? The fate of the original entries, the only entries is now sealed. Forcing an export of the DOP did not change anything.

  4. The delete of the 4 entries deleted from folder Id=11 and that was the end of all entries in the database


  1. I restarted PL6.0.1 and repeated the test but this time I made an edit to the “replacement” images, which were reflected in the renamed images and still only 4 entries so that is going to fail as well!

So something is stopping DxPL from making new entries for the images in the “replacement” directory and it uses the original entries including deleting them even though they belong to another directory by that point!

@DxO_Support-Team Please fix it to restore my confidence in the database coding of DxPL if nothing else!!