Bug in the "Remove" of virtual copies

I don’t understand. With DOP, my first move with a new set of pics was to select them all and create VCs. I kept the “1” unchanged and made editions to the other one(s). I didn’t change this practice to this day. When I want a fresh VC to try new editions I created from the “1”. Very simple and efficient. The new feature is to me useless.
Nick

At the moment, I cannot see the problem. Your original copy is now marked as master instead of one and any virtual copies are numbered one, two and so on. If for argument’s sake you like virtual copy number three. You can make a virtual copy of number three and that will add a new copy to the end of the list. Deleting any of the virtual copies will not delete the master but if you delete the master, it will delete all the virtual copies.

As for me. I prefer the new way of doing it. It is much simpler and logical. This way. For me the master is an untouched copy.

2 Likes

@saint-112 I am writing a longer response to some of the issues but what I would say is whether it is possible to use [1] & [2] together with the [M]aster (@Prem effectively beat me to it), simply because getting any sort of change out of DxO is likely to be … (but that won’t stop me trying and defending the reasons for trying)!

That doesn’t change the original issues of DxPL(MAC) taking old DOPs and playing games with the data on discovering the DOPs.

The overheads of using the [M]aster & at least 2 VCs are slightly bigger DOPs (3 x 1 instead of 2 x 1 entry size) and additional database entries until all the editing is done and the culling begins!

So your workflow would become:

  1. Navigate to a new directory
  2. Select all images and create a VC, i.e. VC[1] so nothing really changed there then!
  3. Sort by Virtual Copy number, [M] is classed as [0] by DxPL in that process, it seems!?
  4. Allocate a ‘5 - No correction’ preset to all M to provide a true original or change your starting preset in preferences or …
  5. Apply an appropriate starting point to all [1]'s (if required) (step 3 & 4 could be reversed)
  6. Remove the sort criteria
  7. Use your original strategy with VCs [1] and [2] with no danger of losing the image until you explicitly delete [M], i.e. when an [M] stands alone then it is a candidate for deleting!

Please note that my “title” should be @BHAYT not some truncated version of that!?

Could you post an image and its DOP file that you know exhibits this problem?

I would suggest you copy the contents of a folder into a test folder and proceed as you would delete VCs on the test folder.

Then send the originals of one that fails.

Apart from the problems that look like they exist the current VC system is a night mare if you use it on a large number if images having to copy and paste each one back to the master (after remembering to close down soft proofing)then deleting the non masters. So very much easier before just deleting the unwanted imiges leaving just the one/master. Yes we were told this was requested but no one appeared top know by who and there are requests long ignored with large numbers requesting them.

Cast your vote here:

1 Like

So, Keith’s suggestion might be worthwhile to be raised as a feature request …

(Rather than unlikely expectation that DxO might reverse the current behaviour)

John M

Edit: Just noticed that Wolfgang raised this before I did - in which case I endorse it.

If you consider spending some time writing a post it would be useful to read carefully what the issue actually is to make sure your response will be relevant.
Nick

I did a couple of tests and it didn’t happen. I’ll send you what you ask when it happens again.
Nick

@Prem I take some exception to what you write, but only because it is less than completely accurate!

The [M]aster is only untouched if the user so chooses, whatever the [M]aster was intended to represent it is still capable of being edited . Being the [M]aster doe not endow the copy with any different “powers” other than controlling the complete deletion of the image, all edits, its own and those for any associated VCs, AND the image on disk.

Personally I didn’t use VCs at all, I was effectively using the only “copy”, i.e. the [M]aster but not so designated when there are no Virtual Copies.

It is certainly possible to always create a VC and only edit that copy and to treat the [M]aster as “inviolate”, i.e. don’t touch it and it effectively holds no data of “importance” except the “key” to preserving the image data on disk or not!

But that is “simply” a matter of choice and I outlined that in my post that I “published” just after your post!

The reason for this topic is because @saint-112 “tangled” with the revised DxPL handling of Virtual Copies and, potentially a bug in DxPL6 when handling DOPs from the DxO OpticsPro era.

I admit that I originally thought it was a “hoax” and tried to reply as politely as I could while doubting @saint-112’2 “sanity” and/or my own. @Platypus helped to set the record straight by describing that DxO changed the product, after consulting with the users, and while you may prefer the [M]aster and VC processing rules they are “arbitrary” and have, at times, caused more trouble than they are worth.

  1. The “only” positive that I can see was mentioned in a post above, namely the ease with which the image and all the edits can be eradicated with a single ‘Remove’ of the [M]aster!

But that was always possible by selecting all the Virtual Copies and deleting them in a single operation. The last one of which would then control and perpetuate the deletion of the image itself.

So nothing gained there then, but what was “lost” in the change. I went back to OpticsPro 9 and tested and installed DxPL 2 and tested and these are my “findings”!

  1. The “old” way was in use at least from DxO 9 through DxPL 2 i.e. 5 releases (DxO 9, 10, 11, DxPL 1, 2) and the change has been in use for releases DxPL 3 to DxPL 6, i.e. 4 releases.

  2. The freedom offered by the original code allowed any VC to be deleted, including the first, and effectively allowed what was to become the [M]aster to be deleted and replaced, albeit only by the next VC in line!

  3. One of the main issues that arose during DxPL 4 and DxPL 5 forum posts was the “Unwanted Virtual Copy” issue, not an inevitable issue as my recent tests and posts have indicated’ but experienced by many and a nightmare to correct. But it is “simply” a consequence of the coding surrounding the [M]aster architecture!

With DxO 9 onwards (possibly earlier) it was possible to delete any VC, as I described above, so in the event of 'Unwanted VCs" there would be two VCs whenever a Uuid mismatch occurred between the database and a DOP.

But a simple sort on ‘Virtual Copy number’ would allow all [1] VCs to be identified. This is an example from DxO 9 of two images containing 1 VC and one image consisting of only the original master

Sorting on ‘Virtual Copy number’ yields the following

Lone masters are sorted to the front, followed by VC[1], followed by VC[2] if there were no VCs before a directory was rediscovered then removing the VC[1] entries would leave the potentially newer edits as the only edit.

This is certainly not a fool proof process but with some care it is a lot better than the newer [M]aster architecture.

As for the workflow you propose @Prem then @saint-112 had worked out his version with the previous scheme without the rigid [M]aster architecture.

In truth, I can just about live with either but what I really detest is the confusion and “danger” created by mixing image file management with edit management!

This “danger” exists with both version when either the deletion of the last VC or the [M]aster results in the deletion of the original image, the two functions should never have been joined in the first place, there should always have been two separate operations

  1. One operation to remove the edits, all edits if desired and

  2. Another separate operation to delete the Image, which would have also have removed all edits, the ‘Remove’ operation as it currently is in the product and has been through both models of VC operation, using a ‘Del’ command pre DxPL 3 and a ‘Shift Del’ command thereafter!

Splitting it this way should prevent any unexpected deletion of the image “all” you would lose is the edit that took hours to perfect and neglected to turn into a preset or assign to another image or …

1 Like

Yes Bryan. I am fully aware that the master can be written to. That is why I wrote “for me…”. In my early days of digital photography I used to use photoshop essentials and soon found out if I wish to edit an image that had already been edited. It was very hard to get back to where I wanted. That is why I believe that the original image (the camera image) should never be written to regardless.

Like you I started with OP9 and never started to use virtual copies until PL4. As far as my images are concerned prior to PL4 they should not be written to if DxO is correct in stating they are not touched.

As far as my posting is concerned, I was replying to @saint-112’s posting immediately above mine. This he obviously didn’t realise and came back with an angry comment thinking I hadn’t studied the whole posting which I had and tried to repeat his problems but could not do so in Windows 11.

@Prem Thank you for the update and @saint-112’s scheme also achieves the same. The difference with the revised scheme that DxO brought in for whatever reason is that it actually creates more work to maintain the “protected” copy plus it must never ever be deleted unless you want to lose the image data.

With the old scheme that would happen only when there was one copy left a rather obvious situation for users surely! With the [M]aster scheme you could have 20 VCs and pick the [M]aster and the message makes it clear that you are going to lose the image but …

Personally I don’t like either scheme!

The removal of edits should be independent of the removal of data.

When there is a “shorthand” in operation it is too risky, someone in DxO thought it was easier to do everything with a single operation and possibly more convenient for the user but I don’t think so, does my opinion count, do I have any right to complain about such coding, well if I don’t after 36 years helping customers build and support 24/7 mainframe database systems then I guess I never will!

That’s the way, I prefer to work too (and labeling with M)
– and for even more consistency maybe to start the first VC with number 2 …

But what I really would like to be able too is to rename the VCs (= add a something to the name), so to signal by the naming, what was done with the VC. – Isn’t that possible in the MAC version?

→ ‘Outside of DxO’s world’ I’ve been doing so all the time and it’s simple to keep track by the naming convention.


And yes, I’ve been experimenting a bit with the order from PL2 vs PL6.

Carrying over a file with two VCs shown as “1 + 2 + 3” turns them into “M + 1 + 2” (also in Compare with …), while keeping the order. The exported jpg-files of course keep their filenames, they got in PL2.

What is not kept is the relation from the ‘source file’ to it’s output – btw the same thing when copying within PL6 from one folder to another.


To delete a complete file, I don’t mind to mark up (1 + 2 + 3) or just the master (M). But I’m not sure, what happens with projects. Up til now I don’t use them like I did in old LR 5.7, where I could sort the file’s order and such …

That’s what I said in my #38 post:

  • When you select the master only and choose Rename Image… you get a dialog box where you can type the new name. In the film strip only the master’s name is changed. The file itself has also the new name as well as the sidecar file.
  • When you select a VC only and choose Rename Image… you get the same dialog box. In the film strip only the VC’s name is changed. The file name is unchanged.
  • When you select the master and a VC and choose Rename Image… you get a different dialog box. In the film strip both images’ name arre changed. The file itself also has the new name as well as the sidecar file.

So you need to shift to a Mac or wait until the feature is implemented in the Win version. :wink:
Nick

On Mac: Select an image, hit command + Enter and change the file or copy name.

@Wolfgang Every copy, including the [M]aster gets an entry in the ‘Items’ structure in the DxPL database and they ALL point to a single entry in the ‘Sources’ structure, which has one entry for every image that has been (“imported”) “discovered”.

So if you have never made a VC then you will have one entry in ‘Items’ pointing to one entry in ‘Sources’.

With 6 VCs [1] … [6] created in PL6 you will have 7 ‘Items’ pointing to ‘Sources’ entry, i.e. [M] + [1] … [6] pointing to arguably the real “Master”, i.e. the ‘Sources’ entry!

NO copy numbers exist in the database or the DOP, it is purely “positional” in both cases. In the database it is done via the “position” of the key entry ('SourceId) in idx_Items_Sourceid, an index on ‘Items’, nothing more and nothing less, or so I believe.

idx_Items_Sourceid is an index on the ‘Items’ Table where each entry in ‘Items’ uses the ‘SourceId’ as a pointer to access the data of the entry in ‘Sources’

So if we import (discover) the first ever image it will have an entry in ‘Sources’ with (Source)‘Id’ =1 and an ‘Items’ entry with an (Item)‘Id’ = 1, both 1 because they are the first entries in the respective Tables, i.e. in ‘Sources’ and ‘Items’.

But as time goes by new VCs might be created, some at the same time, others after a period has elapsed and we might have

  (Item)Id  'SourceId'      "Copy"
     1          1            [M]
     2          1            [1]
   100          1            [2]
   101          1            [3]
   250          1            [4]
   350          1            [5]
   400          1            [6]

Where the “copy” number is a “notional” field and is determined by the position of the ‘Items’ entry in the index, which, with duplicates of the same ‘SourceId’, will be governed by the order in which the entry was added to idx_Items_SourceId with new entries being effectively added to the index after the existing entries for a given ‘SourceId’!

When a “copy” is deleted then it vanishes from ‘Items’ and its entry in ‘inx_Items_SourceId’ will also be removed and all copies (‘Items’) will automatically “shuffle up” in that index, i.e. the item is removed from the position in the index as if it never existed!

              Before                          After deletion of VC[3]
  (Item)Id  'SourceId'      "Copy"        (Item)Id  'SourceId'      "Copy"
     1          1            [M]             1          1            [M]
     2          1            [1]             2          1            [1]
   100          1            [2]           100          1            [2]
   101          1            [3]                                   VC[3] Deleted
   250          1            [4]           250          1            [3]
   350          1            [5]           350          1            [4]
   400          1            [6]           400          1            [5]

@Wolfgang I hope the above made it clear that as far as the storage of data is concerned actual copy numbers are not involved and the “rule” to delete the ‘Sources’ entry AND the image is a coding decision made by DxO!

With respect to projects, on Win 10 they are stored in two structures ‘Projects’ which stores an entry containing the name assigned by the user and a (Project)‘Id’ and another structure ‘ProjectsItems’ that has one entry that joins an ‘Items’ entry to a ‘Projects’ entry using the ‘ItemId’ coupled with the '‘ProjectId’ and indexed by two indexes to provide access from ‘Items’ and from ‘Projects’ to the entry. i.e.
image

In another post someone suggested that deleting items from a project had resulted in loss of the copy entries but according to my tests on Win 10 all that happens is that the ‘ProjectsItems’ Table entry is deleted when you remove an item from a project and that is all!

When you delete a copy from ‘Items’ then the entry in ‘ProjectsItems’ is also removed (verified by my testing) automatically.

Please remember that 'Projects and ‘ProjectsItems’ exist only in the database! No data is held in the DOP that can be used to recreate the project. If you lose the database or deliberately “destroy” it, as I do for testing, then you will lose the Projects data completely.

Hence, regular backups are imperative, as with every system that relies on a database, hence, the prompts from Lightroom, Capture One etc. about “Do you wish to make a backup” etc. as you try to exit!

When deleting a virtual copy, they change position,
e.g. deleting VC1 the next VC2 ‘moves forward’ and is shown as VC1 …

@Wolfgang It appears that way!

But all that actually happens is that ‘Items’ entry for VC[1] is deleted and the inx_Items_SourceId entry that corresponds to the now deleted VC[1] entry is removed from the index.

                                                         inx_Items_SourceId entries
     (Item)Id  'SourceId'      "Copy"    (Item)Id  'SourceId'      "Copy"
        1          1            [M]          1          1            [M]
        2          1            [1]        Deleted
      100          1            [2]        100          1            [1]
      101          1            [3]        101          1            [2]
      250          1            [4]        250          1            [3]
      350          1            [5]        350          1            [4]
      400          1            [6]        400          1            [5]

and PL6 refreshes the screen after the VC[1] ‘Remove’ and simply displays what is finds via the index allocating numbers from [1] upwards (+[M] of course), i.e. what was displayed as VC[2] will now be displayed as VC[1], and PL6 amends the DOP accordingly!

So rather than changing anything specifically, PL6 simply deletes the entry as requested and then refreshes the display using what is left! The only thing that actually changes position are the key entries in inx_Items_SourceId to close the gap after the 1 entry (in this case) has been removed!

In fact the display logic doesn’t really need to be “aware” of the nature the change, other than the fact that one has occurred, which triggers it to refresh the screen with whatever it finds in the database, if anything!

PS:- If you went for a “mega” deletion exercise you could wind up with this

                                                         inx_Items_SourceId entries
     (Item)Id  'SourceId'      "Copy"    (Item)Id  'SourceId'      "Copy"
        1          1            [M]          1          1            [M]
        2          1            [1]        Deleted
      100          1            [2]        100          1            [1]
      101          1            [3]        Deleted
      250          1            [4]        250          1            [2]
      350          1            [5]        Deleted
      400          1            [6]        400          1            [3]

Hi Bryan,

I’m happy with what I find out & understand … to make the most of it
– as an interested user, not as programmer. :slight_smile:

@Wolfgang How sad I was hoping to turn you into a database designer :wink:!?

In truth the database product I worked with for 36 years, starting just one year after its release (Burroughs/Unisys DMS II) was not an SQL product, so SQL and SQLite are new to me, but the principles are not :nerd_face:!