Bug in the "Remove" of virtual copies

Hi, Nick.

That’s explained above.

If you no longer want your "M"aster version, and want to replace it with one of your VCs – then;

  • Switch to the VC … and use image via the Image menu

  • Switch to the "M"aster version … and use image

Now, your "M"aster version is identical to the VC version - - So, the VC version can be safely deleted.

Hope that helps - JohnM

1 Like

As others already pointed out: It’s not a copy (and the sentence of @Pathal about “delete the M copy…” is simply wrong wording, sorry but that’s only increasing the confusion): the “virtual copies” are copies from only your edits of the master RAW file. You’re not copying a file physically. Even the DOP file belonging to the RAW Master file and containing your edit actions won’t be copied as all your edits are stored in one single DOP (as long as you don’t copy physically the master RAW.

I also stumbled into something like this when I thought I could move a VC to another project, maybe for b/w versions or keeping the master in a main project (like holidays in Italy) and having side-projects (like visit of Mount Etna). DxO appears to think Master and VC always need to stick together.

That’s a bit hard to imagine. PL usually doesn’t do things like that. Can you produce it with physical copies in another folder (so the legacy files don’t get hurt)? And has someone already used the word “backup”?

@JoJu @Joanna etc. Sorry another big post!!

@saint-112 As has been stated by a number of users in this post the Virtual Copy (VC) is just that it is “Virtual” and exists only in the PhotoLab “space”. It has no physical presence until a export from any given image or image copy, [M]aster or VC, is created!

So let me explain what I believe happens and why I find some of the situations you describe difficult to understand. Please remember that these tests were conducted on a Windows 10 machine using PL6.0.1 and various database copies were “injured”, even “destroyed”, doing these tests, not to mention the impact on my “sanity”!

The final test scenario consists of 1 image in a test directory and the same image in a directory of photos I took yesterday, while visiting Leonardslee Gardens with my wife, hoping to see some late Autumn (Fall) colour.

This one image has 4 VCs “in tow” in both directories and they look like this in the final database snapshots but only the IPTC data was present in earlier tests!

Where ‘Rating’ has been set by VC number (0 for [M]aster), ‘Colour Label’ set by order and the DeepPrime and DeepPrime XD used to identify two basic edits for ease of checking the thumbnails. In addition the IPTC contact field has been used to hold “Master”, "VC(1), "VC(2), “VC(3)” and “VC(4)” for even further identification (the “only” identification in earlier tests)!!

The details are actually held in the database as separate database records (table entries) but all the edits and metadata from each copy ([M]aster and 4 VCs]) are retained in a single DOP sidecar file, i.e. 5 database entries but one DOP containing 5 lots of edit data and metadata.

The database entries of interest are

  1. ‘Sources’ which holds a reference to the [M]aster only
  2. ‘Items’ which has an entry for each and every incarnation of an image, [M]aster and any associated VCs and
  3. ‘IPTC’.

For the “lone wolf” directory we have:-

I apologise for the size of some of these snapshots, the original tests were more orderly but I managed to “corrupt” the database and PL6 reverted to the default location and …

Some columns have been “hidden” to ensure that the critical fields are visible!

‘Sources’:-

With one entry per original image, essentially the [M]aster image.

‘Items’:-

Please note that all these highlighted entries have a single ‘SourceId’ (the pointer to the original ‘Source’ entry) and I believe that the order of entries in the table is significant! The first entry for the same image being the [M]aster, the next VC[1] etc… when a VC is deleted in PL6 then its corresponding ‘Items’ entry will be deleted and the numbering of VCs will change to maintain the order (or they should!)!

IPTC:-

The “highjacked” IPTC field used for additional identification in this case. Where ‘ItemId’ points to the associated entry in ‘Items’!

Why can’t we delete the [M]aster entry in DxPL:-

In truth I cannot answer that @Musashi & @DxO_Support-Team, the first entry in ‘Items’ is little or no different from the other VC entries unless there are SQL rules that maintain a relationship back to ‘Sources’!? This ability has been has been requested on many occasions’ in relation to the “Unwanted Virtual Copy” situation but …

While there is no particular reason that DxO could not change the code (and possibly have to make changes to the database definition) to allow you to delete the [M]aster and preserve the Virtual Copies that capability does not (currently) exist and with my naïve view it is a very simple coding exercise to delete any entry and leave the ‘Sources’ entry until the last ‘Items’ entry is deleted!

The storage of the [M]aster and VC data is as simple and complicated as that! I have gone to some lengths to identify the various copies because there does not appear to be any such identification in the database!

Using the commands on offer to make any copy “identical”:-

I can make the [M]aster look like one of the other VCs using ‘Copy correction settings’ and ‘Paste correction settings’ and ‘Copy metadata’ and ‘Paste metadata’ from e.g. [2] to [M].

Indeed I can use the same capabilities between any and all copies at any time, arguably one way of “parking” edits without creating presets and then I am free to delete [2] if I so wish but I am never free to delete[M] without losing all the copies AND the DOP!!

Effectively the deletion of the [M]aster deletes the entry in ‘Sources’ and would leave all other VC entries in ‘Items’ pointing at a “missing” entry in ‘Sources’!

Although Mac users can use the ‘Advanced History’ to “park” edits that can be retrieved on a subsequent run of DxPL(MAC) that preservation of the history is not available on DxPL(Win) so using VCs is one way of preserving edits at different stages of the development of an image, a horizontal “stack” rather than a vertical “stack”, but lacking the granularity of the ‘Advanced History’!

I have never seen a listing other than [M] [1] [2] [3] etc. or have I

Miss numbering of Virtual Copies:-

Please note that this was before my more “extreme” identification methods!

As part of the testing of this and another issue I created 4 Virtual Copies but my attempt to delete one of them was impossible because the ‘Remove’ command was “greyed out”. The difference with the[M]aster test image was that I had passed it from FastStone Image Viewer and it was a member of an ‘External Selection’ (possibly only a Win10 feature!?) as were the VCs and the ‘Removing’ any item appears to be prohibited!?

So I first executed a ‘Load original image folder’ which restored the ‘Remove’ command. Adding this image (& copies) to a project still left the ‘Remove’ command active when the project was selected (at this point I was working with the original image in the original directory)!

So I deleted VC[1] and got the following!

Creating “huge” numbers of "Unwanted Virtual Copies:-

The circumstances of this issue are purely of my own making but I am unclear as to exactly why is happens; yes I do understand Uuids clashing cause VC creation.

After deleting all VCs, with the directory still “in scope”, i.e. in view in PL6 and with only the [M]aster image remaining, I copied the DOP from a safe location back into the directory and wound up with the following

I have had triple the number of VCs created in previous tests but have never understood why @DxO_Support-Team. Given that I have deleted all the VCs before “recreating” the DOP I am unclear as to where the Uuid clash occurs!

I then progressively deleted VC[1] and wound up with

and a locked PL6!

I have repeated the test a number of times since and cannot repeat the miss numbering but the 12 VCs happens every time!

Snapshots are important:-

What I have tried to explain above is that I can move the edit “persona” between copies but that doesn’t change their order in the scheme of things (or so I believe)!

@saint-112 If you can recreate the problem then please snapshot the screen so that we can all see it for ourselves!

Possibly Windows only. I can create multiple VCs and delete VC(1) and all the others get renamed in sequence.

A while back I had a problem where the (M) was not in first place. Turns out it was due to having sorted the folder. It took several attempts to find which sort order to specify to restore things to “normal”

The fact that there’s a master at all is just clumsy UI in my opinion. All edits live in the dop file associated with an image file (or the db if you prefer), so there’s no need to distinguish one set of edits as a master and have its removal mean removal of the underlying image file. Just treat all the copies equivalently and equate removal of the last remaining copy with removal of the image file.

Then there’d also no need for copying settings when it’s the current M edit you want to discard. (Or working around that by only editing copies, etc, etc.) DxO could alleviate this need by letting us promote a copy to master, but I just don’t see any reason to distinguish a master at all.

I think it was even like this in an earlier version of PL, but virtual copies were buggy even then. Presumably the master is an implementation detail shining through.

2 Likes

@Joanna Typically Win10 automatically changes the copy numbers automatically and if you watch closely you can sometimes see the [2] change to [1] except on the few occasions where it has remained at one.

The reason for the size and depth of the test, raising “manhole” covers and monitoring the database etc. was to see what was going on and the truth is that there are no VC numbers that I can see in the database.

It is all done by “sleight of hand”, i.e. it is a display issue and is controlled by the position of data in the database and the associated location of data in the DOP! If there were VC numbers in the database then every time one VC was deleted the remaining entries would need to be “fixed”, instead it “only” requires an adjustment of the display only!

How different the code is between the DxPL(MAC) and DxPL(WIN) I do not know and it does not occur that frequently but frequently for me to have captured a snapshot on two occasions!

@asvensson I was not justifying the design but rather documenting it exactly as I saw it. I made the point that I consider the change to be “trivial” and even made the point that “the last copy leaving should turn off the light”, i.e. delete the ‘Sources’ entry!

@platypus and I have asked for better promotion facilities for some time, when you look at the database it becomes clear what work needs to be done, i.e. changing the order in the database and adjusting references accordingly!?

Looking back into the history of virtual copies in DPL, I found the following:

DPL versions 1 and 2

  • numbered all virtual copies and none of the VCs was tagged with “M”
  • any VC could be deleted, which brought up the following message
  • VCs were re-numbered when a VC was deleted
  • the last remaining “copy” lost the number badge and became the “master”

DPL3 introduced the revised VC behaviour with “master” and “copy” files"

While the old scheme was harder to grasp (which is the original?), it allowed to make any copy a master by deleting all other copies. I had my problems understanding that what we see are always “mirages” of our files, reflected by the respective recipes and therefore, none of the previews are real. They only become real when we export.

All things considered, I’d wish that DxO reversed to the old scheme, even though I, at that time, had promoted the current scheme…

1 Like

Hi Asvenson,
if you like it or not – that’s the way how PL works.
As soon you create a VC, the origin file is marked with an M (keeping the relative small dop-file with it).

With ‘independent’ copies, you would duplicate the origin file physically – don’t think you would like it.
Wolfgang

@platypus Thank you for your research and honest comments. It is also possible that not only was there a potential issue with users understanding the logic but were errors creeping into the actual code itself!?

Sadly I “only” have DxO 11, DxPL3, 4, 5 & 6 installed

While I would contend that it is not difficult to code the swapping of one copy with another and allowing one copy to adopt the persona of another, effectively always maintaing the first entry in ‘Items’ as the control on the entry in ‘Sources’

Here is a slightly cleaner set of structure images for the [M]aster and 4 VCs but excluding any metadata!


So after adding some keywords we would also have a keywords structure and I think that is where the problems started

2022-11-01_110543_'Keywords'
2022-11-01_110644_'ItemsKeywords'

and don’t forget ‘Projects’


2022-11-01_111054_'ProjectItems

and, that might be all but ‘ProjectItems’ might get big and complicated!

@Wolfgang But it isn’t you are at risk of confusing the Virtual world with the real world.

Arguably the database is a representation of both but it never marks the images as anything, the interpretations of [M] etc. are “just” put there by the DxPL display process and the DOP is just an output from the database that preserves the ordering so that it can be recreated when the DOP is used as input!

We don’t like it because the restrictions imposed by DxO are just that “unnecessary” restrictions and simply created by the coding rules DxO have chosen to adopt!

Sorry, why do you use PL, if YOU don’t like it?

No worries. I was actually responding to another comment and hadn’t yet read yours. :slight_smile:

The original way of using VC was much better. These posting are a result of a user struggling with the current and if as BHAY1 says “While there is no particular reason that DxO could not change the code (and possibly have to make changes to the database definition) to allow you to delete the [M]aster and preserve the Virtual Copies that capability does not (currently) exist” it should be easy for DxO to revert to the old better way. We were told, with little evidence produced the original change was done due to users not understanding how to use VC but clearly there are those who find the current one a problem and that it has defects so why not admit it was wrong to make the change and go back now soft proofing advises you to use a VC?

Of course. I was just saying that I don’t think it’s a good choice. :slight_smile:

There’s nothing that would require that. A virtual copy need be nothing more than a set of edits in the dop file (or db), associated with the underlying image file. No need at all to duplicate the underlying image file just because you haven’t identified one set of edits as Master.

If that’s why DxO has a Master copy then it is what I suspect, an implementation detail shining though.

1 Like

POTENTIAL WORKAROUND, try at your own risk!

If you like to make a VC the new master, you could edit the .dop file.
There are a few caveats though.

  • The order of the VCs in the sidecar is not as expected as we can see below with manually renamed VCs so that the name reflects the VC badge:
  • The VC’d file has to be removed from the DB by deleting it in DPL
  • Quit DPL, rename its database, reposition the trashed file and its sidecar
  • Edit the .dop file
  • Open DPL…and check if it has worked.

I’ve tested the procedure successfully with DPL 6.0.2 on my Mac…but I do not recommend the procedure to the faint of heart…

@Wolfgang I bought DxoO 11 because it had the best, most sensible export features of any program I had seen and the DxO 7 adjustments made a real difference to my JPGs with just a few tweaks once I had worked out what I wanted.

PhotoLabs 1 saw the demise of my favourite quick adjustment and the start of ‘Local Adjustments’ which in the early releases were mostly a “joke”. However, in spite of the demise of my favourite option I have stuck with it.

But what I see in the forum topics is “infuriating”, when DxO are justifiably seeking increased market share, which benefits us all, otherwise the product goes down with the ship, but steadfastly turns a deaf ear to issues that could be resolved quickly.

They would improve the usability of the product by avoiding issues in the first place or providing facilities to overcome obstacles when they occur.

I will continue to use DxPL but also continue to “complain”, but hopefully in the most professional way possible, in the hope (however slender) that someone is listening and adding suitable “fixes” to the schedule, preferably this side of …

@John7 Agreed

and when have you known me be “faint of heart”. I will test it on Windows later.

I have some way to go with Python and then I need to learn the Python to SQLite interface before I can write a script to do the necessary processes automatically. On my old DMS II database with COBOL it would be written already, but testing would take a bit longer!

But DxO could do it in hours @Wolfgang, if they had the will!

DxO could offer to make an incoming DOP the [M]aster replacement, or the [M]aster but keep the old [M]aster as a VC or preserve the current situation, whenever there is a Uuid clash between a discovered DOP and the database Uuid, within hours (and I am being generous) thereby avoiding some of the need for “promoting” VCs but the facility would still be useful.

1 Like

They really need to do something, the promotion of VC with soft proofing exposes all users to an un user friendly process needing copy and pasting. I just processed 60 images, DxO would have me create all 60 as VC and then copy and paste all changes where needed onto the master copy. Its just not going to happen and I can see why with the changes soft processing is based on it needed but as usual one big change hasn’t been though through to its implications in this case a poor version of VC.
At present it’s a mess as far as I can work out you can’t just copy from the virtual copy to the master as that then becomes soft proofing copy. You have to close down all the virtual copy’s them copy and paste between those edited and wanted and the each master copy then you can delete the now unneeded VC, it’s a mess!

1 Like

@platypus I haven’t got DxPL1 or DxPL 2loaded but I went back to DxO11 and indeed things were easier

The Dx11 ‘Items’ and ‘Sources’ structures:-

With the [M]aster just another copy

@platypus so much easier with no “false” coding rules in place!? I wonder if the coming of keywords (was that DxPL3) had anything to do with the change?

The master/copy implementation was introduced because many posters requested it. While DxO was programming the change, I came to realise that the “mirage” paradigm worked for me…but master/copy came anyway.