Yes, and if I remember well, someone from DxO team explained that metadata in .dop files were there to be used with Virtual copy…
So to be able to import metadata in DPL we must do it from xmp.
1 Like
Yes, I must admit (to be honest) that that’s the case for me … I have a short attention span !
At the same time, I do respect your good intentions …
John M
No need; that’s how I work all the time (with sidecar/.dop files only - having deleted the database before I invoke PhotoLab), and that’s how it works for me … Everytime … No problem (other than loss of metadata that you noted … and which I don’t care about).
John M
Why not just try/test the method we have suggested ?
There’s no danger in doing so, provided you keep your “from” system in place.
John M
1 Like
Thanks John. It will probably be a long time, if ever, that I get a clear, concise recommendation from DXO. So I have been thinking the last couple of days that I will start doing some experimenting.
I am very concerned about losing keywords though with your system. Some of these are many years old and are helpful to us. In fact that is actually more of a concern than losing edits, although I don’t want either to happen.
I will keep a copy of my entire library in a safe place and will start off working with a few copies. To be honest I am really bummed at DXO that any of this should be necessary. I’m also disappointed that they have not been participating in this and other related threads.
On the other hand this ‘Community’ is wonderful with the help of so many of you experts that share freely. Thanks! 
Rod
1 Like
That’s the way to go, I reckon - you’ll learn more from trying it yourself than reading here.
Yes, with my proposal (that you let PL build a new database, on your new PC, from info in the sidecar/.dop files) you will lose any keywords held in the database on your old PC - as they are not stored in the sidecar files. Similarly, metadata (ratings, GPS coordinates, IPTC fields, etc) will not be transferred.
John M
1 Like
As long as one is trying to preserve edits only like you do, your way works smoothly. For all those who also want to transport keywords, ratings and GPS coordinates, XMP sidecars are the way to go.
Those who have added keywords in PL4 or other apps will find the DxO road plastered with obstacles at the moment. Until PL transitions to be a stable, comprehensive metadata and asset manager, I’d recommend to consider a SPOD approach with something other than PL as the SPOD.
1 Like
IPTC metadata belongs in XMP where all the applications capable of reading and writing metadata can access and modify it.
To me, the current decision feels like wantingly shooting one’s own foot.
No way. I don’t much care if DxO keeps a backup of the metadata in a .dop (the space it uses in comparison to image data is negligible) but metadata should be read first from the .xmp if it’s present as the canonical source of data.
The bigger issue is the database, which should not really exist at all except in temporary format (created when opening a folder, deleted when moving to another folder). Every time a photographer opens up a folder or an image in PhotoLab, PhotoLab should be reading the data from the .xmp file or from the .dop (in case the file travelled away to a desktop and back to a laptop in the meantime).
1 Like
Yes, I agree that’s a correct assessment. However, I don’t like the idea of XMPs as well as sidecar files … too much clutter and crud. My preference is to keep things simple.
Q: As someone who uses the database (and with close control over when sidecars are generated, as I understand it), how do you mitigate against the potential (eventual likelihood !) of the database becoming corrupted? What do yo do in that case ?
John
A1: I use Lightroom Classic as my SPOD regarding all things metadata. And yes, I try to not simply trash DPL’s database on a regular basis because it is, imo, the key to DPL’s way to mature asset and metadata management - and don’t forget that managing assets is not about management per se, but a provision to find things again, be it after some time or by someone else.
A2: With all the care I take, the database gets off track occasionally and then, I either restore the DB from a backup or have DPL re-index the collection, which takes about 30 minutes for 25k images on my Mac.
A3: My focus is on managing my photos and I usually re-develop images when I need or want them. This is probably the opposite of what most people do. I (kind of) push DxO to better DPL’s asset/metadata management because I think that, apart from enabling one’s visions through processing, keeping track of things is as important too.
1 Like
This is to me the most important feature. I want to be able to find an image I worked on five years ago and say, “Oh yeah, I have all the corrections in my library. Somewhere.” And then be able to reproduce it. My entire workflow has reproducibility baked into it.
When PL revisions crash up against my workflow, it leads to some consternation. Stars are minor, but they are part of the deal, and when they disappear, I wonder what else is missing. I wonder how I need to change my workflow.
I don’t mind changes per se, especially as an evolutionary step towards a more full-fledged DAM – or integration with other applications – but the changes absolutely need to be communicated, and that did not happen here.
Most of all, I just want to make some damn pictures. 
1 Like
@BHAYT and @uncoy I have a wish that we may judge a text by what it is trying to say instead of how long or short it is or how many words it contains but I am also aware of that most (especially men) seems to avoid reading text longer than a Twitter message these days.
No wonder reading and writing books these days seems to be something mainly women do and as a spin off competent women all over the world to an ever greater extent is taking over a lot of the most important positions in society. To become a skilled expert in any field today you have to be able to read a lot and both read, speak and understand english too at least.
The solution to that problem will not be to lower the thresholds of the world. Men that fail to keep up with these demands are an ever increasing problem all over the world. This Easter the police cars are burning in my country in several places and so is the Koran. Some of these young men are upset despite they hardly have taken the time it takes reading that book and any other book for that matter and in the outback a lot of other men doesn’t give a shit as long as they can drive their snow scooter and fish and hunt. The only thing they seem to have in common is that they are men that failed in school and doesn’t like to read.
The best way to improve reading and writing is to read and write and communicate which in it’s turn will force us to read and write even longer texts better.
1 Like
This is a reply to @platypus’s test and @RexBlock concerns.
I used JPGs and the 'Auto synchronize option is OFF, AS(OFF) but I typically work with DOPs being automatically written and loaded.
Photo 1 - the original JPG before any fixes applied:-
Photo 2 - the photos with fixes and keyword added:-
Photo3 - the photos rediscovered in another directory:-
The photos and DOPs were copied to another directory and “re-discovered” by PL5. The fixes look to be intact but the keyword and ‘Rating’ are missing!!
The keyword and ‘Rating’ are missing because although it is still in the DOP, PL5 expects that data to be in the image.
Why is no metadata discovered?
- AS(OFF) was “selected”
- I did not perform a ‘Files’/‘Metadata’, ‘Write to image’ so the metadata is still in the database assigned to the original photo in its original location but has not been transcribed to the image!
If this had been a PL4 DOP, I still wouldn’t have got the keyword data because it wasn’t written to PL4 DOPs but I would have got the ‘Rating’!
However the data is in the PL5 DOP of the original photo
but after the photo was rediscovered no metadata is present because PL5 takes that from the image and I never forced it to be written to the image.
PL5 then updated the DOP of the re-discovered program in line with the image it has discovered and all the fields have now been nullified!
Hence, the vanishing ‘Rating’ that has been reported in a number of forum topics!
To keep the metadata you must cause it to be written to the image and if it is a RAW image then an xmp sidecar file will be created and this must be secured, i.e. the total “back-up” package for a RAW is xmp sidecar + DOP (for photo adjustments) + Image file, for all other files types it is the DOP + Image iff (if and only if) the metadata has been written from PL5 to the image file!
Edit:- I forgot the after photo sorry!!
@platypus and @RexBlock and @anyone who is interested I should have quit while I was ahead!
New bug detected (I believe) causing Virtual Copies to be created:-
I went back to the original directory and wrote the metadata back to the image
I had missed setting the data in one of the images and missed exporting that image which contained nothing new anyway.
I then copied the images and DOPs to another directory and all hell broke loose @sgospodarenko!! I also failed as a tester at that point because I had wondered if I should secure the original directory before exporting the metadata so I had a copy of the original data to refer back to but I didn’t do that!
The reason for the ‘Virtual Copies’ I discovered some rime ago when I took a DOP from one photo that was currently in the PL5 database and added it to another and then “discovered” the new combination. Because the DOP has a UUID the same as one already in the database PL5 automatically protects the original database entry and the incoming DOP data by creating a ‘Virtual Copy’.
I don’t believe that this should happen in any circumstances when a directory is being “discovered for the first time”! “Yes” I am using an image with a name already in the database and “Yes” I am using a DOP with an identifier (UUID) which is already in use and assigned to another photo also in the database but these are both in the context of a new directory which should automatically control any tests conducted by PL5 and also cause a new identifier (UUID) to be allocated to the DOP of the incoming image (I would contend).
I will agree that this kind of problem is being caused by testing with the same images again and again and again and … but …!
In addition why did it not happen in the previous test!?
In addition why is the actual data not quite as might be expected, i.e.
-
Image 1 has a [M]aster thumbnail that is the original photo without corrections but with a ‘Rating’ that was only added as part of the corrections applied to all the photos (as a group, excluding the last photo)
-
The same thing has happened with some but not all of the images.
Is this a database fault, no, I would contend that it is a programming fault, i.e. possibly it is a test done in the wrong order or a flag not set that indicates a new directory has been discovered so ignore checking the UUID and allocate a new one, the program has taken the wrong path for some reason in this particular case when it didn’t take that path in the previous test!
It is certainly not a typical use of the program unless you are trying to stretch the program in the ways that I tend to do (mostly by accident as in this case) but if the program can hold up to that kind of “abuse” then it should offer a reliable experience in most normal circumstances.
In all cases the Tag survived in VC[1] where it had not survived at all in the basic test in the previous post. All tests where the Tag has been involved have not shown the Tag when a copy of the photo has been re-imported in a new directory.
I have been asking for the ability to “promote” a VC[1] or VC[2] etc. for some time, either "promote or swap with the master or delete the master but keep the copies with the first becoming the [M]aster or any other combination you can think of that just isn’t available currently.
I could copy the metadata from VC[1] to [M] and then copy the corrections from [1] to [M] and then delete [1] but it would be so much easier to have a single menu command to do that!
Please note that it is possible that I have completely screwed up the test and if that turns out to be the case I apologise in advance.
PS I repeated the test and got the following which is what I had expected. But this time no Tag anywhere!!
So, are you saying that PL only checks the UUID and the filename but not the complete path to the file?
@joanna No I am saying that in the case of the test that appeared to somehow go wrong (please remember that could be something I did) either because of some value in some variable, or the direction of the wind or … the test on the directory name which should have ensured that the fact that a new directory had been encountered and conditioned all logic thereafter, failed to do what it should have done or some other “detritus” got in the way! Or a variable had been left set and that caused the problem or …
The problem is that the test before the one that failed, which was similar, passed without incident, the one I then did after the failed test, as near identical as I could get, also worked perfectly but the “little piggy in the middle” failed in quite a “dramatic” way!
When I was responsible for system testing/support and we had one of those I hated it if I didn’t have the code available to check. Intermittent faults are a potential nightmare!
I read and wrote COBOL but could only read ALGOL at a push and much of the operating system and system software code was written in DCALGOL, DMALGOL, ESPOL(the operating system) and NEWP(the ESPOL replacement) (all variants or developments of ALGOL) and I had to rely on others in the U.K as first line defence and then back to the Plant in the US or one of the development centres in the Plant, the U.K., Australia or New Zealand.
But if that call came in the middle of the night when I was on duty, and the customer’s system was flat on its back then…
As mentioned in my previous posts in this thread I have built a new PC and want to move my photo library of about 25,000 images. Those in the last two years are mostly RAW and exported jpg’s, but prior are almost all jpgs.
In reading this and other recent threads I have become very concerned about losing my RAW editing that has been done in PL4 & 5 as well as losing keywords. I put in a Support Request to DXO for their recommendations how to copy the files to my new PC but don’t yet have anything from them. With the gentle nudging of some of you fine folks here I decided to do some testing.
Please note that I only use PL5 as a RAW Developer/Editor. I use Photo Supreme as a DAM and am not interested in what PL may show for Metadata. I only want to have PL not change any of it. All keywording, Ratings, etc are added in PS.
Summary of my Testing:
- I have copied about 150 images in multiple folders from the old PC to the new one.
- Current versions of PL5 & Photo Supreme are installed on both Win 10 PCs. During testing they have all ben running. Auto Synchronize is off on both. I have restarted PL5 a few times and one time I deleted its database so it would start with a fresh one.
- Fairly consistent results: DOP, XMP and all corrections in PL5 were still present but Advanced History was not. I only had one Virtual Copy but it was present on the new PC. Keywords were sometimes in PL but not always. (Keep in mind I don’t care about MD in PL as long as it’s good in my external DAM.
- Not all keywords showed in Photo Supreme on the new PC. Not sure why this is. Using XnView MP it showed they were in the XMPs. I may need to change some settings on Photo Supreme. Before I bring all the photos to the new PC I need to sort that out. I can’t imagine any way that PL5 could prevent PS showing MD that resides in the XMPs.
- Bottom line for me is that I now feel I can safely use PL5 just for Developing/Editing and Photo Supreme, or quite likely Imatch as my DAM.
Rod
1 Like
@nwboater Welcome to the testers “club”. Please note that I had to “park” another longish post to make this response.
I am glad that you are gaining confidence in the product and satisfying yourself about its reliability (or otherwise) with respect to your workflow. Sadly the ‘Advanced History’ is not stored in the database so would never make its way to the DOP and others have expressed concern about this in past posts.
I (personally) don’t know of any plans to maintain what could be an ever increasing list in the database, in the DOP or in some other structure and I would then have to set out see how many changes it could actually cope with before it broke (a test I would never actually try to run).
It would be potentially useful but what I have complained about that is stored in the DOP and in the database is this
the ‘Applied default preset’ which is (mostly) always the same and about as much use as a “chocolate tea pot”.
I asked for something more useful like the ‘Last preset applied’ and/or the ‘Last preset created’ sometime ago which would at least give some provenance to the current state of the photo.
However, a full history with the feature of being able to “roll-back” and then to continue editing would require more than just the storage of a list!.
I am concerned why some keywords were in PL5 but some were not, even though you are not concerned about them. I am also unclear why you are not concerned about PL5 keywords not exactly matching those in Photo Supreme?
I would anticipate that your workflow would be to add keywords and then import (discover) the folders in PL5, edit and then export with the keywords intact, sorry if you have written the details before but I went looking and couldn’t find them. I did notice that you described that you do not use hierarchical keywords which means less “tinkering” from PL5 BUT it will put ‘hr’ keywords for each ‘dc’ keyword present in the image!
When you have time please tell me about the “typical life” of an nwboater image, i.e. your workflow and we can see if it is possible to minimise any potential re-working and what you can expect from PL5.
Because of the “category” issue Photo Supreme does not figure in my analysis of what other packages do with entered keywords and what PL5 then seems to do to those keywords when it encounters them XMP-files gets F*cked up due hierachical mismanagement in dual management - #38 by Joanna.
Please remember all my caveats about potential “feet of clay”, buyer (reader) beware etc. what I write comes from tests and are only as good as the creator of the tests (me), the person who executes the tests (me) and the interpreter of what the tester thought they saw (me) and I make no guarantees!
1 Like
Different to the Mac version, the entries in the history are not kept at all – so no ‘fault’ from copying.
4 Likes
Some clarification here (since the OP was in context of the Mac version of PL);
- Advanced History is stored in the database for the Mac implementation, but not for the PC version.
- This is why the Mac version enables tracking of the minutiae of step-by-step corrections executed to achieve the final result across multiple sessions - whereas, this is available for the PC version only within the current session. (Of course, all final-state corrections are saved for both variants.)
- Some users consider this to be a serious omission of the PC version - - I’m not fussed by this at all.
John M
2 Likes