XMP-files gets F*cked up due hierachical mismanagement in dual management

@Marie @sgospodarenko -

PL5’s problem of rewriting hierarchical keywords in output images is a real issue. DxO should be fully aware. What plans are there to fix this and other metadata issues? I’ve reported these to DxO Support.

Even worse: It appears that the latest update of PL4 (4.3.6.32) has adopted this erroneous re-writing of hierarchical keywords! Users, especially those who take care managing their keywords, are going to be surprised and very unhappy about this!

It just seems so basic to me that there should be an On/Off switch for PL doing ANYTHING with Metadata. The Sync setting is not completely doing that and for some people it’s causing very serious issues.

This should be promptly addressed by DXO or they will certainly be losing customers. They should also reply in this Forum that they recognize it as an issue and what they plan to do about it.

2 Likes

Concentrating on my central heating and other posts I missed this one!

The very first post that I wrote during Beta testing on the 12th. Feb 2021 had an attached document that was even longer. I started the way that I intended to go on, sorry.

In that document I naively wrote the following


(it should have read “A number of users…”)

Effectively what is actually required is at least a keyword “pass through” option. However how much of the metadata should be simply passed through (‘Protected’) and how much should be open to updates from PL5 is a matter of debate!

If the table at XMP-files gets F*cked up due hierachical mismanagement in dual management - #19 by OXiDant existed then we could have forum posts where appropriate combinations are offered by the various DAM users (that was intended to be a serious remark not a fatuous remark).

However, I am afraid I feel that a lot is being laid at the door of PL5 which is actually unfair. If the data from one DAM was passed through another just how much of it would actually remain intact!? But there are a lot of accusations being made about PL5 because it does pretty much what these other products would do and that brings us back to the “age old” debate about the role that different users want PL5 to play in their workflow and to the fact that for many/most/all PL5 is being used foremost as a RAW Processor and photo editor and they would like the behaviour of the program with respect to metadata to be benign.

Against the advice of large swathes of its users who “warned” DxO not to enter the “nightmare” that can be DAM but to leave that to those who had invested years of development into it already, DxO ventured into “DAM” functionality nevertheless. The good news is that they haven’t done too bad a job if you want DAM features in your Photo editor, the bad news is that they have put their own DAM(n) features smack in the way of the work flow path being used by many of DxPL users.

Once again something which could have been addressed so easily by a process of two way communication and joint testing during Beta testing (don’t you just love hindsight) except that I wrote the following on the 29th. March 2021

and

I apologise for feeling “singled out” in the above post, it was my first beta test and so different from any testing I had ever done before, either as a tester or the person whose work was being tested! I felt “unloved” and naively thought that appealing to DxO, including DxO management, might change things but … I am still waiting for a response to that post and have learned to “ignore” being “ignored”, much older and wiser or perhaps just older now.

If any of what I (and others) wrote, in particular DxO reaching out to the DAM users involved with the Beta Testing and then agreeing and arranging specific tests involving those DAMs (and their users) had been acted upon then I believe that we would not be writing posts in this topic now.

But we have what we have and maybe now is the time that DxO come out from behind their “defences” and work with the user base to determine the best way of modifying DxPL to better fit with users work flows and the DAMs they use; only 10 months late

1 Like

Hello,

PhotoLab 5.2 has been released today and should improve the export of hierachical keywords.

Regards,
Marie

1 Like

Hi Marie
I have just updated and found that NO keywords are now copied over when you process from RAW to JPEG. I have added keywords in IMatch to all my RAW files and after processing nothing gets copied over even if you mark all metadata to be copied. Not sure what is happening now.
Andre

What we fixed is preservation of hierarchy in keywords.
Behavior in LightRoom is unchanged as it was fine but now C1 and PhotoMechanic get the correct hierarchy in keywords in exported images from PhotoLab.

About iMatch, @akirstein , can you provide me examples of issues you have (with description of the issue) ? You can use https://upload.dxo.com/

Regards,
Marie

Hi,
My workflowcase is:
Fast RawViewer v2 (creates xmp if i hit a star or colorflag.) Lightroom based.
Then Adobebridge for adding iptcdata and keywords to new images. Xmp (raw only)
Then load into DxOPL v5 and adding GPS data and maybe some forgotten data(which caused the earlier problem).
(now i am keep Bridge open for after metadata edits on the fly and let dxo only read to burn in exported jpegs.This let me still use autosync without the change of corrupting my xmp’s.)

Does the fix you talk about repaired connection between bridge and DxO?
So a change in dxo doesn’t cause writing a editted line on a different location in the xmp as the original place which was written by Bridge?
And honor the way bridge write the hierachical keywords?
(i am just fixed all xmp’s so i am very carefull. :wink:)

My workflow is as follows:
Upload using Fast View, cull as required, save to my drive
Open IMatch, add keywords and other metadata as required, batch rename YYYYMMDD-####
Open PL and edit RAW, export processed JPEG to same file location, noting keywords are to be copied.
In IMatch I can now filter all JPEGS for view or share as required.
In PL5.1 the keywords were copied with no issues.
In PL5.2 no keywords are copied over from RAW to JPEG.
OR
Open PL and edit RAW, export processed TIFF to NIK Collection, complete editing and then export final JPEG to same file location, noting keywords are to be copied.
In IMatch I can now filter all JPEGS for view or share as required.
In PL5.1 all hierarchy keywords copied as well as having the words duplicated into flat keywords.
I have not tried to do this in PL5.2 yet.

I have also seen that PL5.2 now asks you to either write or read metadata to the file as it has detected metadata was added using other software. (Small “S” icon appears on top right of photo.) All I want PL to do is read the metadata, not write anything as I spent months cleaning up the problem created by PL5.1. If this means it will write the data to PL5 then copy that same data back to the JPEG that would be great.

As OXiDant notes I too am treading carefully because it took a long time to fix my keywording up and I don’t want it to happen again. I would be interested to know what will happen if I “wrote” the metadata to the file so will try later t see what happens. As noted, treading carefully.

It works for me with PL5.2
I add keywords in RAW with Bridge and after open PL export JPEG
The keywords are copied on JPEG with all hierarchy

@akirstein I am concerned about your post because although there are changes a simple export test with old test data shows a difference in the exported JPG but the data is still there! The test was using data created by IMatch that was then input to PL5, according to the file naming this was for PL5.1.2 and exported. I re-exported the data to check what might be happening.

At the same time @Marie had posted in another related post and the final (so far) post from @Marie was PL 5 Keywords Handling: Output Files - #16 by Marie indicating that there have been changes made in the format of the output for hierarchical keys.

The following is a screenshot from PIE that indicates the changes are a reduction in the ‘dc’ fields generated within PL5.2, which I presume is what is being described by @Marie . The dataset is a mixture of JPG and RAW (RW2) images and the _DxPL5 JPGs are from the original tests and the _DxPL5.2 images are what was produced a few minutes ago!

This does not repeat your workflow but the original data was from IMatch into an earlier version of PL5 and then exported in that version and then again just now in PL5.2.0 and shows what has changed.

I will try to repeat your workflow when I have time, to see if I can reproduce any data loss!

Having read a number of posts now I decided to remove and reload PL5.2 as a number of other users seem to have had an issue corrected when they did this. After reloading I found all my keywords are seen and correctly listed so I am not sure what happened the first time I loaded PL. It has also read and loaded the old keywords and did not ask about needing to read keywords again so I am very happy with the outcome. I did a lot more testing last night and am very pleased to see the keywords stay as I set them up with no issues when I export.

So thank you team at DXO, really appreciate the work you do and for all the posts. They are really helpful to a non-geeky old timer like me :slight_smile:

If auto sync isn’t active you need to use "file-> metadata-> read metadata. to update xmp iptc data.
By removing and re installing you actualy remove the database and refresh this by installing.
so it sees all new entry’s because there “new”

If I can just chip in here.

I just ran a test with a NEF file with hierarchical keywords embedded in it (as opposed to in an XMP sidecar file)

I used my own keywording app that is fully MWG compliant to prepare the file.

Embedded in RAW

Here is the ExifTool output for that file…

[XMP]           Subject                         : Couleur, Orange, Entreprise, Télécommunications
[XMP]           Hierarchical Subject            : Couleur, Entreprise, Couleur|Orange, Entreprise|Télécommunications, Entreprise|Télécommunications|Orange

… and here is the ExifTool output for an exported JPEG…

[XMP]           Subject                         : Couleur, Entreprise, Orange, Télécommunications
[XMP]           Hierarchical Subject            : Couleur, Couleur|Orange, Entreprise, Entreprise|Télécommunications, Entreprise|Télécommunications|Orange

Written to XMP sidecar

Now, I use my app to clear out the keywords from the NEF file and rewrite the same keywords to an XMP sidecar file…

  <dc:subject>
   <rdf:Bag>
    <rdf:li>Couleur</rdf:li>
    <rdf:li>Orange</rdf:li>
    <rdf:li>Entreprise</rdf:li>
    <rdf:li>Télécommunications</rdf:li>
   </rdf:Bag>
  </dc:subject>
  …
  <lr:hierarchicalSubject>
   <rdf:Bag>
    <rdf:li>Couleur</rdf:li>
    <rdf:li>Entreprise</rdf:li>
    <rdf:li>Couleur|Orange</rdf:li>
    <rdf:li>Entreprise|Télécommunications</rdf:li>
    <rdf:li>Entreprise|Télécommunications|Orange</rdf:li>
   </rdf:Bag>
  </lr:hierarchicalSubject>

Closing PL5, deleting the database and any DOP file, reopening PL5 and exporting to JPEG gives the following ExifTool output from the JPEG…

[XMP]           Subject                         : Couleur, Entreprise, Orange, Télécommunications
[XMP]           Hierarchical Subject            : Couleur, Couleur|Orange, Entreprise, Entreprise|Télécommunications, Entreprise|Télécommunications|Orange

This is totally correct, as expected and MWG compliant.

@OXiDant @akirstein what are you using to write the original keywords and hierarchies and could you please post

  1. the dc:subject and lr:hierarchicalSubject tags that you have before PL5 gets to touch them
  2. the dc:subject and lr:hierarchicalSubject tags that you have after PL5 has done its thing

first Fast raw viewer then Adobe Bridge 2022 and finaly DxOPL.
Adobe Bridge is my main DAM.

I only use IMatch as my DAM software. Fast Raw Viewer is only for culling and selecting photos off the camera card which is then added to IMatch for keywording and the odd marking (one star, two stars etc.).

I tried again to load more photos today and PL showed metadata was written by another software when I first opened the new folder in PL. When I clicked that it should read the data it worked well by adding the information to the PL database and then correctly copied the information from the RAW file to the JPEG when processed. Before I reloaded PL again the original process did not copy the information so hence the reason no keywords were written to the exported file. Not sure what the reason was but it seems to have sorted itself out with the reload. So far I have not seen any issues with the hierarchal structure and there is no errors when reading back to IMatch.

Would you be so kind as to show me the keywords and their hierarchies as you entered them in Bridge? A screenshot should suffice.

@joanna Long time no speak! As a result of @akirstein concerns over what PL5.2.0 had done with keywords I did some more tests and was going to put them in my topic PL5.2.0.4730 Conflict Resolution only detects externally generated conflicts (getting very parochial all of a sudden) but will place them here instead!

Way back in a post far far away and certainly far away from the topic itself (a “bug” reported by me) I asked Joanna and @jch2103 to enlighten me on the “correct” representation of metadata and that started a lot of discussions and tests, amongst which I produced a spreadsheet that looked at what PL5 (PL5.1.2 to be exact) did with the keyword data that other software had “lovingly” placed in the image (in the RAW xmp sidecar file) when that image was then exported from PL5 as a JPG.

The original post that contains the spreadsheet I produced with input from others was contained in PL4.3.6.32 Ghosting Ratings for Same image in different directories (Win10) - #114 by BHAYT, i.e.

Always good to quote yourself!?

So I took the original test data and copied it to a new location and repeated the tests on PL5.2.0 on my Main machine and on PL5.1.4 on my Test machine and produced the following, where the first rows are from the original file, the second from the original PL5.1.2 tests (designated just DxPL5), the third from PL5.1.4 and finally from PL5.2.0 so that we can see exactly what DxO has changed!

  1. Personally I would have retained the original as an option and allowed the users to choose between the two!

  2. I don’t think that this answers the many posts that complained that the metadata so lovingly inserted by their DAM systems has been altered by PL5!!

It is good to see that DxO monitor the posts and take note of the various complaints. it is sad that they then make an arbitrary change and seem to not really have consulted with their users, “You can lead a horse to water but you can’t make it drink and you can’t stop it … in the pond!”.
After that remark and the frosty look I got from my wife, when she kindly brought me a cup of coffee and saw this post etc. on my screen, this might well be my last post!!

ACDSEE:-

Bridge:-

Photo Mechanics:-

IMatch:-

PL5.1.2:-

Lightroom:-

Capture 1:-

Zoner:-

Very difficult to read these spreadsheets but the only one I can see that adheres fully to MWG guidelines (as I understand them) and records the same results as my app, is Capture One.

Given a hierarchy of Animal > Mammal > Bear > Black Bear the guidelines state that the metadata written to an XMP file should be…

  <dc:subject>
   <rdf:Bag>
    <rdf:li>Animal</rdf:li>
    <rdf:li>Mammal</rdf:li>
    <rdf:li>Bear</rdf:li>
    <rdf:li>Black Bear</rdf:li>
   </rdf:Bag>
  </dc:subject>
  …
  <lr:hierarchicalSubject>
   <rdf:Bag>
    <rdf:li>Animal</rdf:li>
    <rdf:li>Animal|Mammal</rdf:li>
    <rdf:li>Animal|Mammal|Bear</rdf:li>
    <rdf:li>Animal|Mammal|Bear|Black Bear</rdf:li>
   </rdf:Bag>
  </lr:hierarchicalSubject>

That way, all the keywords in the hierarchy are available for searching, via the dc:subject tag and the full hierarchical structure of all keywords is transmitted in the lr:hierarchicalSubject tag.

a exported txt file sufficient?
keyword list.txt (1,1 KB)
right:
step one
FRV: create xmp. bij star and colorlabel.
_1080786.xmp (560 Bytes)
_1080791.xmp (560 Bytes)
Then step 2:
Bridge 2022:
meta iptc preset. and keywords: one as main and sub visible in checkbox:
visible row
and one as hide main show sub:
no visible main
_1080786 step 2.xmp (4,6 KB)
_1080791 step 2.xmp (4,7 KB)
Then open in Plv5.2
fill in some fields and add keyword “planten” from list. (always synchronize is active)
seen in bridge
_1080786 step 3.xmp (8,3 KB)
_1080791 step 3.xmp (8,4 KB)
Sins i have always sync active i don’t add new keywords in database of dxopl because i polute then this database.
(if it’s necessary for you to have that also then i
switch off allways sync.
add something to this two files (786/791) and manual export to metadata(xmp) and then delete database and modified xmp’s and re- index my archive to ingest all metadata again. (i don’t add keywords in dxo for now unless i know its safe to do so. :wink: )
if you need a bigger test i will. :slight_smile:

Peter

@joanna Sorry about the legibility they are actually not spreadsheets they are “windows” into 4 simultaneous windows from PIE put together in real-time by 'Sneaky Previews" doing it this way saves me trouble and time trying to copy to the spreadsheet and I have found the colour combinations available in PIE not particularly good when using screen grabs - sorry!

Are any of the following easier to read?