XMP-files gets F*cked up due hierachical mismanagement in dual management

I’ve looked at the XMP file you have posted but don’t know if this is how you intended it to be or how PL has written it.

Here’s what I see in that file…

         <lr:hierarchicalSubject>
            <rdf:Bag>
               <rdf:li>Architectuur</rdf:li>
               <rdf:li>Museum</rdf:li>
               <rdf:li>Type</rdf:li>
               <rdf:li>Vakanties en dagjes weg</rdf:li>
               <rdf:li>Vakanties en dagjes weg|Vakanties</rdf:li>
            </rdf:Bag>
         </lr:hierarchicalSubject>

According to the MWG guidance, this doesn’t make sense, as it says you have three keywords that do not belong to any hierarchy; such entries are not valid.

The last two entries describe a hierarchical keyword Vakanties, which is a child of Vakanties en dagjes weg; the first line describing the root of the hierarchy, the second describing the “path” of the child keyword.

In my keywording software, I interpret the MWG guidance by including all levels of only true hierarchical keywords, so my app would write…

         <lr:hierarchicalSubject>
            <rdf:Bag>
               <rdf:li>Vakanties en dagjes weg</rdf:li>
               <rdf:li>Vakanties en dagjes weg|Vakanties</rdf:li>
            </rdf:Bag>
         </lr:hierarchicalSubject>

… although the MWG guidance can also be interpreted that…

         <lr:hierarchicalSubject>
            <rdf:Bag>
               <rdf:li>Vakanties en dagjes weg|Vakanties</rdf:li>
            </rdf:Bag>
         </lr:hierarchicalSubject>

… is just as valid.

My instinct has been to stick to declaring the root keyword as well as the description of the “child” and, so far, no other software misinterprets that.

MWG guidance also states that all keywords mentioned in the lr:hierarchicalSubject tag must be mentioned in the dc:subject tag and so…

         <dc:subject>
            <rdf:Bag>
               <rdf:li>Architectuur</rdf:li>
               <rdf:li>Museum</rdf:li>
               <rdf:li>Type</rdf:li>
               <rdf:li>Vakanties</rdf:li>
               <rdf:li>Vakanties en dagjes weg</rdf:li>
            </rdf:Bag>
         </dc:subject>

… is totally correct.


In the last couple of years working on my app I have come to the conclusion that the two related tags dc:subject and lr:hierarchicalSubject serve two different purposes…

  • dc:subject → searching
  • lr-hierarchicalSubject - communication

Unfortunately, DxO’s current stance is that all keywords should be written as hierarchical and their search is based on such an assumption; which is why I am inclined to state that the XMP file you posted was written by PhotoLab :crazy_face:

This assumption is just plain wrong as it can contribute to both limited and incorrect search results in other software. For example, if I have some images marked Colour|Orange, some marked Fruit|Orange and yet others marked Enterprise|Telecommunications|Orange, PL5 does not currently allow me to search for all images marked with Orange, regardless of hierarchical context, because it seems to search based on hierarchy and not just the keywords found in the dc:subject tag. Which is why they are forcing all standalone keywords to be also single level hierarchies. This is wrong and incompatible with every other keywording app I have so far found.


At present, the PL5 search is woefully inadequate, in that it doesn’t allow searching based on OR predicates, only AND predicates, and then only for complete hierarchical phrases.

Searching for hierarchical phrases should not be done on the lr:hierarchicalSubject tag, it should be done by ANDing together the keywords that form the hierarchy in a predicate for searching on the dc:subject tag. For example, I should be able to search for Fruit|Orange OR Colour| Orange, something that is currently impossible.


Unfortunately, this misappropriation of keywords in hierarchies has also been propagated to both the DOP files and database and correcting this will now entail invalidating the very keywords that PL5 has already written. Hopefully, not too many people have started using PL5 for keywording.

1 Like

I have just reread your post and saw…

So, I now assume that you have a hierarchy Architectuur|Type|Museum

In which case, the correct hierarchical tags should be…

         <lr:hierarchicalSubject>
            <rdf:Bag>
               <rdf:li>Architectuur</rdf:li>
               <rdf:li>Architectuur|Type</rdf:li>
               <rdf:li>Architectuur|Type|Museum</rdf:li>
               <rdf:li>Vakanties en dagjes weg</rdf:li>
               <rdf:li>Vakanties en dagjes weg|Vakanties</rdf:li>
            </rdf:Bag>
         </lr:hierarchicalSubject>

The problem is how do you get them to change, the history regarding getting DXO to do that isn’t great.

One can but live in hope that they don’t want to get a reputation for getting it wrong with metadata, otherwise all the effort put in so far could be wasted as people stick with what they know rather than moving to PL5. Effort that could have been put into making the image editing side of things even better.

1 Like

i will create a summary for you.
i pm you later.

1 Like

This is the case for most of the "more engaged " iptc users (xmp files by a older dam like bridge) and more professional dam software users would be mift if it “writes” self made iptcmetadata in export files as jpeg/tiff destilated from the xmp files of say imatch (even when "synchronize metadata is off.)

i have a simple and small database so a few day’s/weeks work to repair a corrupted dam system will be getting my mood in a dark place but it’s not work related so it doesn’t cost me money.
any other professional user would be not be pissed off but furious if it’s database is mashed to pieces.
Once bitten you tend to stay away to avoid a second time.
So that can cost you fanbase and users in the long run.

edit:
First modification step must be a "hierarchy of application set up in PL’s syncronizing xmp."
Very important!!!
So you can set which fields it is allowed to alter/modify and which are “read only” a selectivity box like:

me i would say write at:


and i can change every time i like to change in preference.
(this is at this moment database only when sync is off.)
this way i can control the influence the dam from dxopl has and protect my xmp files wile i can still use some of it in my developers stage so i don’t have to switch all the time between external dam and dxopl.
this gives dxo pl the time to perfectionize the dam wile every one can decide for them selfs how much it diggs in there xmp’s.

Second modification is IF xmp sync is set active the DB MUST be Slave: no visual differences even on the NOT selected preferences i described above.

The problem of metadata is that there isn’t 1 standard in how to write and read an xmp file.
The interface between two applications is randomly connected as it seems.
Depends on the interpretation of the reader (viewing corruption) and when you write back, full duplex sync, the risk of corruption of your precious metadata and carefully build iptc data is even bigger, So auto full duplex sync is without carefull writing the interface a hairy thing.
It just not write well in the xmp and the entrylines are meshed up. That’s a no go.
One problem is language, dutch towards english translation.
Lokatie => location, omschrijving => desciption, to connect the right fields it must have correct linking between fields.
And german english, swedisch englisch. Japanese, …
Adobe has far more UI languages and has therefore easier translation dictonairy.

Could be if you never used xmp’s and just start with them inside dxopl.
Then the change it works wel en has no problems is much bigger.
Problem is the UI for keyword ranking and setting is a bit wobbly.
The sync is good because then you can delete DB and re indexing to refresh and cleanout mistakes.

Yes building and maintaining is a dreadfull job. :slightly_smiling_face:
The gain is searchability in specific images, and quick project gathering in those by visualisation on selection.
Other gain is adding location and information on groups of photo’s like holiday and museums so you can see much much later what, where, who.

And this is immidiately the crux, when that data crumbles to a crisp people are not very “pleased”. So responsible software is then to blame even when it’s not directly to blame. Like kicking the dog because you stepped in its turd wile walking and looking on your phone at the same time. :sweat_smile:

1 Like

Yes, exactly! It seems obvious to me that dc:subject should be used by DxO for searching, and lr-hierarchicalSubject should indeed be for communication: to show the hierarchical keyword set by the user. (That’s how IMatch does it.) The ccurrent behavior of PL5 to create new ‘sub-keywords’ in the hierarchical keywords tag just creates a huge mess for users. Worse, it does this even when ‘sync xmp’ is turned off, at least for output files. There needs to be a way to turn this behavior off.

1 Like

So i am repairing 1 folder which i opened for GPS adding.
Google : 52.32507397711865, 4.605446382750082
DxO translate to: 52° 19’ 30" N and 4° 36’ 19" E
Bridge read that from xmp and translate to (not the same location just example): 48,50.2837N and 1,13,22.4589W.

The problem is dxo gps notification can’t be copy paste in Bridge, no translation.
what is possible is take it from old xmp:
exif:GPSLatitude=“48,50,17.0211N”
exif:GPSLongitude=“1,13,22.4589W”
and paste it in new:
problem no “empty gps line”


Rather tricky work i could mesh up the new also.
So i used DxOPL to edit in GPS locations and city and sublocation data, which needed to activate xmp sync. and that corrupted somehow my keyword hierarcie.
so now i am in a twister, can’t create new GPS transferions by dxopl (don’t sync so no writing in xmp.)
Can’t copy past from googlemaps in bridge.
So i need a translater app/website for the google GPS data. in order to finish the task of redoing the hole bunch.

@Marie @sgospodarenko -

PL5’s problem of rewriting hierarchical keywords in output images is a real issue. DxO should be fully aware. What plans are there to fix this and other metadata issues? I’ve reported these to DxO Support.

Even worse: It appears that the latest update of PL4 (4.3.6.32) has adopted this erroneous re-writing of hierarchical keywords! Users, especially those who take care managing their keywords, are going to be surprised and very unhappy about this!

It just seems so basic to me that there should be an On/Off switch for PL doing ANYTHING with Metadata. The Sync setting is not completely doing that and for some people it’s causing very serious issues.

This should be promptly addressed by DXO or they will certainly be losing customers. They should also reply in this Forum that they recognize it as an issue and what they plan to do about it.

2 Likes

Concentrating on my central heating and other posts I missed this one!

The very first post that I wrote during Beta testing on the 12th. Feb 2021 had an attached document that was even longer. I started the way that I intended to go on, sorry.

In that document I naively wrote the following


(it should have read “A number of users…”)

Effectively what is actually required is at least a keyword “pass through” option. However how much of the metadata should be simply passed through (‘Protected’) and how much should be open to updates from PL5 is a matter of debate!

If the table at XMP-files gets F*cked up due hierachical mismanagement in dual management - #19 by OXiDant existed then we could have forum posts where appropriate combinations are offered by the various DAM users (that was intended to be a serious remark not a fatuous remark).

However, I am afraid I feel that a lot is being laid at the door of PL5 which is actually unfair. If the data from one DAM was passed through another just how much of it would actually remain intact!? But there are a lot of accusations being made about PL5 because it does pretty much what these other products would do and that brings us back to the “age old” debate about the role that different users want PL5 to play in their workflow and to the fact that for many/most/all PL5 is being used foremost as a RAW Processor and photo editor and they would like the behaviour of the program with respect to metadata to be benign.

Against the advice of large swathes of its users who “warned” DxO not to enter the “nightmare” that can be DAM but to leave that to those who had invested years of development into it already, DxO ventured into “DAM” functionality nevertheless. The good news is that they haven’t done too bad a job if you want DAM features in your Photo editor, the bad news is that they have put their own DAM(n) features smack in the way of the work flow path being used by many of DxPL users.

Once again something which could have been addressed so easily by a process of two way communication and joint testing during Beta testing (don’t you just love hindsight) except that I wrote the following on the 29th. March 2021

and

I apologise for feeling “singled out” in the above post, it was my first beta test and so different from any testing I had ever done before, either as a tester or the person whose work was being tested! I felt “unloved” and naively thought that appealing to DxO, including DxO management, might change things but … I am still waiting for a response to that post and have learned to “ignore” being “ignored”, much older and wiser or perhaps just older now.

If any of what I (and others) wrote, in particular DxO reaching out to the DAM users involved with the Beta Testing and then agreeing and arranging specific tests involving those DAMs (and their users) had been acted upon then I believe that we would not be writing posts in this topic now.

But we have what we have and maybe now is the time that DxO come out from behind their “defences” and work with the user base to determine the best way of modifying DxPL to better fit with users work flows and the DAMs they use; only 10 months late

1 Like

Hello,

PhotoLab 5.2 has been released today and should improve the export of hierachical keywords.

Regards,
Marie

1 Like

Hi Marie
I have just updated and found that NO keywords are now copied over when you process from RAW to JPEG. I have added keywords in IMatch to all my RAW files and after processing nothing gets copied over even if you mark all metadata to be copied. Not sure what is happening now.
Andre

What we fixed is preservation of hierarchy in keywords.
Behavior in LightRoom is unchanged as it was fine but now C1 and PhotoMechanic get the correct hierarchy in keywords in exported images from PhotoLab.

About iMatch, @akirstein , can you provide me examples of issues you have (with description of the issue) ? You can use https://upload.dxo.com/

Regards,
Marie

Hi,
My workflowcase is:
Fast RawViewer v2 (creates xmp if i hit a star or colorflag.) Lightroom based.
Then Adobebridge for adding iptcdata and keywords to new images. Xmp (raw only)
Then load into DxOPL v5 and adding GPS data and maybe some forgotten data(which caused the earlier problem).
(now i am keep Bridge open for after metadata edits on the fly and let dxo only read to burn in exported jpegs.This let me still use autosync without the change of corrupting my xmp’s.)

Does the fix you talk about repaired connection between bridge and DxO?
So a change in dxo doesn’t cause writing a editted line on a different location in the xmp as the original place which was written by Bridge?
And honor the way bridge write the hierachical keywords?
(i am just fixed all xmp’s so i am very carefull. :wink:)

My workflow is as follows:
Upload using Fast View, cull as required, save to my drive
Open IMatch, add keywords and other metadata as required, batch rename YYYYMMDD-####
Open PL and edit RAW, export processed JPEG to same file location, noting keywords are to be copied.
In IMatch I can now filter all JPEGS for view or share as required.
In PL5.1 the keywords were copied with no issues.
In PL5.2 no keywords are copied over from RAW to JPEG.
OR
Open PL and edit RAW, export processed TIFF to NIK Collection, complete editing and then export final JPEG to same file location, noting keywords are to be copied.
In IMatch I can now filter all JPEGS for view or share as required.
In PL5.1 all hierarchy keywords copied as well as having the words duplicated into flat keywords.
I have not tried to do this in PL5.2 yet.

I have also seen that PL5.2 now asks you to either write or read metadata to the file as it has detected metadata was added using other software. (Small “S” icon appears on top right of photo.) All I want PL to do is read the metadata, not write anything as I spent months cleaning up the problem created by PL5.1. If this means it will write the data to PL5 then copy that same data back to the JPEG that would be great.

As OXiDant notes I too am treading carefully because it took a long time to fix my keywording up and I don’t want it to happen again. I would be interested to know what will happen if I “wrote” the metadata to the file so will try later t see what happens. As noted, treading carefully.

It works for me with PL5.2
I add keywords in RAW with Bridge and after open PL export JPEG
The keywords are copied on JPEG with all hierarchy

@akirstein I am concerned about your post because although there are changes a simple export test with old test data shows a difference in the exported JPG but the data is still there! The test was using data created by IMatch that was then input to PL5, according to the file naming this was for PL5.1.2 and exported. I re-exported the data to check what might be happening.

At the same time @Marie had posted in another related post and the final (so far) post from @Marie was PL 5 Keywords Handling: Output Files - #16 by Marie indicating that there have been changes made in the format of the output for hierarchical keys.

The following is a screenshot from PIE that indicates the changes are a reduction in the ‘dc’ fields generated within PL5.2, which I presume is what is being described by @Marie . The dataset is a mixture of JPG and RAW (RW2) images and the _DxPL5 JPGs are from the original tests and the _DxPL5.2 images are what was produced a few minutes ago!

This does not repeat your workflow but the original data was from IMatch into an earlier version of PL5 and then exported in that version and then again just now in PL5.2.0 and shows what has changed.

I will try to repeat your workflow when I have time, to see if I can reproduce any data loss!

Having read a number of posts now I decided to remove and reload PL5.2 as a number of other users seem to have had an issue corrected when they did this. After reloading I found all my keywords are seen and correctly listed so I am not sure what happened the first time I loaded PL. It has also read and loaded the old keywords and did not ask about needing to read keywords again so I am very happy with the outcome. I did a lot more testing last night and am very pleased to see the keywords stay as I set them up with no issues when I export.

So thank you team at DXO, really appreciate the work you do and for all the posts. They are really helpful to a non-geeky old timer like me :slight_smile: