PL5 keywords, one step closer, many steps still to go

DxO PhotoLab
13

And, as a programmer myself, who has just spent two years writing a keyword management app for Mac, I can reassure you that it takes some getting your head around metadata standards and compatibility.

In brief, it’s a mess but, after extensive research and testing, I can assure you that this is neither over-specified nor unnecessary, especially from a point of view of compatibility with other software, which can behave equally badly.

If we take the case of non-hierarchical keywords, it is simply a matter of adding each required keyword to the xmp-dc:subject tag.

But, with hierarchical keywords, we also need to populate the xmp-lr:hierarchicalSubject tag. In order to do this, we need to write each hierarchical keyword there as well as to the xmp-dc:subject tag. But it is not just a matter of including all keywords in a hierarchy once in an all-encompassing “phrase”.

In your example, you want to use both ungulate and giraffe, but in so doing, you are referring to two different hierarchical keywords:

  • animal|mammal|ungulate
  • animal|mammal|ungulate|giraffe

These hierarchical keywords are integral and indivisible entities - they are not lists of keywords and one cannot be “derived” from the other.

Thus the requirement for both to be included in the xmp-lr:hierarchicalSubject tag.

PL5 effectively does this but it is hidden behind a tooltip that (eventually) appears when you hover over one of the tokens. If duplication is deemed necessary, then, like you, I would rather see the Lr/Bridge style of explicit definition of each hierarchy.

14

Fine. As long as you don´t change the metadata also in Photolab Photo Library and just stick to edit the pictures you are in a one way flow. I just don´t understand why you have to export even smaller sized files carrying the metadata. Isn´t it possible just to open the Lightroom RAW XMP-sidecars associated with your RAW-files directly in Photolab without having to rely on the smaller files? I don´t understand.

If I update metadata on my RAW-files in Photo Mechanic the associated XMP-sidecars gets updated correctly, Photolab will automatically be able to red it as soon as the same RAW is opened in Photolab. If I then export a JPEG from Photolab with that RAW as a metadata master, all that metadata will be transferred to JPEG, TIFF or DNG without any further problem or any other act necessary from my side.

It´s a long time since I gave up on Lightroom but I don´t understand why you need the thumbnails. Shouldn´t the XMP-sidecars to your RAW be sufficient XMP-data containers by themselves? I thought it would be enough to turn on the switch in Lightroom that activates the XMP-sidecar file funktion.

15

could you elaborate?
xmp files are side cars for rawfiles because writing in the rawcontainer, nikon seems to apply that, isn’t a smart thing to do. Produce LR different files then xmp?

The DxOdopfiles are “1 image database pieces” soly for DxO-application use.
reading with a notpad you can see what’s in it.
The reason that exifdata are found inthere is it’s DataBase copy for 1 image edit state.
So 1 image 2 file’s.
xmp and dop.

This means that organising, moving images around should be done inside DxOPL in order to keep side cars stick to image. (Bridge does show all file so when move you can selected all en move around but a error is made quickly.)

16

That I can accept. But…

I disagree. It is simplicity itself to split on the | character. In fact, one could argue that in the perfect world that would be the only field needed as the individual keywords could all be derived.

I would agree with the statement that “these lists of keywords do not get split by software.” But they absolutely could. If you’ve been anywhere near databases you will know about normalisation.

However… the basic hierarchy function as implemented by PL is not my end game. LrC takes things further (and without needing to duplicate the hierarchy at all in the files themselves). Which brings me to…

It took me a while to understand, too, but the key point I haven’t explained well enough so far is the additional capability LrC has:

  • synonyms
  • non-exporting keywords

Consider this real hierarchy I have.

  • !places > Oceania > New Zealand > Wellington > City

When LrC writes the keywords to the DNG files (or XMP files) you will see every entry that I have listed. However, when it exports files, things change.

What I have not shown above are the extra capabilities.

!places is marked as non-exporting, which means it is there purely for structure in the keyword manager in LrC. There is no keyword I want on my published photos that would logically encompass every region of the world. (I’m not one of the 18 people who might reasonably use ‘Earth’ and ‘Moon’.) I considered just having each region at the top level but then I have to scroll for miles to get to Oceania and then miles back in the other direction for Africa. The !places keyword holds together the related regions and their descendants, and by means of having the initial ! character it is always at the top of my keyword list.

New Zealand has a synonym Aotearoa which is the indigenous name for my country. Again, I could work around this by having both keywords individually, or perhaps one hierarchical under the other, but the concept of a synonym is exactly what represents the relationship. Every photo that has New Zealand or Aotearoa on it should have both. With separate keywords or as a hierarchy it is possible to mess that up.

So in LrC if I add City < Wellington then a PL5 export will have exactly the list I have shown above in the Subject field (unrelated). Whereas a LrC export will have these:

  • Oceania, New Zealand, Aotearoa, Wellington, City

The classic case for synonyms for me is airports and airfields. My local airport is named Wellington International Airport, it has an ICAO code of NZWN and an IATA code of WLG. Those latter two are unique in the world, where the former may be but I’m sure there are many airports in the world with the same common names. I have all the airports and airfields under an airfield keyword which is set not to export because it’s not a useful keyword to have on the final product. And when I want to add my airport I just type NZWN and I get Wellington International Airport ‘for free’ without having to deal with suggestions for all other variations of Wellington in my keywords. I currently do not include the IATA codes but I probably should for completeness. While I would have to re-export and re-publish my photos, actually adding WLG to the thousands that already have NZWN is a simple edit of the NZWN keyword to add WLG as a synonym.

17

Turning off the option to two way communication results in no keywords are being shown.image
image

18

As I have said before, it’s not so much a matter of what can be done, it’s much more about being able to edit keywords in one piece of software and have another piece of software understand them and use them.

Before DxO starting doing anything about a DAM, I started writing my own app (macOS) to fill in the gap.

It has been a nightmare! Trying to read and write metadata that works for every other piece of software is meant to be based on standards and the best source of the most universal standard is the MWG (Metadata Working Group) guidelines. The problem is, Adobe, who participated in their drawing up, have decided to diverge from them - to the extent of having two apps, Lr and Bridge, which default to different rules and, what’s more, offer not very easily findable ways of changing how they write stuff!

Then you have all the other software which sort of, possibly, maybe, adhere to the guidelines - at least according to how they have managed to interpret them. And, I must add, that includes my app which, I hope, does its best to bridge those gaps.

To start with, my app has a dedicated Keyword Manager, where keywords can be: added, removed, renamed, arranged in hierarchies, etc. This is key to ease of use, allowing searching for keywords, in their hierarchies, and directly adding them to the keywords box in the main window for one or more images simply by double-clicking on a word in the manager popup window.

Of course, now, @platypus pointed out that a user would have had to enter all their keywords, one at a time, into my app - which, when you see the size of some of the ready-made lists that are available, could take quite some time and effort. So, I am now in the middle of adding in being able to import ready-made lists and, so far, what I have done is working quite well but your post has caused me to think seriously about adding in non-exporting and synonym functionality.

Questions

For synonyms, how do you determine which word “takes the lead”?

Take this example from the Foundation List:

	[~Characteristic]
		beautiful
		cute
			{adorable}
			{lovable}
			{sweet}
		short
		tall
		ugly

… where there are three synonyms for cute and, presumably, are, sort of, “children” and not “siblings”

In your example of airports, which is a domain I have worked in, ICAO, IATA and names could all qualify for being the “master”. Or do we have to have a mutual lookup from each one to the other?

Do you always write the name, or could there be a requirement to write IATA or ICAO? And who decides?

Or would you have simply have two synonyms (for IATA and ICAO) linked to that airport name?

19

Hi, Interesting thread. My own limited research into meta data specifically keywords, rating and IPTC fields with the aim of creating a Rosetta Stone type app just made me realise that it is all rather a mess. Some apps update jpeg, tiff and dngs with keywords others just create an xmp file which might meet the requirements of one application but not another. Other apps read but do not write and some get confused if there is metadata in the file along with an xmp file. In the end I gave up and deleted all of the xmp sidecar files and started adding keywords directly to the file names. While this is limited in someways at least my collection of images may be searched using almost any recent computer OS you care to mention.

I’m sure that hierarchal have their uses but I am not sure what problem they are trying to solve so have concluded that I don’t need them. For those who do - good luck.

Simon

20

sounds very much like → Time stamp expansion for image --DXO PL4 Elite - #2 by Wolfgang

21

Hi Wolfgang,

Yes very similar. My files are renamed on import using the sql date time, they are saved into a day date based folder structure e.g. PhotoLibrary/2021/2021-05-21/2021-05-21-102427-P4527654-Tag1-Tag2-Tag3-TagN-.ORF . The sub seconds are not an issue as I keep the original camera file name, P4527654 and as yet I have never had two cameras using the same original file name on the same day let alone the same second.

I use an application to set and unset tags (a.k.a keywords) and it renames the raw files and any .dop, .xmp or ExposureX6 sidecars. I chose to use dashes as the separator rather than underscore as I read that this is what Google uses and some OS trip up on the underscore character.

DxO PL tolerates external renaming of files except where the files are in a project. It also seems that the database retains records of old file names so I delete the database from time to time. I use an application “Find any File” to search my collection and this allows saved searches. It means that my collection is free of any database applications and can be read on any modern OS. As I said it works for me.

Simon

Screen shot of a few image files:
2021-10-23-175811-Screenshot 2021-10-23 at 17.58.06

1 Like
22

I think this is the key (pardon the pun). The keyword manager is where the battle is won and lost as far as I am concerned. Given the vagaries of keyword storage I think we have to accept that the master has to be a ‘smart’ keyword dictionary that transcends any standard, or non-standard, storage of the keywords in image files. The file storage model should be a translation, to or from the dictionary.

In other words, the dictionary is the ‘truth’ and the user may have the option to import, via a form of translation, from the files into the dictionary, and then export, again via a translation, to the files. Such import translations may cue off other metadata to know what to expect. This would make the handling of the A|B|C, A|B versus just A|B|C a simple choice of correct/desired translation. Though I admit there’s a lot more to it than that and actually pulling this off would still be difficult. But translating directly from, say Adobe to PhotoLab styles, and back is the way madness lies.

But… the keyword manager shouldn’t just stop at showing or searching a tree and allowing clicks or drags. I only work ‘in the tree’ when I am creating keywords, or occasionally if I am unclear what may already exist. When I am applying keywords it is a keyboard activity. This capability in LrC is what makes it all work for me.

Imagine this aircraft picture is in need of keywords. The first thing I am looking for is the registration, 9V-SQN. I will type 9V and immediately see all of the Singapore registered aircraft I have recorded before. With luck, 9V-SQN will be in the list and either by typing two or three more characters and/or arrowing down, I will pick it from the list, at which point I will get 777-200ER, 777, Boeing, and aircraft come along for the ride.

Next, I know it’s taken at Wellington so NZW is enough to get me close to NZWN. And finally, Sing is close enough to get Singapore Airlines. Something like a dozen keystrokes and/or maybe the odd click gets me my 8 keywords (including the NZWN synonym). Note that during this entire process my brain is thinking about the result and not the structure of my tree. As soon as I have to scroll a tree in search of something or type a search and then pick out of the list with a drag or double-click, it requires more concentration.

If 9V-SQN is not present, it will become apparent quickly, at which point I will take to the tree and add it under 777-200ER, simultaneously adding it to the selected photos. This is another reason the low effort for the already present keywords is a huge benefit, because adding new keywords to the hierarchy often involves research. When I’m adding keywords to 500+ photos from an air show (a task I make myself do before processing anything), speed matters. As does not burning out from concentrating too hard.

The only time I actually spend a lot of time ‘in the tree’ is if I am setting up a new hierarchy or, more likely, rearranging an existing one to a new way of thinking (something I do far too often).

For me it comes down to “what am I most likely to think of typing?” For the airports, I just use NZWN because it’s shorter than the full name, and therefore quicker to get a unique match on typing. The IATA code is arguably slightly shorter and more likely will match quicker (all New Zealand airports’ ICAO codes start with NZ) but I’m just more familiar with the ICAO codes.

For New Zealand/Aotearoa, New Zealand is the lead because that’s what I grew up calling it. For birds which have common names and Māori names that I want to include (I don’t espouse Latin names) then mostly I use the common name because, again, that’s what I’ve grown up calling them. An exception might be the Tūi which arguably has the common name Parson Bird, but other than seeing it in books I don’t think I’ve ever heard anyone call them that. In fact, I don’t even have that as a synonym now that I think of it. I probably should for completeness.

23

Oh I am glad that Photo Mechanic Plus can handle that perfectly fine already. When I export from it takes the RAW+ DOP + XMP if there is any. I think I misunderstood since we were talking about hierarchical keywords.

1 Like
24

No problem,
it’s about managing keywords, IPTC and Exif which are written in different parts of the file set.
Exif is in rawfile, keywords and IPTC should be written in xmp and STAY there(i mean not imported and altered by a program.). and propertie file type like tiff and jpeg has containers where some/most of the data can be written.
DNG is also a container which can hold all metadata.

hierachical keywords are very difficult to view properly.
every one wants different kind of default view.
some want to see the hole string.
(which can be filling the view alot.)

Say:
human, male, adult
human, male, juvenile
human, female, adult
human, female, juvenile.

search for juvenile would show:
human,male,female,juvenile.
search for male:
human,male, adult,juvenile
search for human:
human, female, male,adult,juvenile

but if we are in the folder want we to see all the keywords? or only the different ones?
so only male , female, adult and juvenile? (human is on all images so not a keyword which is selective for that folder.)

The only thing i am certain of is i want to determine which program is hierachic leader. That’s the autowriter and all other are autoread but command writers (change has to be confirmed)
so a split in synchronise is preferable.
write and read are two different places in DAM hierachy.

25

Oxidant: There is another parallell discussion about this topic here:

What does the Adobe announcement on LR mean for DXO and PL - DxO PhotoLab - DxO Forums

Quite a lot in that tread revolves around the metadata issues.

26

“Ungulates are members of the diverse clade Ungulata which primarily consists of large mammals with hooves. These include odd-toed ungulates such as horses, rhinoceroses, and tapirs; and even-toed ungulates such as cattle, pigs, giraffes, camels, sheep, deer, and hippopotamuses.”
Come on - someone had to look it up!

2 Likes
27

I didn’t have to look it up for the initial post but I will admit to confirming my memory before hitting publish. :smiley:

28

The development of a Keywords management software should not be so complicated. All is question of respecting international standard to keep the compliance. The problem is that some editors (DXO is one of them) manage it with their personal recipe and here is where the trouble start.
Regarding what I discovered in this new PL5 version, I already use PL4 and once PL5 installed with the migration of my PL4 settings, I opened a picture, jpeg file keyworded with XNviewMP and DXO PL5 still show me the old keywords that was priorly written instead of the new one. Maybe because I already opened this file in PL4 and maybe because there is still a part of the database that is used to save the metadata keyword instead of the xmp or IPTC embeded file. I have no problem to read this updated file with Adobe product. For sure I tried do empty the cache but nothing change…Still messy …

Strong metadata feature was expected for so long time, I’m very cooled to upgrade to PL5 for nothing more than already exist in PL4

1 Like
29

Deneice: Have you tried to update metadata in Photolab 5 with File - Metadata - Read? That import use to populate the metadata-fields in PhotoLibrary. If the metadatalist is not updated to your liking try to reindex the folder you are working with or index all your folders by indexing the top-folder of your folder hierarchy.

Do you work with RAW or just JPEG? If you have a RAW and update it outside Photolab so it get it´s sidecar metadata updated it´s metadata should automatically be transferred if you export a JPEG from Photolab. With what tool did you create the JPEG? Maybe you have to look over your workflow?

I think the integration works fine for me now between my external metadata source Photo Mechanic and Photolab but I have to stick to a certain workflow and it comes with a few conditions. For example I don´t use “structured” / hierarchic keywords. For the moment I´m quite pleased despite I would like the update process to more automatic than it is.

30

I missed this option that can be automatic on other software…with XMP files but not with Jpeg. I tested it also with Exposure X7 and On1 2021 and it failed with them, strange behaviors even with forcing a read option.
So It runs well for DXO even with structured/hierachical keywords !!!
As said Stenis, is that enough to trust in PL5 for managing our metadata ? Let’s give a chance to DXO after waiting so much and complaining about the lack of IPTC support :wink:
Thanx Stenis for this help !

31

Deneice, it DOES NOT work with hierarchic keywords in an integration between Photo Mechanic and Photolab PhotoLibrary yet, but it’s fine with the plain list that Photolab builds when we index the images.

Another condition is ONE WAY workflows from the external DAM/metadata maintenance tool to Photolab OR metadata maintenance ONLY in Photolab PhotoLibrary. Only IPTC is supported and visible in Photolab really.

From what I have seen and as I use Photolab it exports the RAW-mastermetadata correctly to XMP with the above conditions when concerning keywords and that goes for JPEG, TIF and DNG-derivates too, for DNG that has been created with cameras supported by Photolab.

BUT there still doesn’t seem to be any support for import and export of keyword lists to and from Photolab and that has to be fixed so people can migrate to and from Photolab if they please. Just to migrate the RAW and the XMP-files is not enough. You have to be able to migrate keyword lists too .

Other things necessary in a short term is a possibility in References to configure whether Photolab or an external tool owns the metadata to prevent two way updates. If you use an external tool for your metadata it should be possible to inaktivate the update of IPTC in Photolab and inactivate those fields.

1 Like
32

Stenis, it seems to me that if one wants to use multiple applications for metadata it’s essential to avoid hiearchical keywords. In that case, changing title/headline, caption/description and adding a few more keywords shouldn’t do any harm.

Your thoughts on touching metadata in PhotoLab 5 if one avoids hierarchical keyword structure?