PL4.3.6.32 Ghosting Ratings for Same image in different directories (Win10)

DxO PhotoLab Windows topics
,
115

Before we go any further, I think it might be a good idea to see what Phil Harvey writes about IPTC…

IPTC section

In first place, IPTC metadata section was made for archiving purpose. It specifies many “about” (photographer, photo content, etc.) tags, which are ment to be populated by user.
At first, IPTC also allowed using ASCII/ANSI characters only, but now, Unicode/UTF-8 can be officially used as well. Of course, IPTC section has limitations:

1st limitation: Officially, tags defined in IPTC section are length limited. Some tags can only contain 3 characters (i.e. Iptc:Category), while other can contain several hundreds characters (most tags are limited to 32 characters, though).
That’s officially. However, in most cases, more than “allowed” characters can be (and many times are) written into IPTC section, and most software will show them all. But the fact remains: officially, limitation exist.

2nd limitation: Being a bit “old” standard, IPTC section doesn’t specify tags we wish to have and need today. For example, there’s no place, where you could save “rating” of your photo. The same is true for (photographed) people names, etc.

3rd limitation: IPTC metadata specification for IPTC section isn’t maintained anymore -instead, IPTC organisation decided to move IPTC metadata specification into XMP section.
This fact added some confusion among photographers… Anyway, with many software today, when entering “IPTC” metadata, data actually isn’t written into IPTC section only (or not at all) -by most “up to date” software, it is (additionally) saved into XMP section.

To sumarize: Above limitations are only valid for metadata inside IPTC section. That is, in case you use ExifTool command like:
exiftool -Iptc:City=Paris -Iptc:By-line="My name" ...
…values will be written into “old” IPTC section -because metadata section is specified. Ok, By-line tag only exist in Iptc, but you get the idea.

Conclusion: Old IPTC is dead… time to move on.

My conclusion - that being the case, PL5 has no obligation to achieve compatibility with IPTC-driven apps.

116

See my post quoting Phil Harvey.

Just to make sure I understand…

Delimited hierarchical keywords have no place in the dc:subject tag. Its sole purpose is to record every keyword attributed to an image but “hierarchy delimiters” are not normally allowed or even advised.

So, no, animals|mammals|bears|black bears is only allowed if it is intended to be one “keyword” and not a hierarchy. Some software does, out of good grace to legacy users, when reading such, allow this and translates it into a hierarchy. But I suspect this was intended for those who used to record hierarchy under IPTC tags.

Well behaved modern software can allow this but should then go on to “update” it by separating it out into the dc:subject tag and writing it correctly to the lr:hierarchicalSubject tag. IMO, no attempt should be made to update the IPTC tags.

I will comment on the table later, when we have had a chance to discuss this whole IPTC mess a bit more.

1 Like
117

Yes. But note the distinction between Legacy IPTC (‘Old IPTC’ as discussed by Phil) and ‘modern’ XMP-based IPTC. The fact that there are two flavors of IPTC causes considerable confusion when talking about ‘IPTC’. So for clarity (for DxO’s benefit) I would revise your statement to refer to Legacy IPTC/‘Old’ IPTC apps.

118

IPTC has been around for some time, but they adapt their standards every few years. Some interesting info about IPTC metadata use can be found here: Photo Metadata User Guide

IPTC focuses on press and telecommunications industries and pro use. DxO cannot ignore this field, if they think of DPL as a tool for professionals, even though longer standing pros already have a way to handle IPTC metadata by whatever means they use.

Non-pro users can probably get along nicely with keywords and a subset of what IPTC offers.

Nevertheless, DxO needs to get its MD handling right and I’m looking forward to the moment, when this is achieved - for Rating/Rank and all others.

1 Like
119

As noted in posts above, there are two ‘flavors’ of IPTC metadata, Legacy (IIM/‘Old’) and XMP-based IPTC Extension. IPTC provides more details in a section of the document you cited above: Photo Metadata User Guide
In particular:

After development over two decades IPTC Photo Metadata can be embedded in the following ways:

  • IPTC Core fields can be embedded in the IIM format and/or in the XMP format. A key challenge for metadata embedded in parallel in IIM and XMP is that the values are synchronised - this should be taken care of by the image management software. [Emphasis added]
  • IPTC Extension fields can be embedded only in XMP format.

IPTC would have done everyone a big favor by formally declaring IIM to be deprecated and superseded by the XMP-based IPTC Extension. They chose not to and dumped responsibility for keeping this (duplicate) information synchronized on makers of image management software. All of us, including DxO, would benefit from treating IIM as a read-only source of data and perhaps optionally synchronizing and writing IIM metadata only where requested by the user and needed for backwards compatibility. The XMP-based IPTC Extension metadata tags include all the information in IIM and much more, so no information would be lost by this treatment and things would be much simpler for all of us.

Rating is of course one of the standard XMP-based IPTC Extension fields.

1 Like
120

@jch2103 I believe that ACDsee and Zoner in particular are not using the old IPTC standard but are using the newer IPTC-XMP standard and they are both up-to-date versions and ACDSee in particular is being marketed as a general purpose data management system as well as a photo editor.

I am no expert on metadata handling, hence my post reaching out to @Joanna and you and anyone else who had something to say to help me understand this particular aspect of managing photos. I did not intend to “stir up the mud at the bottom of the pond to see what comes up…”

However, I am no novice when it comes to testing (just in tabulating the test results - it is extremely tedious - more so than reading it!) so I became concerned when 2 + 2 = 3 in the case of hierarchical keys in the ‘dc’ fields.

I first encountered this almost a year ago and it left me very confused but if you look at the table you will see that all the ‘hr’ savvy programs effectively “disappear” the “interloper” hierarchical key when one is encountered in the ‘dc’ Subject field, i.e. they all follow the pattern of replacing the “hr” keyword by its flat components in the ‘dc’ Subject field and the “hr” keyword finds its way into the ‘hr’ Hierarchical Subject fields in one form or another.

Unfortunately this leaves the ‘dc’ only software bereft of the very keyword that they created!! How can that be right was what I thought at the time and still think now! I’ve been a software engineer since 1964 and it does not add up, regardless of what the standards may say.

It is a clash of the Titans and one definitely comes off the loser, I feel - not particularly passionately but… Zoner I have used for a very long time on and off, off for some time when it went subscriber only but I got a good “family” version deal some years ago. ACDSee I have used since version 2.5 and the last licence was back in 2018 so I took an interesting offer the other day and now have an up-to-date version, it is the software I use to upload my photos from the memory card and does that job exactly the way that I want it done.

These are reasonably respected pieces of software (not as good as PL5 for “developing” photos) and they have their uses but not, it appears, when they “tangle” with the heavier weight metadata handlers like PL5 etc.

PS I “lied” about being a software engineer since 1964, that was when I started my degree course in Computing.

The following shows the original ACDSee information before adding a new keyword in PL5 and the same record after the key has been added.

image
image2303×460 186 KB

image
image2309×447 200 KB

PPS:- I do understand what is going on and why but the process of “normalizing” the data results in the original photo not visibly carrying the metadata that it started out with. If that program goes on to be processed by an ‘hr’ savvy program all will be well but if it goes on to be processed by another ‘dc’ only program then…!?

If PL5 implemented an option that allowed the “hr” data to be propagated to the ‘dc’ fields that would preserve the data from/for the ‘dc’ oriented programs but what would then happen when/if that data was processed by another ‘hr’ savvy program? I will run some more test, when time permits, processing the ‘dc’ program output from ‘hr’ savvy programs and then also adjust the output to see what would happen if the hybrid solution (‘dc’ = “hr” + and ‘hr’ = ‘hr’ +) is fed back into an ‘hr’ savvy program?

121

Hi,
only a short marker…we discussed all the things here Database Maintenance - #59 by George also the point with the Metadata User Guide.

So again a step backwards in the future :smiley:

have fun

122

@Guenterm I followed your link but then wound up on the database synchronisation topic and made the following “rash” post Synchronise PhotoLab - #23 by BHAYT.

123

Hi Bryan. I have done some research on ACDSee and found an interesting post in one of their forums…

There are two types of keywords in AcDsee

  1. AcDsee Keywords. Image->Properties->Keyword Tab. These are AcDsee keywords own keywords and you can have a Hierarchical keyword tree and a keyword list can be imported and exported. However, this information is only put into the AcDsee metadata and only AcDsee can see them (maybe a few other odd software packages) almost certainly the OS cannot see them. Additionally the Tools->Metadata->Embed AcDsee Metadata should be run to ensure the AcDsee metadata is written to the Image Metadata and not just stored in the database.
  2. IPTC Keywords. Image->Metadata._IPTC->Keywords These are where universally standard keywords are stored. Unfortunately the design standards do not allow a hierarchy (as far as I am aware) so keywords are stored in a flat structure with comma separation. The three little dots next to the keyword box allows you to edit the keywords and full keyword list used in all images. These keywords can be seen by most software including the OS
    Therefore if you export an image, and look at the image properties. The only keywords you will see will be the ones used in the Metadata PTC keywords. The AcDsee keywords are not visible. However if you used the Tools-Metadata-Embed Acdsee keyword option the keywords will be there just hidden and some software may be able to see it

As we already knew, IPTC doesn’t support hierarchical keywords and any such keywords are written to ACDSee tags. But, if we look at the XMP-acdsee tags available, we only find one called Keywords, with nothing explicitly defined for the handling of hierarchical tags.

Then you have this…

ACDSee metadata is proprietary and not viewable in exif (normally it is stored in the database however when you embed it you are writing the proprietary metadata to the files themselves). To get keywords and other acdsee metadata to be readable in exif you need to create a metadata preset to add them to iptc

So, unless you use ACDSee, you are not going to be able to see the “true” hierarchical keywords because they are only written to the ACDSee database, not the metadata. But, I guess, people have instituted the workaround of writing something like animals|mammals|bears|black bears into the IPTC ApplicationRecord Keywords tag because IPTC doesn’t support hierarchical keywords either.

However, this is, to use a technical term, a kludge of the first order and is the reason why we see inconsistencies on interchange of hierarchical keywords so defined in ACDSee.

In summary, ACDSee does not support hierarchical keywords outside of ACDSee, so you are never going to get a perfect solution for compatibility with PL5, or many other apps come to that.

As for Zoner, I couldn’t find anything on their site about hierarchical keywords - they seem to use the concept of categories instead. But, once again, they seem to be more concerned with IPTC, so true hierarchical keywords are not going to be possible without “The Kludge™”

I couldn’t even find help or a manual for ExifPro and it looks more like an open source hobby project that only handles hierarchies by using the same kludge.

124

@joanna I was not using ACDSee keywording I was writing straight into the ‘xmp’ ‘dc’ fields and that is where the various programs discovered that data and acted upon it, i.e. the programs see the hierarchical data in the ‘dc’ fields and use that data but those programs remove the hierarchical keyword from the ‘dc’ field completely.

Please remember that the test was about using the respective packages to set up the keywords using their own interface and then “feeding” the sidecar files that resulted from that into PL5 and documenting the outcome.

I have since started two more test cycles, one where the package adds a key to the ACDSee 'xmp; sidecar and another where the package imports the sidecar data and is then instructed to write/output or sync etc with the photo (sidecar).

The latter tests tend to track the original tests well but the former were something of a surprise and showed the existing keys being treated as a single keyword to which the data was added (you had suggested that might happen).

So we have the ‘dc’ data being treated in different ways depending on what is being done with/to it. Watch this space …

125

@BHAYT

This is just an FYI: IMatch comes with a number of ‘apps’, one of which is the Metadata Analyst. “This app analyzes the metadata embedded in image files, XMP sidecar files and the database itself and helps with identifying and fixing common metadata problems .” IMatch Help System

The app can be very helpful in tracking down metadata issues and problems. You could probably use it to ID inconsistent metadata handling in images from different programs.

126

@jch2103 thank you for that info, I managed to hang IMatch today!! It ius actually hung but it is stuck with “Updating” on the first two of the 4 files!

How do I run MDA!?

127

Go to the App Manager (in the lower right corner panel). It’s a tab in the same panel as Keywords, Map, Categories.

MDAI
MDAI653×889 114 KB

128

@jch2103 thank you I finally found it!!

129

Good. You may also want to take a look at this entry in the IMatch Help. The entire topic has a pretty good discussion of the whole metadata mess, and may help explain why IMatch was having a problem updating those files.

130

Bryan sent me a mail and wanted to drag me into this discussion and I must say I´m glad I stayed out of it so far.

Still I have a few things to say:

We refer to standards when it comes to both IPTC and hierarchical keyword and some call them just that and others call them “structured”, so not even there to begin with we have any consensus.

IPTC exists in it´s own right and also as an embedded IPTC-namespace in XMP as does the old EXIF. Despite XMP exists, quite a few softwares seem to use the old non XMP-version when using the data. Some systems fork data in their own way as does others too - hard to get a standard there I guess. In the other end like the filemanager in Windows (still in version 10) isn’t reading IPTC but EXIF and it also does so by using a set of conditions like if that field is empty read another element instead. So this isn’t straight forward at all.

In Photolab ratings refers to EXIF as for example PM Plus but there in the variable list that variable is called “Rating” and is placed under the label “Camera” which i guess is another word for EXIF. Should we be surprised if people get confused even before we have started to populate all these fields.

I understood Joanna´s opinion about IPTC - after referring to a trustworthy source - was to urge DXO not to bother too much about it because it´s obsolete. Still both a lot of software and the whole newsindustry relies on this old standard still embedded in XMP or not.

The reason I have tried to stay out is this is that I think I have to live in the present to get my own workflows to work properly when it comes to “my own” integration of PM Plus as a metadata editor and image library. I´m extremely deformed by 25 years as an IT-developer always looking to solve problems as quick as possible with what i have and not by what I wish to come in a new version of some software some where in a future.

We have sorted quite a few things in quite a few treads in this forum already but I don´t really see that the DXO-staff participates in these diskussions active enough to get something to happen in a near future, sad to say.

What I have found is working very well for me is to let PM Plus manage all aspects of the metadata and using PhotoLibrary only to search and display metadata when using Photolab.
Never ever update anything in Photolab. The other condision is never ever use hierarchical keywords in either PM Plus or Photolab, because the exchange between Photolab and PM Plus 6 does only work with plain unstructured keywords today.

If I index my images “topfolder” with images that has got their metadata applied in PM Plus 6 with plain unstructured keywords, the images will get indexed and a flat keywordlist with aggregated total numbers will be created automatically.

When I have developed these images in Photolab 5 and export them as JPEG-, TIFF- or DNG-files and open them in PM Plus 6 I will see that Photolab has not messed up anything at all. All the metadata I had in in my RAW-files before the export will get exported to the derivate-files and it´s easy to check that up in PM Plus. If you do like this Photolab and PM Plus is a really good match. If we respect that we won´t get any problems from what I have seen.

Photolab has to be further developed to use structured keywords if you are thinking of exchanging data with other tools. Photolab also needs to get an interface to be able to import tabseparated structures from both Lightroom or other systems.

From what I have read these questions around structured keywords seem to come mainly from Lightroom users but try to consider that there are as many vocabularies out there (lists with bot structured and flat keywords) as there are business branches. Why shall we focus entirelly on the one from Adobe Lightroom?

I started myself importing a Lightrrom list in PM Plus and started to work with it but found it extremely cumbersome both to use and to maintain. I maintained it in by exporting it from PM Plus and imported it after I had updated it.

The last week I have posted a couple of stories about the decline of the graphical industries in Sweden with totally 37 images. Before I really started to add XMP/IPTC-metadata to my blogg- and portfolio-images in 20+ elements/fields, not very many of my images were found by Google. After it´s a really big difference despite just using a flat keyword structure but keywords are only one of these 20+ elements right?

Isn´t that what this is about? To get results? If I had got paralyzed by my structured keywords not very much should have happened and remember, for Google it’s just words to index and search. Sorry to have escaped through the widest hole but I really had a lot of problems getting any efficiency out of using structured keywords.

The compatibility in consuming systems are very much more likely to occur when having a simple flat keyword list that just creates a comma separated list that every other system can relate to in the simplest way. I have seen a bit of how time consuming the task of building vocabularies can be in the historical heritage world in my country and that has put me off a lot to invest a lot in efforts like that. First they had very high spirits and formed a committe and some work groups than then the years went by… I´m not criticizing just saying it´s really hard to build and maintain structured vocabularies. It´s a never ending job. How many will have the endurance to row a boat like that ashore? Not me!

With that said I really like when I can see in Photolab what a search really hit - IPTC - the text in Caption - or something else.

Photolab olso need a few correction in the “Referece”-screens that makes it possible to decide whether to just use Photolab as a selfcontained “DAM” where all metadata is maintained or whether it shall just consume and use data owned by a third party software.

Sorry for the extensive text. I guess the people that use to count the words in the texts needs to get something to drink. :slight_smile:

131

I use PM+ with structured (hierarchical) keywords and they appear fine in PL. Admittedly, I only use structured keywords in a very specific case where I classify my fish pictures with common name and scientific name. Other than that case I find it all works just fine.

One thing I do NOT do is specify each path separately so I only have ONE hierarchical keyword. This is what PM+ does by default.

132

The real problems occur if you by your own will or by mistake save metadata in Photolab to the XMP PM Plus is supposed to own and read because the structures from Photolab will get corrupted. You can’t use bidirectional flows and there is no fool proof way today that prevents oeople from doing so.

This thread is from the beginning about gosting ratings and we have a problem with us using software developed for photo journalists in the media world using IPTC AND EXIF simultaneously while the software only supports IPTC fully at the best.

In my case my workflow is perfectly fine as I use my tools today BUT there are three or four elements in Photolabs IPTC interface under the label “Image …” that doesn’t show at all despite I have filled “the picture taken”-elements in PM. Apparently Photolab uses other element for me unknown and frankly I don’t really care since the dataroundtrip is fine from the PM-horizon.

It’s my opinion Photolab isn’t really ready for structured keywords yet and the reason I have described above. There has to be a possibility to migrate not just the keywords you have used so far in the software you are migrating from but also importing the corresponding vocabulary you have used like in PM but maybee there is a posdibility to just place a tab separated file with a standard name on the right place on the disk as a work around.

I can also think of scenarios where people want to use different vocabularies for different databases or catalogs like you can in PM if you like but maybee that’s too much to ask for in an entry level library as PhotoLibrary. Still these are demands people will come up with in this DAM-like world. If a company like DXO now tries to reach them a hand they will expect at least to get an arm. So I guess the people that raised doubts when DXO went into this DAM-like world because they suspected this will slow the needed development of the image development tools might wake up finding they were right.

With that said I’m extremely pleased now to see that the metadata I add to my images today gets indexed by Google often in just one day. The first reason to put in the metadata work is to get a better order in my own image library but it’s a very important spin off that it really helps making my images more visible than before. To add metadata to the images is pretty time consuming activity and as retired I have the time but I really wonder if the people that could benefit the most from adding metadata really have the time.

133

A few thoughts/comments:

  • Everyone’s got different needs re metadata or lack thereof (and as @KeithRJ points out above, some of us even have specific needs for particular subjects and not for other subjects!). Some people are fully content with just folder-based organization while others pay detailed attention to things like highly structured keywords, etc. That’s certainly fine (although a few folks seem to think that because they have certain perspectives/needs that everyone else should have the same…).
  • To the extent possible, DxO should accommodate and not interfere with different needs. At the same time, DxO should avoid doing harm. They’ve taken some steps in that direction with the Metadata synchronization option in Preferences, but some more work by DxO is needed, especially for keywords and other metadata that’s copied (or not copied) to output files. The whole subject of metadata is a complex mess, and it’s not a surprise that it’s taking DxO some time to grapple with it.
  • I’m in the ‘I want to manage my own metadata and do not want PL to interfere with it’ camp. As long as DxO doesn’t interfere with my management, I’m happy with whatever solutions they provide users who do want to use DxO’s metadata management. (I may still have constructive criticisms of what they’re doing, to help avoid issues for users.)
  • A technical aside: IPTC has two basic ‘flavors’, the old/obsolete IIM flavor and the new XMP-based flavor. It’s confusing enough that these both exist (and that IPTC hasn’t deprecated the old version), but more confusing when users here don’t distinguish between them. (I’m pleased to see that DxO has fixed some IPTC-IIM issues for exports in the next release.)
134

I think the real problem with metadata is there are very few “hard and fast” rules and there are too many options for storing metadata differently. Not all apps support all these different ways of storing metadata. What one app does may be correct but may also not be what another expects and that is where the issue lies.

What you need to do is find what works for you and stick to that and hopefully you won’t have too many issues.

I used Lightroom from version 1 to 6 until it went subscription and used it’s Catalog system very effectively but not too complicated either. When I moved to PL I was happy NOT to rely on the database and that is what I still do. For my fish keywords I now use PM+ and as long as these keywords flow through PL to my final jpg then I am satisfied.