I was responding to @John7’s post at 5.2 What on earth to do with Metadata Conflict Resolution about conflict resolution when I set data in PL5 and no conflict was flagged [AS(OFF)].
Please note that I am a user, not a developer! My only access to the product is by creating scenarios and testing to see if I can empirically determine how the product works, i.e. I have no access to the code, or design documents and what is written here has either been determined by running such tests and/or by looking for solutions to problems reported by other or postulated by me.
This is another long read and the chances that any of what I suggest will be implemented is unlikely, but if you ask you probably won’t get but if you don’t ask you definitely won’t get!
So to re-iterate what I wrote to @John7
I have given an example using Photo Supreme Lite below in the hope that is might make the process a little less opaque! I “imported” the images into PL5 with no ‘Rating’ assigned and assigned a ‘Rating’ to the first photo in PS and that was detected by PL5 and the ‘S’ icon set on the thumbnail.
But then I started to write
OOPS! @sgospodarenko
I then set a rating in PL5 but there is no change in the thumbnail to indicate that the PL5 “copy” of the image metadata now differs from that held by the image! I am afraid that (now) I consider that to be wrong, when there is a mismatch detected by PL5 or created by PL5 then the ‘S’ icon should be set! Without such an indication it is not obvious which images have had their metadata changed in PL5 and should be flushed back to the image (if/when it is the user’s intention to keep the metadata in step).
In addition to flagging that the metadata is ‘out of sync’ (‘Requires Synchronising’) because there is a change in metadata externally or a change in metadata internally PL5 could/should indicate in which direction the mismatch exists.
There is potentially a situation where there may be multiple changes made externally and also made in PL5 and flagging up that situation would at least indicate to the user that a simple ‘Read metadata’ or ‘Write metadata’ command is going to potentially destroy data on one side or the other and make for an “interesting” display with icons indicating that synchronization is required in both directions!
Can I make my plea for a “merge” option yet again at this point!?
In the past I have suggested that a metadata “merge” is simply using existing code used for ‘Always Sync’, however, I finally woke from my “dream” and realised that with AS(ON) what was actually happening was that ‘Read from disk’ and ‘Write to disk’ were being executed sequentially, i.e. metadata read from image, followed by metadata written from PL5, followed by … etc. etc., which gives the impression of the metadata being “merged”.
This is “simply using the existing ‘Read from’ and ‘Write to’ ‘Files’/’Metadata’ commands which is not the same as taking the “excess” of metadata in PL5 and the “excess” of metadata in the image which then need to be amalgamated (and duplicates removed) to create a merged situation where no metadata is lost from either side!
Complexities exist with respect to “uni-valued” data such as ‘Rating’, ‘Rotation’, GPS etc. as to which data actually takes precedence over the other which is why AS(ON) is “arguably” the better strategy if the user wants to be using the metadata settings facilities in PL5. However, In spite of this complexity I still think it is an option well worth having!
Please note that the potential complexity of changes on both sides, in PL5 and in the image, will only occur iff (if and only if) this type of situation can actually occur with a users workflow, simpler or more controlled workflows will typically cause mismatches in one direction only (he says hopefully!) and can be ignored if the user chooses not to keep the metadata in step!
So I feel that the ‘Conflict Resolution’ in PL5 is not yet complete and that some or all of the following are, in my opinion, needed to finish this feature;
1. Flag All Metadata “Conflict” situations:-
Add a ‘Sync required’ icon when PL5 metadata changes result in a metadata mismatch with AS(OFF)
2. Flag the Direction of the Metadata Mismatch:-
Additions to ‘Sync required’ icons that indicate in which direction metadata needs to “flow” to resolve the ‘out of sync’ situation, i.e. a more stylish equivalent to ‘>’, ‘<’, ‘<>’.
3. Add a ‘Merge’ function:-
Add a ‘Merge’ function to the ‘Files’\’Metadata’ options that avoids the need to keep AS(ON) but results in the same (most of the time) equivalent to that feature and ensures that neither set of metadata is overwritten and lost.
With this function metadata is taken from the image and from PL5, combined, de-duplicated and written back to the both PL5 and the image.
The complication of “uni-valued” data such as ‘Rating’, ‘Rotation’, GPS etc needs to be addressed and probably the only way to handle that is via Timestamps. This still poses a problem because each metadata element may have been changed at different times and typically only a single timestamp will exist for all PL5 metadata and another for the image metadata!
4. Add a Metadata ‘Compare’ function:-
Add a ‘Metadata Compare’ function that can exercised at any time to detect when the ‘S’ situation has occurred but not detected and which results in both the ‘S’ icon being set and the direction that metadata must flow to resolve the situation.
5. Add ‘Preferences’ option equivalents to the DOP ‘Load’ and ‘Save’ options for ‘Metadata’:-
PL5 provides the ability to handle DOP data by selecting ‘Save settings in sidecar file (.dop) automatically’ and ‘Load settings from sidecar file (.dop) automatically’ in the ‘Preferences’.
Could this type of feature be added to PL5 to supply the same commands for handling metadata, i.e. ‘Read Metadata from Image’ and ‘Write Metadata to Image?
Arguably setting both of these new options equates to the current ‘Always Synchronize’ option.
However, some users want the metadata set (and updated) by their DAM to be the only source of data and would be able to use the ‘Auto Read Metadata option’ useful to keep the PL5 metadata inline with the DAM output to the image files. This function could have a ‘warning’ attached when the first such situation is detected in a session with an ‘All’ option to accept for the session etc. to prevent potential “corruption” to the PL5 data!?
6. Add an ‘Add from’ and ‘Add to’ feature to the current ‘Read from’ and ‘Write to’ ‘Metadata’ Commands:-
The 'merge function that I described above is effectively a combination of an ‘Add to’ command, i.e. ‘Add to database (from image)’ and ‘Add to image (from database)’ and potentially useful additions to the current ‘Read from’, and ‘Write to’ and (of course) the suggested the ‘Merge’ functions.
Hence, the ‘Files’/’Metadata’ commands would become
- Read from Image
- Write to Image
- Add from Image
- Add to Image
- Merge
- Compare
Would it then also be useful to add the ‘Add to’ commands to the automatic synchronisation options, i.e. item 5 above describes automatic ‘Read from’ and ‘Write to’, both potentially destructive operations where the ‘Add to’ is non-destructive. This would enable users to have the external metadata contents automatically added to or overwrite the PL6 database automatically with no additional intervention required.
The current ‘Always Synchronize’ ‘Preferences’ ‘Metadata Synchronization’ option would effectively become
Read from Image Automatically Add from Image Automatically
Write to Image Automatically Add to Image Automatically
Selecting ‘Read from Automatically’ and ‘Write to Automatically’ will provide the current ‘Always Synchronize’ option but using the ‘Add from Automatically’ and ‘Add to Automatically’ will provide a more secure option with respect to the preservation of metadata.
The caveat about uni-valued metadata still applies, ‘Rating’ etc. are not “additive” items whereas keywords arguably are!
7. Options for Users that are not interested in PL5 handling metadata:-
Are there users who would rather not know about metadata mismatches at all and would like an option that simply ignores what is going on, i.e. not only turn off all the read and/or write metadata options but also the detection and display of the ‘S’ icon?
Whether the ‘Compare’ function at item 4 above (which currently does not exist) should still be available if the user selects to suppress the display of the ‘S’ icon etc. is open to debate?!
8. PL5 versus DAM users:-
Many users of DxPL use the product for its RAW photo editing capabilities rather than any DAM functions.
They typically want the metadata set by their DAM to be taken into DxPL and then transferred unchanged to the exported file. The metadata will currently be captured by PL5 when the file is first ‘discovered’ and to protect the image data from any changes that PL5 might choose to make to that data AS(OFF) will have been selected (i.e. there will have been no selection made for the ‘Always Synchronize’ option).
However, if that image metadata is subsequently changed via the DAM any such changes will not be reflected in the PL5 database and not be included in any ‘Exports’.
Currently a one way import can only be accomplished by manually executing the ‘Metadata’/‘Read from image’ command on selected images.
If either the suggestions at 5 or (better in my opinion) 6 are implemented then there can be an automatic ‘Read of metadata’ from the Image whenever the DAM makes metadata changes to the Image (provided PL5 detects them).
The ‘Compare’ option suggested at 4 above would allow the user to check that the PL5 metadata is up to date before making an ‘Export’ and make a manual ‘Read from’ in the event that something untoward has occurred!
That should keep even the pickiest of users (me) happy!
9. Automatic Metadata Pass through:-
That leaves the thorny subject of PL5 applying its own ‘Reformatting’ rules to the metadata. This has gained more than a few column inches in the Forum from irate users who do not want the keywords that have come from their DAM re-structured in any way.
This effectively requires an option to control whether PL5 applies its inbuilt rules when formatting keywords either to write back to the image, if allowed or blocked by the judicious use of the options currently in place and/or those suggested above (if any are implemented), and when exporting the image.
If the suggestions above were adopted then automatic one way transfer would be possible but PL5 will still apply its reformatting rules as currently implemented, particularly when exporting the image.
A Global ‘No keyword re-structuring’ Option:-
So there needs to be an option somewhere that allows the user to override/inhibit the current PL5 (re-)structuring rules for the keyword metadata. Other packages do include options in the preferences to control how the package handles the metadata in line with the user’s wishes.
The obvious locations for such an option are ‘Globally’ or on ‘Export’ or both!?
A global ‘Keep Image Metadata format’ option would ensure that no restructuring of metadata takes place, what is currently in the ‘dc’ and ‘hr’ fields remains exactly as they are on import and (by implication) on export or on export via an explicit option
That still poses a problem if any metadata is added in PL5 to the imported metadata and what rules should be applied to metadata created in PL5. For those users that have made complaints about the re-formatting it is almost certain that no metadata will be added via PL5 and, if coupled with the automatic ‘Read from Image’ global option suggested in item 5 (and item 6) then any changes made externally via a DAM will be captured and updated and the output format will remain.
Control what metadata is read from an image:-
The metadata is currently read from the image when the image is first “discovered” by PL5 using the ‘Read from Image’ or equivalent code (a guess). Should that function be controlled or inhibited? The problem with either case is that the data will not be available for output unless…
Export of Metadata Direct from the Image (‘Include from’ option added to the Export Options):-
One additional feature that could be considered is that NO Metadata is actually taken into PL5 or it is taken in but simply ignored for export when a particular export option is selected.
This export option would identify the metadata source, i.e. an ‘Include from’ option which indicates that the metadata should be taken from PL5 (the default) or from the original image or a template or ….
This option would precede the current ‘Include’ option so that the items to be included are not controlled by the ‘Include from’ option but are applied regardless of the data source.
An actual Image protection Option:-
Effectively, if users wanted the metadata inserted by a DAM to be fully protected (from DxPL) there could be an ‘Image Write protection’ feature somewhere to prevent DxPL from writing metadata back to the photo as well as a metadata protection feature to ensure that no changes are made to the exported photo meta data “layout” as mentioned above.
I hope DXO listens to these discussions too. They also has to solve the issues with metadata migration between Photolab and other platforms using structured keywords. There has to be possibilities to import vocabularies and to export and import them as for example TAB-separated text in order to make maintenance of structured keywords more efficient for the people who needs that.