Just don’t try searching for this keyword OR that keyword. Impossible 
I don´t know what you are aiming at with that statement.
Sure there is not possible to just select a keyword from i list in the main search field and that´s a pity and entering a keyword will not instantly perform a search BUT if I type a keyword in the same field I get a list of proposals: to use the hits in ITPC, the keywords, the folder name or the filename. For every option I get the number of hits - if there are any. If I select one that has hits the search will get performed. Not the fastest and most efficient way to do it but it sure works and as a bonus you get to know where the hits were aggregated from which I sometimes find useful.
It´s not at all useless but just a bit cumbersome.
Sorry if I have misunderstood or more likely completely ‘lost the plot’…in the above discussion…
If in PL the Sync setting is unticked, is any metadata (including previously assigned LightRoom keywords) read by PL but when one adds keywords in PL they are assigned to the .xmp file of the image but as unsync’ed LR will not read them?
FWIW I have ceased to use LR and am still seeking a DAM solution…and would potentially be content for now to use the DAM function within PL. But if in future I find a good separate DAM (that would utilise the .xmp structure) will that be able to ‘see’ and read the PL generated keywords etc???
TIA for any clarity for me to understand a good/best course of action 
@BoxBrownie There is “no such thing as synchronised” metadata! By that I mean that synchronisation only happens by virtue of the settings of the respective programs that you might be using.
Regardless of the AS setting (AS(AON) or AS(OFF)) the initial “discovery” of a folder triggers the PL5 “import” process (i.e. there is no explicit “import” process) and PL5 executes either the ‘File’/‘Metadata’/‘Read from Image’ command or the equivalent and effectively imports the image and metadata into the DxPL “fold”, metadata included!
So with AS(ON) PL5 is then “watching” for any potential changes in external metadata and, it will execute the ‘File’/‘Metadata’/‘Read from Image’ command whenever such a change is detected and automatically write out any changes made in PL5. For RAW files all such changes will be placed in, or added to, an xmp sidecar file. For JPG, TIFF and DNG the changes will be made directly to the embedded xmp data.
With AS(OFF) no such changes will be imported or exported by PL5 after that initial importation automatically!
With AS(OFF) the process is entirely under the control of the users with the ‘File’/‘Metadata’/‘Read from Image’ and ‘File’/‘Metadata’/‘Write to Image’ commands. These commands can be used at any time with AS(OFF) or with AS(ON) and they are exactly what they state, the data will be read from or written to the image and it will overwrite any data already in place in either that database or the image, depending on the ‘Read …’ or ‘Write …’.
Whether the external programs pick up these changes depends upon the rules defined for those programs and their ability to detect that such changes have occurred automatically.
Even with AS(OFF) PL5 continues with the detection process and sets the ‘S’ icon if any changes are detected that cause a “conflict”. This topic is to request that the icon is set for changes made in PL5 that have not been written to the image and also to indicate the directions of the mismatch > < <>, equivalent to the arrows in LR which are not available with Win 10 PL5.
The last issue that needs some consideration that has been discussed at length in a number of posts is not only the communication between programs, essentially via a “post box”, i.e. the image, but also whether is is written in a “language” the recipient can “understand”, i.e, the format of the metadata. It is with the “language” that some concerns have been expressed, even for simple keywords “dog”, “cat” etc. PL5 will set the ‘hr’ keywords to “dog”, “cat” etc. as well as the ‘dc’ keywords and this conflicts (according to those who know way more about it than I) with guidelines that suggest that ‘hr’ data should be restricted to hierarchical keywords e.g. animal|mammal|bear|black bear!
I hope that helps.
Of course there is but not necessarily when Photolab is involved. When I was working with FotoWare DAM we built a two way hot integration between the City Museum of Stockholms old photo and historical arifact databases (two different SQL-Server databases) and Fotoware. As soon as something was updated in one of the SQL-systems the changes was detected by Fotowares folder watcher and propagated to Fotowares metadatadatabase by that systems Index Manager and after a second the changes were updated in the museums web. The other way around the SQL-database were updated when something got updated in Fotoware’s managenent and maintenance application Fotostation.
The trick is to use transactions and that is a core function in SQL-systems. Many companies using IBM:s AS 400 / I-series systems (very common in bigger companies) use replication between these systems and Micrisoft or Oracle SQL-databases. When I worked in the Nordic countries at that time biggest IT-product distributor we used Vision Solutions replication software Symbiator and I personally helped IBM and Vision to solve the problems they had with our nordic characters ÅÄÖ that they could not handle because it was not part of the character set tgey supported at that time. That three dimensional business data was then analyzed by a software called PowerPlay.
So of course there are something called automatically sunchronised metadata and it can achived even between for example Photo Mechanic and Photoware. It will not be transactional but it xan definitelly be “loosely coupled”.
@Stenis the truth is that PL5 typically detects and displays almost immediately after a change is made to an image in another program with AS(ON)! Better than even Photo Mechanic where there tends to be a delay and sometimes either an ‘F5’ or clicking on any photo in the group prompts PM to check and find the update. From memory I believe that it (PM) will find the update after a short delay automatically.
However, I was conducting a test to check my comments using test data set up for PL6 Beta testing from my Main machine and PM refused to find an update to a RAW that had created a new sidecar file. No amount of ‘F5’ etc. would get PM to detect the changes. So I started to prepare to raise a fault against PL5 and ran another program and discovered that PL5 had made the change correctly but PM had failed to find it until part way through gathering the snapshots, PM woke up and found the new sidecar file across the LAN!
So I re-iterate my point about configuring the products, about the fact that some products can detect automatically (but not necessarily refresh the display), while others may need to be prompted to look before any change will be detected and/or shown. Plus even the good ones can have the occasional “hiccup”.
The detection and display has always been fast in most tests that I have run with my concern over a couple of instances when PL5 seemed to miss the change!? My main concern has been about the creation of unnecessary ‘hr’ data when handling simple keywords but most of the other "heavyweights’ do the same, excluding Photo Mechanic!
This topic was created because I feel that DxO has stopped too early in the development and I want the three icons that Lightroom has to signify that a mismatch has occurred and in what direction (or both) e.g. use the arrow pointers on the ‘S’ icon (is this feature already available on the Mac implementation?) .
Plus the two-way synchronisation options, my ‘Merge’ and 'Add to ’ features.
Capture One talks about an Auto-Sync feature and describes it as a two-way synchronisation, it provides this feature as a global option and as a manual option available, at any time. I have yet to convince myself of the exact nature of that C1 “two way synchronisation” feature but is sounds like what I want added to PL5.
Given people have “poured scorn” on why I would ask for such a feature you can understand my interest in exactly what C1 provides.
Thanks for your input Bryan. Agree fully to your statemant that DXO has to take the data interchange to a higher level and that they have to put in some more efforts to get there.
The C1 two way sync is there I think but it might cost performance to use.
I have tried to convince Cameralabs to develop a folder watch system like FotoWare has. In that case it´s more of a rule than an exception images are updated outside the DAM. So they monitor the filesystem logs in order to find new and updated images to process. In their case it´s not just about metadata but also about updating and storing different derivates of the images but for me it would be fine if they updated their previews to reflect the changes I make in Photolab.
So even PM Plus needs to be developed to achive what I like to achive.
C1 actually advise users to not use that function under certain circumstances but to use the ‘Preferences’ ‘Load’ option and then execute the ‘Sync’ manually to keep I/Os under control!
Did you mean Camerabits rather than Cameralabs in the post? As you will see below I got mixed results from Photo Mechanic with ExifPro but Adobe Bridge seems to have got the detection mechanism working well!?
I have criticised DxO for not being able to detect changes made by ExifPro to JPGs (it works fine for RAW because ExifPro creates a new sidecar file etc.) where other software does.
When I first started testing in early 2021 I was using ExifPro, Bridge (different configuration to the way I use it now), Zoner and ACDSee. PL5 never detected any change because ExifPro doesn’t change the ‘Date modified’ of the JPG yet the other programs seemed to detect the changes without any problem (not quite as the following tests show with the latest versions of those programs).
So I decided to repeat the tests with the latest versions of the programs, amending ‘Rating’ and ‘keywords’ arbitrarily and the results were a little “mixed” and disappointing!
ACDSee detected nothing and could not be induced to check the file for any changes!
Zoner has actually improved but misses the ‘Rating’ changes but detects the keyword changes and could not be forced to re-read the image data?!
Photo Mechanic seems to have the same issues as Zoner, detecting ExifPro keyword changes but not ‘Rating’ changes!?
FastRawViewer is only useful for ‘Ratings’ and never automatically updates but found the ‘Rating’ change on request.
Throughout the tests the ‘Date Modified’ field of the image did not change, ExifPro “sneaks” the limited changes it can make “under the radar”!
Bridge detected ExifPro changes automatically mostly displaying immediately, occasionally requiring a prompt.
Across the LAN there was no automatic detection of ExifPro changes and only double clicking on the image to launch the image in another image browser caused the data to be updated but Bridge changes on one machine were immediately detected by Bridge changes on the other (but then Bridge changes the ‘Date modified’ so there shouldn’t be a problem).
PL5 tracks Bridge changes instantly, Zoner a fraction behind and ACDSee a fraction behind that but while an ‘F5’ in ACDSee causes the entire directory of thumbnails to be refreshed it still doesn’t bother to read the data from the image changed by ExifPro (it is possible there is a setting somewhere in the huge list of setting that might cause it to actually re-read the record but …)
PM requires a click on any thumbnail for the thumbnails to be refreshed, when I am in a hurry (always when testing!)
@Stenis will you please do me a favour, in an earlier post you indicated that you frequently set 20 elements of metadata, I would very much like a RAW image that you are prepared to have published in a post snapshot, complete with (anonymised) data in those 20 metadata fields so that I can use that image in some tests, i.e. the image with the metadata embedded in the RAW and also a sidecar file with the same data present would be ideal (if your internet bandwidth is up to the task, mine simply could not cope!). Thank you.
It´s Camera Bits and not Camera Labs
Was it this article you referred to Bryan where they advise people not to use the two way sync of performance reasons?
Image Alchemist | Sync Metadata Between Photo Mechanic And Capture One • Image Alchemist
By the way, thanks a lot for your empiric testing efforts. Intresting!
Bryan, I have no time to anonymise data just now and my data is anyway published widely on the net already so I am happy to send it to me if you just kan mail me your address.
Isn´t the problem with the loss of the “ratings” explained by the fact that Ratings is EXIF and the rest IPTC? Everybody is talking about XMP but not all are using the namespaces of EXIF and IPTC in XMP, instead they are using the older pre XMP way of reading and writing IPTC. At least in Photo Machanic it´s an active action to write IPTC to both places.
… and it´s not a surprice Photolab seems to interact best with Adobe Bridge and Lightroom. DXO have spent quite a few years to polish these hot links but have just started to be aware of that a lot of users are using other tools than the ones Adobe manufactures.
1 Like
@Stenis I have sent you a message with my email address and thanks for access to an image.
While that is certainly possible the strange thing is that when the keyword changes the products find both the ‘keyword’ and the ‘Rating’. It is possible that they are looking at another file attribute, such as size and if that doesn’t change the image is ignored!?
Sadly, while I knew what was appropriate on the old platforms I worked on and what interrupts and file attributes could be used and interrogated etc. I do not know how the programs are structured to “detect” a potential change without “buzzing” which typically shows as excess CPU usage?
As a consequence I do not know what attributes etc. Bridge is using versus what Zoner and PM are using versus what PL5 is using. If PL5 was coded like Zoner or PM it would be better than it currently is, if it could be codded as Bridge is coded then it would be better still!
The PL5 “report card” still reads “needs improvement” and will continue to do so until they decide to explain in “words of one syllable” what the current mechanism looks like and what alternatives they consider might be even better or why whatever Bridge is doing is actually considered “bad practice”!
The cpu table from Process Lasso is a little scary. ExifPro has consumed the largest proportion of time followed by Creative cloud and I am not sure whether CC is actually consuming resource on behalf of Bridge or …
Some hours later during which time the machine has been unused!
@Stenis thank you for the images. I now have RAWs with no embedded xmp, xmp sidecars with a lot of data, some “ancient” DOPs and I have forced the sidecar data back into the RAW so I can create any combination that I want for testing, when I can be bothered - not testing fatigue just forum writing fatigue!
Once again, thanks for some real data which I will “mess” about with as soon as I feel up to it, hopefully later today!!
You are welcome! Hope you will see some useful patterns.
Stenis
Providing a more complete Conflict Management function:-
I had started to put this in a separate topic but I am not sure I posted it correctly and it really belongs here! I believe that it is as straightforward as I suggest below to provide a more complete Conflict Management function, I hope!?
@sgospodarenko & @alex in this topic I complained about PL5 ignoring changes made to metadata which could not be written back to the image because AS(OFF) and I also requested that the direction of the mismatch could be signalled by DxPL as it is with LR.
I originally suggested >, < or <> to indicate where the mismatch was coming from, i.e. PL5, the image or both! In the meantime I have changed that original suggestion to incorporate the direction indicators as part of the in the ‘S’, in such a case the current double-headed ‘S’ icon would signal the “both” situation.
Recent posts from DxO seem to suggest that what has currently been developed is all the development proposed for Conflict Resolution!
While trying to reproduce a fault I had encountered (as in the above identified post) I started thinking about the database field used to “persist” the ‘out of sync’ status (the ‘IsOutOfSync’ field in the ‘Metadatas’ structure) and decided that far from being a complicated development to extend to the full set of cases it is actually a very straightforward task, unless I have got something very wrong, so here goes!
-
PL5 currently detects a mismatch situation when the metadata in an image changes but AS(OFF) means that the change cannot be captured and entered in PL5 so the ‘IsOutOfSync’ flag is set to ‘1’ and that controls the presence or absence of the ‘S’ icon from then on.
-
Please extend the ‘IsOutOfSync’ situation to all cases where the data appears to have been changed but cannot be read or written to the image or the database for whatever reason, AS(OFF) being the main one but other situations may well arise with AS(ON)! Is there really any good reason to restrict the setting of the 'S’ynchronize icon to only situations detected with AS(OFF)!?
-
Assign an ‘IsOutOfSync’ value of 1 or 2 in PL5 whenever an out of sync situation is detected,
1 ‘IsOutOfSync’ = 1 when external metadata does not match PL5 and cannot be rectified by reading that data and adding to the database (the only situation which is currently recognised and flagged),
2 ‘IsOutOfSync’ = 2 when the PL5 data does not match the external metadata and cannot be rectified by writing the data to the image. A new situation that would answer my “bug” report in PL5.2.0.4730 Conflict Resolution only detects externally generated conflicts -
Check the program ‘IsOutOfSync’ value with the database ‘IsOutOfSync’ value,
1 If the database ‘IsOutOfSync’ value = 3 simply ignore the situation because it is already marked
2 If the database 'IsOutOfSync’value = the ‘IsOutOfSync’ then simply ignore the “new” value because it is already marked.
3 ELSE Simply add the current value of ‘IsOutOfSync’ to the database value of ‘IsOutOfSync’ and set the icon accordingly and set the database ‘IsOutOfSync’ to the new value, i.e. = 1 (image > PL5) , = 2 (PL5 > Image), = 3 both have been encountered!
Unless I have made a mistake it is basically as “complicated” as that!? I have looked at different scenarios, e.g. multiple changes to image metadata while PL5 is shut down followed by restarting PL5 and immediately changing metadata in PL5 but before such data can be changed, PL5 should have detected the earlier changes to the image and marked the images with an ‘S’ before allowing the user to access the directory.
To ‘Compare’ or not to ‘Compare’:-
To avoid potential metadata loss it is imperative that PL5 detects such changes to the image metadata in all cases. My suggestions for a ‘Compare’ function, i.e. a user initiated request to compare the metadata in PL5 with that held in the image file(s) looking for any mismatch may not be as easy to do as the icon handling I have suggested above.
I believe that PL5 currently detects potential mismatches/changes to metadata solely on the basis of timestamp comparisons. Whether any actual data comparisons ever takes place in the current version of PL5 is something I cannot test. If no such comparison code exists then there could be a lot more new work involved to provide a full blown compare function!
But, if PL5 essentially manages automatic updates by recognising mismatches in timestamps then what I am asking for is to simply “trigger” that existing process manually rather than automatically!
Having such a facility would then help close the “window” of uncertainty that follows situations I have encountered during testing, when PL5 has either not detected the change to image metadata or has ignored any changes that might have been detected!
Identifying the different mismatch scenarios:-
I like the idea of using the arrow head pointers on the ‘S’ icon to indicate the “direction” of the mismatch, they are a little small for tired old eyes like mine so it might be useful to change the icon colour to identify the three states of 1 (external image data ahead of database), 2 (database ahead of image data) and 3 (both situations identified!), possibly in addition to using 3 versions of the ‘S’ icon!
If scenario 3 is detected then some loss of metadata is going to occur, with the current implementation this situation and the risk will simply not be recognised and the user will not be informed. I consider that the whole point of using a computer product is to automatically detect such situations whenever possible and help the user as much as possible to make the “best” choice, even though it will be a compromise in this case!
Detecting and reporting Scenario 2 should focus the user to “flush” the changes made in PL5 to the image as soon as possible (or reverse the changes made), thereby helping to eliminate the occurrence of Scenario 3 because at that point the loss of some metadata is “inevitable”!
Handling the “Pop-Up” for resolving 'S’ynchronisation conflicts:-
Currently the pop-up when clicking on the ‘S’ icon highlights the ‘Read from Image’ option, if the above is implemented then the following should take place;
-
For scenario 1 highlight the ‘Read from Image’ option.
-
For scenario 2 highlight the ‘Write to Image’ option.
-
For scenario 3 highlight neither option but when an option is selected by the user emphasise the potential loss of data in the database or in the image, depending on the user option chosen, and request confirmation before proceeding with the chosen option.
Again, editable metadata can be in the following carriers
- image file
- .xmp sidecar
- DPL database
- .dop sidecar
Proposal
- Any difference should be signaled
- Upon clicking on the signal, a window showing the difference(s) should open
- The user should chose, which source should be copied to the other carriers
Note: Mark the image file carrier as read-only as long as it cannot be written to.
The proposed functionality goes well beyond anything I’ve seen in other metadata aware applications and could give DxO a metadata feature that were at the height of DPL’s image tools. I’d still require better database maintenance functionality though.
I suspect the DxO’s problem is that they are relying on the database being the main points of information.
To me the original file should never be written to. When you first click on a new image DxO should write a new DOP file and a new XMP file. Only then should the database read the DOP file and XMP file so that DxO can do its own housekeeping. The editing done to the image should then update it DOP file and any editing to the exif data to update that XMP file. This should then update the database. This should hopefully cause less corruption of the database and people having to delete it and no need for synchronisation on and off. Obviously the off causes corruption.
Of course, if the file has already been edited in a different programme, then it should read the existing XMP file and create its DOP file.
1 Like
Why not? Both Nikon’s and Canon’s own software write both editing data and metadata to their RAW files.
I have been writing keywords, descriptions, ratings and Finder Tags to original RAW files for years and never, let me repeat that, never had the slightest hint of corruption.
However, if I had relied solely on the database and not had DOP files in PL, I would be in right state, because of the number of times moving files (often necessary when reorganising) is tedious in PL but, if I do it in Finder, it creates an almighty nightmare.
As others have said, if you are not happy with writing to RAW files -
- image edits should only ever live in DOP files
- metadata should only ever live in XMP files.
The database should only be used for caching image and metadata changes whilst the app is open and should be scrapped every time it is closed.
Of course, there is still a need for a separate database for other things, but not for the two mentioned here.
But it should never copy anything from the XMP into the DOP, as it presently does.
2 Likes
The writing back to the original file is probably just one of my fads. I’ve always felt that the original file should be pristine. I know that you like to write your data back to the original file which always makes me cringe, thinking that if it ever gets corrupt that’s a valuable file lost. As I say one of my fads.
I agree image edits should only be in the DOP file and the metadata in the XMP file. Photolab should only use the database for its own housekeeping as I have already said. And I 100% agree with you. No XMP data should be copied to the DOP file.
Possibly, as somebody has already said. The database to delete itself at the end of session and make a new one at the start of the next session. Quite a bit of programming though.
1 Like
You are at odds with others who want the original RAW file to be amended, inline with the JPG, TIFF, DNG files that are changed as a matter of course by most software! Personally I agree but don’t really care either way.
I don’t understand this!? The DOP will be written as and when there are any changes worthy of note made to the image editing and/or the metadata (max delay in writing DOP of 20 seconds) and the new XMP file is incorrect if one already exists, as well it might!
Typically PL5 has access to its own database first and the same applies to the DOP but it must potentially “fight” for access with the image files! What I was suggesting in my post above is to use the extended Conflict Resolution model I am proposing above to flag any situation where the image cannot be updated with the ‘S’ icon!
I am lost again, how many genuine “corruptions” are users experiencing or are you referring to mismatches between the PL5 view (database and DOP) and the image data. Although other packages seek to minimise the risk of a mismatch there is always that risk! Without the extension to the conflict management then the product has a serious weakness.
If your proposal is to change the order of updates so that PL5 will never update the database if the write to the image cannot be successful then AS(ON) would be the way to go with that scheme because otherwise you are going to have updates stuck in “limbo” until the user finally executes a ‘Write metadata to disk’ which might never come (particularly if the user forgets the updates that need to be flushed back to the image!).
The order of updates does not need fixing what needs fixing is the following;
-
A full flagging of ‘out of sync’ situations however/whenever they arise, information is power but ignorance is just chaos, in this case.
-
Consider a better scheme for comparison between image metadata and PL5 metadata (I have a much bigger response underway for you @platypus)
-
Encourage more people to use AS(ON) by not playing with the metadata
1 Do not add hierarchical keywords to simple keywords ever!
2 Allow users to specify no adjustment to be made to the metadata (gets complicated when users start to use PL5 even for minor adjustments)
3 Reconsider the way PL5 adds keywords, I am no metadata expert but consider the changes made in PL5.2.0 were a big step in the wrong direction @Marie @Joanna.
4 Do not make changes to metadata handling that leave those that used the previous version “up the creek without a paddle”. Actually start asking users, beware reacting to posts without asking for a wider consensus @sgospodarenko and provide an option to continue using the previous method!
Looking forward to it. Please keep your post short and/or make a summary of what you propose.
1 Like
I think I already said in my first post that if the XMP file already exists. It should read from that.
I personally have not had a corrupt database, but then I do not add keywords, but it seems that many people delete the database. Whether it is through past corruptions or just overcautious. I don’t know.
1 Like