i use it on a network drive yes but not with multiuser
but C1 and Affinity will not close the app if it is still writing to the file
with PL5 it seems i can do other other jobs on the file or close the app and corrupt it
still have to find out how it gets corrupt but had it 3x now
so now i just wait longer before i do another action like opening it in another application after applying changes in PL5 or i look in the finder to see if its still being written or not. (when i say opening in other apps i mean opening it in affinity or other software from PL5 )
What are you doing to the file in PL5? Are you working on. a TIFF file? Are you exporting to a TIFF file?
Patrik (@trikke) I use Win10 and normally only JPG and RAW (RW2 & ORF).
Joanna’s (@Joanna) description of the way that PL5 (versus PL4) works is correct although the ‘Ratings’ and other metadata are stored in the DOP for the [M]aster and any Virtual Copy (VC) edits, PL5 now uses the values stored in the xmp (embedded - JPG, TIFF and DNG and RAW from certain other applications) and sidecar (most other apps and PL5 itself) for the [M]aster image (frequently the only image/copy of interest to a user.
Hence, the updates which are now being made to the TIFF.
Please note that these metadata updates are the result of the ‘Preferences’ ‘Sync’ option setting and/or the use of the ‘Metadata’ Read and write commands, without which no ‘Rating’ data etc. will “escape” from the confines of PL5 - except by being propagated to any exported file.
The database is the primary source of data for processing in PL5 but if lost it can typically be recreated from external files as follows:-
For the [M]aster (or only photo entry)
- Basic photo data from the image file.
- Additional metadata including ‘Rotation’, ‘Rating’ etc. from the xmp data (embedded or sidecar)
- Edits etc. from the DOP
- DxPL unique metadata, i.e. the Tag from the DOP (except there is currently a bug with that item)
For any Virtual Copies:-
- Basic photo data from the image file.
- Additional metadata from the DOP
- Edits pertinent to the VC from the DOP
- DxPL unique metadata from the DOP (haven’t tested if the Tag bug also applies to VCs!!??)
The difference between PL4 and PL5 is that with PL4 item 2 from the list for the [M]aster “copy” was actually a combination of item 2 from both lists but PL4 did not write metadata back (the immutability constraint).
I haven’t investigated which took precedence over the other, i.e. DOP versus xmp for e.g. ‘Rating’ but users who have moved to PL5 didn’t necessarily appreciate the way that the changes would affect their workflow, particularly if all ‘Rating’ was actually done in PL4 when the DOP was the “be all and end all” of what needed to be preserved with the photo but has now been replaced with the need to preserve any external xmp data and to make sure that the metadata required was actually externalized from PL5
I made a response to your post but failed to ask:-
- Under what release did the corruptions occur?
- What was your workflow between PL? and C1 and Affinity?
- Is your current work flow the same or going to remain the same?
- What was the “nature” of the corruption?
So I have an over simplistic test as follows:-
I exported an RW2 image with ratings as a TIFF and it was added to the same directory as the RAW, opened the file in Affinity and changed edited the file (I own Affinity but do not use it enough to be familiar with how to achieve good edits) and saved it and the changes were immediately picked up be PL5 (and FastStone image viewer which was also monitoring the directory).
Changed the ‘Rating’ from 3* to 2* and returned immediately to Affinity and made some outlandish changes and saved and these were immediately reflected in PL5 and FSIV, I am running with 3 screens so I can monitor all three apps simultaneously.
There is no obvious fault and the saves and re-reads are all behaving themselves in this limited test.
I will make changes to the edit in PL5 and watch Affinity and also put the photos on my NAS (DS220J) and repeat the tests. Please note none of the apps were closed during this test (which differs from your scenario where you are being careful not too cause any clash but are experiencing problems!) I will also try closing the apps and see if anything results from that!
@Joanna export tiff from raw and then working on the tiff with other applications never simultaniously
@BHAYT ok thx, at the moment doing some edit work i will test it more in depth later to see where it exactly is going wrong. usually i end up with a tiff file that is only partially visible like the writes where corrupted halfway the file. Im working on mac not on win.
- Under what release did the corruptions occur? => latest version
- What was your workflow between PL? and C1 and Affinity? export in tiff from pl5 to affinity safe in affinity change rating close pl5 open pl5 or open c1 broken tiff. (i think but have to verify to be 100% sure)
- Is your current work flow the same or going to remain the same? i think this is the way to go export a raw in pl5 after changes to tiff to edit in affinity or an other app. dont see how else i can do it. worked like this for years in C1
- What was the “nature” of the corruption? non readable tiff or tiff only half readable
Patrik (@trikke) sorry about my rather “terse” questions, when I read back your answers it was a bit like an inquisition, it wasn’t intended to be like that. Please keep me informed, I have PL5 and Affinity installed on my main machine so I will try to replicate your workflow when I have time later on today.
However, I am confused by the order of processing between Affinity and PL5. I suspected that you exported the TIFF from PL5 and thought that you would do the ‘Rating’ there because from my limited experience of Affinity handling ‘Rating’ doesn’t look like something I would try to do there!?
What exactly is going on here (sorry for my not understanding) “export in tiff from pl5 to affinity safe in affinity change rating close pl5 open pl5”, I presume that you
- Export the TIFF from PL5 to … whatever program can read and process the TIFF, I presume.
- Safe in Affinity means what exactly, e.g. Open in Affinity?
- Change Rating - I presume that is a PL5 action because it does not appear to be a strength of Affinity but certainly is a strength of C1.
- Close PL5, I presume to ensure that the photo is not being “held” (open) by PL5?
- Open PL5, I presume to continue with processing more photos?
The tests I have been conducting have the photo open in PL5 and Affinity at the same time I get easily bored waiting for programs to re-open so when testing I will risk problems for the sake of speed and only change the test flow when things go wrong!
Good coding should ensure that things don’t go wrong (but your situation seems to indicate that to be untrue, at least some of the time!)
Hi Brian
Really ni problem i like straight to the point questions 
- Export the TIFF from PL5 to … whatever program can read and process the TIFF, I presume.
Correct
- Safe in Affinity means what exactly, e.g. Open in Affinity?
Make changes in affinity usually cleanup dodge and burning etc… Safe the file and close
- Change Rating - I presume that is a PL5 action because it does not appear to be a strength of Affinity but certainly is a strength of C1.
Change rating in pl5 yes or open the fiff again in another program from pl5
- Close PL5, I presume to ensure that the photo is not being “held” (open) by PL5?
Close pl5 go sleep do something else…
- Open PL5, I presume to continue with processing more photos
Yes open pl5 app or even in another app.
That’s so far how i think it happens.
On Apple with external Nas mounted via smb
I cannot reproduce the “same timestamp” effect with DPL 5.1.1 on macOS Big Sur on iMac 2019. Whatever I do, .dop files always have a different (newer) timestamp than the TIFF files.
Patrik (@trikke) I have a bit more time this afternoon so thanks for the update to the workflow. I am running on Win10 and will copy the images I want to test to my NAS and continue testing across the LAN trying to follow your flow as closely as possible. Although I can easily change many photos in PL5 and export as TIFF processing them through Affinity will be the botlleneck!
EDIT{
Why were the TIFFs not given a ‘Rating’ in PL5 before the export because that feature was available in PL4. Now there is the possibility to continue with your WorkFlow but the final ‘Rating’ will not require another export if only ‘Rating’ is done because with PL5 that will be applied to the TIFF directly.
}End EDIT
Roughly (inquisition time again - sorry);
- How many photos do you convert to TIFF at one time if PL5?
- Does PL5 remain open while you are processing the photos in Affinity?
- What save or export command do you use in Affinity after you have made your edits there?
- Typically how many photos do you process in Affinity at one time before returning to PL5 etc.
- Do you close down Affinity when you return to PL5 or does it remain open (remember in my tests so far both programs have remained open!).
- Does the corruption occur at the same point in the workflow?
- How many times have things been successful compared to the 3 incidents of corruption?
- Are you sorry that I ever got involved (yet)?
I have copied 180 RAWS to the NAS and will start PL5 generating TIFF files and start opening them in Affinity while the export is going on and make changes as quickly as possible on Affinity. For speed I will not use DeepPrime (at least on the first runs of the test). I stopped the TIFF generation before all the TIFFs were created to address @platypus’s concerns.
@platypus The important fact is that @trikke wasn’t expecting to see any activity on the TIFFs when assigning the ‘Rating’ but because we are now running on PL5 and the ‘Rating’ xmp field in the TIFF is going to be updated then there will be a date/time change for the TIFF!
Just to see what it looked like I assigned 4* to a large number of TIFFs and this is PL5 versus Photo Supreme, i.e. the assignment made in PL5 has been transferred to the TIFF (and the DOP) and is visible externally - no need to re-export the file unless other edits have been done at the same time as applying the ‘Rating’.
PL5 versus Photo Supreme:-
The TIFF:-
The DOP:-
Finally I removed all the DOPs (changed their names) and changed the name of the directory and (re-)introduced the new directory to PL5 and all the ratings were discovered in the TIFF (exactly as was predicted), i.e. no DOPs were used in the making of this image!
But still no corruption!!
Just for clarification, I assume the timestamp you’re referencing is a change in the ‘date modified’ for the tiff? As @BHAYT and others have discussed, if the Rating is applied to the TIF, that’s expected behavior (as opposed to changes in tags like date created and date digitized, which would not be expected).
I suspect the file corruption you’ve seen is related to resource competition/timing in use of a NAS and multiple programs, but I’m just guessing.
1 Like
@jch2103 You know me well enough by now to know that I will be running my machine with all the programs running at the same time, so I was creating TIFFs at the same as editing in Affinity at the same time as drinking a coffee (sorry way too flippant).
Because I haven’t broken it yet doesn’t mean that it won’t break but it all depends how close I can match @trikke’s workflow in the hope that I will fall into the same “black hole”. The programs appear to be generally well behaved but maybe I need to be more creative!
I perhaps should have been more explicit - I suspect there’s an issue w/ the OP’s hardware/software environment that’s led to image corruption. Very hard to track those things down and even harder to re-create on different hardware. (I have a few corrupted images in my collection from several years ago - no idea what caused them, but fortunately no repetition so far.) Enjoy your coffee!
Very possibly and I finished my coffee hours ago!
Creating new TIFs from RW2 files in PL5 while running a batch job in Affinity that was turning the TIFs into TIFFs I have managed to create 5 errors in the PL5 TIF exports, i.e. 184 attempts to export and 179 successful and 5 failures, although they are not consecutive photos they are all clustered together!
This is too early in the WorkFlow to be what I am looking for, i.e. the failure is in the TIF creation not in TIF (or TIFF) processing, but I have decided that I need a faster machine (the wife may not be so easily convinced!?).
monterey 12.1 with pl 5.1.1
iMac (Retina 5K, 27-inch, 2017)
so as discussed it seems to be normal behaviour that the timestamp of the tiff is updated and not impossible
EDIT{
Why were the TIFFs not given a ‘Rating’ in PL5 before the export because that feature was available in PL4. Now there is the possibility to continue with your WorkFlow but the final ‘Rating’ will not require another export if only ‘Rating’ is done because with PL5 that will be applied to the TIFF directly.
}End EDIT
sorry must have been more precise about this:
i gave ratings before the export on the original raw then changed the rating on the tiff after the edit
Roughly (inquisition time again - sorry);
- How many photos do you convert to TIFF at one time if PL5?
one at a time
- Does PL5 remain open while you are processing the photos in Affinity?
yes doing it by making use of the export button in pl5
- What save or export command do you use in Affinity after you have made your edits there?
just close the affinity window (red osx button like X in windows) and press safe
- Typically how many photos do you process in Affinity at one time before returning to PL5 etc.
one by one
- Do you close down Affinity when you return to PL5 or does it remain open (remember in my tests so far both programs have remained open!).
remains open only the picture gets closed
- Does the corruption occur at the same point in the workflow?
yes
- How many times have things been successful compared to the 3 incidents of corruption?
pfff hard to say probably 40±
- Are you sorry that I ever got involved (yet)?
nah not yet
after a few trials i had corruption. I removed the tiff made a copy of the original tiff removed the dop file and tried to open PL5 again and it refused to open the tiff by telling me that the file was not made with a supported camera by by PL5
i deleted the tiff from within PL5 closed PL5
made another copy from the original tiff went back into PL5 and it was able to read the tiff again
one of the other times i got corruption in the tiff
anyway it seems all to work fine if i not just close everything too quick something i never had/have with C1 so its not hardware related imho but anyway it seems to work well if i give it time
Exactly, with PL4 the ‘Rating’ data held in the TIFF was “safe” from DxPL “interference” (the immutability constraint). But now that PL5 is “playing” with the other programs and DAM systems that change metadata then it needs to “externalise” (from the database and the DOP) the changes it makes so that the necessary interaction can take place.
Hence, PL5 updates the TIFF and the timestamps change to indicate that activity has occurred!
Patrik (@trikke) You might not be sorry but I am! I have just made one last attempt to create a problem because all tests conducted previously across a 1 gigabit LAN to the Synology DS220J have not produced a fault. During the tests with the NAS I was changing the ‘Rating’ in PL5 while changing the photo in Affinity, writing back the changes and the new ‘Ratings’ were preserved etc. etc.
So the issue would appear to be what “we in the trade” used to call a “timing window”, always amongst the worse problems to recreate and even worse in this case because I am working with a different NAS, a different network, a different operating system and different copies(builds) of Affinity and PL5 compared to you, other than that everything is perfect!?.
One way to investigate such problems is to open the “timing window” even more! So I attached a USB3 flash drive to the front USB2 port of my Test machine and shared that drive across the network, Not only did that slow things down but I could even “see” the I/Os on the blue LED of the flash drive!
It certainly slowed things down to almost glacial speeds! I copied a small number of the files I had been using and set about testing. The operating system stopped me doing certain things and threatened to terminate ‘not responding’ PL5 and I took that opportunity on one occasion but still no damaged TIFF!
So I will give up on your problem Patrik, for now.
However, @sgospodarenko it might be worth passing this issue on to the development teams(Mac or both) just in case there is something that can be done with the respect to the way that the details are being written that might reduce the “window” that Patrik is managing to find!
In the meantime I used a Trial copy of C1 on my Test machine with PL5 and all worked well for a while but then the programs stopped “seeing” one another, i.e changes to ‘Rating’ went unnoticed by the other program! Returning to my main machine but continuing to use the Flash Drive changes made in PL5 were not being seen by Photo Supreme Lite and vice versa!!
When I checked an xmp sidecar file had been created (probably by me altering C1 settings) and it appears that PL5 started using the sidecar file while the other software, C1 on the Test machine, PS Lite on the Main machine and PIE (used for monitoring) on the main machine, continued to use the TIFF (as all software should!) @sgospodarenko. So I need to start a new “bug” topic if I can remember how to do that, plus get on with my life!! Please ignore the last part of this post, PL5 does not appear to use xmp sidecar files when reading ‘Rating’ (and other metadata) from a TIFF image file.
1 Like
You mean you have a life outside of PL5 metadata? I joined the Sacred Order of the Metadata Sisters during the last beta and have just been promoted to Mother Inferior 
Haha 
(the emoji has more teeth than I do, but that is way too much information)!
The file date change depends on how DPL is set.
The file date will change, when the setting highlighted with the green box is enabled.
DPL writes XMP into TIFF and JPEG files, changing the file date is therefore adequate.
The file date will NOT change, independent of how you set the settings in the pink box, which means that the file timestamp update is independent of .dop file creation.
1 Like
i don’t think its hardware related as the issue only popped up onder PL5 never ever with any other software so far.
anyway i know i need to give things time not be too quick in editing saving closing to its fine since then.
is probably something devs could have a look at but like mentioned these kind of bugs are hard to catch.
Thx anyway guys for the effort and all the feedback
i keep testing PL5 and hope i can find my new home in this software as i like the way of editing my photos so far. need to to some more testing with landscapes.
When I was testing for XMP and .dop sidecar date changes, I noticed that I had to switch folders and that updates were only written upon returning to the folder with the changed image. Note that I tested with only one image in the folder, having more than one image could change DPL’s behaviour, if it doesn’t waitind a while could be time well spent.
1 Like