What is the best way to move a folder of photos with their respective *.DOP and *.XMP files?
I am using a MacBook Pro, M1 Max, Monterey 12.2 and DxO PL5 5.1.1.
Basically, I would like to import and edit photos on my MacBook and then archive them to a desktop hard drive, but still have all of the editing history and ratings available.
I perform my original culling with Photo Mechanic, I then import my selected photos into DxO and then give all photos a 4* rating. After closer examination and editing I change the rejects to 3* and some to 5*. Once the editing is complete and all relevant photos have been exported I move the entire folder, including all of the DOP and XML files to a desktop hard-drive. I then opened the photos on the hard drive to find that their history and star ratings were not to be seen. I managed to import the Sidecars and Metadata which left me with my final edit corrections, cropping and keywords, but not the star ratings nor the history.
Is it possible to use PL5 to move the folder full of files, or is there a better way?
I just do not understand DxO’s connection between their database and the DOP files, and how to use the system to suit my needs. Previously I used Capture One Pro and moving files within the program worked flawlessly, but I need the Deep Prime facility within DxO.
I have never worked with Photo Mechanic - I use Photo Supreme as my DAM. If I move a photo file WITHIN Photo Supreme from folder A to folder B, all sidecars (.dop and i.e xmp files) get moved along automatically. The same happens if I delete a photofile within Photosupreme - all sidecars get deleted too.
Just try it if it is the same with Photo Mechanic.
Moving folders outside of PhotoLab disconnects PhotoLab’s database from the moved files.
Star ratings are stored in .dop and .xmp sidecars, but history is exclusively stored in DPL’s database. Beware: Synchronising star ratings does not work under all circumstances. This has been covered in a few posts already, e.g. here: Star Rating not working in PhotoLab 5 for dop files (without using PL database)
DPL 5 does not support folder moving on macOS, but you can move or copy (group of) files from one folder to another - if the target folder can be seen in DPL’s sidebar. Beware: DPL is picky about where you can move/copy files to. Test your ideas before seriously committing your originals!
Steve - It depends on what you mean by “editing history” …
The “history” that Platypus refers to is the minute detail of all step-by-step corrections you made in achieving the final result for each image.
However, the final state of all individual correction elements (such as your final settings for Smart Lighting, Exposure, Crop, etc, etc, etc, etc) is stored within the sidecar/.dop files … from which PL will re-create database entries for your new folder/image location.
For example, I don’t depend on the database at all; I delete it between PL sessions - Instead, I reply only on the sidecar/.dop files.
Just a word of caution, tho; if you do move RAW files and their associated sidecar/.dop files then be sure to move them together, at the same time - and, for extra caution, do so only with PL NOT running.
Hi Sigi, Thank you for your answer. I found it easier to only import my photos into the the Photo Mechanic DAM after all moves have taken place. I only use Photo Mechanic at the start of the process for its speed of culling and rating photos prior to editing them.
I am trying to work out the relationship between the PL5 database and the DOP and XML files after they have been moved.
Cheers
Steve
Hi Platypus, Thank you for your answer and explanation. I will have a look at the link that you supplied and experiment with a few photos by moving them to a different folder on a different drive within PL5 and see which info is transferred and which is not.
Cheers
Steve
Hi John-M, Thank you for your reply and explanation. That corresponds to my experience of the detailed history disappearing after the move but the final state being shown, minus the star rating that were made within PL5.
I could live with deleting the database between PL sessions, I just wish the star ratings were there in the DOP files.
Fascinating, and counter intuitive, to move the file with PL closed and not running, as I would have thought, in my ignorance, that moving items from within PL5 with the database open would update the database.
More experimentation needed!
Cheers
Steve
The “danger” involved with moving more than one set of [images + associated sidecar/.dop files] whilst PL is running is in the timing of the moves … Depending on the tool being used to move the files, they may not actually be moved in matching pairs.
If, say, a number of sidecar/.dop files were to be moved before their associated image files are moved - then PL may detect the missing sidecar files and re-create them … which can then become confusing. It’s simplest and safest to make such changes while PL is not running. It will recreate the database, as required, the next time it encounters [images + associated sidecar/.dop files] not already in its database.
If moving file outside of PL5 I would move all of the files associated with a particular photo together. I have not tried to move files within PL5, so I do not know if moving the RAW file will move the DOP and XML, or whether it would re-index the RAW in its new location.
However, I can see merit in your idea of treating each session as a separate entity and deleting the database after each move. C1P also uses sessions and catalogues in a similar way. I would like to be able to re-export the photo after the move, perhaps with a different crop, or plus/minus a watermark without having to start from scratch. I just need to find a way to sort out (or separate out) the photos that I have rejected by a star rating.
Considering how advanced DxO are with so many facets of PL5, I am surprised that moving files is so complicated. However, I will persevere and find a way to achieve my goal.
DPL started out as a raw developer a few years ago. Managing the assets has never been an important topic for DPL so far.
DPL5 now starts with a more evolved keywording feature, but it still lacks a few important maintenance features. Until it gets these, DPL is best thought of as a raw converter and all management and metadata actions are best done in the way you’re used to.
Do not manage metadata in more than one app, stick to a single point of definition (SPOD) until you’re sure that switching the SPOD is lossless according to your own tests and expectations.
Hi Platypus, Thank you. I started using DxO Optics Pro 9 back in 2013 but had to move to another raw processor when I started using Fuji cameras. Ironically, I no longer using Fuji, but have returned to DxO due to the noise reduction technology. In the meantime I used Capture One Pro which did and does a very good job of editing raw files, and I got used to that system and managed to make it work to suit my needs.
Actually, this thread is solely to do with managing metadata within DxO, or not as the case may be. I may have found an answer to my original question, and I will be experimenting with that process later this afternoon.
However, I do believe that, along with many other improvements that DxO has made since I last used it, another improvement would be in how it manages it’s own metadata within it’s own DOP files when they are moved to a new folder.
I prefer to import and edit my photos on my computer hard drive, e.g., especially when travelling on land or on a cruise, and then archiving those photos onto a 20Tb RAID system or individual desktop hard drives, thus freeing up space on my computer.
I have been managing metadata in more than one app since 2014 when I first purchased Photo Mechanic and I have found it to work very well when one understands the subtle differences between the applications. My current conundrum is how best to manage the way DxO manages it’s metadata within it’s DOP files, and I may have found a solution.
Thank you for help and comments.
Cheers
Steve
Hi @Sigi, @platypus & @John-M,
With your collective help and assistance I have found a workaround, or two, to my problem:
Option 1.
On my new hard drive create a folder for each star rating that I would like to save, e.g. ‘date’ 3*, ‘date’ 4*, ‘date’ 5*.
In the original folder, filter a particular star rating, e.g., 3*.
Within PL5 Photo Library drag and drop (copy) all of the 3* photos into the new 3* folder on the new hard drive.
Repeat for the 4* and 5* photos.
This procedure still strips out the star rating, but as they are now in their own star rated folder it doesn’t matter. If I want I can select all and re-apply a desired star rating. It does maintain the full editing history in the Advanced Editing tab. It also maintains the ‘pick/reject’ (green/red) rating which leads me on to option 2.
Option 2:
On my new hard drive create a new folder
Reduce my star ratings to two, e.g., 3* and 4*.
Give all the 4s a pick/green rating.
Give all the 3s a reject/red rating.
Within PL5 Photo Library drag and drop all of the photos into the new folder on the new hard drive.
Filter all of the pick/green rated photos and give them a 4*.
Filter all of the reject/red rated photos and give them a 3*.
This seems to work and maintains all of the other metadata apart from the star rating. Interestingly, when the photos are being re-indexed by the system you can see the star rating being stripped out of the thumbnail as it get ingested to the new database.
Yet again, thank you all for your help.
Cheers
Steve
@SGS Sorry I am late to this post but please explain why you believe you are losing your * rating. We have just been covering this in another and another and … post (please note that I am a Win10 user and while there are differences I am unsure that any exist with ‘Rating’ i.e. you should be able to preserve ‘Rating’ with any data you move between systems and make that data visible to other products (Photo Mechanics - PM) and the changes other products (PM) makes also visible to PL5 if you want to take it that far.
Your stated plan is not particularly ambitious and it puzzles me why you have been unsuccessful!
However, let me take on statement you made in one of your posts in this topic namely
I am surprised that know one has told you that PL5 does not manage its metadata in the DOP any more! That statement is true if you don’t include edits, last edit used etc, which are held in and used from the DOP (and database). The Tag is also a PL5 only piece of metadata and that is in the DOP but currently there is a PL5 bug that means it goes in but doesn’t come out of the DOP!
The field you are most interested in is the ‘Ratings’ and that no longer resides as a piece of usable data in the DOP! Pre PL5 DxO had an immutability rule which meant that they would not touch external metadata but that has changed so now you will find the ‘Rating’, ‘Rotation’ keywords etc, in the DOP for the [M]aster photo (or only photo) and a separate set for each Virtual copy associated with that [M]aster (if there are any VCs).
However, the DOP provides the data for all the VCs except the [M]aster!
This is a major change between PL4 and PL5, for PL5 the data will be taken from the photo xmp data (embedded or sidecar). Please note that PL5 only puts data into the xmp sidecar for RAWs, Photo Mechanic can be configured to use the embedded xmp data for RAWs as well (I have used and tested it).
So if all you wanted to preserve is the ‘Rating’ assigned in PL5 for a RAW photo then you do not require the DOP but you do need the xmp sidecar file instead!? You need both if you wish to preserve the edits, ‘Ratings’ etc..
You stated before that you are preserving the DOP and xmp sidecar files and that should be sufficient to move data and add that data to another PL5 database!! BUT only if you are actually writing the data you need into the xmp sidecar files for RAW or directly to the embedded xmp for RGB files.
Moving the xmp sidecar will achieve nothing if you haven’t written the data to the xmp sidecar in PL5 in the first place. The metadata commands at your disposal are:
The ‘Preferences’ ‘Sync’ command. If turned on this will automatically propagate any (external) metadata changes from the photo to PL5 and at the same time propagate any PL5 changes from the PL5 DB to the xmp sidecar file or directly into the embedded xmp data for JPGs etc. Some using the forum do not like this option but personally I feel that it is actually the safest way of making sure that nothing important gets left in the PL5 DB when it should be in external files.
The ‘File’/‘Metadata’/‘Read from image’ overwrites the database keywords, Rating data etc. with the data from the photo (JPG) or xmp sidecar(RAW). Please note that this is a one way destructive process from the photo to the database, and will “destroy” any data in the database that was not already in the photo.
The ‘File’/‘Metadata’/‘Write to image’ overwrites the photo keywords, Rating data etc. with the data from the photo (JPG) or xmp sidecar(RAW). Please note that this is a one way destructive process from the database to the photo, and will “destroy” any data in the photo that was not already in the database.
There is currently no manual equivalent to the Sync (merge) option although I have asked for one! You either have the fairly benign Sync option or the two other (potentially destructive commands). The risk with the manual commands is obviously that you may accidentally overwrite something required but arguably even worse when moving data around that you forget to update the external metadata, when it will remain in the database and not be backed up to the JPG, xmp sidecar etc!
The reason for not moving (or just copying) the data with PL5 shutdown is really to ensure that any pending writes of DOP, or xmp data have been completed and the move is not being executed with an incomplete set of data.
@John-M knows that I do not fully subscribe to his mandatory database destruction process even though it works for him and securing as much data as possible outside the database is always a useful added precaution against disaster.
If I understand your intention correctly you are going to process your photos with PL5 after basic culling with PM and then archive your results to an external drive which is then going to be attached to another (desktop) machine. The following may be of interest
Carrying a hard drive between two systems where the hard drive contains both the data (photos, DOP and xmp) AND the PL5 database. I have tested this and it works but I still have concerns about why it works Cache file - #28 by BHAYT.
With respect to the ‘Advanced History’ this does not exist in Win 10 PL5 except as the data in the DOP which (re-)configures the editing options settings. The settings are held in the database and a copy is held in the DOP but there is nothing in the Win10 database that stores the ‘Advanced History’ so it cannot be written to the database and to the DOP it is essentially session data and (on Win 10) is not persisted anywhere.
The starting point shown at the base of the ‘Advanced History’ is essentially “useless” and is held in the database and the DOP and forms that basis for the ‘Advanced History’ when next you open the image in PL5. But it does not represent anything useful, it does not reflect the last preset used etc. I have complained about that but …
The ‘Advanced History’ before shutdown
The advanced History after a PL5 restart!
Please correct any misunderstanding I have about what you have done and whether my suggestions will work for you and your workflow.
@SGS Sorry about the length of the post I am always afraid that I might miss out an important element or write it as if the reader already knows that part!?
What I have discovered from time to time is that as I write up some test findings I suddenly either doubt what I have written or what I see in the snapshots or the two don’t tally, so yet more testing. It is useful to my own understanding to write it as if I am new to it and see whether it continues to make sense!
You are welcome.
Regards
Bryan
Bebop
(Barb Mac 11.7, PL6, FP6, VP3, Canon, Olympus)
18
I’m not sure if it is helpful @SGS, but I also use Photo Mechanic alongside PL5. My workflow is that I star rate, keyword and cull my images before I open up PL5. The ratings are all there when I open it up. I can then use the filter as required.
If for some reason I want to change my star rating and keywords within PL5, I will then write my metadata to the image as @BHAYT mentioned. This does overwrite what was there (which actually is what I want). If I then go back to Photo Mechanic, the start rating and keywords can be seen updated with the values I set in PL5.
@Bebop, Thank you for the info, I will give that a try:-))
1 Like
Bebop
(Barb Mac 11.7, PL6, FP6, VP3, Canon, Olympus)
20
Just to add… PL5 only reads the xml file when it opens the image the first time. After that any changes will always have to be read or written in the metadata. (I haven’t ticked sync metadata, as my workflow works for me as it is, but perhaps I should).
