I’m still new to DxO’s structures. My Capture One “catalogs” (databases) are on an external TB3 SSD. They grow while my internal SSD doesn’t and I cannot exchange it. I’m currently reading where the DxO database is stored as I also want it to be somewhere else than in the application support folder. Like @platypus I also use a MacBook Air with a 1TB external SSD when abroad and so far I just copied the folders to my RAID after arriving home. That contains DOP and maybe XMP files (if I like to add GPS or keywords). So far that worked and until I know better I didn’t try to merge databases.
Only copying the image folders can get you issues like the following: Add a setting, which will then disappear when you open the image on the other computer. Such effects can be caused by how and when PL reads or writes sidecar files. New files will be read first (imo), but older, reworked files can get messed up through overwriting new settings with old ones from the database.
Automatic sync of .dop and .xmp sidecars can make sense, but there is no way (yet) to tell PhotoLab when to to which action. We’d need settings (or popup questions) like “overwrite old settings/keywords/metadata” or “preserve old settings/keywords/metadata” that let us choose which source of information is the one we currently trust.
One way to control the above is, to disable automatic r/w of sidecar files and manage the actions ourselves.
This may or may not be helpful to others. I think it will allow me to copy folders from my laptop onto my desktop in an organized way. I’ve opened the “Finder” tool on my desktop computer, and it shows only my photos since December 2021 currently on my desktop.
Way at the left I have the Apple folder PICTURES. In my first column to the right of PICTURES I created a folder _2021 December (for work that I will be creating this month).
Next column shows all the folders inside of the _2021 December folder. The folder name is created by PhotoMechanic when I ingest new photos, with folder names in the format year-month-day-short description (as was suggested in this forum).
I clicked on the last entry in that column, and the folder to the right of that column shows the contents of that folder I clicked on, in which PhotoMechanic automatically renames the files as file-number-year-month-day-filetype.
Here’s a screen capture:
My latest plan is to use PhotoMechanic on all my computers to “ingest” photos from my memory cards, re-naming folders and files in the above formats.
Moving ahead, to the topic of this discussion, I hope I can COPY everything I captured last week on my MacBook Pro, which is currently stored in a _2021 December folder on that computer, and PASTE it into the _2021 December folder on my desktop.
As for Using PhotoLab with two computers, I’ll wait until DxO comes up with a proper solution.
In January next year, I’ll be creating a folder named _2022 January, and that’s where photos taken in January 2022 will go.
(I think my computer will show things alphabetically, meaning February will be above January, but I don’t see that as a major problem.)
I rely only on XMP and DOP files and set PL to automatically write to these files. I can then copy my photos along with these sidecar files to a central location when returning from an excursion. I occasionally delete my database to keep things tidy and speedy.
If I need to manage more than what is in the sidecar files I will use my DAM (Photo Mechanic Plus) on a subset of my photos.
I did buy the Photo Mechanic Plus, intending to eventually use it for DAM, but I need to learn how to set that up. Eventually I’ll get around to it.
I have never deleted my PhotoLab to delete its database deliberately, but I think between PL3 and PL4 the installation program did this, if I remember correctly. How often should I do it, and for that matter, how do I tell PL5 to do it safely?
Thanks - my “central location” will be either on the main drive of my Mac Mini, or perhaps on an external high speed drive. In that case, I’ll be asking here about what type of drive would be best to buy.
@mikemyers I am a Windows 10 user not a Mac user but I believe both versions use an SQLite database and I have written at great length (sorry big posts appear to be a habit of mine) about the issues of the wrong DOP meeting the right (or wrong) database entry Unwanted virtual copies - #86 by BHAYT.
If a database is oblivious to a particular Photo in a particular directory/folder it will absorb/ingest/import the DOP automatically based upon the ‘Preferences’ for handling sidecars (DOPs) or when manually directed to ‘Import’ the sidecar (if you manually import you will always wind up with a Virtual Copy - the photo before the import counts as the [M]aster and the import becomes copy [1]). If it has never seen that photo before then all should be O.K…
If the database has “seen” that photo before, i.e there is a database entry for that exact photo in that exact directory (currently PL does not check and start matching up the same photo filename regardless of the directory as some other products do! at least on a PC) then a check swings into place that will determine the fate of the attached DOP, but PL5 will protect the sanctity of both the database entry and the incoming DOP at all costs.
The field is the Uuid just behind ‘ShouldProcess’ as shown below
there will be a similar Uuid for all copies held in the DOP. I haven’t run a test were one of the photos [M] or [VC] matches the database and the other(s) doesn’t (don’t) but certainly a mismatch between the [M] Uuid and the database is going to start the creation of Virtual Copies!
So the only way to import successfully, regardless of what media you use or whether you access the original photo across a LAN (I have done that with some success but only by dividing the exports between two systems and then …) once a Uuid is allocated that will not/does not match the system you are going to import it into there will be Virtual Copies; you can escape this fate by ensuring that there is no corresponding entry in the database you are importing into, i.e. by throwing away the database so there won’t be a problem!!!
If you doubt the above the test is simple;
- Edit a photo in PL5
- Open the DOP in a text editor and change the last digit in the Uuid I identified above, i.e. in my example change the 3 to a 0 and then save the DOP file.
- In my tests PL5 has picked up the change instantly and I have a shining new Virtual Copy to play with (whether I wanted one of not)!!
I have “moaned” about the fact that the VCs cannot be promoted to [M] status etc., there is much that DxO could do to help manage the fall-out from this situation but that is in the future perhaps. Things might be simpler of more complex on a Mac but the above comes from many tests that I have run on a PC.
Hi @mikemyers, if you look in the preferences you will find a setting where you can specify a location for your database. Note down this location then close PL. Go to that location in Finder and delete the files you see there. There should be no other files other than the database files (2 or 3 files?). Now you can start PL and a new clean database will be created automatically.
I delete my database about once a month but it is a personal choice as to how often. The more directories and photos you view in PL the more data is stored in the database.
PMP is quite complex program but once you get used to it you will find it very useful. The great thing about it is it can have multiple databases and you have complete control over what is stored in the databases as well as how they are used (read/write, read-only or not used). You can search all databases too or just selected databases.
Hope this helps.
The database files can be found here:
Instead of “macadmin” you’ll see your user account name 
Note that deleting the database will obliterate everything you have done and not saved to sidecar files.
Hi Bryan, I agree with your assessment and can confirm that the database is indeed SQLite.
I think the aim of DxO is to store all PL specific details on DOP files and the database. All other standards-based data in XMP files and the database. Currently this is not quite the case as some data like keywords and ratings are still stored on DOP files and I suspect is often ignored. This has caused confusion on these forums at times but I hope DxO will fix this soon.
I made the suggestion that DOP files should ONLY contain PL-specific data and other common data stored in XMP files (and database). This should make data management and synchronisation simpler and more understandable.
My desktop computer has never seen the photos I took last week on my trip. After a long talk with a fellow at Apple Tech Support, I took a high speed USB drive, plugged it into my laptop, then copied the folders I created last week. Then I copied them to my desktop computer, where PL5 worked perfectly with them.
This isn’t what I was asking for, but it does solve my immediate need - get my recent photos off the MacBook Pro, and onto my desktop.
All my Mac computers have the updated OS, and no issues.
@KeithRJ I was writing the following when your post arrived so I will try to cover some of the same ground plus what I also wanted to add to my post.
Hopefully from my post above it should be clear that if you take a laptop with PL5 loaded on holiday/to a shoot etc. then photos can be imported onto the laptop, PL5 can be directed to navigate to the photos, presets and other editing applied, plus Ratings and Tags and if it fits your workflow keywords etc also applied.
At some point the Photos, DOPs and ‘xmp’ sidecars etc. can be copied to the main system and PL5 navigated to the directory and iff (if and only if) the database has never seen those photos in that directory before all can be imported successfully into the second/main database, which appears to be what @mikemyers is describing in the post that came in while I was writing this!
I believe that they come in with the Uuid in the DOP I described in my post and that Uuid will be stored in the database. But any change made on the main system or on the laptop will change the respective DOP Uuids and Virtual Copies are then inevitable if any attempt is made to take data in either direction, main to laptop or laptop to main.
Hence the suggestions being made to “trash” the database or rather manage it because as long as you keep a log of what was in each database it can be saved and restored later. I have “trashed” the PL5 database perhaps 20 times since PL5 release because of the testing I have been doing. Life becomes a little more complicated if keywording etc. is being used but providing that is written to the sidecar file or JPG or whatever then it is secure and the loss of the database may not be significant.
DOPs do now contain keywords but I have no evidence that PL5 is doing anything currently other than storing them in the DOP. ‘Ratings’ are in the DOP but also in the ‘xmp’ sidecar, Tags seem to be in the DOP only.
I partly agree with your notion that the split would make synchronisation simpler but to be honest providing you ensure what is in the PL5 database has been reflected in the external elements then you need the photo, DOP and ‘xmp’ sidecar and you are good to go. Except don’t go anywhere near a PL5 database that knows the photo (in the directory) or wallow in VC “heaven” or “hell”.
What is actually needed are some additional data management commands that allow VCs to be “promoted” on a single photo, selection of photos and entire directories and subdirectories basis, i.e. select an entire quarters worth of photos that have loads of VCs and “promote” them!
Except as was well pointed out by someone in another post (it’s late so I will look up the reference tomorrow) how do you distinguish between the VCs caused by importation versus those created deliberately as part of the editing process (the context of the original post was with respect to mass deletions of the VCs, except of course that the imports may actually be the edits you want to keep)!!
This means that the importation process would need to be enhanced to provide the automatic promotion but the advantage of trying to sort it out after importation is the ability to review and control what stays, the import or the existing database and what about “legitimate” VCs for one photo it might be possible to have the [M] and [1] in the database and [2] and [3] come in as part of the importation process could that ever be resolved by any bulk process!?
It is a minefield and I need to be up early tomorrow!!
What good is a database that needs to be trashed that often? Why waste energy and disk space on creating it over and over again?
2 Likes
While this sounds logical, it increases the risk of losing information, e.g. if an external app messes up an XMP sidecar file.
I prefer DxO’s current approach because it creates a distributed backup of the database by storing settings as well as metadata metadata in the .dop files.
@JoJu The database does not need to be trashed if you use it the way that it was intended to be used, i.e. on one system.
The “issue” comes if you want to have multiple systems attempting to access a pool of photos across a LAN where there are two databases and two applications dishing out DOPs with “competing” Uuids. Similarly if you move an external drive, HDD or SSD etc. between two (or more) systems the same thing will happen.
Where I have encountered the problem in the past was DxO 11 versus PL4 and more recently PL4 versus PL5 and I wanted to understand what was going on (yes running PL4 and PL5 at the same time on the same directory is real “fun”)!!
My multiple “trashings” at the moment are as a result of me exploring these issues and wanting to repeat tests with a database that I can easily “peek” into and see the results in the database to compare with the DOP etc… Prior to these recent tests I have rarely “trashed” my PL database.
Here we are laying this at the door of DxO but are there any lessons to be learnt from Lightroom, Capture 1, Photo Mechanic etc., all of which have SQLite databases, and your workflow with these systems?
One remark in general on synchronizing computers, it’s quite difficult to delete items. When an item is deleted on one pc it will come back with next synchronization.
George
@George One real problem with PL5 is that once a directory has been seen and imported it will not be forgotten by the database, the database will hold onto the details regardless of the data on disk!
When I was talking about “promoting” incoming DOP entries so that they effectively replace those in the database rather than getting tacked to the database entry as Virtual Copies I omitted to emphasis the point that PL does not have an explicit import function, it just “discovers” “new” (to it) directories and promptly adds them to the database.
I have seen in posts from some time ago with requests from users to be able to get PL to ignore directories, i.e. to add an import directory function to PL. Personally I hate the Lightroom ingesting process and have always loved the simplicity of the DxO approach but that “freedom” comes at a price. If PL discovers a directory it is importing DOPs, ‘xmps’ and all as soon as the user navigates to that directory.
If you are using only one system that is wonderful, if not then PL is not going to forget that directory regardless of whether it still exists on disk (haven’t verified this so another test coming on) and if it crops up again and any Uuids do not tally then…
I have found as I use an external DAM if I forget to add key words through it first (pl doesn’t see them if done after processing) I can copy the folder, delete the contents in pl than copy the folder back to pl, it than sees the editing and keywords. I use this keeping a desk and laptop identical to enable procesing away from home.
@John7 That makes sense I hadn’t got round to testing that so deleting the directories via PL5 eradicates them from the disk and the database. The situation that concerned me was deleting them from disk (not via PL5) when I suspect that they will remain in the database.
One thought that I have but requires DxO to implement it is to have a preferences options
- To write Uuid free DOPs (or a DOP with a special “wildcard” value) and then overwrite the database when importing such DOPs or create VCs if the date of the incoming DOP is earlier than the date of the database entry or…
- While this would work for future implementations another option would be needed to ignore DOPs with Uuids that exist from before such a strategy.
- The date time check could be made optional i.e. always accept the incoming DOP and overwrite the database editing details etc.
- I have not even vaguely explored all the ramifications of such a scheme.
- I have directories in the database where I have managed to screw up PL5 so I also want an option where I can leave the data intact on disk but expunge the directory from the PL5 database!
Problems only tend to arise if you swap between computers on a regular basis.
If you ingest your camera’s files onto your laptop and work on them in PL there, as long as they have never been seen before on your desktop computer, all you have to do is to copy them, together with their DOP (and XMP) files to the desktop machine. Your desktop will then add them to its database and function perfectly normally.
However, if you were then to work further on those images and then copy them back to your laptop, PL on your laptop will quietly blow its cool in trying to reconcile the original versions that it knows about in its database and the versions of the DOP files it will then read from the files you have just copied back to the laptop.
As long as you only ever perform a one-way move (not a copy) from your laptop to your desktop, thus deleting the originals from your laptop, you shouldn’t have any problems.
I would like to see an option where PL does NOT store metadata for visited directories so that you can explicitly catalog a directory. It would also be nice to have the ability to un-catalog a directory. You then have full control over what is in the database.
I feel rather out of control with how everything is as it stands now. I would also like to have the ability to have multiple databases and you can then select which one to use. LR and PMP do this.