Synchronise PhotoLab

YES,

we discussed it a long time ago, and asked why not to implement a simple function like in LR that every time you close LR getting a message with the the choice to backup your catalog/databas or skip this step.
So you have an automated scenario without needing third party software, or to know how to do within your operating system…

I cannot see what you ask in any photo management program, even Lightroom, considered by many to be the best photo management program, cannot sync between two computers. You described something very specific to answer your needs but doing this in a flexible enough way to satisfy the different needs of different users will require creating something super complex. Only managing the photos as files requires options like two ways Sync, one way update, Mirror, customizable mix of these and considers the sources and destinations, the date/time, the file type, the size and so on, with options to keep or deletes files depending of various criterias. Adding to these basic options, and in order to satisfy different users, the app will have to consider the photographic characteristics from 300+ differents metadata fields of a digital images, the images customization, eventually the contents discovered by AI, the keywords, all of this while maintaining the database integrity.
I agree that what you propose feel right, and yes there is a lot of shortcoming in PL5 catalogue function, it is very new and I hope it will be improve in the next iteration of PL, but this is a sub 200$ photo editing app for everyone, running on a personal computer, it is not a dedicated digital asset management program.
Using a files sync app and the sidecars to export/import between systems or App is certainly not perfect but much more feasible

1 Like

@Guenterm but a backup is not exactly the same as syncing a database? CaptureOne also asks from time to time to backup the database and I see it as one possible way to react once C1 crashes. But it’s not a cross backup, so I could not use the backup “as it is” on the laptop to carry on. And back home then backup again back to the main Mac.

EDIT: The sense of “so I could … use the backup “as it is” on …” did completely say the opposite of that I wanted to say, sorry, need to check more carefully.

I agree with the objective, I have the same requirement: one mobile computer when on the go, for early workflow tasks like sorting and selecting, and doing some image manipulation on those that get me enthusiastic, and a standard computer with multiple nice screens to do all the other image manipulations.

However, using dop sync on both machine is already almost working fine if you move the files with their sidecar, or using a big external SSD drive with good USB3.1 connectivity (excepted some recent issues with rating and select/reject status that need to be addressed and fixed).

I said almost because in this scenario what is missing is the project assignment. I hope that one of my feature requests, having project assignment added to the dop file, will be implemented to help to cover a part of this scenario.

I doubt mixing multiple PL versions is a viable solution, that would require backward edition on the older versions, make any new features more complex to implement and I think it’s not reasonable to spend effort on this when a lot of others features are still lacking.

Maybe what you may do, on Windows at least, would be to move the database to the external drive where your images are stored ? Having that it’s assigned to the same drive letter on both computers, and that the drive is performant in both machines. But this need to be carefully tested.

1 Like

What is being asked for is what looks like a server version which some dams do, as very expensive versions

Mylio does all this - but it’s primarily an asset manager, which is why I use PL as well, and it doesn’t carry sidecars apart from the XMP files. Getting sync right is hard for the developers.

Exposure X explicitly allows for cross-machine sync, but doesn’t automate the sync itself. I’d be very happy if PL did this - I’m in the ‘delete the database’ camp at the moment.

Update: I want this to be launched manually, not as an automatic background task. This should make things more controllable as well as less distracting for people who don’t use the feature.

Note that this feature can be used to completely copy a PhotoLab environment (database, photos etc.) to a new computer…

your scenario has two Photolabs, so you need to be more specific.

Do you have any idea how to deal then with such differences?

I’m working with version management, synchronization etc since decades, and I can tell you that it will be not as simple as you write.

The only solution I would support is a strict “prefer image+sidecar data over database”.

With this preferene, you can use mature, proven tools to synchronize. For example “Beyond Compare”, or a SCM/VCS you like (git, Subversion…) or a sync tool (FreeFIleSync, Syncovery…).

Hi Joachim,
yes I know the difference between RAID, Backup, Synchronise and so on :grinning_face_with_smiling_eyes:
Because platypus made a very good request, I only want to place the hint for the simple backup.
During all the time I remember some posts, where people are crying for a corrupt database backup, not knowing that there is a menu entry for backup, or not knowing where the database is placed.
Not all members are interested in all the IT themes, and to make it more easy for this group of members, DXO developers could apply some smart features to make it more comfortable.

Let’s look forward to the ideas will come.

Enjoy the weekend and the 4th Advent

Guenter

I posted this one in another thread.
One remark in general on synchronizing computers, it’s quite difficult to delete items. When an item is deleted on one pc it will come back with next synchronization.

George

@Guenterm I just corrected my post above, it was saying first “so I could use the backup “as it is” on …” which is not my experience.

Also, my laptop’s HD got it’s name “guinea pig” for a reason. The poor thing is always first in row to “try out new features” :smirk:

@George That depends on how the database is checking “which is the newer version” or the recently edited one. But you’re right, there’s a risk that things can go really wrong. And for sure some explanation and “how to deal with different databases” will become more necessary than it is now.

Would it be possible to store the database on a third, very fast drive (Y) and the two PCs (say X and Z) would backup incrementally their databases to this one? First the newer database to the “external drive in the middle”, and there a last check, before the new changes go into the “elder” database? That way I could come home, backup my new database first to the external drive and then from there to the main computer? Too complicated?

Or, like C1 does that, importing the (completely) new catalog into the main catalog? This way all new files including their edits (stored either in DOP-sidecars or database) arrive nearly the same way like new images do from cards. I don’t need two (at some point) fully identical databases. I need culling, rating and maybe a bit of editing when abroad, I don’t need access to 3 year old pictures.

I disagree, it is complex.

This would open a can of worms, I strongly advise against it.

2 Likes

If it’s gone one can’t compare.

George

Well, the database could log which file was deleted and when. How many “2012-12-17_DC3657.NEF” files would be on my drive taken at the 11:34.54 hrs? Of course, if I would work with physical copies like some do it becomes a different story. But that’s what I strictly avoid, therefore I keep forgetting that’s it’s basically possible and without some kind of file (and link) management sometimes a way to go.

But I assume: With so many users already having to find a way to work with PL and still need to manage images, therefore working with a variety of extra DAMs, DxO would not go from “working as RAW-editor” to “only working as RAW-converter/editor if the new included DAM module would be used exclusively”. With this many options available, no RAW-converter/editor would be “excellent enough” to win users who already have organized their workflow with Photo Mechanic, IView or whatever. A DxO DAM module has to be at least as good as most alternatives and not exclude long term clients.

That is why I propose

  • manual launch
  • one way “integration” sort of sync - call it master/slave, if you still want to use these words
  • exchangeable roles

Note: If .dop sidecars contained “everything” about the respective image, things could get fairly straightforward. As of DPL5, the one thing missing in the sidecar is “advanced history” as far as I know. If AH were excluded from the task, things would be trivial.

Remember: It’s all about using two (or more) computers and getting a built-in feature that ensures that differences can be eliminated, if the user chooses to to so.

I use SyncBackFree. Once it created a sync list I can choose which on I want to delete. Not perfect especial when you work with more places. In my case 2 pc’s and the NAS.

George

@platypus I am sorry but is it really that complicated (putting to one side, for a moment, the issue of projects and indexes).

When a copy of the database is taken and worked upon on another computer then new photos and their attendant “baggage” (the twin sidecars DOP and ‘xmp’) is/are unique and can be added back to the base system, in my case using Beyond Compare and for others their choice of comparison software.

The sidecars are a “sort” of Audit Trail except that they represent the end of the chain of changes rather than the whole chain. However, if photos already in the “base” database are changed while out on assignment then attempting to replace the DOPs will cause issues because of their unique Uuid which will no longer match with the database and results in Virtual Copies.

I believe that any changes to the embedded or sidecar ‘xmp’ data will simply be absorbed as if they had been made externally by another program, photo editor or DAM.

So the gating issue for harmony is the Uuid check. Therefore provide a feature to override the Uuid when a change is detected on an

  1. Individual
  2. ‘All’
  3. Session
  4. Preference basis

and harmony is restored.

Could it be that simple - over to you?

Re-indexing should resolve new keywords etc (I hope) but that does leave transferring projects and anything else I haven’t thought about!?

This does not handle deletion but rather than physical deletion mark images as ‘deleted’ using Tags, ratings, colours(sorry not in PL5), keywords etc. and then execute a physical deletion using the selected criteria on both systems as and when appropriate.

This may not be a “super duper” synchronisation process but would take minimal work from DxO developers and might help those that want this feature.

The issue of preparing the laptop database ready for the next trip then comes up and that is effectively the reverse (I think/hope).

This is still two machines with two databases and two sets of assets which may or may not be identical copies and transferring those assets between the two machines if/as/when required, it requires

  1. A “simple” fix to DxPL (a Uuid override)!?
  2. A change to work habits, i.e. no physical deletion
  3. Good comparison software to transfer the various assets from one platform to the other etc.
  4. A user that understands the implications and possibilities of “corrupting” one or either of the two environments if they lose “focus” at any point. Use database backup before embarking on any such exercise. Being able to have DxPL multiple database aware would be useful (please don’t start the discussion I have seen for Lightroom about one database or many versus indexing versus …, leave it up to the individual user to make the decision but simply provide the facility)
  5. Better directory management within DxPL (the ability to rename a directory exists but it would be useful to have a facility to delete a directory which would be useful in the “battle” to better organise the assets.

PS: It is a “little” tedious to have to (re-)visit every directory in order for DxPL to (re-)discover a directory and “import” as a Virtual Copy, or ask for options (Uuid override ) etc. and hence some form of Import option that can selected at an appropriately higher level of directory would also be useful option potentially in this case and for importing keywords on a wholesale basis etc.

Adding assets to DPL’s database is easy, removing them is a little bit more involved.

Add assts

  • Open a new folder and you’re set (better not spread new images over several folders)

Remove assets (without deleting the image files)

  • Open a folder of images that should be removed from the database
  • Delete images from within DPL, the quit DPL
  • Move images from the trash to their original folder

If one wants to keep the databases of multiple computers “in sync”, the most current database must be copied to the other computer(s). In order for these databases to have the links to all files, new folders must be copied to the other computer(s). While this does not sound too complicated, it takes some discipline, or database(s) and file repositor(y/-ies) drift away from each other.

As of today, DPL offers nothing to simplify a multicomputer sync. Lightroom does a better job in that it

  • discovers discrepancies between catalog and file repository
  • offers a choice for metadata copy (either DB → file or file → DB)
  • can remove assets from the catalog without moving those assets to the trash

I can imagine several ways to implement DPL sync, but I’d be (pleasantly) surprised if DxO came up with DPL sync real soon.

1 Like

@platypus This is true if they are genuinely “New” i.e. never before seen by DxPL in the directory that you navigate to! If DxPL thinks it has seen that directory before then the Uuid check comes into play with all DOPs that may be present in the directory and VCs will be created where there is a Uuid clash!

Unless someone, i.e. DxO, provides a ‘Uuid clash’ ‘override capability’ via ‘yes’/‘all’/ ‘session’ and/or Preference options. This would take a day or so to implement and the user would need to use their own compare software to put the data in the correct place to be discovered. But this basic functionality is arguably simpler to implement than the current VC creation and management logic!!

Remove assets also requires DxO development but is also a lot simpler than many of the things already implemented, i.e. allow actual deletion of directories as well as files and allow the removal of a file, selected files, and directories from the database, i.e. the remove option needs to be extended to directories and both the file and directory remove needs to have an additional option of ‘From disk’, ‘From DB’. The DB option should do just that and leave DOPs intact along with ‘xmp’ sidecar and the original photo!

Another days work for the developers and without it we are spending days writing these posts and countless amounts of time trying to achieve what would be simple with a few “minor” extensions to the code!!

I would like to see a multiple database option added - 1 more days development!

I would like to see an ‘Import’ option to which the Uuid override options I have suggested would also be available. This could import data from any external media, USB disks, memory sticks, another computer across the LAN with an import directory and a target directory. The target would be new or existing and the Uuid clash would be handled using the same process as I have just mentioned.

This is a longer task so I would give the developers a couple of weeks to develop that code. While it might be seen as a retrograde step (to much like Photo Mechanic, LR, C1 etc.) here it is for a very specific reason not as the only way to introduce new photos to the product for the first time but to add new and “remodel” old photos.

As for DxO implementing any of these we need to start pressuring them to look after their current user base as well as chasing new customers, particularly when I believe that some of the items I have mentioned are actually straightforward to implement.

PS: I still consider PL5 is one of the best programs to use, to navigate etc. it is just that it can be even better @sgospodarenko.

PPS: Include a check that the incoming DOP is “newer” than the database/DOP it is replacing in the ‘Uuid override’ situation and offer an additional option to continue etc… Development might take a little longer than my initial estimate which didn’t include formal design and agreement of design, scheduling of development, (development), localization, full testing, regression testing of a release, documentation (localized of course), packaging of the release and the release process itself to name just some of the steps from conception to being in the eager hands of the user.

Plus separate development, testing, release etc. for Win 10 and Mac versions!

@platypus while I believe that my proposals above are not difficult to implement, they require development from DxO and even if that is (hopefully) forthcoming in one form or another it does not solve the problem today or in the intervening period while we are waiting for such features! A lot was discussed in a similar topic How to use PhotoLab on multiple Apple Computers - #27 by John7 and @John7 had described his technique there, i.e.

and @platypus wrote

Unfortunately I would not recommend either of these approaches because although they will work I hate the idea of deleting real data under any circumstances because the onus is on me to get things right and not wind up trashing the data completely!

So will the following provide the solution slightly more safely?

Re-Import Procedure for PL5:-

  1. Change the name of the folders to be re-imported to e.g. -saved 2022-01-18 (i.e. the date of the operation within PL5. All images DOPs and sidecars will be maintained on disk and in the PL5db. The data is intact in all environments except a DAM and/or other file editors you may use which have their own DAMn database etc. etc. but this is supposed to be a temporary operation and what I am proposing is no worse than the wholesale deletion of the data!

  2. Terminate PL5 (not actually necessary but better safe than sorry).

  3. If all the data from the other computer is to be added wholesale, i.e. it is the latest version and nothing is required from the old directory then simply copy the new data to the system with the original name.

  4. Restart PL5 and navigate to the original (original name which now has the new data) directory and it should re-import the data it discovers.

  5. Verify that everything is as it should be and delete the (previously renamed) saved data at you leisure by deleting from disk, PL5 does not currently possess a directory deletion command.

Things are slightly more complicated it you want/need to mix and match data from more than one directory from more than one location because you started working in PL5 on your return and forgot to import the work from the laptop etc. etc.

  1. For maximum safety you could (should) copy the incoming contents to another location on the machine to be updated.

  2. Using your favourite comparison software (Beyond Compare in my case) compare and contrast the two directories, the original to be replaced and the new to be used and copy anything from the old to the new that is required and that must include the photos, the ‘xmp’ sidecar and the DOP sidecar. This then gives the new content that you wish to add to PL5 in a consolidated form and leaves the original and the new still intact ( so the process can be repeated as many times as required until it is it right!!)

  3. Execute the Re-import procedure for PL5 described above.

This process does require “swing space” on the receiving machine, on the main machine when importing from an “away trip” and on the laptop when preparing from an “away trip”.

Re-Syncing Metadata:-

But I have a problem with part of @John7’s original statement

I should have picked it up at the time but I have only recently returned to testing keywords that I hadn’t done for a long time when the original posts were made, principally about later DAM changes to keywords etc. not being picked up by PL5 and forcing PL5 to “catch up” with these changes by the deletion process.

If the ‘Sync’ option is set in the ‘Preferences’ PL5 will automatically pick up any such changes when you navigate to the specific directory (possibly only the specific photo - needs checking) and there is a warning given when you set the ‘Sync’ option about the possibility of data falling down a “crack”.

Alternatively select the photo or all the photos in the directory in PL5 and use the ‘Medata’ ‘Read from image’ command which should refresh all the metadata using the sidecar ‘xmp’ for RAWs and the embedded ‘xmp’ for JPGs etc or RAWs if you are using the DAM to update the RAW itself (I have done tests on that using Photo Mechanic). WARNING:- This is NOT a perfect strategy, I repeat that this is NOT a perfect strategy, I repeat ….

It is not a perfect strategy because it can (will) overwrite any keywords that have been added to PL5 that have not been automatically “flushed” to the external sidecar/embedded ‘xmp’ (please note that PL5 only uses the sidecar for RAW files) as a result of the the ‘Preferences’ ‘Sync’ option, or manually “flushed” to the external sidecar/embedded ‘xmp’ using the ‘Metadata’ ‘Write to image’ command.

Any left in PL5 that have not been “flushed” will be lost with the ‘Metadata’ ‘Read from image’ command - YOU HAVE BEEN WARNED sorry about the use of capitals and bold but it is a trap that could catch the unwary (probably me when I forget my own advice!)

‘Rating’ changes should be in the DOP but also in the ‘xmp’ but ‘Tag’ data is only stored in the DOP and a ‘Sidecar’ ‘Import’ should sort those out but they are only set by PL5 itself!

Keywords in the DOP are not currently used (who knows what might happen in the future)

It is possible @John7 that you had problems with getting PL5 to accept or recognise the changes but we did not have an opportunity to explore the issue because I failed to question why you had to execute your strategy. PL5 has never “played nicely” with ExifPro and refuses to automatically recognise any changes made to keywords (to JPGs) by that program because it ExifPro makes those changes without changing the ‘date last modified’ file timestamp.

In most tests I have been conducting (PL4.3.6.32 Ghosting Ratings for Same image in different directories (Win10) - #114 by BHAYT things work well with PL5 but I have not tested Photo Supreme which I think you have mentioned in one post or another.