Database Maintenance

For those of us who rely on the database, some database maintenance functionality might be helpful to keep DPL running smoothly.

I’d welcome the following functionalities

  1. Clear entries in database (incl. history) and cache per image or selection of images
  2. Clear “Advanced History” for more than one image at a time

Clearing database and cache entries per selection of images helps to keep the database lean, e.g. in cases where we want to scrap an editing session. As of today, we can scrap the cache, which eliminates all previews of whatever number of folders we’ve worked on. Still, all the ballast in the database remains. While cache cleanup is absolute, database cleanup is none, a strange combination imo.

All tools but “Advanced History” can be applied to multiple images at a time. Excluding AH seems inconsistent.

What is “to rely on the database” ??? And by the way, I don’t see any such option or concept in “Preferences”…

1 Like

Only the database contains all info about what has been going on with all those images that have been treated in DPL. The database is DPL’s brain. Deleting the database is DPL’s Alzheimer. Sidecar files only contain a subset (no history, no keywords), which means that deleting the database while keeping the sidecars secures the edits, but nothing else. I’d rather have it all - and be able to manage it all too.

As of today, there is no such functionality except for the “clear cache” part, that is why I post this as a feature request. You can support the request by clicking on the blue “vote” button next to the post’s title.

OK, but how could you delete the database, when Photolab — it seems —offers no interaction with it?

In Windows, in PL, look at the bottom of the General tab in Preferences and note the database location shown there. Then navigate to that location in File Explorer and then delete the files/folders.

I’ve no idea what the equivalent is on a Mac.

stuck

I do not want to delete the whole database but to have selectively removed entries that are related to a selected set of images. Because DPL offers no such functionality today, I request that such functionality be added.

The database used by DPL on Mac is indeed a bunch of files, deleting them will completely flash DPL’s brain, which is not what I’m looking for, see above. Imagine having worked on a few hundred folders after a while. Deleting the database voids all the work that has gone into those folders. All I want is to completely remove all traces of a (selected) few images in order to have a clean slate for a complete makeover. Today, I have to reset the images in question, manually delete the history of each image one by one and simply ignore that the cache is growing out of proportion.

Lightroom has some database maintenance functionality. In Lr, I can remove a folder from the catalog, tell Lr to optimize the database and then reimport the folder and all the images it contains are imported as if they had never been seen by Lr before. DPL cannot do such a thing today.

Good morning, guys,

Let me ask @StevenL to reply here :wink:

Regards,
Svetlana G.

Photo Supreme has also an ability to “optimize the database”. It will even remind you to perform that task after a certain amount of changes to the database. I am no expert but as far as I can read it does some cleaning up, whatever that means. It is especially useful after a big import, it speeds up the app noticable.

1 Like

IMatch behaves similar to Photo Supreme in this respect (regular reminder to “compact and optimize database”).

2 Likes

Hi there,
This is funny, because just a few weeks ago we has a discussion about such options/features.
As usual, I’m not allowed to share our timeline or if a feature will be implemented at the end…all I can say is that we are seriously evaluating options and thinking about it.

BTW, I’ll be more than happy if you all put here your inputs and ideas about “how it should work for you”.

Thanks!
Steven.

4 Likes

Hi Steven; I have just one requirement, which is not to make any change that would prevent (currently implemented) ability to ignore the database and, instead, to rely only on sidecar/.dop files.

Yes, I do understand that there are implications to this approach - such as inability to store keywords and/or multi-session correction history - but there is a good number of users for whom this is not a concern and who prefer the flexibility of sidecar files instead. I am definitely one such user (in fact, this is one of the features of OP/PL that originally convinced me to choose OP), and I know there are other veteran PL users who share this preference.

Regards, John M

8 Likes

It’s confirmed : I don’t understand what this thread is about! WHERE is this so-called « ability » implemented?

Just being able to index / update the database from a file or folder ?

Fundamental Settings

  • address differences between database and sidecar use
    -> add keywords to sidecars (user selectable setting)
    -> add option to save “0” steps to history (user selectable setting)

  • add controlled/controllable oblivion
    -> remove oldest cached previews after a (possibly configurable) time limit
    -> remove oldest cached previews when cache reaches a (possibly configurable) size limit

Database Maintenance

  • secure database health
    -> it is okay to run database optimizations on application quit
    -> keep a (possibly configurable) number of database copies for fallback

  • selective purge and rebirth
    -> selection of images can be purged through a menu item “purge and restart” that pops up the question whether other images’ sidecars should be exported or not
    -> folder can be purged through the same menu item without the popup question
    -> resync a folder or selection of images in order to pick up metadata (also from xmp etc.)

That’s it for the moment. Thanks for your interest.
Update log: added “rebirth” Dec.16th, 16:50

2 Likes

In addition to @platypus reply…

From what I can see, the database structure is/should be, essentially :

Capture d’écran 2020-12-16 à 11.46.38

Each Image is linked to one DopData, which contains all the edits done to the image.

You could argue that you still need this, even if you are exclusively using dop files, in order to cache those changes and avoid re-reading the dop file every time you display an image in the browser/library view.

For those who do not use dop files, this could be said to be the most precious part of the database.

For those who use dop files, this is irrelevant, apart from caching, and could be purged at any time and/or on closing PL.

Each Image can be linked to multiple HistoryItems

Unless you want to persist history between sessions, this should be optional and could be purged at any time and/or on closing PL.

Each Image can be optionally linked to multiple Keywords and each Keyword can be linked to multiple Images

Just because an Image doesn’t link to any keywords doesn’t mean that the Keywords list can be purged. They are useful for lookups as you are adding keywords to an image and PL should provide a tool to manage this list of keywords. Of course, only non-referenced keywords would be deletable, but you should be able to add/import whatever keywords you wish, for future reference.

Each Image can be optionally linked to multiple Projects and each Project can be linked to multiple Images

This should behave in much the same way as the Keywords relationship, except there would be no need to provide an explicit list management tool as this is managed by the present Projects system in the Library tab.

3 Likes

May i add that the database needs to be a function to speed up read write actions and refresh moment’s but in essence a copy of the external data in xml and dopfiles.
No unique info other then wastable temp data. (for instance editing data before switching images, which triggers a dopfile update.)

This is only fine if you don’t use Projects, Keywords or persistent History. If you don’t then, certainly, it should be optional

Somehow we need a function to export this in a seperate file when the database is corrupted. So we can delete and rebuild.
Seperate projects file/folder?
Do you need history to be saved at al time?

Keywords are image related so please keep them sticking next to the image as inside xml.

As someone who (currently) doesn’t use Projects or Key Words, or History, I’m in the camp of those who can manage quite happily without being critically dependent upon the database. I do recognise that, even with my way of working, the database and cache are beneficial, such as in avoiding the need to reprocess the DOP file every time an image is selected. My main concern is that if there was total reliance on the database everything (i.e. all corrections to all images) could be lost if the database ever became corrupted.

I value the flexibility of knowing that all I need is an image and its DOP sidecar in a folder in order to be able to view, further correct and export the image with PL.

For added functionality and future proofing I’d welcome the inclusion of keywords in DOP / XMP sidecars, and also History.

In short, while I suspect the ability to use PL without reliance on the database is not a “designed in” feature I’d be disappointed if future changes made it harder for me to continue working the way I do.

2 Likes

Certainly using an XMP sidecar for each image file that contains keywords would make sense.

However, Projects requires a two-way relationship and internal references to the Image, which can only really be managed in a database.