Database Maintenance

I would like to see explicit options related to database management – mainly location and backup control. In addition, it seems desirable to have a process for checking the integrity of the database.

I appreciate having the sidecar files with all of the current edit settings and wouldn’t want to lose that option.

Thanks.

1 Like

Please add the following:

2 Likes

Isn´t it just what “Index a folder” will do? You can do it for just one folder or every single folder under “the top folder” of your whole folder hierarchy (if you have one)

TIFF, JPEG and DNG are the exception and not the rule. All of them are together with some PDF-formats XMP-compatible. All sorts of RAW are not like any other type of file. There is old fashion IPTC and there is XMP where you can chose for example in Photo Mechanic to write IPTC even to an embedded namespace called IPTC. There are others like Dublin Core and EXIF too and you can even make your own company specific namespaces in more sofisticated DAM-solutions. There is no point in giving the photographer an option to create sidecars to XMP-compatible files.

So the standard is to write to XMP-sidecars wether it is RAW we are talking about or any other file type.

DXO have created some sort of a headache with Photolab of historical reasons. From the beginning there was no monolitic metadata database in most other RAW-converters - just in Lightroom and in Lightroom Adobe introduced a great problem - a single point of failure that have hit me a couple of times with corrupted databases/catalogs. Adobe made a big mistake making it default NOT to export XMP-sidecars - it´s just an option if you want to sclae up an migrate to a real DAM.

Capture One though has always relied on a database too but from the beginning it was per session because CO was session oriented as more of a studiosolution used even for tethering. When they thougt the Lightroom way was the way to go they more or less just implemented the session database as a monolithic database as an alternative.

Lightroom has been around a long time now and the monolitic database approach in Photolab is really 1.0 in PL5 so of course DXO has a few things to do with the database management but still I don´t see anything wrong with the database (as long as XMP-sidecars are supported and kept in sync) because it makes a few things possible, more rational and faster than without it and you don´t miss the history before you need it!

There is no way to get rid of the history what I can see BUT you can definitely ZERO all impact on as many images you want with one single click just with the RESET-buttom in the right corner of the interface. Is that a problem then with a present but inactive history you can call back to any reset point? Isn´t that an asset? I don´t really care it it´s there as long as the impact of the history is ZERO but it might be one idea in a future upgrade to erase orphaned history that has been nullified by using the Reset-funkction.

Another addition: Re-Indexing a folder should check if subfolders and images within still exist or have been added.

  • If folders or images have been added: Ingest (as is)
  • If folders or images are missing: Offer options to
    a) search for the items (applicable if renamed or moved)
    b) forget missing items (applicable if deleted)

Note: if the root folder of the photo collection is re-indexed, this can take a long time. Present a warning like “nnn items will be verified. This can take a long time. Continue/Cancel” (nnn could be taken from the database, verification against spotlight index or similar source in Win)

1 Like

There’s more to be added… vote here:

For the project I allready made a feature request to put project assignment within the dop file, that would allow to refresh it if recreating a new database from scratch. In addition it would allow to sync databases from multiple computers if your images are on a removable drive used on multiple machines.

Although this is an old thread and some of the requests have been implemented, a few vital features remain absent. The things I miss most are

  • configurable automatic compressed backups
  • ability to uncover and fix differences between DB and folder content
  • ability to remove files from DB without trashing the files (implies a manual sync feature)

I didn´t read all the thread, but the topic here says Database Maintenance
Which feature do I need?
Well, first my problem:
Today I was stucked in “Search for images” function. I indexed my folder with photos but when searching I got some pictures that are outside my indexed folder, the Keyword section that I don´t use was loaded some keywords and the pictures linked with those keywords are outside my indexed folder. So, what I tried to do is delete the cache to reset the indexed folders (as a user that was the common option I thought would work), but cleaning the cache didn´t work. Then I started to search for an option where I could manage the indexed folders and delete the undesired ones, but didn´t find anything. After that I searched for the database files hopping there was a file that I could remove only for the “Search for images” or Indexed folders, but no such thing.
Now I have to evaluate between maintaing the stored data, and when searching deail with all the garbage I don´t want to see in the results or create a new clean database and recreate all my projects with their respective pictures.
Last, trying to find more options I found this post and I thought this is the right place to share what I need and maybe more people need
What I need could be:

  1. Move the seach index data to the cache, so when cleaning, the stored cache for seaching dissapears
  2. A seccion in preferences or somewhere else to manage the indexed folders
  3. Separate the database file in multiple files to remove only those files I want to reset
  4. A way to see the content of the database in order to remove only what I want to reset
  5. You are the experts, so I believe you will find the best feature to cover this necessity

I See two things that I’d call “scope of search” and “selective removal of items”

Search: Based on items stored in the database, search can “find” items that have been removed or renamed without PhotoLab knowing about such changes. If DPL had a way to know what’s going on behind its back, it could remove or rename items and thus provide a sane basis for search.

Selective removal: If DPL offered a means for us to remove items without moving the actual files to the trash, the sanity of the database (and therefore basis for search) could be improved.

Automatic cleanup: Now, if DPL had a feature that checked the presence of actual items against what is stored in the database, with or without us having to manage that feature, DPL could keep the list of items listed in the database true to what files there really are.

“can” or “cannot”? The sentence to me indicates the latter.

If a folder is indexed by DPL and removed while DPL was not running, the database remembers the files that were removed. This means that DPL finds absent things. Without inventory control, inventory and reality drift apart. Supermarket employees have to check what’s in the shelves to update the inventory (database) regularly. I think that DPL should do this too - better sooner than later :wink:

Without inventory control, an empty shop’s inventory system can still thing that all is well…

Hmmm. Maybe it’s the wording? DPL doens’t “find absent files”, it just indicates a file as absent and then the user need to search and, at best, find the missing file/s. Or did I miss some sort of automated file search in the tools inventory of DPL?

And the “file is absent” status is also affecting non-absent files which were just renamed outside of PL (which I can understand, the renaming options are pretty poor).

What would this imply when doing search on keywords ? Reading the whole drive(s) to complete the search ?
So what impact on speed search ? (Speed already being a problem in many ways with photolab).

Which is why I wrote my keywording/searching software to leverage the macOS Spotlight database, which is constantly being updated as metadata is written to files.

The problem is that Windows has only recently started to catch up with this technology, so DxO has had to write their own, common, database, instead of relying on the operating system as it can for macOS.

My app takes less than a second to search amongst 14,500 images and, if I remove a keyword from one of the images in a search result, the Spotlight database is instantly updated and the image disappears from the results. But I have never needed to write any code to “manage” the database.

Not forgetting that Spotlight also provides near instantaneous searches for most camera metadata like shutter speed, aperture, ISO, etc.

Didn’t saw this thread last since several years !!! :crazy_face:
So seems I responded to something told several years ago :rofl:

So I don’t know where all this lead since.
This is an habit on this forum to have threads with thousan of posts … :upside_down_face:

Are you saying users should write their code to use photolab ? :face_with_peeking_eye:

1 Like

Would be only helpful for users on a Mac. Windows is too stupid for serious keywording on file level. In a way it’s funny: all couple of weeks @Joanna comes up with her “self-written” keywording app" helping nobody in a current situation. Good for her but a bit of useless information to others? I don’t know…

I agree with that for free open source softwares.
But for professionnal priced softwares … ???

Is it not enough that it’s priced professionally? :grin: Come on, all this expectations cost DxO just another smile :innocent: