Raw + Metadata With DPL + DAM convert in HEIC = THE sustainable solution?

Hi,
According to you, it would be possible / useful to use the HEIC format for :

  • convert raw files to obtain generic, portable, durable and high quality files
  • keep in the same file the metadata (which does not allow Raw - without .xmp).

For years I have been looking for a permanent, open system, independent of any software that allows me to guarantee the reading of the metadata included in the photos while preserving the original photo.

HEIC files will soon be supported by DPL.
If the HEIC format allows to answer the two previous problems I could start to enter the meta data with the DPL DAM.

According to you, it would be possible / useful to use the HEIC format for:

  • convert raw files to obtain a generic, portable, durable and high quality file
  • keep in the same file the metadata (which does not allow Raw - without .xmp).

For years I have been looking for a permanent, open system, independent of any software that allows me to guarantee the reading of the metadata included in the photos while preserving the original photo.

HEIC files will soon be supported by DPL.
If the HEIC format allows to answer the two previous problems I could finally start to enter the metadata via the DPL DAM.

I have several thousands of photos and I have never started to write metadata because until today I have not found a lasting solution.

I don’t understand what you are saying here. RAW files can already contain all sorts of metadata in the EXIF part - including keywords in either the XMP or IPTC sections of the EXIF data in the file.

PhotoLab doesn’t yet read or write keywords to the EXIF data but it could.

You should be aware that HEIC is not open source; its use becomes subject to the licensing of patents on the coding format. https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format

I don’t understand why you are waiting to deal with metadata when adequate open standards (incl. XMP) are available. If your concern is dealing with managing sidecar files (which include .dop, .xmp, etc.), you’ll find that a good DAM program can handle that transparently (I use IMatch https://www.photools.com/ which depends on open source standards, but there are others). As Joanna points out, RAW files already contain all sorts of metadata, both in the EXIF part, but also in various proprietary metadata tags. PhotoLab will certainly use XMP (in raw sidecar files and in image files such as jpg) for keywords. In fact, I understand the Mac version is already doing this to a limited extent.

I want to use a thesuarus (keyword tree with notion of hierarchy) to put the keywords in the photos in order to be able to do research at several levels.

I think the exif are not the right place for this and I do not like the idea to have a sidecar file next to the photos. I find there is a lot of risk of data loss in the long run.

I wish I could take a picture of a quetzal and one other of a swallow, put these two key words in the photos metadata and be able to find them looking for “bird”.

At the risk of repeating myself, this is easy with a decent digital asset management program. And a decent DAM will transparently manage sidecar files so there’s no significant danger of data loss (or, no more than the risk that already exists of losing image files from mishaps - that’s why data backups are essential). See this https://www.photools.com/imatch/ for an illustration of what a DAM can do, even if you have no interest in this particular program.

I am currently doing a head-to-head test between IMatch and Photo Supreme and so far it is complicated. My main headaches come from my RAW library which is 50-50 .NEF and .DNG (My earlier RAW files which I converted and no terribly regret). There are differences between where and how metadata is stored. When I for example work with Capture One, metadata shows up that doesn’t show in DAM software and when I update certain things they don’t sync properly for the .DNG files (which I believe use embedded XMP) while .NEF uses external .xmp files. There are some settings in both Photo Supreme and IMatch to dictate how and where stuff is written but I have the idea that Capture One has a weird way of handling metadata as well.

I am going to run some tests with DXO which should fair better since it doesn’t do any metadata writing as far as I know so it won’t interfere with the DAM. My idea is to manage almost everything in the DAM but I would like to be able to update ratings and color labels in C1/PhotoLab (not possible yet).

TLDR; metadata are terribly complicated and you quickly end-up with synchronization problems or big mess ups where you overwrite proper metadata, or with “shadow” data (like IPTC ratings) that doesn’t show in some programs but it still there, and then suddenly pops up in another application and then messes with the entire system. Allowing the DAM to be the only application to write metadata might be the best solution ultimately.

Hello Floris,
I was in a similar situation like you a few months ago. The developer of Photo Supreme is extremely helpful and quick in replying. I have changed from Media Pro to Photo Supreme finally but in the testing phase I had an e-mail exhange with this guy - a bit more than 250 mails;))) altogether. Moving from one system to another can become complicated. If certain data does not show up in PSU try select the files/files and choose “Convert Metadata to xmp” that usually does the trick.

As said above send off an e-mail to support of PSU

I think both Mario (IMatch) and Hert (Photo Supreme) are very approachable and helpful. Photo Supreme is definitely crashing more for me, especially when batch detecting faces, something that doesn’t happen in Lightroom using a plugin with the same API (it works fine when I do it one image at a time). The other thing I don’t like about Photo Supreme is the incredibly poor documentation. IMatch is so well documented in comparison. For Photo Supreme, there is only a few help documents as PDF available and they don’t cover 20% of the features and rarely give usage examples… took me forever to figure things out since it is mostly trial and error (e.g. searching, filtering, how to handle metadata etc.).

Interface wise I think Photo Supreme is quite nice, a bit lighter and easier to navigate. So for the moment I am leaning towards IMatch since stability is something I crave after all the Lightroom memory leaks and sluggish performance. I am still amazed how compute intensive the proper handling of metadata is apparently (judging on the noise my CPU makes when batch updating files).

Yes, exactly.

ps - Yes, Mario and in fact the entire IMatch user community are very supportive.

I work with windows 10 but I had the same kind of problems.

That is why i am looking for a way to find

I think a lot of users would like to incorporate metadata into their photos so that they can be more easily selected but give up because of incompatibility and durability issues.

Yes, but someday it may be necessary to change the DAM and that day the problem will start.

There should be APIs / tools to customize the import and export of metadata.

All this is undoubtedly utopian.

I thought I would use the DPL DAM but I’m not sure it’s a good idea.

Which DAM do you recommend under windows to simply manage the metadata?

The best way to protect your metadata long-term is to follow open standards, including the work of the Metadata Working Group, and avoid proprietary standards that lock you in to a particular product. IMatch is based on use of open standards. If you later decide to use a different DAM, you can simply transfer your images (and their metadata) to the new system.

What makes metadata difficult is that many of the players (including camera manufacturers) do use ‘secret’ and sometimes encrypted tags even in things like EXIF metadata (so-called maker notes). Thanks to people like Phil Harvey, the creator of the excellent ExifTool, much of this data has been unearthed and is available to users in over 14,000 documented metadata tags (most folks of course only need a small subset of these!). (IMatch uses ExifTool to help manage metadata). And although there are groups that attempt to standardize metadata (e.g., the aforementioned MWG), too many of the influential players in this space prefer to keep things proprietary or otherwise disregard standards.

In any event, all this is a major reason why I use IMatch for my DAM.

[Note: Metadata for still images is a walk in the park compared to the nightmare of metadata for video…]

Yes I like their approach. But what is giving me a headache is that I want to review my images in another application. I like working with Capture 1 sessions where I then rate/label my images and do all the culling. Once that is done, I start adding metadata. The problem is that for those labels/ratings to show up in other applications, I have to “sync metadata” in Capture One but this is a blackhole that creates all kinds of problems since Capture One is terrible at handling metadata.

This means I can’t do my culling in Capture One and need to use either the DAM (I don’t find them that great, especially IMatch in terms of reviewing images) or something else again (e.g. FastRawViewer, but that also has no good way of comparing multiple images, neither does PhotoLab).

Are you using the IMatch Viewer? It has quite a few options to better customize for your individual use (including multiple images).

What’s the advantage in converting raw files to HEIC?

I’ve been using Photo Supreme + DxO since Apple killed off Aperture (IMatch isn’t an option for Mac). Hert has been exceptionally responsive and helpful with issues that came up using Photo Supreme. One handy feature in Photo Supreme is that it reads DxO sidecars and applies some of the edits (rotation, cropping, etc.) when generating thumbnails.

I think it’s best to stick with a single program for writing metadata to images. The problem is that there are numerous fields that serve the same purpose (e.g., both IPTC and XMP have caption, title, etc. fields). It’s easy for data to get out of sync if different programs choose different fields or sets of fields to update when making a change. I’ve had to go in with ExifTool on multiple occasions to clean up messes when fields got out of sync (amazing how many problems lingered for years after transitioning from Aperture to Photo Supreme).

have a look here as a start: https://heic.imobie.com/heic-and-jpg.htm

That’s HEIC vs. JPEG. Why HEIC over raw? Once you’ve converted raw to HEIC, you’ve demosaiced the data, removed defective pixels, applied white balance & noise reduction, picked a color space, etc. These are not reversible processes, and when improved raw-conversion algorithms come back, you won’t be able to benefit from them. A big part of what makes DxO so good is their raw-processing pipeline and the continual improvements they make in that area!