Open individual file rather than directory - impossible?

DxO PhotoLab
21

@md999 I am sorry to hear about your issues with PM Plus.

My directory is now made up of the combined JPGs I copied merged with the RAWs and is now 11,954 photos in total. I changed its name to avoid any “cheating” by the software and opened XnViewMP and navigated to the new folder (name) and off XnViewMP went importing all the thumbnails and any data that went with them, it seemed to run at a reasonable pace until about 45% (@ 2 minutes) when it started to slow, the next 10% took 1 minute it slowed further by 77% and almost stopped at 87% but got there in the end!

It took 12 minutes to add all the photos to its Catalog, currently residing at its default location on my C: drive (a 240MB SATA SSD). The photos are on a 6TB Toshiba 7200 drive and occupy 175GB. Processor utilisation for XNViewMP was 12-13% (I7-4790K @ 4.4, 4 cores + 4 threads) I/O about 17-18MB/s.

Reviewing the catalog in XnViewMP ‘Settings’ showed a direct import capability so I purged the Catalog and started again and the import took 9minutes and 30seconds but with no indication of any % at any point.

Refreshing the thumbnail view takes 10 seconds and how often that will happen depends on your navigation. At first sight it appears to provide the submission of multiple images that you require. The tagging function can span folders (just the one in your case) and tagged images can be isolated and then drag and dropped or passed in the ‘Open As’ function to PL5.

I have always looked upon XnViewMP as a utility rather than a heavyweight application but it has stood up to my throwing large numbers of photos at it as well or better than anything else! FSIV uses large amounts of I/O resources and time to import the thumbnails and is not really practical for directories of the size you need. FRV doesn’t attempt to “swallow” the whole directory in order to allow you to navigate and may or may not fit with your work style, free trial and then £22 for a licence. XnView Classic is as at least as bad as FSIV at importing the thumbnails.

XnViewMP is free but donations are apparently welcomed @£4.67 (inc VAT) per seat and I was offered 2 seats by default.

I hope that this helps.

EDIT:-
I tried the import in PL5 and it took around 10 minutes but these are photos with no editing attached so PL5 will be applying the default preset, Once real editing has been applied this import rate will simply decrease, i.e. the time taken will increase.

I have attached a snapshot of PL5 after an ‘Open with’ export from XnViewMP. I believed that Drag and drop from XnViewMP to PL5 worked in previous tests but when I repeated them I kept being taken to the directory and PL5 started the thumbnail refresh. The only difference is that in previous tests I had never opened the entire directory in PL5 before performing drag and drop!!

So if you ever want to drag and drop never open the directory directly in PL5, once it is in the database then drag and drop appears to result in the whole directory being opened every time!!

2021-12-22_092710
2021-12-22_0927101920×1080 195 KB

EDIT 2:
You might find this interesting (or not)

22

Genuine question: how do you find things in a directory of 10,000 files?

1 Like
23

With a good DAM, a good database it’s much easier than to open hundreds of folders, subfolders. And scrolling through thousands of previews handled in a database is also quicker than rendering RAWs.

Btw., the genuine question

already included that @md999 has no troubles finding his pictures. The trouble only starts once he likes to open a single picture in PL. And most of the contributions were about how different people work around the poor speed of PL’s way to check each file in a huge directory.

24

Oops! I missed this bit…

But given a database is in play, it shouldn’t matter at all where the files are, so there isn’t anything to be lost in putting them in more than one folder. If such folder structure were automated (as many tools can do) then it should make no difference.

I once tried a thought exercise in which I tried to determine what sort of folder structure I might use for files of a given type and realised there are two extremes:

  1. A single folder with files called, for example, 2021-receipts-household-utilities-electric-December.pdf
  2. A nested structure 2021/receipts/household/utilities/electric/December.pdf

I personally think the right answer for me is somewhere in between, but the whole exercise is moot if you can simply search for what you want. Then, the whole concept of what the file is called ceases to matter for the most part (unless you need to see/use it in isolation outside of its ‘home storage’ area).

My photos are in YYYY/MM folders and then the file names are just what came from the camera. If I know roughly when a photograph was taken I may choose to browse to find it, otherwise I will search on keywords. If I had them all in one single folder then browsing would not be an option.

1 Like
25

I really love your post @zkarj ! If a capable search lets me find my files, I do not need dedicated structures, refined keywords with hierarchies and all the other sha-bang. Being a Mac user I got so used to Spotlight searches, that I only use a super simple YYYY-MM structure. Unfortunately, apps developed for Windows, too, can’t rely on an “onboard search”, so everybody has to re-invent the wheel.

But when I read in the manual “you can search in folders but not in projects” then I get my first doubts if the devs live on the same planet like me…

1 Like
26

Mac users are spoilt by Spotlight, but it still cannot recognise people or places in the photos I took - and I’m quite happy with that. Keywords help to find content rather than files, therefore, I consider keywording to be essential for the sake of those who come after us.

Going through a box of photos that came to me because someone died, I was happy to find comments written to the the photo’s reverse by a person who obviously thought farther than her own nosetip.

But again, we’re venturing away from the OP’s question about opening individual files rather than folders…

1 Like
27

So true. As far as I remember, Iridient developer opens a single file and doesn’t need to produce previews for other files in the folder.

One doesn’t need to have a folder of 10.000 images, PL is already slow with 500 - by “slow” I just mean slower than scrolling through 500 ready made previews. I think, DxO needs to decide wether remaining an excellent RAW developer and focus on that only (like Iridient or Affinity Photo also do) or do something to become a better image manager.

But seeing the differences within the app package, the manuals and the different OS versions, I lost hope this will happen in the next future.

Sorry for again drifting off. But on one side there’s a lot of satisfaction in the user group to be not locked into a database system, on the other side users also struggle with the consequences of always need to wait until PL has generated the previews? Or fiddle around with different standards of keywording inside PL and PM and whatever additional source of bugs is involved.

28

A simple setting could help. Lightroom has it, but DxO seems to be too obstinate to make our lives easier. Software is meant to serve us, it’s not the other way 'round.

4 Likes
29

That’s the difficult part to get into the brains of devs… I understand they want to reach their goals: primarily to make the app work as flawless, fast and bug-free as possible. There just needs to be also few persons in the team to watch about usability and handling. Not to forget consistency and terminology.

2 Likes
30

…which serves the user…

I think that DxO needs to review its products for common language, operating concepts, shared resources etc. - and then implement it step by step, all while adding quick wins proposed by the users.

1 Like
31

I have images in month/year folders then once labelled move them to topic folders. The trouble is that over time a 3-4 of the topic folders become very large 10,000+ images. I can sort them by various aspects such as date or location etc in some quite fast programs or use the full database program to view them very quickly by whatever or search for individual images by any aspect of the keywords, title, description etc. An example is a folder called trees where I have about 15,000 images and not really sure how to divide this into more than one, if I do it taxonomically then it will be a total pain to put files into and to make web pages from (without a more complex system than I have for web pages).

As I mentioned I have now set Photomechanic to open DXO and get individual files. However two issues, one when I select each additional file the first one is lost from DXO and two I can’t see how to close files in DXO without closing the program itself. I have searched through the help and there are no references to ‘close’ other than to close menus or the program itself. I have used a number of photo editing programs and they all have a way to close files and have ways to have multiple files open at once so it seems very strange that DXO makes it so difficult to find a way to do this - assuming there is actually a way to do it.
Perhaps I should add that I am not trying to say bad things about the program as it does some things much better than other software which is why I am trying to work out a way to use it properly.

32

@md999 I am not an expert on PM but first are you using PM or PM plus?

I have tried passing more than one photo from PM to PL5 and, like a lot of other programs that I have tried only one photo from the group is passed!! I do not know how more than one can be passed from PM (i.e. what preference settings may be required/is there a special selection method that will result in more than one being passed etc.)?

If that worked for other editing software e.g. Lightroom then the issue is possibly with the PL5 Command Line Interface but as I found with XnViewMP and FRV it can be done by some programs, exactly what others I am not sure. I just opened 4 photos one after the other in PM and all four are available in the PhotoLibrary section of PL5 in the ‘External Selections’ . They can be navigated to and from and as many edits made as required as far as I know.

What I don’t know is whether you wanted all 4 over in one pass from PM to PL5 and what you are expecting back in the PM environment?

Perhaps @Stenis can elaborate but I only have access to a test copy of PM not PM Plus and I am no expert at to how to use the program.

With respect to “closing” a file, PL5 makes any changes (keywords etc. not edits) to the file for JPGs etc. as it goes along and updates the DOP as and when, for RAWs the DOP and the ‘xmp’ sidecar file will also be updated, you close a file by navigating away from the file/folder in PL5 or submitting another file from an external program to be edited, no explicit close is required?

One “problem” with PL5 is that it always starts with the folder/image/images last selected before the shutdown. That proved a bit of a nightmare when testing with the mega directory I created.

Please briefly (or as many lines as it takes) describe the workflow that you seek so that I can try to replicate it and see what I can see!!

2021-12-23_144013
2021-12-23_1440131920×1200 185 KB

EDIT:

Using multiple photos selected in PM (‘Edit selected photos with…’) does submit all the selected images to PL5 but they come in as {no of images} x 1 image entries in the ‘External selection’. Sadly this means that side by side comparisons between the images in PL5 is not possible.

Further EDIT:-
This just carried out on PL5.1.2 with PL5 shutdown submitting 9 photos results in only a single entry in the ‘External selection’ of PL5 PhotoLibrary. If PL5 is already running there will be 9 entries!

Opening XnViewMP using the same command in PM resulted in 9 separate XnViewMP windows being opened!

2021-12-23_180755
2021-12-23_1807551920×1200 251 KB

33

I don’t use Spotlight. For general files I have Alfred which is quicker, mechanically speaking, and lets me do more. But I was referring to DAM searching. If the photos are in a database, that database should offer searching.

On the assumption that PL cannot do what is required in this case, I was looking for alternative ways to attack the problem.

Agreed there is no obvious way to topically divide this without driving yourself crazy. This is why I use date-based folders. As I mentioned, if I want to browse by date I can, but a lot of the time I just use search in my DAM of choice (Lightroom Classic). Often I will do both… search for a photo and then when I find it, I will go to the relevant date folder to see others taken at the same time.

34

@md999 I have so far documented a variety of potential solutions in the forum but with no response from other forum members or you, which is a little disappointing?!

I believe that the issue with large directories is only relevant insofar as when it is not possible to pass a file (preferably more than one) to PL5 it then becomes necessary to open the entire directory in PL5 with all the problems that can cause , i.e. the time taken to open all the thumbnails in the directory which is further exacerbated because PL5 renders and re-renders the thumbnails with PL5 edits (and does this every time that the directory is opened and re-opened and traversed and …).

With respect to improving directory opening I believe that this could be improved if PL5 abandoned its current thumbnail caching scheme and replaced it with a full time SQLite database for the thumbnails, i.e. “persist” the thumbnails.

This would be complicated with PL5 because as photos are edited their thumbnail would need to be updated in the SQLite thumbnail database and worse, if a “rogue” DOP or DOPs were added to a directory the thumbnails would need to re-rendered in the light of these changes and even worse new DOPs being added might cause the introduction of Virtual Copy thumbnails which would need to be added to the thumbnail database and any other complications that you can think of. But it would improve general speed in PL5 and reduce the amount of spinning icons (if all the ramifications can be satisfactorily resolved)!

However, the database would be large and the whole thing becomes extremely complicated when thumbnail sizes as changed and what happens when the main PL5 database is destroyed (in my case mostly deliberately)? and…

I have a DigiKam system only up to date as of early 2021 which is now updating itself automatically so I will get a better idea of the issues of thumbnails in a SQLite database from that.

Failing such a radical new feature the only other alternative is to never browse the big directories in PL5 but do that browsing elsewhere and submit requests for editing directly to PL5. Now the problems really start because while some programs play nicely others do not or (vice versa) PL5 plays well with some programs but not with others, whichever way you want to present the problem, i.e. try to get your other supplier to change or get DxO to change or find a product that does all that you want with respect to management and searching but including (I would suggest) passing one or many photos from one or many directories to PL5 in a single submission!?

Some programs when passing a group of files will be greeted by PL5 starting to scan the entire directory, this also seems to happen with Photo Mechanic when drag and drop is used (at least in my tests).

Some programs when passing a group will actually have only one opened by PL5, better than nothing but …

Others packages will cause all the photos to be opened but effectively in multiple windows, in PL5 this manifests itself as one entry for every photo in the group with each photo represented as a single item in the ‘External selections’, they are all there - well sort of! With PM and PL5 if PL5 is shut down only one photo will make it to PL5 but with PL5 already running all the photos will be available but as separate selections making it impossible to compare and contrast etc. in PL5.

XnViewMP and Photo Supreme seem to “play” nicely and pass the list of files in a manner that results in PL5 opening all the selected files as members of the ‘same external selection’ where they can easily be reviewed and edited etc…

The above must come with a disclaimer that not all variations of the scenarios have been full explored by me, and that the results were actually what I thought I saw etc. etc.

**However, what would be really nice is if those forum users with DAM systems and other photo managers could use those products to take a selection of photos and pass them to PL5 in a single operation with PL5 already running and also with PL5 shutdown. We can then compile a list of those that “play” in any way, those that “play” nicely and those that do not “play” at all (e.g, nothing happens or PL5 starts reading the entire directory).

In addition it would be useful to know which programs work well with other editing products and how well or otherwise they also work with PL5. If there are many that do well with other editors but run into problems with PL5 then there is a real case to press for PL5 to be improved**

The tests do not need to be done on particularly large directories of files just as long as it is easy to determine whether the files passed have been opened versus the entire directory. The tests are pertinent to both Mac and Windows systems albeit it might work out that no MAC system has a problem or …

The next part of the equation is how useful are the programs available for executing metadata searches in order to come up with clusters of photos to be passed to PL5 for editing etc. BUT if there is no way to get the photos across to PL5 in the way that the user wants then are they really that useful or should another program be used or should we be campaigning for changes to PL5.

35

I can confirm that it’s easy to drag and drop a selection of images into PL5, thereby creating a new External Selection. I did this with IMatch, similar to what you’ve done with alternate software.

In IMatch it’s done by using the Favorites feature, which uses stored Windows shortcuts; once that’s done, any selection of images (including ones from different folders) can be dropped onto the Favorite, triggering creation of an External Selection in PL5 (or PL4 or earlier versions).

For some reason, DxO hasn’t documented how External Selections work (or even confirmed in its Help that this feature exists). PL5 is clearly using a standard Windows function (IMatch wasn’t customized for PL5, but rather uses standard Windows conventions, as, I assume, the software you used). This sidesteps problems with too many images, refreshing folder contents, etc. It makes it easy to create PL5 External Selections containing relatively limited numbers of images from multiple folders containing hundreds of images, using arbitrarily complex selection criteria. (IMatch easily handles databases with hundreds of thousands images, although my own database is a bit smaller than 100,000 images.)

I can also drag and drop a selection from a Windows Explorer folder onto the PL5 desktop icon. However, this simply opens the entire folder in PL5 with the selected images highlighted and it doesn’t create an External Selection that only contains the desired images.

If DxO would document the External Selections feature, it would make it much easier to accomplish the wishes of the OP, which are definitely not ‘impossible’.

1 Like
36

@jch2103 Thank you for your response I suspected that IMatch might be able to do something useful.

In truth drag and drop is a recommended solution for Photo Mechanic but this results in the whole directory being opened (at least in my tests and yes I could be doing it wrong). However, if it turns out that more and more products don’t work the way that they should then there is good cause to persuade DxO to be more open with their documentation and perhaps more open with their interface.

Have a good Christmas/ break/whatever you celebrate.

I finally took the plunge and bought myself a “Christmas present” of IMatch with a Christmas discount. It had better be as good as you and others on the forum reckon or I will be chasing you all for a contribution!!??

37

Photo Mechanic has an option for passing selected photos to other programs and when I do this for PL an External Selection is created, so it does work with other software too.

38

Enjoy your new Christmas present! There’s definitely a learning curve, but be sure to check the IMatch Learning Center at IMatch - Organize your digital images and other digital media files easily.. And the IMatch community at photools.com Community - Index (photools.com IMatch) is very helpful if you have questions or issues.

39

@KeithRJ Thank you for your response, I have successfully used PM to pass a number of photos to PL5 BUT I wind up with more than one external selection (1 for each in the list containing 1 photo or only 1 photo in total if PL5 was shut down) rather than one external selection that contains all the selected photos. Plus if I try drag and drop from PM to PL5, PL5 immediately opens the entire directory (I seem to remember that the selected items are also selected in PL5 but with the whole directory displayed)!!

It is possible that this is as a result of some configuration issues with the way that I have set up PM but…

40

@jch2103 I’ll try to enjoy Imatch but I could bought a medium priced cordless drill to add to my collection or 3 of them if I bought them from Lidl. You can never have too many drills!!!