Very simple photo editing question

[Greg7579]

I was quite happy with the 2017 Dell XPS desktop. It wasn't lightning fast with GFX 50R files, but it was fine. When I upgraded to the 100S, it became almost unusable for Lightroom editing. After a lot of masking, it would freeze for up to 30 seconds. While it was processing. I had to replace it in order to get acceptable editing performance.

Rob, I bought a fully loaed XPS laptop in 2017. By 2021 it was a slug. So I bought another XPS 15 in 2021 and it was the best you could get at the time at around 3300 bucks. It's getting a bit sluggish in LR / PS with the AI stuff.

So next week I'm ordering the new the new Dell XPS 16 9640 with the new Meteor Lake chip - the Intel Core Ultra 9 185H. I'm getting 64 RAM, 4080 GPU & 16-inch 4K OLED Touch screen.

A 2017 (or even a 2021 laptop) will really struggle today to do anything in LR with GFX files.

I've always said that if you are going to be a GFX shooter and spend all of that money you need a pro 4K monitor and you need a new mid to top-end desktop and laptop.

Do you have to do that? No. But you will pay a heavy price if you don't and will be making all kinds of compromises and not enjoying the full extent of your GFX or Hassy shooting.

 

[Rob de Loe]

It's a never-ending cycle... This 2023 Dell XPS desktop is holding its own so far.

 

[Rand 47]

Greg,

Sent you a PM.

Rand

 

[DavidMillier]

Not arguing for anything, just saying that it is not a simple matter. There are many factors involved and identifying bottlenecks is not always easy.

I expected my new computer to be substantially faster, but it doesn't appear to be (although it is fast enough). What that means is more RAM is no use unless you are RAM bottlenecked. And a faster CPU is no use unless you are CPU constrained and so on. The days of every generation of hardware being 3x faster than the previous gen are long behind us. Incremental gains. And watch out for software bottlenecks.

--
Photo of the day: https://whisperingcat.co.uk/wp/photo-of-the-day/
Website: http://www.whisperingcat.co.uk/ (2022 - website rebuilt, updated and back in action)
DPReview gallery: https://www.dpreview.com/galleries/0286305481
Flickr: http://www.flickr.com/photos/davidmillier/ (very old!)

 

[DavidMillier]

Greg's solution for everything is another 10 grand

 

[Rob de Loe]

Not arguing for anything, just saying that it is not a simple matter. There are many factors involved and identifying bottlenecks is not always easy.

I expected my new computer to be substantially faster, but it doesn't appear to be (although it is fast enough). What that means is more RAM is no use unless you are RAM bottlenecked. And a faster CPU is no use unless you are CPU constrained and so on. The days of every generation of hardware being 3x faster than the previous gen are long behind us. Incremental gains. And watch out for software bottlenecks.

Am I right that you're using Linux? I have Linux on an old laptop that is grossly underpowered now for Windows. I'm not doing any photo editing on it; its function is simply to give me a way to remote into my desktop from beside my printer. However, I have noticed that this old laptop runs just fine with Linux. It's fast and responsive.

 

[Rand 47]

Not arguing for anything, just saying that it is not a simple matter. There are many factors involved and identifying bottlenecks is not always easy.

I expected my new computer to be substantially faster, but it doesn't appear to be (although it is fast enough). What that means is more RAM is no use unless you are RAM bottlenecked. And a faster CPU is no use unless you are CPU constrained and so on. The days of every generation of hardware being 3x faster than the previous gen are long behind us. Incremental gains. And watch out for software bottlenecks.

Great observation. Look for interviews with Matt Bach from Puget Systems. He’s head of their research lab. Talks a lot about how difficult it is to keep up with all the software programs “creators” use and how there’s no “one answer” re where bottlenecks show up. I just ordered a new high-end box from them that in some ways is WAY over spec’d for “this moment” for PS and LrC. But the trend from Adobe (and almost everyone else) seems to be more utilization of “all” of the various components. CPU, GPU, RAM, speed of SSD’s. I’m hoping that my current new box (will arrive Tuesday, hurrah) will future proof me for a little while.

On my current box, I can turn my NEC PA 301 monitor black for a few seconds at a time when I have HUGE files from GFX (focus stacks, panos, lots of masks and or layers). For most things it is still marginally adequate, but far from quick. It was state of the art 4.5 years ago.

Rand

 

[DavidMillier]

Not arguing for anything, just saying that it is not a simple matter. There are many factors involved and identifying bottlenecks is not always easy.

I expected my new computer to be substantially faster, but it doesn't appear to be (although it is fast enough). What that means is more RAM is no use unless you are RAM bottlenecked. And a faster CPU is no use unless you are CPU constrained and so on. The days of every generation of hardware being 3x faster than the previous gen are long behind us. Incremental gains. And watch out for software bottlenecks.

Am I right that you're using Linux? I have Linux on an old laptop that is grossly underpowered now for Windows. I'm not doing any photo editing on it; its function is simply to give me a way to remote into my desktop from beside my printer. However, I have noticed that this old laptop runs just fine with Linux. It's fast and responsive.

Yep, Ubuntu 22.04 LTS with darktable as my raw convertor/editor/everything but printing which I do on a separate win10/LR 6.14 because it works better.

 

[Greg7579]

Very true. LR has some legacy bottlenecks where SSDs are no faster than the old HDDs on certain tasks that don't yet take advantage of faster ports and drives. Hopefully Adobe if fixing that.

 

[JimKasson]

Rob, I bought a fully loaed XPS laptop in 2017. By 2021 it was a slug.

As we used to say a long time ago, "Andy giveth, and Bill taketh away."

 

[DavidMillier]

Rob, I bought a fully loaed XPS laptop in 2017. By 2021 it was a slug.

As we used to say a long time ago, "Andy giveth, and Bill taketh away."

Interestingly, the darktable manual provides unusually technical performance tuning advice. That's nerdy open source for you!

memory & performance tuning

🔗memory requirements Processing a Raw image in darktable requires a great deal of system memory. A simple calculation makes this clear: For a 20 megapixel image, darktable requires a 4x32-bit floating point cell to store each pixel, meaning that each full image of this size will require...

docs.darktable.org

 

[paulraphael]

I tried a hybrid approach: working with uncompressed files on a scratch volume on my system SSD, and then saving final file compressed to my main media volume. It worked, and made sense on paper, but ended up being too much of a hassle for me. So I'm back to working with compressed files and just dealing with the wait.

I keep the compression disabled, work on my edits. When all the edits are done, when I know I am not going to edit them anymore, I open the PSB files, I compress them, get back the extra space they were taking. And then I archive that project. This is my hybrid approach.

Right, this is what I tried also. I lacked the patience for it. But it makes good sense.

Another argument would be in favor of archiving files in an uncompressed state, because 1) they will be less vulnerable to damage from bit rot and minor disk errors, and 2) there's less chance of discovering one day that there's no software around anymore that can read them.

In fact, it wouldn't be a terrible idea to create an unlayered, uncompressed archive copy (in generic .tif format) for every image you care about.

This could be a time-consuming and expensive proposition, unfortunately.

 

[Greg7579]

Greg's solution for everything is another 10 grand

Hahahahahahaha..... That is pretty funny.

 

[Greg7579]

Greg,

Sent you a PM.

Rand

Rand, that new rig you bought is going to sing and dance.... Perfect specs and the best way to do it.

All of these fantastic new screaming fast SSDs are mostly 4 TB and under. If you are like me and want all your files on one disk, the few 8TB m.2 PCIe 4 models that exist are expensive. Don't be afraid of a SATA 8 TB SSD like the Samsung. They are slow for SSDs but still 4 times faster than spinning rust and way more compact and reliable.

They go on sale from time to time and are around 500 bucks today. That is 4 times higher than a spinning 8 TB rust drive, but I have 3 of them stuck inside my new desktop case. You can add one inside your case very easily (SATA power cable from the PSU and data cable from the motherboard). Or spend 800 bucks and get an 8 TB PCIe 4 M.2 drive and mount it on the extra M.2 slot on the motherboard.

For external backup, keep an eye on the Samsung 8TB T5 Evo.

8 TB M.2 drives are absurdly expensive. Four times more than the 4 TB versions. That is because currently it is hard to get beyond 4 TB for some technical and production reasons, but that is about to change.

For example, the 2TB T5 Evo (external drive) costs about $120 and the 8TB T5 is priced at more than five times that amount.

Unless you want or need all your storage in one single drive, you could pick up two of the company's top-end 4TB T9 portable SSDs, which are rated to more than four times the speed of the T5 Evo, for around $500 – $150 less than the single 8TB T5. Two 4TB T7 Shields (also a much faster drive) will run you about $400.

 

[Rand 47]

Greg,

Sent you a PM.

Rand

Rand, that new rig you bought is going to sing and dance.... Perfect specs and the best way to do it.

All of these fantastic new screaming fast SSDs are mostly 4 TB and under. If you are like me and want all your files on one disk, the few 8TB m.2 PCIe 4 models that exist are expensive. Don't be afraid of a SATA 8 TB SSD like the Samsung. They are slow for SSDs but still 4 times faster than spinning rust and way more compact and reliable.

They go on sale from time to time and are around 500 bucks today. That is 4 times higher than a spinning 8 TB rust drive, but I have 3 of them stuck inside my new desktop case. You can add one inside your case very easily (SATA power cable from the PSU and data cable from the motherboard). Or spend 800 bucks and get an 8 TB PCIe 4 M.2 drive and mount it on the extra M.2 slot on the motherboard.

For external backup, keep an eye on the Samsung 8TB T5 Evo.

8 TB M.2 drives are absurdly expensive. Four times more than the 4 TB versions. That is because currently it is hard to get beyond 4 TB for some technical and production reasons, but that is about to change.

For example, the 2TB T5 Evo (external drive) costs about $120 and the 8TB T5 is priced at more than five times that amount.

Unless you want or need all your storage in one single drive, you could pick up two of the company's top-end 4TB T9 portable SSDs, which are rated to more than four times the speed of the T5 Evo, for around $500 – $150 less than the single 8TB T5. Two 4TB T7 Shields (also a much faster drive) will run you about $400.

Greg,

Thanks, mucho, for the feedback and advice on the backup drive situation. Much appreciated.

Rand

 

[paulraphael]

Don't overlook enterprise u.2 SSDs. They are superior to m.2 in robustness (endurance, quality control, resistance to thermal throttling, power loss protection) and capacity (how high do you want to go?)

At full retail they're stupid expensive. But you can find deals for less than 1/2 price on overstocked previous generation drives. The gen-3 drives are capable of 3.5 gb/s read and write, which will saturate any Thunderbolt or USB-4 connection, and is well past where you'll see gains with either Photoshop or Lightroom.

If you use a Mac, many of them are incompatible, for unknown reasons. The Micron 9300 series work perfectly and are available for relatively cheap right now in the 7.68 TB size.

I have one of these in an OWC Helios Thunderbolt enclosure with active cooling. It inspires much more confidence than a passively cooled consumer sata or m.2 solution. This is for my media drive; there's no reason to pay for performance with backup drives.

 

[JimKasson]

Greg,

Sent you a PM.

Rand

Rand, that new rig you bought is going to sing and dance.... Perfect specs and the best way to do it.

All of these fantastic new screaming fast SSDs are mostly 4 TB and under. If you are like me and want all your files on one disk, the few 8TB m.2 PCIe 4 models that exist are expensive.

Use arrays. I have 8 4TB M.2 SSDs in a SoftRAID striped configuration on a PCIe card for 32 TB that appears to the OS as a single disk.

 

[Greg7579]

Greg,

Sent you a PM.

Rand

Rand, that new rig you bought is going to sing and dance.... Perfect specs and the best way to do it.

All of these fantastic new screaming fast SSDs are mostly 4 TB and under. If you are like me and want all your files on one disk, the few 8TB m.2 PCIe 4 models that exist are expensive.

Use arrays. I have 8 4TB M.2 SSDs in a SoftRAID striped configuration on a PCIe card for 32 TB that appears to the OS as a single disk.

 

[paulraphael]

Use arrays. I have 8 4TB M.2 SSDs in a SoftRAID striped configuration on a PCIe card for 32 TB that appears to the OS as a single disk.

No Raid for me James....

I'm with you on that. For my media drive I want simplicity and robustness above all.

Even with redundant backups, a failure of that volume would be a huge disruption, and I just don't want to worry about it. That's why I've gone the enterprise ssd route.

And also (mercifully) I don't need 30TB. If that were to change, I'd consider splitting it to two volumes, like a high-performance active projects drive and a big RAID-6 (or similar) archive with spinning disks or cheapish SATA ssds.

 

[JimKasson]

Use arrays. I have 8 4TB M.2 SSDs in a SoftRAID striped configuration on a PCIe card for 32 TB that appears to the OS as a single disk.

No Raid for me James....

I'm with you on that. For my media drive I want simplicity and robustness above all.

Even with redundant backups, a failure of that volume would be a huge disruption, and I just don't want to worry about it. That's why I've gone the enterprise ssd route.

I keep all the data on my 32 TB SSD array mirrored to a RAID 5 SSD array accessible over 10GbE. If the main drive fails, I can assign the NAS drive the D: identifier and everything works as before, but a bit slower.

And also (mercifully) I don't need 30TB.

One run of the graffiti images takes a couple of hours and produces about 1.5 TB. A thousand images at 1.5 GB each. Of course, I throw most of those away.

If that were to change, I'd consider splitting it to two volumes, like a high-performance active projects drive and a big RAID-6 (or similar) archive with spinning disks or cheapish SATA ssds.