Adobe scratch disk and Work file on the same m.2 drive?

Started Jan 25, 2020 | Discussions
Arazi New Member • Posts: 9
Adobe scratch disk and Work file on the same m.2 drive?

I'm trying to optimize my PC for Photoshop and Lightroom to work with very large image files from the a7rIV.

I installed an m.2 Samsung 970 Evo intending to use it as a scratch disk. Though considering its a 1TB drive, I was thinking to using it as a LR catalog and PS work file drive as well.

Will this diminish the drives efficiency as a scratch disk?

(unknown member) Regular Member • Posts: 214
Re: Adobe scratch disk and Work file on the same m.2 drive?

Back in the days, scratch disks were used to speed up your workflow by having temp data on a separate hard drive. That way your system files and applications were on you startup disk and temp data and user data each on their separate disk (regular hard drive). Nowadays with nmve / m2 ( or even sata ssd) there’s no need anymore for a dedicated scratch disk.

Though I would look into the option of storing all user data (i.e. photos and catalogs) on dedicated storage like a larger sata ssd or nvme storage.

This way you can easily make images of the startup volume for safety backups, if user data would be on the start up volume, the image would be rather large.

Backingup user data requires a different mechanism tha has been discussed on a regular base on this forum.

-- hide signature --

-e 
Ron

Ho72
Ho72 Senior Member • Posts: 2,555
Re: Adobe scratch disk and Work file on the same m.2 drive?
4

Arazi wrote:

I'm trying to optimize my PC for Photoshop and Lightroom to work with very large image files from the a7rIV.

I installed an m.2 Samsung 970 Evo intending to use it as a scratch disk. Though considering its a 1TB drive, I was thinking to using it as a LR catalog and PS work file drive as well.

Will this diminish the drives efficiency as a scratch disk?

If you have enough RAM, scratch is a non-factor. 32GB should be enough unless you plan to stitch large panos.

Morris0
Morris0 Forum Pro • Posts: 29,457
Re: Adobe scratch disk and Work file on the same m.2 drive?

Arazi wrote:

I'm trying to optimize my PC for Photoshop and Lightroom to work with very large image files from the a7rIV.

I installed an m.2 Samsung 970 Evo intending to use it as a scratch disk. Though considering its a 1TB drive, I was thinking to using it as a LR catalog and PS work file drive as well.

Will this diminish the drives efficiency as a scratch disk?

I don't agree with the previous answers and neither dose Adobe if you read there optimization guide.

The full guide:

https://helpx.adobe.com/photoshop/kb/optimize-photoshop-cc-performance.html

Scratch disk guide:

https://helpx.adobe.com/photoshop/using/scratch-disks-preferences.html

I've tried every combination and these are my findings:

With your system drive as an SSD, you get pretty good performance with no other drives.  Even with 64-GB of RAM, A separate paging drive and scratch drive will make your system snappier.  The ideal setup is:

System Drive: Very Fast SSD

Swap Drive (Paging Drive) any SSD, this drive is not used much yet windows prepages so it's always being hit a little bit and separating this IO from the system drive will allow for a slightly snappier system.

Scratch Disk: Very Fast SSD.  Remember to move Bridge and Camera RAW scratch to this drive as well

Photo Storage:  A large drive.  Note that this storage will impact your loads and saves.  Some people like to use an SSD for there work and then move it to a big hard drive or NAS for storage.  Another nice option is to use disk caching software.

Morris

 Morris0's gear list:Morris0's gear list
Fujifilm X-T3 Fujifilm X-H2S Canon EF 200mm f/2.8L II USM Nikon AF-S Teleconverter TC-17E II XF 90mm +11 more
Ho72
Ho72 Senior Member • Posts: 2,555
Re: Adobe scratch disk and Work file on the same m.2 drive?
4

Morris0 wrote:

I don't agree with the previous answers and neither dose Adobe if you read there optimization guide.

With regard to my own answer, you may not agree but I suspect it's because you don't understand. Yes scratch is required and it is always written to but the writes are not what slow things down, it's the reads which occur once RAM is full. Enough RAM means PS does not have to read from scratch, ever — or mostly so. Anyone can go nuts with open files and layers and fill up their RAM. I've done it myself just to see what would happen.

Yes, putting catalogs and cache on a fast drive is a good idea. A case can be made that, for some users, assuming a fast SSD is used for the system disk, a separate drive for catalogs and cache is not needed but most people choose to install one anyway.

Morris0
Morris0 Forum Pro • Posts: 29,457
Re: Adobe scratch disk and Work file on the same m.2 drive?
1

Ho72 wrote:

Morris0 wrote:

I don't agree with the previous answers and neither dose Adobe if you read there optimization guide.

With regard to my own answer, you may not agree but I suspect it's because you don't understand. Yes scratch is required and it is always written to but the writes are not what slow things down, it's the reads which occur once RAM is full. Enough RAM means PS does not have to read from scratch, ever — or mostly so. Anyone can go nuts with open files and layers and fill up their RAM. I've done it myself just to see what would happen.

Yes, putting catalogs and cache on a fast drive is a good idea. A case can be made that, for some users, assuming a fast SSD is used for the system disk, a separate drive for catalogs and cache is not needed but most people choose to install one anyway.

Oh, I understand at a level that most never achieve as I was a system programer, systems manager and one of my scalties was performance tuning.

"Yes scratch is required and it is always written to but the writes are not what slow things down, it's the reads which occur once RAM is full."

Those writes that are always happening are I/Os that take up space in the I/O Queue and must be completed before other I/Os to the same device can happen.  If these I/Os are for the drive that Photoshop is loaded on, then loading modules must wait.  Photoshop is constantly loading modules such as filters, and other tools as we work.  Being set up this way will reduce the snappyness of the system.

If scratch I/O shares the swapper, then there are two sets of constant I/O queuing for the same device and slowing response.

If scratch shares the data drive, then data reads and writes must get in line with the constant scratch I/O.

You are correct that Photoshop is constantly writing to scratch.  This prepaging like I/O allows for Photoshop to fetch data quickly when it's needed thus speeding up the system by preventing starvation.   It works best when it is isolated on it's own.

Best is relative, and it's a question of what level of responsiveness the individual who is using the system wants.

Morris

 Morris0's gear list:Morris0's gear list
Fujifilm X-T3 Fujifilm X-H2S Canon EF 200mm f/2.8L II USM Nikon AF-S Teleconverter TC-17E II XF 90mm +11 more
edispics
edispics Veteran Member • Posts: 4,408
Re: Adobe scratch disk and Work file on the same m.2 drive?

Morris0 wrote:

Arazi wrote:

I'm trying to optimize my PC for Photoshop and Lightroom to work with very large image files from the a7rIV.

I installed an m.2 Samsung 970 Evo intending to use it as a scratch disk. Though considering its a 1TB drive, I was thinking to using it as a LR catalog and PS work file drive as well.

Will this diminish the drives efficiency as a scratch disk?

I don't agree with the previous answers and neither dose Adobe if you read there optimization guide.

The full guide:

https://helpx.adobe.com/photoshop/kb/optimize-photoshop-cc-performance.html

Scratch disk guide:

https://helpx.adobe.com/photoshop/using/scratch-disks-preferences.html

I've tried every combination and these are my findings:

With your system drive as an SSD, you get pretty good performance with no other drives. Even with 64-GB of RAM, A separate paging drive and scratch drive will make your system snappier.

With respect to PS - after I upgraded to 32GB mem a while back I monitored what PS was writing to the scratch drive (I use SSDs). Even with the increased mem PS always writes some files out on startup. However, for the duration of the session, unless your work requires more mem that allocated to PS, PS does not write out to the scratch drive at all.

Correct that. I just tried the same thing using the current version of PS. Using the Resource Monitor, Disk tab, opening the Disk Activity section, I can see that whenever I open a file or make any adjustments, there is I/O to the scratch disk. Not much. But it is there.

The ideal setup is:

System Drive: Very Fast SSD

Swap Drive (Paging Drive) any SSD, this drive is not used much yet windows prepages so it's always being hit a little bit and separating this IO from the system drive will allow for a slightly snappier system.

Scratch Disk: Very Fast SSD. Remember to move Bridge and Camera RAW scratch to this drive as well

Photo Storage: A large drive. Note that this storage will impact your loads and saves. Some people like to use an SSD for there work and then move it to a big hard drive or NAS for storage. Another nice option is to use disk caching software.

Morris

Yup, I do the same

I would also store the various LR caches and cat on an SSD.

 edispics's gear list:edispics's gear list
Panasonic ZS100 Nikon D7000 Nikon D7200 Nikon Z6 Nikon D70 +25 more
Sean Nelson
Sean Nelson Forum Pro • Posts: 15,783
Re: Adobe scratch disk and Work file on the same m.2 drive?
1

Morris0 wrote:

Those writes that are always happening are I/Os that take up space in the I/O Queue and must be completed before other I/Os to the same device can happen.

By default Windows caches write operations and uses a "lazy write" strategy to flush the cache to disk, giving priority to queued read operations.   It's true that writes add to the overall disk load, but the application doesn't have to wait for them to complete before continuing the way it does for read operations.   So the impact of writes is a lot lower than it is for reads.

Morris0
Morris0 Forum Pro • Posts: 29,457
Re: Adobe scratch disk and Work file on the same m.2 drive?

Sean Nelson wrote:

Morris0 wrote:

Those writes that are always happening are I/Os that take up space in the I/O Queue and must be completed before other I/Os to the same device can happen.

By default Windows caches write operations and uses a "lazy write" strategy to flush the cache to disk, giving priority to queued read operations. It's true that writes add to the overall disk load, but the application doesn't have to wait for them to complete before continuing the way it does for read operations. So the impact of writes is a lot lower than it is for reads.

That queue is very small.  It's goal is to allow the elevator algorithm to reduce hard drive latency and of cause not usefull on an sad which is why some say to turn off delayed writes to sad.

I've benched the different conigs.   With a fast SSD and 16 GB of RAM or more one gets a very fast system and the other drives only make the system a bit snappier.  I love speed yet the returns diminish for the other drives.

Morris

 Morris0's gear list:Morris0's gear list
Fujifilm X-T3 Fujifilm X-H2S Canon EF 200mm f/2.8L II USM Nikon AF-S Teleconverter TC-17E II XF 90mm +11 more
Sean Nelson
Sean Nelson Forum Pro • Posts: 15,783
Re: Adobe scratch disk and Work file on the same m.2 drive?

Morris0 wrote:

Sean Nelson wrote:

Morris0 wrote:

Those writes that are always happening are I/Os that take up space in the I/O Queue and must be completed before other I/Os to the same device can happen.

By default Windows caches write operations and uses a "lazy write" strategy to flush the cache to disk, giving priority to queued read operations. It's true that writes add to the overall disk load, but the application doesn't have to wait for them to complete before continuing the way it does for read operations. So the impact of writes is a lot lower than it is for reads.

That queue is very small. It's goal is to allow the elevator algorithm to reduce hard drive latency and of cause not usefull on an sad which is why some say to turn off delayed writes to sad.

The size of the write cache can be fairly large. My system has 24GB of RAM and when I'm running my backups with nothing else going on I see the system typically buffering around 1 to 2GB of writes before starting to dump them to disk. Of course if you're memory constrained then the queue will be a lot smaller.

And write caching is even more comprehensive for temporary files. If the application creates a file with the temporary attribute, then writes are never actually sent to disk unless the system has memory constraints or the application explicitly requests it.

When you combine write caching with the synchronous nature of reads (applications reading data typically can't continue until the read completes), writes tend to be a lot less performance-critical to an application than reads are.

I'm not disagreeing with your conclusions, just trying to provide some insight for everyone who's reading the thread.

robertfel Senior Member • Posts: 2,002
This is old information
1

Morris0 wrote:

Scratch disk guide:

https://helpx.adobe.com/photoshop/using/scratch-disks-preferences.html

It's interesting how old timers insist on scratch disks and selectively read the guidelines.

"An SSD, on the other hand, performs well as both the primary startup and scratch disk. In fact, using an SSD is probably better than using a separate hard disk as your primary scratch disk."

It's simple math, your NVME SSD is operating at 3000 MBps, 2 SATA drives operating at 500MBps each won't have the speed of one NVME. That's for sequential read/write. For random reads it's a lot closer, but the advantage is still NVME.

You can have the OS, apps and data on a 1 NVME, it will be faster than splitting them across 3 SATA drives. 3,000 > 1,500. 1/10 - 1/50th of those speeds are for random reads, I wish drive makers would increase those speeds, then you'd notice how much snappier your system would be. Again, this is strictly NVME, if your m.2 slot if SATA, then yes, split up the OS/apps from data.

Now if you have the latest AMD boards with multiple PCIe 4.0 NVME slots, you might notice a difference splitting things up in benchmarks, but you'd never notice it in real life.

With 32GB RAM 99% of users will never access the Photoshop scratch file once it's written. It's easy enough to test this. Put your scratch file on an external drive with an LED. Once Photoshop opens, it will create the scratch file, but it will never access it again unless you're working on MASSIVE file with multiple layers. The LED will never light up.

 robertfel's gear list:robertfel's gear list
Nikon D800 Nikon D850 Nikon Z6 Nikon Z fc Nikon AF-S Nikkor 14-24mm f/2.8G ED +29 more
Sean Nelson
Sean Nelson Forum Pro • Posts: 15,783
Re: This is old information

robertfel wrote:

It's simple math, your NVME SSD is operating at 3000 MBps, 2 SATA drives operating at 500MBps each won't have the speed of one NVME. That's for sequential read/write. For random reads it's a lot closer, but the advantage is still NVME.

The real bottleneck isn't the transfer speed, it's the latency. Hard disks have to wait for data to come around to the read head, and they usually have to move the read head too. SSDs just read, immediately. So SSDs have latencies that are a couple of orders or magnitude (that's 100's of times) better than hard drives. That makes a huge difference for workloads that involve a lot of I/O operations per second.

You have to pile an awful lot of I/O operations onto one SSD before you get anywhere near to slowing it down to the speed of a hard disk.  It's something to worry about in a server environment where you might be servicing hundreds or thousands of client requests, but in a single-user environment the more active files you can put on your SSD, the better it will usually be.

edispics
edispics Veteran Member • Posts: 4,408
Re: This is old information

robertfel wrote:

Morris0 wrote:

Scratch disk guide:

https://helpx.adobe.com/photoshop/using/scratch-disks-preferences.html

It's interesting how old timers insist on scratch disks and selectively read the guidelines.

"An SSD, on the other hand, performs well as both the primary startup and scratch disk. In fact, using an SSD is probably better than using a separate hard disk as your primary scratch disk."

It's simple math, your NVME SSD is operating at 3000 MBps, 2 SATA drives operating at 500MBps each won't have the speed of one NVME. That's for sequential read/write. For random reads it's a lot closer, but the advantage is still NVME.

You can have the OS, apps and data on a 1 NVME, it will be faster than splitting them across 3 SATA drives. 3,000 > 1,500. 1/10 - 1/50th of those speeds are for random reads, I wish drive makers would increase those speeds, then you'd notice how much snappier your system would be. Again, this is strictly NVME, if your m.2 slot if SATA, then yes, split up the OS/apps from data.

Now if you have the latest AMD boards with multiple PCIe 4.0 NVME slots, you might notice a difference splitting things up in benchmarks, but you'd never notice it in real life.

With 32GB RAM 99% of users will never access the Photoshop scratch file once it's written. It's easy enough to test this. Put your scratch file on an external drive with an LED. Once Photoshop opens, it will create the scratch file, but it will never access it again unless you're working on MASSIVE file with multiple layers. The LED will never light up.

I'm all SSD, so no issue there.

However, you can do a lot better than try to read the LED on your drive. Go into Resource Monitor, click on the Disk tab and go to the second panel which lists all IO on your system. If you sort on the name it will group all filenames with the same drive letter together. I use one SSD for PS scratch, so easy to monitor.

Try to open files in PS, make mods, save files and watch what happens. There is IO against the scratch drive. Not much, PS indicates 100% efficiency on my system with 32GB, but it's there alright. Lots of interesting information on that disk activity display.

You can check the boxes for the photoshop processes in the top panel and then watch which files are accessed in the disk activity panel below. A sample below. Good way to determine specifically what processes are doing:

In my case F is my PS scratch drive.

 edispics's gear list:edispics's gear list
Panasonic ZS100 Nikon D7000 Nikon D7200 Nikon Z6 Nikon D70 +25 more
OP Arazi New Member • Posts: 9
Re: Adobe scratch disk and Work file on the same m.2 drive?

Thanks for all the info guys.

For a bit more context. With the 61mp sensor the files are massive. I've hit 138MB on RAW. Add the amount of layers I tend to add to images when post processing  in photoshop and you're looking at some massive files. I have 64 GB RAM, while I know that logically that should be enough I do overflow every so often. Maybe its time for a clean windows install?

Working in photoshop I experience pretty significant tool-response lag , which is a serious distracting pain.

My current system has:

- Windows 10 - SATA SSD

-  Adobe software (same Drive as OS)

- files while they are being worked on - Smaller  SATA SSD (this is the one I was wondering if I can replace with the m.2 drive I use for scratch disk)

- Scratch and LR catalog on an m.2 drive

- Long term storage on dedicated RAID server

I have a pretty good GPU on the GTX 1080 though an older i7 CPU (which seems like not too big of a deal).

I'm trying to figure out what else I can do to speed things up.

edispics
edispics Veteran Member • Posts: 4,408
Re: Adobe scratch disk and Work file on the same m.2 drive?

Arazi wrote:

Thanks for all the info guys.

For a bit more context. With the 61mp sensor the files are massive. I've hit 138MB on RAW. Add the amount of layers I tend to add to images when post processing in photoshop and you're looking at some massive files. I have 64 GB RAM, while I know that logically that should be enough I do overflow every so often. Maybe its time for a clean windows install?

Working in photoshop I experience pretty significant tool-response lag , which is a serious distracting pain.

My current system has:

- Windows 10 - SATA SSD

- Adobe software (same Drive as OS)

- files while they are being worked on - Smaller SATA SSD (this is the one I was wondering if I can replace with the m.2 drive I use for scratch disk)

- Scratch and LR catalog on an m.2 drive

- Long term storage on dedicated RAID server

I have a pretty good GPU on the GTX 1080 though an older i7 CPU (which seems like not too big of a deal).

I'm trying to figure out what else I can do to speed things up.

What percentage of your 64GB have you allocated to PS in settings and what efficiency does PS report in the efficiency window at the bottom left of the PS screen

Also, what are your settings for system paging

 edispics's gear list:edispics's gear list
Panasonic ZS100 Nikon D7000 Nikon D7200 Nikon Z6 Nikon D70 +25 more
Morris0
Morris0 Forum Pro • Posts: 29,457
Re: Adobe scratch disk and Work file on the same m.2 drive?

Sean Nelson wrote:

Morris0 wrote:

Sean Nelson wrote:

Morris0 wrote:

Those writes that are always happening are I/Os that take up space in the I/O Queue and must be completed before other I/Os to the same device can happen.

By default Windows caches write operations and uses a "lazy write" strategy to flush the cache to disk, giving priority to queued read operations. It's true that writes add to the overall disk load, but the application doesn't have to wait for them to complete before continuing the way it does for read operations. So the impact of writes is a lot lower than it is for reads.

That queue is very small. It's goal is to allow the elevator algorithm to reduce hard drive latency and of cause not usefull on an sad which is why some say to turn off delayed writes to sad.

The size of the write cache can be fairly large. My system has 24GB of RAM and when I'm running my backups with nothing else going on I see the system typically buffering around 1 to 2GB of writes before starting to dump them to disk. Of course if you're memory constrained then the queue will be a lot smaller.

And write caching is even more comprehensive for temporary files. If the application creates a file with the temporary attribute, then writes are never actually sent to disk unless the system has memory constraints or the application explicitly requests it.

When you combine write caching with the synchronous nature of reads (applications reading data typically can't continue until the read completes), writes tend to be a lot less performance-critical to an application than reads are.

I'm not disagreeing with your conclusions, just trying to provide some insight for everyone who's reading the thread.

The write delay is 2 seconds. What you are seeing build up looking at the monitor is a cache that has already been written to disk. It is available to avoid reads yet for a backup application we know that will never happen.  In the case of the Photoshop scratch file, it might.

Morris

 Morris0's gear list:Morris0's gear list
Fujifilm X-T3 Fujifilm X-H2S Canon EF 200mm f/2.8L II USM Nikon AF-S Teleconverter TC-17E II XF 90mm +11 more
OP Arazi New Member • Posts: 9
Re: Adobe scratch disk and Work file on the same m.2 drive?

edispics wrote:

What percentage of your 64GB have you allocated to PS in settings and what efficiency does PS report in the efficiency window at the bottom left of the PS screen

Also, what are your settings for system paging

Correction, this specific system I'm working with has 32GB RAM. 25GB dedicated to PS.

My efficiency is showing as 100% (I'm not entirely sure what efficiency that is referring to ).

I haven't set anything on the paging side.

Archer66 Veteran Member • Posts: 4,885
Re: Adobe scratch disk and Work file on the same m.2 drive?

Arazi wrote:

My efficiency is showing as 100% (I'm not entirely sure what efficiency that is referring to ).

It shows you that PS is having enough memory so it does not need to use scratch disk.

 Archer66's gear list:Archer66's gear list
Sony a7R III
Ho72
Ho72 Senior Member • Posts: 2,555
Re: Adobe scratch disk and Work file on the same m.2 drive?

Arazi wrote:

Working in photoshop I experience pretty significant tool-response lag , which is a serious distracting pain.

Are there specific tools that lag? I'm still using a 3770K based machine with which I create some very large files and lags are exceedingly rare... can't recall the last time I saw one.

Things to check: GPU acceleration, turn it down a notch if set to anything but Basic; Wacom driver (some have issues with WIn10), also disable "Use Windows Ink" inside the Wacom Tablet Properties/Mapping; exclude the PS process from AV/malware apps; corrupt fonts and color profiles have also been known to constipate PS.

There are also ways to reset PS prefs and launch without plugins. Might be worth a shot if you can't find any other fix.

Morris0
Morris0 Forum Pro • Posts: 29,457
Re: This is old information

edispics wrote:

robertfel wrote:

Morris0 wrote:

Scratch disk guide:

https://helpx.adobe.com/photoshop/using/scratch-disks-preferences.html

It's interesting how old timers insist on scratch disks and selectively read the guidelines.

"An SSD, on the other hand, performs well as both the primary startup and scratch disk. In fact, using an SSD is probably better than using a separate hard disk as your primary scratch disk."

It's simple math, your NVME SSD is operating at 3000 MBps, 2 SATA drives operating at 500MBps each won't have the speed of one NVME. That's for sequential read/write. For random reads it's a lot closer, but the advantage is still NVME.

You can have the OS, apps and data on a 1 NVME, it will be faster than splitting them across 3 SATA drives. 3,000 > 1,500. 1/10 - 1/50th of those speeds are for random reads, I wish drive makers would increase those speeds, then you'd notice how much snappier your system would be. Again, this is strictly NVME, if your m.2 slot if SATA, then yes, split up the OS/apps from data.

Now if you have the latest AMD boards with multiple PCIe 4.0 NVME slots, you might notice a difference splitting things up in benchmarks, but you'd never notice it in real life.

With 32GB RAM 99% of users will never access the Photoshop scratch file once it's written. It's easy enough to test this. Put your scratch file on an external drive with an LED. Once Photoshop opens, it will create the scratch file, but it will never access it again unless you're working on MASSIVE file with multiple layers. The LED will never light up.

I'm all SSD, so no issue there.

However, you can do a lot better than try to read the LED on your drive. Go into Resource Monitor, click on the Disk tab and go to the second panel which lists all IO on your system. If you sort on the name it will group all filenames with the same drive letter together. I use one SSD for PS scratch, so easy to monitor.

Try to open files in PS, make mods, save files and watch what happens. There is IO against the scratch drive. Not much, PS indicates 100% efficiency on my system with 32GB, but it's there alright. Lots of interesting information on that disk activity display.

You can check the boxes for the photoshop processes in the top panel and then watch which files are accessed in the disk activity panel below. A sample below. Good way to determine specifically what processes are doing:

In my case F is my PS scratch drive.

Yes, and in fact every operation you make is recorded to the scratch disk so you can review the history and undo as desired.  If you run a complex action, the difference in time to complete when scratch is on separate SSD is both noticeable and measurable.  One must decide if this small yet noticeable difference is worth there investment in the drive.  If you have an ssd laying around from when you upgraded, it's a no cost option to have a snappier system.  Have another one, even SATA, use it for the Page File (Swap File).

Morris

 Morris0's gear list:Morris0's gear list
Fujifilm X-T3 Fujifilm X-H2S Canon EF 200mm f/2.8L II USM Nikon AF-S Teleconverter TC-17E II XF 90mm +11 more
Keyboard shortcuts:
FForum MMy threads