Switching from PC to Apple for big file processing

[MasterOfGoingFaster]

Thomas, I will try that but would this issue have an incidence on the delay when I hit « p » for pick? (for example)

And, can I run APFS in a windows machine?

I am currently struggling with LrC's performance

Windows11 Machine Specs:

Norton 360

Antivirus Enabled

That's one of the first places I'd check. Disconnect from the internet, then shut down Norton (everything Norton-related). Now try LRc.

I use to be a sysadmin at a 150,000 user network. Norton and McAfee were alway a performance suspect.

Thanks Ken, it’s one of the first things I checked based on a few recommendations like yours. I actually de-installed Norton altogether and ran my computer without it for a few days and saw no noticeable difference. I also entered exceptions for photo directories and drives in Norton antivirus (I still have those).

I’m pretty desperate honestly.

Another hint: Maybe it is a problem with the FS (File System).

On Mac, APFS is the actual FS. The former one was HPFS/HPFS+.

On Linux systems extFAT the common used.

On former Windows system use(d) FAT32 - Windows 11 uses NTFS. This was the standard FS on Win NT 3.51. We are again in history

APFS/HPFS performs best with large files > 0,5 to 0,75 GB, but struggles with smaller files and performs worse as compared to ExtFAT or FAT32.

RAW files perform much better using ExtFAT/FAT32. Usually "too small".

Besdes the FS the applications might be a second bottleneck, because it might be that e. g. LR on Mac uses "the wrong FS" with respect to RW speed.

Same strange behaviours are known on Norton/Symantec products as well.

Test/Solution:

Try the following: Take a stick and format it using Disk Utility

1. with ExtFAT - copy 100 pictures to it, check the time

2. Open LR using the ExtFAT device and check speed while using/accessing/edit files

now do the same, using the same stick, but formatted with APFS and check times and performance.

If you detect the better performing FS consider to change the FS of your "data drive.

The Mac OS partitions (internal SSD/HD) need to run with APFS - DONT CHANGE THE FS of your SYSTEM!

I had similar problems using COne - it performed ⅓ better on external drive using ExtFAT compared to running the same under APFS.

I never tried to access APFS drives from windows because I don't use Windows besides customers systems, and upon this I'll never do tests for my own

So my post relates to MacOS and the FS being applicable on this platform.

In former times it was necessary to have SAMBA installed to be able to mount NTFS drives from Mac.

My experience is to use ExtFAT with Windows systems.

FAT was used with Windows 95. NT-based systems (NT, 2000, XP, Vista, 7, 8,10 and 11) all use NTFS. FAT is only used for portable drives used on multiple systems. Only Macs use APFS.

So system drives should use the native file system (Mac APFS, Windows NTFS), and portable devices (cards, portable drives) can use FAT/FAT32. FAT/FAT32 does not have features to protect the file structure, so it is slightly faster, but can be easily corrupted. FAT32 has a 4GB size limitation, so larger drives would need APFS, NTFS, or ExFAT. ExFAT is less compatible across devices than FAT32.

Rule of thumb - system drives should use the native file system. Cards should use FAT32. Portable drives should use the native file system UNLESS they are used to transfer files between Mac and Windows systems, then use Fat32/ExFAT. Test this before you use it for real work.

 

[JimKasson]

Thomas, I will try that but would this issue have an incidence on the delay when I hit « p » for pick? (for example)

And, can I run APFS in a windows machine?

I am currently struggling with LrC's performance

Windows11 Machine Specs:

Norton 360

Antivirus Enabled

That's one of the first places I'd check. Disconnect from the internet, then shut down Norton (everything Norton-related). Now try LRc.

I use to be a sysadmin at a 150,000 user network. Norton and McAfee were alway a performance suspect.

Thanks Ken, it’s one of the first things I checked based on a few recommendations like yours. I actually de-installed Norton altogether and ran my computer without it for a few days and saw no noticeable difference. I also entered exceptions for photo directories and drives in Norton antivirus (I still have those).

I’m pretty desperate honestly.

Another hint: Maybe it is a problem with the FS (File System).

On Mac, APFS is the actual FS. The former one was HPFS/HPFS+.

On Linux systems extFAT the common used.

On former Windows system use(d) FAT32 - Windows 11 uses NTFS. This was the standard FS on Win NT 3.51. We are again in history

APFS/HPFS performs best with large files > 0,5 to 0,75 GB, but struggles with smaller files and performs worse as compared to ExtFAT or FAT32.

RAW files perform much better using ExtFAT/FAT32. Usually "too small".

Besdes the FS the applications might be a second bottleneck, because it might be that e. g. LR on Mac uses "the wrong FS" with respect to RW speed.

Same strange behaviours are known on Norton/Symantec products as well.

Test/Solution:

Try the following: Take a stick and format it using Disk Utility

1. with ExtFAT - copy 100 pictures to it, check the time

2. Open LR using the ExtFAT device and check speed while using/accessing/edit files

now do the same, using the same stick, but formatted with APFS and check times and performance.

If you detect the better performing FS consider to change the FS of your "data drive.

The Mac OS partitions (internal SSD/HD) need to run with APFS - DONT CHANGE THE FS of your SYSTEM!

I had similar problems using COne - it performed ⅓ better on external drive using ExtFAT compared to running the same under APFS.

I never tried to access APFS drives from windows because I don't use Windows besides customers systems, and upon this I'll never do tests for my own

So my post relates to MacOS and the FS being applicable on this platform.

In former times it was necessary to have SAMBA installed to be able to mount NTFS drives from Mac.

My experience is to use ExtFAT with Windows systems.

FAT was used with Windows 95. NT-based systems (NT, 2000, XP, Vista, 7, 8,10 and 11) all use NTFS. FAT is only used for portable drives used on multiple systems. Only Macs use APFS.
So system drives should use the native file system (Mac APFS, Windows NTFS), and portable devices (cards, portable drives) can use FAT/FAT32. FAT/FAT32 does not have features to protect the file structure, so it is slightly faster, but can be easily corrupted. FAT32 has a 4GB size limitation, so larger drives would need APFS, NTFS, or ExFAT. ExFAT is less compatible across devices than FAT32.

Rule of thumb - system drives should use the native file system.

How are you defining system drives? I'm using ZFS on some of my attached storage and the Apple native file system on others.

Cards should use FAT32. Portable drives should use the native file system UNLESS they are used to transfer files between Mac and Windows systems, then use Fat32/ExFAT. Test this before you use it for real work.

 

[Stan Disbrow]

Hi,

They weren't, exactly. They were 19" but the vertical rails weren't drilled with all the holes. Just the holes needed for their original use. But those unbolted and I swiped ones drilled with all the holes from a couple 24" racks. So, an easy change.

And some instruments I wanted in there weren't rack mount so I made some plywood shelves as well.

Stan

 

[MasterOfGoingFaster]

Rule of thumb - system drives should use the native file system.

How are you defining system drives? I'm using ZFS on some of my attached storage and the Apple native file system on others.

Well, the system drive is the primary (drive 0) drive with the OS. But I'm sure most people will assume it applies to all internal storage. As a rule of thumb, I think it works.

The rule of thumb does not apply to you, Jim. If you've got the cpu cycles to spare ZFS is pretty awesome. But I'm not about to suggest it to most photographers.

 

[JimKasson]

Rule of thumb - system drives should use the native file system.

How are you defining system drives? I'm using ZFS on some of my attached storage and the Apple native file system on others.

Well, the system drive is the primary (drive 0) drive with the OS.

Gotcha. I do use the Apple native file system for that.

But I'm sure most people will assume it applies to all internal storage. As a rule of thumb, I think it works.

The rule of thumb does not apply to you, Jim. If you've got the cpu cycles to spare ZFS is pretty awesome.

ZFS runs in my NASs.

But I'm not about to suggest it to most photographers.

 

[Thomas Guenther]

Rule of thumb - system drives should use the native file system.

How are you defining system drives? I'm using ZFS on some of my attached storage and the Apple native file system on others.

Well, the system drive is the primary (drive 0) drive with the OS. But I'm sure most people will assume it applies to all internal storage. As a rule of thumb, I think it works.

The rule of thumb does not apply to you, Jim. If you've got the cpu cycles to spare ZFS is pretty awesome. But I'm not about to suggest it to most photographers.

Maybe on windows. on Mac/Linux disk0 is the physical disk. Then disk3 is usually synthesized and mounted as the logical drive that contains system partition/dis

isk and data portion/disk

 

[paulraphael]

Rule of thumb - system drives should use the native file system.

How are you defining system drives? I'm using ZFS on some of my attached storage and the Apple native file system on others.

Well, the system drive is the primary (drive 0) drive with the OS.

Gotcha. I do use the Apple native file system for that.

As you must. The boot drive is always APFS, always encrypted, and always structured in a complex manner that synthesizes the "Macintosh HD" image from a group of elements (including read-only snapshots) that reside on multiple logical APFS volumes.

All this complexity is in the name of security and stability. Excellent explanations on Howard Oakley's site .