Is flash memory unreliable due to exFAT?

[CAcreeks]

Have you experienced external SSD corruption? I have not, yet.

Last week I lost my 6th USB thumb drive, if I recall correctly. More than can be counted on one hand. On both Mac and PC, this ruined thumb drive claimed reformatting was required. Unmountable. Data not readable.

Is flash memory unreliable because of exFAT, or because it is flash memory?

I'm wondering if I should reformat my external SSD devices, mostly Samsung T7. They come with exFAT, which is convenient because of its near universality. Here's an informative writeup about exFAT reliability; save time by skipping to conclusion.

Notes on exFAT and Reliability

Is exFAT as fragile as some say?

pawitp.medium.com

 

[Carey Brown]

Have you experienced external SSD corruption? I have not, yet.

Last week I lost my 6th USB thumb drive, if I recall correctly. More than can be counted on one hand. On both Mac and PC, this ruined thumb drive claimed reformatting was required. Unmountable. Data not readable.

Is flash memory unreliable because of exFAT, or because it is flash memory?

I'm wondering if I should reformat my external SSD devices, mostly Samsung T7. They come with exFAT, which is convenient because of its near universality. Here's an informative writeup about exFAT reliability; save time by skipping to conclusion.

https://pawitp.medium.com/notes-on-exfat-and-reliability-d2f194d394c2

In general, thumb drives are made cheaply. It is not the file system that is at fault.

As for T7, I formatted mine with NTFS because:
a) faster copy NTFS to NTFS (I think)
b) journalling, so more robust
c) had an issue one time with a file/path name that was ok on NTFS but not on exFAT
d) I'm not in a multi-OS environment

--
Photos at http://inasphere.com

 

[Soaptrail]

Have you experienced external SSD corruption? I have not, yet.

Last week I lost my 6th USB thumb drive, if I recall correctly. More than can be counted on one hand. On both Mac and PC, this ruined thumb drive claimed reformatting was required. Unmountable. Data not readable.

Is flash memory unreliable because of exFAT, or because it is flash memory?

I'm wondering if I should reformat my external SSD devices, mostly Samsung T7. They come with exFAT, which is convenient because of its near universality. Here's an informative writeup about exFAT reliability; save time by skipping to conclusion.

https://pawitp.medium.com/notes-on-exfat-and-reliability-d2f194d394c2

I am curious what brand USB thumb drives you are buying and how much use they get. perhaps you can check the others with a disk check for a health assessment.

While I barely used thumb drives I don't think I have even had a SD/microSD or old Sony memory stick die or get corrupted.

Perhaps Brand and not getting a counterfeit drive are part of your issue. I hear at Amazon even when buying a reputable brand you could get a counterfeit because other sellers products get dumped into the same bins as Amazon products so no matter which seller you buy from you could get counterfeits.

 

[bmoag]

Most USB thumb drives are cheap crud, so there's that to explain why they fail. Does not matter which OEM is on the label, drives at a particular price point are likely sourced from the same places.

Failure is more likely due to the controller and not the data format type.

Also Apple has never solved its problems with USB based storage. Hence eject before disconnect. I doubt there are many people who regularly use both macOS (x86 or ARM) who have not experienced USB storage based media that functions perfectly on Windows but hiccoughs on macOS.

"Thumb" drives are not optimal for reliable long term storage period.

I use the same drives regularly on macOS and Windows, all formatted in NTFS. There are reliable apps for mac, like Paragon, for writing to those drives but one does need to periodically check data integrity if written from macOS (something I have sadly learned to be the case for Apple's native formatting).

 

[CAcreeks]

Last week I lost my 6th USB thumb drive. More than can be counted on one hand.

Last one that failed is a Verbatim. That's a low-end brand AFAIK. Can't recall all the previous ones, but I'm positive a Sandisk failed. That's a high-end brand IMO. One was an unknown brand from Frye's Electronic (RIP) and failed on first use. My longest lasting USB stick was a 16MB, yes you read that right, MB not GB. More space between the flashes?

The Verbatim did allow me to pull off a very large MP4 video before failing.

While I barely used thumb drives I don't think I have even had a SD/microSD or old Sony memory stick die or get corrupted.

I've only had one SD(XC) card fail. It was Adata. Possibly when you leave SD/microSD in the camera or phone, it gets enough power to keep the memory bits oriented correctly.

Perhaps Brand and not getting a counterfeit drive are part of your issue. I hear at Amazon even when buying a reputable brand you could get a counterfeit because other sellers products get dumped into the same bins as Amazon products so no matter which seller you buy from you could get counterfeits.

Agreed. I've had good luck with Sandisk Fit. One I formatted EXT-4 to run Linux, and it was fairly reliable, although I had to run fsck occasionally when the OS recommended it. Linux can read and write NTFS, which was on the laptop HDD.

 

[Tom_N]

Have you experienced external SSD corruption? I have not, yet.

Last week I lost my 6th USB thumb drive, if I recall correctly. More than can be counted on one hand. On both Mac and PC, this ruined thumb drive claimed reformatting was required. Unmountable. Data not readable.

Is flash memory unreliable because of exFAT, or because it is flash memory?

Flash memory can lose data if left unpowered long enough (years). It can also wear out if rewritten enough times, although today's flash memory can handle a lot of rewriting. I wouldn't be surprised to find that the flash memory and controllers that you find in a typical USB thumb drive area are lower-end than those you would find in a typical SSD, and USB thumb drives are more likely to be treated in a manner that exposes them to hazards like dirt building up on the contacts, or static.

exFAT doesn't have journaling, so if you eject a volume without dismounting it properly, exFAT is more likely than NTFS to become corrupt / lose data.

Do people lose all of their photos off exFAT-formatted SDXC memory cards as often as they lose all of their files off exFAT-formatted USB flash drives?!?

 

[Joep van Steen]

Flash memory can lose data if left unpowered long enough (years).

A lot quicker actually, specially modern TLC/QLC NAND has very small margins. Also, the more NAND was written to, the quicker it bleeds data (retention errors, most common cause for NAND errors).

although today's flash memory can handle a lot of rewriting

Other way around. First generation (SLC) NAND was considerably better than modern NAND.

Failure is more likely due to the controller and not the data format type.

Mostly the NAND. Controller is secondary factor that comes in to play when handling the bad NAND. With most USB Flash Drives I recover data from it's the condition of the NAND.

IOW, moving NAND to intact donor often yields nothing and if I dump NAND bypassing controller, tons of bit errors reveal themselves.

--
Joep

 

[Joep van Steen]

I'm wondering if I should reformat my external SSD devices, mostly Samsung T7. They come with exFAT, which is convenient because of its near universality. Here's an informative writeup about exFAT reliability; save time by skipping to conclusion.

https://pawitp.medium.com/notes-on-exfat-and-reliability-d2f194d394c2

In general, thumb drives are made cheaply. It is not the file system that is at fault.

As for T7, I formatted mine with NTFS because:
a) faster copy NTFS to NTFS (I think)
b) journalling, so more robust
c) had an issue one time with a file/path name that was ok on NTFS but not on exFAT
d) I'm not in a multi-OS environment

I won't argue against NTFS being more robust than exFAT, but neither file system stands a chance when errors are result of bad NAND memory or other NAND errors.

 

[CAcreeks]

In general, thumb drives are made cheaply. It is not the file system that is at fault.

As for T7, I formatted mine with NTFS because:
a) faster copy NTFS to NTFS (I think)
b) journalling, so more robust
c) had an issue one time with a file/path name that was ok on NTFS but not on exFAT
d) I'm not in a multi-OS environment

I won't argue against NTFS being more robust than exFAT, but neither file system stands a chance when errors are result of bad NAND memory or other NAND errors.

Also in the environments where I would use it (Linux or POSIX) NTFS is very slow compared to exFAT. And in some cases, read-only.

F2FS Results Mixed Against Microsoft's exFAT On Linux - Phoronix

www.phoronix.com

This Wikipedia page does not list NTFS among filesystems optimized for Flash memory.

List of file systems - Wikipedia

en.wikipedia.org

 

[Carey Brown]

Have you experienced external SSD corruption? I have not, yet.

Last week I lost my 6th USB thumb drive, if I recall correctly. More than can be counted on one hand. On both Mac and PC, this ruined thumb drive claimed reformatting was required. Unmountable. Data not readable.

Is flash memory unreliable because of exFAT, or because it is flash memory?

I'm wondering if I should reformat my external SSD devices, mostly Samsung T7. They come with exFAT, which is convenient because of its near universality. Here's an informative writeup about exFAT reliability; save time by skipping to conclusion.

https://pawitp.medium.com/notes-on-exfat-and-reliability-d2f194d394c2

In general, thumb drives are made cheaply. It is not the file system that is at fault.

As for T7, I formatted mine with NTFS because:
a) faster copy NTFS to NTFS (I think)
b) journalling, so more robust
c) had an issue one time with a file/path name that was ok on NTFS but not on exFAT
d) I'm not in a multi-OS environment

'C' was the clincher. Sorry, can't remember what the exact issue was. I know it was a long name and long path with lots of special characters, but that's it.

A file that came from an install. Not one of mine.

--
Photos at http://inasphere.com

 

[foot]

Flash memory can lose data if left unpowered long enough (years).

A lot quicker actually, specially modern TLC/QLC NAND has very small margins. Also, the more NAND was written to, the quicker it bleeds data (retention errors, most common cause for NAND errors).

although today's flash memory can handle a lot of rewriting

Other way around. First generation (SLC) NAND was considerably better than modern NAND.

Failure is more likely due to the controller and not the data format type.

Mostly the NAND. Controller is secondary factor that comes in to play when handling the bad NAND. With most USB Flash Drives I recover data from it's the condition of the NAND.

IOW, moving NAND to intact donor often yields nothing and if I dump NAND bypassing controller, tons of bit errors reveal themselves.

also the general issues with removable drives, for example:

unexpected power interruptions

removal before all writes are finished (so no cached writes!!!)

usb connector issues

cabling issues

 

[kelpdiver]

CA - a bit confused here. You open with a question and then post a citation that answers that question. So what is the intent of this thread?

Bmoag, I think, hit it correctly. OSX really *ucks with flash drives if you don't eject them in software before physical removal. It's a pretty bad flaw to me, and I've been hit by it.

But look at the price of a 256 or 612g microsd card and compare to an SSD and that shows the quality difference.

 

[JerseySailor]

...............
exFAT doesn't have journaling, so if you eject a volume without dismounting it properly, exFAT is more likely than NTFS to become corrupt / lose data.

...........

One item buried in Tom's post was the need to remove the drive from the OS, before pulling the thumb drive out of the USB port.

On a Windows PC, that means right-clicking on the device, and selecting "Eject", and waiting two seconds for the message saying that it is safe to remove the drive. Pulling the drive before going through this eject process is playing Russian Roulet with the data on the drive.

 

[kelpdiver]

On a Windows PC, that means right-clicking on the device, and selecting "Eject", and waiting two seconds for the message saying that it is safe to remove the drive. Pulling the drive before going through this eject process is playing Russian Roulet with the data on the drive.

if you did some writes and then immediately pulled it, that would be a risk. But with any sort of time delay, that write completes. This is why MS stopped putting up the 'you done wrong' nag message in Win10.

 

[Joep van Steen]

In general, thumb drives are made cheaply. It is not the file system that is at fault.

As for T7, I formatted mine with NTFS because:
a) faster copy NTFS to NTFS (I think)
b) journalling, so more robust
c) had an issue one time with a file/path name that was ok on NTFS but not on exFAT
d) I'm not in a multi-OS environment

I won't argue against NTFS being more robust than exFAT, but neither file system stands a chance when errors are result of bad NAND memory or other NAND errors.

Also in the environments where I would use it (Linux or POSIX) NTFS is very slow compared to exFAT. And in some cases, read-only.

https://www.phoronix.com/review/linux_f2fs_exfat/3

This Wikipedia page does not list NTFS among filesystems optimized for Flash memory.

https://en.wikipedia.org/wiki/List_...optimized_for_flash_memory,_solid_state_media

And exFAT is more optimized towards NAND (less wear) than NTFS.

--
Joep

 

[CAcreeks]

CA - a bit confused here. You open with a question and then post a citation that answers that question. So what is the intent of this thread?

That citation did not answer my question. The article was essentially a "what me worry?" opinion piece about exFAT. Could have been authored by Alfred E. Newman. My question is:

Have you experienced external SSD corruption [due to exFAT], or know anybody who did?

P.S. Windows "Quick Removal" setting is nice, but I'm used to Eject from the olden days (especially CD/DVD) so failure to Eject is not my issue. The Verbatim product failed on my wife's Windows laptop.

 

[Billiam29]

...............
exFAT doesn't have journaling, so if you eject a volume without dismounting it properly, exFAT is more likely than NTFS to become corrupt / lose data.

...........

One item buried in Tom's post was the need to remove the drive from the OS, before pulling the thumb drive out of the USB port.

On a Windows PC, that means right-clicking on the device, and selecting "Eject", and waiting two seconds for the message saying that it is safe to remove the drive. Pulling the drive before going through this eject process is playing Russian Roulet with the data on the drive.

Starting with Windows 10 1809, Microsoft changed the default removal behavior for external storage to the 'Quick Removal' setting which disables write caching.

I think it's entirely plausible there could be application or user service processes that might be touchy-feely with files on a thumb drive at any given time. It's probably a complete coin toss whether or not those actions would be gracefully stopped by using the Windows Eject command. I do belive the Eject command's original purpose though was to flush the write cache to the external storage device before removing it. As said, that hasn't been a concern for roughly 4.5 years now.

Reference: https://learn.microsoft.com/en-us/w...default-removal-policy-external-storage-media

 

[Joep van Steen]

Flash memory can lose data if left unpowered long enough (years).

A lot quicker actually, specially modern TLC/QLC NAND has very small margins. Also, the more NAND was written to, the quicker it bleeds data (retention errors, most common cause for NAND errors).

although today's flash memory can handle a lot of rewriting

Other way around. First generation (SLC) NAND was considerably better than modern NAND.

Failure is more likely due to the controller and not the data format type.

Mostly the NAND. Controller is secondary factor that comes in to play when handling the bad NAND. With most USB Flash Drives I recover data from it's the condition of the NAND.

IOW, moving NAND to intact donor often yields nothing and if I dump NAND bypassing controller, tons of bit errors reveal themselves.

also the general issues with removable drives, for example:

unexpected power interruptions

removal before all writes are finished (so no cached writes!!!)

usb connector issues

cabling issues

Yes, and of course you get what you pay for. Ranging from cheap USB flash drives mage using with low grade NAND and controllers, soldered using low grade solder on a flimsy easy to bend PCB up to high quality external SSD's with quality NAND and an array of super capacitors to protect from sudden power loss.

 

[Tom_N]

I won't argue against NTFS being more robust than exFAT, but neither file system stands a chance when errors are result of bad NAND memory or other NAND errors.

Also in the environments where I would use it (Linux or POSIX) NTFS is very slow compared to exFAT. And in some cases, read-only.

https://www.phoronix.com/review/linux_f2fs_exfat/3

If you're talking about a SDXC memory card used in a digital camera, you really have no choice. Cameras expect SDXC and exFAT to go hand-in-hand, and I haven't heard of any camera that understands NTFS (or HFS+ Journaled, or APFS, or Linux/BSD filesystems).

 

[David Lal]

P.S. Windows "Quick Removal" setting is nice, but I'm used to Eject from the olden days (especially CD/DVD) so failure to Eject is not my issue. The Verbatim product failed on my wife's Windows laptop.

I'm surprised by the negative references to Verbatim in this thread. I thought it had a good name, certainly in terms of CDs/DVDs - I hope so as I have a pack of 50 Verbatim dual layer recordable DVDs arriving today :-(

See: https://www.digitalfaq.com/reviews/dvd-media.htm

CDs/DVDs seem to be going out of fashion but I prefer them to flash sticks for distributing photos and so on to friends and family. Also they are a lot easier to file along with relevant paperwork.