bit Rot examples

JPG compression is done the same each time, and the loss should be the same for the same photo, but its isn't - even thought its very close - due to small insignificant copy errors compounding over time ...
The difference there is not due to copy errors. It's due to different calculation results when the compression algorithm determines how much data to discard. That has nothing to do with 'bit rot', which is about errors occurring in the recording medium.
Well then I mislabel the term. I am talking about photos that deteriorate over time faster than the recording medium decays.
No, that simply doesn't happen with digital storages. It doesn't happen with non-digital (analogue) storages either btw.

The information is the medium in certain sense. If the medium is intact, the information is intact. Full stop.
Please back with data. This is contrary to what I have read, and experienced.
There's no need to back that with data - the above statement is in the same league as flat earth conspiracy and alike.
I am sure your right, I m no exptert but lets play Devils Advocate:

That is a fundamental misunderstanding of particle science and Thermodynamics.

Reminder Entropy

plus we have not idea how effective quantum tunneling is.

However your right election loss is almost certainly from some from of medium decay, localized Carrington like events are rare on any large scale. However regular ionize backgrounder causes digital flips and what not and is more or less continuous minus radiation harding of the device. The famous heavy magnate distortions of yesteryear being an early example of such flips.

I guess you could argue that this are all destroying the medium -that a chalk board and its chalk are one and if you remove the chalk its the medium that suffers as well.

But to supprt you contention about bit rot is this


which points out that its the medium no t being made perfect that allows "flipping" so even if the device never aged at best your gong to have areas prone to flipping over time which is a medium isue

Furhter reading



"

Electron and temperature diagram

The Solid-State Technology Association (JEDEC) has already made relevant regulations. It is no problem to store data in consumer SSDs for one year without power saving at 30 °C, but if the temperature is too high, the risk of losing data will increase dramatically."




Net discussions


 
JPG compression is done the same each time, and the loss should be the same for the same photo, but its isn't - even thought its very close - due to small insignificant copy errors compounding over time ...
The difference there is not due to copy errors. It's due to different calculation results when the compression algorithm determines how much data to discard. That has nothing to do with 'bit rot', which is about errors occurring in the recording medium.
Well then I mislabel the term. I am talking about photos that deteriorate over time faster than the recording medium decays. Though I suppose I am talking about that too. How ever you make make the unsubstantiated claim that in somehow all jpgs fail at the same rate.

Please back with data. This is contrary to what I have read, and experienced.
PLease define the problem, if you want a useful discussion it must be clear what you want to talk about.
 
"Rot" is a bad term IMHO since it implies some sort of organic decay. That doesn't happen to data -- a 1 or a 0 will always be a 1 or a 0, unless the media on which it is written goes bad.
It does not work like that. Simply put, taking NAND as example, a threshold voltage decides if something is either 0 or 1. If voltage drops over time it causes the bit to flip. But the NAND cell does not store a 0 or 1, it stores a charge.

This is not the same as the media going bad. If caught in time 'refreshing' the cell's content will prevent the issue.
 
"Rot" is a bad term IMHO since it implies some sort of organic decay. That doesn't happen to data -- a 1 or a 0 will always be a 1 or a 0, unless the media on which it is written goes bad.
It does not work like that. Simply put, taking NAND as example, a threshold voltage decides if something is either 0 or 1. If voltage drops over time it causes the bit to flip. But the NAND cell does not store a 0 or 1, it stores a charge.
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
This is not the same as the media going bad. If caught in time 'refreshing' the cell's content will prevent the issue.
With a USB stick or SDxC card, how is it possible to "refresh" the cell contents? It's not like an SSD, where firmware will take care of wear leveling and sector reallocation.

Flash memory can be reformatted and rewritten, assuming content is available to replace.
 
Last edited:
However your right election loss is almost certainly from some from of medium decay, localized Carrington like events are rare on any large scale. However regular ionize backgrounder causes digital flips and what not and is more or less continuous minus radiation harding of the device. The famous heavy magnate distortions of yesteryear being an early example of such flips.
I am hereby accusing you of a word salad. This is so silly I am going to go further and accuse you of...

... kale.
 
However your right election loss is almost certainly from some from of medium decay, localized Carrington like events are rare on any large scale. However regular ionize backgrounder causes digital flips and what not and is more or less continuous minus radiation harding of the device. The famous heavy magnate distortions of yesteryear being an early example of such flips.
I am hereby accusing you of a word salad. This is so silly I am going to go further and accuse you of...

... kale.
It kind of looks like it could be a result of using a virtual keyboard and relying too much on predictive functionality without making sure all suggestions are actually valid. Or a result of using Google Translate (or other similar translation services) to transfer from some other language to English.
 
Digital media frequently has errors. It happens all the time.

Under the hood, digital media is analog. Real world systems are never perfect, and it is not uncommon to see a voltage of 2.5, when you are hoping to see either zero or five volts.

However, digital media is also quite reliable. That's because it stores redundant data to provide error correction. The controller uses the redundant data to detect and correct those errors.

Occasionally, we will see errors that are such that they cannot be corrected using the available redundant data. For instance, a Seagate 8TB drive (model ST8000DM004) is specified to have up to one bit error per 100,000,000,000,000 bits read.

We can improve on this in a number of ways. RAID is a common method.

As storage media ages, the internal error rates go up, and it becomes more likely that you will see an uncorrectable error.

Of course, these un correctable error rates are very low. If you make a millions of copies of a JPEG file, it is unlikely that you will see a bit error in the data.

Strangely, drives marketed for NAS (Network Attached Storage) systems have higher error rates. If a normal desktop drive has trouble reading data off the disk, it will make multiple attempts, before eventually giving up. If the drive is part of a RAID, this can pause all of the RAID activity. A NAS drive is programmed to give up quickly, and return an error if it has trouble reading a sector. The theory is that there is no need to pause the RAID, as the data can be reconstructed from the redundant data stored on other drives.
 
"Rot" is a bad term IMHO since it implies some sort of organic decay. That doesn't happen to data -- a 1 or a 0 will always be a 1 or a 0, unless the media on which it is written goes bad.
It does not work like that. Simply put, taking NAND as example, a threshold voltage decides if something is either 0 or 1. If voltage drops over time it causes the bit to flip. But the NAND cell does not store a 0 or 1, it stores a charge.
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.

In my book media is 'bad' when a sector becomes unusable. Which is not the case here.

I'd almost say it's a property of NAND memory that it leaks some charge. And to counter that ECC is a vital part of NAND memory (but also hard drives, which also do not store zeros and ones). But it can only recover so many bits, depending on size of the ECC.

In addition the NAND based drive can play and does play with these thresholds. Say ECC can recover 6 bits but we have 7 corrupt bits. It can see if higher or lower threshold results in fewer corrupt bits and if so write contents of a page to another.

The corrupt page however is not per se unfit to store new data, so it's not bad in that sense.
This is not the same as the media going bad. If caught in time 'refreshing' the cell's content will prevent the issue.
With a USB stick or SDxC card, how is it possible to "refresh" the cell contents? It's not like an SSD, where firmware will take care of wear leveling and sector reallocation.
Although firmware of an SSD is more complex, it is not impossible for an SD card either, I don't see any theoretical objections. SD Cards do sector reallocation BTW, but I am unsure what you mean by it. And wear leveling is more than just backgrounds tasks shuffling data around. SD Cards and USB flash drives do wear leveling.

I was discussing NAND as an example which includes both SSD and SD cards.

Edit:

I was thinking how to explain (static) wear-leveling at it's simplest form. Key is that NAND flash, all NAND flash translates LBA to PBA (logical block to physical block).

This translation layer almost implies some degree of wear leveling by definition. Assume we store a contiguous JPEG file in cluster 100 + however many clusters we need. First translation is cluster(x) to LBA(y), so simplified boot sector + FAT * (number of FATs) + cluster * (sectors/cluster).

Firmware now assigns PBA(z) to LBA(y).

Now we delete that file, so file system is aware of clusters assigned to our JPEG are vacant. We assume no TRIM like mechanism so card's firmware is not aware!

Now we write a new JPEG to the drive, firmware 'FAT driver' selects our now vacant cluster 100 which translates to LBA(y). Since card can not write to PBA(z) as it needs erased memory for that, it has to select a different PBA for LBA(y). Ergo, data is written to different PBA which in itself is a form of wear leveling: the file written to cluster 100 ends up being written in a different physical location. PBA previously assigned to LBA(y) now becomes stale and can be erased by garbage collector.

To enhance this all we need is a simple table to keep track of number of uses per page so we can even more efficiently level wear over pages.
Flash memory can be reformatted and rewritten, assuming content is available to replace.
Oh really?

--
Joep
 
Last edited:
The information is the medium in certain sense. If the medium is intact, the information is intact. Full stop.
Please back with data. This is contrary to what I have read, and experienced.
There's no need to back that with data - the above statement is in the same league as flat earth conspiracy and alike.
I am sure your right, I m no exptert but lets play Devils Advocate:

That is a fundamental misunderstanding of particle science and Thermodynamics.
Reminder Entropy

plus we have not idea how effective quantum tunneling is.
We do. There are tunneling semiconductors that use it.
However your right election loss is almost certainly from some from of medium decay,
You always need a physical carrier to store the information. You can't store information without making physical changes in the medium. Maybe you imply that there's silicone medium and electrons in it are something different (non-medium)?
I guess you could argue that this are all destroying the medium -that a chalk board and its chalk are one and if you remove the chalk its the medium that suffers as well.
The chalk is the medium.

However you can lose information when the medium is intact, the information is intact, but the protocol is broken, so the information is inaccessible or corrupted. It can be broken on different levels. An SSD controller may break, or on a higher level, your software may get an upgrade and stop supporting an old file format. Or it can be a glitch in the software that incorrectly decodes certain files, or a glitch on a lower level, e.g. faulty RAM.

With your original examples, things could go wrong on many different levels, it didn't necessarily have to be a faulty storage.
 
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.

It's encoded as voltage level (in certain types of digital storages).
In my book media is 'bad' when a sector becomes unusable. Which is not the case here.
It's 'bad' when it breaks the protocol: you encoded a 1 and it reads a 0. Whether it's recoverable and whether it's marked as 'bad' is another story.
 
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.

It's encoded as voltage level (in certain types of digital storages).
The issue is that real world media does not encode zeros and ones. The encoding is analog. For instance we want to use a no magnetic field to encode a zero, and a magnetic field of intensity 1 to store a one. When we are reading the data back, the analog values won't be exactly 0 and 1. We may see values like 0.1, 0.9, 0.2 and 0.8. It's when we stat seeing values around 0.5 that we lose confidence as to which binary value it represents.

Fields on Magnetic media can fade over time. The charge on Solid State Drives can fade over time. One strategy is to periodically read and refresh the data, but even this is not 100% reliable.

Modern devices are exceedingly reliable, but they are not perfect. However, we can control how often we expect to see an error. By using various techniques we can reduce the expected error rates to less than one bit error in a thousand years. However, we can save money by living with error rates that are a little less than that.

In my book media is 'bad' when a sector becomes unusable. Which is not the case here.
It's 'bad' when it breaks the protocol: you encoded a 1 and it reads a 0. Whether it's recoverable and whether it's marked as 'bad' is another story.
=====

"To the best of our knowledge, our computer has never had an undetected error."
 
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.

It's encoded as voltage level (in certain types of digital storages).
The issue is that real world media does not encode zeros and ones.
All digital media 'encodes' zeros and ones. 0s and 1s are what's being encoded, and there's physical medium that has its state changed in order to 'encode' 0s and 1s.
The encoding is analog.
You may say so, but generally it's 'physical'.
For instance we want to use a no magnetic field to encode a zero, and a magnetic field of intensity 1 to store a one.
In case it's stored though magnetic field, 0s and 1s are literally encoded/recorded in magnetic dipoles that have just two states (if not faulty).
When we are reading the data back, the analog values won't be exactly 0 and 1. We may see values like 0.1, 0.9, 0.2 and 0.8. It's when we stat seeing values around 0.5 that we lose confidence as to which binary value it represents.

Fields on Magnetic media can fade over time. The charge on Solid State Drives can fade over time. One strategy is to periodically read and refresh the data, but even this is not 100% reliable.

Modern devices are exceedingly reliable, but they are not perfect. However, we can control how often we expect to see an error. By using various techniques we can reduce the expected error rates to less than one bit error in a thousand years. However, we can save money by living with error rates that are a little less than that.
Everything decays in this Universe.
 
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.
I can argue a user isn't interested in zeros and ones either. He stores a document, or a photo so I do not think that's a strong argument. And yes a computer handles all that data as zeros and ones. But once written to storage it is no longer zeros and ones. We intend to store zeros and ones, but we don't. And due to physical properties data can leak without the media being 'bad'. Or, we could simply assume all media is bad, I can live with that too. Because eventually electric charges will fade.

A battery will leak charge over time, not because it's bad but because that's just the way it is. The outcome may be undesired because we have no power or in case of NAND memory the bit flips, but it's not due to bad media. We discussed 'bit rot', bit rot can occur on perfectly healthy media. It occurs because information stored as not as binary as we intend it to be. It's truly decay, it's not a one that drops all of a sudden, it gradually gets closer to a threshold until it's no longer decoded as a 1 but a 0 instead or vice versa.
It's encoded as voltage level (in certain types of digital storages).
In my book media is 'bad' when a sector becomes unusable. Which is not the case here.
It's 'bad' when it breaks the protocol: you encoded a 1 and it reads a 0. Whether it's recoverable and whether it's marked as 'bad' is another story.
It isn't. It's functional, like I said if the decay is detected early enough data can be refreshed, and this is possible because the media isn't actually bad. Media is bad, at least that's my definition, it's all a matter of definition, when it loses it's capability to store data. A bad sector can not be read from or written to.

Modern storage devices are only possible due to data being recoverable to a degree. Flash based drives, but spinning disks too, ECC recover data all the time. We encode data before writing it to a storage device to work around the limitations of the media and prevent bit errors.

Point is, it is not possible to write plain zeros and ones to storage devices it's simply to 'fuzzy' for that. What we read from a drive is information that needs to be decoded to turn it back into zeros and ones, into digital data.
 
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.
I can argue a user isn't interested in zeros and ones either. He stores a document, or a photo so I do not think that's a strong argument. And yes a computer handles all that data as zeros and ones. But once written to storage it is no longer zeros and ones.
Zeros and ones is an abstraction. The meaningful data being recorded is a sequence of 0s and 1s.
We intend to store zeros and ones, but we don't.
We intend and we do. We do store zeros and ones recorded/encoded as changes in the physical state of the media. We then need to follow a certain protocol to read them back.
A battery will leak charge over time, not because it's bad but because that's just the way it is. The outcome may be undesired because we have no power or in case of NAND memory the bit flips, but it's not due to bad media. We discussed 'bit rot', bit rot can occur on perfectly healthy media. It occurs because information stored as not as binary as we intend it to be. It's truly decay, it's not a one that drops all of a sudden, it gradually gets closer to a threshold until it's no longer decoded as a 1 but a 0 instead or vice versa.
Yes. Which means there was a physical change in the corresponding medium. Or something happened in the controller (e.g. voltage became too low for whatever reason). Or there's a parasitic capacitance because of a manufacturing defect.
In my book media is 'bad' when a sector becomes unusable. Which is not the case here.
It's 'bad' when it breaks the protocol: you encoded a 1 and it reads a 0. Whether it's recoverable and whether it's marked as 'bad' is another story.
It isn't. It's functional, like I said if the decay is detected early enough data can be refreshed, and this is possible because the media isn't actually bad.
You have no clue if it's actually functional or not, all you can do is to follow the reading/decoding protocol and verify the result against some redundancy data (parity bits, CRC etc.)
Media is bad, at least that's my definition, it's all a matter of definition, when it loses it's capability to store data. A bad sector can not be read from or written to.
It's bad when it loses data (fails to read/decode after following the protocol). You write A, but read B. Whether or not the controller or operating system marks the sector (or the whole drive) as 'bad' is another story. With similar errors, different controllers/systems may mark or not mark the faulty sector as 'bad' and they may employ different recovery protocols.

What matters for this conversation - 'bit rot' always happens due to unexpected changes in the medium.
Point is, it is not possible to write plain zeros and ones to storage devices it's simply to 'fuzzy' for that.
Depends on the implementation (again in HDD 0s and 1s are encoded by means of flipping magnetic dipoles which map to {0, 1} set exactly.
What we read from a drive is information that needs to be decoded to turn it back into zeros and ones, into digital data.
It all true in general, but that wasn't the point actually. :)

The discussion was around the OP's statement "I am talking about photos that deteriorate over time faster than the recording medium decays."

Another point, the OPs examples of 'bit rotting' could've occured because of huge range of different reasons, not necessarily because of the corrupted media.

--
https://www.instagram.com/quarkcharmed/
https://500px.com/quarkcharmed
 
Last edited:
The issue is that real world media does not encode zeros and ones.
All digital media 'encodes' zeros and ones. 0s and 1s are what's being encoded, and there's physical medium that has its state changed in order to 'encode' 0s and 1s.
The encoding is analog.
You may say so, but generally it's 'physical'.
For instance we want to use a no magnetic field to encode a zero, and a magnetic field of intensity 1 to store a one.
In case it's stored though magnetic field, 0s and 1s are literally encoded/recorded in magnetic dipoles that have just two states (if not faulty).
Now again you make it sound as if we store zeros and ones on a hard drive, as if individual magnetic grains represent individual bits and it simply isn't so. Disk heads don't read zeros or ones, they measure voltage spikes that are result of flux reversals, and these spikes don't signal zero or one, they're within an analog range we then interpret as zero or one. The information we get from a drive is analog, our interpretation, our decoding is what makes it digital.

It's not like I store a simple text document that contains the the word HELP, that I will be able to find individual magnetic grains that spell out 01001000 to represent the H in the word HELP.

--
Joep
 
Last edited:
The issue is that real world media does not encode zeros and ones.
All digital media 'encodes' zeros and ones. 0s and 1s are what's being encoded, and there's physical medium that has its state changed in order to 'encode' 0s and 1s.
The encoding is analog.
You may say so, but generally it's 'physical'.
For instance we want to use a no magnetic field to encode a zero, and a magnetic field of intensity 1 to store a one.
In case it's stored though magnetic field, 0s and 1s are literally encoded/recorded in magnetic dipoles that have just two states (if not faulty).
Now again you make it sound as if we store zeros and ones on a hard drive, as if individual magnetic grains represent individual bits and it simply isn't so.
The logical data being stored is a sequence of 0s and 1s. The physical carrier, obviously, isn't logical.
Disk heads don't read zeros or ones, they measure voltage spikes that are result of flux reversals, and these spikes don't signal zero or one, they're within an analog range we then interpret as zero or one. The information we get from a drive is analog, our interpretation, our decoding is what makes it digital.
Well if you think about it, decoding doesn't make it digital (the way you describe it) either. The signal from the HDD magnetic head turns into series of electrical impulses, goes through the bus, gets stored in RAM as charge in its cells - there's only electricity in the circuits and no abstract 0s and 1s at all.

DRAM and NAND memory cells aren't hugely different - would you say that NAND doesn't store 0s and 1s but DRAM does? :)
It's not like I store a simple text document that contains the the word HELP, that I will be able to find individual magnetic grains that spell out 01001000 to represent the H in the word HELP.
Yeah - but if you dive deeper, you won't find it anywhere in a computer.
 
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.
I can argue a user isn't interested in zeros and ones either. He stores a document, or a photo so I do not think that's a strong argument. And yes a computer handles all that data as zeros and ones. But once written to storage it is no longer zeros and ones.
Zeros and ones is an abstraction. The meaningful data being recorded is a sequence of 0s and 1s.
You're twisting this. All I was saying user is as less interested in charge levels as he is in zeros and ones. To the user everything that happens after he clicks the save button s a black box. He does not care about charge level, nor zeros and ones. To put is simply, I don't see what a 'user' has to do with this.

My point is that storage media store analog information, not digital data.
We intend to store zeros and ones, but we don't.
We intend and we do. We do store zeros and ones recorded/encoded as changes in the physical state of the media. We then need to follow a certain protocol to read them back.
A battery will leak charge over time, not because it's bad but because that's just the way it is. The outcome may be undesired because we have no power or in case of NAND memory the bit flips, but it's not due to bad media. We discussed 'bit rot', bit rot can occur on perfectly healthy media. It occurs because information stored as not as binary as we intend it to be. It's truly decay, it's not a one that drops all of a sudden, it gradually gets closer to a threshold until it's no longer decoded as a 1 but a 0 instead or vice versa.
Yes. Which means there was a physical change in the corresponding medium. Or something happened in the controller (e.g. voltage became too low for whatever reason). Or there's a parasitic capacitance because of a manufacturing defect.
No, there wasn't a change in the medium, there was a change in information it contained.
In my book media is 'bad' when a sector becomes unusable. Which is not the case here.
It's 'bad' when it breaks the protocol: you encoded a 1 and it reads a 0. Whether it's recoverable and whether it's marked as 'bad' is another story.
It isn't. It's functional, like I said if the decay is detected early enough data can be refreshed, and this is possible because the media isn't actually bad.
You have no clue if it's actually functional or not, all you can do is to follow the reading/decoding protocol and verify the result against some redundancy data (parity bits, CRC etc.)
No, it's bad when media is unfit to store data is my point. If I write data and I can read it back then it's not bad, it's functional.
Media is bad, at least that's my definition, it's all a matter of definition, when it loses it's capability to store data. A bad sector can not be read from or written to.
It's bad when it loses data (fails to read/decode after following the protocol). You write A, but read B.
That's not due to bad media, it's due to bad information. That's my whole point.
Whether or not the controller or operating system marks the sector (or the whole drive) as 'bad' is another story.
Which was exactly my point. It will only do so if it is unable to store new data. The information being corrupt is not a measure for the media being bad.
With similar errors, different controllers/systems may mark or not mark the faulty sector as 'bad' and they may employ different recovery protocols.

What matters for this conversation - 'bit rot' always happens due to unexpected changes in the medium.
No, it happens due to changes in analog information.
Point is, it is not possible to write plain zeros and ones to storage devices it's simply to 'fuzzy' for that.
Depends on the implementation (again in HDD 0s and 1s are encoded by means of flipping magnetic dipoles which map to {0, 1} set exactly.
No, that's not how this works. It's a simplistic layman interpretation.

The rest I ignored, it's same point over and over IMO.

Picture a grid of 8 x 8 buckets. It it allows us to store digital data, 64 bits. In your world the zero / one is represented by a bucket being full or empty if I correctly interpret what you're writing.

But PC storage does not work like that, it works more like: If bucket is > 50% full we interpret this as a one. If bucket is < 50% full it's a zero. So IOW we measure ranges, and if within certain range it's either a 0 or 1. So whether a bucket is 56% filled or 78% filled, we decode that as a 1. And whether a bucket is 0%, 23% filled or 48% we decode it as a 0. This is pretty much how NAND works. The information itself is not digital, where we set the threshold and decide what counts as 0 or 1 is what makes it digital. Magnetic storage also is interpretation of voltage spike due to flux reversal and thresholds we decide upon. The information itself is not digital, it's much more fluid.

We'd like to store zeros and ones, but we don't, simply because the physical world doesn't allow us. Instead we have to work with analog information the best we can.

Now without changing the media (the buckets) but due to external causes the analog information inside the buckets (amount of fluid) can change (rain, drought). In NAND this could for example translate to read/write disturbance, we change analog information in cells we do not intend to change. But even without those external factors and disturbances, the fluid slowly evaporates like a NAND cell will slowly leak electrons. It's stuff like this that results in what we tend to describe (I believe) as bit rot.

IOW, the media did not deteriorate, The analog information did. If we detect it using ECC for example we can even correct it on the fly but also update the information in the buckets as long as the media itself isn't bad.

Over time indeed, due to for example corrosion buckets may lose some capacity to hold fluid, it simply leaks out. That's bad media in my book. The media (buckets) lost it's ability to store information. Or a buckets gets punctured, like a hard drive that was dropped, heads scratching the surface as a result, scraping off the layer of magnetic material, that's bad media in my book and it's clearly not what's meant by bit rot.

--
Joep
 
Last edited:
You're both saying approximately the same thing. The media has gone bad because it lost one or more charged bits.
No, I feel we do not. First the assumption that data will always be a 0 or 1 is incorrect IMO, an over simplification. The media simply does not store zeros and ones. And it does not have to go bad for bits to flip.
Depends on how you define 'data'. A user is not interested in storing the charge level. The digital data is 0s and 1s.

It's encoded as voltage level (in certain types of digital storages).
The issue is that real world media does not encode zeros and ones.
All digital media 'encodes' zeros and ones. 0s and 1s are what's being encoded, and there's physical medium that has its state changed in order to 'encode' 0s and 1s.
While we are conceptually writing 0s and 1s, it's far more complicated under the hood.

Consider a situation where you were encoding zeros and ones by either punching, or not punching hole in a piece of paper. On the surface, it seems it should be obvious exactly what is encoded in each location. Yet, back in 2000, it became clear that sometimes it isn't clear whether a hole is punched or not (Google "Hanging Chad").

In the real world, an analog signal that should be 0 or 5 volts, might end up being 2.5 volts.

Things get even worse as some recording media does not record 0's and 1's, but multiple bits. Some SSDs store 5 bits, That's 32 discreet analog levels.

The encoding is analog.
You may say so, but generally it's 'physical'.
It's a physical encoding, but there are seldom only two possible states for the medium to be in. Generally there's a wide range of possible states. Some of them clearly represent 0s, some clearly represent 1s, and some can be ambiguous.
For instance we want to use a no magnetic field to encode a zero, and a magnetic field of intensity 1 to store a one.
In case it's stored though magnetic field, 0s and 1s are literally encoded/recorded in magnetic dipoles that have just two states (if not faulty).
Ah, but the world is inherently faulty.

While you see the recorded magnetic data as having two states, that isn't quite true. The strength of the magnetic field can vary. If the strength is week, the noise inherent in the real world can dwarf the actual signal.

Remember, in an effort to pack as much data onto a platter as possible, the drives write the smallest (and therefore weakest) signal that they can get by with. There is always a tradeoff between denser storage and more underlying errors.
When we are reading the data back, the analog values won't be exactly 0 and 1. We may see values like 0.1, 0.9, 0.2 and 0.8. It's when we stat seeing values around 0.5 that we lose confidence as to which binary value it represents.

Fields on Magnetic media can fade over time. The charge on Solid State Drives can fade over time. One strategy is to periodically read and refresh the data, but even this is not 100% reliable.

Modern devices are exceedingly reliable, but they are not perfect. However, we can control how often we expect to see an error. By using various techniques we can reduce the expected error rates to less than one bit error in a thousand years. However, we can save money by living with error rates that are a little less than that.
Everything decays in this Universe.
In the real world, the underlying storage mechanisms are inherently error prone. Digital storage systems address this by storing redundant information that allows the controller to detect and correct errors. We can never guarantee that 100% of all errors will be detected and corrected. However, by adding additional redundancy, we can get very close to 100% error free operation. It's a cost vs. reliability tradeoff.
 
The issue is that real world media does not encode zeros and ones.
All digital media 'encodes' zeros and ones. 0s and 1s are what's being encoded, and there's physical medium that has its state changed in order to 'encode' 0s and 1s.
The encoding is analog.
You may say so, but generally it's 'physical'.
For instance we want to use a no magnetic field to encode a zero, and a magnetic field of intensity 1 to store a one.
In case it's stored though magnetic field, 0s and 1s are literally encoded/recorded in magnetic dipoles that have just two states (if not faulty).
Now again you make it sound as if we store zeros and ones on a hard drive, as if individual magnetic grains represent individual bits and it simply isn't so.
The logical data being stored is a sequence of 0s and 1s. The physical carrier, obviously, isn't logical.
I fail to see the point. In the end, data is stored as analog information.
Disk heads don't read zeros or ones, they measure voltage spikes that are result of flux reversals, and these spikes don't signal zero or one, they're within an analog range we then interpret as zero or one. The information we get from a drive is analog, our interpretation, our decoding is what makes it digital.
Well if you think about it, decoding doesn't make it digital (the way you describe it) either. The signal from the HDD magnetic head turns into series of electrical impulses, goes through the bus
It does not just flow, it's processed, decoded, converted, digitized and whatnot.
, gets stored in RAM as charge in its cells - there's only electricity in the circuits and no abstract 0s and 1s at all.
Exactly.
DRAM and NAND memory cells aren't hugely different - would you say that NAND doesn't store 0s and 1s but DRAM does? :)
NAND cells trap electrons is what I said. We 'count' them and then decide it value is interpreted as zero or one. If DRAM works like that too (I don't know) then it's the same deal.
It's not like I store a simple text document that contains the the word HELP, that I will be able to find individual magnetic grains that spell out 01001000 to represent the H in the word HELP.
Yeah - but if you dive deeper, you won't find it anywhere in a computer.
Point? You were the one that made it sound that storage media store digital data, rather than analog information.

--
Joep
 
Last edited:

Keyboard shortcuts

Back
Top