Which is more trustworthy for file storage, RAM or the good ol' hard drive?

[Plusiotis]

http://news.bbc.co.uk/1/hi/programmes/click_online/6897265.stm

 

[Leon Wittwer]

I assume by "RAM" you are talking about flash memory. Both media are quite reliable and with mulitple copies should meet most needs. Whether flash memory becomes preferred for storage will depend mostly on cost per byte to store. Another factor is read/write speed. This is not very important for archival storage but it important to normal day to day file access. Flash memory is still somewhat slow. Time and the market will tell how all this comes out.
--
Leon
http://homepage.mac.com/leonwittwer/landscapes.htm

 

[Leon Wittwer]

Flash memory will have to come in large sizes to be competitive.
--
Leon
http://homepage.mac.com/leonwittwer/landscapes.htm

 

[JRP64]

It's just going to be a matter of time before flash memory replaces hard drives, they offer no mechanical parts and faster retrieval of data. Doing Photo editing in the future is going to be so much faster than on today's computers, it's going to be fun.

 

[Mark Bessey]

It's just going to be a matter of time before flash memory replaces
hard drives, they offer no mechanical parts and faster retrieval of
data.

I'd been wondering about this, so I did some math...

Flash memory is running around $10/GB at retail, and a 500 gigabyte hard drive is about $100. So currently, Flash is about 50 times more expensive than a hard drive. Assuming that Flash doubles in size every 18 months (Moore's Law), then it'll be about 6 years until Flash is price-competitive with a hard drive, assuming of course that hard drive prices and capacities stay the same.

For what it's worth, current Flash memory devices are not actually faster than hard drives in all cases. Read speed is much better, write speed is less impressive, and random-write performance is abysmal. You won't see Flash drives in database servers for a while yet.

 

[Plusiotis]

.. hard drives are proven to be archival ... flash memory isn't ... and never will be.,,, this is why digital movies (e.g. Shrek, Toy Story etc.) are archved onto emulsion film and stored in an atomic-bomb-proof vault somewhere in the USA.

 

[GodSpeaks]

Hard drives are NOT archival. Not only do they fail, but the failure rate is higher with non-use than with constant use.

--

The greatest of mankind's criminals are those who delude themselves into thinking they have done 'the right thing.'

• Rayna Butler

 

[nickleback]

Another factor is read/write speed. This
is not very important for archival storage but it important to normal
day to day file access. Flash memory is still somewhat slow.

Currently, flash memory read/write is in the same ballpark as hard disks. Remember with hard disks actual performance is the sustained times. The peak performance given is simply copying data in/out of the on-board RAM cache.

--
Seen in a fortune cookie:
Fear is the darkroom where negatives are developed

 

[Nathan Yan]

In my experience, hard drive speed has blown away flash memory. My 7200rpm external hard drive writes files way faster than my USB key.

 

[nickleback]

Flash memory is running around $10/GB at retail, and a 500 gigabyte
hard drive is about $100. So currently, Flash is about 50 times more
expensive than a hard drive. Assuming that Flash doubles in size
every 18 months (Moore's Law), then it'll be about 6 years until
Flash is price-competitive with a hard drive, assuming of course that
hard drive prices and capacities stay the same.

And of course hard drives will improve, too. But flash as desktop replacement is a ways off. Flash makers are targeting sub-notebook (1.8") drives first, where low power and high shock resistance are strong points in favor of flash.

For what it's worth, current Flash memory devices are not actually
faster than hard drives in all cases. Read speed is much better,
write speed is less impressive, and random-write performance is
abysmal.

You've got this completely backwards. Flash clobbers disk in random I/O operations per second. Hard disks can do maybe 50 to 100. Flash is in the 5000 range. Hard disks need to move the head and wait for the correct sector to spin under the head. This takes significant time, even with 15k RPM disks. The difference is even greater with slower disks, as used in laptops and ipods.

You won't see Flash drives in database servers for a while yet.

Not for the database itself, as that's too expensive. But as a DB cache, it's got possibilities. Actually doing database cache in flash is a hot topic, precisely because of the slow random I/O performance of disks.

--
Seen in a fortune cookie:
Fear is the darkroom where negatives are developed

 

[nickleback]

In my experience, hard drive speed has blown away flash memory. My
7200rpm external hard drive writes files way faster than my USB key.

Wow, that's a fair comparison. FWIW, most cheap (and some expensive) USB key drives use MLC memory that is pretty fast to read but abysmally slow to write. That isn't the type of memory that you'd want to use in a hard-disk replacement.

For a better comparison, take a fast CF or SD card and stick it in a good USB reader. It'll still be slower than the hard disk, but that'll be the fault of the USB reader.

Now imagine building a compact drive using 4 fast SD cards bundled together using RAID0. Or better yet, skip the SD interface, solder 16 flash devices (i.e. the individual components inside an SD or CF card) together on a little circuit board with RAID0 controller integrated.

--
Seen in a fortune cookie:
Fear is the darkroom where negatives are developed

 

[James Arnold 1971]

Not for the database itself, as that's too expensive. But as a DB
cache, it's got possibilities. Actually doing database cache in
flash is a hot topic, precisely because of the slow random I/O
performance of disks.

I am not a real expert on databases but I know large ones separate indexes from the the actual data on either separate drives or seaparate computers? Is this something that the flash drives could also be used for? To reduce the time required to look up the location of data?

Cheers
J

 

[Bad Sneakers]

Hard drives are NOT archival. Not only do they fail, but the failure
rate is higher with non-use than with constant use.

Not challenging this statement (re constant use vs. non-use), um, er, GodSpeaks (sheesh!), but I never heard this before - do you have a link/reference to which you might point me?

-----------------------------------
I'm just cropping reality...

 

[James Arnold 1971]

.. hard drives are proven to be archival ... flash memory isn't ...
and never will be.,,, this is why digital movies (e.g. Shrek, Toy
Story etc.) are archved onto emulsion film and stored in an
atomic-bomb-proof vault somewhere in the USA.

Hard drives most certainly not archival, if they were why would they archive digital data on film?

BTW that is really interesting about the archiving on film! The one failing of diital media is that it requires a lot of support.. media readers, compatable software, media that is stable and lasting, and ultimatley electricity. If someone did want something to stand a chance of survning a catastrphic event digital may not be the best choice.

Do they say what the life expectancy of the archival emulsion they are using will be? I would think that they should be able to retain something for several hundred years?

Jamie

 

[Kendall Helmstetter Gelner]

Hard drives are NOT archival. Not only do they fail, but the failure
rate is higher with non-use than with constant use.

Hard drives are more archival than anything else we use today. Just what then would you advise, if not a hard drive?

CD/DVD? That media decays over time.

Flash? The charge holding the memory also gives out over time.

Hard drives, when not in use, do not suffer loss of memory. The data is more deeply imprinted on the media, and even if the mechanics of the drive give out (BTW, I reject your assertion that non-use leads to a higher failure rate - what is the proof for that??) the media is still quite readable in ways that CD/DVD or flash will simply not be - when that data is gone, it is gone. Consider that recovery experts can actually recover data overwritten several times on a hard drive, and you'll realize just how more stable that form of storage is.

In some ways printing to film is of course the most tried and true archival solution - but it does not represent a perfect copy the way hard drive duplication can, and is subject to environmental decay if not carefully stored in a way that is not true of hard drives..

Basically with any archiving process, they key is duplication and dispersal, and hard drives make that easy. I think Blu-Ray will aid a little beyond that, the discs may not last as long as hard drive but they make it even easier to send images to a number of different location since they can be mailed flat.

--
---> Kendall
http://InsideAperture.com
http://www.pbase.com/kgelner
http://www.pbase.com/sigmadslr/user_home

 

[James Arnold 1971]

Hard drives are NOT archival. Not only do they fail, but the failure
rate is higher with non-use than with constant use.

Hard drives are more archival than anything else we use today. Just
what then would you advise, if not a hard drive?

CD/DVD? That media decays over time.

Flash? The charge holding the memory also gives out over time.

Hard drives, when not in use, do not suffer loss of memory. The data
is more deeply imprinted on the media, and even if the mechanics of
the drive give out (BTW, I reject your assertion that non-use leads
to a higher failure rate - what is the proof for that??) the media is
still quite readable in ways that CD/DVD or flash will simply not be

• when that data is gone, it is gone. Consider that recovery experts

can actually recover data overwritten several times on a hard drive,
and you'll realize just how more stable that form of storage is.

I don't know about the non-use part (sounds a little like an urban legend) but hardrives are not close to being the most archival medium today. Tape still gets this nod. no-one I know considers the hardrive reliable for stata storage. Of any kind really (thus the nightly backups)

http://www.cci-icc.gc.ca/PID/faq_2_e.shtml#2

I have never heard anyone any article or technical document discuss data on a hard disk being more deeply imprinted and therfore more reliable. if you have more info on this I would be interested in it. Data recovery from a damaged drive while sometimes possible is extremly costly.

As well I would also be interested in learning more about how data that has been truley written over can be recovered. i have never heard of this before (well beyond TV fiction). I understand that deleted or damaged files can be recovered after more data has been added to the disk, so long as the physical location containing the data was not written to. If you have any more information on this I would be interested to read about it.

Thanks
james

In some ways printing to film is of course the most tried and true
archival solution - but it does not represent a perfect copy the way
hard drive duplication can, and is subject to environmental decay if
not carefully stored in a way that is not true of hard drives..

Basically with any archiving process, they key is duplication and
dispersal, and hard drives make that easy. I think Blu-Ray will aid
a little beyond that, the discs may not last as long as hard drive
but they make it even easier to send images to a number of different
location since they can be mailed flat.

--
---> Kendall
http://InsideAperture.com
http://www.pbase.com/kgelner
http://www.pbase.com/sigmadslr/user_home

 

[nickleback]

I don't know about the non-use part (sounds a little like an urban
legend) but hardrives are not close to being the most archival medium
today.

Nope, but they are cheap enough that disk backup is becoming an option. Have 3 copies of everything in different places. That's Google's strategy...

http://storagemojo.com/?page_id=152

Data recovery from a damaged drive while sometimes possible is
extremly costly.

And data recovery from a damaged tape? Neither is a good strategy. Keep multiple backups so you don't have to resort to data recovery services.

As well I would also be interested in learning more about how data
that has been truley written over can be recovered. i have never
heard of this before (well beyond TV fiction)

Take a pencil, eraser and paper. Write a "1" on the paper. Now erase it and write a "0". Now erase it and write another one. Do you see the old 0 and 1 beneath the new 1? Similarly with disk drives, there is a residual charge that can be read. I can't find a link right now, but I think US military requires data be overwritten with random garbage 15 times before it is considered gone.

That's another advantage of flash. No residual charge to be read. It's truly binary. Overwrite the data once and the data is gone.

--
Seen in a fortune cookie:
Fear is the darkroom where negatives are developed

 

[excal]

For all those yapping away above, it really depends on whether you are talking NOR or NAND flash. Don't know the different... tusk tusk... go an look it up!

XIP NOR is the way to go, but it's not cheap.

Excal

 

[Mark Bessey]

For what it's worth, current Flash memory devices are not actually
faster than hard drives in all cases. Read speed is much better,
write speed is less impressive, and random-write performance is
abysmal.

You've got this completely backwards. Flash clobbers disk in random
I/O operations per second. Hard disks can do maybe 50 to 100. Flash
is in the 5000 range. Hard disks need to move the head and wait for
the correct sector to spin under the head. This takes significant
time, even with 15k RPM disks. The difference is even greater with
slower disks, as used in laptops and ipods.

I did mention that reads were much faster. When you write to a flash device, you have to erase an entire block and re-write it, which takes significant time (blocks on a Flash device are also usually much larger than the blocks on a hard disk).

So writing a little bit of data here, then a little over there, is actually slower on a Flash device than it is on a magnetic drive. Actually, you can fake your way around that for a while by just writing to a new block every time, but that only works until you've written to all the blocks on the device.

What you see in those cases is that write performance looks really good until you get to a certain amount of data transferred, then it suddenly drops to 10x or 100x lower performance. I've actually measured this effect on real Flash devices.

I can't find a good reference online for worst-case performance on Flash disks, but it is a lot worse than you might expect. The transfer-rate specs you see quoted are always best-case numbers, both for Flash and Hard Disks.

 

[nickleback]

So writing a little bit of data here, then a little over there, is
actually slower on a Flash device than it is on a magnetic drive.
Actually, you can fake your way around that for a while by just
writing to a new block every time, but that only works until you've
written to all the blocks on the device.

You can say the same about hard disks. Randomly seeking and writing takes a lot of time.

What you see in those cases is that write performance looks really
good until you get to a certain amount of data transferred, then it
suddenly drops to 10x or 100x lower performance. I've actually
measured this effect on real Flash devices.

A 100x hit would bring it into the hard disk ballpark. And if you want to see truly awful performance, stick a fast SD card into a Palm device. Any one I've ever used writes tiny amounts of data at a time, which really hits SD performance. It would be similarly slow if you could stuff a microdrive in the system. Why Palm OS designers haven't heard of buffered I/O is beyond me.

I can't find a good reference online for worst-case performance on
Flash disks, but it is a lot worse than you might expect.

Which means that a caching controller with a decent amount of RAM can make a lot of difference, just as it does on hard disks.

The
transfer-rate specs you see quoted are always best-case numbers, both
for Flash and Hard Disks.

Of course, there is a difference between peak and sustained, not to mention marketing vs reality.

Can you find pathological cases for disks and flash? Sure. Is flash a lot slower than disks? No.

--
Seen in a fortune cookie:
Fear is the darkroom where negatives are developed