Value System Build

[kelpdiver]

SushiEater wrote:

There is a huge difference between a $250000 and $700. Not the same reality at all. I have never lost anything except maybe for a few emails that were not backed up. But certainly not photos.

because you only have 1 or 2 computers. And you're prepared to lose a few emails or recent photo edits if the dice come up snake eyes. Funny enough, the odds of rolling two 1's is pretty close to the odds of a 2 stripe volume failing in a year.

The time it didn't it was not my fault, it was fixed and restored from backup. It was a fluke. Plane and simple. Backup is the key. And if you are advocating that people should not backup or doing less backups if using a single disk you are a fool.

I think this is what you call "skewing the conversation in your direction." Or more classically, making up a straw man. No one has spoken of skipping backups. I advocate for a remote backup at least 50-100 miles away from your home.

And unless your backups are continuous (they're not), you're losing data with a failure. Goes back to how much or little you value your data.

What a relief! And when those bad block writes happen, you probably won't even know.

Stop inventing something that does not exists. You are still not going to win.

the concern around bit errors became real a few years ago, most notably for the RAID5 crowd. But if you're writing as much as you claim, so much so that you "ate up a year of SSD life," then it definitely should be a concern to you as well. It's no coincidence that data scrubbing is part of MS's roadmap for their replacement to ntfs.

 

[malch]

SushiEater wrote:
Regardless of statistics you are never going to get in to the car accident if you don't drive. Right?

You mean:

* Passengers are never injured?

* Bicyclists are never hit by vehicles?

* Pedestrians are never hit by vehicles?

Folks sitting at home watching TV have been killed by trucks crashing into their house.

So you will never lose your data if you backup!!!

Wrong, at so many levels it's hard to know where to start!

Let's put it this way. Large numbers of people have run into ever so big problems when they lost data and attempted to restore it from a backup copy only to discover that the backup was missing, corrupt, incomplete, out-of-date, or otherwise less than they hoped when they made it.

 

[Tom_N]

SushiEater wrote:

Tom_N wrote:

A RAID 0 setup with two drives would have only an 81% chance (0.9 squared = 0.81) of surviving intact for that time. One that striped over four drives would have only a 66% chance (0.9 to the fourth power = 0.6561) of surviving.

Lets say you have 4 drives not in the RAID. All 4 drives contain data. One has crashed and you lost that data. So you still lost some data maybe not all of it but 25% of it.

This is the point. With independent drives, if one fails, you only lose the files on it. With RAID 0, if any drive fails, you lose everything.

But you still lost it because you did not have a backup. But if you do backups anyway what difference does it make where the data is? None.

You can lose backups, too. If you lose the original and all your backups (for a particular file), that file is gone. For any given backup strategy ("risk of losing all backups"), the risk of losing your files is higher if you use RAID 0 for the original, than if you don't.

Maybe the risk does not bother you, or you have extra backups to compensate. Either way, that does not negate the math.

 

[malch]

SushiEater wrote:

Again, stop inventing stuff just to skew conversation in your direction. Not going to work.

Bit rot is real. Today, the concern mainly applies to corporate users. But data volumes are exploding and home users with a few TB of data are now rather common. As those volumes reach 10, 20 and more TB the risk of data loss due to bit rot is becoming legitimate and significant.

It won't be long before most home users need error correcting file systems with scrubbing etc. A lot of power users are already taking steps to mitigate the risk with checksums, ZFS-based file servers etc.

 

[SushiEater]

kelpdiver wrote:

SushiEater wrote:

There is a huge difference between a $250000 and $700. Not the same reality at all. I have never lost anything except maybe for a few emails that were not backed up. But certainly not photos.

because you only have 1 or 2 computers. And you're prepared to lose a few emails or recent photo edits if the dice come up snake eyes. Funny enough, the odds of rolling two 1's is pretty close to the odds of a 2 stripe volume failing in a year.

Actually I have 4, one is a backup.

The time it didn't it was not my fault, it was fixed and restored from backup. It was a fluke. Plane and simple. Backup is the key. And if you are advocating that people should not backup or doing less backups if using a single disk you are a fool.

I think this is what you call "skewing the conversation in your direction." Or more classically, making up a straw man. No one has spoken of skipping backups. I advocate for a remote backup at least 50-100 miles away from your home.

And unless your backups are continuous (they're not), you're losing data with a failure. Goes back to how much or little you value your data.

And what happens if their server goes down? And don't tell me it is not possible. Those hackers love hacking servers especially those containing data.And what if for some reason or another they decide to go out of business?

It is totally stupid to use remote servers to store data.

What a relief! And when those bad block writes happen, you probably won't even know.

Stop inventing something that does not exists. You are still not going to win.

the concern around bit errors became real a few years ago, most notably for the RAID5 crowd. But if you're writing as much as you claim, so much so that you "ate up a year of SSD life," then it definitely should be a concern to you as well. It's no coincidence that data scrubbing is part of MS's roadmap for their replacement to ntfs.

If you would bother to read what actually I did to "eat up" of the SSD life you would understand. And I don't do it anymore. It was more of the experiment to prove few things.

 

[kelpdiver]

malch wrote:

Let's put it this way. Large numbers of people have run into ever so big problems when they lost data and attempted to restore it from a backup copy only to discover that the backup was missing, corrupt, incomplete, out-of-date, or otherwise less than they hoped when they made it.

The worst example I encountered was in the late 90s. Someone or somehow mangled an array, wiping out at least the first cylinder on each drive. So we went to tape...and found that we had a batch that appeared to write fine, but were not readable. (which is why if you use offline storage media, you need to occasionally go through the routine of trying to restore). We were able to get nearly all of the files restored (for big $$) by Ontrack or one of the others, but lost directory paths so it was still fun work for us.

Anyone that uses DVD or Blu-rays for backup purposes should be doing verification scans on a periodic basis.

 

[SushiEater]

malch wrote:

SushiEater wrote:

Again, stop inventing stuff just to skew conversation in your direction. Not going to work.

Bit rot is real.

I agree with you.

Today, the concern mainly applies to corporate users. But data volumes are exploding and home users with a few TB of data are now rather common. As those volumes reach 10, 20 and more TB the risk of data loss due to bit rot is becoming legitimate and significant.

No, it also applies to CPUs.

It won't be long before most home users need error correcting file systems with scrubbing etc. A lot of power users are already taking steps to mitigate the risk with checksums, ZFS-based file servers etc.

DOOM DOOM DOOM.

DOOM!

 

[kelpdiver]

SushiEater wrote:

And what happens if their server goes down? And don't tell me it is not possible. Those hackers love hacking servers especially those containing data.And what if for some reason or another they decide to go out of business?

It is totally stupid to use remote servers to store data.

I live in earthquake country. Some live in hurricane land, or tornado, or flood plains. Some just live in areas with a lot of burglars. It's totally stupid to keep all copies of your data on site. One fire wipes you out. And it's nearly guaranteed that any offline backup (like safety deposit boxes) don't get updated as often as they should. So you may be losing weeks or months of updates. I can update as often as I want, and I can verify the backups as often as I want (and with automated processes).

Downtime is irrelevant for backups - if they're down for 3 hours, why would you care? It is a valid concern for businesses using it for operations, however few of them are competent at datacenter management. The cloud companies are far more reliable. It's just that when they have a hiccup, many companies get hit at once. Very visible.

Going out of business? A legitimate fear if you shop for the lowest rates. I use Amazon who is going nowhere.

Data loss? You're the one flying without any sort of parachute. They would be using RAID6 at a minimum, along with incremental and full backup sets.

Security is pretty low on the list of problems. Again, the cloud people run SaaS as their core competency. Few companies can match this - their core competency is whatever business they're in.

If you would bother to read what actually I did to "eat up" of the SSD life you would understand. And I don't do it anymore. It was more of the experiment to prove few things.

You refuse to repeat it- keep saying look it up from a few weeks ago. I translate that "I don't really want any more scrutiny of the facts," so I ignore it for the most part. Someone whose workload is truly write dominated may in fact do better with R0. But most people are 90% reads or higher.

If you're saying now that you don't do this (whatever) any more, then perhaps your justification for slower R0 doesn't stand anymore. Frankly, if speed matters and your pano work requires wearing down SSDs, it may still worth it for the performance gains. $200 for a 240GB drive (Or 650 for the 1TB 840 EVO) that needs to be replaced every 3 years is cheap for a power user. But you're not paying me to solve your problems, so I leave it to you.

 

[SushiEater]

malch wrote:

SushiEater wrote:

Regardless of statistics you are never going to get in to the car accident if you don't drive. Right?

You mean:

* Passengers are never injured?

* Bicyclists are never hit by vehicles?

* Pedestrians are never hit by vehicles?

Folks sitting at home watching TV have been killed by trucks crashing into their house.

No, I meant you hide under the rock.

So you will never lose your data if you backup!!!

Hasn't happen yet.

Wrong, at so many levels it's hard to know where to start!

You are correct! You are wrong on so many levels.

Let's put it this way. Large numbers of people have run into ever so big problems when they lost data and attempted to restore it from a backup copy only to discover that the backup was missing, corrupt, incomplete, out-of-date, or otherwise less than they hoped when they made it.

Yesterday I was on the red carpet shoot. While my wife was driving I backed up CF cards to the Hyperdrive. After I came home and before I did anything at all I copied all CF cards to the RAID 0 HD. Before I even touched anything at all I turned on my 3TB external backup (RAID 0) and copied folder from my internal RAID 0. Then I edited and posted some pictures on my website and replaced backup on 3TB backup. And then I turned on my 6TB RAID 0 and backed up there as a final destination. Today I burnt two blue-ray disks so I can take one to another location.

And while I was doing it I still had original copies on those CF cards and Hyperdrive. And those will stay on CF cards for a few days just in case and on Hyperdrive for at least until I need space. So no I am not one of those people you described above.

It is called discipline and you should learn some.

 

[malch]

kelpdiver wrote:
Anyone that uses DVD or Blu-rays for backup purposes should be doing verification scans on a periodic basis.

Yeah but folks should also verify the whole restore process from time to time. Media issues are not the only source of catastrophe. The whole process needs to be verified and tested to an appropriate degree.

Suppose a system drive fails but you have a nice image on a USB drive. Can you find that bootable recovery disk? You did make one, right? Does it actually boot on your current hardware? Can it see, read, and restore the image?

Backups are useless if your restore process is flawed/broken!

 

[malch]

kelpdiver wrote:

If you're saying now that you don't do this (whatever) any more, then perhaps your justification for slower R0 doesn't stand anymore. Frankly, if speed matters and your pano work requires wearing down SSDs, it may still worth it for the performance gains. $200 for a 240GB drive (Or 650 for the 1TB 840 EVO) that needs to be replaced every 3 years is cheap for a power user. But you're not paying me to solve your problems, so I leave it to you.

When I've bought SSD's, I expect them to last 3 years. Most modern drives will last way longer in normal workstation settings but they'll be very obsolete within 3 years. The better products come with a 3 year manufacturer warranty anyway.

This makes it easy. Plan on 3 years useful life and if the economic proposition works for you, go for it. I've purchased 4 SSD's on that basis so far.

 

[SushiEater]

kelpdiver wrote:

SushiEater wrote:

And what happens if their server goes down? And don't tell me it is not possible. Those hackers love hacking servers especially those containing data.And what if for some reason or another they decide to go out of business?

It is totally stupid to use remote servers to store data.

I live in earthquake country.

I live few mile from San Andreas Fault. So what?

Some live in hurricane land, or tornado, or flood plains. Some just live in areas with a lot of burglars. It's totally stupid to keep all copies of your data on site.

It is. Read my other post. I don't keep all data on site. And I also have big dog.

One fire wipes you out. And it's nearly guaranteed that any offline backup (like safety deposit boxes) don't get updated as often as they should. So you may be losing weeks or months of updates. I can update as often as I want, and I can verify the backups as often as I want (and with automated processes).

Chance of that happening is very slim. Probably 1000000 times slimmer than crashed RAID 0.

And if your internet service goes down (very good chance for many reasons) you would not be doing anything you described above.

Downtime is irrelevant for backups - if they're down for 3 hours, why would you care?

If the earthquake, flood etc... hits, you will not have internet for a while and you will not have backup either. And it could happen for many other reasons.

It is a valid concern for businesses using it for operations, however few of them are competent at datacenter management. The cloud companies are far more reliable. It's just that when they have a hiccup, many companies get hit at once. Very visible.

Going out of business? A legitimate fear if you shop for the lowest rates. I use Amazon who is going nowhere.

I simply don't trust my data to anyone but me. Your backup maybe automated but people who work there are not. Someone can and will read your files. Amazon can get hacked.

Data loss? You're the one flying without any sort of parachute. They would be using RAID6 at a minimum, along with incremental and full backup sets.

Read my other post what I do. I am not going to repeat myself.

Security is pretty low on the list of problems. Again, the cloud people run SaaS as their core competency. Few companies can match this - their core competency is whatever business they're in.

The more security they have the more pleasure it would be for a hacker to break in.

If you would bother to read what actually I did to "eat up" of the SSD life you would understand. And I don't do it anymore. It was more of the experiment to prove few things.

You refuse to repeat it- keep saying look it up from a few weeks ago. I translate that "I don't really want any more scrutiny of the facts," so I ignore it for the most part. Someone whose workload is truly write dominated may in fact do better with R0. But most people are 90% reads or higher.

Just for you: I took a lot of my large panos and reprocessed them on SSD. It took couple of weeks. And after that SSDlife reported a year less than I had before. So translate all you want but you still be wrong.

If you're saying now that you don't do this (whatever) any more, then perhaps your justification for slower R0 doesn't stand anymore.

It wasn't because of the speed. I did it because of the speed and stop doing it because of the SSD life.

Frankly, if speed matters and your pano work requires wearing down SSDs, it may still worth it for the performance gains.

So you do know what I did after all. As far as performance gain, it wasn't much if not at all. I wasn't counting the seconds.

$200 for a 240GB drive (Or 650 for the 1TB 840 EVO) that needs to be replaced every 3 years is cheap for a power user. But you're not paying me to solve your problems, so I leave it to you.

I am not paying you to be involve here and distributing the disinformation and yet you are still here.
My problem is in your head only. Just get it out there before it eats you up.

Like I said before there is nothing you can say that will stop me from using RAID 0 which I have been using for a very long time now. The same thing applies to overclocking that allegedly shortens life of the CPU or could even burn it.

 

[malch]

SushiEater wrote:

Yesterday I was on the red carpet shoot. While my wife was driving I backed up CF cards to the Hyperdrive. After I came home and before I did anything at all I copied all CF cards to the RAID 0 HD. Before I even touched anything at all I turned on my 3TB external backup (RAID 0) and copied folder from my internal RAID 0. Then I edited and posted some pictures on my website and replaced backup on 3TB backup. And then I turned on my 6TB RAID 0 and backed up there as a final destination. Today I burnt two blue-ray disks so I can take one to another location.

OMG, you backup RAID 0 to RAID 0 and RAID 0?

Now, that's what I call a serious case of RAIDitis!

Logout and call 911 or go to the nearest emergency room!

 

[SushiEater]

malch wrote:

kelpdiver wrote:

Anyone that uses DVD or Blu-rays for backup purposes should be doing verification scans on a periodic basis.

Yeah but folks should also verify the whole restore process from time to time. Media issues are not the only source of catastrophe. The whole process needs to be verified and tested to an appropriate degree.

Suppose a system drive fails but you have a nice image on a USB drive. Can you find that bootable recovery disk? You did make one, right? Does it actually boot on your current hardware? Can it see, read, and restore the image?

Backups are useless if your restore process is flawed/broken!

Don't image anything. Just straight copy. This way you can just look at your backup the normal way.

Plus a single backup is not enough. Right now drives are cheap and fast so there is no excuse not to have multiple backups.

BTW, I periodically use EASEUS software to clone my SSD to another SSD so if something happens I can have my computer running in a few minutes. No need to create image and restore it and pray it will restore. Since there is absolutely no data but software and OS on SSD if something happens I might only lose Windows update.

 

[malch]

SushiEater wrote:

One fire wipes you out. And it's nearly guaranteed that any offline backup (like safety deposit boxes) don't get updated as often as they should. So you may be losing weeks or months of updates. I can update as often as I want, and I can verify the backups as often as I want (and with automated processes).

Chance of that happening is very slim. Probably 1000000 times slimmer than crashed RAID 0.

"An estimated 360,900 residential building fires were reported to United States fire departments each year and caused an estimated 2,495 deaths, 13,250 injuries and $7 billion in property losses."

Source: FEMA

 

[SushiEater]

malch wrote:

SushiEater wrote:

Yesterday I was on the red carpet shoot. While my wife was driving I backed up CF cards to the Hyperdrive. After I came home and before I did anything at all I copied all CF cards to the RAID 0 HD. Before I even touched anything at all I turned on my 3TB external backup (RAID 0) and copied folder from my internal RAID 0. Then I edited and posted some pictures on my website and replaced backup on 3TB backup. And then I turned on my 6TB RAID 0 and backed up there as a final destination. Today I burnt two blue-ray disks so I can take one to another location.

OMG, you backup RAID 0 to RAID 0 and RAID 0?

Now, that's what I call a serious case of RAIDitis!

Yep, and I build it all myself using NexStar and Startech enclosures and all connected with eSATA.

Logout and call 911 or go to the nearest emergency room!

Better than have the case of nonconformism and dissidence. Someone can call government on you and you have a chance to rot in jail somewhere in Cuba for the rest of your life. :- -x

Can't call 911, using multiple MagicJacks. I am doomed!!!!!

 

[malch]

SushiEater wrote:

Don't image anything. Just straight copy. This way you can just look at your backup the normal way.

It depends on the situation. Some things are best imaged. Others (like photos) are best copied.

Since there is absolutely no data but software and OS on SSD if something happens I might only lose Windows update.

This is not a safe assumption for most folks. Windows tucks away a lot of user data on the C drive and in some cases you really have to jump through hoops to prevent it. In other cases, it really can't be relocated elsewhere.

Besides, it's not just Windows updates. How about all of your application updates? In my case, it take 3-4 days of full time work to install, configure, and update all of my many applications. Better to clone or image.

 

[SushiEater]

malch wrote:

SushiEater wrote:

One fire wipes you out. And it's nearly guaranteed that any offline backup (like safety deposit boxes) don't get updated as often as they should. So you may be losing weeks or months of updates. I can update as often as I want, and I can verify the backups as often as I want (and with automated processes).

Chance of that happening is very slim. Probably 1000000 times slimmer than crashed RAID 0.

"An estimated 360,900 residential building fires were reported to United States fire departments each year and caused an estimated 2,495 deaths, 13,250 injuries and $7 billion in property losses."

Source: FEMA

My immediate neighborhood has 3000 homes and not a single home fire in the last 12 years.
The only fires I have seen are brush fires, far away on the hills.

I also survived 94 earthquake. Because my computer was on it actually caught on fire internally. HD and motherboard were gone. Thanks to monitor cable and tower being on the floor nothing went flying unlike in my kitchen where refrigerator and trash compactor met in the middle. Top of the Townhome was moving several feet in the aftershock so I can only imagine how much it moved during the actual quake.

Yet after all settled down and I re-build the computer all my data was just fine thanks to tape backup.

 

[SushiEater]

malch wrote:

SushiEater wrote:

Don't image anything. Just straight copy. This way you can just look at your backup the normal way.

It depends on the situation. Some things are best imaged. Others (like photos) are best copied.

Name one thing that is best imaged besides software which on SSD anyway and being cloned after software installation.

Since there is absolutely no data but software and OS on SSD if something happens I might only lose Windows update.

This is not a safe assumption for most folks. Windows tucks away a lot of user data on the C drive and in some cases you really have to jump through hoops to prevent it. In other cases, it really can't be relocated elsewhere.

Hence the SSD clone. But my email, profiles, databases etc.. are all on RAID 0 drive which is not only have at least a double backup but email and profiles have triple backup by using MozBackup and storing on the old CF card. This way it can be restored within minutes without any kind of futzing around.

All bases are covered. Been doing it for a very long time.

Besides, it's not just Windows updates. How about all of your application updates? In my case, it take 3-4 days of full time work to install, configure, and update all of my many applications. Better to clone or image.

I don't know, you either don't understand or don't want to understand.

If I clone SSD every time I install new software (which is not happening very often now since most of it installed already) how do I lose software updates? At worst I will download update again which does not happen very often anyway. I don't make several hundreds software updates every week and if for some stupid reason I did I would clone SSD right after.

 

[SushiEater]

Tom_N wrote:

SushiEater wrote:

Tom_N wrote:

A RAID 0 setup with two drives would have only an 81% chance (0.9 squared = 0.81) of surviving intact for that time. One that striped over four drives would have only a 66% chance (0.9 to the fourth power = 0.6561) of surviving.

Lets say you have 4 drives not in the RAID. All 4 drives contain data. One has crashed and you lost that data. So you still lost some data maybe not all of it but 25% of it.

This is the point. With independent drives, if one fails, you only lose the files on it. With RAID 0, if any drive fails, you lose everything.

No, the point is if you lose data you lost data. I don't want to lose data, any data so I backup religiously.

And since I backup religiously it does not matter where data is, on the single drive, multiple drives or RAID 0.

The only point you could possibly make for me to stop using RAID 0 if the array was constantly breaking and I had to constantly rebuild it.

So now it would depends how often it would happen. Since it only happened once and it was a backup RAID 0 in the enclosure and it was fixed (I am still using it) there is no chance I would stop using RAID 0.

If the problem was reoccurring I would think about it but it would still depend on how often.

But you still lost it because you did not have a backup. But if you do backups anyway what difference does it make where the data is? None.

You can lose backups, too.

Anything could happen but losing 2 backups plus original plus Bluerays all at the same time is impossible in any scenario.

Your single HD will have more chance of crashing before I lose backups.

If you lose the original and all your backups (for a particular file), that file is gone. For any given backup strategy ("risk of losing all backups"), the risk of losing your files is higher if you use RAID 0 for the original, than if you don't.

Name one thing how it would happen beside nuclear attack. I don't think I would care about backup at that point.

Maybe the risk does not bother you, or you have extra backups to compensate. Either way, that does not negate the math.

Your math is silly.