Value System Build

SushiEater · Aug 13, 2013

Member said:
malch wrote:

Member said:

SushiEater wrote:

Nope. It depends what other work you do in the meantime. Just because most of the CS6 files are cached in RAM doesn't mean they will stay there indefinitely. They won't. They will get discarded over time in favor of other more recently used files. With suitable tools you can manually flush the cache of memory mapped files too.

Yep, if I use the same software every day, several times per day, it never gets discarded.

Click to expand...

It depends what else you do on the system. If you actually use it to do real work, the cached files can soon be discarded.

I loaded CS6 and quit. Most of CS6 was cached. A one line program and a few seconds and CS6 was gone (confirmed by SysInternals RAMMap). CS6 only remains in cache until Windows decides it has a better use for that RAM which won't be long if you're using other memory hungry apps.

I don't know how your system works because I am not there but my CS6 stays put in the memory because I use it constantly. I probably load it and reload it on average 10 times per day. I don't have to but it is just a habit of mine and has nothing to do with my system because many programs cluttering my screen and quick launch bar.

I also use Dreamweaver, In-Design (very memory hungry), many instances of Cyberfox etc... so my 32GB memory loaded up to 5GB.

Member said:
Memory mapped files are great. But it's just a cache, using the fastest and most expensive storage technology (RAM).

But if you already have as part of the system for other uses cost justifies the means.

Member said:
The slower your physical file system storage the better it looks; so it does more for HDD data than it does for SSD data. But it says nothing about the performance of your disks (SSD, HDD, or RAID).

Every time we have argument you are trying to skew things.

These are totally separate issues.

To start with SSD has the lowest WOW factor I have ever encountered after upgrade because, regardless, I keep my system asleep then not in use. So basically SSD is used maybe once per week in my system on Wednesday then I do update from Microsoft and it requires reboot.If no reboot requires I could keep my system for months without rebooting.

Plus, not all programs benefit from loading off SSD. Only large programs like Photoshop and plug-ins, In-Design because of all the fonts and plug-ins, Act! use SSD efficiently. Microsoft Word does not. Many small programs load just as fast from HD (especially from RAID) if they are not broken in to small files like DLLs.

And since SSD seeing such a small usage it is the most expensive storage technology.

I already explained to you few weeks ago that I tried to use SSD for redoing all my panos and I lost 1 year of SSD usage. So now I am still doing the panos but off the RAID. Just as fast with less destruction.

malch · Aug 13, 2013

Member said:
SushiEater wrote:

Member said:

I loaded CS6 and quit. Most of CS6 was cached. A one line program and a few seconds and CS6 was gone (confirmed by SysInternals RAMMap). CS6 only remains in cache until Windows decides it has a better use for that RAM which won't be long if you're using other memory hungry apps.

Click to expand...

I don't know how your system works because I am not there but my CS6 stays put in the memory because I use it constantly. I probably load it and reload it on average 10 times per day. I don't have to but it is just a habit of mine and has nothing to do with my system because many programs cluttering my screen and quick launch bar.

I also use Dreamweaver, In-Design (very memory hungry), many instances of Cyberfox etc... so my 32GB memory loaded up to 5GB.

Member said:

Memory mapped files are great. But it's just a cache, using the fastest and most expensive storage technology (RAM).

Click to expand...

But if you already have as part of the system for other uses cost justifies the means.

But if you actually use the RAM for the other things, an inactive copy of CS6 will get swapped out. This isn't complicated.

Now, if you're happy with your 32GB of RAM and the very good Windows caching that's fine too. But it just doesn't tell us anything about SSD or RAID or disk performance. It's tell us about the cache performance.

And if CS6 is cached in memory, RAID 0 certainly ain't gonna make it load any faster, is it?

Member said:
Plus, not all programs benefit from loading off SSD. Only large programs like Photoshop and plug-ins, In-Design because of all the fonts and plug-ins, Act! use SSD efficiently. Microsoft Word does not. Many small programs load just as fast from HD (especially from RAID) if they are not broken in to small files like DLLs.
And since SSD seeing such a small usage it is the most expensive storage technology.

No, your RAM cache is actually quite a bit more expensive per GB. And the CPU cache is even faster and more costly.

HDD's are one of the slower options with optical and tape behind that. RAID is basically a tweak on the HDD.

SSD's provide a relatively new and, in my view, indispensable option between RAM and disk.

Having the SSD option is nothing but a Good Thing. The trick is figuring out how and when to use it well in your specific situation. Frankly, that's not very hard. The biggest problem is with all of the misinformation out there posted by folks who think they read a story on the Internet about someone who said that a few writes will destroy the drive.

SushiEater · Aug 13, 2013

kelpdiver said:
kelpdiver wrote:

kelpdiver said:

SushiEater wrote:

kelpdiver said:

kelpdiver wrote:

Once a feature is created, it tends to last.

Click to expand...

Seriously????? Why bother to keep it if it is not popular?

Click to expand...

Seriously?

Given a choice of a motherboard with 10 features commonly used, or one with those 10 features and 30 uncommon ones, guess which one sells better? People like options,

People like option they can use and discard option they can't. At least those who buy motherboards.

kelpdiver said:
or sometimes they just like shiny knobs (hint). How long did Canon keep that useless print button on their cameras? Is anyone out there really plugging their camera directly into a printer to print photos?

Actually YES. Go to the shows like CES/PMA and you will see it a lot. As far as the button Canon replaced it with the very "useful" RATE button.

kelpdiver said:
That button could have been so much more useful doing something (anything!) else.

They do allow sometimes for buttons to be remapped.

kelpdiver said:
It costs virtually nothing to keep around the old stuff. Simple regression tests and you're good to go.

Actually it costs something because they have to keep up with all the new drives.

kelpdiver said:
kelpdiver said:

I was not talking about commercial environment obviously but RAID 0 originated in that environment in commercial networks and it trickled down to home computers.

Click to expand...

Your data should be treated just as well as a commercial environment would, at least in terms of reliability. The home user can (likely) afford a longer outage window, but losing your last week's worth of edits (because of your backup schedule) should be unacceptable.

But better than loosing the photos all together and that would be disaster. I at least could re-edit.

kelpdiver said:
kelpdiver said:

kelpdiver said:

no, you pick 2. R0 gives up reliability.

Click to expand...

Only according to you. One one of my machines have 3 RAID 0, one internal and 2 external.

On the other 2 computers one internal each. No problems at all.

Click to expand...

don't confuse lack of incidents as being reliable. Running a 10% chance of losing the array isn't reliable to me, You may deem it reliable enough, but there's no question it's less reliable than the single disk or the mirror.

If it is good for insurance companies it is good enough for me. How do you determine 10%? In my case it is close to ZERO. If your cameras was breaking constantly would you use it? Nope. If it doesn't break at all would you use it?

kelpdiver said:
kelpdiver said:

Actually a lot of experience and I back up my data religiously.

Click to expand...

daily? hourly? R0 means the potential loss of everything since the last backup.

Don't be ridiculous. But I have never lost anything. But I also use Hyperdrive as soon as I can in the field and I don't erase from it until I do backup. And I have 4 64gb cards 2 32GB and 4 16gb and try to keep photos on the cards until every is backup. It is called redundancy.

BTW, by your logic one should not buy a large HD but a whole bunch small ones and keep multiple copies of everything and not to keep all the eggs in one basket. Is that correct? Paranoia runs a long way.

kelpdiver said:
kelpdiver said:

More like 100% more. I read and write terabytes of data very often because I take over a 100K pictures, at least, every year all in RAW.

Click to expand...

you only get that 100% improvement when doing simple copies. And the 840 Pro is faster at doing that as well. You need to go to a 4 stripe mirror just to catch up to one of them.

Nope. I have already tried that and lost one year of SSD usage. The complete explanation is in another post. You can't replace the storage capacity with SSD, not yet anyway. But you can get close to SSD performance by using RAID. At least close enough without pain.

kelpdiver said:
malch already demonstrated that you are cpu bound on the raw processing. That is a meaningful benchmark, not CS6 load times.

Malch did not demonstrate anything. I knew that before he was born. Why do you think I upgrade my computer so often? I have been shooting RAW since Nikon D1 came out. And CS6 does load faster from RAID than from a single HD just not as fast as from SSD. But it loads even faster from memory.

RAID 0 might not have uses for you but it has huge uses for me and nothing you can say can convince me otherwise as I said before.

SushiEater · Aug 13, 2013

Member said:
malch wrote:

Member said:

SushiEater wrote:

Member said:

I loaded CS6 and quit. Most of CS6 was cached. A one line program and a few seconds and CS6 was gone (confirmed by SysInternals RAMMap). CS6 only remains in cache until Windows decides it has a better use for that RAM which won't be long if you're using other memory hungry apps.

Click to expand...

I don't know how your system works because I am not there but my CS6 stays put in the memory because I use it constantly. I probably load it and reload it on average 10 times per day. I don't have to but it is just a habit of mine and has nothing to do with my system because many programs cluttering my screen and quick launch bar.

I also use Dreamweaver, In-Design (very memory hungry), many instances of Cyberfox etc... so my 32GB memory loaded up to 5GB.

Member said:

Memory mapped files are great. But it's just a cache, using the fastest and most expensive storage technology (RAM).

Click to expand...

But if you already have as part of the system for other uses cost justifies the means.

Click to expand...

But if you actually use the RAM for the other things, an inactive copy of CS6 will get swapped out. This isn't complicated.

Now, if you're happy with your 32GB of RAM and the very good Windows caching that's fine too. But it just doesn't tell us anything about SSD or RAID or disk performance. It's tell us about the cache performance.

And if CS6 is cached in memory, RAID 0 certainly ain't gonna make it load any faster, is it?

CS6 loads faster from RAID 0 than from a single HD. Not as fast as from SSD but still faster. But RAID 0 has more uses than just loading software. Try to load or save 2GB or more pano on a single HD or RAID and you will find out.

Member said:
Member said:

Plus, not all programs benefit from loading off SSD. Only large programs like Photoshop and plug-ins, In-Design because of all the fonts and plug-ins, Act! use SSD efficiently. Microsoft Word does not. Many small programs load just as fast from HD (especially from RAID) if they are not broken in to small files like DLLs.

And since SSD seeing such a small usage it is the most expensive storage technology.

Click to expand...

No, your RAM cache is actually quite a bit more expensive per GB. And the CPU cache is even faster and more costly.

The cost is determined by usage and not by the dollar amount.

Again, you are not the power user so you think no one needs 32GB of memory or one of the faster CPUs.

Member said:
HDD's are one of the slower options with optical and tape behind that. RAID is basically a tweak on the HDD.

So? Why do you think someone came up with that tweak?

Member said:
SSD's provide a relatively new and, in my view, indispensable option between RAM and disk.

On paper only. Like I said it was the lowest WOW factor I have ever encountered.

Member said:
Having the SSD option is nothing but a Good Thing. The trick is figuring out how and when to use it well in your specific situation. Frankly, that's not very hard. The biggest problem is with all of the misinformation out there posted by folks who think they read a story on the Internet about someone who said that a few writes will destroy the drive.

Few will not but a lot of writes will. Been there, tried that.

malch · Aug 13, 2013

Member said:
SushiEater wrote:

Member said:

And if CS6 is cached in memory, RAID 0 certainly ain't gonna make it load any faster, is it?

Click to expand...

CS6 loads faster from RAID 0 than from a single HD. Not as fast as from SSD but still faster. But RAID 0 has more uses than just loading software. Try to load or save 2GB or more pano on a single HD or RAID and you will find out.

Sure. SSD is faster than RAID 0 which is faster than a single HDD.

That's all a bit moot if the program (or 2GB data file which isn't really any different) is already in the Windows file system cache. Cache will be faster still although there's probably going to be some delay assuming the file incorporates compression. The cache data will still have to be decompressed and that will burn quite a few cycles. That decompression is not typically multi-threaded so you'll see a single core saturated for a few seconds.

Of course, if CS6 (or whatever software) has cached a decompressed bitmap, that may be the fastest of all.

Member said:
Member said:

No, your RAM cache is actually quite a bit more expensive per GB. And the CPU cache is even faster and more costly.

Click to expand...

The cost is determined by usage and not by the dollar amount.

Not really. Cost is cost. Value depends on actual benefit/usage delivered.

Member said:
Again, you are not the power user so you think no one needs 32GB of memory or one of the faster CPUs.

You don't really know squat about what kind of user I am.

But I stand by the position that folks don't need supercomputers or more than 32GB of RAM to run MS Office, a web browser, and Photoshop Elements which was the topic at hand.

Member said:
Member said:

HDD's are one of the slower options with optical and tape behind that. RAID is basically a tweak on the HDD.

Click to expand...

So? Why do you think someone came up with that tweak?

I'll give you a hint -- it wasn't for photo workstations. It wasn't for performance either. RAID stands for Redundant Arrays of Inexpensive Disks. Yet, you'll note that RAID 0 doesn't incorporate any redundancy!

RAID was invented for redundancy. RAID 0 fell out of that work almost by accident.

Member said:
Member said:

SSD's provide a relatively new and, in my view, indispensable option between RAM and disk.

Click to expand...

On paper only. Like I said it was the lowest WOW factor I have ever encountered.

No, not only on paper. My first SSD delivered one of the biggest WOW factors I've seen in 40 years of computing.

Member said:
Member said:

Having the SSD option is nothing but a Good Thing. The trick is figuring out how and when to use it well in your specific situation. Frankly, that's not very hard. The biggest problem is with all of the misinformation out there posted by folks who think they read a story on the Internet about someone who said that a few writes will destroy the drive.

Click to expand...

Few will not but a lot of writes will. Been there, tried that.

In a test of two 256GB Samsung 840 Pro drives, they both survived around 740TB of writes before the first uncorrectable error. According to generally accepted guidelines a consumer workstation can be expected to generate 5-10GB per day. Of course, any device can be subject to infant mortality. But it's now possible to make reasonable estimates of expected SSD lifespans under given operating conditions.

kelpdiver · Aug 13, 2013

Member said:
SushiEater wrote:

If it is good for insurance companies it is good enough for me. How do you determine 10%? In my case it is close to ZERO. If your cameras was breaking constantly would you use it? Nope. If it doesn't break at all would you use it?

Disks have failure rates that start high in the first 90s days, then drop for a few years, then begin to climb upwards again. I made simple rounding to 5% per year, but if you want to use 3 or 4%, fine. Double 5% and you get 10%. Your tiny sample size doesn't matter. Large scale environments remove random chance from the equation....and I've replaced enough dead drives in mirrors to know better. It's a trivial process to recover from a failed disk in a mirror. Remove, insert, wait. No downtime, no restores to do, no lost data.

Member said:
BTW, by your logic one should not buy a large HD but a whole bunch small ones and keep multiple copies of everything and not to keep all the eggs in one basket. Is that correct? Paranoia runs a long way.

Mirroring is hardly paranoia. My primary storage is 0+1 - 4 x 2 TB drives.

The flaw is simply having a lot of copies and relying on them if your stripe fails is that the potential for bit rot. If you're insisting that SSD write exhaustion is a problem for you, then so is a BER of 1 in 10^14. Are you doing cksumming of any sort to prevent this?

Member said:
Nope. I have already tried that and lost one year of SSD usage. The complete explanation is in another post. You can't replace the storage capacity with SSD, not yet anyway. But you can get close to SSD performance by using RAID. At least close enough without pain.

Going from 300 IOPS to ... 300 IOPS (R0 contributes nothing) while SSDs do tens of thousands of IOPS is not "close enough." Again, you need a 3 disk stripe just to match that SSD on sequential performance, and you'll never do better than 1 MB/s for random 4k reads.

Member said:
Malch did not demonstrate anything. I knew that before he was born.

Really? The first white paper on RAID as a concept came out in 1987. I believe he predates that, as well as any knowledge you have of CS6.

Member said:
RAID 0 might not have uses for you but it has huge uses for me and nothing you can say can convince me otherwise as I said before.

Point isn't about convincing you. You drank the koolade, great.

SushiEater · Aug 13, 2013

Member said:
malch wrote:

Member said:

SushiEater wrote:

Member said:

And if CS6 is cached in memory, RAID 0 certainly ain't gonna make it load any faster, is it?

Click to expand...

CS6 loads faster from RAID 0 than from a single HD. Not as fast as from SSD but still faster. But RAID 0 has more uses than just loading software. Try to load or save 2GB or more pano on a single HD or RAID and you will find out.

Click to expand...

Sure. SSD is faster than RAID 0 which is faster than a single HDD.

That's all a bit moot if the program (or 2GB data file which isn't really any different) is already in the Windows file system cache. Cache will be faster still although there's probably going to be some delay assuming the file incorporates compression. The cache data will still have to be decompressed and that will burn quite a few cycles. That decompression is not typically multi-threaded so you'll see a single core saturated for a few seconds.

How many seconds does it take on your system?

Member said:
Of course, if CS6 (or whatever software) has cached a decompressed bitmap, that may be the fastest of all.

Member said:

Member said:

No, your RAM cache is actually quite a bit more expensive per GB. And the CPU cache is even faster and more costly.

Click to expand...

The cost is determined by usage and not by the dollar amount.

Click to expand...

Not really. Cost is cost. Value depends on actual benefit/usage delivered.

You know what I meant so there is no need to parrot.

Member said:
Member said:

Again, you are not the power user so you think no one needs 32GB of memory or one of the faster CPUs.

Click to expand...

You don't really know squat about what kind of user I am.

You have quite slow CPU by today's standard, old motherboard...etc... So you are either not a power user or you are not value your time.

Member said:
But I stand by the position that folks don't need supercomputers or more than 32GB of RAM to run MS Office, a web browser, and Photoshop Elements which was the topic at hand.

Member said:

Member said:

HDD's are one of the slower options with optical and tape behind that. RAID is basically a tweak on the HDD.

Click to expand...

So? Why do you think someone came up with that tweak?

Click to expand...

I'll give you a hint -- it wasn't for photo workstations. It wasn't for performance either. RAID stands for Redundant Arrays of Inexpensive Disks. Yet, you'll note that RAID 0 doesn't incorporate any redundancy!

Actually it was Independent and some idiot said Inexpensive so it stuck.

When Raid was invented where was no photo workstations at that time. So your analogy is pointless.

Member said:
RAID was invented for redundancy. RAID 0 fell out of that work almost by accident.

Regardless, I have been experimenting with all kinds of ways to speed up read/write for very large files and the RAID is the only way to go.

Member said:
Member said:

Member said:

SSD's provide a relatively new and, in my view, indispensable option between RAM and disk.

Click to expand...

On paper only. Like I said it was the lowest WOW factor I have ever encountered.

Click to expand...

No, not only on paper. My first SSD delivered one of the biggest WOW factors I've seen in 40 years of computing.

That is because you had system so slow that it felt like it. I never had slow system since 1985. My system is always close to the cutting edge of technology.

Member said:
Member said:

Member said:

Having the SSD option is nothing but a Good Thing. The trick is figuring out how and when to use it well in your specific situation. Frankly, that's not very hard. The biggest problem is with all of the misinformation out there posted by folks who think they read a story on the Internet about someone who said that a few writes will destroy the drive.

Click to expand...

Few will not but a lot of writes will. Been there, tried that.

Click to expand...

In a test of two 256GB Samsung 840 Pro drives, they both survived around 740TB of writes before the first uncorrectable error. According to generally accepted guidelines a consumer workstation can be expected to generate 5-10GB per day. Of course, any device can be subject to infant mortality. But it's now possible to make reasonable estimates of expected SSD lifespans under given operating conditions.

And yet I lost a year worth of SSD life after several weeks of writing and erasing pano files.

SushiEater · Aug 13, 2013

kelpdiver said:
kelpdiver wrote:

kelpdiver said:

SushiEater wrote:

If it is good for insurance companies it is good enough for me. How do you determine 10%? In my case it is close to ZERO. If your cameras was breaking constantly would you use it? Nope. If it doesn't break at all would you use it?

Click to expand...

Disks have failure rates that start high in the first 90s days, then drop for a few years, then begin to climb upwards again. I made simple rounding to 5% per year, but if you want to use 3 or 4%, fine. Double 5% and you get 10%. Your tiny sample size doesn't matter. Large scale environments remove random chance from the equation....and I've replaced enough dead drives in mirrors to know better. It's a trivial process to recover from a failed disk in a mirror. Remove, insert, wait. No downtime, no restores to do, no lost data.

You know people die in the car accidents everyday yet everybody is driving. If it was 10% chance of dying no one would be driving. That is why we take precautions. And by backing up at least once a week plus very frequent incremental backups I think I am pretty safe.

kelpdiver said:
kelpdiver said:

BTW, by your logic one should not buy a large HD but a whole bunch small ones and keep multiple copies of everything and not to keep all the eggs in one basket. Is that correct? Paranoia runs a long way.

Click to expand...

Mirroring is hardly paranoia. My primary storage is 0+1 - 4 x 2 TB drives.

Good for you but I need space and speed without blocking the front of my tower from air circulation.
So no mirror for me.

kelpdiver said:
The flaw is simply having a lot of copies and relying on them if your stripe fails is that the potential for bit rot. If you're insisting that SSD write exhaustion is a problem for you, then so is a BER of 1 in 10^14. Are you doing cksumming of any sort to prevent this?

If my stripe fails, but it hasn't. And if it does I have a backup. And more likely by the time drives get old I will replace them anyway with the faster drives.

kelpdiver said:
kelpdiver said:

Nope. I have already tried that and lost one year of SSD usage. The complete explanation is in another post. You can't replace the storage capacity with SSD, not yet anyway. But you can get close to SSD performance by using RAID. At least close enough without pain.

Click to expand...

Going from 300 IOPS to ... 300 IOPS (R0 contributes nothing) while SSDs do tens of thousands of IOPS is not "close enough." Again, you need a 3 disk stripe just to match that SSD on sequential performance, and you'll never do better than 1 MB/s for random 4k reads.

No I don't and I don't use worthless synthetic benchmarks to prove it. You guys just stuck like a broken record on 4K reads then in real life 4K reads is just a small insignificant measure not even related to photo files. So get your record changed because it is worn.

kelpdiver said:
kelpdiver said:

Malch did not demonstrate anything. I knew that before he was born.

Click to expand...

Really? The first white paper on RAID as a concept came out in 1987.

But when was it actually available to home users?

kelpdiver said:
I believe he predates that, as well as any knowledge you have of CS6.

That is your opinion and you are entitled to it. My opinion is different and considering that I do real life tests without listening to a bunch of misinformation posted here, my opinion is more important to me than yours.

kelpdiver said:
kelpdiver said:

RAID 0 might not have uses for you but it has huge uses for me and nothing you can say can convince me otherwise as I said before.

Click to expand...

Point isn't about convincing you. You drank the koolade, great.

To much sugar!!!! I drink water.

SushiEater · Aug 13, 2013

Member said:
And by the way, that didn't include any really heavy lifting like noise reduction, lens corrections and the like which will increase the CPU load by dramatic increments.

I can only say one thing. Throw your system under the bus if you have dramatic increment in CPU activity while doing everything you said above. I got 1% while doing all of the above and more. Nothing dramatic about it.

CPU load went from 35% to 36%. All 6 cores were used. In both cases it took the same amount of time of 2 seconds to convert D800 compressed (no loss) RAW at 12800 ISO file and display it.

malch · Aug 14, 2013

Member said:
SushiEater wrote:

Member said:

That's all a bit moot if the program (or 2GB data file which isn't really any different) is already in the Windows file system cache. Cache will be faster still although there's probably going to be some delay assuming the file incorporates compression. The cache data will still have to be decompressed and that will burn quite a few cycles. That decompression is not typically multi-threaded so you'll see a single core saturated for a few seconds.

Click to expand...

How many seconds does it take on your system?

That will depend on the type of file, what compression, if any, is employed, and more.

Member said:
Regardless, I have been experimenting with all kinds of ways to speed up read/write for very large files and the RAID is the only way to go.

No, it's one way. In some settings, it's the best way, and you only have to look at how many large corporations use RAID to see some examples. Hint: they're not writing out little panos.

Member said:
That is because you had system so slow that it felt like it. I never had slow system since 1985. My system is always close to the cutting edge of technology.

Oh, I'm quite sure your thing is bigger than mine. It certainly doesn't bother me and I doubt anyone else here gives a poop either.

Member said:
And yet I lost a year worth of SSD life after several weeks of writing and erasing pano files.

Make and model and terabytes written?

SushiEater · Aug 14, 2013

Member said:
malch wrote:

Member said:

SushiEater wrote:

Member said:

That's all a bit moot if the program (or 2GB data file which isn't really any different) is already in the Windows file system cache. Cache will be faster still although there's probably going to be some delay assuming the file incorporates compression. The cache data will still have to be decompressed and that will burn quite a few cycles. That decompression is not typically multi-threaded so you'll see a single core saturated for a few seconds.

Click to expand...

How many seconds does it take on your system?

Click to expand...

That will depend on the type of file, what compression, if any, is employed, and more.

That depends on if you are going to answer or not. So far I asked twice.

Member said:
Member said:

Regardless, I have been experimenting with all kinds of ways to speed up read/write for very large files and the RAID is the only way to go.

Click to expand...

No, it's one way. In some settings, it's the best way, and you only have to look at how many large corporations use RAID to see some examples. Hint: they're not writing out little panos.

Again, big corporation. I think I made it clear several times I don't care about big corporations. You don't model your setup based on big corporations.

Member said:
Member said:

That is because you had system so slow that it felt like it. I never had slow system since 1985. My system is always close to the cutting edge of technology.

Click to expand...

Oh, I'm quite sure your thing is bigger than mine. It certainly doesn't bother me and I doubt anyone else here gives a poop either.

It is not bigger. Just more powerful to do the work I need to be done to meet the deadline.

How can you possibly give advise here to people based on what you don't have knowledge of.

Your motherboard is not even capable of RAID. Something like MSI B75MA-E33? One memory slot.

No wonder your system saturates everything because you don't have system capable of doing any meaningful work. Your system is designed to browse Internet and write emails. You are not a power user by any means.

You can't even test anything yourself so all your advices, based on what you read on the web, are null and void!!!! And that is a cold truth!!!!

Member said:
Member said:

And yet I lost a year worth of SSD life after several weeks of writing and erasing pano files.

Click to expand...

Make and model and terabytes written?

I am not going to repeat myself. We already had this discussion. Look it up.

kelpdiver · Aug 14, 2013

Member said:
SushiEater wrote:

You know people die in the car accidents everyday yet everybody is driving. If it was 10% chance of dying no one would be driving. That is why we take precautions. And by backing up at least once a week plus very frequent incremental backups I think I am pretty safe.

People do a lot of things, unaware of the risk. Roughly 1 in 6000 Americans dies every year in a car. Pretty high rate, really, when skydiving is only a bit worse at 1 in 1000. Not drinking and driving cuts the risk considerable - roughly half. So consider Raid 0 to be like driving with a .10 BAC. After people get caught by the odds, the tendency is to now overestimate the risk.

Member said:
Member said:

The flaw is simply having a lot of copies and relying on them if your stripe fails is that the potential for bit rot. If you're insisting that SSD write exhaustion is a problem for you, then so is a BER of 1 in 10^14. Are you doing cksumming of any sort to prevent this?

Click to expand...

If my stripe fails, but it hasn't. And if it does I have a backup. And more likely by the time drives get old I will replace them anyway with the faster drives.

I don't think you understand the question here. That BER means one bad block every 14 Terabytes written. And you're doing a lot of writing, you say. So your backup may easily have corruption...or maybe even your original, which gets copied to your backup.

Member said:
No I don't and I don't use worthless synthetic benchmarks to prove it. You guys just stuck like a broken record on 4K reads then in real life 4K reads is just a small insignificant measure not even related to photo files. So get your record changed because it is worn.

you seem happy to use sequential write - the most synthetic benchmark available.

For a multicore cpu running a multitasking OS - which we all have - IOPS matters a lot more than gigabyte writes.

Member said:
Member said:

Member said:

Malch did not demonstrate anything. I knew that before he was born.

Click to expand...

Really? The first white paper on RAID as a concept came out in 1987.

Click to expand...

But when was it actually available to home users?

Member said:

I believe he predates that, as well as any knowledge you have of CS6.

Click to expand...

That is your opinion and you are entitled to it. My opinion is different and considering that I do real life tests without listening to a bunch of misinformation posted here, my opinion is more important to me than yours.

You said you knew this stuff before he was even born. That's not an opinion, that's an absolute falsehood. I know he's older than CS6, and I'm pretty sure he's older than 1987. Which is stupidly obvious, but this sort of hyperbole is common in your writing.

SushiEater · Aug 14, 2013

kelpdiver said:
kelpdiver wrote:

kelpdiver said:

SushiEater wrote:

You know people die in the car accidents everyday yet everybody is driving. If it was 10% chance of dying no one would be driving. That is why we take precautions. And by backing up at least once a week plus very frequent incremental backups I think I am pretty safe.

Click to expand...

People do a lot of things, unaware of the risk. Roughly 1 in 6000 Americans dies every year in a car. Pretty high rate, really, when skydiving is only a bit worse at 1 in 1000. Not drinking and driving cuts the risk considerable - roughly half. So consider Raid 0 to be like driving with a .10 BAC. After people get caught by the odds, the tendency is to now overestimate the risk.

I don't think you are reading the news correctly. Usually if drunk driving involves the drunk driver survives but the innocent sober people die.

kelpdiver said:
kelpdiver said:

kelpdiver said:

The flaw is simply having a lot of copies and relying on them if your stripe fails is that the potential for bit rot. If you're insisting that SSD write exhaustion is a problem for you, then so is a BER of 1 in 10^14. Are you doing cksumming of any sort to prevent this?

Click to expand...

If my stripe fails, but it hasn't. And if it does I have a backup. And more likely by the time drives get old I will replace them anyway with the faster drives.

Click to expand...

I don't think you understand the question here. That BER means one bad block every 14 Terabytes written. And you're doing a lot of writing, you say. So your backup may easily have corruption...or maybe even your original, which gets copied to your backup.

And the answer is it hasn't happened yet.

kelpdiver said:
kelpdiver said:

No I don't and I don't use worthless synthetic benchmarks to prove it. You guys just stuck like a broken record on 4K reads then in real life 4K reads is just a small insignificant measure not even related to photo files. So get your record changed because it is worn.

Click to expand...

you seem happy to use sequential write - the most synthetic benchmark available.

For a multicore cpu running a multitasking OS - which we all have - IOPS matters a lot more than gigabyte writes.

kelpdiver said:

kelpdiver said:

kelpdiver said:

Malch did not demonstrate anything. I knew that before he was born.

Click to expand...

Really? The first white paper on RAID as a concept came out in 1987.

Click to expand...

But when was it actually available to home users?

kelpdiver said:

I believe he predates that, as well as any knowledge you have of CS6.

Click to expand...

That is your opinion and you are entitled to it. My opinion is different and considering that I do real life tests without listening to a bunch of misinformation posted here, my opinion is more important to me than yours.

Click to expand...

You said you knew this stuff before he was even born. That's not an opinion, that's an absolute falsehood. I know he's older than CS6, and I'm pretty sure he's older than 1987. Which is stupidly obvious, but this sort of hyperbole is common in your writing.

Stop skewing this post. You know exactly what I meant. The truth of the matter is then two of you losing the argument you start throwing useless statistic numbers of useless synthetic tests. Real life computing is based on experience with the items you have in hand. Otherwise advice you give based on the web reading which might or might not be true. Everything you and Malch post is just that kind of the opinion. According to Malch admitting what kind of computer he has he can't have more than 8gb of memory or RAID because he has only one SATA3 connector. And the only motherboards that have only one SATA 3 connector are some cheap mini-ITX boards.

I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

If everyone listen to people like you no one would buy electronics ever. All electronics fail eventually. I had Canon digital camera once that was used once and put away in the box. I bought it for my mother in law. Several years later I opened the box and camera did not work at all.

People complain here all the time about equipment failures but that does not prevent them from using it.

Does the RAID justifies to be used? Absolutely!!!!! Otherwise I would not be using it.

kelpdiver · Aug 14, 2013

Member said:
Stop skewing this post. You know exactly what I meant. The truth of the matter is then two of you losing the argument you start throwing useless statistic numbers of useless synthetic tests. Real life computing is based on experience with the items you have in hand. Otherwise advice you give based on the web reading which might or might not be true.

You need to make up your mind. At times you claim we have no idea what we're talking about. Then when we point out that we deal with thousands of servers for millions of people and trillions of dollars, you claim that what is done in the corporate high tech world doesn't matter for home users.

The fact is, we know our stuff and yes, it does matter at home, unless you consider your data unimportant. The realities for servers that cost a quarter million dollars don't change for the $700 system. It's been a long time since I lost data, and not just my personal photos. Many drives have failed in this period of time.

Member said:
I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

So in your tiny experience set, it works all the time, except for the time it didn't. What a relief! And when those bad block writes happen, you probably won't even know.

Member said:
Does the RAID justifies to be used? Absolutely!!!!! Otherwise I would not be using it.

As Malch noted already, you're using a nonRedundant Array of Inexpensive Disks. Or non raid as I would call it. So you can load up CS6 slower than I can with a more reliable single SSD.

malch · Aug 14, 2013

Member said:
SushiEater wrote:
I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

I don't think critical thinking and analysis is your strong suit.

You're saying that RAID 0 is absolutely safe. Except when it breaks. You ignore the simple math which tells us that two drives in RAID 0 are at least twice as likely to fail as a single non-RAID drive.

And you ignore the fact that the vast majority of credible and experienced users here consider a single drive to be far from "absolutely safe". Most of us assume otherwise and have backup and recovery plans formulated on that premise.

A single disk drive is not "absolutely safe". And two of them in RAID 0 are way less than "absolutely safe". In the wrong hands they are positively dangerous. And that, my friend, is the reason why I tend to discourage folks from embarking upon such adventures in RAID without very careful thought and preparation.

You think peak data transfer rates are the Holy Grail. They are not in the vast majority of workstation applications. You promote RAID 0 which mitigates (to some extent) the fundamental mechanical limitations of hard drives. i.e. seek times and rotational latency. And you dismiss SSD's which almost eliminate those critical factors.

You are **** sure of everything and wrong about most of it!

Tom_N · Aug 15, 2013

Member said:
malch wrote:

Member said:

SushiEater wrote:

I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

Click to expand...

You're saying that RAID 0 is absolutely safe. Except when it breaks. You ignore the simple math which tells us that two drives in RAID 0 are at least twice as likely to fail as a single non-RAID drive.

The math on RAID 0 reliability is not additive. It's exponential. Your data is only safe as long as ALL of the drives in the RAID avoid crashing or other catastrophic data loss events. Lose a drive, and you've probably lost a part of every file or close to every file in the filesystem. At that point, you better have backups, or it sucks to be you.

Let's say that a single hard disk has a 90% chance of surviving intact for five years (not the real number; just a back of the envelope number for purposes of illustrating the RAID math), and that odds of a disk surviving are independent of capacity. A RAID 0 setup with two drives would have only an 81% chance (0.9 squared = 0.81) of surviving intact for that time. One that striped over four drives would have only a 66% chance (0.9 to the fourth power = 0.6561) of surviving.

The figure that's easiest to compute is the chance that all of the disks survive, but once you have that, it's easy to "reverse" it to get the chance of loss: in this example, 10% with one hard drive, 19% with a two-drive RAID 0, 34% with a four-drive RAID 0.

SushiEater · Aug 15, 2013

kelpdiver said:
kelpdiver wrote:

kelpdiver said:

Stop skewing this post. You know exactly what I meant. The truth of the matter is then two of you losing the argument you start throwing useless statistic numbers of useless synthetic tests. Real life computing is based on experience with the items you have in hand. Otherwise advice you give based on the web reading which might or might not be true.

Click to expand...

You need to make up your mind. At times you claim we have no idea what we're talking about. Then when we point out that we deal with thousands of servers for millions of people and trillions of dollars, you claim that what is done in the corporate high tech world doesn't matter for home users.

How does your sentence apply to my sentence above? It doesn't!!!

You are losing the argument and try skewing the whole conversation.

kelpdiver said:
The fact is, we know our stuff and yes, it does matter at home, unless you consider your data unimportant. The realities for servers that cost a quarter million dollars don't change for the $700 system. It's been a long time since I lost data, and not just my personal photos. Many drives have failed in this period of time.

There is a huge difference between a $250000 and $700. Not the same reality at all. I have never lost anything except maybe for a few emails that were not backed up. But certainly not photos.

kelpdiver said:
kelpdiver said:

I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

Click to expand...

So in your tiny experience set, it works all the time, except for the time it didn't.

The time it didn't it was not my fault, it was fixed and restored from backup. It was a fluke. Plane and simple. Backup is the key. And if you are advocating that people should not backup or doing less backups if using a single disk you are a fool. Regardless how the data stored on your computer you should have multiple backups. I do and I will never lose any data.

kelpdiver said:
What a relief! And when those bad block writes happen, you probably won't even know.

Stop inventing something that does not exists. You are still not going to win.

kelpdiver said:
kelpdiver said:

Does the RAID justifies to be used? Absolutely!!!!! Otherwise I would not be using it.

Click to expand...

As Malch noted already, you're using a nonRedundant Array of Inexpensive Disks. Or non raid as I would call it. So you can load up CS6 slower than I can with a more reliable single SSD.

Again, skewing the conversation. I can load CS6 even faster from memory than SSD which anyone reading this can test for themselves and see how full of poop you are.

Again, stop inventing stuff just to skew conversation in your direction. Not going to work.

SushiEater · Aug 15, 2013

Member said:
malch wrote:

Member said:

SushiEater wrote:

I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

Click to expand...

I don't think critical thinking and analysis is your strong suit.

You're saying that RAID 0 is absolutely safe. Except when it breaks. You ignore the simple math which tells us that two drives in RAID 0 are at least twice as likely to fail as a single non-RAID drive.

So don't go outside because you are most likely to be run over by a car, robbed at the store, AC might fall on your head etc... How do you prevent all those things not happening to you? You use caution. How do you prevent RAID 0 failure? You use backup.

Member said:
And you ignore the fact that the vast majority of credible and experienced users here consider a single drive to be far from "absolutely safe". Most of us assume otherwise and have backup and recovery plans formulated on that premise.

Apparently not most of you. Actually most people don't do backups very frequently. So from RAID 0 point of view if someone makes frequent backups it really does not matter. You read many stories that SSD fail too without warning. It only matters if you don't have contingency plan.

Member said:
A single disk drive is not "absolutely safe". And two of them in RAID 0 are way less than "absolutely safe". In the wrong hands they are positively dangerous.

I have heard that someone actually exploded RAID 0 and shrapnel hit that person who spend 2 weeks in the hospital. :-D

Member said:
And that, my friend, is the reason why I tend to discourage folks from embarking upon such adventures in RAID without very careful thought and preparation.

It needs the same preparation regardless. It is called BACKUP.

Member said:
You think peak data transfer rates are the Holy Grail. They are not in the vast majority of workstation applications. You promote RAID 0 which mitigates (to some extent) the fundamental mechanical limitations of hard drives. i.e. seek times and rotational latency. And you dismiss SSD's which almost eliminate those critical factors.

And you are dismissing the fact that SSD performance can be simulated by the sleep mode. High transfer rate for reading and writing can't be. Reading and writing and storing large files is not yet possible on SSD because of the limited life of the SSD and capacity vs cost. You obviously did not encounter what I have so you don't know. So the second best thing to do is to use RAID 0.

Member said:
You are **** sure of everything and wrong about most of it!

I am only sure of what I have tested myself unlike you who doesn't even have the equipment to do any kinds of tests.

SushiEater · Aug 15, 2013

Tom_N said:
Tom_N wrote:

Tom_N said:

malch wrote:

Tom_N said:

SushiEater wrote:

I, on the other hand give advices on what I have hands on experience. And my experience is that RAID 0 is absolutely safe. But just to be on the safe side I do frequent backups. This strategy has worked for many years without fail (okay one fail but not because HD broke) and will continue to work.

Click to expand...

You're saying that RAID 0 is absolutely safe. Except when it breaks. You ignore the simple math which tells us that two drives in RAID 0 are at least twice as likely to fail as a single non-RAID drive.

Click to expand...

The math on RAID 0 reliability is not additive. It's exponential. Your data is only safe as long as ALL of the drives in the RAID avoid crashing or other catastrophic data loss events. Lose a drive, and you've probably lost a part of every file or close to every file in the filesystem. At that point, you better have backups, or it sucks to be you.

I do have many redundant backups.

Think about it. Even if you have a single drive you still have to have backups. So what is the difference where your data resides a single drive or two drives? None, if you have a backup.

Tom_N said:
Let's say that a single hard disk has a 90% chance of surviving intact for five years (not the real number; just a back of the envelope number for purposes of illustrating the RAID math), and that odds of a disk surviving are independent of capacity.

Actually it matters but that is another topic.

Tom_N said:
A RAID 0 setup with two drives would have only an 81% chance (0.9 squared = 0.81) of surviving intact for that time. One that striped over four drives would have only a 66% chance (0.9 to the fourth power = 0.6561) of surviving.

Lets say you have 4 drives not in the RAID. All 4 drives contain data. One has crashed and you lost that data. So you still lost some data maybe not all of it but 25% of it. But you still lost it because you did not have a backup. But if you do backups anyway what difference does it make where the data is? None.

Regardless of statistics you are never going to get in to the car accident if you don't drive. Right?

So you will never lose your data if you backup!!!

malch · Aug 15, 2013

Member said:
SushiEater wrote:

So don't go outside because you are most likely to be run over by a car, robbed at the store, AC might fall on your head etc... How do you prevent all those things not happening to you?

You can't. One can only take measures that reduce the risk to (hopefully) acceptable levels.

Member said:
Member said:

And you ignore the fact that the vast majority of credible and experienced users here consider a single drive to be far from "absolutely safe". Most of us assume otherwise and have backup and recovery plans formulated on that premise.

Click to expand...

Apparently not most of you. Actually most people don't do backups very frequently.

Correct. I guess even a stopped clock is right twice a day.

But the "credible and experienced users here" are not "most people". Many of them execute comprehensive backup procedures and a number have helped less experienced members understand some of the issues.

Member said:
You read many stories that SSD fail too without warning. It only matters if you don't have contingency plan.

HDD's fail without warning too. I had one fail yesterday!

Member said:
And that, my friend, is the reason why I tend to discourage folks from embarking upon such adventures in RAID without very careful thought and preparation.

It needs the same preparation regardless. It is called BACKUP.

It needs a great deal more than that. One needs to study and understand RAID in a lot more detail than "most people" to stand any hope of designing and implementing a decent backup strategy. And as you correctly point out, "most people" don't know how to make proper backup arrangements for a single HDD.

Member said:
And you are dismissing the fact that SSD performance can be simulated by the sleep mode.

You can't be serious. Just think about the nonsense you've just written!

Member said:
High transfer rate for reading and writing can't be. Reading and writing and storing large files is not yet possible on SSD because of the limited life of the SSD and capacity vs cost.

Rubbish, I read and write large (GB) files to SSD every single day. It's true that I move them to HDD for storage (for economy). For storage, SSD life is just fine and probably no worse than HDD although SSD technology is still rather expensive. But, as a matter of policy, I don't store critical data on disk drives that are more than 5-6 years old.

I hammered one SSD for three years. Media wear: 2%. It's now retired, not because of media wear but because at 80GB it's too small be very useful to me today.

Member said:
I am only sure of what I have tested myself unlike you who doesn't even have the equipment to do any kinds of tests.

You have no clue what I equipment I have. You took one tiny bit of info and leaped to all kinds of completely erroneous conclusions (which seems to be a recurring pattern with you).

Value System Build

SushiEater

Guest

malch

Guest

SushiEater

Guest

SushiEater

Guest

malch

Guest

kelpdiver

Veteran Member

SushiEater

Guest

SushiEater

Guest

SushiEater

Guest

malch

Guest

SushiEater

Guest

kelpdiver

Veteran Member

SushiEater

Guest

kelpdiver

Veteran Member

malch

Guest

Tom_N

Forum Pro

SushiEater

Guest

SushiEater

Guest

SushiEater

Guest

malch

Guest

Similar threads

Keyboard shortcuts