Copying 2 Terabytes from one External HDD to another External HDD

I just finished copying 1 Terabytes worth of photos from a 1TB External HDD to another. I merely plugged them both into a PC, and used Windows Explorer to Copy/Paste from one HDD to another. It took about 24 hours to complete

Now I want to copy nearly 2 Terabytes of Video on a 2 TB External drive to another empty 2 TB external drive. I don't mind it taking 2 days, as I can just use a spare PC to copy/paste. Is there a better/faster way, or am I limited by the speed of the USB-2 throughput? I don't really want to crack the case of the HDD's and install them into a desktop, as this is a one-time copy, so the 2-day run is my default choice.




TIA

USB 2 is limited in practice to about 23 MB/sec, which is about a quarter of the speed a hard drive can do.

USB 3.0 can easily cope with the full speed of a hard drive, and if your system is well configured you can get about 90-110 MB/sec from them ( the typical max of a HDD ).

It will still be at least an overnight job.

dad_of_four said:

dad_of_four wrote:

I just finished copying 1 Terabytes worth of photos from a 1TB External HDD to another. I merely plugged them both into a PC, and used Windows Explorer to Copy/Paste from one HDD to another. It took about 24 hours to complete

Now I want to copy nearly 2 Terabytes of Video on a 2 TB External drive to another empty 2 TB external drive. I don't mind it taking 2 days, as I can just use a spare PC to copy/paste. Is there a better/faster way, or am I limited by the speed of the USB-2 throughput? I don't really want to crack the case of the HDD's and install them into a desktop, as this is a one-time copy, so the 2-day run is my default choice.

TIA

Click to expand...

If you were only using a smaller percentage of the source drive, then some solutions may work better than others (as some backup and restore software only copies the used space on a drive).

But, there's no "magic bullet" to copying a given amount of data from one drive to another, as even if you use backup software that compresses the data, you're still going to need to decompress that data when restoring it to another drive.

It sounds like you're just trying to clone the contents of a virtually full 2TB Drive to another 2TB drive. In that case, I'd probably use a linux utility called ddrescue for that purpose, since you're not going to save any time using a different method anyway if most of the space on the source drive is already in use, and it would be able to work around any sector read errors that could "choke" other techniques.

See this post for details on using it:

http://forums.dpreview.com/forums/post/50338666


If your source drive had plenty of free space on it, another method may save some time, since some backup software would only copy the data on it, without copying any unused space.

But, if you're trying to copy the data from a drive that's almost full (as it sounds like you want to do), then I'd just opt for the most reliable method, which IMO would be using ddrescue for that purpose, which is going to copy *all* sectors from the source drive to a destination drive, including unused sectors.

dad_of_four said:

dad_of_four wrote:

I just finished copying 1 Terabytes worth of photos from a 1TB External HDD to another. I merely plugged them both into a PC, and used Windows Explorer to Copy/Paste from one HDD to another. It took about 24 hours to complete

Now I want to copy nearly 2 Terabytes of Video on a 2 TB External drive to another empty 2 TB external drive. I don't mind it taking 2 days, as I can just use a spare PC to copy/paste. Is there a better/faster way, or am I limited by the speed of the USB-2 throughput? I don't really want to crack the case of the HDD's and install them into a desktop, as this is a one-time copy, so the 2-day run is my default choice.

Click to expand...

USB2 is probably the rate limiting factor. USB3 would be fast but it's still going to take a long time.

However, I would definitely use something more efficient and robust than Explorer.

Robocopy (included with Win 7/8) would be ideal. It is a command line program but there are some GUI front ends available if you absolutely need one.

Deleted1929 said:

sjgcit wrote:

USB 2 is limited in practice to about 23 MB/sec, which is about a quarter of the speed a hard drive can do.

It will still be at least an overnight job.

Click to expand...

IIRC, I was averaging about 10 MB/sec throughput on my last run. But then remember that I have to read from one, and then write to the other, so this may explain 23 vs. 10

dad_of_four said:

dad_of_four wrote:

dad_of_four said:

sjgcit wrote:

USB 2 is limited in practice to about 23 MB/sec, which is about a quarter of the speed a hard drive can do.

It will still be at least an overnight job.

Click to expand...

IIRC, I was averaging about 10 MB/sec throughput on my last run. But then remember that I have to read from one, and then write to the other, so this may explain 23 vs. 10

Click to expand...

Nah.. 10MB/Second is too slow, unless you're doing something like using the same USB port for a dual slot docking station with a very slow PC.


You'll usually get twice that speed or more copying from one USB attached drive to another on a newer PC model.


Transfer speeds will vary, depending on the USB chipset being used and more. If your PC is using an internal hub with more than one port on the same USB controller, that will also slow things down.

But, as a general rule of thumb, I'd expect 20MB/Second+ transfer speeds copying from one USB Attached Drive to another on a typical PC; and with some PCs, you may see closer to 30MB/Second or even more.


In theory, you can get even faster transfer times. But, around 30MB/Second is what I see on newer PCs. 10MB/Second? Nah... Something is wrong if you're seeing transfer times that slow on a modern PC.

Now, what you have running may also impact throughput. For example, some Antivirus and Firewall software may be trying to do things like scan each file being copied, slowing down transfer times. So, disabling that kind of software temporarily may help speed up transfer times.

You could also have programs running taking up CPU resources, slowing down copy times.

That's one reason I prefer to do larger copies by booting into a Live Linux distro (either running from DVD or booting into a Linux distro running from USB flash drive).

That way, I know I don't have any programs running impacting performance, or any AV software slowing down transfers because they're scanning everything being copied.

Member said:

Jim Cockfield wrote:

Nah.. 10MB/Second is too slow, unless you're doing something like using the same USB port for a dual slot docking station with a very slow PC.

Click to expand...

I'm inclined to agree depending on the age and speed of the drives.

Also, when copying very large numbers of very small files, the data rate slows down dramatically because of all of the updating to the file system pointers etc.

As Jim mentioned, AV software can result in a big hit too.

after completing the copy, you probably should do a second run using something that does checksums on each side to make sure there were no bad reads/writes in the process.

dad_of_four said:

dad_of_four wrote:

Now I want to copy nearly 2 Terabytes of Video on a 2 TB External drive to another empty 2 TB external drive....

Click to expand...

I've copied recently 2TB via USB2.0 using a nice Win utility : TeraCopy. Took a looong time but each file checksums were verified - it's important not to be left with corrupted files.

-- M.

24 hours is not too bad. I migrated about 700G of data to my personal cloud via Wifi. I left my PC on for about a week. Ouch.

But within next year, a new standard will emerge WiGig with a "theoretical" throughput rate of 7Gbps. It is supposedly faster than USB3. Will see

nofumble said:

nofumble wrote:

24 hours is not too bad. I migrated about 700G of data to my personal cloud via Wifi. I left my PC on for about a week. Ouch.

But within next year, a new standard will emerge WiGig with a "theoretical" throughput rate of 7Gbps. It is supposedly faster than USB3. Will see

Click to expand...

Won't help - USB3 is already faster (or pretty close) to hard drive speed. But it's pretty hard finding consumer connections even at 100mbit - Google is rolling it out in Kansas City as a trial and Sonic.net is doing it in Santa Rosa area and a small section of San Francisco in another year or two.

kelpdiver said:

kelpdiver wrote:

Won't help - USB3 is already faster (or pretty close) to hard drive speed. But it's pretty hard finding consumer connections even at 100mbit - Google is rolling it out in Kansas City as a trial and Sonic.net is doing it in Santa Rosa area and a small section of San Francisco in another year or two.

Click to expand...

Sonic.net are offering 100 Mbps for $40/mo? Yikes! I might just have to move to Santa Rosa!

More on their trial here:


http://www.pressdemocrat.com/article/20120506/BUSINESS/120509761/1036/business?p=2&tc=pg

Sadly, I'm limited to old fashioned DSL (not even Fusion here due to a RT) but at least it's the best DSL you can get, thanks to Sonic. They are such a joy to work with compared with AT&T et al.

First Copy always takes forever, Increments thereafter are quicker. Use Karen's Replicator FREE, you will not regret it ever. Let it backup your 1TB or 2TB to other hard drive and go for a Coffee break or make Love to wife.