Online raw storage solutions

Dervast

Senior Member
Messages
2,795
Reaction score
111
Location
Germany
Dear all,

I am looking for an online raw storage solution for my files. I would like to also have a support tool so I can upload my photos with my current folder structure (and not need to make each folder manually).
An upload tool that also has resume and pause uploading would be desirable too (So I can upload at nights and pause uploading when I need my fast internet connection).Even better would be if I can customize the upload tool so it runs every midnight and pauses at 8:00 am, but I guess in the real marketing world I would need to spend 300 euros just for such tool.RegardsAlex
 
Thanks for the answer.

Currently I have more than 1Tb for my files and unfortunately these reviews do not give enough information about the prices when you go over their normal offers.

It is also not clear what theirs syncing windows tools offer. Do they allow me to upload, for example, all my folders structure or I have to set each one individually.

I guess I would have to start with the demos version first and see from there...

Regards

Alex
 
I'm using OneDrive and Amazon Photos for RAW storage. OneDrive offers a sync program that you can manually pause and restart. Amazon Photos is a manual process where you can move a folder structure in Google's Chrome Browser on Windows, to the service. I'm using a date folder structure, YYYY/MM/DD, so I just grab the top level, year, and move that into Amazon, which copies all the month and day sub folders. The moving in Chrome works the same as a copy would in Windows, where files that are already in the service can be overwritten or not, by your command.

OneDrive offers unlimited file storage with Office 365 for US $80 a year, and Amazon Photos offers unlimited Photo storage for $13 a year, or as part of the US Amazon Prime membership, which costs US $99 a year. The US $99 Amazon Prime membership include Prime Video and Prime Music service, well as the free two day shipping for goods bought through Amazon.

For your request of being able to turn things on and off, I just use the traffic management feature in my router to manage the bandwidth to my desktop which runs the sync software. I turn down the upload bandwidth to the desktop, so that it doesn't interfere with the other computers use of the Internet. Most are just pulling down data, so as long as there is a little bit of upload bandwidth left to acknowledge the download data, things work well.

Thank you

Russell
 
Last edited:
Currently I have more than 1Tb for my files and unfortunately these reviews do not give enough information about the prices when you go over their normal offers.
This is one of the many things I hate about online services; they all have a morass of pricing schemes, deals, extras, offers, teasers etc. I look at the list prices to make a decision and disregard the teasers to focus on the "real" price.
It is also not clear what theirs syncing windows tools offer. Do they allow me to upload, for example, all my folders structure or I have to set each one individually.
CrashPlan definitely has all of that -- and the ability to backup to another of your own machines as well as the cloud. They seem to have pretty decent reviews so I think they're worth a look.
 
Currently I have more than 1Tb for my files and unfortunately these reviews do not give enough information about the prices when you go over their normal offers.
Once you're passing a terabyte, they're no longer interested in offering one price fits all plans, and would rather simply charge by the gig. Part of the allure (from their perspective) of the 100G/1Tb tiered pricing is that so many users will fall in the middle in usage but round up in terms of payment.
 
Thanks for the answer.

Currently I have more than 1Tb for my files and unfortunately these reviews do not give enough information about the prices when you go over their normal offers.
Is that compressed? If you compress your raw files (using gzip or similar), they'd take up a *lot* less space.

IOW, you may want to compress some of your folders into extra copies of them that way. Then, just sync the compressed versions of them.

But, make sure to read the "fine print" with any cloud storage plan you use if you're using compressed folders, as some plans have a file size limitation (sometimes 1GB, sometimes 2GB, etc.)

Google Drive lets you choose from plans that offer more than 1TB. But, the pricing jumps from 1TB (at $9.99/month) to 10TB (at $99.99/month), with nothing in between.

https://support.google.com/drive/answer/2375123?hl=en
It is also not clear what theirs syncing windows tools offer. Do they allow me to upload, for example, all my folders structure or I have to set each one individually.
There are third party tools you can use to give you more features. For example, Insync is a good one for use with Google Drive.

https://www.insynchq.com/

It lets you keep your existing folder structure, and gives you a "right click" option to add any existing folder to Insync. You'll see that shown on their Why Insync page here:

https://www.insynchq.com/why

For a single Google Drive Account, you can get it for a one time fee of $20 (no recurring subscriptions or payments). They also offer a Pro plan that's $20/year with more features that allows multiple accounts.

As Malch mentioned, you may want to consider something like Crashplan, too (as they have unlimited plans at reasonable prices, and seem to get fairly good reviews for the most part).

http://www.code42.com/crashplan/

BTW, have you thought about just how long it could take to upload that much data, and if your ISP has any monthly caps on bandwidth usage?

For example, I'm using a Comcast Cable Modem service and they now cap my monthly bandwidth at 300GB/Month. I can use more, but they tack on another $10 for each 50GB over the 300GB limit.

You also need to consider how long it can take to upload all of your files. Most Broadband Services have much slower upload times compared to download times. For example, a plan with 50Mbps Download Speed may only allow 10Mbps Upload Speed. In my case, I have a "Performance Plan" where I get around 30Mbps Download Speed, but I only get 6Mbps upload Speed.

Current Speed:

4395165485.png


That's Megabits per Second, not Megabytes per Second. There are 8 bits in a byte. So, 6Mbps (Megabits per second) is only 0.75 Megabytes per second (less than 1 Megabyte per second.

So, if you have 2TB of Data you want to sync at 6Mbps, that works out to a transfer time of over 854 hours (approx. 36 days running 24x7).

But, in reality, you're not going to get maximum speed though due to a variety of reasons (overhead of sync software, multiple smaller files vs one larger file, other applications sharing the same internet service, etc.) So, you'll probably only get around half of what your max upload speed is if you're lucky

Here's a transfer speed calculator that can come in handy:

http://techinternets.com/copy_calc?do

Of course, if you have other devices accessing the internet, that's going to impact your transfer speed, too.

But, considering bandwidth limitations that Comcast has for my broadband service, and that I tend to use around 200GB of Bandwidth each month for other purposes (browsing, downloads, watching videos, etc.), I could only transfer around 100GB of files to a cloud based storage site per month without incurring overage fees, meaning I'd need over a year to finish syncing that much data. :-)

Some services like Crashplan do have options where they'll send you a physical hard drive to help out with the initial sync though. But, I think the largest they offer is 1TB (with no ability to do it more than once either). Then, you have to sync the rest via the internet.

--
JimC
------
 
Last edited:
Thanks for the answer.

Currently I have more than 1Tb for my files and unfortunately these reviews do not give enough information about the prices when you go over their normal offers.
Is that compressed? If you compress your raw files (using gzip or similar), they'd take up a *lot* less space.

IOW, you may want to compress some of your folders into extra copies of them that way. Then, just sync the compressed versions of them.
--
JimC
------
There are two problems with compression:

1) Most RAW files are already compressed so any additional amount of compression is miniscule. I just tested this on a large (3.7GB) directory of .CR2 (Canon) files with WinZip and only achieved a 1% reduction in size.

2) If you compress multiple files into a single zip file you thwart the cloud's ability to do incremental backup.
 
If you archive DNG files to a cloud service, archiving your original raw files as you do now locally, (always keep your original files), you can use Adobe's DNG converter to remove the jpeg previews and that will save a lot of space and bandwidth. If you wanted, you could extract the jpeg previews from the RAW files, and save them to a private Flicker account, so that both the RAW data and the jpeg previews are in cloud storage.

Thank you

Russell
 
Last edited:
Thanks for the answer.

Currently I have more than 1Tb for my files and unfortunately these reviews do not give enough information about the prices when you go over their normal offers.
Is that compressed? If you compress your raw files (using gzip or similar), they'd take up a *lot* less space.

IOW, you may want to compress some of your folders into extra copies of them that way. Then, just sync the compressed versions of them.
Yea.. the savings are not that great. I expected more.

If I take a 24.0MB raw file (.arw extension) from the A6000 (what it looks like the OP is shooting with now), I get a a 21.9MB file when zipped. I see similar results with other current Sony dSLR models.

I tried it with raw files from the A6000 review (by selecting RAW for comparisons and clicking on download links)

http://www.dpreview.com/reviews/sony-alpha-A6000/12

So, the OP would be looking at a 2.1MB/File savings (or around 9% less space needed).

That's probably not worth the effort given the down sides. With your .cr2 files, the savings are even less. Again, not worth the effort.

--
JimC
------
 
Last edited:
I think it might be possible to have desktop syncing in Amazon's Photo service with a work around.

DuOS allows you to specify local storage on the host system as the Picture directory on the emulated Android machine. Amazon's Photo app on Android automatically uploads photos on a Android device to its cloud service. Put them together and it might offer a solution for the lack of syncing with Amazon's service.


I'm going to set this up as soon as I've finished loading my Desktop photo file to Amazon using the manual pick via Chrome. Unfortunately that will take a while still, so someone else might like to give this a go to try out. DuOS has a 30 day, fully functional, trial.

Amazon's US $13 a year unlimited photo storage, includes RAW, makes it the least expensive photography solution, and its US $60 unlimited, store anything, a year subscription is also the least expensive cloud storage, that I know of. It's only the manual syncing that limits its appeal for me. If this works I probably will drop the OneDrive 365 yearly subscription and pick up the Amazon, store anything, subscription.

Mostly my other large storage data is just wav files from the audio CDs we own and a few ripped DVDs that we also own. I've got some home movies files as well now, as owning a video capture device is now part of the other things we buy, ILC cameras, and cell phones. All the rest of our personal data can be stored on any of the free accounts that have syncing software, Google and Microsoft, that offer 15Gb of free cloud storage, so there isn't a lot pressing need to be able get those files into Amazon's storage.

Thank you

Russell
 
Unlimited free storage for photos up to 16 mb (yay - m43s!) larger photos count against your 15 gig free (so I assume you can buy more)

 
Unlimited free storage for photos up to 16 mb (yay - m43s!) larger photos count against your 15 gig free (so I assume you can buy more)

http://www.google.com/photos/about/
The free service doesn't support raw file types (what the OP is trying to store), unless you only want the embedded jpeg files.

I spent some time with today trying all sorts of things, including uploading a variety of photos to it in order to find out how it handles resizing, compression, etc.

Basically, you can set it up for "High Quality" so you get free storage, and it automatically downsizes any images over 16 Megapixels to 16 Megapixel dimensions. It also recompresses images (even if they're not larger than 16 Megapixels).

The compression is surprisingly good though (I could see no difference in image quality uploading jpeg files saved at 90% quality originally and what Google Photos did when recompensing them.

There was very little difference in file size either for photos under 16 Megapixels. For example, if I saved a 16MP or smaller file as jpeg with an 85% quality setting using something like irfanview, I actually got a slightly smaller file size than how Google Photos recompressed them (Google Drive's compression appeared to be closer to around an 88% quality setting if using something like irfanview from what I could see with a few crude tests).

You'd need to test a greater variety of images (and scene types) to draw any conclusions though (but, again, the compression algorithms being used appear to be excellent from what I could tell by "eyeballing" the images after uploading them, then downloading the compressed versions and viewing them after zooming in with an editor)

Even videos worked fine, with huge file size reductions with some .mov files I tried (with info about the videos showing the same codecs, bit rate, etc. for the most part when they were using mpeg 4 codecs). I'm sure we'll see reviews about it from people that know more about video formats commenting on what they're doing in the near future.

Anyway, Google Photos is not going to work for raw files.

Although not technically supported (if you look at the file types it filters your Windows Explorer view with that you can select from when using the Upload Button from their service), you can switch it to "all files" and it will let you upload raw files.

I tried using .cr2, .arw and .dng files to Google Photos and they uploaded OK. But, you can't download them in their original format.

It's storing a jpeg image only (probably just the embedded jpeg inside of the raw files). If you want to set quality to "Original", then you can upload them and download them with their original format. But, then you're using the paid vs free [unlimited] service.

See the setup options you get below. You'll need to use your Google Drive storage for raw files in their original format (versus only their embedded jpeg files using the unlimited option with Google Photos), meaning anything after your first 15GB of storage will require a paid plan).

654b1fd04b0c4636a692ba171ba652c9.jpg

--
JimC
------
 
Last edited:
Unlimited free storage for photos up to 16 mb (yay - m43s!) larger photos count against your 15 gig free (so I assume you can buy more)
yeah, that was a not accidentally picked value, wasn't it? With the exception of my GH4 and other m43s, virtually every SLR is at 18, 22, 24, etc.
 
Last edited:
I didn't catch that "No Raw" qualifier. To be honest, I had my doubts that the typical upload speeds would be adequate for raw anyway, they haven't been adequate so far. I guess we have to wait a bit longer before technology can catch up to our wishes.
It doesn't specifically say no raw files. But if you want unlimited photo storage, the only way to get it is using the "High" (versus "Original") Quality setting that is designed to resize and recompress your images

But, if you use the "High" (free, unlimited) vs "Original" (uses your Google Drive Storage) with raw files, it is only storing a jpeg, instead of letting you store or download the original file format.

I even tried it with a raw file that was only 12 Megapixels (versus > 16 Megapixels) to see if it would keep the original format that way.

Nope...I uploaded a raw file (.arw) from a Sony A700 that was a 12MP resolution image with a 18.3MB file size, and it's storing a 1.9MB JPEG image (original file name with .jpg tacked on to the end of it), with a resolution of 12 Megapixels (it kept the original resolution in that case).

Interestingly, it converted the raw file to jpeg using a Picasa engine according to the Exif Metadata in the case of the Sony A700 raw file. With other raw files I've tried, it sometimes uses what appears to be the embedded jpeg instead.

In any event, you don't get to store your raw files using Google Photos if you want unlimited [free] storage.

See more about the storage sizes and associated quality settings on this page:

https://support.google.com/photos/answer/6220791?p=storage&rd=1

--
JimC
------
 
Last edited:
Unlimited free storage for photos up to 16 mb (yay - m43s!) larger photos count against your 15 gig free (so I assume you can buy more)
El Reg has raised a potentially major issue with the Google Photos terms of service:


Be aware.

Since I already use Gmail, Google Contacts, Google Drive and more, this looked like a natural for me. So I spent a little time playing with it today. Overall I was not too impressed:

* I am a committed RAW shooter so the free service doesn't work that well for me.

* Although their compression seems to work well (extremely well) these things always worry me when I don't have full control.

* Organizational options were not that well suited to my needs/preferences.

Nevertheless, it'll probably come in handy if I want to quickly put a collection of images on-line.
 
84b5941d3f93421182af4f0062f1bd4c.jpg
 

Attachments

  • c849419eac4648e990267f1d1db958e6.jpg
    c849419eac4648e990267f1d1db958e6.jpg
    41.2 KB · Views: 0
Last edited:
I'd look at this product (Syncovery, formerly known as Super Flexible File Synchronizer).


The latest beta (7.20 Beta 15) of it supports Amazon Cloud:Drive. Get that here (scroll down past the main downloads and you'll see links to the beta)


Direct links to 7.20 Beta 15 downloads for Windows:
It will work as a trial without buying it for 15 days. If you decide you want it, you'll probably want the Pro version to get features like real time folder monitoring and sync, etc. That would run you $69.90. See a pricing grid here:


Feature comparison:

 

Keyboard shortcuts

Back
Top