Storage Solution for Large Photo Collection

Started 2 months ago | Discussions thread
NilsDecker
NilsDecker Junior Member • Posts: 31
Re: Storage Solution for Large Photo Collection

DerKeyser wrote:

NilsDecker wrote:

Hi Wolff,

Thanks for sharing this, very helpful. I just finished reading the blog-post you wrote too which is even better (and coincidentally I literally got the same NAS in the mail today :-)).

How did you... figure out the NAS? It seems "a little" overwhelming at first. Which RAID configuration did you apply?

Thanks, Nils

Yes, right at start it looks a little daunting, but it’s not that bad once you get a few pointers. Hopefully these will help:

Wow, amazing! It did/ does help a LOT!

1: The NAS works with a concept called a storage pool. This is basically just the underlying disks RAID’ed together to provide gross space for different storage needs. Those needs are things like:

- Reserved space for the NAS OS itself
- Volumes (Filesystems) for your storage
- Snapshot space if you decide to snapshot volumes you created
So you need to create a storage pool from your disks in order to be able to create a volume (filesystem) for your data. When creating the storagepool you can select between different RAID levels. I only have two drives in my NAS, and hence i choose Raid 1 (Mirror). This allows one drive to fail without dataloss/problems.
If you have more drives, you could fx. look into Raid 6 which stripes data across 4 or more spindles of which two are used for “parity data” - IE: additional data calculated to allow for loss of up to two drives. So in Raid 6 you get your numbers of drives minus two’s capacity for userdata (fx: 6x8Tb equals 4x8Tb usable capacity).
You could also use Raid 5 which has only one drives capacity for parity data, but with todays HUGE drives, you do run a decent risk of not actually being able to rebuild a dead drive if you encounter a dead block on one of the remaining drives. If you make sure to have proper running backup, that could be an acceptable risk because of the drive capacity gained (or saved by buying one less drive).

I have 4x4tb. Would you recommend RAID 5 or 6 in that case? 4TB IronWolf 5900 rpm SATA III 3.5" Internal NAS to be precise which seems to be what you recommend in your next comment?

2: Once your storage pool is created - lets in this example use 3x10Tb drives in Raid 5 - you will have 20Tb of net. Space (one drives capacity is used for parity in Raid5).
The NAS itself will reserve some 200Gb of the space for NAS features so you cannot use that. The remaining 19.7Tb can be used by you to create a volume on. There are two kinds of volumes:
- A thick volume that reserves all its space right away
- A Thin volume that reserves space from the pool when you add data, and give it back when you delete data.

IMPORTANT: The key point here is NOT to create a volume that uses up all the space in the storage pool because you want space for snapshots as well. So fx. Only create a volume of 17Tb to allow for snapshotspace.

I'm still in "play mode" so the way I had it set up was: 4x4tb in RAID 5, giving me 10.9tb usable storage and I created a volume of 10tb on it. I reserved some space for system applications etc.

Based on your recommendation, I now shrunk the actual volume to 7.9tb and allocated "guaranteed snapshot space" of 5%. Is that too low? I found somewhere in the documentation that the default is 20%.

It can be a little confusing that a Thin volume can be created much bigger that the actual free storagepool space. But it’s pretty neat as you can add drives to the pool one space starts to run out. So in this case we could make a 60Tb volume up front even though we only have 19.8Tb in the pool.

I will strongly recommend NOT overallocating like that since it requires you to monitor free space, and it can cause SERIOUS problems if something allows the pool to run out of space because data is added to the thin volume. So stick to creating a 17Tb volume - you can always increase its size once you have added more disks to the storage pool.

Once the 17Tb volume (Thick or Thin) is created, setup a snapshot schedule of that volume. fx. Create a snapshot every night at 02:00 and keep up to 14 snapshots (14 days).
Those snapshots will use the remaining space in the storagepool, and if the get bigger than the remaining 2.7Tb the oldest ones will delete automatically. For this very reason I can recommend using a Thin volume (but NOT over allocated) vs. A thick. That way snapshots can use the 2.7Tb remaining space + whatever is not used by the automatically growing/shrinking volume

So I think I understand. BUT. I currently set up the volume as a thick type, but, with a guaranteed snapshot space (see above, might increase to more than 5% which is what it's currently at). Isn't that the same then, somehow?

Hopes all this makes sense

After your volume is ready and you have added data, make sure to download the APP called HBS 3 (Hybrid Backup Sync 3). This app allows you to setup automatic jobs that copy/backup or syncs data (I use Sync) to a cloud service on scheduled basis.
It also allows you to configure Local Backups via the One Touch Copy button on the front of the NAS (IE: When a USB drive is connected, pressing the button will sync all your data to the USB drive)

I will look into all of that once I feel I've found the right setup of RAID/ thick vs. thin/ etc.

Thanks a MILLION, I really really appreciate it! 100x better than watching YT tutorials and reading manufacturer documentation.

Good luck with your NAS - I’m sure you will love as much as I do once you get it all set up.

PS: There are a lot of APPs for the NAS that allows your images to be shared to clients around the house. Several uPnP/DNLA apps exist - fx. Photostation, but also more elaborate servers like PLEX are great. I use PLEX because that handles all my Music, Photos and Video in the same app on clients, and it allows offline sync of music/movies.

Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow