Free Windows file sync programs? FreeFileSync, etc.

LynniePad said:

Eric Carlson said:

LynniePad said:

Robocopy has been around for a while, Win7 at least.

Click to expand...

Since Windows NT 4.0 resource kit. A really long time ago. It's great for IT work, and for people who grew up using the command line, though not necessarily ideal for people not used to using a command prompt.

Click to expand...

Yes, but some early versions were rather limited. I mentioned Win7 because it was likely the first version that many users employed. Wikipedia discusses the history.

I’d suggest that the computer skills required for command line operation are rather less than those required for wrangling LR and PS, and for the many other complex activities often recommended by our correspondents. Craig, for example, makes my head hurt. :-D

Click to expand...

Well, I *am* an 'IT Professional'. But my example at least had comments to explain the settings.

LynniePad said:

I’ve been using Robocopy for backup since 2019 with good results. There does seem to be some reluctance to use the program, but it’s ideal for my requirements.

Click to expand...

The biggest issue I have is one that faces every type of backup program - what happens to deleted files?

In my case I back up multiple folders - one for videos, one for webpages, one for stills, etc, but one of my folders is a temporary holding area while I rate, cull and process before moving the finished files to their permanent location. I end up old files in backup that were moved or deleted.

My solution is to periodically delete the entire temp holding area folder in the backup and let the next backup repopulate it with whatever is current. Puts the temp folder contents at risk, so I try to do this cleanup between photo projects.

I could use options to sync files in both directions, so that deleting a file in the source also deletes it in the backup. I just find that too risky.

LynniePad said:

Eric Carlson said:

LynniePad said:

Robocopy has been around for a while, Win7 at least.

Click to expand...

Since Windows NT 4.0 resource kit. A really long time ago. It's great for IT work, and for people who grew up using the command line, though not necessarily ideal for people not used to using a command prompt.

Click to expand...

Yes, but some early versions were rather limited. I mentioned Win7 because it was likely the first version that many users employed. Wikipedia discusses the history.

I’d suggest that the computer skills required for command line operation are rather less than those required for wrangling LR and PS, and for the many other complex activities often recommended by our correspondents. Craig, for example, makes my head hurt. :-D

I’ve been using Robocopy for backup since 2019 with good results. There does seem to be some reluctance to use the program, but it’s ideal for my requirements.

Click to expand...

Does Robocopy support versioning?

rem copy photos to QNAP
robocopy e:\photos m:\photos /MT:64 /R:2 /E /XO /TIMFIX /FFT /xf d:\pagefile.sys /xd d:\database d:\nxcache d:\temp d:\temp2 d:\$recycle.bin d:\recycler "d:\System Volume Information"

Why all the exclude directories when they are not sub-directories of e:\photos?

Carey Brown said:

Does Robocopy support versioning?

Click to expand...

No.

PHXAZCRAIG said:

LynniePad said:

Eric Carlson said:

LynniePad said:

Robocopy has been around for a while, Win7 at least.

Click to expand...

Since Windows NT 4.0 resource kit. A really long time ago. It's great for IT work, and for people who grew up using the command line, though not necessarily ideal for people not used to using a command prompt.

Click to expand...

Yes, but some early versions were rather limited. I mentioned Win7 because it was likely the first version that many users employed. Wikipedia discusses the history.

I’d suggest that the computer skills required for command line operation are rather less than those required for wrangling LR and PS, and for the many other complex activities often recommended by our correspondents. Craig, for example, makes my head hurt. :-D

Click to expand...

Well, I *am* an 'IT Professional'. But my example at least had comments to explain the settings.

Click to expand...

I wasn’t particularly referring to your recent Robocopy post, but some of your other topics are rather specialised. Note that I wasn’t being critical.

PHXAZCRAIG said:

The biggest issue I have is one that faces every type of backup program - what happens to deleted files?

Click to expand...

In my case, there are very few deleted files. A recent check revealed six accumulated files in the backup copy, but these were not an issue.

Carey Brown said:

LynniePad said:

Eric Carlson said:

LynniePad said:

Robocopy has been around for a while, Win7 at least.

Click to expand...

Since Windows NT 4.0 resource kit. A really long time ago. It's great for IT work, and for people who grew up using the command line, though not necessarily ideal for people not used to using a command prompt.

Click to expand...

Yes, but some early versions were rather limited. I mentioned Win7 because it was likely the first version that many users employed. Wikipedia discusses the history.

I’d suggest that the computer skills required for command line operation are rather less than those required for wrangling LR and PS, and for the many other complex activities often recommended by our correspondents. Craig, for example, makes my head hurt. :-D

I’ve been using Robocopy for backup since 2019 with good results. There does seem to be some reluctance to use the program, but it’s ideal for my requirements.

Click to expand...

Does Robocopy support versioning?

Click to expand...

No, but when necessary, I keep multiple versions of a developing project in the working folder: (thesis001.docx, thesis002.docx, thesis003.docx etc.).

In addition to the quick Robocopy backup, I regularly back up the whole working folder system, without overwriting, so there’s about 2 years of versioning. Note that the working folder is only 30Gb, and there’s a separate backup system for my photo archive (which is also supplemented by a Robocopy routine).

What some correspondents don’t seem to understand is that, after the initial backup, subsequent runs only copy new or changed files by default. That, plus the implementation of multi-tasking makes for very quick backup. The first time that I used MT with SSD source and destination drives, I thought that the procedure had failed because it was done in an instant.

Carey Brown said:

rem copy photos to QNAP
robocopy e:\photos m:\photos /MT:64 /R:2 /E /XO /TIMFIX /FFT /xf d:\pagefile.sys /xd d:\database d:\nxcache d:\temp d:\temp2 d:\$recycle.bin d:\recycler "d:\System Volume Information"

Why all the exclude directories when they are not sub-directories of e:\photos?

Click to expand...

Probably a leftover from copying a section of a batch file and changing the drive letter. (One size fits all.) I set this up maybe 10 years ago or more.

Eric Carlson said:

Chris Noble said:

Yes, but the versioning option is very complicated compared to typical backup software.

Click to expand...

It's super simple. I set up the versioning

Click to expand...

Having to set up versioning is not simple.

Eric Carlson said:

to go to a separate folder on the backup drive(s),

Click to expand...

Setting up separate folders is not simple.

Eric Carlson said:

and if you find the current version of one of your files has a problem, just go to the same path under the separate directory

Click to expand...

Manual searching is not simple, especially if multiple files are involved.

Eric Carlson said:

to find previous versions. Simpler than restoring a backup.

Click to expand...

Actually, your system sounds super complicated. In contrast, restoring to any point in the past is a single click operation with versioning backup software.

And how do you manage your archiving? (How many previous versions to save).

LynniePad said:

Carey Brown said:

LynniePad said:

Eric Carlson said:

LynniePad said:

Robocopy has been around for a while, Win7 at least.

Click to expand...

Since Windows NT 4.0 resource kit. A really long time ago. It's great for IT work, and for people who grew up using the command line, though not necessarily ideal for people not used to using a command prompt.

Click to expand...

Yes, but some early versions were rather limited. I mentioned Win7 because it was likely the first version that many users employed. Wikipedia discusses the history.

I’d suggest that the computer skills required for command line operation are rather less than those required for wrangling LR and PS, and for the many other complex activities often recommended by our correspondents. Craig, for example, makes my head hurt. :-D

I’ve been using Robocopy for backup since 2019 with good results. There does seem to be some reluctance to use the program, but it’s ideal for my requirements.

Click to expand...

Does Robocopy support versioning?

Click to expand...

No, but when necessary, I keep multiple versions of a developing project in the working folder: (thesis001.docx, thesis002.docx, thesis003.docx etc.).

Click to expand...

I use FreeFileSync which has versioning. On two critical folder hierarchies that are in constant change I set up FFS script to back them up with versioning every hour. Runs in the background and is very fast.

LynniePad said:

In addition to the quick Robocopy backup, I regularly back up the whole working folder system, without overwriting, so there’s about 2 years of versioning. Note that the working folder is only 30Gb, and there’s a separate backup system for my photo archive (which is also supplemented by a Robocopy routine).

What some correspondents don’t seem to understand is that, after the initial backup, subsequent runs only copy new or changed files by default. That, plus the implementation of multi-tasking makes for very quick backup. The first time that I used MT with SSD source and destination drives, I thought that the procedure had failed because it was done in an instant.

Click to expand...

Coming from Mac, I used TimeMachine for backup to external disk and rsync to keep a copy of my latest files on another external disk.

Now on Windows, I still use rysnc (thru cygwin unix). It is command line. I like rsync in that it will copy only when needed, when files changed. rsync can also delete orphaned files (files deleted from source).

It's super simple to set up and use versioning. If you can use backup and restore, you can use versioning without much thought or effort at all.

Eric Carlson said:

It's super simple to set up and use versioning. If you can use backup and restore, you can use versioning without much thought or effort at all.

Click to expand...

i took a quick look at:

"FreeFileSync - Open Source File Synchronization"

https://freefilesync.org/manual.php?topic=versioning

from what I read, (clip from their documenation pasted below) it uses a folder-based system to store both the current versions, and a folder structure to store the previous file versions. Personally I avoid folder-based archives. I find there are several issues with them:

1) Different OS have much different file-naming specifications. A big one is that Linux type OS supports case-sensitive naming. Windows is case-insensitive. So right there is a conflict.

2) also different OS allow different characters in the file names. And the current windows OS versions (win10, win11) still have issues with long file names (> around 260 characters, total). Programs have to be modified at the source code level! to use the actual limit of around 32,000+ characters...

3) the archive consists of a huge number of files...

3.a) So this slows down file copy/backup (you do back up your archives, right?? I sure do!!)

3.b) Individual files can be corrupted (as can entire sub-folders). Windows will often "mask" this by automatically "fixing the program" which means the corrupted folders/files are removed. Unless you are watching carefully, all the time, this can go un-noticed

3.c) file date/time stamps can accidently be set, losing the actual file date/times

3.d) not all file systems support the create date/time stamps!!

I prefer my archive versioning/journalling program to store the entire archive in a single file. To me this is more convient, and more reliable

Currently I use Robocopy to take all the important folders/files into a single backup folder. Then I use a versioning/journaling program ("zpaq", open source so it will never become an orphaned program) to update the archive file. Periodically I backup the archive file.

I do this all thru a batch file, which makes the process easy and convienient. Plus I can backup/archive the batch files!!

Robocopy and zpaq are both plenty fast enough that I run them often during the day

"2. Keep multiple versions of old files
Set deletion handling to Versioning and naming convention to Time stamp [File]. FreeFileSync will move deleted files into the provided folder and add a time stamp to each file name. The structure of the synchronized folders is preserved so that old versions of a file can be conveniently accessed via a file browser.

Example: Last versions of the file Folder\File.txt inside folder D:\Revisions
D:\Revisions\Folder\File.txt 2020-12-11 111111.txt
D:\Revisions\Folder\File.txt 2020-12-12 122222.txt
D:\Revisions\Folder\File.txt 2020-12-13 133333.txt

With naming convention Time stamp [Folder] files are moved into a time-stamped subfolder of the versioning folder while their names remain unchanged. This makes it easy to manually undo a synchronization by moving the deleted files from the versioning folder back to their original folders.

Example: Last versions of the file Folder\File.txt inside folder D:\Revisions
D:\Revisions\2020-12-11 111111\Folder\File.txt
D:\Revisions\2020-12-12 122222\Folder\File.txt
D:\Revisions\2020-12-13 133333\Folder\File.txt"

Eric Carlson said:

It's super simple to set up and use versioning. If you can use backup and restore, you can use versioning without much thought or effort at all.

Click to expand...

OK; let's say I want to roll back my entire system to the state it was in on Sept 1, except for my data files that I want to roll back to the state they were in on Sept 25th, without affecting the newly created data files since Sept 25th. What is the super simple sequence of mouse clicks to do that with FreeFileSync?

This would be the situation if there is some corruption in the OS that exhibited itself between Sept 1 and Sept 25th, but has not affected the data files.

Chris Noble said:

Eric Carlson said:

It's super simple to set up and use versioning. If you can use backup and restore, you can use versioning without much thought or effort at all.

Click to expand...

OK; let's say I want to roll back my entire system to the state it was in on Sept 1, except for my data files that I want to roll back to the state they were in on Sept 25th, without affecting the newly created data files since Sept 25th. What is the super simple sequence of mouse clicks to do that with FreeFileSync?

This would be the situation if there is some corruption in the OS that exhibited itself between Sept 1 and Sept 25th, but has not affected the data files.

Click to expand...

Sync is not for the OS, just for data. Use file backup and/or image backup for the OS and apps. Obviously. Or just reinstall.

No big deal if I lose my OS or apps. I'll just switch to another computer. And I have my sync'd data, so it's all good.

--
- Eric, http://www.invisiblerobot.com/

foot said:

1) Different OS have much different file-naming specifications.

2) also different OS allow different characters in the file names.

Click to expand...

Not relevant to someone who only uses Windows as their desktop OS, like me, and many people.

foot said:

3) the archive consists of a huge number of files...

3.a) So this slows down file copy/backup (you do back up your archives, right?? I sure do!!)

Click to expand...

I make multiple syncs to multiple drives.

It only takes a few minutes when done regularly. Just like if you do incremental backups regularly. No difference there.

Backups have their uses, and so does sync. If your using robocopy, to copy files, that's roughly analogous to a sync. Sounds like a sync (copy) before backup. I do that at work sometimes. Nothing wrong with that.
Everyone can backup however they see fit. Even using many ways in parallel, like file sync + backup + image backup, if you like. They all have their advantages and disadvantages. If you want to quickly restore 1 or 2 files or folders, of use them elsewhere: Sync. If you want to restore files to a specific day: File backup. If you want to restore your OS, and apps: Image backup.

I'm helping solve the problem of losing older versions of data files when doing a data sync, and versioning is perfect for that. And a sync is fully accessible on any computer (that has the same OS, and even mostly on other OSes if you're coming from a Windows sync) with no special software.

--
- Eric, http://www.invisiblerobot.com/

Eric Carlson said:

foot said:

1) Different OS have much different file-naming specifications.

2) also different OS allow different characters in the file names.

Click to expand...

Not relevant to someone who only uses Windows as their desktop OS, like me, and many people.

Eric Carlson said:

3) the archive consists of a huge number of files...

3.a) So this slows down file copy/backup (you do back up your archives, right?? I sure do!!)

Click to expand...

I make multiple syncs to multiple drives.

It only takes a few minutes when done regularly. Just like if you do incremental backups regularly. No difference there.

Backups have their uses, and so does sync. If your using robocopy, to copy files, that's roughly analogous to a sync. Sounds like a sync (copy) before backup. I do that at work sometimes. Nothing wrong with that.
Everyone can backup however they see fit. Even using many ways in parallel, like file sync + backup + image backup, if you like. They all have their advantages and disadvantages. If you want to quickly restore 1 or 2 files or folders, of use them elsewhere: Sync. If you want to restore files to a specific day: File backup. If you want to restore your OS, and apps: Image backup.

I'm helping solve the problem of losing older versions of data files when doing a data sync, and versioning is perfect for that. And a sync is fully accessible on any computer (that has the same OS, and even mostly on other OSes if you're coming from a Windows sync) with no special software.

Click to expand...

we agree on just about everything. So these are minor points:

The word "synch" can be ambigous, since there is "one way" vs "two-sync" synch.

I use Robocopy "one-way", all changes are only in the destination folder, never in the source folder.

Yes, I'm using robocopy to update/create the backup folder. Then the versioning/journaling archive programis run This is the step that updates/creates the arfile file

and of course the periodic backup of the archive file to a seperate disk, then at some point to a offline storage

foot said:

Eric Carlson said:

foot said:

1) Different OS have much different file-naming specifications.

2) also different OS allow different characters in the file names.

Click to expand...

Not relevant to someone who only uses Windows as their desktop OS, like me, and many people.

foot said:

3) the archive consists of a huge number of files...

3.a) So this slows down file copy/backup (you do back up your archives, right?? I sure do!!)

Click to expand...

I make multiple syncs to multiple drives.

It only takes a few minutes when done regularly. Just like if you do incremental backups regularly. No difference there.

Backups have their uses, and so does sync. If your using robocopy, to copy files, that's roughly analogous to a sync. Sounds like a sync (copy) before backup. I do that at work sometimes. Nothing wrong with that.
Everyone can backup however they see fit. Even using many ways in parallel, like file sync + backup + image backup, if you like. They all have their advantages and disadvantages. If you want to quickly restore 1 or 2 files or folders, of use them elsewhere: Sync. If you want to restore files to a specific day: File backup. If you want to restore your OS, and apps: Image backup.

I'm helping solve the problem of losing older versions of data files when doing a data sync, and versioning is perfect for that. And a sync is fully accessible on any computer (that has the same OS, and even mostly on other OSes if you're coming from a Windows sync) with no special software.

Click to expand...

we agree on just about everything. So these are minor points:

The word "synch" can be ambigous, since there is "one way" vs "two-sync" synch.

I use Robocopy "one-way", all changes are only in the destination folder, never in the source folder.

Yes, I'm using robocopy to update/create the backup folder. Then the versioning/journaling archive programis run This is the step that updates/creates the arfile file

and of course the periodic backup of the archive file to a seperate disk, then at some point to a offline storage

Click to expand...

I've been banging on about Robocopy for some time now, and it's pleasing to see that you use it to good effect. I regularly update the archive folder using the program and, as mentioned, multi-threading speeds this up remarkably.

I am very selective about versioning as I only keep versions of significant projects in the main data folder (and backup folder, of course).

In addition to Robocopy, I back up everything monthly, and this is my "time machine". As mentioned, there's only about 30Gb of data, so storage is not an issue.

We need to distinguish between "project versioning" and "system error versioning". I'm not so concerned about the latter.

Eric Carlson said:

Chris Noble said:

Eric Carlson said:

It's super simple to set up and use versioning. If you can use backup and restore, you can use versioning without much thought or effort at all.

Click to expand...

OK; let's say I want to roll back my entire system to the state it was in on Sept 1, except for my data files that I want to roll back to the state they were in on Sept 25th, without affecting the newly created data files since Sept 25th. What is the super simple sequence of mouse clicks to do that with FreeFileSync?

This would be the situation if there is some corruption in the OS that exhibited itself between Sept 1 and Sept 25th, but has not affected the data files.

Click to expand...

Sync is not for the OS, just for data. Use file backup and/or image backup for the OS and apps. Obviously. Or just reinstall.

Click to expand...

Super simple to use backup for both OS and data. Then no additional need for sync.

Eric Carlson said:

No big deal if I lose my OS or apps. I'll just switch to another computer. And I have my sync'd data, so it's all good.

Click to expand...

Super simple to avoid having to get another computer and reinstall and reconfigure all your apps -- by just having a backup. Sounds even more super than your super, doesn't it?

Chris Noble said:

Eric Carlson said:

Chris Noble said:

Eric Carlson said:

It's super simple to set up and use versioning. If you can use backup and restore, you can use versioning without much thought or effort at all.

Click to expand...

OK; let's say I want to roll back my entire system to the state it was in on Sept 1, except for my data files that I want to roll back to the state they were in on Sept 25th, without affecting the newly created data files since Sept 25th. What is the super simple sequence of mouse clicks to do that with FreeFileSync?

This would be the situation if there is some corruption in the OS that exhibited itself between Sept 1 and Sept 25th, but has not affected the data files.

Click to expand...

Sync is not for the OS, just for data. Use file backup and/or image backup for the OS and apps. Obviously. Or just reinstall.

Click to expand...

Super simple to use backup for both OS and data. Then no additional need for sync.

Chris Noble said:

No big deal if I lose my OS or apps. I'll just switch to another computer. And I have my sync'd data, so it's all good.

Click to expand...

Super simple to avoid having to get another computer and reinstall and reconfigure all your apps -- by just having a backup. Sounds even more super than your super, doesn't it?

Click to expand...

If there are several computers available, each with an installed suit of programs, work can continue almost uninterrupted by simply accessing a shared folder, or maybe by loading a backup data set.

In my experience, a crashed/failed computer generally needs more that an OS restore.

LynniePad said:

If there are several computers available, each with an installed suit of programs,

Click to expand...

I prefer a backup program to that option. We're talking about backup vs. sync in this thread.