I use rsync to send files to a remote server mostly using a limited bandwidth connection that can also have packet loss or jitter, sometimes randomly rsync will fail with the following error:
default_perms_for_dir: sys_acl_get_file(photos/2019-05-11_01-31-51, ACL_TYPE_DEFAULT): No such file or directory, falling back on umask
rsync: mkstemp "/photos/2019-05-11_01-31-51/.data.txt.CJXeor" (in remote_bkp_site) failed: No such file or directory (2)
rsync: mkstemp "/photos/2019-05-11_01-31-51/.output_20190511-013202.mp4.Bhka2P" (in remote_bkp_site) failed: No such file or directory (2)If I run the rsync again it will work and successfully send all the necessary files to the remote site. This error happens randomly, not on the same file.
I use rsyncd daemon on the server side and my root directory may have about 1000 files. The old files doesn't change, I mostly use rsync to push new files to the server.
I did a lot of research and found out this error mostly happen to mounts ( CIFS ) or permission error but it's not my case. Also if I run rsync again a couple of times it will eventually sync all the files.
I believe this error has to do with the fact I sync the servers over a 4G broadband connection that has a limited bandwidth and packet loss can occur.
Do anyone has any suggestion or encounter a similar situation ?
13 Answers
I have found under linux that when transferring files across different fs types, the mkstemp call can fail due to a valid name on the source system being invalid on the target system.
This means even a touch of the filename will fail & not all fs seem to accept escape sequences to avoid this. I verify that I can touch into the target folder, but it fails when I incorporate the first invalid character (eg, ':') even when escaped.
Cause
Rsync can't seem to find some of the files it's supposed to sync. This happens when you're adding and/or deleting files in the source while the rsync is running.
The probable reason is that rsync does an initial scan on the source at the start of the sync and some of the files are already gone when rsync finally comes around to syncing them to the destination. It will then throw an error.
This also explains why, after a couple of attempts, the sync succeeds (as the time between initial scan and copying the file's data gets shorter so there is less chance of a file changing between scan and copy).
The files that fail look like temporary files. Which might indicate that the fact they are not synchronized to the destination is not a real problem.
Solution
The solution is to prevent your source from changing during the sync, e.g. by syncing from a static (snapshot) copy of the data. You might also learn to live with the errors.
1Same thing happened to me. The problem was I was running multiple rsync processes at the same time in the background, while using the option which deletes source files.
Solved by killall rsync. Then ran the script one last time and everything was ok.
