[CPANEL-21358] Unable to validate rsync backup destination
Hi,
I'm currently testing the new "Rsync" Backup Transport in version 74.0.2.
Current System Specs:
CentOS 7.5
cPanel / WHM 74.0.2
I've validated the host, user, password and destination folder by using cli commands, destination port is 23 (not 22) and everything works via cli.
With the same settings used as new backup destination validation always fails with the message
"Error: Validation for transport "Backup (Rsync)" failed: Could not create path directory "server03rs/": child exited with code 1"
When I connect to the destination via sftp I see the server03rs folder got created and a file in there (tmp.test.10504-1533208029) was uploaded which contains the text
"This is a test file created to determine if backups are operable::
10504
1533208029"
I can also download this file without problems via rsync cli command, sync via rsync into the folder etc. but the validation always fails.
As I can't find any logs about what's happening here I'm a bit clueless. Did anyone else ran into this problem or is there any way to find out what's exactly happening here?
Server + backupspace are located at Hetzner (Germany), SSH-Support is activated on backupspace.
kind Regards,
Sven B. 8)
-
Hello @Cyber911, It looks like this relates to the topic discussed on the following thread: Can you review this thread and let me know if that helps? Thank you. 0 -
Hi Michael, this does not help. I can't enter /../ or /../server03rs/ or /../server03rs as the system declines paths with "." or "..". I've already tried using "server03rs/" and "server03rs", if I enter the slash after path ("server03rs/") the validation fails with errormessage "Error: Validation for transport "Backup (Rsync)" failed: Could not upload test file: child exited with code 1", without slash ("server03rs") it gives "Error: Validation for transport "Backup (Rsync)" failed: Could not create path directory "server03rs/": child exited with code 1". In both cases the folder got created and a file was uploaded into it. 0 -
Hi Michael, Support Request ID is: 10066019 :) 0 -
Hello, To update, here's part of the response to the support ticket explaining why validation failed for the the Rsync backup destination on this system: [QUOTE] Currently our implementation of "rsync" backup destination relies on the rsync command over ssh. While often they are interchangeable, if the rsync user is restricted to only being able to use the "rsync" command, then it is possible that issues as you are experiencing would occur. For example on your server during the rsync process, the command "mkdir -p server123" is being run. Since this user is restricted there is a "Command not found" message being generated preventing the backup destination from validating. We currently have an internal case open, CPANEL-21358, to look into supporting restricted rsync users with our backup system. If a solution to this is reached, it will be published to our change logs when it is released in a public cPanel build. You can view the changelogs here: Change Logs - Change Logs - cPanel Documentation Unfortunately at this time we do not have a valid work-around as there are other functions of the rsync backup destination that rely on other normal ssh commands.
I'll update this thread with more information on the status of case CPANEL-21358 as it becomes available. Thank you.0 -
Still same problem for me too with Hetzner, can You please update what is the status of this ?? 0 -
I can confirm that this was a resent change, where as a "/" in the beginning of the path is denied by the GUI. A workaround, until it gets sorted: Go to: /var/cpanel/backups Edit the file named: somthing_UID_##some-key##_backup_destination ad a "/" in the beginning of the "path:" to change it to absolute. I have tested this with */backup --force, and it seems to work. 0 -
Nope, that doesn't help. 0 -
Nope, that doesn't help.
Nope.. sry, I posted that in the wrong thread! This was to fix the backup transport going to /root folder. You need to get past the validation before that is relevant :D However, I resonantly had issues with validation. Because of latest change, I had to remove the transport settings completely, and recreate a new one, and refill the data. Remember to choose "SFTP". as the new standard selection is "Custom" which requires a script. Also my top workaround is relevant, because the new default path is Relative to the user, so it tries to create /user/path Test with a root user, as that folder already exists, and has the correct permission.0 -
I'm having the same issue in this thread, I'm trying to configure an Rsync destination to a Hetzner storage box. Rsync works with no issues when running the command manually. However validating my destination is giving me the error Error: Validation for transport "Hetzner Storage Box" failed: Could not upload test file: child exited with code 1. I can see files tmp.test.68555-1543257347 being created in the backup folder. The backup directory value is empty. Does anyone have a working solution to this? I'd like to stick with Rsync so I can use incremental backup. 0 -
Hi, there's no solution for this, yet. The only (useable) possibility for Hetzners storage boxes / backup systems to use incremental rsync backups is to mount the box as CIFS Volume and select that mountpoint as rsync destination. Works pretty good on my current machines. kind regards, Sven B. 8) 0 -
Hello, I don't have new information to report on the status of internal case CPANEL-21358 at this time, however I'll continue to monitor the case and update this thread with more information as it becomes available. Thank you. 0 -
Hi, there's no solution for this, yet. The only (useable) possibility for Hetzners storage boxes / backup systems to use incremental rsync backups is to mount the box as CIFS Volume and select that mountpoint as rsync destination. Works pretty good on my current machines. kind regards, Sven B. 8)
Sven, how did you make it work with CIFS mount? I tried mounting as CIFS, but got lot of mknod failed invalid argument (22) error messages resulting in an incomplete backup. - Michael0 -
Sven, how did you make it work with CIFS mount? I tried mounting as CIFS, but got lot of mknod failed invalid argument (22) error messages resulting in an incomplete backup. - Michael
Hi Michael, I was able to get a CIFS share mounted using systemd on CentOS 7, however in the end I didn't use it as I was getting file/folder permission issues. SSHFS might be a better way to mount the share but I haven't looked into that yet. The steps below should work just fine, this assumes you're using /mnt/backup as the mount point. If it's any different then your systemd script filename needs to be modified. 1. Create /etc/systemd/system/mnt-backup.mount with the contents below: [QUOTE][Unit] Description=Backup Share Mount Script Requires=network.target After=network.service [Mount] What=//u000000.your-storagebox.de/backup Where=/mnt/backup Options=username=u000000,password=XXXXXXXX,cache=none,rw Type=cifs [Install] WantedBy=multi-user.target
2. Run systemctl daemon-reload 3. Run systemctl enable mnt-backup to auto mount on boot 4. Mount the share using systemctl start mnt-backup Best of luck. - Asad0 -
Hi Michael, I was able to get a CIFS share mounted using systemd on CentOS 7, however in the end I didn't use it as I was getting file/folder permission issues. SSHFS might be a better way to mount the share but I haven't looked into that yet. The steps below should work just fine, this assumes you're using /mnt/backup as the mount point. If it's any different then your systemd script filename needs to be modified. 1. Create /etc/systemd/system/mnt-backup.mount with the contents below: 2. Run systemctl daemon-reload 3. Run systemctl enable mnt-backup to auto mount on boot 4. Mount the share using systemctl start mnt-backup Best of luck. - Asad
Hi Asad, I know how to mount things :) My question was not how to Mount volumes, but that CIFS doesn't work with CPanel backup. Also SSHFS doesn't work (maybe issue with symlink support) The only workaround I've got was to mount volume as SSHFS to fx /mnt/storage and then create an image file and format it as ext4 or equivalent and then mount the image as fx /mnt/backup and then run CPanel backup towards that. It works quite well, but it's kind of a "bad" workaround in my opinion. - Michael0 -
Hi Asad, I know how to mount things :) My question was not how to Mount volumes, but that CIFS doesn't work with CPanel backup. Also SSHFS doesn't work (maybe issue with symlink support) The only workaround I've got was to mount volume as SSHFS to fx /mnt/storage and then create an image file and format it as ext4 or equivalent and then mount the image as fx /mnt/backup and then run CPanel backup towards that. It works quite well, but it's kind of a "bad" workaround in my opinion. - Michael
Hey Michael, Sorry, misread what you wrote. I thought about doing something similar with EncFS but I agree, all bad workarounds. I hope cPanel get the bug causing this fixed as I've just left it as it is - incremental backups to a local folder. - Asad0 -
Hello people! Same problem here. Trying to setup remote incremental rsync with Hetzner backup and rsync works manually from a terminal window but not from inside the cpanel remote destination interface. it would be great if you could help figure this out. Thanks Yiannis 0 -
Hello, To update, CPANEL-21358 is still open at this time. I don't have a specific time frame to offer on the solution at this time, but the case is receiving attention due to the amount of customer feedback we've received. I'll update this thread with more information as soon as it's available. Thank you. 0 -
Any updates/timeframe? :-) 0 -
Hello @Michael Svendsen, I don't have a specific time frame to provide at this time. I'll update this thread as soon as a tentative release date is available. Thank you. 0 -
I am also facing the same issue. I could connect the destination manually using the Terminal. Until the fix is released, I will be using FTP or other types. 0 -
I am being affected by this serious bug. Did purchase server with Cpanel using Hetzner provider, and did figure out being able to use incremental / rsync backups over Hetzner Backup Space, which is completelly functional if using rsync manually. This way, my hard drive space available is tight and fixed up to my account requirements. Now, Cpanel is not able to validate a perfectly functional remote rsync source, although using it to copy temp files inside without problems. "?"? Now I cannot use incremental rsyncs because of this, which will harm me because of CPU/MEM requirements for backups, as well as required space. And discovered Cpanel development team could not fix this bug for months... :( Seriously? Hello @Michael Svendsen, I don't have a specific time frame to provide at this time. I'll update this thread as soon as a tentative release date is available. Thank you.
0 -
Hello @feanorknd, The Rsync backup destination type included with cPanel & WHM requires full SSH access to the remote destination due to the need to execute multiple commands over SSH during the backup transportation stage. For example, directory creation tasks and backup pruning tasks are performed via the execution of the mkdir and rm commands on the remote backup destination over SSH. Backup providers such as Hetzner permit the use of the Rsync command, but they do not permit the execution of additional commands over SSH. CPANEL-21358 remains open to request a feature improvement to the Rsync destination type so that the additional SSH commands are not required, however I don't have a firm time frame to offer on when we might see a change to the current functionality. I'll continue to monitor the case and update this thread with any new information as it becomes available. Thank you. 0 -
Great explanation, sincerely Hope you could improve these in the future. Regards, Hello @feanorknd, The Rsync backup destination type included with cPanel & WHM requires full SSH access to the remote destination due to the need to execute multiple commands over SSH during the backup transportation stage. For example, directory creation tasks and backup pruning tasks are performed via the execution of the mkdir and rm commands on the remote backup destination over SSH. Backup providers such as Hetzner permit the use of the Rsync command, but they do not permit the execution of additional commands over SSH. CPANEL-21358 remains open to request a feature improvement to the Rsync destination type so that the additional SSH commands are not required, however I don't have a firm time frame to offer on when we might see a change to the current functionality. I'll continue to monitor the case and update this thread with any new information as it becomes available. Thank you.
0 -
Also experiencing this issue trying to backup to a Synology NAS. Will follow the thread for future updates. 0 -
Also experiencing this issue trying to backup to a Synology NAS. Will follow the thread for future updates.
I am waiting for an update on this issue too. Has any progress been made?0 -
Watching this with interest, but I'm guessing with the lack of updates a workaround isn't currently possible. 0 -
We are now testing Hetner storage. What does seem to work is if you backup to another vps with root access first, then rsync with the -H hardlinks preserved this works. Then Hetner would be your secondary storage. Also tried backblaze with rclone but doesn't support hardlinks! EDIT: Of course if you use ESXi and have a server with SSD and HDD you could add a backup datastore then rsync from there to hetner for offsite. 0 -
Hello, It's been too long and unfortunately this issue still exists. Any news on that @cPanelMichael ? 0 -
I am also facing the same issue, and I checked this thread and found people are facing it from 2+ years and yet the cpanel didn't solve it. can you tell us any update on this thread? 0
Please sign in to leave a comment.
Comments
50 comments