CPANEL-40457 - Restarting cpbackup_transporter skips the current backup file
I occasionally see an issue where uploads to Backblaze slow or perhaps hang. I then restart the process with cpbackup_transporter --stop and then --start. It would appear that the shutdown process marks the file in progress done, because when it starts back up it grabs the next file.
[2022-04-11 15:29:56 -0400] info [cpbackup_transporter] Uploading account backup /backup/2022-04-11/accounts/account1.tar.gz to 2022-04-11/accounts/account1.tar.gz (from 2022-04-11/accounts/account1.tar.gz)
[2022-04-11 15:29:56 -0400] info [cpbackup_transporter] Attempting to upload /backup/2022-04-11/accounts/account1.tar.gz to 2022-04-11/accounts/account1.tar.gz for destination: Backblaze-CP
[2022-04-11 15:29:56 -0400] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-04-11/accounts/account1.tar.gz to 2022-04-11/accounts/account1.tar.gz for destination: Backblaze-CP
# /usr/local/cpanel/bin/cpbackup_transporter --stop
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] Initializing log file
Graceful shutdown of cPanel Backup Transporter Queue Daemon requested.
Graceful shutdown due to SIGHUP 34064
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] Graceful shutdown due to SIGHUP
Shutdown complete
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] cpbackup_transporter - started
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] Instantiating Object
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] Starting a "copy" operation on the "Backblaze-CP" destination ID "".
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] Validating destination path 2022-04-11/accounts/
[2022-04-11 16:45:40 -0400] info [cpbackup_transporter] Path exists
[2022-04-11 16:45:40 -0400] info [cpbackup_transporter] Uploading account backup /backup/2022-04-11/accounts/account2.tar.gz to 2022-04-11/accounts/account2.tar.gz (from 2022-04-11/accounts/account2.tar.gz)
When the transport finishes the skipped files will still exist in the backup directory and are not present in the Backblaze bucket.
cpanel_and_whm: 11.102.0
operating_system_name: centos
operating_system_version: 7
-
Hey there! Do you know if you are using the B2 or S3 protocol? We see *much* better results when using the "S3Compatible" transport in the backup system than the B2. 0 -
We are using the B2 protocol. I'll flip it to S3 to see about fixing the periodic glitches, but the skipping files bit would likely still be there. 0 -
If you can try that and then report back about any lingering issues I can make a case from there. 0 -
I made the switch last night and the troubles continue. I'm investigating the throughput issues with my provider (I can get 30 MB/s for a while and then it throttles back to a couple hundred KB/s), but the skipping backup bit is still the case. A backup being transported gets "lost" when cpbackup_transporter is stopped and restarted. 0 -
We have also seen this behaviour of losing the backup that was in progress when you stop and restart the cpbackup_transporter process. Again this was with Backblaze B2 using the B2 protocol. (It is not easy for us to swap protocols, but the bucket is not S3 enabled.) 0 -
@AGY or @phil99 - could you submit a ticket to our team so we can see this on a server experiencing the issue? 0 -
Submitted #94436570. 0 -
Thanks for that - I'm following along with that ticket on my end now. 0
Please sign in to leave a comment.
Comments
9 comments