SFTP Backup stopped working

I have remote backup for WHM 62.0 (build 15) on Amazon S3 and on a Synology NAS using SFTP. Two day ago the SFTP stopped working and now I can access the NAS from another computer but from WHM I can"t validate the destination. This it the log /usr/local/cpanel/logs/cpbackup_transporter.log

[2017-02-20 08:27:11 +0100] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2017-02-20 08:27:11 +0100] info [cpbackup_transporter] Instantiating Object
[2017-02-20 08:30:12 +0100] warn [cpbackup_transporter] Error connecting to DS916+:  Cpanel::Transport::Exception::Network::Connection=HASH(0x3b2af90) at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
    Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x1367a50), "Error connecting to DS916+:  Cpanel::Transport::Exception::Ne"...) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 462
    Cpanel::Backup::Queue::transport_backup::get_ctf_obj(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), HASH(0x3ae2b60)) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 192
    Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), cPanel::TaskQueue::Task=HASH(0x202e670), Cpanel::LoggerAdapter=HASH(0x1367a50)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 586
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 589
    cPanel::TaskQueue::__ANON__() called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    cPanel::StateFile::Guard::call_unlocked(cPanel::StateFile::Guard=HASH(0x3ac1bf8), CODE(0x3a602c0)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 594
    cPanel::TaskQueue::process_next_task(cPanel::TaskQueue=HASH(0x1edd170)) called at /usr/local/cpanel/bin/cpbackup_transporter line 151
    eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 149
[2017-02-20 08:30:12 +0100] info [cpbackup_transporter] Error encountered
[2017-02-20 08:30:12 +0100] info [cpbackup_transporter] Error is an object reference of type ref Cpanel::Transport::Exception::Network::Connection=HASH(0x3b2af90)
[2017-02-20 08:33:12 +0100] warn [cpbackup_transporter] Error connecting to DS916+:  Cpanel::Transport::Exception::Network::Connection=HASH(0x3a592b0) at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
    Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x1367a50), "Error connecting to DS916+:  Cpanel::Transport::Exception::Ne"...) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 462
    Cpanel::Backup::Queue::transport_backup::get_ctf_obj(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), HASH(0x3ae2b60)) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 192
    Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), cPanel::TaskQueue::Task=HASH(0x202e670), Cpanel::LoggerAdapter=HASH(0x1367a50)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 586
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 589
    cPanel::TaskQueue::__ANON__() called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    cPanel::StateFile::Guard::call_unlocked(cPanel::StateFile::Guard=HASH(0x3ac1bf8), CODE(0x3a602c0)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 594
    cPanel::TaskQueue::process_next_task(cPanel::TaskQueue=HASH(0x1edd170)) called at /usr/local/cpanel/bin/cpbackup_transporter line 151
    eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 149
[2017-02-20 08:33:12 +0100] info [cpbackup_transporter] Error encountered
[2017-02-20 08:33:12 +0100] info [cpbackup_transporter] Error is an object reference of type ref Cpanel::Transport::Exception::Network::Connection=HASH(0x3a592b0)
[2017-02-20 08:36:13 +0100] warn [cpbackup_transporter] Error connecting to DS916+:  Cpanel::Transport::Exception::Network::Connection=HASH(0x3afea10) at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
    Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x1367a50), "Error connecting to DS916+:  Cpanel::Transport::Exception::Ne"...) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 462
    Cpanel::Backup::Queue::transport_backup::get_ctf_obj(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), HASH(0x3ae2b60)) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 192
    Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), cPanel::TaskQueue::Task=HASH(0x202e670), Cpanel::LoggerAdapter=HASH(0x1367a50)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 586
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 589
    cPanel::TaskQueue::__ANON__() called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    cPanel::StateFile::Guard::call_unlocked(cPanel::StateFile::Guard=HASH(0x3ac1bf8), CODE(0x3a602c0)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 594
    cPanel::TaskQueue::process_next_task(cPanel::TaskQueue=HASH(0x1edd170)) called at /usr/local/cpanel/bin/cpbackup_transporter line 151
    eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 149
[2017-02-20 08:36:13 +0100] info [cpbackup_transporter] Error encountered
[2017-02-20 08:36:13 +0100] info [cpbackup_transporter] Error is an object reference of type ref Cpanel::Transport::Exception::Network::Connection=HASH(0x3afea10)
[2017-02-20 08:36:13 +0100] warn [cpbackup_transporter] Unable to get connection to transport id:  1_ad9pjISVOz7siCuaawIagq at /usr/local/cpanel/Cpanel/LoggerAdapter.pm line 27.
    Cpanel::LoggerAdapter::warn(Cpanel::LoggerAdapter=HASH(0x1367a50), "Unable to get connection to transport id:  1_ad9pjISVOz7siCua"...) called at /usr/local/cpanel/Cpanel/Backup/Queue.pm line 200
    Cpanel::Backup::Queue::transport_backup::process_task(Cpanel::Backup::Queue::transport_backup=HASH(0x1938c50), cPanel::TaskQueue::Task=HASH(0x202e670), Cpanel::LoggerAdapter=HASH(0x1367a50)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 586
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 589
    cPanel::TaskQueue::__ANON__() called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/StateFile.pm line 238
    cPanel::StateFile::Guard::call_unlocked(cPanel::StateFile::Guard=HASH(0x3ac1bf8), CODE(0x3a602c0)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/cPanel/TaskQueue.pm line 594
    cPanel::TaskQueue::process_next_task(cPanel::TaskQueue=HASH(0x1edd170)) called at /usr/local/cpanel/bin/cpbackup_transporter line 151
    eval {...} called at /usr/local/cpanel/bin/cpbackup_transporter line 149

Where I can find additional informations about the error occurred?

Jcats

February 20, 2017 16:16

Are you able to manually connect using SSH? This would be a good place to start testing.
# sftp user@sftpserver

cPanelMichael

February 20, 2017 17:07

Hello, Please verify you can make SFTP connections from the cPanel server to the remote SFTP server, as mentioned in the previous post. It's possible the port used for SFTP is filtered by a firewall rule on either the cPanel server, or the remote backup server. Thank you.

teo_

February 20, 2017 17:27

Trying a connection I get:


sftp -oPort=22916 bkremoto@bk.MYHOST.com
Connecting to bk.MYHOST.com...
ssh: connect to host bk.MYHOST.com port 22916: Connection timed out
Couldn't read packet: Connection reset by peer

And if I look at my router port forwarding rules stat and firewall router just after the command, the couter stats grows.

February 20, 2017 17:40

Do you have a firewall on your cPanel server? Try whitelisting your SFTP server there first, if it persists, then the issue is most likely the SFTP server. Do you have access to another server that you can try and access the SFTP server from?

February 20, 2017 19:58

Thank, I tried but without success: another CentOS with WHM gives the same result. If I connect from macOS I get:

sftp -v -oPort=22916 bkremoto@bk.MYHOST.com
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /Volumes/SSD1TB/Users/matteo/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to bk.MYHOST.com [MYIP] port 22916.
debug1: Connection established.
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Volumes/SSD1TB/Users/matteo/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.8p1-hpn14v6
debug1: match: OpenSSH_6.8p1-hpn14v6 pat OpenSSH* compat 0x04000000
debug1: Authenticating to bk.MYHOST.com:22916 as 'bkremoto'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC:  compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC:  compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:kvgIIYf/+6q3hDZF+nH298RvepX9ISHiDhQHa64flWQ
debug1: Host '[bk.MYHOST.com]:22916' is known and matches the ECDSA host key.
debug1: Found key in /Volumes/SSD1TB/Users/matteo/.ssh/known_hosts:75
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Volumes/SSD1TB/Users/matteo/.ssh/id_rsa
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Trying private key: /Volumes/SSD1TB/Users/matteo/.ssh/id_dsa
debug1: Trying private key: /Volumes/SSD1TB/Users/matteo/.ssh/id_ecdsa
debug1: Trying private key: /Volumes/SSD1TB/Users/matteo/.ssh/id_ed25519
debug1: Next authentication method: keyboard-interactive
Password:
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to bk.MYHOST.com ([MYIP]:22916).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Sending environment.
debug1: Sending env LANG = it_IT.UTF-8
debug1: Sending subsystem: sftp
Connected to bk.MYHOST.com.
sftp>

from CentOS where I have WHM I get:

sftp -vvv -oPort=22916 bkremoto@bk.MYHOST.com
Connecting to bk.MYHOST.com...
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to bk.MYHOST.com [MYIP] port 22916.
debug1: connect to address MYIP port 22916: Connection timed out
ssh: connect to host bk.MYHOST.com port 22916: Connection timed out
Couldn't read packet: Connection reset by peer

February 20, 2017 20:26

Assuming both of those WHM servers have outgoing port 22916 opened or you have the SFTP server IP white listed, I would say the issue is on the SFTP server.

February 21, 2017 12:58

Hello, You'll need to verify if any firewall/network rules on the SFTP server are blocking the connection, as the issue looks to stem from a connection issue as opposed to a problem with the cPanel/WHM software. Thank you.

February 21, 2017 17:26

You'll need to verify if any firewall/network rules on the SFTP server are blocking the connection, as the issue looks to stem from a connection

For the last tests I"ve temporary set the firewall in front of the WHM/cPanel machine to accept all and the firewall in front of the SFTP server to accept all traffic of all protocols from the WHM/cPanel machine IP. I"ve also tried to create a new instance of the server from scratch, but with the same results. I don"t think it is a firewall related issue because other SFTP client can connect to both SFTP" May be some incompatibility between SFTP client/server or something related to certificate management?

February 21, 2017 17:49

This is sort of one of those things where its hard to assist further without checking personally. I would maybe open a ticket up with cPanel and at least have them confirm the issue is/isn't on the cPanel server but the error is pretty clear "Connection timed out", if it was SSL related we would see different output. Connection timeout means its sitting there waiting for a reply back from the server and its just not happening. What if you change the SSH port on the other cPanel server you have access to, to 22916 and see if you can ssh/sftp from the one cPanel server to the other.

February 21, 2017 17:50

Hello, Feel free to open a support ticket using the link in my signature so we can take a closer look. You can post the ticket number here so we can update this thread with the outcome. Thank you.

SFTP Backup stopped working

Comments

Didn't find what you were looking for?