Transfer Tool processes get stuck at the end of a transfer due to an open connection to the source in a CLOSE_WAIT state

Symptoms

While using the transfer tool, a process gets stuck after seeming to complete a user's migration. This will either prevent the transfer from reaching 100%, or prevent it from resuming the other users in the transfer. No other errors will be logged in the transfer's logs.

Description 

This will happen if the transfer process is stuck waiting for an open connection to close. The stuck connection prevents the transfer process from completing and marking the user transfer as finished.

We've opened an internal case for our development team to investigate this further. For reference, the case number is CPANEL-46206. Follow this article to receive an email notification when a solution is published in the product. 

Workaround

The processes can be safely killed from the command line or WHM:

1. Use "ps" to identify the transfer processes that are running on the server:

   [root@server]# ps faux | grep "transfer_session"
   root 6545 0.0 0.0 157228 12472 ? S 00:30 0:05 transfer_session - [transfer session ID] - MASTER
   root 6550 0.1 1.1 434664 182528 ? Sl 00:30 0:20 \_ transfer_session - [transfer session ID] - RESTORE:1

2. Before killing the process, we should make sure that the process is stuck waiting for a closing connection. We can list the open files and sockets with the "lsof" command, and the process ID (PID) of the "RESTORE" process from the last step:

   [root@server]# lsof -p 6550 | grep TCP
   transfer_ 6550 root 21u sock 0,9 0t0 324088 protocol: TCP
   transfer_ 6550 root 22u IPv4 94104 0t0 TCP [source server]->[destination] (CLOSE_WAIT)

3. If the process is stick waiting on a connection to close as illustrated above, we can kill the process using the same PID:

   kill 6550

4. If you have any users that were not transferred, you can launch a new session using the Transfer Tool for only those users.