How To: Manual Account Transfer With Rsync

Introduction

Why would I need to perform a manual transfer?
If you have ever needed to transfer cPanel accounts from one server to another, you know this can be a stressful task to take on. With the introduction of our Transfer Tool, we tried to streamline this process to take the burden of work off of you, and in almost every case, this is the recommended way to transfer accounts from one cPanel server to another. However, when it comes to accounts that have a large amount of disk space, large databases, or a general lack of free disk space on the source server, using the transfer tool may not be possible or optimal. Fear not though, as hope is not lost! Our pkgacct script has a number of options that can help assist in just such a case. Using this in conjunction with the rsync command, we can bypass the need to make an entire backup on the source server as well as speed up the transfer of large amounts of data.

Procedure

Special Note
To simplify the steps that are being taken, there are a few terms to define before we get started. Going forward, we will refer to the server you are migrating FROM as the 'Source' server, and the server you are migrating TO as the 'Destination' server.

Setup
To begin, please review the following checklist:

1. Establish an SSH connection on the Source server as the 'root' user.

2. Establish a second SSH connection on the Destination server as the 'root' user.

3. (Optional) Install the 'screen' command on both servers. If you do not have the screen utility installed, a brief article on how to install and use it can be found here. Screen is recommended to ensure the completion of tasks in case of any disconnects. A full usage of the command and installing it extends beyond the scope of this article.

4. To determine the home directory path for the user you would like to transfer, run the following command on the Source server:

Code:

awk -F ':' '/cptech/ {print $6}' /etc/passwd

You can see this command in action in our example below:

Code:

awk -F ':' '/cptech/ {print $6}' /etc/passwd
/home/cptech

Please note that you will need to replace 'cptech' in the aforementioned commands with the cPanel username you are trying to transfer. Make a note of this path as we will be needing it later!

Warnings
1. The rsync command examples that are provided within this article will need to be tailored to your specific situation. We will be covering examples, and providing methods of identifying the information that needs to be added to the commands, but it is always best to have a fundamental awareness of the command that is being run and exactly what it does.

2. While rsync is a fast and extraordinarily versatile file copying tool, it can be quite unforgiving. It will replace files that if they already exist without a prompt or warning, and a single missing '/' (trailing slash) from the command could have catastrophic consequences. Please keep this in mind when attempting this type of transfer, and always make sure to check and re-check any commands before running them.

Getting started
For each example in the following sections, we will be using 'cptech' as our cPanel username, 'cptech.test' as our domain name, '192.168.0.1' as the source server IP address and '10.0.10.1' as the destination server IP address. These will need to be replaced by your username, source server IP address and destination server IP address.

Section 1. Perform the following steps on the source server:

(Optional) Establish a screen session, this can be accomplished with this command:
Code:
```
screen -S transfer
```
The '-S' flag allows us to name our screen session. 'transfer' in the above command can be substituted with anything that will help you or other users that have access to your server denote what the purpose of the screen session is. If at any point during the following steps, you would like to exit the screen session, but keep the current task running (pkgacct, rsync etc.), you may use the key combination - ctrl + a + d - which should result in output similar to this:
Code:
```
[detached from 109728.transfer]
```
If you would like to terminate the screen without continuing any processes that are currently running, you will want to use this key combination - ctrl + d - which should result in output similar to this:
Code:
```
[screen is terminating]
```
This means that your screen session has ended and any process that was running in there has was also terminated. Reconnecting to a screen you have disconnected from can be accomplished by the following commands:
Obtain a list of the currently open screen sessions on the server:
Code:
```
[root@74 source-server]# screen -ls
There is a screen on:
    109728.transfer    (Detached)
1 Socket in /var/run/screen/S-root.
```
Reattach to the screen using the name you provided after the '-S' flag when the screen was opened (We used 'transfer' as the name of our screen in this case):
Code:
```
screen -r transfer
```
The first step we will take is to setup a 'working directory' to keep our work separate from any critical system or site files. We will do this by running the following command:
Code:
```
mkdir -pv /root/site-migration/source-server && cd /root/site-migration/source-server && pwd
```
This will create the directories 'site-migration' and 'source-server' inside of your /root directory, change your current working directory to - /root/site-migration/source-server - and then print your current working directory which should result in output similar to this:
Code:
```
[root@72 ~]# mkdir -pv /root/site-migration/source-server && cd /root/site-migration/source-server && pwd
mkdir: created directory ‘/root/site-migration’
mkdir: created directory ‘/root/site-migration/source-server’
/root/site-migration/source-server
```
Determine the cPanel username of the account you would like to transfer. If you are unsure of this username, you may use the following command to find it, where you replace 'cptech.test' with the domain name of the account you are transferring. Example:
Code:
```
grep '^cptech.test' /etc/userdomains | cut -d ' ' -f2
```
This will result in just the username for that domain being printed, you can see this in action as an example below:
Code:
```
[root@74 source-server]# grep '^cptech.test' /etc/userdomains | cut -d ' ' -f2
cptech
```
Once the username has been determined, we can use our pkgacct script to make a backup of the account using the '--skiphomedir' option, which is short for 'skip home directory'. This will backup all of the cPanel specific files, configurations and preferences along with the databases for the specified account. Please note that if have large databases which are preventing you from using the transfer tool, you will also want to use the '--dbbackup=schema' flag for the command (We will cover the additional steps needed in the transfer process denoted by the 'Optional Sections' below). Example:
Code:
```
/usr/local/cpanel/scripts/pkgacct --skiphomedir cptech /root/site-migration/source-server
```
This will start listing output into your terminal as it begins the package account process, however, no further input should be needed until this has been completed. Once completed, output will cease and the last line should be similar to this:
Code:
```
[2018-08-29 02:08:17 -0500] pkgacct completed
```
We will want to list the files and directories to ensure the backup is present which we can use with the command:
Code:
```
ls -lah
```
which should return output similar to this:
Code:
```
[root@72 source-server]# ls -lah
total 8.0K
drwxr-xr-x 2 root root   34 Aug 29 02:11 .
drwxr-xr-x 4 root root   42 Aug 29 02:10 ..
-rw------- 1 root root 6.8K Aug 29 02:11 cpmove-cptech.tar.gz
```
Once we have confirmed the backup is present in this directory, we can move on to the next step.
We will now want to check the integrity of our newly created backup. We can do this using the 'tar' command. This command will ensure the archive is complete, displaying no output if the archive is complete and displaying an error if the archive is invalid. The ideal outcome of this command is no output. If you receive any output at all, this generally means the archive is bad and will need to be re-created. An example of a good archive:
Code:
```
[root@74 source-server]# tar -tzf cpmove-cptech.tar.gz
[root@74 source-server]#
```
An example of a bad archive:
Code:
```
[root@74 site-migration]# tar -tzf cpmove-cptech.tar.gz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
```
Once your newly created archive has passed our integrity check, we can begin the process of transferring it to the destination server.

Section 1A (Optional). Databases too large - Source:

This section is for the corner cases where the databases are too large to be packaged with our package account script which necessitates the use of the '--dbbackup=schema' flag in the fourth step of the above section. Please note that all work in this section will need to take place on the source server.

We will first want to create a directory to place our database files into to keep everything organized, this can be done with the following command (Replace 'cptech' with the user you are transferring):
Code:
```
mysql -se 'show databases like "cptech_%";'
```
Which will result in output similar to this:
Code:
```
[root@74 source-server]# mysql -se 'show databases like "cptech_%";'
Database (cptech_%)
cptech_database1
cptech_second
cptech_third
```
From the output of our command, we can see that the databases present for this user are:
cptech_database1
cptech_second
cptech_third
We now need to export these databases into importable SQL files, this can be accomplished using the mysqldump command. The command will be formatted as such:
Code:
```
mysqldump ${name-of-database} -r ${desired-name-of-sql-file}
```
The '${desired-name-of-the-sql-file}' variable should be replaced by whatever you would like the name of the SQL file to be, followed by '.sql'. For the sake of simplicity, I would highly recommend making the name of the file the exact same as the database and adding '.sql' to the end of it. We can see this in action in our example here:
Code:
```
[root@74 source-server]# mysqldump cptech_database1 -r cptech_database1.sql
[root@74 source-server]#
```
Please note that there will be no output or 'progress bar' of any kind for this command unless an error occurs, so just as in step five above, no output is ideal. To ensure this was created, we can list the files in our current directory once again using the ls command:
Code:
```
[root@74 databases]# ls -lah

total 4.0K
drwxr-xr-x 2 root root   34 Aug 30 03:25 .
drwxr-xr-x 3 root root   51 Aug 30 03:25 ..
-rw-r--r-- 1 root root 1.3K Aug 30 03:22 cptech_database1.sql
```
Please note that depending on the size of the database, this could take some time to complete, but no further input will be needed once the command has been executed.

Section 2. Perform the following steps on the destination server:

(Optional) Use the instructions provided in step number 1 of the 'Getting Started' section to start a screen on this server as well.

Next, we will formulate the rsync command to move the home directory that we excluded from our package account. Please note that we are using 192.168.0.1 as the source server IP address, and 10.0.10.1 as the destination IP address, but these will need to be replaced with the IP addresses of your servers. Example:

Code:

rsync -avPhz root@192.168.0.1:/root/site-migration/source-server /root/site-migration

You may receive a warning regarding the authenticity of the host IP address, and it will prompt you if you would like to continue. You will type 'yes' and will then be prompted for the 'root' user's password on the source server. The full output will look similar to this:

Code:

[root@74 ~]# rsync -avPhz root@192.168.0.1:/root/site-migration/source-server /root/site-migration
The authenticity of host '192.168.0.1' can't be established.
ECDSA key fingerprint is <REDACTED>.
ECDSA key fingerprint is <REDACTED>.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.1' (ECDSA) to the list of known hosts.
root@192.168.0.1's password:
receiving incremental file list
created directory /root/site-migration
source-server/
source-server/cpmove-cptech.tar.gz
          6.91K 100%    6.59MB/s    0:00:00 (xfr#1, to-chk=0/2)

sent 47 bytes  received 7.06K bytes  232.98 bytes/sec
total size is 6.91K  speedup is 0.97

We will now want to ensure this was properly transferred, and the packaged account is now on the destination server. We can accomplish that by using the following:

Code:

cd /root/site-migration/source-server/ && ls -lah

Which should result in output similar to this:

Code:

[root@74 ~]# cd /root/site-migration/source-server/ && ls -lah
total 8.0K
drwxr-xr-x 2 root root   34 Aug 29 02:11 .
drwxr-xr-x 3 root root   27 Aug 29 02:31 ..
-rw------- 1 root root 6.8K Aug 29 02:11 cpmove-cptech.tar.gz

Now that we have confirmed our package account file was transferred over, we will begin the restoration of the package. Luckily, cPanel provides a script to do just that! Example:
Code:
```
/usr/local/cpanel/scripts/restorepkg cpmove-cptech.tar.gz
```
Please make sure to replace 'cpmove-cptech.tar.gz' with your archive name. This will begin filling your screen with output once again, but no further input is needed until this has been completed. The last line will denote its completion and will look similar to this:
Code:
```
[108429][MASTER    ]: Session Complete
```
Once the restoration of our archive has been completed, we will need to determine the user's home directory path on this server. We can do so using the following command:
Code:
```
grep 'cptech' /etc/passwd | awk -F ':' '{print $6}'
```
You can see this command in action in our example below:
Code:
```
[root@74 source-server]# grep 'cptech' /etc/passwd | awk -F ':' '{print $6}'
/home/cptech
```
Please note that you will need to replace 'cptech' in the aforementioned commands with the cPanel username you are trying to transfer.
Now we need to formulate our rsync command to transfer the home directory data from the source server that we left out of our archive. Remember the note we made in our 'setup' section of our user's home directory path on the source server? Now is the time we will be needing it! More often than not, these paths will be the same, but it is always best to be safe and check before-hand. Please substitute '192.168.0.1' for your source server IP address, and substitute '10.0.10.1' for your destination server IP address in the following rsync command, example:
Code:
```
rsync -avPhz 192.168.0.1:/home/cptech/ /home/cptech/
```
***Please make note of the trailing slash (/) following the paths above (/home/cptech/), as this is vital to rsync transferring the files properly.***
You will be prompted for the 'root' user's password on the source server once again, just as you were when we transferred our archive over. Once you have entered that password, your screen will begin filling with output once more. This will likely be the portion of the transfer that will take the most amount of time. The timeframe of this depends on quite a few different factors such as the amount of data being transferred, transfer speeds of each server, bandwidth allocation etc. so there would be no real way to calculate an estimated time of completion. (Now would be the time to use the 'ctrl + a + d' key combination if you opted into using the screen session)

Once the transfer of data has been completed, we will want to do one final check of the files present on the server to ensure everything was completed properly. We can do this by using the 'ls' command and reviewing the file structure of the user's home directory:
Example command:

Code:

ls -lah /home/cptech/

Example output:

Code:

[root@74 source-server]# ls -lah /home/cptech/
total 20K
drwx--x--x 15 cptech cptech 4.0K Aug 29 01:46 .
drwxr-xr-x  7 root   root    127 Aug 29 02:51 ..
lrwxrwxrwx  1 cptech cptech   32 Aug 29 01:46 access-logs -> /etc/apache2/logs/domlogs/cptech
-rw-r--r--  1 cptech cptech   18 Aug 29 01:40 .bash_logout
-rw-r--r--  1 cptech cptech  193 Aug 29 01:40 .bash_profile
-rw-r--r--  1 cptech cptech  231 Aug 29 01:40 .bashrc
drwxr-xr-x  2 cptech cptech    6 Aug 29 02:51 cache
-rw-r-----  1 cptech cptech    0 Aug 29 01:40 .contactemail
drwx------  4 cptech cptech   83 Aug 29 01:40 .cpanel
drwx------  4 cptech cptech   70 Aug 29 01:40 .cphorde
drwxr-x---  3 cptech mail     25 Aug 29 01:40 etc
drwxr-x---  2 cptech cptech    6 Aug 29 01:40 .htpasswds
drwx------  2 cptech cptech    6 Aug 29 01:46 logs
drwxr-x--x 11 cptech cptech  172 Aug 29 01:40 mail
drwxr-x---  3 cptech cptech   22 Aug 29 01:40 public_ftp
drwxr-x---  4 cptech cptech   40 Aug 29 01:41 public_html
drwx------  2 cptech cptech   24 Aug 29 01:40 .spamassassin
-rw-r--r--  1 cptech cptech    0 Aug 29 01:40 .spamassassinenable
drwxr-xr-x  5 cptech cptech   77 Aug 29 01:40 ssl
drwx------  2 cptech cptech   28 Aug 29 02:51 .subaccounts
drwxr-xr-x  2 cptech cptech    6 Aug 29 01:40 tmp
lrwxrwxrwx  1 cptech cptech   11 Aug 29 01:40 www -> public_html
-rw-r--r--  1 cptech cptech  658 Aug 29 01:40 .zshrc

The general outline of the account should be similar to the output provided above, with the addition of any custom folders that may have been added to the user's home directory on the source server.

Section 2A (Optional). Databases too large - Destination:

This is the second section for cases where the database could not be packaged with the account in the source server section above. Please note that all work in this section will need to take place on the destination server.

Our 'databases' folder that we created in the first section should have been transferred when we ran our initial rsync command, so we will want to change our current working directory to that folder.
Code:
```
cd /root/site-migration/source-server/databases/
```
We can now import our transferred SQL files into them. This can be done using the following command, please be sure to replace 'cptech_database1' with your newly created database name and replace 'cptech_database1.sql with the SQL file that relates to that particular database:
Code:
```
[root@74 databases]# mysql cptech_database1 < cptech_database1.sql
[root@74 databases]#
```
Similar to when we used the mysqldump earlier, this has no output or progress bar, but no further input is needed. Repeat these steps as needed to import all of your transferred SQL files.

Closing Notes:
We highly recommend that you double check each website present on the account using something similar to a hosts file change to ensure it is working properly before updating your DNS entries/nameservers. This would fall outside of the scope of this article, but we have some documentation to help you get started which can be found here.

We hope this guide was able to assist you with your transfer and as always, thank you for choosing cPanel!

Articles in this section

Introduction

Procedure

Comments

Articles in this section

Introduction

Procedure

Related articles