Stagger cron jobs as root
I need to set up 17 cron jobs as root, and I need them to run as often as possible (I'm moving an active site to a new server, and I'm copying over user-uploaded files from the old server to the new while waiting for the DNS to propagate).
AFAIK, this should work for one if I put it in /etc/crontabs:
This would make it run every 5 minutes. But is there a way for me to make it run one, wait 1 minute, then run the next? This way, I wouldn't be running two at once, and it would give it 1 minute for the CPU load to go down before moving on to the next. FWIW, this would run for two weeks (max), at which time the old server goes offline permanently. [COLOR="silver">- - - Updated - - - I don't have a lot of SH experience, but would this work?:
Then, since there would be 17 of these with a 30 second pause between (meaning a minimum of 8 1/2 minutes) set the cron to run the shell script every 10 minutes. Is there an easier / better option?
*/5 * * * * root rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir/ /home/new/public_html/dir/
This would make it run every 5 minutes. But is there a way for me to make it run one, wait 1 minute, then run the next? This way, I wouldn't be running two at once, and it would give it 1 minute for the CPU load to go down before moving on to the next. FWIW, this would run for two weeks (max), at which time the old server goes offline permanently. [COLOR="silver">- - - Updated - - - I don't have a lot of SH experience, but would this work?:
#!/bin/sh
rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir1/ /home/new/public_html/dir1/
sleep 30
rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir2/ /home/new/public_html/dir2/
sleep 30
rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir3/ /home/new/public_html/dir3/
sleep 30
# and so on
Then, since there would be 17 of these with a 30 second pause between (meaning a minimum of 8 1/2 minutes) set the cron to run the shell script every 10 minutes. Is there an easier / better option?
-
First of all, don't put it in /etc/crontab - use /var/spool/cron/root or dump it into /etc/cron.d/ folder. You probably don't want to manage the parallelism issue with crond itself - you should put these commands into a script and have the script create a lock file to prevent it from running the same command when another is in progress. This way the script may be executed while another is running, but won't actually do anything. Here's an example, not saying it's the prettiest one out there: #!/bin/bash LOCKFILE=/var/run/myrsync.lock [ -e $LOCKFILE ] && exit 1 # If lock exists, exit touch $LOCKFILE rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir1/ /home/new/public_html/dir1/ rm -f $LOCKFILE0 -
Good plan, thanks Vanessa. Should I still use sleep 30 after each rm command? I'm concerned that sleep might still tie up the CPU (which is exactly what I'm trying to avoid), but I also need to give a break for a few seconds between each rsync to let the CPU load go down. 0 -
While we're on the subject... any suggestion on putting an end date on the cron, so that it doesn't run past the end of the month? I know that I can remove it manually, I'm just wondering if there's a way to make it stop automatically. 0 -
[quote="GoWilkes, post: 1597331">While we're on the subject... any suggestion on putting an end date on the cron, so that it doesn't run past the end of the month? I know that I can remove it manually, I'm just wondering if there's a way to make it stop automatically.
You could create an additional cron job that removes the existing script or cron job on a certain day of the month. Thank you.0
Please sign in to leave a comment.
Comments
4 comments