Improve reboot time?
Hi everyone,
Looking for some insight into my reboot cycle and timings
I have my own dedicated box hosted in a data centre in UK, its a Xeon E3-1270 V2 @ 3.50GHz (8 core) with 32GB RAM and 500GB 7200RPM SATA HD (Not SSD)
The rig itself I believe is a Dell PowerEdge R210 II
It runs Centos 7.4 -
######### 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
(I am aware this is not the latest kernel, the latest one had to be rolled back as it broke the server and failed to boot for some reason)
When issuing a reboot it can take around 5 minutes sometimes more before the system will respond, i.e. allow SSH login or load WHM page. Which for me seems excessive especially as the server is not actually all that busy just now and has not been active that long.
Anyway I will try and pre-empt the first requests for info:
[root@# ~]# systemd-analyze
Startup finished in 1.567s (kernel) + 11.180s (initrd) + 1min 47.671s (userspace) = 2min 419ms
[root@# ~]# systemd-analyze blame
1min 25.734s mysql.service
1min 19.836s cpanel-dovecot-solr.service
30.494s clamd.service
28.692s spamd.service
28.515s ea-php70-php-fpm.service
14.512s cpanel.service
8.801s httpd.service
8.045s cpanel_php_fpm.service
7.713s lvm2-monitor.service
6.938s cpdavd.service
6.847s csf.service
6.146s network.service
5.534s dev-mapper-centos\x2droot.device
5.485s securetmp.service
5.228s tailwatchd.service
4.616s maldet.service
3.628s systemd-hwdb-update.service
2.809s boot.mount
2.368s fastmail.service
2.108s lvm2-pvscan@8:2.service
2.106s named.service
1.901s mailman.service
1.640s systemd-udevd.service
I understand the OS is supposeduly loaded and ready in around 2 mins however I assume the hardware / kernel stage is taking a little longer.
mysql may be taking a while due to my my.cnf settings an excerpt is below I suspect the 1min 25 for sql service may be due to the "GB buffer pool? :
default-storage-engine = InnoDB
#innodb_file_format = BARRACUDA # deprecated
#innodb_file_format_max = BARRACUDA # deprecated
innodb-buffer-pool-instances = 2
innodb-buffer-pool-size = 2G
innodb-file-per-table = 1
innodb-flush-log-at-trx-commit = 0
innodb-flush-method = O_DIRECT
innodb-log-buffer-size = 64M
innodb-log-file-size = 512M
innodb_compression_algorithm = zlib
innodb_default_row_format = Dynamic
innodb-thread-concurrency = 6
thread_concurrency = 6
innodb_fast_shutdown = 1
Anyway can someone advise if there is anything I can do to improve boot times without the obvious things like "buy a better rig" or "upgrade to SSD" The hardware is fixed and not changeable (unless it breaks!)
Thanks in advance!
p.s. I uploaded a dmesg output aswell
-
You could try Kexec [LIST] - Kexec is a system which allows a system to be rebooted without going through BIOS. That is, a Linux kernel can directly boot into another Linux kernel, without going through firmware.
0 -
Hi Greg, You could try tuning your MySQL configuration, but generally the time for services to start after a reboot is related to the hardware you are using. Or, if you are interested in reducing the frequency of reboot requirements, you may want to consider a third-party application such as KernelCare: CloudLinux - Main | New template Thank you. 0 -
Hi UHLHosting & Michael, Thank you both for the replies. I have considered KernelCare in the past but it's not something I am going to go ahead with at this time for different reasons but still thank you for the suggestion. I haven't really looked into Kexec in great detail before however initial information does seem like it may be quite useful Part of the issue is that I like to reboot the server at least 1ce a month (one of the reasons for not using kernelcare) 0 -
If you have KVM/IP on your server (or can get it connected) then you'll find that a chunk of the reboot time is the server BIOS & devices initialising (network, RAID controller, etc). You can speed up this by doing things such as disabling BIOS memory testing, checking PXE boot is disabled (this is commonly enabled as default on a Dell and it will sit there for about 30 seconds looking for a DHCP server), checking the boot order, etc - these will all improve the time for reboots. 0 -
Hi PenguinInternet, Thanks for the reply, great insight. I can request KVMoIP access but there is a delay at the DC end and I believe they limit access/number of uses (unless I wish to buy one from them ;) ) so I have been avoiding it for as long as possible unless there is a failure etc, that being said I will clearly need to take a look at the bios settings as you have suggested so I will probably try and schedule one for the weekend and plan some downtime. I do believe you are also correct in regards to PXE on Dells. I have a repurposed dell tower set up as a local server in the office and it also used to have PXE enabled by default but that thought never occurred to me for the server I just thought it was a workstation feature. Anyway I will see if I can find a bios demo or image I can mount for the server so I can familiarise myself with it so I don't have too much downtime Thanks again 0 -
PXE can be useful for servers for performing OS installs where you can boot from a separate environment to remotely install. Generally though, it just slows the reboot down so if you're not using it then definitely disable it. If you get stuck, give me a shout - I can probably give you access to an R210 over KVM/IP so you can remotely have a play with it before doing it on your production server, or otherwise guide you through the settings on one by sharing my desktop. 0
Please sign in to leave a comment.
Comments
6 comments