Skip to main content

MySql restarts every 6 days exactly at 12:03 pm

Comments

5 comments

  • rbairwell

    I assume you've checked /var/log/messages and Apache's logs (in case there is high demand at that time)?

    I also take it that network activity is nominal/normal, no backups are running at the time, no recent crons have been kicked off (/var/log/cron) etc? You may need to "backtrack" in the logs around 30 minutes before the event as MySQL "issues" do tend to have an initial stage which then grows into the issue.

    Can you try manually restarting MySQL via WHM/command line after 5 days (i.e. nearly a day before it "auto-restarts") and see if the problem reoccurs at the same point in time (i.e. it restarts 22 hours later) or after the same sort of time delay (i.e. it auto-restarts 5hrs 22days later), It might give a clue as to what is causing it.

    1
  • MiloÅ¡ Srnić

    Hi, yes i checked everything you mentioned. My first thought was it was backup or cron, but nothing in that time frame is happening. 
    I restarted mysql manually, and even the whole server, and mysql restarted 6 days later in both cases. 
    The weird thing is that this server had a lot more traffic months ago, and we moved bunch of databases to other servers etc. When it was under heavy load, mysql was working fine. 
    I am looking at the mysql logs but all i have 30,40 minutes before is some random user denied. 
    Every time i check it has bunch of memory waiting to be called, i am thinking that maybe there is some bug with releasing standby ram memory.

    0
  • rbairwell

    If it was memory, there should be an out of memory message in dmesg ( dmesg | grep -i oom ) - and the fact you are only occasionally seeing that does make it appear deceiving - and with the logs not showing any other activity, it is a puzzler.

    Could you post your current /etc/my.cnf file just in case (feel free to "anonymise" server hostname etc) - just to see if there is anything unexpected set there which might be causing it.

    You might get some luck running something like MySQLTuner :

    mkdir /root/mysqltuner
    cd /root/mysqltuner
    wget https://github.com/major/MySQLTuner-perl/archive/refs/tags/v2.2.12.tar.gz
    tar -xvf v2.2.12.tar.gz
    cd MySQLTuner-perl-2.2.12
    perl mysqltuner.pl

    It should pickup the root MySQL login credentials automatically. Things like the MyISAM metrics ("InnoDB migration request for...") can be safely ignored. I wouldn't actually make any of the recommended changes at this point, but it might highlight things to check/investigate.

    Percona's toolkit is another group of handy tools to look and perhaps try: pt-mysql-summary ( wget percona.com/get/pt-mysql-summary ; bash ./pt-mysql-summary ) should give a good overview and pt-stalk looks like it might help "capture" data about when MySQL dies - however, I've not tried it myself so I can't comment on how easy/hard/useful it is.

    0
  • MiloÅ¡ Srnić

    Okay, i went through every log again. Checked mysql, one to two hours before the last 2 restarts, checked exim, apache logs. Nothing.
    This is my my.cnf file:

    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid
    innodb_buffer_pool_size=10485760000
    max_allowed_packet=268435456
    open_files_limit=80000
    default-storage-engine=MyISAM
    innodb_file_per_table=1
    sql_mode=""
    max_connections=500

    I don't see anything wrong with the setting, i have other servers with 2,3 times the load and the same settings, and they are doing fine. 
    Cpanel won't even send me an email before it goes out of memory. I received the email only once, and i had like 15 restarts. I just get the mysql failed email.

    0
  • cPRex Jurassic Moderator

    It sounds like it's time to create a ticket on this one, so either us or your host can take a look.

    0

Please sign in to leave a comment.