SQL Server crashing and restarting, log file attached, what might be the solution?

maestroc

• June 10, 2014 10:14

The MySQL server on my CentOS cpanel account keeps crashing/restarting at random intervals. The most it ever seems to run is for a day or two. Pasted below is the tail of my sql error log. Anyone have any idea of what I need to adjust or try to repair in order to get the MySQL server to run reliably? As you can see from the first few lines I have had issues occasionally with crashed tables in various databases. I go in through phpmyadmin and repair them but then they crash again... Any help would be greatly appreciated. As you can tell I am not incredibly experienced in this area so please speak slowly for me. :) 140605 14:49:54 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:49:54 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:00 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:01 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:02 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:04 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:05 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:06 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:23 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:23 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:24 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:35 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:38 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:42 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140605 14:50:43 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and should be repaired 140606 04:19:09 mysqld_safe Number of processes running now: 0 140606 04:19:09 mysqld_safe mysqld restarted 140606 4:19:09 [Note] Plugin 'FEDERATED' is disabled. 140606 4:19:09 InnoDB: Initializing buffer pool, size = 110.0M 140606 4:19:09 InnoDB: Completed initialization of buffer pool InnoDB: Log scan progressed past the checkpoint lsn 6 576517682 140606 4:19:09 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 6 576538928 140606 4:19:10 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed 140606 4:19:10 InnoDB: Started; log sequence number 6 576538928 140606 4:19:10 [Note] Event Scheduler: Loaded 0 events 140606 4:19:10 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.1.73-cll' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPLv2) 140607 01:03:18 mysqld_safe Number of processes running now: 0 140607 01:03:18 mysqld_safe mysqld restarted 140607 1:03:18 [Note] Plugin 'FEDERATED' is disabled. 140607 1:03:18 InnoDB: Initializing buffer pool, size = 110.0M 140607 1:03:18 InnoDB: Completed initialization of buffer pool InnoDB: Log scan progressed past the checkpoint lsn 6 821854091 140607 1:03:18 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 6 821916080 140607 1:03:18 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed 140607 1:03:19 InnoDB: Started; log sequence number 6 821916080 140607 1:03:19 [Note] Event Scheduler: Loaded 0 events 140607 1:03:19 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.1.73-cll' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPLv2) 140608 21:47:33 mysqld_safe Number of processes running now: 0 140608 21:47:33 mysqld_safe mysqld restarted 140608 21:47:33 [Note] Plugin 'FEDERATED' is disabled. 140608 21:47:33 InnoDB: Initializing buffer pool, size = 110.0M 140608 21:47:33 InnoDB: Completed initialization of buffer pool InnoDB: Log scan progressed past the checkpoint lsn 6 1207548833 140608 21:47:33 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 6 1207579994 InnoDB: 8 transaction(s) which must be rolled back or cleaned up InnoDB: in total 9 row operations to undo InnoDB: Trx id counter is 0 35638528 140608 21:47:33 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting in background the rollback of uncommitted transactions 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638155, 1 rows to undo 140608 21:47:34 InnoDB: Started; log sequence number 6 1207579994 InnoDB: Rolling back of trx id 0 35638155 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638153, 1 rows to undo InnoDB: Rolling back of trx id 0 35638153 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638136, 1 rows to undo InnoDB: Rolling back of trx id 0 35638136 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638135, 1 rows to undo InnoDB: Rolling back of trx id 0 35638135 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638134, 1 rows to undo InnoDB: Rolling back of trx id 0 35638134 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638123, 2 rows to undo InnoDB: Rolling back of trx id 0 35638123 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638116, 1 rows to undo InnoDB: Rolling back of trx id 0 35638116 completed 140608 21:47:34 InnoDB: Rolling back trx with id 0 35638115, 1 rows to undo InnoDB: Rolling back of trx id 0 35638115 completed 140608 21:47:34 InnoDB: Rollback of non-prepared transactions completed 140608 21:47:34 [Note] Event Scheduler: Loaded 0 events 140608 21:47:34 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.1.73-cll' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPLv2)

• 

  cPanelMichael

  • June 10, 2014 18:00

  Hello :) It looks like there is InnoDB corruption based on the error output. Could you post the entries added to your /etc/my.cnf file? Thank you.

  0
• 

  maestroc

  • June 10, 2014 19:12

  [quote="cPanelMichael, post: 1661722">Hello :) It looks like there is InnoDB corruption based on the error output. Could you post the entries added to your /etc/my.cnf file? Thank you.
  Here it is: [mysqld] innodb_file_per_table=1 local-infile=0 open_files_limit=22164 query_cache_type=1 query_cache_size=90M query_cache_limit=100M tmp_table_size=156M max_heap_table_size=156M thread_cache_size=16K key_buffer_size=64M join_buffer_size=900K max_connections=200 innodb_buffer_pool_size=110M table_cache=3K table_definition_cache=4K
  

  0
• 

  cPanelMichael

  • June 10, 2014 19:36

  The following document is a good place to start when handling InnoDB corruption: [url=http://dev.mysql.com/doc/refman/5.5/en/innodb-backup.html]MySQL :: MySQL 5.5 Reference Manual :: 14.19 InnoDB Backup and Recovery Thank you.

  0
• 

  maestroc

  • June 10, 2014 19:48

  So it sounds like I need to drop and recreate that database from a backup. If this was for a normal account I would know how to do that with no problem (I've done it before, deleting an account then restoring it from backup), but since this is in reference to the mailserver tables I am not familiar with how to do that part of things. Can it be done through WHM as I have done with the others? Also will doing so screw up the way mail is processed while the deletion/restore process is carried out? Like I said, I'm new to this part of server management...

  0
• 

  cPanelMichael

  • June 10, 2014 19:54

  The references in the error logs to the eximstats database is not related to the InnoDB corruption. Those are separate, unrelated, error messages in the MySQL error log. Thank you.

  0
• 

  maestroc

  • June 10, 2014 20:53

  Like I said, I am kind of clueless here. In the help file you referenced it talks like the only way to fix the corruption is to restore the database from a backup. If the eximstats db is not the culprit how do I figure out what is doing it or what should my steps be to fixing the issue?

  0
• 

  cPanelMichael

  • June 11, 2014 14:35

  I don't recommend attempting InnoDB recovery if you are not comfortable doing so. Instead, you may want to consult with a qualified system administrator for assistance with this particular issue. Thank you.

  0

Please sign in to leave a comment.