Couldn't execute 'show fields from `messages`': Unknown storage engine 'InnoDB' (1286)

ttremain

February 28, 2023 01:13

MariaDB 10.2 at this moment. We first noticed message came up in a backup report a couple hundred of these.


[2023-02-26 02:11:15 -0800] mysqldump: Couldn't execute 'show fields from `messages`': Unknown storage engine 'InnoDB' (1286)
[2023-02-26 02:11:15 -0800] mysqldump: Couldn't execute 'show fields from `messages`': Unknown storage engine 'InnoDB' (1286)
[2023-02-26 02:11:15 -0800] mysqldump: Couldn't execute 'show fields from `messages`': Unknown storage engine 'InnoDB' (1286)
[2023-02-26 02:11:15 -0800] mysqldump: Couldn't execute 'show fields from `messages`': Unknown storage engine 'InnoDB' (1286)
[2023-02-26 02:11:15 -0800] mysqldump: Couldn't execute 'show fields from `messages`': Unknown storage engine 'InnoDB' (1286)

Then tonight I was having trouble with some queries, and accessing table in phpMyAdmin with what I believe was also: "Unknown storage engine 'InnoDB'" some of /var/lib/mysql/---HOSTNAME---.err


2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Uses event mutexes
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Compressed tables use zlib 1.2.7
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Using Linux native AIO
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Number of pools: 1
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Using SSE2 crc32 instructions
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2023-02-27 22:09:06 139746927843520 [ERROR] InnoDB: mmap(137232384 bytes) failed; errno 12
2023-02-27 22:09:06 139746927843520 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
2023-02-27 22:09:06 139746927843520 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2023-02-27 22:09:06 139746927843520 [Note] InnoDB: Starting shutdown...
2023-02-27 22:09:06 139746927843520 [ERROR] Plugin 'InnoDB' init function returned error.
2023-02-27 22:09:06 139746927843520 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2023-02-27 22:09:06 139746927843520 [ERROR] mysqld: Out of memory (Needed 128663552 bytes)
2023-02-27 22:09:06 139746927843520 [Note] Plugin 'FEEDBACK' is disabled.
2023-02-27 22:09:06 139746927843520 [Note] Server socket created on IP: '::'.
2023-02-27 22:09:06 139746926884608 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1286: Unknown storage engine 'InnoDB'
2023-02-27 22:09:06 139746927843520 [Note] Reading of all Master_info entries succeeded
2023-02-27 22:09:06 139746927843520 [Note] Added new Master_info '' to hash table
2023-02-27 22:09:06 139746927843520 [Note] /usr/sbin/mysqld: ready for connections.

A few things here imply memory, but I'd like to find what I should specifically check. Please advise!

Comments

17 comments

andrew.n

February 28, 2023 09:04
Check your available memory with "free -m" command and set the buffer size in /etc/my.cnf accordingly: innodb_buffer_pool_size=XXXM
0
cPRex Jurassic Moderator

February 28, 2023 19:08
I agree with @andrew.n - it looks like your MySQL server is running out of memory before it fully restarts.
0
ttremain

March 06, 2023 20:55
Do you see anything new now? Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-06 12:18:05 140649880459008 [Note] InnoDB: Buffer pool(s) load completed at 230306 12:18:05 2023-03-06 13:11:59 140027518089408 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-06 13:11:59 140027518089408 [Note] InnoDB: Uses event mutexes 2023-03-06 13:11:59 140027518089408 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-06 13:11:59 140027518089408 [Note] InnoDB: Using Linux native AIO 2023-03-06 13:11:59 140027518089408 [Note] InnoDB: Number of pools: 1 2023-03-06 13:11:59 140027518089408 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-06 13:12:00 140027518089408 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-06 13:12:00 140027518089408 [Note] InnoDB: Completed initialization of buffer pool 2023-03-06 13:12:00 140018012681984 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-06 13:12:00 140027518089408 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-06 13:12:00 140027518089408 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1343910767 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: 128 out of 128 rollback segments are active. 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: Waiting for purge to start 2023-03-06 13:12:01 140027518089408 [Note] InnoDB: 5.7.38 started; log sequence number 1343910776 2023-03-06 13:12:01 140018238326528 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2023-03-06 13:12:01 140027518089408 [Note] Plugin 'FEEDBACK' is disabled. 2023-03-06 13:12:01 140027518089408 [Note] Recovering after a crash using tc.log 2023-03-06 13:12:01 140027518089408 [Note] Starting crash recovery... 2023-03-06 13:12:01 140027518089408 [Note] Crash recovery finished. 2023-03-06 13:12:01 140027518089408 [Note] Server socket created on IP: '::'. 2023-03-06 13:12:02 140027518089408 [Note] Reading of all Master_info entries succeeded 2023-03-06 13:12:02 140027518089408 [Note] Added new Master_info '' to hash table 2023-03-06 13:12:02 140027518089408 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-06 13:12:03 140018238326528 [Note] InnoDB: Buffer pool(s) load completed at 230306 13:12:03 2023-03-06 13:12:06 140018188326656 [ERROR] mysqld: Table './nodyss5_southrivercondo/visitors' is marked as crashed and should be repaired 2023-03-06 13:12:06 140018188326656 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-06 13:12:06 140018188326656 [Warning] Checking table: './nodyss5_southrivercondo/visitors' 2023-03-06 13:12:07 140018188633856 [ERROR] mysqld: Table './nodyss5_lexingtonplaceorlando/visitors' is marked as crashed and should be repaired 2023-03-06 13:12:07 140018188633856 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-06 13:12:07 140018188633856 [Warning] Checking table: './nodyss5_lexingtonplaceorlando/visitors' 2023-03-06 13:13:31 140018188019456 [ERROR] mysqld: Table './nodyss5_southseaswest/users' is marked as crashed and should be repaired 2023-03-06 13:13:31 140018188019456 [ERROR] mysqld: Table 'users' is marked as crashed and should be repaired 2023-03-06 13:13:31 140018188019456 [Warning] Checking table: './nodyss5_southseaswest/users' 2023-03-06 13:13:31 140018187405056 [ERROR] mysqld: Table './nodyss5_southseaswest/visitors' is marked as crashed and should be repaired 2023-03-06 13:13:31 140018187405056 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-06 13:13:31 140018187405056 [Warning] Checking table: './nodyss5_southseaswest/visitors' 2023-03-06 13:17:36 140018188633856 [ERROR] mysqld: Table './nodyss5_myccstuart/visitors' is marked as crashed and should be repaired 2023-03-06 13:17:36 140018188633856 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-06 13:17:36 140018188633856 [Warning] Checking table: './nodyss5_myccstuart/visitors' 2023-03-06 13:20:21 140018188019456 [ERROR] mysqld: Table './nodyss5_1800atlantickw/visitors' is marked as crashed and should be repaired 2023-03-06 13:20:21 140018188019456 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-06 13:20:21 140018188019456 [Warning] Checking table: './nodyss5_1800atlantickw/visitors' 2023-03-06 13:21:29 140018188019456 [ERROR] mysqld: Table './nodyss5_whisperwalkgreenleaf/visitors' is marked as crashed and should be repaired 2023-03-06 13:21:29 140018188019456 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-06 13:21:29 140018188019456 [Warning] Checking table: './nodyss5_whisperwalkgreenleaf/visitors'
0
cPRex Jurassic Moderator

March 06, 2023 21:20
Thanks for posting that. While I don't see the memory error any longer, it does look like there are several crashed InnoDB tables. While cPanel support can't help you fix those, we have some helpful details on dealing with that here:
0
ttremain

March 07, 2023 04:41
Those logs start showing after we get the Unknown storage engine 'InnoDB' errors, but the "visitors" tables listed are in the logs aren't even Innodb. I try repairing in phpMyAdmin, but there they show fine. Do you still think I should be looking at that link for more crash fixes? The "crashed" tables appear to me to be a symptom of the service failing, but that's why I'm asking. 2023-03-06 21:03:55 140318766040832 [ERROR] mysqld: Table './nodyss5_labrisacondo/users' is marked as crashed and should be repaired 2023-03-06 21:03:55 140318766040832 [ERROR] mysqld: Table 'users' is marked as crashed and should be repaired 2023-03-06 21:03:55 140318766040832 [Warning] Checking table: './nodyss5_labrisacondo/users' 2023-03-06 21:03:55 140318766040832 [ERROR] mysqld: Table './nodyss5_labrisacondo/log' is marked as crashed and should be repaired 2023-03-06 21:03:55 140318766040832 [ERROR] mysqld: Table 'log' is marked as crashed and should be repaired 2023-03-06 21:03:55 140318766040832 [Warning] Checking table: './nodyss5_labrisacondo/log' 2023-03-06 21:27:28 140318766348032 [Warning] Access denied for user 'nodyss5_oAkH2m0TH0A'@'localhost' (using password: YES) 2023-03-06 22:53:39 140318765426432 [Warning] Access denied for user 'nodyss5_oAkH2m0TH0A'@'localhost' (using password: YES) 2023-03-06 23:11:03 140318764812032 [Warning] Access denied for user 'root'@'localhost' (using password: YES) 2023-03-06 23:21:30 140318765119232 [Warning] Access denied for user 'root'@'localhost' (using password: YES) 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Uses event mutexes 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Using Linux native AIO 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Number of pools: 1 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-07 6:28:39 140250486614208 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Completed initialization of buffer pool 2023-03-07 6:28:40 140240971593472 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1352375362 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: 128 out of 128 rollback segments are active. 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-03-07 6:28:40 140250486614208 [Note] InnoDB: Waiting for purge to start 2023-03-07 6:28:41 140250486614208 [Note] InnoDB: 5.7.38 started; log sequence number 1352375371 2023-03-07 6:28:41 140241197225728 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2023-03-07 6:28:41 140250486614208 [Note] Plugin 'FEEDBACK' is disabled. 2023-03-07 6:28:41 140250486614208 [Note] Recovering after a crash using tc.log 2023-03-07 6:28:41 140250486614208 [Note] Starting crash recovery... 2023-03-07 6:28:41 140250486614208 [Note] Crash recovery finished. 2023-03-07 6:28:41 140241197225728 [Note] InnoDB: Buffer pool(s) load completed at 230307 6:28:41 2023-03-07 6:28:41 140250486614208 [Note] Server socket created on IP: '::'. 2023-03-07 6:28:41 140250486614208 [Note] Reading of all Master_info entries succeeded 2023-03-07 6:28:41 140250486614208 [Note] Added new Master_info '' to hash table 2023-03-07 6:28:41 140250486614208 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Uses event mutexes 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Using Linux native AIO 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Number of pools: 1 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-07 6:28:54 140037555460288 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-07 6:28:55 140037555460288 [Note] InnoDB: Completed initialization of buffer pool 2023-03-07 6:28:55 140028040726272 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-07 6:28:55 140037555460288 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-07 6:28:55 140037555460288 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1352375399 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: 128 out of 128 rollback segments are active. 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: Waiting for purge to start 2023-03-07 6:28:56 140037555460288 [Note] InnoDB: 5.7.38 started; log sequence number 1352375408 2023-03-07 6:28:56 140028266350336 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2023-03-07 6:28:56 140028266350336 [Note] InnoDB: Buffer pool(s) load completed at 230307 6:28:56 2023-03-07 6:28:56 140037555460288 [Note] Plugin 'FEEDBACK' is disabled. 2023-03-07 6:28:56 140037555460288 [Note] Recovering after a crash using tc.log 2023-03-07 6:28:56 140037555460288 [Note] Starting crash recovery... 2023-03-07 6:28:56 140037555460288 [Note] Crash recovery finished. 2023-03-07 6:28:56 140037555460288 [Note] Server socket created on IP: '::'. 2023-03-07 6:28:56 140037555460288 [Note] Reading of all Master_info entries succeeded 2023-03-07 6:28:56 140037555460288 [Note] Added new Master_info '' to hash table 2023-03-07 6:28:56 140037555460288 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-07 6:29:03 140028215736064 [ERROR] mysqld: Table './nodyss5_mycitrusvalley/users' is marked as crashed and should be repaired 2023-03-07 6:29:03 140028215736064 [ERROR] mysqld: Table 'users' is marked as crashed and should be repaired 2023-03-07 6:29:03 140028215736064 [Warning] Checking table: './nodyss5_mycitrusvalley/users' 2023-03-07 6:29:03 140028215736064 [ERROR] mysqld: Table './nodyss5_mycitrusvalley/visitors' is marked as crashed and should be repaired 2023-03-07 6:29:03 140028215736064 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:29:03 140028215736064 [Warning] Checking table: './nodyss5_mycitrusvalley/visitors' 2023-03-07 6:29:26 140028215428864 [ERROR] mysqld: Table './nodyss5_pacc2info/users' is marked as crashed and should be repaired 2023-03-07 6:29:26 140028215428864 [ERROR] mysqld: Table 'users' is marked as crashed and should be repaired 2023-03-07 6:29:26 140028215428864 [Warning] Checking table: './nodyss5_pacc2info/users' 2023-03-07 6:31:03 140028216043264 [ERROR] mysqld: Table './nodyss5_burgundy1/visitors' is marked as crashed and should be repaired 2023-03-07 6:31:03 140028216043264 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:31:03 140028216043264 [Warning] Checking table: './nodyss5_burgundy1/visitors' 2023-03-07 6:33:29 140028214814464 [ERROR] mysqld: Table './nodyss5_riverbendfl/visitors' is marked as crashed and should be repaired 2023-03-07 6:33:29 140028214814464 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:33:29 140028214814464 [Warning] Checking table: './nodyss5_riverbendfl/visitors' 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Uses event mutexes 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Using Linux native AIO 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Number of pools: 1 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-07 6:49:33 139923511498944 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-07 6:49:34 139923511498944 [Note] InnoDB: Completed initialization of buffer pool 2023-03-07 6:49:34 139914009155328 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-07 6:49:34 139923511498944 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-07 6:49:34 139923511498944 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1352375436 2023-03-07 6:49:47 140519621306560 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-07 6:49:47 140519621306560 [Note] InnoDB: Uses event mutexes 2023-03-07 6:49:47 140519621306560 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-07 6:49:47 140519621306560 [Note] InnoDB: Using Linux native AIO 2023-03-07 6:49:47 140519621306560 [Note] InnoDB: Number of pools: 1 2023-03-07 6:49:47 140519621306560 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-07 6:49:48 140519621306560 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-07 6:49:48 140519621306560 [Note] InnoDB: Completed initialization of buffer pool 2023-03-07 6:49:48 140510116435712 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-07 6:49:48 140519621306560 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-07 6:49:48 140519621306560 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1352375436 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: 128 out of 128 rollback segments are active. 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: Waiting for purge to start 2023-03-07 6:49:49 140519621306560 [Note] InnoDB: 5.7.38 started; log sequence number 1352375445 2023-03-07 6:49:49 140510342076160 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2023-03-07 6:49:49 140519621306560 [Note] Plugin 'FEEDBACK' is disabled. 2023-03-07 6:49:49 140519621306560 [Note] Recovering after a crash using tc.log 2023-03-07 6:49:49 140519621306560 [Note] Starting crash recovery... 2023-03-07 6:49:49 140519621306560 [Note] Crash recovery finished. 2023-03-07 6:49:49 140510342076160 [Note] InnoDB: Buffer pool(s) load completed at 230307 6:49:49 2023-03-07 6:49:49 140519621306560 [Note] Server socket created on IP: '::'. 2023-03-07 6:49:49 140519621306560 [Note] Reading of all Master_info entries succeeded 2023-03-07 6:49:49 140519621306560 [Note] Added new Master_info '' to hash table 2023-03-07 6:49:49 140519621306560 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-07 6:50:12 140510290847488 [ERROR] mysqld: Table './nodyss5_oakbrookcondominium/visitors' is marked as crashed and should be repaired 2023-03-07 6:50:12 140510290847488 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:50:12 140510290847488 [Warning] Checking table: './nodyss5_oakbrookcondominium/visitors' 2023-03-07 6:50:35 140510290847488 [ERROR] mysqld: Table './nodyss5_southrivercondo/visitors' is marked as crashed and should be repaired 2023-03-07 6:50:35 140510290847488 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:50:35 140510290847488 [Warning] Checking table: './nodyss5_southrivercondo/visitors' 2023-03-07 6:51:05 140510291461888 [ERROR] mysqld: Table './nodyss5_beachandtennisclub/visitors' is marked as crashed and should be repaired 2023-03-07 6:51:05 140510291461888 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:51:05 140510291461888 [Warning] Checking table: './nodyss5_beachandtennisclub/visitors' 2023-03-07 6:55:48 140510290233088 [ERROR] mysqld: Table './nodyss5_auburnvillage/users' is marked as crashed and should be repaired 2023-03-07 6:55:48 140510290233088 [ERROR] mysqld: Table 'users' is marked as crashed and should be repaired 2023-03-07 6:55:48 140510290233088 [Warning] Checking table: './nodyss5_auburnvillage/users' 2023-03-07 6:59:34 140510290847488 [ERROR] mysqld: Table './nodyss5_auburnvillage/visitors' is marked as crashed and should be repaired 2023-03-07 6:59:34 140510290847488 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 6:59:34 140510290847488 [Warning] Checking table: './nodyss5_auburnvillage/visitors' 2023-03-07 7:20:22 140510289311488 [ERROR] mysqld: Table './nodyss5_riverbendfl/realestate' is marked as crashed and should be repaired 2023-03-07 7:20:22 140510289311488 [ERROR] mysqld: Table 'realestate' is marked as crashed and should be repaired 2023-03-07 7:20:22 140510289311488 [Warning] Checking table: './nodyss5_riverbendfl/realestate' 2023-03-07 7:30:40 140510289618688 [Warning] Access denied for user 'nodyss5_oAkH2m0TH0A'@'localhost' (using password: YES) 2023-03-07 8:20:01 140510289618688 [Warning] Access denied for user 'nodyss5_oAkH2m0TH0A'@'localhost' (using password: YES) 2023-03-07 8:42:56 140510288389888 [ERROR] mysqld: Table './nodyss5_princesscondo/visitors' is marked as crashed and should be repaired 2023-03-07 8:42:56 140510288389888 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 8:42:56 140510288389888 [Warning] Checking table: './nodyss5_princesscondo/visitors' 2023-03-07 8:56:39 140510288697088 [Warning] Access denied for user 'nodyss5_oAkH2m0TH0A'@'localhost' (using password: YES) 2023-03-07 8:59:57 139879386273984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-07 8:59:57 139879386273984 [Note] InnoDB: Uses event mutexes 2023-03-07 8:59:57 139879386273984 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-07 8:59:57 139879386273984 [Note] InnoDB: Using Linux native AIO 2023-03-07 8:59:57 139879386273984 [Note] InnoDB: Number of pools: 1 2023-03-07 8:59:57 139879386273984 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-07 8:59:59 139879386273984 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-07 8:59:59 139879386273984 [Note] InnoDB: Completed initialization of buffer pool 2023-03-07 8:59:59 139869871138560 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-07 8:59:59 139879386273984 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-07 8:59:59 139879386273984 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1352757841 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: 128 out of 128 rollback segments are active. 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: Waiting for purge to start 2023-03-07 9:00:00 139879386273984 [Note] InnoDB: 5.7.38 started; log sequence number 1352757850 2023-03-07 9:00:00 139870096762624 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2023-03-07 9:00:00 139879386273984 [Note] Plugin 'FEEDBACK' is disabled. 2023-03-07 9:00:00 139879386273984 [Note] Recovering after a crash using tc.log 2023-03-07 9:00:00 139879386273984 [Note] Starting crash recovery... 2023-03-07 9:00:00 139879386273984 [Note] Crash recovery finished. 2023-03-07 9:00:00 139870096762624 [Note] InnoDB: Buffer pool(s) load completed at 230307 9:00:00 2023-03-07 9:00:00 139879386273984 [Note] Server socket created on IP: '::'. 2023-03-07 9:00:00 139879233410816 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown 2023-03-07 9:00:00 139879386273984 [Note] Reading of all Master_info entries succeeded 2023-03-07 9:00:00 139879386273984 [Note] Added new Master_info '' to hash table 2023-03-07 9:00:00 139879386273984 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-07 9:00:00 139879233410816 [Note] Event Scheduler: Purging the queue. 0 events 2023-03-07 9:00:00 139870138726144 [Note] InnoDB: FTS optimize thread exiting. 2023-03-07 9:00:00 139879233410816 [Note] InnoDB: Starting shutdown... 2023-03-07 9:00:00 139870096762624 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool 2023-03-07 9:00:00 139870096762624 [Note] InnoDB: Buffer pool(s) dump completed at 230307 9:00:00 2023-03-07 9:00:03 139879233410816 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-07 9:00:03 139879233410816 [Note] InnoDB: Shutdown completed; log sequence number 1352757869 2023-03-07 9:00:03 139879233410816 [Note] /usr/sbin/mysqld: Shutdown complete 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Uses event mutexes 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Compressed tables use zlib 1.2.7 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Using Linux native AIO 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Number of pools: 1 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Using SSE2 crc32 instructions 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 8, chunk size = 128M 2023-03-07 9:00:04 139870554659008 [Note] InnoDB: Completed initialization of buffer pool 2023-03-07 9:00:05 139861058160384 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-03-07 9:00:05 139870554659008 [Note] InnoDB: Highest supported file format is Barracuda. 2023-03-07 9:00:05 139870554659008 [Note] InnoDB: 128 out of 128 rollback segments are active. 2023-03-07 9:00:05 139870554659008 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-03-07 9:00:05 139870554659008 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-03-07 9:00:05 139870554659008 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-03-07 9:00:05 139870554659008 [Note] InnoDB: 5.7.38 started; log sequence number 1352757869 2023-03-07 9:00:05 139861265630976 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2023-03-07 9:00:05 139861265630976 [Note] InnoDB: Buffer pool(s) load completed at 230307 9:00:05 2023-03-07 9:00:05 139870554659008 [Note] Plugin 'FEEDBACK' is disabled. 2023-03-07 9:00:05 139870554659008 [Note] Server socket created on IP: '::'. 2023-03-07 9:00:05 139870554659008 [Note] Reading of all Master_info entries succeeded 2023-03-07 9:00:05 139870554659008 [Note] Added new Master_info '' to hash table 2023-03-07 9:00:05 139870554659008 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2023-03-07 9:01:19 139861215938304 [ERROR] mysqld: Table './nodyss5_islandiaone/visitors' is marked as crashed and should be repaired 2023-03-07 9:01:19 139861215938304 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 9:01:19 139861215938304 [Warning] Checking table: './nodyss5_islandiaone/visitors' 2023-03-07 9:02:01 139870375315200 [ERROR] mysqld: Table './nodyss5_polyvill/visitors' is marked as crashed and should be repaired 2023-03-07 9:02:01 139870375315200 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 9:02:01 139870375315200 [Warning] Checking table: './nodyss5_polyvill/visitors' 2023-03-07 9:02:37 139861215323904 [ERROR] mysqld: Table './nodyss5_southseaswest/visitors' is marked as crashed and should be repaired 2023-03-07 9:02:37 139861215323904 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 9:02:37 139861215323904 [Warning] Checking table: './nodyss5_southseaswest/visitors' 2023-03-07 9:03:48 139861215323904 [ERROR] mysqld: Table './nodyss5_lakesidevillagefla/visitors' is marked as crashed and should be repaired 2023-03-07 9:03:48 139861215323904 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 9:03:48 139861215323904 [Warning] Checking table: './nodyss5_lakesidevillagefla/visitors' 2023-03-07 9:04:44 139870375315200 [ERROR] mysqld: Table './nodyss5_polyvill/users' is marked as crashed and should be repaired 2023-03-07 9:04:44 139870375315200 [ERROR] mysqld: Table 'users' is marked as crashed and should be repaired 2023-03-07 9:04:44 139870375315200 [Warning] Checking table: './nodyss5_polyvill/users' 2023-03-07 9:12:55 139870375315200 [ERROR] mysqld: Table './nodyss5_forestlakescondos/visitors' is marked as crashed and should be repaired 2023-03-07 9:12:55 139870375315200 [ERROR] mysqld: Table 'visitors' is marked as crashed and should be repaired 2023-03-07 9:12:55 139870375315200 [Warning] Checking table: './nodyss5_forestlakescondos/visitors' 2023-03-07 9:13:19 139861214709504 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown 2023-03-07 9:13:19 139861214709504 [Note] Event Scheduler: Purging the queue. 0 events 2023-03-07 9:13:19 139861307594496 [Note] InnoDB: FTS optimize thread exiting. 2023-03-07 9:13:19 139861214709504 [Note] InnoDB: Starting shutdown... 2023-03-07 9:13:19 139861265630976 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool 2023-03-07 9:13:19 139861265630976 [Note] InnoDB: Buffer pool(s) dump completed at 230307 9:13:19 2023-03-07 9:13:21 139861214709504 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-03-07 9:13:21 139861214709504 [Note] InnoDB: Shutdown completed; log sequence number 1352757897 2023-03-07 9:13:21 139861214709504 [Note] /usr/sbin/mysqld: Shutdown complete
0
cPRex Jurassic Moderator

March 08, 2023 13:53
Yes, as long as the logs show the "marked as crashed and should be repaired" error you'll have to go through that process. Or, you can restore the tables from a backup.
0
ttremain

March 08, 2023 15:22
Yes, as long as the logs show the "marked as crashed and should be repaired" error you'll have to go through that process. Or, you can restore the tables from a backup.

How about export the table, drop it, and re-import it? All the data seems to be in tact.
0
cPRex Jurassic Moderator

March 08, 2023 15:23
Sure, that's basically a restore with just a different process. Although if you're working with the currently crashed table, I wouldn't expect good results.
0
ttremain

March 08, 2023 15:25
Sure, that's basically a restore with just a different process. Although if you're working with the currently crashed table, I wouldn't expect good results.

The visitors table is temporary anyway. It's a logs table, I could truncate it without worry, so drop and re-create with no concern of data integ.
0
ttremain

March 09, 2023 10:10
Watching the logs, each table that said it was corrupt so far, has been deleted and restored. As of this time, mysql hasn't failed in about 8 hours. Thank you!
0
cPRex Jurassic Moderator

March 09, 2023 14:30
Nice - I'm so glad to hear that!
0
ttremain

March 09, 2023 18:09
Nice - I'm so glad to hear that!

15(ish) more tables corrupt before morning. Running MySQL Tuning-Primer, even though mysql hasn't been up for 48 hours.
0
cPRex Jurassic Moderator

March 09, 2023 18:25
I wonder if there is some underlying disk issues on the machine that could be causing this type of behavior to happen.
0
ttremain

March 09, 2023 21:22
I wonder if there is some underlying disk issues on the machine that could be causing this type of behavior to happen.

We are wondering the same thing. Probably time to migrate to a new dedicated anway. This one is 6 years old, that's about 60 in computer years.
0
ttremain

March 11, 2023 18:02
I wonder if there is some underlying disk issues on the machine that could be causing this type of behavior to happen.

Ran tuning-primer and adjusted a few settings. Seems to have held for the last 48 hours! WOOHOO!!!!
0
cPRex Jurassic Moderator

March 13, 2023 07:06
Nice - so there's hope!
0
andrew.n

March 14, 2023 15:16
Good job @ttremain!
0

Please sign in to leave a comment.

Comments

Didn't find what you were looking for?