MySQL constant high CPU usage
For the last couple of days, MySQL has been obliterating my CPU for some reason I do not understand. I first realised this when my site was unresponsive, logging into SSH/WHM were also painfully slow. htop showed mysqld was using high CPU constantly. I checked WHM process manager and it shows the same thing.
In checking everything else, I found one account which was over its disk usage quota due to two analog stats cache files being very large. I deleted both of those but it seems unrelated.
A trace of the mysqld process in WHM shows this being repeated constantly:
That is just a snippet. It is constant. In the /var/lib/mysql/server.hostname.err file I see this a few times:
I'm not sure whether this is caused by my recent constant killing of mysqld or restarting server. Monitoring database usage with watch mysqladmin pr there is nothing out of the ordinary, in fact there is extremely infrequent access other than one wordpress site one of my users has which is using an activity tracker script. Centos 6.6 2x Xeon E5620 4GB RAM
setsockopt(200, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)
futex(0x1321f64, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1321f60, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
poll([{fd=12, events=POLLIN}, {fd=14, events=POLLIN}], 2, -1) = 1 ([{fd=14, revents=POLLIN}])
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
accept(14, {sa_family=AF_LOCAL, NULL}, [2]) = 310
fcntl(14, F_SETFL, O_RDWR) = 0
setsockopt(310, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)
futex(0x1321f64, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1321f60, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
poll([{fd=12, events=POLLIN}, {fd=14, events=POLLIN}], 2, -1) = 1 ([{fd=14, revents=POLLIN}])
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
accept(14, {sa_family=AF_LOCAL, NULL}, [2]) = 200
fcntl(14, F_SETFL, O_RDWR) = 0
setsockopt(200, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)
futex(0x1321f64, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1321f60, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x131f900, FUTEX_WAKE_PRIVATE, 1) = 1
poll([{fd=12, events=POLLIN}, {fd=14, events=POLLIN}], 2, -1) = 1 ([{fd=14, revents=POLLIN}])
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
accept(14, {sa_family=AF_LOCAL, NULL}, [2]) = 200
fcntl(14, F_SETFL, O_RDWR) = 0
setsockopt(200, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)
futex(0x1321f64, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1321f60, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
poll([{fd=12, events=POLLIN}, {fd=14, events=POLLIN}], 2, -1) = 1 ([{fd=14, revents=POLLIN}])
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
accept(14, {sa_family=AF_LOCAL, NULL}, [2]) = 310
fcntl(14, F_SETFL, O_RDWR) = 0
setsockopt(310, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)
futex(0x1321f64, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1321f60, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x131f900, FUTEX_WAKE_PRIVATE, 1) = 1
That is just a snippet. It is constant. In the /var/lib/mysql/server.hostname.err file I see this a few times:
InnoDB: Error: trying to access page number 3925863297 in space 0,
InnoDB: space name ./ibdata1,
InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that
InnoDB: your my.cnf matches the ibdata files that you have in the
InnoDB: MySQL server.
2017-02-08 00:15:59 7f1a0f2ba700 InnoDB: Assertion failure in thread 139749900396288 in file fil0fil.cc line 5664
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to MySQL Bugs.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: MySQL :: MySQL 5.6 Reference Manual :: 14.21.2 Forcing InnoDB Recovery
InnoDB: about forcing recovery.
00:15:59 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
I'm not sure whether this is caused by my recent constant killing of mysqld or restarting server. Monitoring database usage with watch mysqladmin pr there is nothing out of the ordinary, in fact there is extremely infrequent access other than one wordpress site one of my users has which is using an activity tracker script. Centos 6.6 2x Xeon E5620 4GB RAM
-
Hello, You may want to try running a MySQL tuning application to see if the results show any helpful information. This thread will help you get started: mysqlmymonlite.sh server stats gathering tool for cPanel Server Thank you. 0
Please sign in to leave a comment.
Comments
1 comment