MySQL won't start - "(Errcode: 28 - No space left on device)"

Symptoms

MySQL won't start due to "(Errcode: 28 - No space left on device)" errors in /var/log/mysqld.log

mysqld: Can't create/write to file '/tmp/ibBszhgf' (Errcode: 28 - No space left on device)

Description 

This may be caused by data space consumption in /tmp and is usually easily caught running

df -h

There is a number of file limit in each partition as well and these limits are called inode limits. If it's not space consumption then run

df -i

In this case /tmp may be at 100% usage like this:

[root@server /var/lib/mysql]# df -i /tmp
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/loop0 147744 147742 2 100% /tmp
[root@server /var/lib/mysql]#

Workaround

Files will need to be moved or removed from /tmp and typically the space usage occurs because a user sets a PHP session.save_path to /tmp instead of /var/cpanel/php/sessions/ea-phpXX where XX is the PHP version being used. Before removing any session files note the user and group so yo or your administrator can check that user's sites for session.save_path set to /tmp

If that's the case there is a large number of session files.

You can remove all but one days worth by running the following

find /tmp ! -mtime -1 -name 'sess_*' -exec rm -f '{}' \;

or remove them all with

rm -f /tmp/sess_*

If that gives an argument too long error then yo may need to loop through the files but this will take longer to process

find /tmp -name 'sess_*' -exec rm -f '{}' \;

PLEASE NOTE: Use EXTREME caution while using the rm command. Incorrectly run rm commands can severely damage a server. Please have anything run via rm reviewed by an obtained qualified server administrator.