Skip to main content

MariaDB Upgrade from 10.5 to 10.6 Failed

celiac101
I tried to upgrade mysql but the upgrade failed. My upgrade steps and the error log is below. Initially my site was totally down and throwing 500 errors, and I ran this command which brough it back up: mysql_upgrade -u root -h localhost -p --verbose --force The web site seems to be fine, and I do not see any errors being recorded in the mysql log, however, the MySQL/MariaDB Upgrade (Step 5 of 5) there is this notice: " The upgrade process failed. You should correct any errors listed in the output window and retry the upgrade process" and I do not see PHPMYADMIN as an option anymore in cPanel. I did make backups of the site databases, and the mysql before I upgraded. MY UPGRADE STEPS: 1) Turn website off 2) Backup DB's: mysqldump --all-databases > all_databases_20220721.sql 3) turn off tailwatched mysql 4) systemctl stop mysqld 5) cp -rav /var/lib/mysql /backup2/mysql_20220719.bak 6) systemctl start mysqld 7) Run MariaDB upgrade - Stuck here 8) turn on tailwatched mysql 9) Test 10) Turn website on I see multiple errors like this in my mysql log: 2022-07-22 3:25:26 0 [ERROR] InnoDB: Operating system error number 2 in a file operation. 2022-07-22 3:25:26 0 [ERROR] InnoDB: The error means the system cannot find the path specified. 2022-07-22 3:25:26 0 [ERROR] InnoDB: Cannot open datafile for read-only: './celiac_com_advertising/phpads_account_preference_assoc.ibd' OS error: 71 2022-07-22 3:25:26 0 [ERROR] InnoDB: Operating system error number 2 in a file operation. 2022-07-22 3:25:26 0 [ERROR] InnoDB: The error means the system cannot find the path specified. 2022-07-22 3:25:26 0 [ERROR] InnoDB: Could not find a valid tablespace file for celiac_com_advertising/phpads_account_preference_assoc. Please refer to Running transaction check ---> Package MariaDB-server.x86_64 0:10.6.8-1.el7.centos will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: MariaDB-server x86_64 10.6.8-1.el7.centos MariaDB106 25 M Transaction Summary ================================================================================ Install 1 Package Total download size: 25 M Installed size: 122 M Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : MariaDB-server-10.6.8-1.el7.centos.x86_64 1/1 /etc/yum/universal-hooks/pkgs/MariaDB-server/posttrans/100-build_mysql_conf: running in the background. Verifying : MariaDB-server-10.6.8-1.el7.centos.x86_64 1/1 Installed: MariaDB-server.x86_64 0:10.6.8-1.el7.centos Complete! MariaDB installed from official repository. Building configuration. This step may produce some errors or warnings in the log. The errors are usually harmless and are a result of table changes between versions of MySQL or MariaDB. Running: /usr/local/cpanel/bin/build_mysql_conf Done building configuration. Hooks system enabled. Checking for and running RPM::Versions 'post' hooks for any Packages about to be installed All required 'post' hooks have been run Running: /usr/local/cpanel/scripts/check_cpanel_pkgs --targets=MySQL41,MySQL50,MySQL51,MySQL55,MySQL56,MySQL57,MySQL80,MariaDB100,MariaDB101,MariaDB102,MariaDB103,MariaDB105,MariaDB106 --fix --no-broken --no-digest The system could not start MariaDB 10.6 Startup Log Jul 22 01:32:26 scott.celiac.com systemd[1]: Starting MariaDB 10.6.8 database server... Jul 22 01:32:27 scott.celiac.com mariadbd[12683]: 2022-07-22 1:32:27 0 [Note] /usr/sbin/mariadbd (server 10.6.8-MariaDB) starting as process 12683 ... Jul 22 01:32:27 scott.celiac.com systemd[1]: Started MariaDB 10.6.8 database server. Jul 22 01:35:12 scott.celiac.com systemd[1]: Stopping MariaDB 10.6.8 database server... Jul 22 01:35:13 scott.celiac.com systemd[1]: Stopped MariaDB 10.6.8 database server. Jul 22 02:45:30 scott.celiac.com systemd[1]: Starting MariaDB 10.6.8 database server... Jul 22 02:45:30 scott.celiac.com mariadbd[25210]: 2022-07-22 2:45:30 0 [Note] /usr/sbin/mariadbd (server 10.6.8-MariaDB) starting as process 25210 ... Jul 22 02:45:31 scott.celiac.com systemd[1]: Started MariaDB 10.6.8 database server. Jul 22 03:14:55 scott.celiac.com systemd[1]: Stopping MariaDB 10.6.8 database server... Jul 22 03:14:56 scott.celiac.com systemd[1]: Stopped MariaDB 10.6.8 database server. Restarting mysql service. (XID uthx9v) The "mysql" service is disabled. MariaDB did not start up after upgrade; there may be a problem in /etc/my.cnf. at /usr/local/cpanel/Whostmgr/Mysql/Upgrade.pm line 859. Obtained version information from system. Proceeding with MySQL/MariaDB upgrade despite the following: Critical: The system failed to query the MySQL server to check for the presence of pre-4.1-style MySQL passwords. ((XID 4dey6e) The system failed to connect to the "MySQL" database "mysql" because of an error: CR_CONNECTION_ERROR (Can't connect to local server through socket '/var/lib/mysql/mysql.sock' (2))) This issue could also hinder mysql_upgrade's ability to run, which could potentially leave MySQL in an unusable state if you proceed. Normal: The selected MariaDB version (10.6) is the same as the currently installed MariaDB version (10.6). No upgrade will be performed at this time, though the normal upgrade steps will still be executed. This is only useful if a previous upgrade failed while partially completed. Normal: MariaDB 10.6 introduced a new reserved word: OFFSET. This can no longer be used as an identifier without being quoted. Normal: From MariaDB 10.6, tables that are of the `COMPRESSED` row format are read-only by default. This is the first step towards removing write support and deprecating the feature. The `innodb_read_only_compressed` variable must be set to `OFF` in order to make the tables writable.

Comments

4 comments

Date Votes
  • cPRex Jurassic Moderator
    Hey there! I would recommend not stopping the MySQL service - it has to be online for the update to complete. Can you reactivate that and try again?
    0
  • celiac101
    Thank you, but if you notice in my list, I turn it off only to back up the mysql, and then turn it back on after, and then run the upgrade.
    0
  • cPRex Jurassic Moderator
    It might be best to submit a ticket to our team so we can check that directly. That's one of those "it should just work automatically" items.
    0
  • celiac101
    This got resolved, but I'm still not sure exactly how it got fixed. The state my MariaDB got stuck in--MySQL/MariaDB Upgrade showed that I needed to continue the upgrade because it did not finish the upgrade. I had no access to PHPMyAdmin, and I tried re-running continue upgrade process 3 times, but it stayed in this state. I created a ticket on this, and here are the two things that were done: 1) "It looks like the mysql service was just disabled and hadn't been re-enabled. Both the "enabled" and "monitored" boxes in WHM -> Service Manager were unchecked for mysql" 2) Disabled the skip-name-resolve in my.cnf (I had this enabled because I heard it makes sites faster...not sure, but I'm always trying to increase site speed). I'm not sure how either could have fixed this, but in my original checklist I will admit that I did disable tailwatched mysql on step 3 to backup my mysql, and if this caused the issue, then I likely should have made turning it back on step 7, and the upgrade should have been step 8. Interestingly nobody had to restart the finish mysql upgrade, as simply doing #1 above seemed to fix everything.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post