Major Issues Lately with MariaDB Upgrades and Updates via WHM
After going through 2 days of hell in the last few months, I am sharing my thoughts here:
1) I no longer trust anything to do with WHM's ability to run any type of upgrade or update on MariaDB databases. Within the last few months I've had 2 major crashed database incidents due to:
- a) a more major upgrade from MariaDB 10.5 to 10.6 using WHM update mysql app;
- b) a "minor" update to MariaDB 10.6 using the standard yum update command;
- "success-a new DB was created that was tested to have no issues with the name of db-name-copy. To test this with your app be sure to change the DB name in the config file of your site,"
-
Hey there! If you're referring to the issues yesterday, that was a problem with MariaDB breaking their own repositories, so there wasn't anything we could do on our end for that. MariaDB said that was resolved as of 12:30 yesterday, although some people reported odd behavior for another few hours while the mirrors were getting back in sync. If you have ideas about adding repair features, please submit a feature request and I can bring that up during our weekly developer meeting. We're always working on and entertaining new features so I'd be happy to discuss that with them. 0 -
No, I'm not referring to that. The auto update that ran did alert me to an issue, but when I ran yup update ~6 hours later the update ran fine, found the repositories, and then broke my databases--all of them on different web sites. 0 -
Any chance you could submit a ticket so we can see that? I'm not saying it's not possible, but a Yum transaction wouldn't touch any data in the database itself, so I'm not sure how the two would be related without us doing some investigation. 0 -
I did, but fixed this myself in the several hours I was waiting for anyone there to respond...oh...did I mention my issue with how long it is now to get any help from you guys? 0 -
and yes, yum update, which updated MariaDB, did crash every database on my server to the point where the only way to repair them was a full export import routine, which took a ton of time. 0 -
How long did you wait for a ticket response? Is your license direct through us so you receive priority support? Could you let me know a ticket number so I could review that? I'm happy to listen to feedback, but I do need some data to act on in order to help. As far as the Yum issues, I suppose anything is possible but I'm just not sure how a Yum transaction would corrupt database data. If that did happen between Yum and MariaDB, at least cPanel tools wouldn't have been involved. 0 -
#94475125 is the ticket number, and I believe I'm on the misnomer "Priority" support ;) Apparently the paid upgraded "Premium" support is no longer an option, as it now throws an error when I try to do that. I can't tell the exact times on the ticket, but it looks like I submitted it at ~4:30PM Pacific time, and the first response on the ticket was ~4 hours later. It was enough time for a non-expert like me to research how, and to rebuild 5 mysql database, 3 of them being over 5 gigs. I'm only glad I was able successfully dump all the data, so that nothing was lost. But really, I'd love to hear a comment about #2 above. Why the heck, given that essentially all of your customers run mysql databases, doesn't WHM offer more automated recovery options (especially if your software updates/upgrades might be breaking them more often going forward)? BTW, I posted a reply on that ticket 6 hours ago and nobody has replied. I guess it's a good think my sites are not still down and I was able to recover the DBs myself! 0 -
Thanks for the details. I checked the ticket and confirmed that your license isn't purchased through us, so the ticket is part of the complimentary queue, which has no official timeframe for a reply. You are correct that there are issues with the support bump in the ticket system, and our developers are working to address that. I see that Justin created case CPANEL-41340 with our development team to have them explore the possibility of making some type of automatic database copy during the upgrade process on the server, just in case something goes wrong. Traditionally we try and stay hands-off with data, but we'll let the team explore this and see what they come up with. We also weren't able to access the server in the ticket, so for more details on what may have caused the MySQL problems we'd have to get access to the affected system. 0 -
Thanks for looking into this...regardless of the license type I'm paying your regularly for my license, which should include reasonable support times, but again, your escalation system has been broken for months now, and I have no issues paying $30 or so in an emergency, but I could not even do that the last two emergencies I had (both caused by WHM). I hope you can fix it soon, but I would fire the person who's been working on it. I could have gotten that fixed for you within a few days... 0 -
I've got someone looking into this today, so hopefully that is resolved soon. Since the license isn't purchased through cPanel, your first contact for support would normally be the hosting provider where the license was purchased. 0 -
Hi my server gives me an update error .. this is on my MariaDB106.repo [MariaDB106] name = MariaDB106 baseurl = gpgcheck=1 But that folder does not exist on the MariaDB server? I am running Almalinux 8.6 with latest WHM/cPanel and MariaDB 10.6.9 [root@host ~]# yum -y update MariaDB106 862 B/s | 10 B 00:00 Error: Failed to download metadata for repo 'MariaDB106': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried Is there a way to fix this? Because i can not even update EA now. 0 -
The is no centos8 folders on the MariaDB servers 0 -
CentOS 8 is not a supported operating system, so that makes sense to me. You should be using Ubuntu, CentOS 7, AlmaLinux 8, or CloudLinux for a cPanel system at this time. 0 -
I am running Almalinux 8.6 with latest WHM/cPanel and MariaDB 10.6.9 0 -
this is the text in my MariaDB106.repo on the server. [MariaDB106] name = MariaDB106 baseurl = gpgcheck=1 0 -
I updated from MariaDB 10.6.8 to 10.6.9 a few day ago with no problems. 0 -
Do i need to change the baseurl in the MariaDB106.repo file? And if yes ... what do i need to change it to? 0 -
Thanks @ServerHealers ! 0 -
Working now .. thank you for your help :) 0 -
Now i have another issue MariaDB stopped working today .. no idea why .. i was not on the server when it happened.
How to fix the i already tried to set /usr/lib/systemd/system/mariadb.service to # Number of files limit. previously [mysqld_safe] open-files-limit LimitNOFILE=infinity and LimitNOFILE=650000 But none of them did it .. how do i fix this?(XID 2qsvqs) The "mysql" service is down. The subprocess "/usr/local/cpanel/scripts/restartsrv_mysql" reported error number 2 when it ended. Aug 19 18:05:54 host.****.com systemd[1]: Starting MariaDB 10.6.9 database server... Aug 19 18:05:54 host.****.com mariadbd[757800]: 2022-08-19 18:05:54 0 [Note] /usr/sbin/mariadbd (server 10.6.9-MariaDB) starting as process 757800 ... Aug 19 18:05:54 host.****.com mariadbd[757800]: 2022-08-19 18:05:54 0 [Warning] Could not increase number of max_open_files to more than 32768 (request: 640863) Aug 19 18:05:54 host.****.com mariadbd[757800]: 2022-08-19 18:05:54 0 [Warning] Changed limits: max_open_files: 32768 max_connections: 800 (was 800) table_cache: 15969 (was 40000) Aug 19 18:05:56 host.****.com systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE Aug 19 18:05:56 host.****.com systemd[1]: mariadb.service: Failed with result 'exit-code'. Aug 19 18:05:56 host.****.com systemd[1]: Failed to start MariaDB 10.6.9 database server. 0 -
Unable to upgrade from mysql 5.7 to mariadb 10.3 In the error logs, 404 error response comes from mariadb 10.2 repository, the update cannot continue because the updates were made to 10.2 first to update to 10.3. [QUOTE] The system was not able to ensure the availability of the "MariaDB-client" package: (XID se5akg) "/usr/bin/yum" reported error code "1" when it ended: : [Errno 14] HTTPS Error 404 - Not Found 0 -
... 0 -
@vacancy - that sounds right, and until the fix is 100% applied to all systems, which will happen this week, people will continue to see errors. If you need to manually perform an update you can run this: echo '142.4.217.28 yum.mariadb.org' >> /etc/hosts then run the update, then remove that hosts file entry. 0 -
After going through 2 days of hell in the last few months, I am sharing my thoughts here: 1) I no longer trust anything to do with WHM's ability to run any type of upgrade or update on MariaDB databases. Within the last few months I've had 2 major crashed database incidents due to: - a) a more major upgrade from MariaDB 10.5 to 10.6 using WHM update mysql app;
- b) a "minor" update to MariaDB 10.6 using the standard yum update command;
- "success-a new DB was created that was tested to have no issues with the name of db-name-copy. To test this with your Adding more repair options might at least mitigate the many problems now created when using WHM to upgrade or update MariaDB.
0 -
It's important to note the most recent issue was caused by MariaDB changing their repository structure. I did see the feature request that was submitted about repair options, so we'll see if we can get something like that added in the future. 0 -
Thank you @ServerHealers the solution provided by you worked like charm .
Happy to help always! :)0
Please sign in to leave a comment.
Comments
29 comments