Apache HTTP response codes go 403 (forbidden) at random
Hello,
We seem to be having an issue with 1 of our WHM servers of which we can't replicate on any other of our 10 others.
Completely at random HTTP requests will come back with the 403 (forbidden) status code instead 200, even though the PHP application does present a 200 response. The payload in the request is completely correct as if it should be status 200. -- Well I say random, you'll usually get 5-10 good requests for 1 bad. The URL of the request or what PHP processing it entails doesn't seem to have any implications.
I'm trying to rule out the PHP application being the cause as we've currently got 15+ instances of it on other servers (both ours and clients), but so far only 1 specific server is having this problem. I've tried disabling mod_security in both WHM and cPanel as I read that it can sometimes cause stuff like this, but that didn't change.
Other things I've checked are file system permissions for the cPanel account
So yeah, I have no idea where to look next other than spinning up an alternative server. (That will cost though, this is a production one.)
Are there any magic check/fix scripts I should be using? (Or other suggestions?)
-
Just for the record we've noticed this happens on both WordPress and Symfony based PHP applications, which leads us to believe the issue is at server level. 0 -
I've seen this occur when mod_evasive is installed and enabled - it may be worth checking the settings for that if you've got it installed. The logs in /var/log/apache2 may also be useful. 0 -
Very interesting! I had completely forgotten about that mod. Yes, it's very installed on all of our WHM servers. I thought "mod_security" might have been a culprit but not that. I've taken mod_evasive out of our EA4 profiles and suddenly the 403s seem to have disappeared. -- There are indeed a ton of files (1 per IP address) at "/var/log/apache2/mod_evasive", each just contains a number. (Probably a hit count?) Thank you very much indeed for that tip. :) -- Unrelated, not really sure what the point of changing the HTTP response code is if the payload is left unchanged. Oh well. 0 -
Glad to hear that you were able to identify the cause of the issue and @orudge thanks for providing the useful advice. @adamreece.webbox it sounds like the status code wasn't being changed but that mod_evasive was actually blocking access to the site which would return a 403 error 0 -
Hi Lauren, That's possibly what was going on, though each time the status code was 403 the correct+intended response body from the PHP application was being delivered with it. The issue was particularly noticeable with small but frequent AJAX requests, but the expected JSON body was correct every time. 0 -
The explanation of how it works might be helpful as well as checking for the mod_evasive log messages:
I'm also curious what the PHP 200 response looked like/where it was in conjunctions with the 403's you received0
Please sign in to leave a comment.
Comments
6 comments