ModSecurity is missing connections it also blocks.
Hi All,
Currently getting zip attacks agains my server - request uri is /1.zip, /ALL_FILES.zip etc.
I have a custom rule that drops these requests - it works great (rule at the end of this post.)
What is odd thought, is modsec will block a connection for one domain, but for the next domain attacked, the server sends a 404.
No domains on the server are excluded from modsec, no domains have custom/specific rules and in theory no domain should be treated any differently.
Any suggestions?
Thanks!
Steve
SecRule REQUEST_METHOD "GET|POST|HEAD" "chain, id:25001 ,phase:1, log, drop, status:403, t:compressWhiteSpace, t:replaceNulls, msg:' *** GPH zip attack drop', skipAfter:END_CUSTOM_RULES"
SecRule REQUEST_URI "@pmFromFile text_match/zip-attack.txt"
Rule looks at
GET, HEAD, POST data; super low rule id to have the act early; drops the connection on match; cleans up whitespace and nulls, skips over all other rules on match [chain]
Checks the request url using a text match against a file with a list of URI's -- /1.zip, /ALL_FILES.zip etc.
-
Hey there! Does it matter how much time is in between the correct Modsec block request and the 404 request? If there is an active attack, I'm wondering if it is happening too quickly for Modsec to check them all. 0 -
Hi - it's a thought, that the server would be swamped - but no, that's not it - the server responds just fine. Here's a hit from Amazon EC2 in the last few minutes: /usr/local/apache/domlogs/[REDACTED].com:13.232.110.216 - - [21/Apr/2021:15:51:41 -0500] "GET /wp-load.zip HTTP/1.1" 404 7941 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36" It's as if the request just sailed through mod sec to apache. Going to add logging to that rule and see if it is even being triggered. 0 -
If you don't find anything with the logging, feel free to submit a ticket to our team so we can take a look at that directly on the system. 0 -
@servtastic If I was trying to block request attempts to known dodgy zip files I would use a rule like this... SecRule REQUEST_FILENAME "@pmFromFile text_match/zip-attack.txt" "msg:'*** GPH zip attack blocked', id:25001, phase:1, block, status:403, t:none"
- Your first rule to match the REQUEST_METHOD serves no purpose, so is unnecessary. If the attack was directed to the POST method only then a chain beginning with SecRule REQUEST_METHOD "POST" would provide a fast fail mechanism for all other request methods.
- If you drop a request in phase 1 (Request Headers), then no Phase 3 (Response Headers) occur, so status:403 is irrelevant. For this reason I would choose block disruptive action over drop. Once you have Response Status of 403 being returned and logged in the apache_log, then iptables can be used to drop repeat offending IPs. In OWASP CRS3 the drop disruptive action is only used in 2 rules, both of those rules are DOS-PROTECTION rules after repeat DOS behavior is identified.
- REQUEST_FILENAME is a much smaller haystack than REQUEST_URI. (especially in non-matching requests) All the Request Header variables are loaded by the time modsec applys the Phase 1 rules whether the rules use these variables or not.
- t:compressWhiteSpace is not really nescesary for a REQUEST_URI or a REQUEST_FILENAME because Apache will return a 400 Bad Request status if white-space is encountered. Unless you were trying to replace a null byte with a space then compress that space (not sure if that works) Converts any of the whitespace characters (0x20, \f, \t, \n, \r, \v, 0xa0) to spaces (ASCII 0x20), compressing multiple consecutive space characters into one.
- t:replaceNulls is not used at all in OWASP CRS3. Quote from feistyduck.com/library/apache-security/online/apachesc-CHP-10.html " Apache will respond with a 404 (Not found) response to any request that contains a URL-encoded null byte in the filename." *** This may be the answer. apache Bad Request 400 occurs before modsecurity, so perhaps apache Null Byte 404 does the same.***
- That is an unothodox use of skipAfter. It is usually used to skip blocks of rules that have not been configured to be used. Not saying that it doesn't work the way you intended though.
- @pmFromFile and @pm both do a partial case-insensitive match of their listed phrases. End of line markers (both LF and CRLF) will be stripped from each phrase and any whitespace trimmed from both the beginning and the end.
0 -
Hey, What an awesome answer! I learned several new things about mod_sec - thanks! So - I have dropped your rule in place of mine and will monitor for the next few days. And the switching between 403/404 does seem to be apache's decision as there is nothing in the rule's custom error log for those connections that return a 404. Todays connections (each ip had 25-80 requests - showing just one of each): /usr/local/apache/domlogs/[ DOMAIN A ]:82.220.91.140 - - [24/Apr/2021:07:56:22 -0500] "GET /.htaccess.zip HTTP/1.1" 403 91 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36" /usr/local/apache/domlogs/[ DOMAIN B ]:95.111.240.163 - - [24/Apr/2021:15:29:10 -0500] "GET /.htaccess.zip HTTP/1.1" 301 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36" /usr/local/apache/domlogs/[ DOMAIN C ]:80.82.215.95 - - [24/Apr/2021:12:28:35 -0500] "GET /.htaccess.zip HTTP/1.1" 404 22 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36" /usr/local/apache/domlogs/[ DOMAIN D ]:150.95.9.124 - - [24/Apr/2021:19:41:46 -0500] "GET /.htaccess.zip HTTP/1.1" 403 91 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36" As you can see, the first attack IP / domain combo gets blocked, the next redirects (https), the next is a 404 and then the fourth gets blocked. NOTES: Block vs drop. I use drop, phase 1 for these types of rules and indeed a lot of my rules as I do not want to waste any resources on a "bad" connection. Our hacker friends on amazon / azure / google have no issues with sending 100+ requests per second to probe / inject etc. In that instance I need the connection gone asap. This is the *HUGE* issue with apache choosing a 404 for these rules, as it is then a full php/mysql processing response from wordpress to return the 404 content - and the server choking due to serving that content (when it's a more ddos like connection.) Yes, csf/lfd would block these IP's but long after they are gone and have choked the server. It would be really nice to figure out why apache chooses to 404 sometimes. Steve 0 -
And 90 seconds after I post this, the exact issue I describe happens: 1 Min Load Avg: 22.92 5 Min Load Avg: 6.46 15 Min Load Avg: 2.34 Running/Total Processes: 17/528 81.92.207.82 responsible for 412 connections, trying all domains on the server, a-z, for wlwmanifest.xml, some home pages, wp authors, xmlrpc and more in ~20 seconds. sigh. 0 -
With that data it does sound like an attack. If the IP is consistent, it would be best to just block that in the firewall. 0 -
Checking the logs, it's the same with the fuzzylogic rule, some 4040, some 403. So, it does appear to be apache making the choice of applying the rule or not And @cPRex - yes it is an attack. Once upon a time blocking at the firewall was a quick and easy solution. But now, hacks/probes can be down to one IP per attempt. The only attacks/probes that come in a block anymore are zip, wlwmanifest, (rarely) xmrpc and that's about it, everything else is usually in two's (ips). This is why I have an urgent need to figure out why apache routes some requests to 404 and others the mod sec / 403. 0 -
Firstly are the phrases... /.htaccess.zip /wp-load.zip in the file text_match/zip-attack.txt? (you have not said that they are but are using them as examples) I personally would not use the forward slash as part of the phrases to be matched (unsure of how the forward slash would be interpreted and sought after file may be at /test/.htaccess.zip) Secondly are static 403.shtml files present at the root of the domains returning 404 for these requests. The 404 response may be in response to the absence of 403.shtml file. Test the response of these sites when you request domain.com/wp-load.zip from your browser. (try https and http because redirects may confuse issue) Put a 403.shtml file in place and test again. Thirdly you may get more information about the problem requests by logging them to modsec_audit.log then examining the full request. You could try... SecRule REQUEST_FILENAME "\.zip" "msg:'*** GPH zip 404', id:25002, phase:3, pass, t:none, auditlog, log, chain" SecRule RESPONSE_STATUS "404" "t:none"0 -
Firstly - yes, you are correct on the pattern. For example, a wlwmanifest.xml attack-probe will try /blog/wlwmanifest.xml, /site/wlwmanifest.xml, /sito/wlwmanifest.xml, /wp/wlwmanifest.xml and so on. So yes, I want to catch all those probes, and, by habitually adding a slash I am making the text match just that bit more specific. Secondly - thank you! You are spot on with the 403.shtml files. Quite a few of my sites go back though many servers (over 2 decades in some cases) and at some point I must have missed setting up the shtml files in the skel directory, so yes, some sites were missing all those files. I just became blind to it. Fixed - yay! Thirdly - this is fixed/not-needed as the shtml files was the issue - but I will make a note of this for other debugging. Thanks so much for suggesting the shtml fix. Steve 0
Please sign in to leave a comment.
Comments
10 comments