Lots of 404 requests that should be made to domain-A made to domain-B?
I just spent 10 minutes composing a post and when I hit Submit, the whole form reset and nothing got posted. Are you kidding me?
Basically, in access_log files, I discovered that some requests that should be made to domain-A.com are instead incorrectly made to our main SSL host domain. It's not an isolated case as many different IP addresses are logged doing these requests. I also looked for those IP addresses in abuse databases online and no one shows as reported. They're all clean.
The website domain-A.com works perfectly fine for me from my cell phone network or from my office network. No issue whatsoever.
I tried for a day to switch our main SSL host to another domain and lo and behold, those incorrect requests were now all shifted towards that new main SSL host (still incorrectly of course, since they should really be requested to domain-A.com)
So the question I have is: How/Why is this happening that every day, random IP addresses that have never been reported to any abuse databases request many pages to domain-A.com when they should be requested to domain-B.com ...and should I care that the logs contain many occurrences of those every day?
-
Hey hey! So the issue isn't that the main SSL host isn't working properly, but just that you're seeing odd traffic to the sites? In your original scenario before the switch, did domainA.com have a valid SSL certificate? If not, that's exactly the behavior I would expect, as Apache will serve the first secure vhost as the "best match."
Is that what you were experiencing?
As far as if you should be concerned, probably not. There's always going to be odd traffic in public webserver logs. If you'd like to try and mitigate some of that, see if there are any commonalities - for example, if all the traffic is coming from outside your home country, you could setup a country block in the firewall if you know you shouldn't be getting much international traffic.
0 -
SSL certs work flawlessly for me and for anybody I asked around with all sorts of ISP. To answer your first question: Yes, all our domains always have a valid SSL cert. It's issued by AutoSSL and I double-checked over the last few weeks and they all are fine.
The issue is that if our main SSL host is domain-A.com, then I will see many requests made to domain-A.com/flower-shop.html while that flower-shop.html page does NOT EXIST on domain-A.com but it DOES EXIST on domain-B.com (Note that all those requests are made from random/non-malicious IP addresses from ISP, not from shady cloud server farms. Those are easy to spot as they request 100 non-existent .php scripts like password.php or l.php or whatever, I'm not referring to that, I'm referring to very obviously good pages like flower-shop.html that DO exist on one of the domains that we host. By "very obvious" I mean that the URL will often contain very explicit brand names or even the owner's real name.)
After switching the main SSL host from domain-A.com to domain-C.com then lots of requests from other/random IP addresses are made to domain-C.com/flower-shop.html and again flower-shop.html only exists on domain-B.com not on any other domain.
So whatever the main SSL host is, there will be logs of GET /flower-shop.html 404 made every day for it. The day I switch the main SSL host to something else, then that something else domain will have lots of logs like GET /flower-shop.html 404 and the previous main SSL host GET /flower-shop.html 404 logs will stop.
To reply to your suggestion, unfortunately, blocking whole countries is not an option for several reasons, the first being that we do international business and the second is that it comes from several countries. I've seen some from UK, some from Canada, some from France, some from Australia even. It's everywhere. And we cannot block the whole planet. I'm thinking that perhaps SSL certs or something related to SSL is cached in *SOME* other countries ISP? (certainly not the norm, just 1% of them maybe?) And like, when the SSL certs renew every 2-3 months, perhaps they have an old/cached SSL cert that they serve to their customers and that since it fails, then the fallback of my server is to redirect the traffic to our main SSL host? I don't know how it works but what I do know is that it's bizarre that none of the IP addresses that make those requests is reported or blacklisted on abuse databases. This means, it's most likely legitimate traffic.
0 -
Thanks for the additional details - that's exactly what I needed to know.
Is it possible they are just trying random common page names? I see that in my personal logs all the time where they try and access something like "admin.php" or "login.php" or "settings.txt." I'm wondering if they just found the page about your server and some web scraper has it cached.
Either way, this doesn't sound like abnormal behavior on the server side to me based off your recent update.
0 -
Perhaps you missed a paragraph in my message above your reply.
0 -
Nope, I'm good :D
0 -
Then why do you ask "Is it possible they are just trying random common page names?"
0 -
And I also supplied the second option of "perhaps they indexed your pages somewhere and that web scraper has it cached."
It ultimately doesn't matter - since the issue isn't on the server the only thing you can do to stop incoming traffic is to block it.
0 -
I will grep my own IP address from time to time just to see if I can catch my own browser requesting legitimate web pages from the main SSL host instead of the actual domain name they're supposed to be on. I am concerned by the fact that none of the 20-ish IP addresses I've caught requesting legitimate "cross domain" pages over the last 2 weeks are listed anywhere as a scraper or malicious IP address. Odds seem next to impossible.
Moreover, if a scraper had it cached and shared the list of pages with other non-malicious IP addresses, what would explain those from requesting those pages on the main SSL host and not on the actual domain name?
0 -
Alright, so yesterday it actually happened to somebody I am in contact with and I tried to test further with them but we could not reproduce the issue. They said that they had multiple tabs open when it happened. They left the computer unattended for something like an hour (not sure how long exactly, but between 45 minutes and 1h30m) and so the laptop went to sleep.
When they came back and resumed, the issue was gone. They mentioned using Chrome as a browser. I checked the logs against their IP address and for approximately 20 minutes in a row, I could see requests that were made to the main SSL host instead of the actual domain and they of course all resulted in 404 errors. After leaving the computer unattended (sleeping) and coming back, the issue was gone and the logs show the same story: not a single 404 error from their IP address was logged afterwards.
0
Please sign in to leave a comment.
Comments
9 comments