apache master/custom/global access log
Recently I had to rebuild a WHM server due to filesystem corruption. I was able to recover most of the functionality/customisation (the cpanel backup is bloody useless btw, will never ever rely on it again).
The version is WHM 11.44.0 (build 18).
One problem is I cannot replicate one customisation (even though it works on other WHM machines that we run).
The customisation in question is combined apache access log.
Right now it is real pain to monitor abuse on the machine, due to no central/master apache log. I cannot tail domlogs due to there are thousands of accounts on each machine.
The way I was able to have combined log is by creating/editing /var/cpanel/templates/apache2/main.local
and adding the following:
just after this:
and before this:
Basically I have replicated what we have on other multiple machines (the machine that the combined log works is running WHM 11.38.2 (build 23) ). It does not work on new machine. The only entires that are being logged are those that are not in virtual hosts.... Basically what I am after is a central/master/combined apache access log, so I can tail it and see which domain get what requests from which ip. I have searched this forum for this, and documentation, there is a lot of misleading information, most of the forum posts are simply unanswered or answered incorrectly (eg I am not after /usr/local/apache/domlogs/, nor I am after /usr/local/apache/logs/access_log). I don't understand how people get away without having one? Do they not care about abuse? This log is a must when running shared hosting. It should be on by default. So the question is: what is the correct way of setting up the combined apache log for all vhosts?
LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon
CustomLog /var/log/httpd/vhost-access_log vcommonjust after this:
[% FOREACH dir IN main.ifmodulelogconfigmodule.ifmodulelogiomodule.logformat.items -%]
LogFormat [% dir.logformat %]
[% END -%]
and before this:
[% FOREACH dir IN main.ifmodulealiasmodule.scriptalias.items -%]
ScriptAlias [% dir.url %] [% dir.path %]
[% END -%]
Basically I have replicated what we have on other multiple machines (the machine that the combined log works is running WHM 11.38.2 (build 23) ). It does not work on new machine. The only entires that are being logged are those that are not in virtual hosts.... Basically what I am after is a central/master/combined apache access log, so I can tail it and see which domain get what requests from which ip. I have searched this forum for this, and documentation, there is a lot of misleading information, most of the forum posts are simply unanswered or answered incorrectly (eg I am not after /usr/local/apache/domlogs/, nor I am after /usr/local/apache/logs/access_log). I don't understand how people get away without having one? Do they not care about abuse? This log is a must when running shared hosting. It should be on by default. So the question is: what is the correct way of setting up the combined apache log for all vhosts?
-
[quote="SergeiF, post: 1676071">Right now it is real pain to monitor abuse on the machine, due to no central/master apache log. I cannot tail domlogs due to there are thousands of accounts on each machine.
Hello :) To clarify, is the goal here to have one log file with the data from all of your Apache domlogs combined? If so, you could develop a bash script that combines all of the Apache domlogs into one single log file after the initial logging. Feel free to clarify or let us know if that's not what you are after. Thank you.0 -
Hi, The idea is to have a "live" log of all the domains combined. The bash script will not do, as it is much more efficient to tail the log (and perhaps grep for stuff of interest). Sometimes there is no time before server runs out of memory due to DoS. In anyway, while my message was being aproved by moderator I figured out the answer: The correct place to put the log line is in /var/cpanel/templates/apache2/vhost.local (cp from vhost.default): ........... LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon CustomLog /var/log/httpd/vhost-access_log vcommon
It can be placed right before closing tag in the file. The ssl_vhost.local should have similar done. Don't forget to configure logrotate to rotate the logs. Now you can tail the access log for all sites to figure which site or IP is affecting your server. Very easy to spot for example wordpress or joomla bruteforces and deal with them (be that botnet or a single machine). This log can also be used as signle watch point for stuff like fail2ban (to deal with bruteforces above) or similar. I hope this will be helpfull for someone in the future. I wander sometimes why this is not done by default. Logs are very important for security purposes. I am not the first one to ask for it: All these were unaswered. Another problem I found is that the WHM will delete all the access logs without archival, once the stats were processed by default. We have disabled that anti-feature.0 -
Hello :) I am happy to see you were able to resolve the issue. Thank you for updating this thread with the solution. Note that you are welcome to submit a feature request to have this behavior utilized by default: Submit A Feature Request Thank you. 0 -
You Sir, are a champion! 0 -
isn't /usr/local/apache/logs/access_log already a combined log of requests? I was happy to find these: /usr/local/apache/logs/fpcgisock /usr/local/apache/logs/access_log /usr/local/apache/logs/error_log /usr/local/apache/logs/httpd.pid /usr/local/apache/logs/suexec_log /usr/local/apache/logs/ssl_scache.dir /usr/local/apache/logs/ssl_scache.pag /usr/local/apache/logs/suexec_log.offset /usr/local/apache/logs/suphp_log
It may also be worth looking at piped logging and ALSO this site discussing optimization Be sure to check out the cPanel config option for piped logging, too .vB0
Please sign in to leave a comment.
Comments
5 comments