Skip to main content

mod_remoteip and correct ip in vhost access logs

Oderland David
Good morning from Sweden :) We've decided to use mod_remoteip instead of mod_rpaf to translate the XFF header (apache 2.4) However, mod_remoteip uses the %a log format instead of the %h to get the client IP. If there is no XFF header then %a (or if it's not trusted) then %a equals %h. So, this is fixable in httpd.conf by changing these %h into %a.
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost LogFormat "%v %{%s}t %I .\n%v %{%s}t %O ." bytesvhost LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent CustomLog "|/usr/local/cpanel/bin/splitlogs --main=box01.oderland.com --suffix=-bytes_log" bytesvhost CustomLog "|/usr/local/cpanel/bin/splitlogs --main=box01.oderland.com --mainout=/usr/local/apache/logs/access_log" combinedvhost
so they'll be like this:
LogFormat "%v:%p %a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost LogFormat "%v %{%s}t %I .\n%v %{%s}t %O ." bytesvhost LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%a %l %u %t \"%r\" %>s %b" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent CustomLog "|/usr/local/cpanel/bin/splitlogs --main=box01.oderland.com --suffix=-bytes_log" bytesvhost CustomLog "|/usr/local/cpanel/bin/splitlogs --main=box01.oderland.com --mainout=/usr/local/apache/logs/access_log" combinedvhost
However, after distilling and rebuilding httpdconf, one of the entries above reverts to %h:
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost
That's of course the logformat that controls virtual host logs. I've had this issue up with cPanel twice already during the last year. They couldn't solve it and asked me to file a feature request. I've since added the whole part of the config in /usr/local/apache/conf/includes/pre_virtualhost_global.conf to overwrite the incorrect one, but it feels like a hack (since cPanel could change the template in the future or decide to move the splitlogs binary). Has anyone else solved it in a better way? Care to share how?

Comments

14 comments

Date Votes
  • DavidN.
    Hi there, Can you check the directive in /var/cpanel/conf/apache/main ? By default, it looks like this:
    ifmodulelogiomodule: logformat: directive: logformat items: - logformat: "\"%h %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\" %I %O\" combinedio" logformat: directive: logformat items: - logformat: "\"%h %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\"\" combined" - logformat: "\"%h %l %u %t \\\"%r\\\" %>s %b\" common"
    You should be able to just change %h to %a, then run /scripts/rebuildhttpdconf and have it do what you want. Please let me know if that doesn't work. Please be careful editing this file; it's YAML, which is very sensitive to whitespace changes. Edit: I fail at formatting. Make sure you preserve the indentation in this file, or you may get a broken Apache configuration.
    0
  • Oderland David
    Hi David! Thanks for getting back to me. I've been down this road before ;) The
    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost
    is missing from the yaml conf. I just realized I sent you guys a feature request January 2015. Here's what I wrote in the ticket about it: I found the source - it would seem that the LogFormat line is hardcoded in: /usr/local/cpanel/Cpanel/AdvConfig/apache.pm Therefor it leaves no way for us to change the LogFormat to be used with mod_remoteip module for domlogs (using split logs). I've made an diff which i'm sending to you below. This is in no way a complete patch as I'm just changing one hard coded value for another, but I just want to show what we need to change for mod_remoteip to work for domlogs. The problem otherwise is that the IP of the proxy will get blocked in the firewall due to CSF/LFD or cpHulk when we're checking modsec or our own LFD modules. Please - make that logformat line configurable in main.default or local in /var/cpanel/conf/apache/ in case it exists.. otherwise use default if you want to. Steps to reproduce: # [/usr/local/cpanel/Cpanel/AdvConfig]# diff -c apache.pm apache.pm.new *** apache.pm 2015-01-28 14:07:43.388000008 +0100 --- apache.pm.new 2015-01-28 14:05:20.870000009 +0100 *************** *** 371,377 **** # Add new log formats. $logconf->{'logformat'}->{'items'} = [ { ! 'logformat' => q["%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost], }, ( $has_logio --- 371,377 ---- # Add new log formats. $logconf->{'logformat'}->{'items'} = [ { ! 'logformat' => q["%v:%p %a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost], }, ( $has_logio
    0
  • Oderland David
    Ticket id: 6031689
    0
  • Oderland David
    Any updates on this @cPDavidN? Is it a "feature" or a bug? ;)
    0
  • DavidN.
    Hi Internetfab, I apologize for missing your reply. The way that Apache templates and those .yaml files interact is complex and strange and, since they're generated on the server itself, comparisons between machines aren't helpful. (I've been dealing with different aspects of this subsystem for the last couple of days, so I've gotten some new insights into why my last reply was not so helpful.) Would it be possible for you to re-open your ticket and provide access to an affected server? I'd like to examine your configuration and see if I can come up with something that will work for you long-term, which will survive cPanel updates, distiller runs, and Apache configuration rebuilds.
    0
  • santrix
    We also see this problem. on 11.48.4 (build4). From /var/cpanel/conf/apache/main
    logformat: "\"%v:%p %a %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\"\" combinedvhost" - logformat: "\"%v %{%s}t %I .\\n%v %{%s}t %O .\" bytesvhost" - logformat: "\"%a %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\"\" combined" - logformat: "\"%a %l %u %t \\\"%r\\\" %>s %b\" common" - logformat: "\"%{Referer}i -> %U\" referer" - logformat: "\"%{User-agent}i\" agent"
    After /scripts/rebuildhttpdconf we see in the httpd.conf
    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost LogFormat "%v %{%s}t %I .\n%v %{%s}t %O ." bytesvhost LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%a %l %u %t \"%r\" %>s %b" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent CustomLog "|/usr/local/cpanel/bin/splitlogs --main=orcus.uksrv.co.uk --suffix=-bytes_log" bytesvhost CustomLog "|/usr/local/cpanel/bin/splitlogs --main=orcus.uksrv.co.uk --mainout=/usr/local/apache/logs/access_log" combinedvhost
    The combinedvhost entry, for some reason totally ignores the line entry in /var/cpanel/conf/apache/main - I've inserted extra spaces etc., and none of it has any effect on the LogFormat directive for combinedvhost
    0
  • cPanelMichael
    The combinedvhost entry, for some reason totally ignores the line entry in /var/cpanel/conf/apache/main - I've inserted extra spaces etc., and none of it has any effect on the LogFormat directive for combinedvhost

    Try using /var/cpanel/conf/apache/local instead of /var/cpanel/conf/apache/main in order to implement custom changes. You will need to create "/var/cpanel/conf/apache/local" if it does not yet exist. Thank you.
    0
  • santrix
    Try using /var/cpanel/conf/apache/local instead of /var/cpanel/conf/apache/main in order to implement custom changes.

    While both files look like YAML, they appear to provide quite different functions /var/cpanel/conf/apache/local - appears to hold the settings from WHM "Service Configuration "Apache Configuration "Global Configuration /var/cpanel/conf/apache/main - appears to hold the settings that are directly translated into httpd.conf I'm taking a wild guess, but I doubt adding the following to our pre-existing local file is going to help - or would probably break a rebuild (I haven't tested)
    --- defaultvhost: userdirprotect: '' main: ifmodulemodlogconfigc: customlog: directive: customlog items: - format: --main=ourhost.co.uk --suffix=-bytes_log" bytesvhost target: "\"|/usr/local/cpanel/bin/splitlogs" - format: --main=ourhost.co.uk --mainout=/usr/local/apache/logs/access_log" combinedvhost target: "\"|/usr/local/cpanel/bin/splitlogs" logformat: directive: logformat items: - logformat: "\"%v:%p %a %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\"\" combinedvhost" - logformat: "\"%v %{%s}t %I .\\n%v %{%s}t %O .\" bytesvhost" - logformat: "\"%a %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\"\" combined" - logformat: "\"%a %l %u %t \\\"%r\\\" %>s %b\" common" - logformat: "\"%{Referer}i -> %U\" referer" - logformat: "\"%{User-agent}i\" agent" sections: ifmodulemodlogconfigc: supported: mod_log_config: 1
    0
  • santrix
    For the time being - the only way around this is to directly edit the following files...
    sed -i.original -e 's/\(logformat.*\)%h\(.*\)/\1%a\2/' /var/cpanel/conf/apache/main sed -i.original -e 's/\(.*\)%h \(.* combinedvhost.*\)/\1%a \2/' /usr/local/cpanel/Cpanel/AdvConfig/apache.pm
    This will ensure that %h is replaced with %a for the combinedvhost LogFormat entry (in addition to the other LogFormat entries) in httpd.conf This will get overwritten after a upcp, so you may want to put these instructions in a script and add that script to /usr/local/cpanel/scripts/postupcp
    0
  • santrix
    It's probably more tidy to put them into a short bash script and pop that into /usr/local/cpanel/scripts/postupcp instead. If you want the changes to take effect immediately then you will need to run those two sed commands, and then run /scripts/rebuildhttpdconf, and /scripts/rebuildnginxvhost, and then service httpd restart. It should be noted that this technique has been tested and works with 11.54 (Build 19), but you know how cPanel change things ;-)
    0
  • santrix
    Then, check /usr/local/apache/conf/httpd.conf for each of the LogFormat directives to ensure that the %h options have been converted to %a. If so, then it may be that your nginx reverse proxy isn't adding the X-Real-IP header for the correct server IP addresses, or maybe you aren't loading the remoteip module... Depending on your implementation this could be in various places. Check it's loaded by running /usr/local/apache/bin/apachectl -M and look for remoteip_module in the output. If it is loaded then you need to ensure that your server's local IP addresses are configured as RemoteIPInternalProxy's with the following directives - these could be in your httpd.conf or in an include file depending on your setup. You should fine one or more lines like this, with your IP addresses listed: RemoteIPInternalProxy x.x.x.x If you have all that, then it's going to be quicker to let cPanel take a look ;)
    0
  • AnthonyFrancis
    The custom_opt_mod-remoteip.tgz seems to be missing from the downloads. The download link 404's.
    0
  • AnthonyFrancis
    Thanks for fixing the link (whomever did it)!
    0
  • cPanelMichael
    Thanks for fixing the link (whomever did it)!

    I'm happy to see it's working well. It may have been a temporary server issue as I don't see any reports that a case was opened to address the problem. Let us know if you encounter any additional issues with the download. Thanks!
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post