Breaking down AWstats

As technical analysts, we often run in to various questions and issues surrounding AWstats, and how it works. For this we've created this rundown of AWstats, its files, how AWstats data is populated, and even a few quick fixes. List of AWstats files System files: [LIST]

/usr/local/cpanel/3rdparty/bin/awstats.pl - Main AWstats script.

/usr/local/cpanel/etc/awstats.conf - This is the main AWstats configuration file from which user templates are built (~/USER/tmp/awstats.domain.com.conf). If you make changes to this file, it will also change all new ~/USER/tmp/awstats.domain.com.conf files.

/usr/local/cpanel/3rdparty/share/awstats - This directory contains all of the AWstats images files, the copyright file, and the language files. You can upload new images for the AWstats page to the image directories, and slightly change the look of AWstats.

/etc/stats.conf - Retains changes made in WHM's Server Configuration "Statistics Software Configuration page

/scripts/restartsrv_cpanellogd - This is a script to restart the cpanellogd service which processes bandwidth and domain logs (also service level logs). This does a lot beyond logs for AWstats, but this should be mentioned because if its not running, logs will not process.

/usr/local/cpanel/base/awstats.pl - This file is a symlink to "../3rdparty/bin/awstats.pl"

/usr/local/cpanel/3rdparty/bin/js - This is a directory containing the AWstats javascript files. These are not directly used by the AWstats script, but are included in the distribution.

/usr/local/cpanel/3rdparty/bin/lang - This directory contains files for AWstats language-specific message files. This includes various translations.

/usr/local/cpanel/3rdparty/bin/lib - This directory contains the AWstats perl libraries.

/usr/local/cpanel/3rdparty/bin/plugins - This is the directory for the AWstats plugins. User files: [LIST]

/home/USER/logs/ - This contains compressed files of the logs used to generate stats. These look to be separated by service (Apache logs: domain.com-Jun-2015.gz, FTP logs ftp.domain.com-ftp_log-Jun-2015.gz, etc).

/home/$USER/tmp/awstats/ - This directory holds the AWstats data files and user configuration.

/home/USER/tmp/awstats/awstatsMMYYYY.domain.com.txt - This is the actual datafile that AWstats will use to display information. This is built using the /home/USER/tmp/awstats.domain.com.conf configuration file and and /usr/local/apache/domlogs/domain.com.bkup log.

/usr/local/apache/domlogs/domain.com - These files are built from the "CustomLog" entries for each Virtual Host in the httpd.conf. From these, the /usr/local/apache/domlogs/domain.com.bkup files are made, and then used.

/usr/local/apache/domlogs/domain.com.bkup - Made from the above file, and is actually used in the data generation. This looks to be a backup so the original is kept in tact.

/home/USER/tmp/awstats.domain.com.conf - This is the per domain configuration file built from the /usr/local/cpanel/etc/awstats.conf file when the domain is added.

/var/cpanel/users/USERNAME - This user file also contains configuration information for AWstats [LIST]

awwstatsreversedns=0 " Enabling allows AWstats to interpret domains as IPs.

Skipawstats=1- disable/enable awstast for the user

awstatsbrowserupdate=0 " disable/enable updates via browser for user. How data moves from Apache to AWstats To populate data for statistics in AWstats, you would first go to the domain in a browser. Apache recognizes the connection to this domain then uses the "CustomLog" entires in the httpd.conf, to copy the visit into the individual domlog for that domain (these look like /usr/local/apache/domlogs/domain.com). From there, the daily stats processing will run (or force it with /scripts/runweblogs user). When this runs, a temporary file is created that looks like (/usr/local/apache/domlogs/domain.com.bkup), using that file in this process, along with /home/user/tmp/awstats/awstats.domain.com.conf generates a new data file that lives at /home/USER/tmp/awstats/awstatsMMYYYY.domain.com.txt. This is the data that AWstats displays. Finally, the /usr/local/apache/domlogs/domain.com.bkup merges into the current /home/user/logs/domain.com-MONTH-YEAR.gz file. You can find an easier to read version of this paragraph down below. Step-by-step instructions to get AWstats to populate with data

]
• Visit domain.com in a web browser.
• Apache uses the "CustomLog" entry to copy the visit into a /usr/local/apache/domlogs/domain.com log.
• Allow the auto stats processing to run, or force an account to run with /scripts/runweblogs USER [LIST]
• This creates a temporary /usr/local/apache/domlogs/domain.com.bkup file.
• This file is used in conjunction with the configuration file /home/user/tmp/awstats/awstats.domain.com.conf to generate the data file:
• /home/USER/tmp/awstats/awstatsMMYYYY.domain.com.txt. This is what gives your AWstats data.

/usr/local/apache/domlogs/domain.com.bkup is then merged into and saved to file /home/user/logs/domain.com-MONTH-YEAR.gz file.

Done. Common issues and Resolutions awstats.pl has incorrect permissions, preventing stats from processing AWstats is not able to run and process stats if the permissions on the /usr/local/cpanel/3rdparty/bin/awstats.pl are incorrect. The permissions on this file should be set to 755, with owner:group of root:root root@cpanel01 [~]# ls -alh /usr/local/cpanel/3rdparty/bin/awstats.pl -rwxr-xr-x 1 root root 667K Jan 27 2015 /usr/local/cpanel/3rdparty/bin/awstats.pl*
CustomLog entries commented out (aka. AWstats not generating data) AWstats was not generating any data. After investigating,, it was found that the "CustomLog" entries were commented out in a custom /var/cpanel/templates/apache2/vhost.local and /var/cpanel/templates/apache2/ssl_vhost.local. This will not allow apache to separate the logs into its own file, thus stats processing has nothing to process. Heres how they were commented out: [% IF logstyle == 'combined' -%] [%- IF !enable_piped_logs || !supported.mod_log_config -%] #CustomLog [% paths.dir_domlogs %]/[% wildcard_safe(vhost.log_servername) %] combined [%- END %] [% ELSE %] TransferLog [% paths.dir_domlogs %]/[% wildcard_safe(vhost.log_servername) %] [% END -%] [% IF supported.mod_log_config && supported.mod_logio && !enable_piped_logs -%] #CustomLog [% paths.dir_domlogs %]/[% wildcard_safe(vhost.log_servername) %]-bytes_log "%{%s}t %I .\n%{%s}t %O ." [% END -%]
Stats processing times totally blacked out (aka AWstats not generating data) AWstats was not showing any data, due to stats processing just not running. Checking every single box in the WHM >> Server Configuration >> Statistics Software Configuration >> Statistics Schedule >> Configure Statistic Process Time Schedule page, tells AWstats which dats not to run. Checking all (any) of these will cause AWstats not to process at those times. NGINX preventing AWstats from gathering information (aka AWstats not generating data) While unsupported, this may be a workaround that will work for Nginx users for this type of no data issue. AWstats stopped providing information at the moment that Nginx was installed. This was easily fixed by doing the following:

]
• Enable Piped Apache Logs - WHM "Service Configuration "Apache Configuration "Piped Log Configuration. Check the box to enable, and save
• Rebuild Vhosts - This is from within the Nginx configuration, not apache.
• Restart Nginx - Self explanatory
• Restart Apache
• If this doesnt work you can try lucasrolff.com/nginx/nginx-support-for-cpanel-awstats/

Active by default not set This one is incredibly simple, but easy to miss. AWstats will not out of the box on all accounts without setting "Active by default" for the accounts. 301 Redirect causing no data Another simple yet missed issue is, AWstats was not generating data for one single domain. It ends up that the domain had a 301 redirect directing to another place. This is actually still shown, but ends up showing up under the "HTTP Status Codes" in AWstats. Comodo CAWF may prevent httpd.conf from rebuilding properly, and not writing to domlogs In this case, CAWF seemed to allow httpd.conf to rebuild, but without proper syntax. Webstats probe found this HTTPD CONF: Syntax Errors (Run: httpd configtest) *** This means that Apache can't do a graceful restart and that the domlogs will be 0 bytes in size, so therefore no new stats will be processed until httpd.conf is fixed! ***
Trying to rebuild httpd.conf manually, gave further errors regarding COMODO CAWF Configuration problem detected on line 31 of file /var/cpanel/cwaf/rules/cwaf_01.conf: ModSecurity: Found another rule with the same id
After fixing this, domlogs started being generated again. Thank you for taking the time to read through, and I hope this is useful for everyone who needs it. Feel free to leave comments, questions, or anything extra that you believe should be a part of this.