Skip to main content

Bandwidth usage issues

jacobvw
Hi, I am having issues with bandwidth usage not being calculated for IMAP/POP traffic. I have had a looked through the maillog and each session is reporting usage in there. Any pointer on where else to look to find the issue? cpanel_and_whm: 11.86.0.18 operating_system_name: cloudlinux operating_system_version: '7.8' Thanks Jacob

Comments

11 comments

Date Votes
  • cPanelLauren
    Are you utilizing Statistics gathering in WHM>>Server Configuration>>Statistics Software Configuration? This would be the process that gathers the information on bandwidth usage for the server.
    0
  • jacobvw
    Yes this enabled for all users. Bandwidth statistics set for every 2 hours web statistics every 24 hours Do you have a log file location for this process?
    0
  • cPanelLauren
    The stats log should tell you if there was issues processing stats: /usr/local/cpanel/logs/stats_log
    The error_log
    in the same location should tell you about cPanel errors as well.
    0
  • jacobvw
    have looked through the error log, I have a bunch of disk quota exceeded messages for users who have reached their maximum disk quota and a bunch of errors like warn [Internal Warning while parsing [stdin] 477232] Error loading module API::Email - Can't locate Cpanel/API/Email.pm in @INC (you may need to install the Cpanel::API::Email module) (@INC contains: /usr/local/cpanel /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib/x86_64-linux-64int /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib /usr/local/cpanel/3rdparty/perl/530/lib/perl5/5.30.0/x86_64-linux-64int /usr/local/cpanel/3rdparty/perl/530/lib/perl5/5.30.0 /opt/cpanel/perl5/530/site_lib/x86_64-linux-64int /opt/cpanel/perl5/530/site_lib) at (eval 159) line 1. BEGIN failed--compilation aborted at (eval 159) line 1. at /usr/local/cpanel/Cpanel/LoadModule.pm line 27. Cpanel::LoadModule::_logger_warn("Error loading module API::Email - Can't locate Cpanel/API/Ema"...) called at /usr/local/cpanel/Cpanel/LoadModule.pm line 170 Cpanel::LoadModule::_modloader("API::Email") called at /usr/local/cpanel/Cpanel/LoadModule.pm line 109 Cpanel::LoadModule::loadmodule("API::Email") called at /usr/local/cpanel/Cpanel/API/StatsBar.pm line 50 Cpanel::API::StatsBar::get_stats(Cpanel::Args=HASH(0x4c73410), Cpanel::Result=HASH(0x4c6b4c8)) called at /usr/local/cpanel/Cpanel/API.pm line 342 eval {...} called at /usr/local/cpanel/Cpanel/API.pm line 344 Cpanel::API::_run_module_function(Cpanel::Args=HASH(0x4c73410), Cpanel::Result=HASH(0x4c6b4c8), "StatsBar", "get_stats") called at /usr/local/cpanel/Cpanel/API.pm line 222 Cpanel::API::execute("StatsBar", "get_stats", HASH(0x3dde378)) called at /usr/local/cpanel/Whostmgr/API/1/Cpanel.pm line 580 Whostmgr::API::1::Cpanel::_run_uapi_for_public("jejamanc", "StatsBar", "get_stats", HASH(0x3dde378)) called at /usr/local/cpanel/Whostmgr/API/1/Cpanel.pm line 556 Whostmgr::API::1::Cpanel::__ANON__() called at /usr/local/cpanel/Cpanel/AccessIds/ReducedPrivileges.pm line 93 eval {...} called at /usr/local/cpanel/Cpanel/AccessIds/ReducedPrivileges.pm line 93 Cpanel::AccessIds::ReducedPrivileges::call_as_user("jejamanc", CODE(0x4c6b750)) called at /usr/local/cpanel/Whostmgr/API/1/Cpanel.pm line 556 Whostmgr::API::1::Cpanel::uapi_cpanel(HASH(0x3dde378), Whostmgr::API::1::Utils::Metadata=HASH(0x3dde2a0), HASH(0x3dde198)) called at whostmgr/bin/xml-api.pl line 3750 whostmgr::bin::xml_api::__ANON__(Whostmgr::API::1::Utils::Metadata=HASH(0x3dde2a0), HASH(0x3dde378), HASH(0x3dde198), undef) called at /usr/local/cpanel/Whostmgr/API/1/Data/Wrapper.pm line 234 Whostmgr::API::1::Data::Wrapper::__ANON__() called at /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib/Try/Tiny.pm line 97 eval {...} called at /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib/Try/Tiny.pm line 88 Try::Tiny::try(CODE(0x3dde498), Try::Tiny::Catch=REF(0x3dd3b08)) called at /usr/local/cpanel/Whostmgr/API/1/Data/Wrapper.pm line 253 Whostmgr::API::1::Data::Wrapper::execute_internal(CODE(0x4c4c210), HASH(0x3dde378), HASH(0x3dde198), HASH(0x3dde288), undef) called at whostmgr/bin/xml-api.pl line 3925 whostmgr::bin::xml_api::runapp("uapi_cpanel", HASH(0x3dde198), HASH(0x3dde378), 1, undef) called at whostmgr/bin/xml-api.pl line 4001 eval {...} called at whostmgr/bin/xml-api.pl line 4001 whostmgr::bin::xml_api::batch_runapp(HASH(0x17e6b30), Whostmgr::API::1::Utils::Metadata=HASH(0x17df9c0), HASH(0x3d433d8)) called at whostmgr/bin/xml-api.pl line 3750 whostmgr::bin::xml_api::__ANON__(Whostmgr::API::1::Utils::Metadata=HASH(0x17df9c0), HASH(0x17e6b30), HASH(0x3d433d8), CODE(0x3cdfc50)) called at /usr/local/cpanel/Whostmgr/API/1/Data/Wrapper.pm line 234 Whostmgr::API::1::Data::Wrapper::__ANON__() called at /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib/Try/Tiny.pm line 97 eval {...} called at /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib/Try/Tiny.pm line 88 Try::Tiny::try(CODE(0x3d13b28), Try::Tiny::Catch=REF(0x3d13cc0)) called at /usr/local/cpanel/Whostmgr/API/1/Data/Wrapper.pm line 253 Whostmgr::API::1::Data::Wrapper::execute_internal(CODE(0x3d42e98), HASH(0x17e6b30), HASH(0x3d433d8), HASH(0x3d430d8), CODE(0x3cdfc50)) called at whostmgr/bin/xml-api.pl line 3925 whostmgr::bin::xml_api::runapp("batch", HASH(0x3d433d8), HASH(0x16d13d8), 0, CODE(0x3cdfc50)) called at whostmgr/bin/xml-api.pl line 3619 whostmgr::bin::xml_api::script(CODE(0x3cdfc50), "-json", "./batch") called at whostmgr/bin/xml-api.pl line 3562
    stats_log does not contain any errors but a bunch of warnings like the follow [2020-05-04 23:05:25 +1200] [dologs] execute: analog for user: USERNAME, log: /etc/apache2/logs/domlogs/DOMAINNAME-ssl_log.bkup. [2020-05-04 23:05:25 +1200] /usr/local/cpanel/3rdparty/bin/analog: analog version 6.0/Unix [2020-05-04 23:05:25 +1200] /usr/local/cpanel/3rdparty/bin/analog: Warning M: Cache file [2020-05-04 23:05:25 +1200] /home/USERNAME/tmp/analog/ssl/DOMAINNAME/cache doesn't contain [2020-05-04 23:05:25 +1200] last-seven-day statistics; so these may be underestimated [2020-05-04 23:05:25 +1200] (For help on all errors and warnings, see docs/errors.html) [2020-05-04 23:05:26 +1200] /usr/local/cpanel/3rdparty/bin/analog: Warning R: Turning off empty Virtual [2020-05-04 23:05:26 +1200] Host Report
    0
  • cPanelLauren
    The quota errors should be addressed - if your users don't have any space left they won't be able to store files. For stats it looks like the stats cache files don't have data for the last 7 days which would align with the issues you're having. Is there anything in those logs that indicate that they haven't been able to run for X number of days? Do you get any errors when running the following: /scripts/runstatsonce
    0
  • jacobvw
    Yes, I understand it will not work correctly on users with quota issues but I am also having issues with accounts that do not. Most of the accounts were transferred with the transfer tool in WHM if that could cause anything? When running /scripts/runstatsonce i get the following ==> cPanel Log Daemon version 26.0 ==> Ignoring lastrun files and running all stats now ==> cpanellogd will exit after stats have run
    This then generates logging data in the stats_log. I have captured the entire log for a domain without quota errors and that is not reporting IMAP/POP traffic [2020-05-06 08:14:33 +1200] Disk Usage for USERNAME on /dev/sdb1 (26781744/31457280) [2020-05-06 08:14:33 +1200] Archive Status for USERNAME: 1 [2020-05-06 08:14:33 +1200] Processing USERNAME, fork() required to drop privs with (domains:1 domains) [2020-05-06 08:14:33 +1200] [setuid] USERNAME (uid=1148,gid=1151) [2020-05-06 08:14:33 +1200] Can't find /home/USERNAME/tmp/stats.conf ... skipping [2020-05-06 08:14:33 +1200] [dologs] execute: analog for user: USERNAME, log: /etc/apache2/logs/domlogs/DOMAINNAME-ssl_log.bkup. [2020-05-06 08:14:33 +1200] /usr/local/cpanel/3rdparty/bin/analog: analog version 6.0/Unix [2020-05-06 08:14:33 +1200] /usr/local/cpanel/3rdparty/bin/analog: Warning M: Cache file [2020-05-06 08:14:33 +1200] /home/USERNAME/tmp/analog/ssl/DOMAINNAME/cache doesn't contain [2020-05-06 08:14:33 +1200] last-seven-day statistics; so these may be underestimated [2020-05-06 08:14:33 +1200] (For help on all errors and warnings, see docs/errors.html) [2020-05-06 08:14:33 +1200] /usr/local/cpanel/3rdparty/bin/analog: Warning R: Turning off empty Virtual [2020-05-06 08:14:33 +1200] Host Report [2020-05-06 08:14:33 +1200] /usr/local/cpanel/3rdparty/bin/analog: Warning R: In Request Report, turning [2020-05-06 08:14:33 +1200] off pie chart because no wedge large enough [2020-05-06 08:14:33 +1200] [dologs] execute: awstats for user: USERNAME, log: /etc/apache2/logs/domlogs/DOMAINNAME-ssl_log.bkup. [2020-05-06 08:14:34 +1200] Create/Update database for config "/home/USERNAME/tmp/awstats/ssl/awstats.DOMAINNAME.conf" by AWStats version 7.7 (build 20180105) [2020-05-06 08:14:34 +1200] From data in log file "/etc/apache2/logs/domlogs/DOMAINNAME-ssl_log.bkup"... [2020-05-06 08:14:34 +1200] Phase 1 : First bypass old records, searching new record... [2020-05-06 08:14:34 +1200] Direct access to last remembered record is out of file. [2020-05-06 08:14:34 +1200] So searching it from beginning of log file... [2020-05-06 08:14:34 +1200] Phase 2 : Now process new records (Flush history on disk after 20000 hosts)... [2020-05-06 08:14:34 +1200] Jumped lines in file: 0 [2020-05-06 08:14:34 +1200] Parsed lines in file: 3 [2020-05-06 08:14:34 +1200] Found 0 dropped records, [2020-05-06 08:14:34 +1200] Found 0 comments, [2020-05-06 08:14:34 +1200] Found 0 blank records, [2020-05-06 08:14:34 +1200] Found 0 corrupted records, [2020-05-06 08:14:34 +1200] Found 0 old records, [2020-05-06 08:14:34 +1200] Found 3 new qualified records. [2020-05-06 08:14:34 +1200] [dologs] execute: webalizer for user: USERNAME, log: /etc/apache2/logs/domlogs/DOMAINNAME-ssl_log.bkup. [2020-05-06 08:14:34 +1200] Webalizer V2.23-08 (Linux 3.10.0-962.3.2.lve1.5.33.el7.x86_64 x86_64) English [2020-05-06 08:14:34 +1200] Using logfile /etc/apache2/logs/domlogs/DOMAINNAME-ssl_log.bkup (clf) [2020-05-06 08:14:34 +1200] DNS Lookup (10): 3 addresses in 1 seconds, 3/sec [2020-05-06 08:14:34 +1200] Using DNS cache file /home/USERNAME/tmp/webalizer/ssl/DOMAINNAME/dns_cache.db [2020-05-06 08:14:34 +1200] Creating output in /home/USERNAME/tmp/webalizer/ssl/DOMAINNAME [2020-05-06 08:14:34 +1200] Hostname for reports is 'DOMAINNAME' [2020-05-06 08:14:34 +1200] Reading history file... webalizer.hist [2020-05-06 08:14:34 +1200] Reading previous run data.. webalizer.current [2020-05-06 08:14:34 +1200] Saving current run data... [05/06/2020 06:02:53] [2020-05-06 08:14:34 +1200] Generating report for May 2020 [2020-05-06 08:14:34 +1200] Saving history information... [2020-05-06 08:14:34 +1200] Generating summary report [2020-05-06 08:14:34 +1200] 3 records in 1 seconds, 3/sec [2020-05-06 08:14:34 +1200] The system has archived any ModSecurity logs.
    0
  • cPanelLauren
    How long have the accounts been on the server though, if they're sending/receiving mail on the server there should be *some* data. If you look at the raw logs at /etc/apache2/logs/domlogs/ is there anything there for the IMAP/POP data in the bytes logs? These is noted in the maillogs as well at /var/log/maillog
    It looks like the following: (for the ccs user is the same as for the rest of the users) May 5 16:21:00 server dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=15634, TLS, session= May 5 16:21:00 server dovecot: imap(cpanel-ccs)<15634>: Logged out in=50, out=850, bytes=50/850
    0
  • jacobvw
    They were migrated around a month ago. The domain I posted logs for in the previous post does not have a bytes log. That domain does generate a lot of IMAP traffic ~100gb/month Another domain does with the following 1588748803 489 . 1588748803 15846 . 1588748837 420 . 1588748837 56168 . 1588749155 1037 . 1588749155 5504 . 1588749153 893 . 1588749153 6936 . 1588750395 1037 . 1588750395 5504 . 1588750393 543 . 1588750393 11876 .
    dovecot is correctly reporting bytes as shown in your example
    0
  • cPanelLauren
    I think this might be best looked into by our analysts. Can you please open a ticket using the link in my signature? Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved. Thanks!
    0
  • jacobvw
    Thanks for your help ticket #93480980
    0
  • cPanelLauren
    Thanks for that - I'll note the ticket as well with what we've gone over.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post