Skip to main content

cpanelsolr high resource usage after recent cPanel update

Comments

42 comments

  • Stefan Devroey

    I can confirm i have the exact same problem.  Almalinux 8.10. Did not find a solution yet.

    0
  • mtindor

    Even though some of you still have SOLR running but with high usage, versus SOLR not starting at all, did you downgrade your tzdata-java per the instructions in this thread?

    https://support.cpanel.net/hc/en-us/community/posts/33211716182679

    If it were me I'd stop SOLR, follow the procedure to downgrade and versionlock tzdata-java, and then restart SOLR and see if fixes it.

    0
  • Stefan Devroey

    SOLR was working (so not stopped) on tzdata-java v11 jre-11 after the patch. But using high resources.

    Reverted back to Jre-1.8.0 and restarted Solr bur remains using high resources

     

     

    0
  • tui

    I also followed the steps provided in CPANEL-48037 and downgraded the tzdata-java package and this did not resolve the issue.

    I then restarted the entire server to ensure everything was reloaded cleanly, but the behavior persisted.

    After that, I left the cpanelsolr process running to monitor it for a longer period. As of now, it has been running for over 27 hours continuously with the same excessive resource usage. The process does not appear to cycle or release — it just stays active indefinitely

    0
  • cPRex Jurassic Moderator

    Hey all!  I'm not finding much on my end related to high resource usage after the change, so it would be best to create a ticket so we can see this problem in action.

    0
  • Jose Dieguez

    I can confirm the SOLR issue on many of our servers

     

    clients reporting the search is not working

    and SOLR using 1 full core permanently.

    0
  • cPRex Jurassic Moderator

    Jose Dieguez - are you able to make a ticket?

    0
  • Stefan Devroey

    In the meantime, I have two servers experiencing the exact same problem. Unfortunately, I can't open a support ticket directly with cPanel, as I purchased the license through my hosting reseller.

    0
  • cPRex Jurassic Moderator

    Your hosting provider should be the first point of contact for any issues, and then they will escalate the issue to us if necessary.

    So far I do not have a server I can reproduce this on so even though several of you are reporting this we still haven't seen it in action on our end so we can't fix it.

    0
  • Stefan Devroey

    Yes, I find this a rather odd approach to providing effective support. My hosting provider simply delivers the dedicated VPS and licenses. Expecting me to go through their support, only for them to relay the issue back to you, seems highly inefficient — not to mention far from customer-friendly. It’s comparable to Microsoft suggesting that Windows errors should be handled by the laptop manufacturer.

    That said, I currently have two servers experiencing this issue, and I’ve seen at least three other cPanel users reporting the same problem. Hopefully they are able to submit a ticket. I’d be more than willing to assist, but if direct contact isn't permitted, there's little more I can do.

    0
  • Jose Dieguez

    cPRex

    Managed to fix it reinstalling solr on the 3 cases i have seen.

     

    Sadly, when this case arises, the client needs the fix in the moment, so we can't leave the system like that to create a ticket and wait the reply :(

    0
  • cPRex Jurassic Moderator

    Jose Dieguez - that's good to know, though!

    Stefan Devroey - unfortunately that's how the license system works.  You're always welcome to purchase the cPanel license directly from us and then you can contact us directly through WHM if you run into something.  Your host *should* be offering you good support, though, although I understand that doesn't always happen.

     

    0
  • Stefan Devroey

    Jose Dieguez How did you reinstall ? I tried this by going to WHM - Cpanel - Manage Cpanel plugin and Uninstall Solr. Reboot the server and did the same and install Solr here. But the problem remains after this.

     

    0
  • Jose Dieguez

    Stefan Devroey

     

    after that we ran

    # /scripts/restartsrv_cpanel_dovecot_solr --stop 
    # /scripts/restartsrv_cpanel_dovecot_solr --start
    # /scripts/rescan_user_dovecot_fts --user cpaneluser

    that fixed our specific issue in these cases (client was able to search again, and solr used high cpu for a short period during the reindex)

    1
  • tui

    Jose Dieguez

    So you need to run the rescan_user_dovecto_fts for each user on the server? 

    Unfortunately, I'm in the same boat — my license is issued by my VPS provider, so I can't open a ticket directly

    0
  • Stefan Devroey

    Jose Dieguez No luck for us. This did not solve the problem here. Do you mind sharing your Solr commandline ?

    For us this is : /usr/lib/jvm/jre-11/bin/java -server -Xms512m -Xmx512m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+ExplicitGCInvokesConcurrent -Xlog:gc*:file=/home/cpanelsolr/server/logs/solr_gc.log:time,uptime:filecount=9,filesize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/home/cpanelsolr/server/logs -Djetty.port=8984 -DSTOP.PORT=7984 -Dhost=127.0.0.1 -Duser.timezone=UTC -XX:-OmitStackTraceInFastThrow -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/home/cpanelsolr/server/logs/jvm_crash_%p.log -Djetty.home=/home/cpanelsolr/server -Dsolr.solr.home=/home/cpanelsolr/server/solr -Dsolr.install.dir=/home/cpanelsolr -Dsolr.install.symDir=/home/cpanelsolr -Dsolr.default.confdir=/home/cpanelsolr/server/solr/configsets/_default/conf -Xss256k -Djava.security.manager -Djava.security.policy=/home/cpanelsolr/server/etc/security.policy -Djava.security.properties=/home/cpanelsolr/server/etc/security.properties -Dsolr.internal.network.permission=* -DdisableAdminUI=false -Dlog4j2.formatMsgNoLookups=true -Dsolr.autoSoftCommit.maxTime=3000 -Dsolr.log.muteconsole -jar start.jar --module=http --module=requestlog --module=gzip


    It seems there was an update on Cpanel Solr but the rebuilding of the cache did also nothing for us : cPanel Dovecot Solr search does not work after the update to version 9.8+ – cPanel

    0
  • tui

    Stefan Devroey Jose Dieguez

    Seems that its resolved with this:

    https://support.cpanel.net/hc/en-us/articles/33251591779863-cPanel-Dovecot-Solr-search-does-not-work-after-the-update-to-version-9-8

    I just followed the workaround and it seems that it fix it as i can see less cpanelsolr processes and they are dissapearing, i will monitor the usage on the next hours


    0
  • tui

    Forget everything, that didn’t solve the problem either — it did exactly the same thing again.

    0
  • James Birchmore

    Just joining the discussion to say we have the exact same issue, and haven't been able to resolve it.

    Command line is

     /usr/lib/jvm/jre-11/bin/java -server -Xms512m -Xmx512m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+ExplicitGCInvokesConcurrent -Xlog:gc*:file=/home/cpanelsolr/server/logs/solr_gc.log:time,uptime:filecount=9,filesize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/home/cpanelsolr/server/logs -Djetty.port=8984 -DSTOP.PORT=7984 -Dhost=127.0.0.1 -Duser.timezone=UTC -XX:-OmitStackTraceInFastThrow -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/home/cpanelsolr/server/logs/jvm_crash_%p.log -Djetty.home=/home/cpanelsolr/server -Dsolr.solr.home=/home/cpanelsolr/server/solr -Dsolr.install.dir=/home/cpanelsolr -Dsolr.install.symDir=/home/cpanelsolr -Dsolr.default.confdir=/home/cpanelsolr/server/solr/configsets/_default/conf -Xss256k -Djava.security.manager -Djava.security.policy=/home/cpanelsolr/server/etc/security.policy
    -Djava.security.properties=/home/cpanelsolr/server/etc/security.properties -Dsolr.internal.network.permission=* -DdisableAdminUI=false -Dlog4j2.formatMsgNoLookups=true -Dsolr.autoSoftCommit.maxTime=3000 -Dsolr.log.muteconsole -jar start.jar --module=http --module=requestlog
    --module=gzip

    1
  • cPRex Jurassic Moderator

    James Birchmore - have you submitted a ticket on the issue?

    0
  • James Birchmore

    cPRex Not directly, as I also have a partner supported license

    0
  • ruzbehraja

    We have exactly the same issue on 2 out of 3 similar config servers. 

    0
  • tui

    Was anyone able to resolve this?

    0
  • Stefan Devroey

    Still the same problem here.

    No ticket has been raised. No possible solution in the release notes for upcoming versions as well.

    0
  • Petri Hind

    Just confirming that I have the same exact issue and following the topic, if somebody will find a solution. Also, I have a partner supported license, so I cannot raise a ticket directly. 

    At the time of writing this, the cpanelsolr process has exceeded the time threshold by 488307 > 7200 (seconds) & I will add the process to CSF monitoring exception list for now.

    0
  • ruzbehraja

    I have opened a ticket with Id #95758258

    1
  • ruzbehraja

    cPanel Supprt has just opened a case to the developers regarding this issue.
    For reference, the case number is CPANEL-48463.

    0
  • Stefan Devroey

    ruzbehraja Yes !! Thanks...

    0
  • cPRex Jurassic Moderator

    Thanks so much! 

    0
  • tui

    Please keep us updated!

    0

Please sign in to leave a comment.