What type of events or periodic task trigger Cpanel NGINX with Reverse Proxy to delete the cached files?
According to Nginx cached files are delete based on "inactive" so expired cached files based on Expires Header or proxy_cache_valid don't get deleted until "inactive" expires and in between if the page is visited the "inactive" resets.
In the Cpanel NGINX with Reverse Proxy set up there's something that keeps deleting the cached files I would say after around 6, 12 or up to 24 hours not honoring my "inactive_time" : "14400m", (10 days) here /etc/nginx/ea-nginx/cache.json that is correctly added by cpanel to each individual username conf file as part of proxy_cache_path.
I know, for example, that cpanel has some sort of task that overwrites username conf files and it seems there's a similar task (or tasks) run once or twice a day approximately that is deleting my cached files overwriting my nginx inactive=14400m ( I also tried inactive=10d in days instead of minutes same problem)
Could Cpanel share what are the events that trigger the deleting of all cached files everyday? Or if there's some sort of maximum imposed?
In my /etc/nginx/ea-nginx/cache.json I have "zone_size" : "200000m" which is 200gb per user, plenty, more than enough. I tried adding Nginx max_size, but is not considered by Cpanel NGINX with Reverse Proxy, but based on Nginx guidelines if no max_size is included Nginx won't impose a limit until the disk is full, so files don't get deleted until that point which is not by far the case.
There's definitely something that Cpanel is doing based on a task cron or so that deletes all files regardless of "inactive" which becomes a big problem specially when you have a big site with many cached files, sharing what are those tasks in which Cpanel will delete the entire cache will be very useful to know.
Thanks,
-
Hey there! I reached out to the developers about this issue since I couldn't find anything on my end, and they confirmed there is no standard scheduled clearing of the cache files that cPanel performs. As you already know, the cache entries are time-based to begin with, so Nginx will expire it automatically when it reaches that threshold. If a file changes in a way that modifies the mtime, or if an app specifically sends a "do-not-cache" header, this can alter the cached data as well. After hearing the above from the team, I don't have a good explanation for the behavior you're seeing. If you're able to reproduce this issue, it might be best to create a ticket so we can see this happening in real-time on the system. 0 -
OK, I have two tickets about the issue, one I closed (decided to keep monitoring myself) and another one currently open, but stuck with the analyst. I think is better when you contact the developers of Cpanel's NGINX with Reverse Proxy directly. In regards to processes in which that Cpanel Nginx set up might be deleting cached files, I think one is, for example, when cpanel overwrites NGINX conf as part of that process it deletes all the cache, is that right? Could you confirm that the conf files overwriting process that I think is done once or twice daily won't empty the cache? I want to make sure that if I modify a conf like for example: /etc/nginx/conf.d/users/myusername/mydomain/myconf.conf cpanel as part of the modification of conf files check process won't delete the cache. What I am looking at as a potential cause of the issue is that I noticed in the Nginx error log the following repeating like every 6 hours or so related to passenger : [ N 2023-06-05 12:11:21.1479 46764/T8 age/Cor/CoreMain.cpp:671 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown) [ N 2023-06-05 12:11:21.1480 46764/T1 age/Cor/CoreMain.cpp:1246 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected... N 2023-06-05 12:11:21.1480 46764/T8 Ser/Server.h:902 ]: [ServerThr.1] Freed 0 spare client objects [ N 2023-06-05 12:11:21.1480 46764/T8 Ser/Server.h:558 ]: [ServerThr.1] Shutdown finished [ N 2023-06-05 12:11:21.1480 46764/Te Ser/Server.h:902 ]: [ServerThr.4] Freed 0 spare client objects [ N 2023-06-05 12:11:21.1480 46764/Ti Ser/Server.h:902 ]: [ServerThr.6] Freed 0 spare client objects [ N 2023-06-05 12:11:21.1480 46764/Te Ser/Server.h:558 ]: [ServerThr.4] Shutdown finished ... [ N 2023-06-05 12:11:21.1899 2826/T1 age/Wat/WatchdogMain.cpp:1377 ]: Starting Passenger watchdog... [ N 2023-06-05 12:11:21.2053 2839/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core... [ N 2023-06-05 12:11:21.2054 2839/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode. [ N 2023-06-05 12:11:21.3224 2839/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 2839 2023/06/05 12:12:59 [alert] 3922#2968: ngx_http_pipelog_module: ngx_http_pipelog_command_exec(): SIGTERM detected, gracefully shutting down... 2023/06/05 12:12:59 [alert] 3046#3046: ngx_http_pipelog_module: exit_process: killpg(2968, SIGTERM) failed Cpanel's NGINX with Reverse Proxy relies on ruby27-mod_passenger and that error that I see in the error log here: /var/log/nginx/error.logs seems that passenger shuts down NGINX. Is passenger causing all the cached files to be deleted when I see that error? It seems to coincide when I notice I get a MISS on files that were supposed to show STALE. PER NGINX RULES NOTHING IS SUPPOSED TO BE DELETED PRIOR TO "INACTIVE" HAS EXPIRED AND I HAVE IT AS 10 DAYS, not supposed to be deleted in about 6 hours or 12 hours (approximately) Could you have the developers to take a look at a potential bug with passenger causing this issue? The problem with tickets is that they say "' all looks good" while they review the ticket they see the HIT, UPDATING and STALE, the problem is that I have the time and is supposed to last on STALE for 10 days, 6 hours later than the time they reviewed the ticket I get all MISS, really not going anywhere through the ticket system. thanks, 0 -
We can definitely leave a ticket open for an extended period of time - just let us know that needs to happen and we can give it the "on hold" status so it doesn't auto close. If you already have a ticket opened, that'll be better than any information I can provider through a Forums post. 0 -
I will really appreciate if you pass this information to the cpanel NGINX team, the ticket #95070894 seems to be abandoned The passenger issue seems to be a bug that might be affecting many websites, specially those with Nginx Inactive greater that 5 of 6 hours, specially those sites with large number of cached pages, huge impact when you think that all is OK based on your Nginx Inactive time and instead all files getting wiped out, huge impact on Google speed factor for indexation and user interaction thanks, 0 -
I did reach out to the devs about that, and they said Passenger itself would not be involved in the cache clearing. I don't see that the ticket is abandoned at all, it just isn't in the high priority queue so it's not getting immediate replies. 0 -
Any answer about this part from my prior message: "In regards to processes in which that Cpanel Nginx set up might be deleting cached files, I think one is, for example, when cpanel overwrites NGINX conf as part of that process it deletes all the cache, is that right? Could you confirm that the conf files overwriting process that I think is done once or twice daily won't empty the cache? I want to make sure that if I modify a conf like for example: /etc/nginx/conf.d/users/myusername/mydomain/myconf.conf cpanel as part of the modification of conf files check process won't delete the cache." In reference to passenger did they get to see to error log passenger sending a signal to shut down nginx, are they sure that wont delete the cache files? In reference to my ticket I don't see how that is not a priority remember Cpanel Nginx Reverse Proxy is now offered from cpanel comes with a manager like any other feature cpanel offers if that feature is no working as intended it should be a priority as affecting many of your users It is a big problem to rely on that feature and your cache files get deleted, need to warn up the entire cache every 6 hours Thanks 0 -
I meant the ticket itself isn't in the priority queue, as that queue is for license holders that purchase directly from us. 0 -
OK, I will tell my hosting provider that I will cancel the Cpanel license I bought through then because that puts me at the end of the queue and I will purchase one from Cpanel directly. Thanks for the tip. How about the questions in my previous message? thanks for your help, 0 -
You could also reach out to your hosting provider directly as they *should* be the ones to provide the frontline support for those types of issues. At this point, we really need to work through the ticket since this seems like a unique situation with your machine. 0 -
I am posting this out of the obligation to warn others, after denial after denial I finally got this answer via ticket, bottom line is INACTIVE and serving STALE until INACTIVE expires, one of the most important functionalities of NGINX proxy cache is overwritten by CPANEL daily, the cached files needed to serve the fast STALE version while updating, unless is less that 1 day becomes useless, if you have 1 day or longer in your INACTIVE in this file /etc/nginx/ea-nginx/cache.json as per here: As you can see the 1 day thing from the Cpanel Nginx Proxy developers makes this CPanel NGINX proxy cache useless if you have more that 1 day in INACTIVE, I learned the hard way until getting provided proper disclosure now. One of the most important features of NGINX PROXY CACHE which is the option to have a long INACTIVE (10 days for me) for the cached files to be served as STALE (fast) while updating is completely overwritten by cpanel, if you think your site is serving STALE cached files based on INACTIVE longer that one day, those cached were deleted by CPANEL in 1 day, website visitors will get a slow MISS Hope this helps others 0
Please sign in to leave a comment.
Comments
11 comments