cron problems
OK, I'm in "cron hell". I have asked cPanel to create a simple cron job that just appends one file to another, once a day, at 11:55pm. Thusly
55 23 * * * cat ~/access-logs/subdomain.myaccount.net >> ~/subdomain.myaccount.net/totalarchive
This is just to make a running access-log called "totalarchive" out of the daily access logs.
That command works fine, when I give it over ssh. It doesn't work at all when I schedule it with cron. So the command is executable. BTW, I have asked that cron notify me by e-mail when it does that command. I never get notified. Not even an error. So evidently it doesn't get executed because it isn't even trying to execute it!
My host providers are clueless: "Cron jobs aren't our responsibility!" Sez Level 2 support. Duh?
So my question is ... is this a cPanel issue, or a provider issue? I there somewhere in cPanel that you can simply turn off cron jobs? Help!
-
Have you tried running it as a script ? Try creating a file eg cronjob.sh (don't forget to make the permissions executable) with something similar to the following content and call that via cron #!/bin/sh cat ~/access-logs/subdomain.myaccount.net >> ~/subdomain.myaccount.net/totalarchive0 -
I have not tried that, and probably should. Is there some reason why cron would run a script and not a shell command? 0 -
Generally, a shell command in cron would need to have fully qualified path to the executable, so you might make it work using /bin/cat (if that is where your cat executable resides). You may well still need to define the full path to an executable inside a shell script - much will depend on who (eg system admin root user, or non-privileged user) is calling the cron, and what access they have been allocated. You should be able to determine where the cat command is (and if it is run as an external executable, or from inside a shell) by running $ type cat $ type cat cat is /bin/cat
You can confirm there are no external executables by using $ which cat$ which cat /bin/cat
(Try the above commands with other instructions like echo and help instead of cat to see the difference)0 -
Hello, Let us know if the previous post helps. Thank you. 0 -
My cat is all /bin/cat. So you mean I have to command in cron, /bin/cat ~/access-logs/subdomain.myaccount.net >> ~/subdomain.myaccount.net/totalarchive ? 0 -
Cant hurt to try it - it will either work or you will have to go down the shell script route You may be able to increase the verbosity of the logged output by adding -vvv to the command. (You are changing your ~ to the fully qualified paths of where your access-logs files are residing ? Cron may not be running from the folder root of those files) 0 -
Well, this just gets snarkier and snarkier. I tried it with a script. My script file was /home1/memyselfani/job.sh, which contained fully qualified addresses ... #!/bin/bash cat /home1/memyselfandi/access-logs/subdomain.myaccount.net >> /home1/memyselfandi/subdomain.myaccount.net/test1 and the cron job I invoked was bash /home1/memyselfandi/job.sh You'd like to believe at the scheduled time, a file test1 would appear in the folder subdomain.myaccount.net and I'd get an e-mail telling me it happened. Nope. Nada. Oh, when I run that bash command from the shell, it works perfectly. WTF??? 0 -
OK, now I just tried a simple (enhanced) cron job /bin/cat /home1/memyselfandi/access-logs/subdomain.myaccount.net >> /home1/memyselfandi/subdomain.myaccount.net/test1 No joy. I'm just baffled. No errors, no responses. Nothing. 0 -
Does your created file test1 have any content? Is content concatenated into the test1 file after multiple manual runs in shell? I would have thought that the first part of your cat command should refer to a file to read (or to a wildcard pattern to read more than one file) but you seem to have it only referring to a folder /subdomain.myaccount.net (Be careful with wildcard matches and writing to the same folder - you can get in a run-away loop. Better to use a pattern match) It is entirely possible that running the command under a users shell will create a zero byte file (since no input has been defined) called test1, but under cron will error out and do nothing. 0 -
The problem is that once the scheduled time for the cron job is reached, there is no test1. test1 never happens. When I run the command in the shell, it all works the way I want it to, and I get a test1 file with the right access-log content in it. The cat command does refer to a file to read. That file is named subdomain.myaccount.net, and it is sitting in the access-logs folder. I can cat, more, or even vi that file. It's a file. Really. 0 -
I think I can see where this is going. If I am not mistaken, access-logs is a symlink to /usr/apache/domlogs/ which in turn is symlinked to /etc/apache2/logs/domlogs which is symlinked to /var/log/apache2/domlogs On my servers the ownership of the files are root, and the Group is the user. In order that a cron call will execute the command as you intend, it may have to be run as root. There has been some discussion about whether a file run by cron should be named filename.sh or just filename and be executable. Personally I have various executable files named filename.sh run under root, but have never tried to run them as a user. There is a handy quick reference at 6 Reasons Your cron Job is Not Running - and further reading at Why is my crontab not working, and how can I troubleshoot it? 0 -
That's very interesting. In fact, that file in access-logs IS owned by root (but it is in my group, which is the user). root has rw permission on that file, and I have r permission on it. That's why I can view the file in a shell script. So cron doesn't have the same privileges that I do? Geez, does that mean that I have NO cron capabilities for access-logs, but I'm just free to manually access them? Thank you for the interesting reference links. Need to study those. 0 -
One (there are many) of the biggest irritations of the cron shell is that it runs it's own shell in a very restricted environment. This shows up as the command you want to run working perfectly if you execute it from your shell, but then it stamps its foot and pouts petulantly (much like politicians) and refuses to do anything when you try and have it run under cron. I usually side-step the issues by running any problematic cron job as root - but this is certainly not the correct answer, and anyway, you may not always have that option ! 0 -
Oh goodness. I certainly don't have the option of running as root. I'm using a shared server. I don't even have WHM access. I had no idea that cron couldn't do everything I could do manually. So if I can't use cron to schedule certain shell scripts, what other options might I have? 0 -
If you have PHP exec() available (not a great idea as this is ofton used by hackers to inject code), you might be able to create a PHP file that can be executed by cron and that then calls the shell script you want to execute. PHP: exec - Manual PHP: shell_exec - Manual and you might want to read exec() disabled when PHP-FPM enabled if you are using FPM Alternately, since your user can read the files, you may be able to use PHP file_get_contents() Function and then write the string to your output file located in the users home folder with their full RW permissions. 0 -
Well, I'll see what I can do. Thank you for your insights. I am *stunned* that there are such restrictions on cron usage. One would think that whatever you're able to do in the shell, you should be able to do with cron. So this is an OS deficiency, rather than a cPanel deficiency. One wonders what motivated the OS developers to make it so screwy. Doesn't sound like a security issue. 0 -
This is just to make a running access-log called "totalarchive" out of the daily access logs.
Can you provide any more details as to what you're hoping to do exactly? Are you experiencing some issue that you'd like to track the logs for or something?0 -
Pretty simple. This implementation of cPanel, or the underlying Linux, produces DAILY access logs, and overwrites them, starting at midnight. So on Tuesday, I don't get to look at the logs from Monday. I'd like to review the logs for a given WEEK. My Mac Apache 2.3 produced weekly logs automatically. You set that up in /etc/newsyslog.d/apache.conf as root. So, not being able to access that file in my cPanel, what I was trying to do was, at the end of each day, append the daily log to a master log file. At the end of the week, that master log file would be moved to a safe place, so a new one could be started. I have a service I do each week, so my operational time scale is one week. In principle, that should have been a trivial job in cron. In reality, evidently it is not. Ideas? 0 -
cPanel, or the underlying Linux, produces DAILY access logs, and overwrites them, starting at midnight.
Are the access logs of no use to you? 49935 You might consider getting your own VPS with full root access. Then you'll have plenty of logs to look at. :)0 -
Yes, the raw logs are monthly. That's better than nothing. I don't have a Configure Logs option. What's that? Looks like it might be useful. But yes, a VPS with root access would be the way to go. That's pricier. My present shared server implementatino is adequate, but it would just be nice to have this kind of log access. 0 -
In your cPanel > METRICS > Raw Access .......you should have options to Configure Logs If you are not seeing this, you might try reaching out to your host to switch on the options for you 0 -
Oh, I see what you mean. Yes, I default to archiving daily logs. But that just writes over that archive every day.. It doesn't preserve daily logs. And while I now preserve my monthly logs, as I say, it would be nice to examine a weeks worth at a time (starting, say, on Sunday), rather than an archive that starts on the first of the month. Having said that, about the monthly logs, I see it offers me the access-logs from the first of the month up to, er, yesterday. So if I want to examine previous logs for the whole month, I really have to add to that file the daily log. That's not hard, but just a PIA. One other thing I miss with my shared server that I had when I managed Apache on a private server is real-time logging. If I look at my access log now, it shows me accesses up to maybe an hour ago. Yes, a VPS would fix that! I run a large telecon meeting once a week where we discuss downloadable documents. It would be nice to have a real-time log that told me how many downloads were done before my telecon starts. But I don't get that anymore. 0 -
Speaking of the monthly access logs files, those files in ~/logs are owned by the user. Not by root! Bizarre. So I can play with those monthly log files in cron all I want, but cron can't touch those daily log files. In my mind, this is a major OS design error. What were they thinking? 0 -
Hello Dan, I believe the difference you notice is because the daily log files do not actually exist under the account, but are symbolic links pointing to the corresponding log files in the /etc/apache2/logs/domlogs/ directory. Also, here's some additional information that it looks like you are already aware of, but might help anyone else seeing this thread for the first time: 1. Here's some general information about the options available to your web hosting provider under the "Stats and Logs" tab in "WHM >> Tweak Settings". It's important to understand these options, as their use may affect how you can access the access logs associated with your account: Delete each domain"s access logs after statistics are gathered - This setting deletes all access logs after each statistics generation (by default it happens every 24 hours). Archive logs in the user's home directory at the end of each stats run unless configured by the user. - This option will only be used if the user does not configure their log archiving preferences in cPanel. Remove the previous month's archived logs from the user's home directory at the end of each month unless configured by the user. - This option will only be used if the user does not configure their log archiving preferences in cPanel. 2. In cPanel, under the "Raw Access" option, the following settings are available: Archive logs in your home directory at the end of each stats run every 24 hours. Remove the previous month"s archived logs from your home directory at the end of each month. On most servers, you can simply enable "Archive logs in your home directory at the end of each stats run every 24 hours." via the "Raw Access" option in cPanel, and you will then have access to your raw access logs. Thank you. 0 -
Thank you for your assistance in this, Michael. When I "archive logs in your home directory", I believe it just writes one log file every day, and writes over it the next day. What I was trying to do was preserve those daily files, if just by concatenating them into one file in cron. But I guess that monthly files will have to do, unless I want to do a manual job every evening at 11:59pm. I do NOT have WHM access, unfortunately. Sounds like there might be tools in WHM that would be helpful. Perhaps oddly I do not have /etc/apache2. Not sure where the Apache stuff is kept. I can't get into / to look for it. I appreciate the advice here at cPanel forums. I'm new here, but you guys are very responsive. 0 -
Perhaps oddly I do not have /etc/apache2. Not sure where the Apache stuff is kept. I can't get into / to look for it.
You won't be able to access that directory without root access to the system. Thank you.0 -
OK, this turned out to be not that hard. One just has to have cron execute a script that addresses the root files. As in, cron needs to do "bash archivejob" where basharchivejob is a file containing "cat /home1/me/access-logs/mylog >> archivehold" So you end up with a file archivehold that gets added to every day with the daily log. At the end of a week, you have a weeks worth of archive. I still think it is kinda perverse that I manually have access to root-owned files while cron does not. 0
Please sign in to leave a comment.
Comments
27 comments