Issue with robots.txt denying access
Hey everyone, I recently took over everything IT and website related for a friend of mine and I'm having some issues:
When attempting to access cpanel or roundcube email on anything other than my cell phone I constantly get the error:
[Thu Jan 21 07:24:04.785920 2021] [access_compat:error] [pid 10677] [client 114.119.155.200:34580] AH01797: client denied by server configuration: /home/nypalawc/public_html/robots.txt.
When I go to file manager I do not have a robots.txt where it states it is even with show hidden files. When I view my robots with my domain I get
"user agent *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: (my site url).com/wp-sitemap.xml
-
Hey there! If you just move the robots.txt file out of the way, does that get things working normally for you? I believe that may be misconfigured as the same user agent, the "*" in your case, is acting on both the Disallow and the Allow sections, meaning you'd only be allowed to see the wp-admin/admin-ajax.php file and the sitemap. It could be there is caching on the phone network allowing you to see other pages, but I'd try just moving that file first to see if that helps get things working. 0 -
I appreciate the reply! So I'm unable to actually find the robots.txt file. I'm assuming something else is dynamically creating the robots.txt. I created a new robots.txt but when I tried to log in it instantly said login failed when trying to log into another users email. It usually takes about 5 seconds then says connection failed, so I removed the robots.txt because I didn't want other things to stop working. Any ideas? Thanks again! Hey there! If you just move the robots.txt file out of the way, does that get things working normally for you? I believe that may be misconfigured as the same user agent, the "*" in your case, is acting on both the Disallow and the Allow sections, meaning you'd only be allowed to see the wp-admin/admin-ajax.php file and the sitemap. It could be there is caching on the phone network allowing you to see other pages, but I'd try just moving that file first to see if that helps get things working.
0 -
That's interesting - normally I would expect the robots.txt file to be in /home/username/public_html/robots.txt if it is being used for just your site. If you aren't able to see the existing file, you may need to contact the hosting provider or datacenter to have them check the system. 0 -
Hmm alright, I'll give them a call. I appreciate your help! 0 -
Sure thing - if you run into anything else in the future feel free to let me know! 0
Please sign in to leave a comment.
Comments
5 comments