Where exactly is the root directory?
Apologies if this is the wrong forum... This may seem a silly question, but is the public_html directory and the root directory one and the same, or is the root directory the top-level one that everything else is in, including the public_html one?
Also, for controlling bots/crawlers/spiders, do I need to use both the robots.txt and .htaccess methods, or can I use one or the other?
-
Hey there! In Apache we define the "document root" for a domain, and that is /home/username/public_html by default. You would not have any web content outside of the public_html directory, so that's the only place you would need to control access to the files. robots.txt works well and you don't need anything in .htaccess for that work. Let me know if that helps! 0 -
Hey there! In Apache we define the "document root" for a domain, and that is /home/username/public_html by default. You would not have any web content outside of the public_html directory, so that's the only place you would need to control access to the files. robots.txt works well and you don't need anything in .htaccess for that work. Let me know if that helps!
OK, so if I use robots.txt that would go in the public_html directory or folder, thanks for putting me straight there :) About if I use robots.txt or not ... I've been told that the .htaccess method is the way to go because many bad bots will ignore requests in robots.txt and also because one can easily find out what's in it by entering somedomain.com/robots.txt in the URL bar, and therefore it's not very secure. Sorry for contradict... ;)0 -
Sorry to bump thread... OK, I can see why my last post might have been 'taken the wrong way', this wasn't my intention at all, it's just that you're saying I only need robots.txt but someone else is saying that the .htaccess method is what I should use, so I'm still a bit confused here. Maybe I should use both? Not sure though.:confused: 0 -
There are definitely ways to use both, but it really depends on the situation. You may want to try using just the robots.txt first and then check the site traffic to see if that is accomplishing everything you want. If not, you could then move on to using .htaccess to block additional bots. 0 -
There are definitely ways to use both, but it really depends on the situation. You may want to try using just the robots.txt first and then check the site traffic to see if that is accomplishing everything you want. If not, you could then move on to using .htaccess to block additional bots.
OK, thanks for that, this more or less clears it up for me. :)0 -
You're welcome! 0
Please sign in to leave a comment.
Comments
6 comments