Disable bots from accessing anything on the server

Is there a way to globally disable Google etc from indexing anything on the server? I have /~xxxxxxx/ that keeps getting indexed. I am aware that some might be solved with robots or disable it with mod but I need the open links to work. Is there a foolproof way to block all traffic from all bots within WHM?