############################### # # User-agent: * # # # First - list folders robots are not allowed # to index... # # Disallow: /.cpaddons/ Disallow: /.cpanel-datastore/ Disallow: /etc/ Disallow: /mail/ Disallow: /public_ftp/ Disallow: /public_html/ Disallow: /tmp/ Disallow: /www/help/ # # I'm leaving out the www folder, as I want the robots # to look in it of course. I want the bots to look at # just the index pages (htm and txt) excluding any # images. # # # Then - list specific files robots are not allowed # to index... # # # # # Finally - upload this text file to # the Root Directory of your site's index page. # # ################################