# ----------------------------------------------------------------------------------------------------------------------------- # Author : simon.spark@virgin.net # Title : Robots.txt file for http://www.rsslotracing.com # ----------------------------------------------------------------------------------------------------------------------------- # Description : This file will be accessed by Search Engine spyders (eg. # Google, Alta Vista, etc,.) to determine which files within # this web site may and may not be listed within their search # engines. # # This file lists the 'Disallow' rules (i.e. which files & folder # contents may not be indexed) for the specified User-Agents # (spyders) [note that '*' is a catch-all wildcard]. # # Check the web site access logs on a regular basis to see # if any spyders have accessed the robots.txt file. # ----------------------------------------------------------------------------------------------------------------------------- # Example Syntax: # User-Agent: [agent_name] - use * to catch all agents. # Disallow /[folder_name]/ - disallows access to all folder contents. # Disallow [file_name] - disallows access to specified file only. # ----------------------------------------------------------------------------------------------------------------------------- # More Info : http://www.searchengineworld.com/robots/robots_tutorial.htm # ----------------------------------------------------------------------------------------------------------------------------- User-agent: * # Catch all search engine spyders Disallow: /img/ # Disallow indexing of any files within the 'img' folder Disallow: /rsc/ # Disallow indexing of any files within the 'rsc' folder Disallow: /forms/ # Disallow indexing of any files within the 'forms' folder Disallow: page_template.htm # Disallow indexing of the header_template.htm file.