# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # forget it User-agent: * # applies to all robots Disallow: style/ Disallow: stats/ Allow: index.html Allow: /annonce/default.asp?cat=1 Allow: /annonce/default.asp?cat=5