#ROBOTS.TXT - keeps robots from indexing specified resources User-agent: * Disallow: /_css/ # stylesheets Disallow: /_js/ # JavaScript Disallow: /_restricted/ # internal Disallow: /access_db/ # databases Disallow: /blogs/ # blogs Disallow: /download/ # Disallow: /images/ # images Disallow: /libraries/ # library collection Disallow: /license/ # licensing software Disallow: /products/pip/ # Parent Information Program software Disallow: /products/pip/_lib/ # Parent Information Program software Disallow: /products/pip/_xml/ # Parent Information Program software Disallow: /products/pip/_admin/ # Parent Information Program software Disallow: /stats/ #