## robots.txt User-agent: * Disallow: /cgi-bin/ # various scripts that should not be indexed Disallow: /blog/comment.php Disallow: /blog/docs Disallow: /blog/pingback.php Disallow: /blog/submit.php Disallow: /blog/trackback.php