User-agent: * Disallow: /resource/ Disallow: /404.php Disallow: /clearcache.php Disallow: /common.php Disallow: /maint.html