文章内容
1、wordpress内置默认robots.txt
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
2、推荐robots.txt规则
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Disallow: /trackback/
Disallow: /comments/
Disallow: /attachment/
Disallow: /comments/feed
Disallow: /feed
Disallow: /*/feed
Disallow: /*/comment-page-*
Disallow: /*?replytocom=*
Disallow: /*/trackback
Disallow: /?s=*
Disallow: /*/?s=*
Disallow: /wp-*.php