从robotstxt.info链接出的网站列表
| No. | 链出域名 | 关键词 |
|---|---|---|
| 1. | robotstxt.org | Robots Exclusion |
| 2. | conman.org | |
| 3. | yahoo.com | |
| 4. | msn.com | |
| 5. | ask.com | |
| 6. | google.com | |
| 7. | bing.com | |
| 8. | ysearchblog.com | |
| 9. | serbanghita.com | |
| 10. | mediawiki.org | |
| 11. | kloth.net | List of Bad Bots |
| 12. | searchtools.com | |
| 13. | wikipedia.org | |
| 14. | mcanerin.com | |
| 15. | howrank.com | |
| 16. | diovo.com | |
| 17. | antipope.org | |
| 18. | slashdot.org | original comment |
| 19. | psu.edu | |