« Basic solr works | Main | Eric Moore's Comments on the UDC Indexer »

New robots.txt file for dspace

I am going to use the robots.txt file below it is based on information from the dspace wiki User-agent: * Disallow: /browse-subject Disallow: /browse-author Disallow: /browse-title Disallow: /suggest Disallow: /*/browse-subject Disallow: /*/browse-author The "/suggest" corresponds to a page that sends an email to a friend.

Posted by Jeffrey Silvis on May 8, 2009 12:29 PM |

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)