[csw-maintainers] robots.txt

Maciej (Matchek) Blizinski maciej at opencsw.org
Mon Aug 31 02:32:06 CEST 2009

On Sun, Aug 30, 2009 at 11:08 PM, Dagobert Michelsen<dam at opencsw.org> wrote:
> Hi Maciej,
> Am 30.08.2009 um 10:24 schrieb Maciej (Matchek) Blizinski:
>> Can we set up a robots.txt file on www.opencsw.org? I would suggest
>> the following content:
>> User-agent: *
>> Disallow: /search
> Do you really mean www.opencsw.org? Or mirror.opencsw.org? The awstats
> are for mirror.opencsw.org only.

I meant www.opencsw.org specifically, but it really concerns all of
our (sub) domains that are being visited by indexer bots.


More information about the maintainers mailing list