Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

We've moved the forum!

Please use forum.silverstripe.org for any new questions (announcement).
The forum archive will stick around, but will be read only.

You can also use our Slack channel or StackOverflow to ask for help.
Check out our community overview for more options to contribute.

General Questions /

General questions about getting started with SilverStripe that don't fit in any of the categories above.

Moderators: martimiz, Sean, Ed, biapar, Willr, Ingo, swaiba

What to put in robots.txt ?


Go to End


1613 Views

Avatar
theAlien

Community Member, 131 Posts

30 March 2009 at 7:56am

Edited: 30/03/2009 8:53am

Hi,

What folders should be allowed in robots.txt to make sure the pages (/mypage/, /myblogpage/ etc) are still indexed by SE-bots (while everything else is blocked)?
In the forums there seem to be different opinions on this one.
Maybe there is someone who knows alot about SS who can shed some light on the topic?