Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

General Questions /

General questions about getting started with SilverStripe that don't fit in any of the categories above.

Moderators: martimiz, Sean, biapar, Willr, Ingo, swaiba, simon_w

What to put in robots.txt ?


Reply


1004 Views

Avatar
theAlien

Community Member, 131 Posts

30 March 2009 at 7:56am

Edited: 30/03/2009 8:53am

Hi,

What folders should be allowed in robots.txt to make sure the pages (/mypage/, /myblogpage/ etc) are still indexed by SE-bots (while everything else is blocked)?
In the forums there seem to be different opinions on this one.
Maybe there is someone who knows alot about SS who can shed some light on the topic?