• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

The robots meta directive in the view temlates gets edited out during the rendering process.

This mean that indexing robots can walk publicly accessible sites whether the sitemaster wishes this or not.

The code seems to have soemthing to do with whether this is a top level revision or not.

I feel this is mispalced. If the site master wants the site to be publicly accessible then he should have the option of not having the meta tag and this should be part of the site cofigure process.

Conversely, if he wants the site accessible via the Internet but not indexed then the tag should be left.

This should be an option in configure not hard wired into the code.

There was a problem due to misinterpretation of the CGI::Vars function that was resulting in the tag always being removed. Someone also screwed up the definition of an indexable view. I also added the following option:

# By default, a META tag is generated to tell robots not to index pages
# viewed in TWiki. You can ask for this tag to be <b>removed</b> in the event
# that you <b>want</b> topics to be indexed by robots, such as Google. Note
# that only "normal  view" pages can be indexed. If there are <i>any</i>
# parameters to the view, then it can never be indexed. This is to help
# prevent inappropriate pages (like the raw view) from ever being spidered.
$cfg{AntiSpam}{RobotsAreWelcome} = $TRUE;

Thanks for the reminder, I also added the tag and rel=nofollow to all links generated by configure.

Checkin incorrectly attributed to Item109 frown

SVN 4658


Summary The 'robots' directive gets edited out
ReportedBy TWiki:Main/AntonAylward


Priority Urgent
CurrentState Closed

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r4 - 2005-07-17 - TWikiGuest
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback