Scroll

Disable spiders crawling the web to list your surveys

Disable spiders crawling the web to list your surveys

Follow
Summary This article describes how to disable spiders crawling the web to list your surveys.
Applies to askiaweb
Written for Fieldwork manager
Keywords spiders; crawling

There are two ways this can be done: by adding a meta tag in the page generated by AskiaExt, or with a robots.txt file at the root of the web server.
The latter is less fool-proof (the file can be deleted or corrupted), but it has the benefit that clients could allow search engines access if they want to (by changing file).

Example of a meta tag:

<meta name="robots" content="noindex,nofollow" />

Example of a robots.txt file:

User-agent: *
2Disallow: /cgi-bin/
Have more questions? Submit a request

Comments