Disable spiders crawling the web to list your surveys
Summary | This article describes how to disable spiders crawling the web to list your surveys. |
Applies to | AskiaWeb |
Written for | Fieldwork Manager |
Keywords | spiders; crawling |
There are two ways this can be done: by adding a meta tag in the page generated by AskiaExt, or with a robots.txt file at the root of the web server.
The latter is less fool-proof (the file can be deleted or corrupted), but it has the benefit that clients could allow search engines access if they want to (by changing file).
Example of a meta tag:
<meta name="robots" content="noindex,nofollow" />
Example of a robots.txt file:
User-agent: * 2Disallow: /cgi-bin/