How do I block a search engine from indexing my site?
One of two things will easily prevent most search engines’ robots from indexing your site:
- add
<meta name="robots" content="noindex, nofollow">to theheadof the document (“noindex” stops the page from being added to a search engine, “nofollow” tells the robot from following the links; either can be removed). - add a
robots.txtfile
The latter is a text file that indexing robots read to learn what parts off your site are “off limits.” Find out more about how to use this file, read A Standard for Robot Exclusion