megri
12-10-2007, 04:53 PM
Yahoo recently announced (http://www.ysearchblog.com/archives/000508.html) that they are supporting four new types of exclusion tags in the robots.txt file: NOINDEX, NOARCHIVE, NOSNIPPET, and NOFOLLOW. The benefits of being able to declare these directives in the robots.txt file enables folks who store PDFs, Word Documents, and other files on the web and cannot easily place these directives in the header.