robots.txt

Ken Moffat kmoffat
Mon May 17 11:44:10 PDT 2004


Ken Moffat wrote:
> Anyone know the function of robots.txt? I have seen attempted access to 
> it in my apache logs.


Ah, never mind, I found a page describing this functionality.


http://www.robotstxt.org/wc/exclusion.html

-- 
Ken Moffat
kmoffat at drizzle.com




More information about the Linux-users mailing list