I'm curious what the difference between the bottom two are? What happends when you leave out the "/" or put one there? I've been trying to exclude the bots from a directory, but I'm not sure if I'm doing it right.
Which one is right? with the / after the folder?
User-agent: *
Disallow: /administrator/
and
User-agent: *
Disallow: /administrator
Which one is right? with the / after the folder?
User-agent: *
Disallow: /administrator/
and
User-agent: *
Disallow: /administrator