The White House is blocking search engines from accessing much of the White House web site, particularly web pages relating to Iraq. As a result, these web pages cannot be directly accessed through common search engines such as google.com.
The White House employs a "robots.txt" file which "disallows" search engine access to over a thousand specified directories. See:
http://www.whitehouse.gov/robots.txt
The matter was first noticed and discussed on this web page:
http://www.bway.net/~keith/whrobots/
Say, for example, that you want to find the White House web page where President Bush made this remarkable statement on October 3:
"See, free nations are peaceful nations. Free nations don't attack each other. Free nations don't develop weapons of mass destruction."
You can't get there from Google, because the page is blocked. But it's here:
http://www.whitehouse.gov/news/releases/2003/10/20031003-4.html
Interesting reads.
The White House employs a "robots.txt" file which "disallows" search engine access to over a thousand specified directories. See:
http://www.whitehouse.gov/robots.txt
The matter was first noticed and discussed on this web page:
http://www.bway.net/~keith/whrobots/
Say, for example, that you want to find the White House web page where President Bush made this remarkable statement on October 3:
"See, free nations are peaceful nations. Free nations don't attack each other. Free nations don't develop weapons of mass destruction."
You can't get there from Google, because the page is blocked. But it's here:
http://www.whitehouse.gov/news/releases/2003/10/20031003-4.html
Interesting reads.
Facebook
Twitter