HOSTS file not working on Win7 64-bit?

[VirtualLarry]

I found a HOSTS file in C:\WINDOWS\SYSTEM32\DRIVERS\ETC\HOSTS.

I added these entries:
127.0.0.1 b.scorecardresearch.com
127.0.0.1 pixel.quantserve.com
127.0.0.1 pagead2.googlesyndication.com
127.0.0.1 a.tribalfusion.com
127.0.0.1 tenzig.fmpub.com
127.0.0.1 tag.contextweb.com

Because my browser (Waterfox 5.0) was lagging on loading all of those "tracking URLs" on the forums here.

What I don't understand is, after changing the hosts file, and restarting my browser (is that even required), I still see the domains loading in Waterfox as an overlay at the bottom. Shouldn't those domains, resolved to localhost, basically resolve instantly, such that I wouldn't see them resolving anymore? It seems like they are still resolving out on the internet. What's going on here?

Is there a seperate 32-bit and 64-bit HOSTS file that I don't know about?

Edit: This suggests that Firefox ignores the HOSTS file? I thought that the HOSTS file was used system-wide for DNS resolution, and handled by the OS?
http://answers.microsoft.com/en-us/...-firefox/dcad50c8-4242-435b-a87b-26ecd2ff7d97

Hmm, according to this article, there is only one HOSTS file, exactly where I found it.
http://www.sepago.de/helge/2009/06/04/where-is-the-hosts-file-on-windows-x64/

ping b.scorecardresearch.com gives 127.0.0.1. nslookup returns the actual internet IP address.

Edit: This site suggests a workaround, of copying the entire contents of the HOSTS file, and copying and pasting it to another notepad instance, and saving it over the old one.
http://www.tweaksforgeeks.com/windows7/2011/02/windows-7-hosts-file-ignored

I tried opening an Administrative command-prompt window, and typing IPCONFIG /flushdns , and then another nslookup, but it won't return 127.0.0.1

I just tried that, opened two Administrator Notepad windows, opened hosts in one, copied and pasted to the other instance, closed the first one, opened an Administrator Explorer.exe, deleted HOSTS, then saved the new copy correctly. Still doesn't work.

 

[FishAk]

nevermind

 

[postmortemIA]

C: \Windows\System32\drivers\etc

 

[nickbits]

only if you have a webserver running locally, otherwise it has to timeout trying to connect to your local machine

 

[imagoon]

You may need to flush the DNS cache. Also programs are not *required* to use HOSTS.

 

[VirtualLarry]

You may need to flush the DNS cache. Also programs are not *required* to use HOSTS.

I thought it was automatic, as long as the app uses Windows' built-in DNS resolution APIs, that the system DNS service respected the HOSTS file.

I just added www.google.com to my hosts file, and it still resolves just fine in my browser.

 

[FishAk]

I thought you had to reboot to get Windows to read the host file again.

 

[VirtualLarry]

I thought you had to reboot to get Windows to read the host file again.

I thought that I had read that Windows monitors the filesystem (there are calls to do that), for processes that change/replace the hosts file, and the DNS service reloads it when modified. I'll try rebooting though.

I just tried a full reboot. No dice. Tried Waterfox. Then tried Firefox. Then tried IE8. All of them were able to access www.google.com, even though I added that to the HOSTS file.

I'm starting to hate Windows 7, I want XP back. This is a HUGE HOLE, and a definate negative for privacy and getting rid of ad servers.

 

[VirtualLarry]

WTF. Now I just noticed something, the icon for the file is different, and the file type is "Text File". When I copied and pasted and saved, I typed "hosts" (no extension), but Notepad must had added one itself.

Edit: Now it seems to be working, now that I've renamed the file back. Interestingly enough though, when I first added those entries, it was not a ".txt" file, it saved it back with the original filename.

So it looks like there may be a bug in the original Win7 HOSTS file, that has to be fixed by copying and pasting the file into notepad, to strip out the invalid characters. Simply editing and added entries like XP, does not work.

 

[imagoon]

I thought it was automatic, as long as the app uses Windows' built-in DNS resolution APIs, that the system DNS service respected the HOSTS file.

I just added www.google.com to my hosts file, and it still resolves just fine in my browser.

It isn't. Flush the DNS cache.

 

[imagoon]

WTF. Now I just noticed something, the icon for the file is different, and the file type is "Text File". When I copied and pasted and saved, I typed "hosts" (no extension), but Notepad must had added one itself.

It has done that since NT 3.51....

You have to show the extensions and it will let you save it without one.

As to the "bug" you say you found... I have edited it without issues. <ip>[tab]<dns name>[cr][lf] Not sure where you are picking up "extra characters" from. HOSTS is an ascii (on windows at least) text file with no extension.

 

[VirtualLarry]

It's still not working. The web browser is hanging on loading some of those domains, and the length of time it hangs on any particular one is variable, suggesting that it is still going out on the internet to look those up.

I would expect if it were timing out on localhost, that those timeouts would be brief and regular. That is not how it is working.

I just tested editing the hosts file, and removed www.google.com, and I didn't have to re-open my browser, nor did I have to do a IPCONFIG /flushdns , it started resolving immediately.

Thus verifying what I said, that Windows' DNS client monitors the hosts file, and automagically re-compiles the list when edited.

I'm now starting to think that Windows has something more serious in it, like a Microsoft DNS white-list, that INCLUDES WELL-KNOWN AD SERVERS.

It's already known that MS white-lists their own DNS entries in the binaries, you cannot block out certain MS websites, either with Windows Firewall entries, NOR with certain HOSTS entries.

But did they take that to the next level with Win7, by including certain well-known ad servers in the whitelist?

 

[imagoon]

From technet:

* Parsing of the "hosts" file: The lookup functions read only the hosts file if they cannot off-load their task onto the DNS Client service and have to fall back to communicating with DNS servers themselves. In turn, the DNS Client service reads the "hosts" file once, at startup, and only re-reads it if it notices that the last modification timestamp of the file has changed since it last read it. Thus:
o With the DNS Client service running: The "hosts" file is read and parsed only a few times, once at service startup, and thereafter whenever the DNS Client service notices that it has been modified.
o Without the DNS Client service running: The "hosts" file is read and parsed repeatedly, by each individual application program as it makes a DNS lookup.

And "lol" at your conspiracy theory. Learn how to use it first, then worry about the tin foil hat.

 

[VirtualLarry]

Actually, I just proved that MS does whitelist certain DNS entries.

Try adding www.microsoft.com to a HOSTS file, and point it at 127.0.0.1.

Then try going to www.microsoft.com in your browser. Funny how you can still access their web site, after trying to block it, eh?

 

[VirtualLarry]

and thereafter whenever the DNS Client service notices that it has been modified.

And that's exactly what I said, it re-compiles the internal list, whenever the hosts file is modified. No need for IPCONFIG /flushdns, or a reboot.

 

[imagoon]

And that's exactly what I said, it re-compiles the internal list, whenever the hosts file is modified. No need for IPCONFIG /flushdns, or a reboot.

pics.bbzzdd.com is not accepting uploads at the moment otherwise I have a step by step showing you why you are wrong. You need to flush the client. It seems in win7 ipconfig /flushdns no longer includes the HOSTS file. You need to net stop dnsclient && net start dnsclient, wait for a refresh or reboot the pc to get the file to process.

Blocking www.google.com worked entirely as expected. There is no "internal MS whitelist."

 

[VirtualLarry]

pics.bbzzdd.com is not accepting uploads at the moment otherwise I have a step by step showing you why you are wrong. You need to flush the client. It seems in win7 ipconfig /flushdns no longer includes the HOSTS file. You need to net stop dnsclient && net start dnsclient, wait for a refresh or reboot the pc to get the file to process.

Blocking www.google.com worked entirely as expected. There is no "internal MS whitelist."

Yes, as I had already explained, adding www.google.com, works to block it. Try adding www.microsoft.com, you can't block it.

And as I already explained, no need to flushdns or reboot, changing the HOSTS file is enough.

Edit: My original example to test the whitelisting, was using www.microsoft.com. So I don't know why you said you tested it with www.google.com. You are either being disingenious, or you don't want to admit the truth.

 

[Nothinman]

pics.bbzzdd.com is not accepting uploads at the moment otherwise I have a step by step showing you why you are wrong. You need to flush the client. It seems in win7 ipconfig /flushdns no longer includes the HOSTS file. You need to net stop dnsclient && net start dnsclient, wait for a refresh or reboot the pc to get the file to process.

Blocking www.google.com worked entirely as expected. There is no "internal MS whitelist."

I can verify what VirtualLarry says, I just added www.microsoft.com to my hosts file and it still returned their akadns.net address, restarted dnscache and got the same thing. Added www.google.com and it magically resolved to whatever I wanted. I disabled the NIC in my VM, did it all again and www.microsoft.com failed to resolve, so while it doesn't look like there's any static addresses built into their resolver, it does appear that they're forcing DNS queries for their own domain and maybe more. And it does it with the dnscache service stopped so it looks like it may be part of the base resolver library.

 

[imagoon]

Yes, as I had already explained, adding www.google.com, works to block it. Try adding www.microsoft.com, you can't block it.

And as I already explained, no need to flushdns or reboot, changing the HOSTS file is enough.

Edit: My original example to test the whitelisting, was using www.microsoft.com. So I don't know why you said you tested it with www.google.com. You are either being disingenious, or you don't want to admit the truth.

http://forums.anandtech.com/showpost.php?p=31951918&postcount=10

You started your rant here.

It looks like MS announced that microsoft.com etc would not be cached in the DNS poisoning fixes from a few years ago. So I guess you are right that there is at least a microsoft.com whitelist. Granted that makes sense if they are trying to block people from running activation server / deploying exploited patch etc. I concede on that point.

 

[imagoon]

I can verify what VirtualLarry says, I just added www.microsoft.com to my hosts file and it still returned their akadns.net address, restarted dnscache and got the same thing. Added www.google.com and it magically resolved to whatever I wanted. I disabled the NIC in my VM, did it all again and www.microsoft.com failed to resolve, so while it doesn't look like there's any static addresses built into their resolver, it does appear that they're forcing DNS queries for their own domain and maybe more. And it does it with the dnscache service stopped so it looks like it may be part of the base resolver library.

It didn't always used to be that way. We used to mess with people in the computer club and redirect them to pages like "Redhat acquires Microsoft" pages etc. IE mucking with the MS "fanbois" That was on 2000 / XP / 2003 circa 2004.

 

[VirtualLarry]

Thank you Nothinman for testing. Now that we've established a whitelist exists, has anyone else tried adding those entries that I listed in the first post to their HOSTS file, and see if it still goes out on the internet? Nothinman, is it possible to sniff the network connection on your VM and see if those are also whitelisted?

 

[imagoon]

    a.tribalfusion.com
    ----------------------------------------
    Record Name . . . . . : a.tribalfusion.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 86400
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 127.0.0.1


    a.tribalfusion.com
    ----------------------------------------
    No records of type AAAA


    pixel.quantserve.com
    ----------------------------------------
    Record Name . . . . . : pixel.quantserve.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 86400
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 127.0.0.1


    pixel.quantserve.com
    ----------------------------------------
    No records of type AAAA


    tenzig.fmpub.com
    ----------------------------------------
    Record Name . . . . . : tenzig.fmpub.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 86400
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 127.0.0.1


    tenzig.fmpub.com
    ----------------------------------------
    No records of type AAAA


    pagead2.googlesyndication.com
    ----------------------------------------
    Record Name . . . . . : pagead2.googlesyndication.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 86400
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 127.0.0.1


    pagead2.googlesyndication.com
    ----------------------------------------
    No records of type AAAA


    tag.contextweb.com
    ----------------------------------------
    Record Name . . . . . : tag.contextweb.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 86400
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 127.0.0.1


    tag.contextweb.com
    ----------------------------------------
    No records of type AAAA


    b.scorecardresearch.com
    ----------------------------------------
    Record Name . . . . . : b.scorecardresearch.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 86400
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 127.0.0.1


    b.scorecardresearch.com
    ----------------------------------------
    No records of type AAAA

C:\Windows\system32>ping b.scorecardresearch.com

Pinging b.scorecardresearch.com [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0&#37; loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>ping pixel.quantserve.com

Pinging pixel.quantserve.com [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>ping pagead2.googlesyndication.com

Pinging pagead2.googlesyndication.com [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>ping a.tribalfusion.com

Pinging a.tribalfusion.com [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>ping tenzig.fmpub.com

Pinging tenzig.fmpub.com [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>ping tag.contextweb.com

Pinging tag.contextweb.com [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Windows\system32>

 

[Nothinman]

http://forums.anandtech.com/showpost.php?p=31951918&postcount=10

You started your rant here.

It looks like MS announced that microsoft.com etc would not be cached in the DNS poisoning fixes from a few years ago. So I guess you are right that there is at least a microsoft.com whitelist. Granted that makes sense if they are trying to block people from running activation server / deploying exploited patch etc. I concede on that point.

There's a huge difference between not caching and ignoring the hosts file...

 

[imagoon]

There's a huge difference between not caching and ignoring the hosts file...

I know that ignoring the file vs caching is different. What I meant is they were doing it for variety of reasons. Malware sends www.microsoft.com to hotnudewomen.com and downloads a "patch" from there? In theory if DNS is botched, the certificate could be faked well enough to make it work. I am not attempting to explain what is going on under the hood there but they might have done something to verify the address like implement an under the covers version of DNSSEC for just their site. I didn't test that hard but it seemed to ignore any non-authoritative replies for example.

NM the non-authoritative thing I manged to get it to use one.

I am not all that interested in trying to tear it apart really.

 

[VirtualLarry]

imagoon, can you try actual browsing of the forums.anandtech.com with Firefox 5.0 or Waterfox 5.0, after adding those entries in my first post to the HOSTS file, and see if the browser shows resolving those domain names for several seconds at times, like I see?

I don't quite understand, if they are resolving to 127.0.0.1, then why would I see the browser sit there resolving those domains, for several seconds at a time? Surely, the browser should timeout faster than that. And not all of them for the same duration, some of them pop up only for a split second, some sit there for like 3-4 seconds. If all of the activity was on the local machine, surely there should be consistency in the delays, and there is not.

This is an entirely unloaded quad-core Q9300 @ 3.0Ghz, plenty of CPU time to play with, on a FIOS 25/25 connection.

My earlier testing was on a Zacate 1.6Ghz dual-core, on a Comcast 16Mbit connection.

Edit: Here's a list of the documented DNS whitelist entries for XP SP2:
http://www.dslreports.com/forum/remark,17385255

Here's a list, directly ripped from DNSAPI.DLL in SYSTEM32:
www.msdn.com msdn.com www.msn.com msn.com go.microsoft.com msdn.microsoft.com office.microsoft.com microsoftupdate.microsoft.com wustats.microsoft.com support.microsoft.com www.microsoft.com microsoft.com update.microsoft.com download.microsoft.com microsoftupdate.com windowsupdate.com windowsupdate.microsoft.com

No ad networks that I can see. Hmm.