Difference between revisions of "Talk:Hosts"

From Pin Eight
Jump to: navigation, search
(Discussions I need to revisit)
(perhaps I should have done this instead of reverting. Hey Tepples, should we put in an editnotice?)
Line 1: Line 1:
 +
== APK, please read ==
 +
My reversion of your edits had ''nothing whatsoever'' to do with their content – there was no censorship intended. I had no intention whatsoever to endorse the views of other Anonymous Cowards and impostors over yours.
 +
 +
The problem with your first round of edits was that you did not comply with [[wikipedia:Wikipedia:Talk page guidelines|common conventions regarding formatting and etiquette]].
 +
 +
You improved your formatting the second time around, but I still see problems with [[wikipedia:Wikipedia:Civility|civility]]. ''Please keep that in mind'' if you wish to discuss further. (Generally, we apply Wikipedia guidelines here where it makes sense to do so.)
 +
 +
None of the unregistered IP users on this talk page are me (Eighty5cacao) or the site administrator (Tepples), and none have any known conflict of interest with either of us.
 +
 +
Thank you for your attention. --[[User:Eighty5cacao|Eighty5cacao]] ([[User talk:Eighty5cacao|talk]]) 20:23, 15 December 2015 (UTC)
 +
 +
__TOC__
 +
 
== Purpose of this page ==
 
== Purpose of this page ==
 
On Slashdot, there is a frequent anonymous poster going by "APK" who makes off-topic posts containing multi-page guides to computer security through hosts files. There is another who claims to refute all of APK's posts.
 
On Slashdot, there is a frequent anonymous poster going by "APK" who makes off-topic posts containing multi-page guides to computer security through hosts files. There is another who claims to refute all of APK's posts.

Revision as of 20:23, 15 December 2015

APK, please read

My reversion of your edits had nothing whatsoever to do with their content – there was no censorship intended. I had no intention whatsoever to endorse the views of other Anonymous Cowards and impostors over yours.

The problem with your first round of edits was that you did not comply with common conventions regarding formatting and etiquette.

You improved your formatting the second time around, but I still see problems with civility. Please keep that in mind if you wish to discuss further. (Generally, we apply Wikipedia guidelines here where it makes sense to do so.)

None of the unregistered IP users on this talk page are me (Eighty5cacao) or the site administrator (Tepples), and none have any known conflict of interest with either of us.

Thank you for your attention. --Eighty5cacao (talk) 20:23, 15 December 2015 (UTC)

Purpose of this page

On Slashdot, there is a frequent anonymous poster going by "APK" who makes off-topic posts containing multi-page guides to computer security through hosts files. There is another who claims to refute all of APK's posts.

So to help shift the clutter from Slashdot, I called for proposals for a subspace within this wiki about the pros and cons of improving security by blocking specific hostnames from resolving. It is G8-exempt for now. --Tepples (talk) 15:24, 27 March 2013 (UTC)

In fact, this other anonymous poster has recently declared all out war against APK, and in an appalling display of arrogance they're using Slashdot forums for their battleground whether that forum has anything to do with any of this or not. (unsigned post by 68.146.22.71)
Tepples, did you accidentally get logged out here? If not, how should we advise the IP of proper wikiquette? --Eighty5cacao (talk) 18:07, 25 April 2013 (UTC)
Wasn't me. --Tepples (talk) 20:55, 25 April 2013 (UTC)

A few Thoughts on the Hosts file

I admit to telling a friend about the hosts file some years ago, when he was asking about a way to prevent advertisement spam from showing up when browsing. For completeness' sake, this was before the advent of AdBlock Plus, back when dialup was king. Since he only had one computer he used the internet on, and was using dialup at the time, as well as having never been formally trained in network administration, I felt introducing him to the Hosts file was convenient, and effective for his specific instance. He wasn't trying to block access for a whole local LAN trying to talk to the outside world, and a hosed up Hosts file is reasonably painless to fix. (As opposed to a hosed up static routing table, or a bunch of hosed DNS entries served by a local DNS server, which are 'less' trivial to fix.)

Given that he has since switched to a cable modem, and has several internet connected PCs on a home network now, I am contemplating giving him some instruction on how to administer routing tables and DNS servers via ssh, so he can tweak his local router's behavior instead. I have long since introduced him to ABP, and he loves it, but has computer paranoia concerning advert scripts, and using noscript carelessly simply breaks more pages than it sanitizes. For him, it's just easier to prevent a request from resolving, and preventing outbound communication, and he is content with the deleterious consequences.

Some of the Pros of using a local hosts file:

  • Easily edited in case of a screwup
  • Mangles resolution for only that specific PC
  • Allows intranet resolutions for static addresses without running a local DNS
  • Trumps whatever is in the DNS server's resolution entry for a specific name
  • Easy for untrained users to make use of

Some of the Cons associated with a local hosts file:

  • Is a poor fit when trying to manage multiple systems due to having to maintain many copies
  • APK EDIT: It is easy to migrate a hosts file across a LAN from a central point by a network administrator (either by chronjob/taskscheduler tasks OR logon scripts)
  • Its ease of maintenance allows malware to blackhole useful/desirable external locations programmatically (win32 platform especially vulnerable. Unix/Linux/BSD flavors have better FS security by default.)
  • APK EDIT: Nothing in usermode can blow past APK Hosts File Engine protecting hosts (above & beyond Windows' own WFP/SFP mind you) while it runs resident AND on update, a brand new hosts file is created by the program, overwriting ANY old entries & the old file in its entirety.
  • Large hosts files slow down name resolution of the local stack, slowing network performace
  • APK EDIT: ONLY TRUE if you can't follow directions - APK Hosts File Engine instructs the user to shutdown & disable the local dnscache usermode slower client which has a problem with larger hosts files (faulty design, fixed sized datastructure/buffer - limited vs. resizeable/redimmable as it should have been)

So, when is a Hosts file modification reasonable in my opinion?

  1. only one system needs to be impacted (say, you are offline testing a webpage you are building in a sandbox, or this is a stand alone kiosk with a separate fileserver physically inside the cabinet.)
  2. the modification is small
  3. the user is NOT a trained admin, but has a specific need to block a resolution request and
  4. the forced resolution table is temporary, and/or meant to not impact other users on the network
  • APK EDIT: All of my edits above put THAT malarkey in its place (the bin).

Any other scenario is probably better served by having a properly configured local DNS, and maybe a custom static routing table on the default gateway, if you are super paranoid. For a commercial env, such as an enterprise, using a local DNS for the local intranet is a no-brainer, and preventing access to outside hosts is more flexibly handled with a quality firewall. (unsigned post by 138.210.219.83)

  • APK EDIT: LOCAL DNS has MASSIVE security issues & is wasteful for a home network (especially if there's only a single machine there) in terms of both electrical

power used (especially if you only have 1 system & moreso if you setup the DNS as a separate system) + complexity in moving parts for exploit or breakdown. Kaminsky redirect poisoning anyone? Routers being bushwhacked in DNS settings anyone?? Open DNS being abused by malware anyone??? Rogue DNS servers anyone????

The above is correct. Also, on most systems the hosts file will be indexed in memory, unless this is disabled. For a multi-GB hosts file, that's a rock set agin a hard place. Also, on Windows, the only way to disable the hosts file being cached is to disable DNS caching entirely. You can manually cache often-used entries at the top of your hosts file, but any URI requested that is not in that list will result in the entire hosts file being read and *then* a DNS query. This is unlikely to be a highly performing operation.
DNS-level blocks will prevent a multitude of issues, but for blocking web advertisements specifically, there is no better solution than Adblock. It offers fine-grained control over what it blocks, based on any part of the URI and/or regex filtering. It also operates at the "content policy" stage, that is, when the browser is deciding how to handle the requested URI, before it actually sends the request. 50.137.30.129 19:09, 30 March 2013 (UTC)
  • APK EDIT: DNS level blocking introduces a PLETHORA of issues in security, resource abuse, & more (see just above).
Why must it be a linear search? An OS designed to work with a hosts file over 10 MB will sort the hosts file when loading it and then use an O(log n) binary search. No, I don't know whether any popular PC OS does this. --Tepples (talk) 20:38, 30 March 2013 (UTC)
The hosts file will not be sorted on disk. The OS will not rewrite the hosts file for you. If the file is loaded into memory, it is probably sorted. If not, welcome to linear search. 50.137.30.129 01:10, 31 March 2013 (UTC)
  • APK EDIT: Using my APK Hosts File Engine hosts IS sorted on disk and in memory... apk
BSD, Linux, and Windows (with DNS Cache disabled) work the same way: every time you perform a name lookup, they open the hosts file, parse it line-by-line looking for a match, then close it. If there was no match, it then does a DNS lookup. This isn't top-secret information -- you can check the *BSD or GNU libc code to verify it. (unsigned post by 193.234.198.236)

Like them because they're simple and direct; unless it's a monster-big file it's plenty fast on modern hardware. Don't like them because figuring out what to comment out to restore desired function to a given site is for a noob like me at best tedious. I find something such as AdBlock or AdBlockPlus usually gives me enough control without me needing to really know what I'm doing. (unsigned post by 70.92.185.140)

  • APK EDIT: Try editing almostalladsblocked regex rules (especially non-programmers) vs. editing hosts easily understood 2 column line entries (far less work)

A few Thoughts on APK

I've noticed that there's a whole lot of talk about the hosts file, but not much about APK, or APK's hosts file.

  • APK EDIT: There is on /. where I "pr" the program I wrote for hosts files. You admit this yourself next:

Keep in mind, the reason for the extended spam battle on slashdot is not because people debate the value of one modifying their own hosts file to suit their network-blocking needs. It is instead focused entirely on the infamous APK's hosts file specifically.

  • APK EDIT: EXACTAMUNDO - & nobody there (or here) validly technically disproves my points on hosts files... apk

This discussion would be more informative if it focused on APK's hosts file, as it is claimed that only APK has the divinely inspired knowledge to blacklist all the right hosts, with no false positives and no false negatives. I see people debating the virtues of using a hosts file to improve computer security, but they miss the entire point. Of course you can't craft a hosts file that is effective for this purpose; you are not APK.

  • APK EDIT: WHERE DID I EVER SAY "MY WAY IS THE ONLY WAY TO EDIT HOSTS?" - answer that! You can't... I just point out my program does the job better than other methods.

Of particular interest is APK himself. Why is his hosts file the one true hosts file to use for these purposes? What is it about APK that allows him to craft a hosts file that no one else could? By what chance did he find himself in such a privileged state, and is there any way that we can hope to become more like him? Will there ever be a day when the common computer user will be able to fire up their own plaintext editor and perform hosts file wizardry the likes of which have only been seen by APK himself?

  • APK EDIT: WHERE DID I EVER SAY "MY WAY IS THE ONLY WAY TO EDIT HOSTS?" - answer that! You can't... I just point out my program does the job better than other methods.

The fact that this discussion has digressed to encompass such off-topic issues as dynamic hostname resolution and ad blocking leaves me feeling sad. I thought this was supposed to be a place where great philosophers would congregate to share their musings on the fabled APK and his hosts file. I fear that I am leaving disappointed. (unsigned post by 108.58.122.186)

From the article: "APK [...] wrote a tool to manage hosts files in Windows." Then it links to a Slashdot comment by APK listing the "14++ reputable & reliable sources" that his tool checks by default. I too am slightly disappointed that I haven't been able to find more info about the methodology used by APK to choose these blocking lists over others, nor about the methodologies used by the authors of these lists. If you know of a better set of blocking lists, go ahead. --Tepples (talk) 19:15, 3 May 2013 (UTC)
Sorry if I don't make a new account for a 1-2 time posting. I do get grumpy with the APK/Clone-APK thing because it is indeed SO DAMNED LONG! It's a small part of why I am finally reading at 0 instead of -1 these days because I don't care to scroll through TWELVE SCREENS of stuff! 24.193.34.113 23:22, 4 May 2013 (UTC)
APK now has a 2 million (!) line hosts file. Are there 2 million active malware and ad sites? And what kind of filth is he visiting where it's even a problem?
  • APK EDIT: Correction - 4++ million lines currently... apk

If you're that concerned about ads and malware, white list instead of blacklist. I.E. - turn off DNS and add your top 100 sites to your hosts file. (APK already includes his favorite sites at the top of his hosts file since a 2 million line hosts file kills name resolution performance). -- 96.44.189.98 23:03, 7 May 2013 (UTC)

If a hosts file with 2 million entries kills performance on a modern PC, then there's a problem with how the operating system's hostname revolver searches the hosts file. (I'll write some notes on implementation in the article.) And with a whitelist, how would any web search engine be useful to you? As soon as you find a search result on a hostname that you've never seen before, you'd have to wait for your administrator to get home and seek your administrator's permission to add a particular hostname to the whitelist. --Tepples (talk) 02:13, 8 May 2013 (UTC)
  • APK EDIT: Correction - again, 4++ million lines here and growing, and FAR FROM SLOWING DOWN (if you can follow directions, tepples, myself, & others note to turn off the local usermode slower clientside dnscache that's faulty with larger hosts files in Windows - & there are methods for increasing hosts read priority in the registry I've posted before also to further speed it up above the normal default).

So I finally got a response out of him (or his impostor), and it's a wall of text. --Tepples (talk) 20:48, 11 May 2013 (UTC)

At the moment I'm probably the most active APK impostor, and that wasn't me or anyone I know. Based on my experience, I would estimate at least an 80% chance that that's the real APK. Cheers. 199.48.147.39 15:17, 18 May 2013 (UTC)
No way. APK always uses lots and lots (and lots) of blank lines. (unsigned post by 99.103.126.38)
  • APK EDIT: THE FACT YOU ADMIT IMPERSONATING ME INDICATES YOU ARE A LOSER and WEASEL OF THE HIGHEST ORDER "ne'er-do-well"... apk

To do: Integrate this APK post and something sent to my e-mail. --Tepples (talk) 20:04, 4 July 2013 (UTC)

jansal's advice

I don't specifically recall whether we've ever mentioned this on the wiki. I'm aware that their explanation of 0.0.0.0 isn't exactly correct. Just leaving this here as food for thought. --Eighty5cacao (talk) 23:26, 21 May 2013 (UTC)

kozz's advice

I've done my best to ignore the APK posts. That being said, I recall that at least in Windows XP, hosts files could become a problem. Anyone who ever installed Spybot Search & Destroy on an XP machine can understand. In an attempt to blacklist the malware-laden domains, Spybot adds... I don't know, probably thousands of lines to the hosts file. The result is that the machine is so damned slow (especially on startup) that I blew it away, never to recommend it to anyone again. (Usually only installed it on PCs I had to support, you know, like stepmom)

  • APK EDIT: You're not "ignoring me" - TRUTH IS, you & yours can't validly technically disprove my points on hosts files' superiority to redundant, inefficient, sold out to advertisers & crippled by default browser addons like AlmostALLAdsBlocked (or Ghostery for example) -> http://it.slashdot.org/comments.pl?sid=8457871&cid=51107411

I realize this is more of an indictment of Spybot, but the end is the same: hosts files as blacklists is generally not a good idea. (unsigned post by 50.123.253.195)

For the record: jansal's advice above includes the use of a hosts file based on that of Spybot. And if just a couple thousand are enough to slow Windows XP down, it's not an indictment of Spybot as much as it is an indictment of Windows XP's resolver. I explained in the article how to process even a super-sized APK hosts file efficiently. --Tepples (talk) 12:35, 22 May 2013 (UTC)

Memory usage of Adblock Plus

Sorry I'm a little late to the game (I hadn't gotten around to posting this because of other things on my mind):

There's been a lot of talk lately about the memory usage of Adblock Plus (and similar extensions for other browsers) and the implementation changes that would need to be made to improve that, especially with regard to the element-hiding feature. See also the Adblock Plus team's reply, where APK verifiably attempted to comment and had his post deleted as off-topic. --Eighty5cacao (talk) 19:11, 26 May 2014 (UTC)

  • APK EDIT: There's NO DISPUTING DOCUMENTED PROOF FROM REPUTABLE SOURCES' TESTS THAT ALMOSTALLADSBLOCKED (& other redundant bloated inefficient RAM, CPU, + other forms of I/O abusing (messagepassing overheads) browser addons) -> cdn.ghacks.net/wp-content/uploads/2014/06/adblocker-memory-consumption.jpg (hosts use 3-11mb w/ my program initially). Even FireFox 41 adblock eats 65++mb www.ghacks.net/2015/06/30/firefox-41-ships-with-massive-memory-improvements-for-adblock-plus/ & that IS truly that! IF my ware & posts are "so bad", then WHY ON EARTH did AlmostALLAdsBlocked people DELETE MY POSTS? They tried hiding facts I put out like these is why, lol... weak! apk

Discussions I need to revisit

(No prompt reply needed or expected.)

Has APK made any substantial effort to ensure that blocking of CDN IPs does not break major functionality on legitimate sites?

  • APK EDIT: WHEN THAT BECOMES A REALITY? THEN WE'LL TALK. UNTIL THEN NOTHING ADDS MORE SECURITY, SPEED, RELIABILITY, & ANONYMITY ONLINE FOR USERS FOR LESS! I don't deal with 'phantasyland' theoreticals but rather reality... try it sometime. Until then & even then? Hosts files provide excellent protection for less and more speed, security, reliability & anonymity - more than ANY OTHER SINGLE SOLUTION DOES & for a lot less resource bloat + electrical power consumption with less complexity using what you already have natively built in (ip stack which hosts are a part of in kernelmode faster operations).

Also, I still need to read the post in which it is claimed that "APK Hosts File Engine does things that ad blocker browser extensions can't do." This makes sense in the limited sense of strictly blocking malicious sites, especially by IP (with firewall rules such as those added by APK Hosts File Engine). However, my intuition would suggest that the more general case is the opposite; a hosts file cannot apply URL-specific filters or cosmetic (DOM element-hiding) filters without a local Web proxy and a locally-generated root certificate for TLS MITMs. See mention of the issue in the documentation for uBlock Origin. --Eighty5cacao (talk) 23:13, 18 November 2015 (UTC) (+ 05:30, 22 November 2015 (UTC))

  • APK EDIT: HOSTS WILL ALWAYS BE ABLE TO BLOCK MALICIOUS SITES SERVED UP BY HOST-DOMAIN NAMES FOR SECURITY VS. ONLINE THREATS AND IT DOES SO FOR LESS THAN BROWSER ADDONS IN SLOWER USERMODE BY FAR (DOCUMENTATION OF THAT IS ABOVE FOR ALMOSTALLADSBLOCKED AND NOW HERE FOR UBLOCK TOO FROM REPUTABLE SOURCES:

Hosts @ 3mb-11mb w/ current data vs. threats + ads - test yourself.

UBlock uses 63++ MB -> www.ghacks.net/2014/06/24/ublock-chrome-resource-friendly-adblocker-http-switchboard-author/

SCREENSHOT -> cdn.ghacks.net/wp-content/uploads/2014/06/adblocker-memory-consumption.jpg

I just summarized the major "things". DNS blocking applies to all applications, even if they aren't designed for use with web browser extensions. Some native applications fetch advertisements or report excessive telemetry to the publisher; DNS blocking can block that. I seem to remember past news stories about certain native online applications being compromised through the Trident or WebKit browser embedded in the client.
  • APK EDIT: I ENUMERATE MANY OF DNS SECURITY AND RESOURCES OVERUSE FAULTS ABOVE... apk
The other advantage that APK likes to trumpet is that the OS's hosts file parser runs in kernel mode, without an allegedly time-consuming context switch in and out. But that's more dubious for two reasons. One is that because major OS developers haven't spent any blooming time on Blooming, the time for a linear search through a multi-megabyte hosts file greatly outweighs context switch time, which APK Hosts File Engine works around by caching commonly used "good" sites' IPs at the top of the hosts file. The other is that a browser extension avoids the context switch into kernel mode in the first place. --Tepples (talk) 15:23, 22 November 2015 (UTC)
Sorry for not thinking about non-browser applications, but I still feel that the other objections are decisive. I personally use both a (rather small) hosts file and an in-browser ad blocker. --Eighty5cacao (talk) 18:54, 22 November 2015 (UTC)
  • APK EDIT: KERNELMODE IS 1000's of times faster than usermode (& the ip stack has over 45++ yrs. of programmatic refinement in it & is using what you already natively have vs. stupidly & illogically "Bolting on 'MoAr'" in browser addons OR dns servers since hosts is part of the kernelmode faster IP stack) & caching using the local kernelmode diskcaching subsystem as I do it (due to larger hosts file use here) avoids ANY context-switch overheads to usermode (you can't with

addons - they ARE in usermode, & slower because of it as well as their tendency to bloat memory, cpu, & i/o badly with more complexity, room for breakdown, OR exploit due to it).

(FINAL THOUGHTS HERE: YOU CAN KEEP EDITING OUT MY POSTS THAT PROVE ALL OF YOURS COMPLETELY WRONG - GOOD JOB BOYS - YOU'VE GRADUATED TO THE LEVEL OF ALMOSTALLADSBLOCKED DELETING MY POSTS ON THEIR FORUMS! DO YOU THINK THAT FOOLS ANYONE? IT MAKES YOU ALL LOOK LIKE IMBECILES TRYING TO HIDE THE TRUTH THAT I PUT OUT VS. YOUR BULLSHIT!)