Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
How We Detect Adblock
52 points by pagefair on Oct 2, 2013 | hide | past | favorite | 79 comments


So has anybody ever tried to address the root cause of adblocker use, i.e. ads are really fucking annoying? I don't like ads in general, nobody does, but that alone wouldn't be enough to make me bother to install and maintain AdBlock. What does it is strobing "YOU MAY HAVE ALREADY WON" and animated "one weird trick" scams and softcore porn. This shit is everywhere, even on allegedly respectable news sites.

If you want me to turn off AdBlock, you need to insist on reasonable, non-offensive, non-animated ads. If your ad provider doesn't do that, get a better one, or lean on yours until they do. If this industry spent one-tenth as much energy pushing ad services for better quality standards as they do wringing their hands about ad blocking, shit would happen.


I'll continue to ad-block even if ads are non-intrusive or annoying. I prefer no ads to any ads.

If your website requires a monetization scheme to exist, and you can't find a monetization scheme that doesn't pollute your product with (even nice simple) ads, that's not really my problem. I didn't force you to publish.

If I want your content, then I'd be happy to help pay for its generation. If it satisfies my terms, of course (open licensing, no ads).


One site that annoys the hell out of me for adverts is http://www.autosport.com/ I read an article every so often, they appear in some Formula 1 feeds I am subscribed to. They also already have monetization scheme (you can few 10-20 articles "free" each month) but still insist on these horrendeous video ads with sound.

I'm on a limited data plan here in Canada (unlimited plans are not common and very expensive). So I have adblock, very rarely visit the site and now refuse to buy a subscription.


You are, fortunately for the rest of us, not a representative sample. If nothing else, most people prefer to take the path of least resistance, and won't bother learning about, installing, and maintaining an ad blocker until they get really irritated; and even then, as I said, many will happily whitelist sites that reach out to AdBlock users and say "We promise our ads aren't annoying, please help us out!"

(This does not require PageFair's services, BTW. Every site I've seen that does it has a text message under their ad panel, so if the ad doesn't load you see the message. Simple.)

The problem now is that advertisers have poisoned the well. It's hard to get people to change--that means it's hard to get them to install AdBlock in the first place, but it's also hard to get them to use their whitelist after installing it. Inoffensive ad vendors and users have an uphill battle to regain the trust of their users.


I'm curious. What websites do you currently pay for access to? Any?


I donate to Wikipedia, Democracy Now (http://www.democracynow.org/), and TVO (http://www.tvo.org).


Agreed. Any site that offers me real value -- enough that I would return regularly -- is a site I'd probably give a few dollars a year to, rather than have ads. Of course, if I'm just a hit and run seeing your site once or twice a year or only once ever, then I'm not going to do anything and there is no relationship there.

Unfortunately, the problem with this is:

1) Most sites try to make the best of both options. They put ads everywhere, which essentially makes them beholden to the advertiser and not the reader and then offer an option to give them some money instead of seeing ads. That only addresses "I don't want to see 10,000 ads every day" part. It doesn't address the "I want to be your focus - not your product that you deliver to your advertisers" part.

2) Why is it that when places offer a "support us with money" option, they are so ridiculously expensive? Instead of spamming my face with ads every day, give me an option where I can go ad-free for maybe $5 a year. Nobody does that, though. They say "you can see ads or you can pay us $50/yr". Seriously, you are not getting $50 in value from advertising to me in a year, so why are you jacking up the price ten times over just so I can go without the ads?!

Ultimately, none of this matters. Internet advertising is a joke and a scam and advertisers will slowly catch on. The whole "it's super targeted you guys!" thing really isn't turning out to be that valuable and it is all rampant with fake click-farms and other inherent problems.

Also, the only thing I hate more than a huge site throwing ads in my face for their content is when every little mommy-blogger or part-time ranter on the internet with a Blogger blog has to monetize their stuff. Look, you are not making a living at this. You are not ever going to make a living at this. Why would you do something so gross as to fill a page with advertising for those six people you might get each month who actually bother to read your site?

Do people not realize that there was a time when we did things online (or even back in the BBS days) for the joy of it? For the joy of making and maintaining systems and services? Or for the joy of having people read what we have to say or make use of services we have to offer? Not only didn't we expect to make money (much less a living) from it, we actually spent plenty of our own money.

But now, if you can't monetize something, it just isn't worth doing, I guess. Gross. And you'd rather advertise to me rather than just take a buck from me in a value-for-value trade. Double gross.

PS: I pay for or donate to a number of online services and websites from RDIO and GiantBomb to a couple torrent sites, charities, and chipping in a dollar or two when I find something really special on one site or another that I really particularly appreciated. I'm not cheap. I just don't want to stare at tens of thousands of ads for 8-12hrs a day every day of my life. And, ideally, I'd also rather be a customer than a product... which I guess depends on everyone else also wanting to be a customer, but they're too cheap and would rather just look at a page that is 70% ads than give a dollar now and then.


I imagine most web devs don't mind about AdBlock if they run that kind of advert -- I mean, it's a self-selecting filter.

The people who are savvy enough to use AdBlock (and I'm sorry for my condescension here, I really am) are generally not going to be the people who convert from a YOU MAY HAVE ALREADY WON WEIRD TRICKS advert. Generally.

So the point of stuff like this for most web devs should not be ultimately to circumvent AdBlock, or to try to encourage/guilty AdBlocker users into quitting, but to make sure that you don't count AdBlock views as ad views.

I'm not saying that's what the article is arguing, but it's the useful information web devs should take from the article in my opinion.

And of course running ads which are well-made and unintrusive should absolutely be the goal of any site owner who relies on ads for income.


Webdevs reading this: it does work. I whitelist all sites that have responsible ads on their pages, reddit being a prime example. They have ads in a couple predetermined spots on their pages, and they're never intrusive. I've actually clicked on a couple (something I never do). Be responsible and open about your ad policy and I'll listen.


This brings up another case for detecting AdBlock: if you think your users are receptive to whitelisting non-annoying ads, you could detect the plugin and let them know that you strive to incorporate ads in a non-intrusive way and kindly ask them to disable AdBlock.

Edit: sounds like this is what reddit does. Props to them!


Agreed. I was the original complainer in this thread, and I've whitelisted several sites that asked nicely and promised non-intrusive ads.

I believe they all used https://www.projectwonderful.com/ as their ad provider, by the way. I haven't really inspected them, but if you want inoffensive ads on your page, that might be a good place to start looking.


I quite like the idea of projectwonderful.

It seems to be very heavily used on webcomic sites I frequent, and I haven't seen a bad ad yet.

I also like the idea of purchasing time on a site, rather than using targeted ads. Meanwhile, the hosting site can discontinue the ad if they feel it doesn't match their culture.

It's more analogous to renting a billboard from a friendly business you like, rather than hiring a bunch of people to follow your customers around with posters.

It just seems like a stand-up way to do ads.


Sometimes it is not the positioning or the non-intrusiveness which bothers me, it's the following me around and tracking. I find it a little creepy.


As do a lot of other people.

The HTTP DNT process at the W3C was supposed to fix this, but it got derailed by the DAA (the Digital Advertising Alliance).

You can manually stop this tracking with AdChoices. Click on that weird little blue icon that's in most ads, and you'll be able to disable tracking by DAA members.

Notably, AdChoices doesn't entirely prohibit tracking - I think you just need to be "minimized" to a certain extent. AdChoices is stupid, but it looks like we won't get a real solution in DNT unless something changes within the next month or so.


Many sites do this, but the problem with Adblock that most people are not happy about is that it AUTOMATICALLY blocks everything. You have to whitelist, and most people just turn adblock on and leave it be... If it was required to blacklist instead of whitelist it'd be better. Many sites with non-intrusive ads are being blocked because of this and that's detrimental. It's already hurting many businesses to the point that soon content online is gonna be behind a paywall and you're gonna end up having to pay $5/mo for every site you want to view.


There are two problems with your claims:

I, as a user, wouldn't be happy having to blacklist all sites.

I won't pay a monthly fee for most sites, because I read a little on a lot of sites - I don't mind paying a 1 penny per article fee if it is convinient.


Unless you visit some very shady sites, having a auto-blacklist isn't needed, and a whitelist is best to keep both parties happy. And if Adblock continues to grow the way it does, the second route is gonna be what happens, nobody is gonna be able to make any money off advertising therefore they'll have to take other routes to pay for their servers to provide you with your content free of charge.


Fark does that. If they detect an ad blocker you get a little message at the top of the page asking you to allow ads to support the site. Their ads don't get in the way so I did it.


Incidentally, you don't need PageFair to communicate with AdBlock users. I've seen several sites that just have a text message behind their ad image, so you don't see it unless the ad doesn't load.

I guess PageFair additionally lets you collect statistics on how many users are blocking ads, which may be useful.


Suicidegirls (nsfw) (this is 2004) had subscribers, and they could turn on ads if they wanted to. The ads were carefully selected, and had rules about not flashing or having noises. The ads appeared on on place on the page and were a fixed size.

This was great for everyone. People who hate ads turned them off. People who don't care left them on, and they were unobtrusive. Some people liked the ads. (Yes, they clicked relevant ads, and bought stuff from the vendors.)

I have no idea how that can be expanded.


Sites don't run crappy ads because they hate you. Those bottom-of-the-barrel network ads don't even pay well.

They run those ads because they can't figure out how to more effectively monetize their traffic.


Wasn't that Google's big schtick 8 years ago? That their text ads were as effective but not annoying as hell?

Then they bought doubleclick and the animated display ads came right back...


And now Google has released a new IDE for making annoying animated ads.


What have I missed? Do you have a link?



Blame the websites for that, when you create an ad unit you have the option of selecting text only or text and image ad units.


AdBlock Plus already has a system for exempting "acceptable" ads. Though the implementation is pretty controversial.

http://www.salon.com/2013/09/02/the_internets_next_victim_ad...


I can pay ABP and have all my ads whitelisted. It's funny to me that so much people are on their nuts, when they just make money like everyone else.


Please note that AdBlock is different to AdBlock Plus, despite the almost same names (I assume this was on purpose)

https://getadblock.com/

But yes, agree with your point if you are a ABP user.


Actually AdBlock Plus had the name first!


Ha, didn't know that! Cool.


I don't think it will be enough for most.

I have turned the non-obtrusive adblock option of, I don't want to see any ads ever.

There is a reason I don't want that - because ads are content that suck. Ads that don't suck are content - such as the funny videos I am watching.

I don't give a hoot if the site I am visiting goes down because it can't afford its bandwidth bills - there are plenty other sites.


The problem is that just because a site uses non-annoying ads today doesn't mean it will not use annoying ads tomorrow.


Is it really worth potentially breaking your site for innocent, non-adblock folks in order to serve ads to a user who has explicitly and deliberately taken steps to not be served ads?

Take the reddit approach: put a humanizing message kindly asking users to disable it and then don't make offensive/hostile ads. And then make some percentage of the ads a "thanks for whitelisting us, we really appreciate it" message.


What potential break are you referring to? They're not recommending disabling functionality for users with ad block on.

It's nice that they're discussing these techniques, hopefully other sites will do it better. I browse with click-to-play for plugins and it's often mistaken for ad block.


>We then carefully observe what happens when a page loads. onLoad and onError events tell us if they’re successfully retrieved or if requests have been blocked

Unfortunately for them many Adblock users also use Noscript or Javascript Blocker. Circumventing Adblock this way will just drive more people to prevent untrusted websites running code on their machines. A net win for web security but probably not the intended effect.


Can anyone else confirm this? Personally, I've always used AdBlock, but never used a Javascript blocker because it is substantially more effort and I am lazy. I do use Disconnect, though, to block that certain flavor of JS.


The first two things I install on any fresh browser is NoScript and AdBlock. I then make a point of permanently blocking JS from any domain looking like it might be used only for ads. I'll selectively turn on JS for one (very constrained!) domain at a time if I need to do something (like login to purchase or comment), but if I'm just browsing and a site no-worky-without-js? Tab closed, no second thoughts. There's a certain blogging platform site (can't remember what it's called) that gets linked here a lot that doesn't work with JS off; I've never read anything posted there.


I use adblock and Ghostery. Partly because I'm tired of ads but mostly because I'm tired of sites connecting to ten external services making pages load slower and sending my details to third parties without my consent.


You want to confirm other people disable javascript?

https://addons.mozilla.org/En-us/firefox/addon/noscript/


I use Adblock, NoScript and Disconnect. Been using the former two in conjunction for years.


If a website popups up a message "we detected you are running adblock, please whitelist us to continue" I will press that little x button and move on. My feeling is they are most likely trying to spam me with something or malware me.


I've used another site by the pagefair guys while running adblock and it didn't prevent me from using the site but rather asked me nicely to perhaps turn adblock off. But I didn't...


Extremely non-scientific survey:

https://addons.mozilla.org/En-us/firefox/addon/noscript/ has 2MM users and https://addons.mozilla.org/En-us/firefox/addon/adblock-plus/... has 17MM

IMHO, NoScript is too annoying to be practical for most users. It's not the sort of thing you can set it and forget it.


I hope you're right, but I fear the people who block scripts are a minority that'll be ignored. Both by those who think doing all in Javascript is "The Way", also by those who want more control over the user.

More and more sites are requiring JS to display content that could be served just fine (if not better) as a static document. I don't know how much of that is done with the intent of having control over the user, but I've witnessed at least one web developer admit to making JS non-optional because of an advertising contract -- the idea being that users will allow scripts on an all-or-nothing basis, so to view the site, they'll also run the ad scripts.


You guys should take a look at piracy and how well that worked out for content creators. They still get paid, but it'll always be whack-a-mole.

I have a comfy six figure IT job, and I'd still spend a few hours for fun just to figure out how to work around your javascript. Now think about how much time someone determined will spend on it.

Pivot and go do something with a higher possible success rate.


The question is not whether this will be 100% effective, the question is whether people will pay for it.

I'm not arguing that content distributors are killing their own biz with these techniques, however, the companies selling the tech are doing quite fine.

The more people hacking their products the better, if they have a client and the ad rate goes down, who do you think the client is going to call? And of course the client would love upgrade to the latest version that defeats that technique.

Do you think McAfee is upset or happy that people continue to write viruses?

They're selling arms and you're starting wars.


I'm not so sure. There are only a handful of popular ad-blockers, and on most people's browsers, they update automatically. If PageFair's techniques can be defeated, they'll be defeated systematically and simultaneously for every ad-blocking user as soon as the workaround is implemented. So it's basically binary. If PageFair just doesn't work at all (or can't be expected to for a meaningful amount of time), then it's not going to sell well.

Contrast that to McAfee. They're fighting a long, generally losing arms race with virus writers, but note that in practice they still protect your computer from a lot of viruses. So users need them. If all the viruses in the wild could all be simultaneously upgraded to circumvent the scanner, it would be a different story and everyone would just think of virus scanners as a thing that doesn't work.


I agree. Google's business model doesn't seem sustainable and I doubt they'll ever be profitable.

\s


He's talking about being in the ad block circumvention business, not being in the ad business.

However, I don't like Google's business model either. Being profitable isn't a measurement of making the world a better place.


Show me a business who serves ads as unobtrusively as Google while providing the same immense amount of value (Search, Gmail, Chrome, etc).


AdblockPlus, NoScript and RequestPolicy are my tools to control the content my web browser displays. My general rule is that I'll only disable ad blocking on a few trusted sites and never on a site that uses a third-party advertising network.

Most advertising networks have intrusive or annoying ads that actually block my ability to read a site by using pop-overs if my mouse cursor accidentally touches a keyword, divs that obscure content until I've paid sufficient attention to locate and click the "X", etc. If that wasn't bad enough, these third-party networks also present a security risk and have been used in the past as delivery vehicles for malware.

In the end if a web site relies solely on intrusive advertising for support and they plead with me to disable adblock, I'll most likely choose to stop reading/participating on the website rather than make an exception for them.


Coincidentally I just open sourced some JavaScript I've been using to detect adblock: http://esd.io/blog/detecting-adblock-javascript.html

There's not much to it, but it's important to note that many ad filters actively try to avoids detection by whitelisting "test ads" or blocking externally loaded adblock detection code. That's why I suggesting inlining this javascript snippet but all of this, of course, is ultimately a cat and mouse game.


Destructoid reported an ad-block rate of 30-45%. In general the wider estimate is 30% of ALL ads across the web are getting blocked.

http://www.destructoid.com/half-of-destructoid-s-readers-blo...

At Quartz we're lucky - it appears to be 13%. I've used Ad-Blocker for years in the past. Not sure why I stopped (new computer, never installed?) but its been a few years at least.

However now that I work more on the ad side of things (how to make site experience better and maintain reading experience) I can see that blanket ad-blocking is harmful to sites bottom line.

I'm not sure what the answer is, its still Users right to manage their browser VS site operators trying to stay in business


I don't have much data yet, but on our B2B news sites it appears to be in the 4% to 9% range depending on the industry.

I don't think ad blocking specifically is behind the trend away from leaderboards and towards "native content" advertising, but it's only going to accelerate it.


Way fewer than 30% of visitors will ever click on an ad, and way fewer than 30% of visitors that click will actually convert after clicking (for non-CPC ads). So the question should be how many of that 30% that's using Adblock would have ever clicked on an ad or converted?


I assume that you are just injecting some snippet of JS into pages. What does protects you from getting (ad-)blocked?

If you get more customers somebody will create ad-blocker-blocker-blocker (we are going deeper). And ad-blockers have much better position than you have, they can execute anytime and intercept code before it is evaluated.


I find AdBlock to be superfluous.

I just block advertising websites from my hosts file.

In particular, using this: http://someonewhocares.org/hosts/


I know several people (me included) that installed AdBlock because of YT annoying pre-roll ads. Google seems to be shooting themselves in the foot.


To all the bashers: detecting AdBlock or other plugins like Disconnect is sometimes necessary for ad-free sites as well. Some sites malfunction when the user has such an extension installed, and it is a great user experience when the developer is thoughtful enough to put up a banner at the top of the page saying "This site may not work correctly if you are using AdBlock". (sorry for the "some sites"; I was browsing recently and saw such a banner, but cannot recall where). Of course the better option is to test your site under AdBlock and make sure it works regardless, but sometimes the detection is needed.


What does PageFair do, exactly? I mean, they're very proud of themselves for detecting Adblock, but it's not clear what they do with that information.


We won’t bore you with complaints here though; hopefully you’ve now got enough information to understand what’s going on in the background when you sign up to use our free adblock measurement service.

From the homepage: Measure: How many of your visitors are blocking ads and how much does it cost you in advertising revenue?

etc etc. They are selling a service.


While I get the point, you can't claim revenues you never received are a "cost."


It does cost you if the ad revenue is supposed to cover your bandwidth costs. If you're hosting something heavy and users don't offset it by clicking the ads every once in a while, it's a problem. Also there are plugins out there which will load the ad content, but never display it - that means if you pay to advertise somewhere, you just paid for nothing (service was provided, but noone saw the ad).


Having ads that nobody likes and ads that nobody sees pays the same.

The case where you need to know if ads are being blocked or not is when you're trying to determine if you should change the ads or find another way to monetize the site.


PageFair lets publishers reach out to adblock users: to explain why ad revenue is necessary to keep the site running, and ask them to support the site by whitelisting it so ads appear or by making a donation towards the running costs.


Do they think we are stupid? I mean I know it costs them money, I just don't give a shit.


How to get me to turn off Adblock:

1) Make ads that aren't annoying 2) Make ads that are low bandwidth 3) Make ads that don't take over the whole page (hiding the real content) 4) Don't show offensive ads (pornography, crude humor, etc) 5) Only advertise stuff that is of high quality and an excellent value for the money -- no junk.

Of course I realize that this will never happen, but one can dream.


I've long thought that a useful feature for Adblock to include would be the option to load ads into the cache, then immediately delete them. This way the websites would make their money, the costs being pushed onto the advertisers instead.


But he script supposedly detects elements in page, css etc so this would not solve the issue of views, but I have no idea what the payment model is - is it just views or clickthroughs?


Is there somewhere I can go to see an example of Pagefair in action?


You can check out this article on Destructoid to see how they used the tool (fyi- we used to be called "Block Metrics")

http://www.destructoid.com/half-of-destructoid-s-readers-blo...


I'm curious if Destructoid paid a licensing fee to use Hopper's art. I'm skeptical that adding another figure (sad Keanu or not) qualifies this for "fair use" in this particular case.

And if they're skipping out on that payment then calling out people who block ads makes me think of a certain kettle and pot.

If they are paying for that content then props to them for practicing what they preach.


Only a matter of time until someone writes code to detect PageFair.


AdBlock -> detected by PageFair AdBlock -> detects PageFair and destroys it

Seems easy enough.


My prediction; Websites/Services will eventually start providing an Advertising Policy similar to today's Privacy Policy or Terms of Use. This Advertising Policy will help the content consumer understand the Websites/Services Stance on Advertising, what types of Ad's they allow, etc. This will help consumers choose what Sites they want to allow (aka support Ad revenue), and what Sites they want to Block.


Twitter detects if you're using an adblocker before logging into their ad platform: https://ads.twitter.com/

"To use this site, you need to disable AdBlock or any other ads-blocking extension you are using, or customize it to show ads on this site."


"Filter lists also name particular files for which requests should be blocked regardless of domain; for example any javascript file called ads.js"

We have a file called that which has nothing to do with showing ads. If that's correct it will effectively break the service for anyone using AdBlock.


These days, it's a good idea to test your site with AdBlock the same way you test it with multiple browsers. When a page appears to be broken the first thing I try is temporarily disabling AdBlock; the second thing is loading it in IE.

It does suck that you have to jump through these hoops because flagrantly intrusive ads have poisoned the well for innocent site-owners. I wish it were otherwise.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: