![\"WhoTracks.Me\"](\"https://raw.githubusercontent.com/ghostery/whotracks.me/master/static/img/who-tracksme-logo.png\"){kind=logo}
\n\n \n
\n \n Transparency\n · Privacy\n · Tracking landscape\n · Built by Ghostery\n \n
\n \n Trackers\n · Websites\n · Explorer\n \n
![\"Composition](\"../static/img/blog/adblockers_performance/requests-composition.svg\")
\n\n### Key takeaway: how to make an adblocker more effective\n\nThis observation suggests that content-blockers will perform better on\naverage if they can efficiently decide which requests to *not block*.\n\nThe filters used to determine whether or not a request is to be blocked\nare the ones from [Easylist](https://easylist-downloads.adblockplus.org/easylist.txt),\nwhere we removed all the cosmetic rules before running the benchmarks.\nThe final list contains *38978 network filters* and is available here:\n[easylist.txt](https://github.com/cliqz-oss/adblocker/blob/master/bench/comparison/easylist.txt).\n\nIt should be noted at this point that a larger proportion of requests\nwould be blocked by enabling extra filters lists such as *EasyPrivacy*.\n\n### 2. Time To Match All Requests\n\nWe first look at all of the requests (whether they will eventually\nbe blocked or not).\n\nWe use a log scale for the x-axis (time in\nmilliseconds) to facilitate the comparison of the cumulative\ndistribution of the time it takes for adblockers to decide whether\nor not a request should be blocked.\n\nHere is a break-down of the 99th percentile and median times for each\ncontent-blocker:\n\n| \n | 99% OF REQUESTS | \nMEDIAN | \n
|---|---|---|
| Ghostery | \n0.050ms | \n0.007ms | \n
| uBlock Origin | \n0.124ms (2.5x slower) | \n0.017ms (2.7x slower) | \n
| Adblock Plus | \n0.103ms (2.1x slower) | \n0.019ms (2.9x slower) | \n
| Brave | \n1.288ms (25.9x slower) | \n0.041ms (6.3x slower) | \n
| DuckDuckGo | \n12.085ms (242.5x slower) | \n8.270ms (1258.4x slower) | \n
![\"Time](\"../static/img/blog/adblockers_performance/ghostery-ublock-origin-brave-duckduckgo-adblock-plus-all.svg\")
\n\n### 3. Time To Match Requests Which Are Not Blocked\n\nThe following table details 99th percentile and median timings for requests not\nblocked:\n\n| \n | 99% OF REQUESTS | \nMEDIAN | \n
|---|---|---|
| Ghostery | \n0.049ms | \n0.006ms | \n
| uBlock Origin | \n0.112ms (2.3x slower) | \n0.018ms (2.8x slower) | \n
| Adblock Plus | \n0.105ms (2.2x slower) | \n0.020ms (3.1x slower) | \n
| Brave | \n1.270ms (26.2x slower) | \n0.038ms (5.9x slower) | \n
| DuckDuckGo | \n11.190ms (230.5x slower) | \n6.781ms (1060.5x slower) | \n
![\"Time](\"../static/img/blog/adblockers_performance/ghostery-ublock-origin-brave-duckduckgo-adblock-plus-not-blocked.svg\")
\n\n### 4. Time To Match Requests Which Are Blocked\n\nThe following table details 99th percentile and median timings for requests blocked:\n\n| \n | 99% OF REQUESTS | \nMEDIAN | \n
|---|---|---|
| Ghostery | \n0.052ms | \n0.007ms | \n
| uBlock Origin | \n0.165ms (3.1x slower) | \n0.016ms (2.2x slower) | \n
| Adblock Plus | \n0.099ms (1.9x slower) | \n0.014ms (1.9x slower) | \n
| Brave | \n1.468ms (28.0x slower) | \n0.062ms (8.5x slower) | \n
| DuckDuckGo | \n13.025ms (248.5x slower) | \n8.31ms (1130.6x slower) | \n
![\"Time](\"../static/img/blog/adblockers_performance/ghostery-ublock-origin-brave-duckduckgo-adblock-plus-blocked.svg\")
\n\n### Summary of findings: How the adblockers performed\n\nOn these graphs, we observe a plateau for *Adblock Plus*, *Brave* and\n*Duckduckgo*.\n\nThis can be explained by the fact that these engines\nimplement some form of caching internally, thus having a very fast\nresponse time for some requests (redundancy in\nrequests comes from both common third parties seen on multiple websites\nas well as the fact that we load several pages for each domain).\n\nThis caching can be implemented on top of any content-blocker and does not\ntell much about the efficiency of each. We can see this as a means to\ntrade *memory* against *CPU usage*.\n\n### Ghostery’s adblocker performance\n\nFrom the previous measurements, we see that Ghostery outperforms other\nlibraries in terms of matching speed. Without going into too many\ndetails, here are some of the optimizations which can explain these\nresults:\n\n* Ghostery makes use of a reverse index associating tokens to filters. Contrary\n to other libraries, we make sure that we pick *the best* token for each filter\n at construction time (best being defined as the *least seen token*). This incurs\n a one-time extra cost but results in maximized dispatching capabilities.\n* Filters are stored in a very compact form, in typed arrays, and only loaded in\n memory lazily, when there is a chance they will be blocked (if we encounter\n identical tokens in URLs).\n* Filters loaded in memory are optimized on-the-fly and multiple filters can be\n combined for increased efficiency. The optimizations were carefully crafted\n based on common cases observed in Easylist.\n\n### 5. Serialization And Deserialization\n\nIn this section, we have a look at the performance of content-blockers\nwhen it comes to serializing their internal representation for faster\nsubsequent loading.\n\nOnly *DuckDuckGo*'s engine does not provide this\nfeature. *uBlock Origin*, *Ghostery*, *Adblock Plus* and *Brave* all allow to\nserialize or cache (*uBlock Origin*'s terminology is: *selfies*) the\nentire blocking engine to either a string or a buffer, which can then be\nused to speed-up subsequent loads.\n\n### The impact of load time on user experience\n\nAs this is a one-time operation, having a higher loading time does not\nimpact desktop users significantly. On the other hand, the ability to quickly\ninitialize the content-blocker is critical on mobile.\n\n### Comparison of serialization and deserialization times\n\nAnother use-case allowed by such capability is to perform the parsing\nof the lists on the backend and ship the serialized form of the\ncontent-blocker to clients directly. This removes the cost of\ninitialization completely.\n\nWe performed 100 serializations for each content-blocker and display the\nresults below:\n\n![\"Serialization](\"../static/img/blog/adblockers_performance/ghostery-ublock-origin-brave-adblock-plus-serializationtimings.svg\")
\n\nThis bar plot contains the median time taken to serialize the engine for each\ncontent-blocker:\n\n![\"Serialization](\"../static/img/blog/adblockers_performance/serializationtimings.svg\")
\n\nSimilarly, we measure the time it takes to restore the content-blocker from its\nserialized form:\n\n![\"Deserialization](\"../static/img/blog/adblockers_performance/ghostery-ublock-origin-brave-adblock-plus-deserializationtimings.svg\")
\n\nAnd here is the median time:\n\n![\"Deserialization](\"../static/img/blog/adblockers_performance/deserializationtimings.svg\")
\n\nLast but not least, we measured the size of the serialized buffer for each\ncontent-blocker:\n\n![\"Cache](\"../static/img/blog/adblockers_performance/cache-size.svg\")
\n\nFrom these measurements, we see that *Ghostery* offers both significantly\nfaster serialization and deserialization times as well as a smaller\ncache size.\n\nThe reason is the following:\n* The internal representation is already\n mostly stored in a compact form (using typed arrays).\n* This means that serialization only adds a small amount of metadata\n alongside the already available arrays. Deserialization is\n *essentially instantaneous* since it's enough to create some typed array\n views on top of the serialized buffer (think of `mmap` but using typed\n arrays).\n* This also explains the very low memory consumption: after\n initialization, the memory footprint is only slightly higher than the size\n of the serialized form.\n\n### 6. Memory Consumption at Start-up\n\nHere, we consider the memory usage of each content-blocker, initialized\nfrom lists (not from cache) after one full garbage collection.\n\n### How memory consumption was measured\n\nThe measurements were performed using Chrome's devtools memory snapshot. We\ndid not measure *Brave* here since the memory used from C++ side does not\nseem to be taken into account in the snapshot. Keep in mind that\nthis memory usage can vary at run-time as content-blockers might cache\nfrequently used resources, etc.\n\n![\"Memory](\"../static/img/blog/adblockers_performance/memory-usage-at-startup.svg\")
\n\nAs mentioned in the previous section on serialization, the very low\nmemory usage of *Ghostery* can be explained by the fact that the\ninternal representation mostly consists of very compact typed arrays\nwith some small overhead for extra meta-data.\n\nAgain, we need to stress\nhere that this measures the network filtering engine of Ghostery only,\nnot the full extension, as described in the introduction.\n\n### 7. Parsing Lists\n\nIn this graph, we present the time it takes for each content-blocker to\nbe initialized from the lists (without any prior caching, which means\ninitializing all internal resources by parsing the raw list).\n\nWe see that only *Brave* seems to be significantly slower and that *uBlock Origin*,\n*Ghostery*, *Adblock Plus* and *DuckDuckGo* all perform well.\n\n![\"Time](\"../static/img/blog/adblockers_performance/time-to-parse-easylist-all.svg\")
\n\nIt seems that the long parsing time for *Brave* is a [known\nissue](https://github.com/brave/ad-block/issues/158) tracked on their\nGitHub repository.\n\n### Key findings on parsing lists\n\nIf we remove *Brave*, we see that there are still differences between\n*uBlock Origin*, *Ghostery*, *Adblock Plus* and *DuckDuckGo*. One reason\n*Ghostery* is slower than *uBlock Origin* and *AdblockPlus* here is that to\nachieve maximum performance while matching as well as minimize memory\nusage, there is a bit more work to do up-front.\n\nIn practice, this does\nnot matter so much since it is a one-time operation and subsequent\nloads are performed from cache. This is really fast. In fact, we\ncan even perform the parsing backend-side and just ship the serialized\nversion of the blocker, which removes this step completely.\n\n![\"Time](\"../static/img/blog/adblockers_performance/time-to-parse-easylist-without-brave.svg\")
\n\n### 8. Conclusion\n\nIn this study, we looked closely at the performance of some of the most\npopular content-blockers in use today. In particular, we focused on the\nefficiency of their network filtering engines, which is the most CPU\nintensive task they perform.\n\nThis work was motivated by one of the claims formulated in the [Manifest V3\nproposal](https://bugs.chromium.org/p/chromium/issues/detail?id=896897)\nof the Chromium project: *\"the extension then performs arbitrary (and\npotentially very slow) JavaScript\"*, talking about content-blockers'\nability to process all network requests.\n\n### Key findings:\n\n* We do not think the Manifest V3 claim is accurate as all popular content-blockers are already\n very efficient and should not incur any noticeable slow-down for users.\n* Moreover, the efficiency of content-blockers is *continuously improving*,\n either thanks to more innovative approaches or using technologies like\n WebAssembly to reach native performance.\n* While most content-blockers are indeed efficient, they are not\n equivalent. We observed that *Ghostery* performs consistently as well\n or better across all dimensions, often surpassing other libraries.\n\nWe hope these benchmarks will give content-blocker developers the opportunity\nto measure their own progress against other popular libraries.\nThis will benefit all users, no matter which extension they use, as the efficiency of\ncontent-blockers improves.\n\n[*Edit of 20-02-2019*](https://github.com/ghostery/whotracks.me/pull/154): The study has been updated with the specific version of each content-blocker measured.\n\n[*Edit of 15-03-2019*](https://github.com/ghostery/whotracks.me/pull/161): DuckDuckGo's description has been amended to more accurately describe the way their content-blocker is used in practice: focusing on blocking third-party trackers, but not ads.\n"
},
{
"path": "blog/block-third-party-cookies.md",
"content": "title: Third-party cookies - the guests who won't leave\nsubtitle: How the web ecosystem is preventing us from reverting the third-party cookie mistake.\nauthor: privacy team\ntype: primer\npublish: True\ndate: 2018-08-27\ntags: blog, cookies\nheader_img: blog/blog-third-party-cookies.jpg\n+++\n\nSummary: _In this post we describe_\n\n * _How third-party cookies are the cause of multiple privacy and security issues on the web._\n * _That, despite the cookie spec writers recommending against it, all major browsers ended up allowing third-party cookies by default._\n * _How several major sites and services, including those from Microsoft and Google, fail badly when third-party cookies are blocked._\n * _How Cliqz and Ghostery are defusing the privacy issues of third-party cookies by blocking them, while preventing the breakage issues caused by developers assuming a cookie free-for-all._\n\n---\n\nCookies are a fundamental browser technology which enables state to be kept between browser and servers over the normally stateless HTTP protocol. Cookies enable sites to remember your login, what you've put in your shopping cart, and allow a site to remember language or currency preferences.\n\nThese features enable the sites you're visiting, 'first parties', to improve your experience and provide some content behind authentication, only accessible to you. However, browsers also, by default, send cookies to any third-parties embedded by the site operators. In some cases these can be used to allow third-party widgets, such as [Disqus](../trackers/disqus.html) comments, to automatically log you in to embedded content in the page. However, it also enables these third-parties to track your browsing across the web.\n\nAllowing cookies to third-parties opens up a privacy hole in your browser. On [many sites](../websites.html), just visiting a page will set cookies for over 50 different third-party domains. Each of these are setting cookies so they can [correlate requests](./how_facebook_knows_exactly_what_turns_you_on.html) coming from your browser over days, months, or even years. For example, when you visit any page with a Facebook widget (or visit Facebook itself), they will set a cookie which will only expire in 2 years time. Some google.com cookies expire in 20 years. The `facebook.com` and `google.com` domains are present as a third-party to [24%](../trackers/facebook.html) and [30%](../google.html) of page loads\non the web respectively, allowing these services to tracker this proportion of the average user's web browsing history.\n\n\n![\"Google's](\"../static/img/blog/cookie_block/google_cookie.png\")
\n ![\"A](\"../static/img/blog/cookie_block/economist_cookie.png\")
\n Google's consent cookie lasts for 20 years; A tracking cookie on the Economist which lasts for 68 years.
\n\nThird-party cookies also represent a security risk to you. [Cross-site request forgery](https://en.wikipedia.org/wiki/Cross-site_request_forgery) (CSRF) attacks are based on the idea that I can make a third-party request to a site that the browser has previously authenticated with, and the browser will send the credentials with the request. If browsers did not allow third-party cookies these attacks would be much harder to exploit than they currently are. These kinds of attacks have been around for over 15 years, and methods to mitigate them are [still being proposed](https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/), while\nbrowser-side protection, such as [first-party isolation](https://wiki.mozilla.org/Security/FirstPartyIsolation), have very limited distribution.\n\nFurthermore, the use-cases which legitimately use third-party cookies, like Single-Sign-On portals, or third-party authentication mechanisms, have alternatives which do not require cookies. Sites using a centralised authentication domain can obtain authenication tokens via first-party redirects, and OAuth[^2] can be used to log in to sites using third-party credentials. These mechanisms have the added bonus of transparency and implied consent: When a user logs in with Facebook on a site, the user is actively allowing this connection between the site and Facebook to proceed.\n\nSo why do we have third-party cookies? Actually, the original 1997 [RFC Specification](https://tools.ietf.org/html/rfc2109) of the cookie standard proposed that third-party cookies should not be allowed on privacy grounds:\n\n> This restriction prevents a malicious service author from using unverifiable transactions to induce a user agent to start or continue a session with a server in a different domain. The starting or continuation of such sessions could be contrary to the privacy expectations of the user, and could also be a security problem.\n\nand browsers should have this setting by default:\n\n> User agents may offer configurable options that allow the user agent, or any autonomous programs that the user agent executes, to ignore the above rule, so long as these override options default to \"off\".\n\nHowever, these recommendations were not implemented by browser developers at that time, and the default of _'allow all cookies'_ has remained since then.\n\nCurrently, almost all major browsers have a default to allow all cookies. The one exception is Safari, which only allows third-party cookies for domains which have been visited as a first party. This setting mitigates tracking from unknown domains, but still allows others to track, and does not prevent CSRF attacks. Mozilla also previously [attempted](https://blog.mozilla.org/netpolicy/2013/02/25/firefox-getting-smarter-about-third-party-cookies/) to change Firefox's default handling of third-party cookies in 2013, but pressure from the Ad industry led to a [U-turn](https://blog.mozilla.org/blog/2013/05/10/personalization-with-respect/) before these changes went live. The failure of browsers to handle third-party cookie tracking [has argueably led](https://medium.com/the-graph/how-to-reverse-publisher-revenue-drain-c33e41bf0665) to the increase in adblocker usage since then.\n\nThe effect that this default has had over the last 20 years, is that developers now assume that cookies are allowed in all contexts. This causes many workflows to break once this assumption is broken. This leads to a vicious cycle, where attempts to limit third-party cookies are foiled because they break too many sites. Apple's push to reduce third-party cookie tracking with their [Intelligent Tracking Prevention](https://webkit.org/blog/7675/intelligent-tracking-prevention/) technology had to include a section to explain to developers how to solve several use-cases when their cookies are limited. This technology still allows third-party cookies from visited sites however, and this method is also recommended for implementing single sign-ons.\n\n## Moving away from third-party cookies\n\nIn 2015 Cliqz[^1] released an anti-tracking technology which [aggressively blocks third-party cookies](./how_cliqz_antitracking_protects_users.html). Third-party cookies are blocked unless certain heuristics are triggered. These heuristics aim to mitigate common cases where cookie blocking breaks workflows, but also require user action to trigger. A Facebook button can be loaded without cookies, but if the user then clicks on it, there is an implied consent to allow the cookies in this case. This method blocks 97% of third-party cookies, with minimal breakage of pages.\n\n| Browser | \nDefault Cookie setting | \n
|---|---|
| Google Chrome | \nAllow all. | \n
| Mozilla Firefox | \nAllow all. | \n
| Apple Safari | \nAllow from visited; tracking cookies limited. | \n
| Cliqz Browser / Ghostery extension | \nBlock all third-party, unless user interaction or compatibility exception. | \n
![\"Office](\"../static/img/blog/cookie_block/office_signed_out.png\")
\nLooks like I'm logged out...
\n\nAfter the confirmation of a successful logout, simply navigate back to www.office.com, and one is returned to the view after login, including an up-to-date feed of recently changed documents.\n\n![\"Office](\"../static/img/blog/cookie_block/office_documents.png\")
\nDocument change feed still shown after logout.
\n\nThe API that makes this information available after logout is fetched via the [SharePoint REST API](https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/sharepoint-net-server-csom-jsom-and-rest-api-index), and the authentication token for this is not deleted nor expired after the failed logout - hence the page can continue to access this information. The token can be collected from the developer tools and then reused for API calls, for example to list folders in this organisation's SharePoint:\n\n```javascript\nvar accessToken = \"eyJ0...\";\nvar baseUrl = 'https://org-my.sharepoint.com/_api/';\nvar headers = new Headers();\nheaders.append('Authorization', `Bearer ${accessToken}`);\nheaders.append('Accept', 'application/json;odata=verbose');\nfetch(`${baseUrl}web/lists`, { headers })\n .then(resp => resp.json())\n .then(res => console.log(res))\n```\n\nThe broken logout state can only be resolved by manually deleting office.com cookies. We also found the session may eventually be expired, but this only happened after multiple hours. Hence, users affected by this will 1) likely not be aware that they're not logged out properly, as the logout appears to be successful, and 2) would not be able to logout anyway if they noticed the issue.\n\nThe issues with Office continue when trying to purchase an Office365 trial from `https://products.office.com/try`. This time, the source of the problem is detected, but the user is given no choice to continue unless they compromise their security and privacy by enabling cookies. Ironically, they also imply that allowing third-party cookies is somehow safer.\n\n![\"Cookie](\"../static/img/blog/cookie_block/ms_payment_fail.png\")
\nIt is not possible to buy Office without allowing third-party cookies.
\n\n### Pay with your Cookies\n\nIt is common practice for E-Commerce sites to embed payment systems from third-party vendors, such as Paypal, on their checkout pages. Such widgets should not require third-party cookies - usually the user can be redirected to pay at the payment provider's site. This method is preferable, as it reduces the chances of phishing: loading the payment page as a first party will make the url and certificate status visible, and only prompting users to enter payment information on the first party site is also good practice.\n\nDespite this, we see examples of payment being blocked when third-party cookies are disabled. One such example is on the German E-Commerce site [Thomann.de](../websites/thomann.de.html). When attempting to checkout with Amazon pay, we get an error mentioning that third-party cookies are being blocked:\n\n![\"Amazon](\"../static/img/blog/cookie_block/amazon_pay_thomann.png\")
\n\"There was an error processing the Amazon payment. A possible cause is third-party cookie blocking.\"
\n\n### Connect with Google? Third-party cookies required\n\nMany sites use Google's connect SDK, to allow users to login to sites with their Google account. When testing cases on [www.tripadvisor.com](https://www.tripadvisor.com) and [www.stumbleupon.com](https://www.stumbleupon.com) with third-party cookies disabled, the 'Connect with Google' button fails to do anything when clicked. Both these sites also offer Facebook login too which works with cookies disabled. It is not clear why the Google implementation requires third-party cookies to be allowed.\n\n![\"Tripadvisor](\"../static/img/blog/cookie_block/tripadvisor_connect.png\")
\n Tripadvisor signup buttons.
\n\n### Please let me track your tracking opt-out\n\nFollowing GDPR, websites using third-party services which collect data about users [acquire consent](./update_jun_2018.html) for this, as well as provide a reasonable way of opting-out of data collection and processing. While many publishers have converged on a solution which [gathers consent as a first-party cookie](https://iabtechlab.com/standards/gdpr-transparency-and-consent-framework/) which can then be passed to third-parties, other still rely on an older system of setting opt-out cookies for each vendor. Obviously, if third-party cookies are blocked, this mechanism will not work, as can be seen on the [Telegraph](../websites/telegraph.co.uk.html):\n\n![\"Cookie](\"../static/img/blog/cookie_block/telegraph_optout.png\")
\n\"You browser is currently blocking 3rd party cookies ... you will need to enabled 3rd party cookies if you want all of the opt-outs on this page to work.\"
\n\nIn this case, users with third-party cookies disabled will be denied their right to opt-out (though blocking these cookies will effectively prevent a large proportion of tracking).\n\nThird-party vendors may say that this mechanism is required in order to remember a user's consent settings. However, previous attempts to allow browsers to convey tracking consent explicitly to servers, via the ['Do Not Track'](https://www.w3.org/TR/tracking-dnt/) standard were killed by the same vendors collectively saying they would [ignore this signal](https://blogs.harvard.edu/doc/2015/09/23/how-adtech-not-ad-blocking-breaks-the-social-contract/).\n\n## Conclusion\n\nWhen the idea of cookies was first proposed, the standard writers were concerned about the privacy implications of allow third-party cookies, and specified that browser vendors should disable them by default. Fast-forward 20 years and the majority of browsers on the web will allow all third-party cookies. The result of this are significant challenges to protect against Cross-site request forgery, with countless sites and accounts compromised along the way, and pervasive privacy invasion in the form of cross-site tracking of users.\n\nWe argue that we should aim to return to a web where third-party cookies are blocked by default, and are making that possible for users of our anti-tracking technology in [Cliqz](https://cliqz.com/) and [Ghostery](https://www.ghostery.com/), however this is made difficult by the prevailing assumption that cookies are a free-for-all, making many sites fail to function properly in this environment. In this regard we are constantly improving heuristics to mitigate the breakage issues we do find.\n\nWe showed multiple cases where the assumption that third-party cookies will be allowed lead to both benign and potentially dangerous issues for users who block cookies. Some of these cases affect payments, so perhaps if cookie-blocking becomes more common and companies' bottom lines are effected these issues will be fixed. This is a chicken and egg problem though, if the web is broken for users blocking cookies, then we may never achieve the critical mass required to get it fixed.\n\n**For users**, getting control over which cookies your browser sends out, and to whom, is a key part of protecting privacy online, but also something that is not universally recognised by\nbrowser privacy tools. Most adblockers, for example, do nothing to the cookies of third-party requests which are not on their blocklists. More adoption of the kind of cookie blocking that Cliqz and Ghostery do help us to achive this critical mass, and push more websites to ensure that their services still work correctly for users who chose more private browser configurations.\n\n**Developers** have a part to play here too. By building services which do not require third-party cookies, or at least continue to function without them, it becomes easier for users to turn off third-party cookies, and the web becomes more privacy-friendly. As we have seen in this article, even the biggest tech companies are currently failing at this, but this seems to be more due to a lack of awareness, than any difficultly in implementation.\n\n[^1]: Disclosure: WhoTracks.Me is operated by Cliqz.\n[^2]: Note that both of these methods also have some privacy issues. First-party redirection has been exploited for [user tracking](https://brave.com/redirection-based-tracking/), and OAuth dialogs can trick users into granting [many more permissions](https://lifehacker.com/how-to-revoke-pokemon-go-s-extensive-permissions-to-you-1783466118) than they actually need.\n" }, { "path": "blog/cookie-consent.md", "content": "title: Improving Cookie Consent\nsubtitle: Cliqz' new feature to make consent fairer\nauthor: privacy team\ntype: article\npublish: True\ndate: 2019-11-28\ntags: blog, gdpr, consent\nheader_img: blog/autoconsent/cookie-blocker-prompt.png\n+++\n\nSince the GDPR came into force in May last year, the Cookie-Consent Popup has become a fixture of browsing the web. These popups are ostensibly there to allow you to choose whether you agree or disagree to your data being used for certain purposes on the site, but confusing UI design and tricks mean that many users are not able to select their desired consent settings. A recent [study](https://arxiv.org/pdf/1909.02638.pdf) showed that user fatigue with consent popups, and simple UI tricks are able to artificially inflate the opt-in rate. The study also showed that, when opt-out is the default, only 0.1% of users would consent to all data processing. This is in stark contrast to the over 90% opt-in rate that the [industry claims](https://www.thedrum.com/news/2018/07/31/over-90-users-consent-gdpr-requests-says-quantcast-after-enabling-1bn-them), and uses to justify that users are OK with tracking.\n\nHow can we restore balance to this situation, and allow users a fair choice about how their data is used? At Cliqz we have been developing a new feature to aim to address the difficulty of denying consent based around 3 core principles:\n\n1. Opt-out and opt-in should both require maximum of one click, i.e. the time-cost should be the same, no matter which choice is made.\n2. The user should not have to decide individually for every site. Their default choice can be used to give consent after their initial decision.\n3. Consent banners only offering an 'OK' or 'Allow' option do not allow user choice. The are at best a distraction for the user, and at worst drive consent fatigue and encourage the bad practice of automatically clicking away message prompts. These should be hidden.\n\nUnfortunately, implementing an automated consent choice in the browser is made challenging by the lack of adoption or adherence to browser standards. The [Do Not Track](https://www.w3.org/blog/2018/06/do-not-track-and-the-gdpr/) standard enables users to broadcast preferences around tracking, and for sites to communicate tracking status to the browser. Before that, the [P3P Project](https://www.w3.org/P3P/) attempted to standardise privacy practices and allow automated decision making around them. Both of these standards have been rejected by the tracking industry, who prefer to present consent on their terms. The industry have instead proposed and implemented the [Transparency and Consent Framework](https://iabeurope.eu/transparency-consent-framework/), which primarily focuses on communicating consent between vendors. It is a read-only API, so the browser can only read the consent status as set by the site, and not modify it. This means that consent can currently only be expressed by clicking through HTML forms.\n\n![\"Navigating](\"../static/img/blog/autoconsent/cookie-blocker-before.gif\")
\nNavigating a Cookie-Consent Popup manually.
\n\nLuckily, the number of vendors offering consent solutions is limited, and browser extensions can simulate clicking through forms. Thus, [autoconsent](https://github.com/cliqz-oss/autoconsent) was born - a library of rules standardising the navigation of consent forms for the most popular sites and vendors. This library is able to:\n\n* Detect the presence of supported Consent Management Providers on a page.\n* Determine whether a popup or overlay is being shown on the page.\n* Execute an opt-in (allow all purposes) or opt-out (reject all purposes).\n* Where available, re-open the popup to allow modification of the settings.\n\nIn practice, this allows consent popups to be rapidly dismissed when loading a new site. The speed depends on the provider and how quickly their UI can be manipulated. In all cases, however, this is faster than a user could navigate the interface.\n\n![\"Automatic](\"../static/img/blog/autoconsent/cookie-blocker-after.gif\")
\nAutomatic navigation of the Cookie-Consent Popup.
\n\nFor popups that are informational only, or force affirmative consent, we apply simple cosmetic rules. These are CSS rules that define elements in the page that should be hidden. As with the consent rules, we benefit from the defacto standardisation of tools for displaying of popups, such that a small number of rules can support the majority of popups shown by websites.\n\nThese elements combined mean that we now just have to ask the user once whether they want to opt-in or opt-out, then they will not be bothered by consent popups on the majority of sites they visit. At the same time, they will signal to these sites their approval or dissapproval of their data collection practices. \n\nThis signal of non-consent is important to encourage and incentivise a shift in data usage practices on the web. When sites realise they cannot just trick users into allowing invasive data collection, they will have a strong incentive to change the way they operate and respect users more.\n\nThe new Cliqz Cookie-Popup blocker is available in the latest version of the Cliqz browser. Get it at [cliqz.com](https://cliqz.com/download).\n" }, { "path": "blog/cookies.md", "content": "title: Cookies\nsubtitle: A small piece of data sent from a website, meant to 'help', used to track.\nauthor: privacy team\ntype: primer\npublish: True\ndate: 2017-07-22\ntags: primer, tracking\nheader_img: blog/blog-cookies.jpg\n+++\n\nAn HTTP cookie (also called web cookie, Internet cookie, browser cookie, or simply cookie) is a small piece of data sent from a website and stored on the user's computer by the user's web browser while the user is browsing. Cookies were designed to be a reliable mechanism for websites to remember stateful information (such as items added in the shopping cart in an online store) or to record the user's browsing activity (including clicking particular buttons, logging in, or recording which pages were visited in the past). They can also be used to remember arbitrary pieces of information that the user previously entered into form fields such as names, addresses, passwords, and credit card numbers.\n\nOther kinds of cookies perform essential functions in the modern web. Perhaps most importantly, authentication cookies are the most common method used by web servers to know whether the user is logged in or not, and which account they are logged in with. Without such a mechanism, the site would not know whether to send a page containing sensitive information, or require the user to authenticate themselves by logging in. The security of an authentication cookie generally depends on the security of the issuing website and the user's web browser, and on whether the cookie data is encrypted. Security vulnerabilities may allow a cookie's data to be read by a hacker, used to gain access to user data, or used to gain access (with the user's credentials) to the website to which the cookie belongs (see cross-site scripting and cross-site request forgery for examples).[[1](http://news.cnet.com/8301-10789_3-9918582-57.html)]\n\n## Tracking Cookies\nThe tracking cookies, and especially third-party tracking cookies, are commonly used as ways to compile long-term records of individuals' browsing histories – a potential privacy concern that prompted European [[2](http://webcookies.org/faq/#Directive)] and U.S. lawmakers to take action in 2011. European law [[3](http://www.bbc.co.uk/news/technology-12668552)] requires that all websites targeting European Union member states gain \"informed consent\" from users before storing non-essential cookies on their device.\n\nThe excerpt above has been retrieved from [wikipedia](https://en.wikipedia.org/wiki/HTTP_cookie).\n\n\n#### References:\n\n[1] [Gmail cookie stolen via Google Spreadsheets](http://news.cnet.com/8301-10789_3-9918582-57.html)Percentage change in traffic to google search result pages, April 2018
\n\nWe can further see the magnitude of this change by focusing on data for Germany. If we look at the\nrelative proportion of pages loaded on `www.google.de` and `www.google.com` in Germany over the last\nmonth, we see a marked increase, with the share of traffic to `www.google.com` going up from around\n5% to over 40%.\n\n\nSearch results pages used in Germany, April 2018
\n\nWhy is Google doing this? We don't know - we're not aware of any official announcement. However,\none reason for this could be a reaction to increased usage of restrictive cookie settings, such\nas allowing cookies only from visited sites, or Apple's [Intelligent Tracking Prevention](https://webkit.org/blog/7675/intelligent-tracking-prevention/).\nIf a user is rarely visiting the google.com domain, these technologies can expire this cookie earlier,\nor prevent its use in third-party contexts.\nAs `google.com` is the domain used to authenticate with Google services, if the browser sends\n`google.com` cookies in third-party context, these visits can be directly attributed to one's\nGoogle profile. Therefore, this change increases the likelihood that the user will have recently\nvisited `www.google.com`, and therefore Google's tracking can continue uninterrupted.\n\nTracking from `google.*` domains [reaches 30% of web traffic](https://whotracks.me/trackers/google.html),\nand the majority of this reach is contributed by the [`google.com` domain](https://github.com/ghostery/whotracks.me/blob/master/whotracksme/data/assets/2018-03/global/domains.csv#L5).\nAs, with this change, it is very difficult to avoid visiting `google.com` domain as a first party,\npreventing this tracking in a vanilla browser would require disabling all third-party cookies.\nAlternatively, [Cliqz](https://cliqz.com/) and [Ghostery's](https://www.ghostery.com/) AI anti-tracking\ntechnologies block all third-party tracking cookies (Disclosure: the author works on this product).\n[Privacy Badger](https://www.eff.org/privacybadger) is also able to block third-party tracking cookies.\n" }, { "path": "blog/government_websites_september.md", "content": "title: Government websites\nsubtitle: If you are not the product, you're the taxpayer\nauthor: privacy team\ntype: article\npublish: True\ndate: 2018-10-10\ntags: trackers, government\nheader_img: blog/gov_trackers/gov.png\nredirect_url: https://www.ghostery.com/blog/government-websites-trackers\n+++\n\n\n_This post is one of our regular monthly blogs accompanying an update to the data\ndisplayed on WhoTracks.Me. In these posts we introduce what data has been added as\nwell as point out interesting trends and case-studies we found in the last month._\n\nAverage number of trackers seen on selected government websites from\nthe WhoTracks.Me September dataset.
\n\nHere's a list of the government websites ending up in this month's release:\n| Country | \nSite | \nNotable trackers | \n
|---|---|---|
| Australia | \nbom.gov.au | \n\n Google Analytics,\n Doubleclick\n | \n
| Europe | \neuropa.eu | \n\n Google Analytics,\n Google,\n Twitter\n | \n
| France | \nants.gouv.fr | \n\n Google Analytics,\n Doubleclick\n | \n
| France | \nlegifrance.gouv.fr | \n\n AT Internet\n | \n
| France | \nimpots.gouv.fr | \n\n AT Internet\n | \n
| Russia | \nzakupki.gov.ru | \n\n Yandex\n | \n
| UK | \ntax.service.gov.uk | \n\n Google Analytics,\n Optimizely\n | \n
| US | \nca.gov | \n\n Google Analytics,\n Google,\n AddThis\n | \n
| US | \ndhs.gov | \n\n Google Analytics,\n Doubleclick\n | \n
| US | \nirs.gov | \n\n Google Analytics,\n New Relic,\n AddToAny,\n Youtube,\n Foresee\n | \n
| US | \nnih.gov | \n\n Google Analytics,\n Doubleclick,\n Google\n | \n
| US | \nnoaa.gov | \n\n Google Analytics\n | \n
| US | \nstate.gov | \n\n Google Analytics,\n Google,\n Youtube,\n Qualtrics\n | \n
| US | \nweather.gov | \n\n Google Analytics,\n AddThis\n | \n
![](\"../static/img/blog/antitracking_intro/antitracking-comparison-v2.png\"/)
\n\nSecondly, as these lists are manually maintained, they will always be playing catchup against trackers. Tracking companies can constantly change their URLs, domains and methods to prevent blocking, and those generating the lists must respond in each case.\n\nFinally, blocking lists bestow significant power to their curators. With blocking browser extensions being used ever more ([over 40% of users](https://downloads.pagefair.com/wp-content/uploads/2016/05/Adblocking-Goes-Mainstream.pdf) for some market segments), those who write the block list would have the power to cut off a significant proportion of a company’s traffic — deservedly or not.\n\nHeuristic approaches like Privacy Badger are limited by just having local knowledge. In many cases we will not know if data sent is unique to us until we have tested it in another browser and seen a different value, like in the fingerprinting example in the previous post. Thus some kind of collaboration is required between users to determine what data is safe, and what is not – and this is the method Cliqz’s anti-tracking uses.\n\n\n# Cliqz Anti-tracking\nAs we outlined in the previous section, blocklists have several drawbacks, and we did not want such an aggressive system. Likewise, relying purely local evaluation of whether data is a UID or not has significant limitations. Therefore, we designed a system which combines local with global evaluation of tracking data. It is also designed to be conservative — we only remove data which we determine to be UIDs, and leave the rest alone. Our system modifies request URLs instead of blocking. This aims to reduce site breakage, and enable services to collect data, provided it does not compromise the user’s privacy.\n\nLike existing tools we focus on removing UIDs in transmission, rather than trying to prevent UID generation. Therefore we have three transmission vectors: HTTP Headers, URL Path and Post data. The latter we currently do not handle as our data shows that the reach of this method is very low, however our system allows us to continually monitor the situation, should this change.\n\nThe Cliqz anti-tracking system is split into two subsystems. One that handles only Cookies, and the other which handles all other data sent in headers and the URL path.\n\n\n## Cookie Protection\nProtecting from tracking cookies is relatively simple. This is because the vast majority of third-party cookies have no function beyond tracking. Therefore we can very simply strip these from the request without breaking the page.\n\nHowever, blanket third-party cookie blocking is not an ideal solution, because some third-party widgets do require cookies, to, for instance, authenticate with their service. In order to enable this use case, our system allows cookies in cases when user interaction with the widget is detected. When this happens, the third party is temporarily whitelisted to allow cookies.\n\nThis implementation effectively prevents all cookie tracking, and rarely breaks the user-experience on web pages. This is despite the fact that over 96% of third-party cookies are removed.\n\n\n## Unsafe Data Removal\nThe second anti-tracking subsystem deals with the non-trivial problem of identifying whether the data sent in a request is ‘safe’ or not. By this we mean, that the data point could be used as a UID, i.e. it is unique to the user. Once we identify what data is unsafe we remove it from the request before it is sent by the browser. This means that if a tracker tries to aggregate their data using this UID, then all Cliqz users will appear as one, and thus will gain crowd anonymity. The algorithm therefore runs as follows:\n\nWhen a page is loaded, for each third-party request:\n\n1. Analyse the URL, headers and postdata of the request.\n2. Tokenise this data into key-value pairs.\n3. Evaluate the safeness of each key-value pair.\n4. If there are unsafe values, remove the data from the request.\n\nHow can we determine what data is safe or not? A UID is characterised as a value which is unique to a single user, and which is repeatedly seen by this user. Such values can be detected by aggregating the data seen by multiple users over a period of browsing time. However, by the time this aggregation would tell us what the UIDs were, it would be too late — the trackers would already have the data. Therefore, our algorithm does the inverse: Detecting the values which cannot be UIDs, and removing all other data.\n\nThe advantage of this method is that the protection for new users is available straight away. New UIDs will not be known by the system, and therefore be removed by default. Furthermore, the set of safe values is significantly smaller than the set of unsafe. Safe values will be categorical in nature, and therefore be only ever be `O(1)` in size, while the set of UIDs will be `O(n)`, where n is the number of users.\n\nWe can also classify many values as safe locally, without having to consult the global safe value set. Our system uses the following rules for local classification:\n\n- If a value has not been seen previously for the `(third party, key)` pair, it is safe.\n- If a value is too short, i.e. has too little entropy to be a UID, it is _safe_.\n- If more than 3 different values have been seen for a `(third party, key)` pair over a two day period, then the value is not persistant, and therefore safe.\n\nIf none of these rules are able to classify the value as safe, we use the global safe set, which tells us which values have achieved a quorum of users who all saw the same value.\n\n\n### Example\n\nConsider a hypothetical visit to the site `example.com`, which has tracker.de as a third party. After processing a request we generate a set of `T = [(s, d, k, v)]` tuples as follows:\n\n```javascript\nT = [\n (s= example.com, d= tracker.de, k= z, v= 1459866821),\n (s= example.com, d= tracker.de, k= fl, v= 21.0),\n (s= example.com, d= tracker.de, k= u, v= CCAAAABI),\n (s= example.com, d= tracker.de, k= vr, v= 1440x1024),\n (s= example.com, d= tracker.de, k= c7, v= e9d4a7e4d2185cec),\n]\n```\n\nWe can then evaluate these values:\n\n- `(s= example.com, d= tracker.de, k= z, v= 1459866821)`: This was the first time we saw the value `1459866821` for the given `(d, k)`, so this data is _safe_.\n- `(s= example.com, d= tracker.de, k= fl, v= 21.0)`: The value `21.0` is too short to be a UID, so it is _safe_.\n- `(s= example.com, d= tracker.de, k= u, v= CCAAAABI)`: More than 3 different values seen in the last two days for the same `(d, k)`, so _safe_.\n- `(s= example.com, d= tracker.de, k= vr, v= 1440x1024)`: Always the same value seen for this `(d, k)` pair, however the value is in the global safe set (as it represents a common screen resolution), therefore _safe_.\n- `(s= example.com, d= tracker.de, k= c7, v= e9d4a7e4d2185cec)`: Always the same value seen, and not in the globally safe set, so this value is _unsafe_ and will be removed.\n\n\n### Building the Global Safe Set\nWe build the global safe set daily using the data sent from users’ clients. Clients collect the tuples of data from each request while browsing and send this back to us every hour, adding a timestamp parameter. The values in the tuple are hashed to prevent user-identifiable information being sent to us. The client guarantees that a maximum of one message is sent per user per hour, so this can be used instead of a user id to count the number of users for each (third party, value) tuple. If the number of users exceeds the quorum threshold for a given hour, it is added to the safe set.\n\nThis model allows users to create this collaborative safe value set without compromising their privacy. Any privacy sensitive information in the data is obfuscated by the hash function, and, as no UID is required for our aggregation, we cannot derive any browsing history beyond the single hour granularity (there is no way to link messages from different hours), and the most information that could be gained is the first-party domain names visited. This protection is further strengthened by the use of our [Human Web](https://events.linuxfoundation.org/sites/events/files/slides/collecting-user-data-socially-responsibly.pdf) technology which further obfuscates the source of each message we receive.\n\n\n# Cliqz vs Other Anti-tracking Systems\nThe algorithmic, data-driven system for removing UIDs from third-party requests which we have described has several advantages over other anti-tracking solutions. As an online system we can respond much quicker, and without human oversight, to changes in trackers and their techniques. If a tracking company switched domains to try and avoid blocklists, we would have the data to block this tracking within a day. Human-curated blocklists would take comparatively longer to update.\n\nIn our [paper](https://static.cliqz.com/wp-content/uploads/2016/07/Cliqz-Studie-Tracking-the-Trackers.pdf) we did several tests to measure the difference between our system and other blocklist systems. Our tests indicated a reduction in breakage on web pages caused by our system, compared to adblockers. We also saw that blocklist-based systems blocked more often, but a large proportion of these blocks were false positives: requests which did not contain any UIDs.\n\nThe downside of our method is that, unlike other blockers, we see a net performance loss when loading complex web sites. This is because, other blocking systems simply block the javascript which will then attempt to calculate and send a fingerprint, while we will block just the outgoing request with UID. Thus blockers get a performance benefit of avoiding running this resource-heavy tracking javascript code.\n\nHowever, we believe that in the long term, this property is a net benefit. Unlike other anti-tracking systems, Cliqz can forgive. If a tracker updates its code, and switches to a method which no [longer sends UIDs](http://www.slideshare.net/jmpujol/data-collection-without-privacy-sideeffects-at-big2016-www-2016), then our system will immediately stop blocking their data. With blocklists, there is no such mechanism — trackers are then incentivised to circumvent the block (for example using a new domain name), rather than improving their data collection methods.\n\n\n# Summary\nOur anti-tracking system has now been running successfully for over a year, blocking around 300 million cookies and removing around 10 million UIDs per day to keep our users free from tracking. The same system, as described here, is now also available on [our mobile browsers](https://cliqz.com/en/mobile) to provide the same protection across even more devices, and give users control over what data third-parties can collect about them.\n"
},
{
"path": "blog/how_facebook_knows_exactly_what_turns_you_on.md",
"content": "title: How facebook knows exactly what turns you on\nsubtitle: A technical analysis of the methods used to track users as a third party. Deep dive into a couple of case studies.\nauthor: privacy team\ntype: article\npublish: True\ndate: 2017-07-22\ntags: privacy, circle\nheader_img: blog/blog-facebook.jpg\n+++\n\nThe modern web is built around advertising. A multi-billion dollar industry ([$42bn in 2013 in the US](http://www.iab.net/about_the_iab/recent_press_releases/press_release_archive/press_release/pr-041014) and about [6bn in 2015](https://de.statista.com/statistik/daten/studie/456157/umfrage/umsaetze-im-markt-fuer-digitale-werbung-in-deutschland/) in Germany) primarily concerned with one question: How to make the most money by serving exactly the right ad, at the right time, to the right user.\n\nWeb pages are extremely complicated constructions, often meshing together multiple software tools and services from different providers, from analytics and social sharing widgets, to dynamic advertising and content recommendation engines. Consider an average news site with social media sharing buttons. More often than not, these are created by linking to scripts from Facebook, Google, Twitter, etc., which then inject the required content into the page. These third parties may then in turn load other required services into the page.\n\nIn isolation, this seems mainly harmless. Services are being provided to the website owners to better integrate third-party services such as social networks, add extra widgets such as comment sections and related content, and improve the website’s monetisation through targeted advertising.\n\nHowever, the implementation of these services often cause a privacy side-effect: they allow third-parties to track your web-browsing across the web, and in some cases even link this history to you personally. When a user visits a new site the third parties included in the page can then look up the browsing history they have collected for this user, and then generate a personalised response based on this information. This is akin to being given a personalised newspaper where the adverts have been selected based on which articles you have read previously, in both this and other newspapers, any magazine articles you might have read, where you shop and what items you were looking for, [where you bank](http://cliqz.com/magazine/pressemitteilung-cliqz-tracking-beim-online-banking) and more. Our data shows that largest of these tracking third party services can be seen on [almost half of all pages you might visit](http://josepmpujol.net/public/papers/pujolTrackingTheTrackers.pdf), and many others share and [trade user data](https://big.exchange/) amongst each other in order to build a comprehensive user browsing history.\n\nLuckily, as the web is an open system, we can see what these companies are up to, and equip the browser with the capability of foiling their attempts to send tracking data.\n\nThis post is the first part of a two-part series. In the second part we will describe how our Anti-tracking system works. This part acts as a background to that, describing the how and why behind online tracking. The methodology and data we present here is based on our [published work](http://josepmpujol.net/public/papers/pujolTrackingTheTrackers.pdf) on Anti-tracking, which we [presented](http://www.slideshare.net/jmpujol/tracking-the-trackers-www-2016) at WWW2016.\n\n\n\n## How online tracking works\n\nThe mechanism behind online user tracking is simple enough. First, one must be a third party to many page loads across the web. Our data shows that almost 30% of web sites require 10 or more different service providers to fully load their content, and dynamic advertising alone can bring this many different companies into the page. Secondly, the request a third party receives when loaded into a page should contain some kind of user identifier (uid) for the client visiting the page, as well as the address of the first party page visited (usually provided by the Referer [sic] header). Collecting together the first-party pages seen for each uid will then yield the browsing histories of all the users seen.\n\nA simple list of visited web pages may not seem like a significant privacy violation to some, however further analysis can yield much more information than one might expect. Trackers can collect users’ browser and operating system, which can be used for [price discrimination](http://news.northeastern.edu/2014/10/ecommerce-study/), and rough geographical location can be checked using [IP geolocation](https://en.wikipedia.org/wiki/Geolocation). One can also find private urls in the history to determine membership of certain services, such as some [online banking portals](https://static.cliqz.com/wp-content/uploads/2016/07/Cliqz-study-tracking-in-online-banking.pdf) which contain trackers. Another example is the twitter analytics dashboard (e.g. analytics.twitter.com/user/sammacbeth/home). This url is only accessible when logged in as a specific user, and when accessed the browser will transmit this user name in the url to the trackers in this page (in testing, these included Google, Microsoft and tellapart.com), thus enabling these services to add a user’s twitter handle to the previously collecting browsing history. Private urls, such as this, are particularly dangerous, because they often contain Personal Identifiable Information (PII) which puts a real identify on the other urls that are being collected in that session (See [http://www.slideshare.net/jmpujol/data-collection-without-privacy-sideeffects-at-big2016-www-2016](http://www.slideshare.net/jmpujol/data-collection-without-privacy-sideeffects-at-big2016-www-2016) for an example.).\n\n\n### UID Generation Techniques\n\nThe uid that trackers need in order to attribute page loads to specific users can be generated in several different ways:\n\n1. Cookies – This is the simplest and most common method for generating uids. Cookies are a web standard for sharing state between a client and server over the stateless HTTP protocol. It is an important part of the web, which enables sites to keep track of your login and/or preferences between visits. Cookies work as follows: When a client makes a request to a server, in the response the server can set a header Set-Cookie with a value of its choosing. The client will, from then on, send this value in headers for any subsequent requests for this domain, and thus the server will know which user it was who sent the request.\n\n2. Network fingerprint – This method uses the properties of the network from which the request comes from as an identifier, usually the IP address. This varies in effectiveness based on whether users have unique IPs or not.\n\n3. Client fingerprint – Here, code is run in the client browser to try and build a unique identifier from data accessed in Javascript, Flash and other APIs, for example installed fonts, browser plugin versions, screen resolution, browser version and more. Techniques such as canvas fingerprinting are further able to fingerprint the specific hardware configuration of the user’s computer. Together this can generate a unique fingerprint which is stateless, and endures even when private data is cleared, and private tabs are used.\n\nOnce generated, these uids must be transmitted to the tracker with information about the page the user is visiting. Again there are three primary methods:\n\n1. HTTP Headers – This is metadata send along with a request with information for the server. This is where Cookies are transmitted, but also other data can be sent here. Our data shows that 45% of requests to third parties on web pages seen by our users contain a cookie header.\n\n2. URL Path – Arbitrary data can be sent in the URL path requested from the server. This is commonly in the form of a query or parameter string — key/value pairs separated by & or ; characters at the end of the query. 52% of third party requests have some kind of query string, and 1.5% a parameter string.\n\n3. Post data – This is data sent from the client as part of the main body of the request. We see this kind of request in 0.05% of cases.\n\n\n### Case Study 1: Facebook cookie tracking\nFacebook use cookies to link your web-browsing behaviour to your Facebook account. Facebook widgets are embedded in various sites around the web, and will send the address of the page you are viewing along with your Facebook cookie, enabling Facebook to build a list of sites you have visited. Our data shows that Facebook’s widget reaches 25% of pages loaded by our users – this means that Facebook could collect 25% of an average user’s browsing history.\n\nWe can see this tracking in action by inspecting requests in the web browser. First, if we visit the Facebook home page, we can see a cookie called datr being set:\n\n![](\"../static/img/blog/how_facebook_post/facebook-casestudy.png\"/)
\n\nNow, upon visiting a site which has a Facebook widget, in this case bild.de, we can see a request to facebook.com. As third-party cookies are enabled in the browser (the default setting in all major browsers), we will send the cookie we got on the previous page along with the request. The Referer header of this request will also contain the site I am visiting: www.bild.de.\n\n![](\"../static/img/blog/how_facebook_post/bild-tracker.png\"/)
\n\nAs I continue to browse the web, this process will repeat, and Facebook will collect a series of requests with this datr cookie and the pages I was viewing.\n\nFinally, if I now log into my Facebook account, we see that the datr cookie remains, and now alongside a cookie with my Facebook user ID. This means that Facebook can now attribute all the pages I have viewed with my personal Facebook account.\n\n![](\"../static/img/blog/how_facebook_post/facebook-tracker.png\"/)
\n\n\nThis mechanism allows Facebook to collect your browsing habits across the web, in order to tailor adverts and recommendations within their site. [Our measurements](http://josepmpujol.net/public/papers/pujolTrackingTheTrackers.pdf) show that this tracking covers around 25% of pages visited by our users.\n\nFacebook were banned last year from using this tracking on European users who had not logged into their site, however this was recently [overturned](http://www.theverge.com/2016/6/30/12069626/facebook-belgian-privacy-commission-cookie-user-tracking-case-overturned), so this practice continues.\n\n\n### Case Study 2: Moatads fingerprinting\n[Moat](https://moat.com/) is an analytics and advertising provider. They are present on many popular news sites, where their JavaScript is loaded into the page, and then a tracking pixel is sent back to their servers. We can observe this behaviour by opening two different sites in our web browser and inspecting the requests to moatads.com:\n\n![](\"../static/img/blog/how_facebook_post/moat-mac-uai-1440x774.png\"/)
\n\nHere we can see many parameters are sent in the request, and many values match across both requests. However, we cannot know for sure if these represent uids, or just other values used legitimately for the service. However, the qn value is suspicious, as a long cryptic value which remains the same when visiting different sites.\n\nWe now try opening the same sites in a different browser:\n\n![](\"../static/img/blog/how_facebook_post/moat_linux.png\"/)
\n\nAgain, pixels are generated with various parameters set in the request URL. Some are the same as we saw in the first test, for example the qq parameter. However, looking at the qn value we see that it is again the same on both web pages, but different to the value we saw on Mac. We can hypothesise that this is a fingerprint of this browser which functions as a uid, however we would need more examples from more unique browsers to properly test this.\n\nFinally, we test the qn in a private tab in the first browser. As shown below, we see that the same fingerprint is generated. Therefore, Moat are able to also tag page views in private tabs with the same uid as in a normal window, suggesting that they can bypass this protection for their tracking purposes.\n\n\n![](\"../static/img/blog/how_facebook_post/bild-tracker2.png\"/)
\n\n\n# Where are the trackers\nThese two case studies have shown the technical means with which companies can collect the pages you visit, and group them by a particular user, be that against a specific facebook profile or just a hash value which uniquely identifies one’s computer. Having established that third parties may snoop on some pages you visit on the web (with the first party’s permission), the question is how far does this tracking reach, and how much of our browsing habits can these third parties collect?\n\nWe presented our data on the [online tracking](http://josepmpujol.net/public/papers/pujolTrackingTheTrackers.pdf) seen by 200,000 users over a two week period at the [WWW2016](http://www2016.ca/) conference in April, which analysed over 13 million page loads by our users. A large study of 1 million sites has also been done by researchers at Princeton with similar findings to ours, although the study is not based on real user traffic but rather on data collected by instrumented browsers that download and scan for trackers top sites on the Web. We present some updated results from our on-going browser telemetry, during August 2016, and containing over 140 million page loads over 1.8 million unique domains. Multiple visits to the same site and/or page are counted multiple times, thus the data set weighs more popular pages more strongly, and represents the tracking observed by an average user of our browser.\n\nThe first result we observe from our data, is that a small collection of third parties are are installed in a huge number of visited pages. From a list of 2000 domains, representing the top tracking domains, we see that 96% of page loads include a request to a third party in this list. Over 80% of these page loads (and 78% of the total) contain some kind of tracking attempt. Thus, a user browsing the web with no tracking protection could be tracked on 78% of the pages they visit.\n\n\n![](\"../static/img/blog/how_facebook_post/tracking-per-page-load.png\"/)
\n\n\nWe can further look at how much of an average user’s browsing history each third-party company might be able to tracking. We analyse the ‘reach’ – the proportion of total page loads in the data set seen – by domains associated with particular companies or products. The figure below shows the top companies in terms of total reach, and for each we indicate the types of behaviour seen on each page. ‘Safe’ means that no tracking behaviour was seen, just that a request was made to the domain; ‘cookie’ and ‘qs’ mean that there was an attempt to transmit a uid with one of these methods, and ‘both’ means that both methods were used.\n\n\n![](\"../static/img/blog/how_facebook_post/tracking-reach-uai-1032x678.png\"/)
\n\nThe figures show that the big players – and particularly Google, with their products taking the top three places – have significant reach across the web. Some specific company behaviours can also be observed, for example Google Analytics does not use cookies, using a weaker kind of fingerprint. Also, Amazon, offer CDN services on their cloudfront.net and amazonaws.com domains, thus a high proportion of their reach is safe.\n\nThe other feature of the tracking landscape is the long tail of tracking companies. There are 27 companies/services with over 5% reach, 110 with over 1%, then 450 over 0.1%. This 0.1% still corresponds to over 140,000 pages seen on this data set.\n\nFinally, we can look at how many trackers are seen on each page load. The figure below shows how many distinct tracking domains were contacted for page loads in the dataset. We see that over 10% of pages have over 20 different trackers in them, and the vast majority of page loads have multiple trackers. Therefore, not only are users tracked across most of the web, after there are many companies who are able to generate comprehensive user profiles.\n\n![](\"../static/img/blog/how_facebook_post/domains.png\"/)
\n\n\n# Conclusion\nIn this post we’ve given a general description of how online tracking works, and looked at the extent of tracker companies’ reach across the web. In the next post we will look at how we can stop this tracking, and give an in depth description of how our Cliqz Anti-tracking technology works to prevent tracking without an adverse effect on user experience.\n"
},
{
"path": "blog/manifest_v3_privacy.md",
"content": "title: Chrome's Manifest V3 - Improving Privacy?\nsubtitle: How Chrome's changes will reduce user privacy\nauthor: privacy team\ntype: article\npublish: True\ndate: 2019-06-18\ntags: blog, extensions, privacy, chrome\nheader_img: blog/adblocker-perf-study.jpg\nredirect_url: https://www.ghostery.com/blog/manifest-v3-privacy\n+++\n\nThe Chrome team's proposed changes to browser extension APIs, known as Manifest v3, have proven controversial due to their expected impact on adblockers and privacy extensions. Of particular concern are the changes to the `webRequest` API, whose blocking capabilities are being replaced by the `declarativeNetRequest` API. In repeated posts the Chrome team claim that these changes are required to improve the *performance*, *security* and *privacy* of extensions. In a [previous post](./adblockers_performance_study.html) we showed that, for the most popular adblocker engines, performance is already very good, and these changes are unlikely to improve much. In this post we assess the privacy argument for the changes to request handling, if the proposed changes do improve privacy, and how Ghostery specifically will be affected. We find that:\n\n* The Chrome team have only belatedly stated specific privacy concerns with the `webRequest` API, and these are still not included in the design document.\n* The proposed changes do not provide any protections against the stated privacy issues.\n* Privacy extensions like Ghostery will be negatively impacted by the changes, reducing their ability to keep users safe online.\n\n## Extension privacy\n\nBrowser extensions have the potential to cause many privacy problems - when granted permissions, they can see every page you visit in the browser, view their contents, read and write form data, and send requests to any server on the internet. These powers are required for some of the valuable features extensions provide. Therefore, as the Chrome team rightly [point out](https://blog.chromium.org/2019/05/taking-action-on-deceptive-installation.html), ensuring extensions are consentfully installed is the first step to address privacy.\n\nThe Manifest v3 changes, however, primarily address extensions' capabilities post install. As privacy at this point is also a stated goal, what are the privacy concerns and attacks that the changes seek to address? In the Manifest V3 [design document](https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3NzzhHzc-qnk4w4PX-0XMw8/edit#heading=h.9lwe237fxtp2) this goal is stated as follows:\n\n> Users should have increased control over their extensions. A user should be able to determine what information is available to an extension, and be able to control that privilege.\n\nLater in the document the changes to the `webRequest` API are described, but only using a performance-based reasoning:\n\n> … the extension then performs arbitrary (and potentially very slow) JavaScript, and returns the result back to the browser process. This can have a significant effect on every single network request, ...\n\nThey also acknowledge that the `webRequest` API should remain in place for observation.\n\n> The non-blocking implementation of the webRequest API, which allows extensions to observe network requests, but not modify, redirect, or block them (and thus doesn't prevent Chrome from continuing to process the request) will not be discouraged.\n\nThis implies that the potential privacy impact of extensions being able to observe all requests going out of the browser are not a concern for these API changes. While the `webRequest` API remains, the switch to allow blocking only via the `declarativeNetRequest` API does nothing for the stated privacy goal of increasing user control over the information extensions can access.\n\nDespite this, since [our study](./adblockers_performance_study.html) showed that the performance cost of `webRequest` blocking for leading adblockers was not an issue, the Chrome team have focused on privacy reasons for the changes. In their [recent blog](https://blog.chromium.org/2019/06/web-request-and-declarative-net-request.html) about web request and declarative net request changes, they state:\n\n> In order to improve the security and privacy guarantees of the extensions platform, we are rethinking some of the extension platform's core APIs. That's why we're planning to replace the blocking Web Request API with the Declarative Net Request API.\n\nThis shift in angle has also come up in public statements by Chrome devs:\n\n> \"… The big problem with webRequest is unfixable privacy and security holes. …\" @justinschuh ([Source](https://twitter.com/justinschuh/status/1134060703231254528))\n\nIn the blog post they also mention one potential malicious use of webRequest:\n\n> Because all of the request data is exposed to the extension, it makes it very easy for a malicious developer to abuse that access to a user’s credentials, accounts, or personal information.\n\nIf this is the single privacy loophole the `webRequest` changes are targeting, then it seems strange that the solution is to remove the blocking capabilities of `webRequest` and leave the observational ones. Post Manifest V3, the exact same malicious extension will be possible. We can imagine that the Chrome team's strategy may be, that by providing a simple alternative API for blocking use-cases, the extension review process can be tougher for extensions asking for `webRequest` permissions. This, however, would also be possible by just introducing the new API, leaving `webRequest` as it is, and providing developer incentives to switch unless they really need `webRequest` for their use-case.\n\nIt is strange that this privacy issue was not stated in the original design document, and the proposed change to `webRequest` is seemingly just collateral damage that does not address the stated goals. More transparency is needed on what the strategy is here, and why keeping `webRequest` observation with blocking removed should be the solution.\n\nTo summarise:\n\n- The stated privacy improvements of Manifest V3 are addressed elsewhere in the proposals.\n- The privacy and security issues with `webRequest` blocking have not been fully articulated by the Chrome team, with only a brief mention of malicious behaviour in a blog post last month.\n- The removal of `webRequest` blocking does not improve the privacy of extensions.\n\nTherefore at this point, the primary impact on privacy from the proposed changes will be the neutering of the capabilities of several privacy extensions. Privacy Badger devs [expect their core functionality to be broken](https://github.com/EFForg/privacybadger/issues/2273) by the changes. Similarly, we expect it to be difficult to provide the same level of protection in Ghostery should these changes come into effect, and we will describe why in the rest of this post.\n\nIt is ironic that a change ostensibly aimed at improving user privacy will actually reduce it for many users who rely on privacy extensions to protect them online. Some have suggested that the changes simply align Chrome with Apple's Safari, which provides a similar declarative blocking API for extensions. This overlooks the fact that Safari comes with significant privacy protections by default, having been blocking most third-party cookies by default for years, and recently bringing in advanced anti-tracking measures in the form of [ITP](https://webkit.org/blog/8613/intelligent-tracking-prevention-2-1/). Chrome on the other hand, ships with zero tracking protection by default, and is now hindering extensions which try to provide comparible protections to other browsers.\n\n## How removing webRequest blocking affects Ghostery\n\nThis analysis is based on the `declarativeNetRequest` [API documentation](https://developer.chrome.com/extensions/declarativeNetRequest) as of 17th June 2019. The primary features of the API are:\n\n1. A matching grammar for specifying rules that will trigger blocking, header modification or redirects.\n2. Up to 30,000 static rules per extension\n3. The ability to add _dynamic_ rules at runtime, up to a maximum of 5,000 rules.\n4. Rules can have a white- or black-list of first-party sites, to control triggering.\n5. Individual sites can be dynamically whitelisted, up to a maximum of 100 per extension.\n\nGhostery contains the following components which will be affected by the webRequest API changes:\n\n### 1. Tracker matching and blocking\n\nGhostery contains a blocklist of over 4,000 filters which are used to detect and block trackers. The extension allows users fine-grained control over these, allowing or blocking specific trackers on specific sites or globally. The list of detected trackers is shown in the Ghostery UI for each page visited.\n\nTo support the `declarativeNetRequest`, these 4,000 filters would have to be re-written to the new filter grammar that Chrome offers. We are likely to lose some filters in the process, as certain types of matching rule, for example Regex's likely cannot be implemented in the more restrictive grammar.\n\nThe more challenging issue, however, is maintaining Ghostery's rich configurability with the low threshold of dynamic rules allowed. As every rule should be toggleable, all 4,000 filters would have to be _dynamic_ rules. This means that we are already using 80% of our allowance from the start, before we have even started adding supplementary rules for adblocking and cookie blocking.\n\nLikewise, the limit of 100 whitelisted sites is prohibitively low, as many users may us the Ghostery 'Trust Site' feature for more sites than this. It is unclear how to handle hitting this limit, as to the user it will seem like the feature is broken if they trust a site, but it does not get saved.\n\nFurthermore, the new API, in it's current form, does not report the results of blocking back to the extension. This means that we will still have to run our filters on all urls via the `webRequest` API anyway, in order to display the list of trackers seen and blocked. This means that the user pays the cost of keeping the block list loaded in memory and matching against each url twice.\n\n### 2. Cookie blocking\n\nThe Ghostery extension uses a heuristic third-party cookie blocker as part of the 'Enhanced Anti-Tracking' feature. This feature blocks third-party cookies in most cases, using a set of heuristics to decide when cookies should be allowed. It is currently not clear if these heuristics will be able to work correctly without the webRequest API, nor if the dynamic filter cap is sufficient to even hold the basic cookie blocklist.\n\nOur cookie heuristics respond to user input, for example clicking on a Facebook like button or Google login form, in order to trigger a temporary cookie whitelist for a specific domain. To implement this with `declarativeNetRequest`, we would have to add or modify our cookie blocking rule temporarily. As the API for this is asynchronous, we introduce a race condition that we did not have before. If the rule is not added before the request we want to whitelist, the mechanism will fail. This can, for example, break Google logins on third-party sites.\n\nThe cookie blocking is done based on a dynamically generated list of tracker domains of between 2,000 and 3,000 entries. For these domains, third-party cookies should be blocked, unless a heuristic allows it. Again, the limited rule threshold of the `declarativeNetRequest` API means that this list would have to be reduced.\n\nAnother concern is that the [Rule condition specification](https://developer.chrome.com/extensions/declarativeNetRequest#type-Rule) can distuingiush between `firstParty` and `thirdParty` contexts for a request, but this is done on a frame level, rather than relative to the page document. This means that we would not, for example, be able to block Google cookies inside a Google Ads iFrame, as in this context the API would consider requests from the frame as first party.\n\n### 3. Removing private data points\n\nThe other component of Ghostery's 'Enhanced Anti-Tracking' feature is the dynamic removal of url parameters seen to be used for cross-site tracking. This uses a [k-anonymity](./how_cliqz_antitracking_protects_users.html) based algorithm, using anonymously contributed data from our users.\n\nAs the `declarativeNetRequest` API does not support dynamic redirects, this component cannot be implemented with it.\n\n### 4. Adblocker\n\nGhostery includes an additional adblocker component which is able to further block ads based on standard blocklist. As this feature should also be toggleable on-and-off at runtime, we would need to use _dynamic_ rules for these filters. With only 1,000 rules available after adding the Ghostery tracker matching, the coverage of this feature would be drastically reduced.\n\n### 5. WhoTracks.Me Data\n\nGhostery is the primary source of data for this website, using our [anonymised telemetry system](https://arxiv.org/abs/1804.08959) to report on global tracker trends. This largely relies on the webRequest API in order to observe which trackers are on which page. Changes caused by the introduction of `declarativeNetRequest` will reduce the quality of this data. Namely, cookies blocked by the declarative API will not be visible to webRequest listeners. This means that we will not be able to distinguish between trackers setting cookies, which are then blocked, and those who do not set cookies.\n\n### Summary\n\nTo summarise, the Manifest V3 changes to the webRequest API will require a significant re-write of the Ghostery extension to be able to fit the existing features into the constraints of the `declarativeNetRequest` API. The result will be:\n\n- Slower: URL matching will have to be done twice in order to show tracker counts in the UI.\n- Less configurable: Configuration may have to be limited to fit within the very low dynamic rule limit.\n- Break sites more often: We will have to evaluate the trade-offs of relaxing the third-party cookie blocking vs. breaking sites.\n- Less private: As the private data removal feature will have to be removed.\n\n## Conclusion\n\nIn this post we have shown that the current proposed changes to the webRequest API by Chrome do not improve privacy, and in fact reduce it, by severely hindering the operation of privacy extensions like Ghostery. The limitations on dynamic rules in the new `declarativeNetRequest` API are particularly taxing for extensions which aim to the give user control over what is blocked and what is not.\nThis forces extensions into a 'dumb blocker' model, where block lists are fixed, and the only controls are an on/off toggle. At the same time,\nthe changes increase the difficulty and practicality of implementing dynamic heuristic mechanisms for detecting and blocking tracking.\n\nThe webRequest API powers much innovation in browser extensions, however it does implicitly provide access to private user data. While the Chrome team state that privacy is a reason for the proposed changes to this API they have not stated which specific concerns they aim to address. The Manifest V3 changes do not prevent extensions accessing private user data via webRequest, nor have other potentially dangerous APIs like content scripts been limited. Therefore the claims that this change improves extension privacy are misleading and disingenuous.\n\nThe fact that very few of the initial concerns regarding Manifest V3 have been addressed in the months since the original announcement, means that it currently looks like the changes will be forced through, despite community objections. This means that Chrome users will become second class web citizens with regards to their access to tracking protection. This is however just a continuation of a trend where Chrome stands still or actively reduces privacy while the rest of the competition have been pushing forward. At this point we recommend considering switching away from Chrome, if you haven't done so already, to browsers with privacy built-in by default. For example, the [Cliqz Browser](https://cliqz.com/en/download) has Anti-tracking built in and enabled by default, and Firefox now ships with [tracking protection on by default](https://blog.mozilla.org/blog/2019/06/04/firefox-now-available-with-enhanced-tracking-protection-by-default/).\n\n_Disclosure: WhoTracks.Me is a joint effort by Cliqz and Ghostery._\n"
},
{
"path": "blog/private_analytics.md",
"content": "title: Tracking visits without tracking people\nsubtitle: A privacy-by-design approach.\nauthor: privacy team\ntype: article\npublish: True\ndate: 2018-05-03\ntags: analytics, privacy-by-design\nheader_img: blog/analytics/analytics.png\nredirect_url: https://www.ghostery.com/blog/private-analytics\n+++\n\nAnalytics are one of the most common use-cases on the web. You want to know how many people are\nvisiting your website, whether anyone actually clicked the link you posted on social media, or who\nis sending traffic to your website. For most sites, the solution is to just drop a\n[Google Analytics](../trackers/google_analytics.html) script into the page - it's free, after all...\nThis has led us to the current situation, where we see Google Analytics having presence across 87%\nof the top half a million websites, and, despite using reasonably short-lived identifiers, the way the data is collected can be used to\n[track users across these sites](https://www.slideshare.net/jmpujol/data-collection-without-privacy-sideeffects-at-big2016-www-2016#13).\n\nIs counting page visits such a difficult problem that only Google has solved it? No, there are\n[paid](https://get.gaug.es/) and [open source](https://matomo.org/) alternatives available, but\nwhy pay when you can use a free version which does more, and why host a server with the extra\ncosts that entails, when you don't have to?\n\nBut is Google Analytics actually better than the competition? We would argue that, at least among\nprivacy conscious users (i.e. those\n[who contribute to the WhoTracks.Me dataset](../blog/where_is_the_data_from.html)), Google\nAnalytics will report vastly incorrect figures, for two main reasons:\n\n1. Our data shows that on 29% of pages with Google Analytics some of the requests will be blocked\ndue to Ghostery blocking settings.\n2. On 19% of pages with Google Analytics, Cliqz and Ghostery's AI anti-tracking will remove\npotential identifiers from the request, often causing unique visitors and conversions to be\nincorrectly measured.\n\n## Analytics without tracking\n\nSo how can we _accurately_ measure the traffic coming to our site without exposing the user to\ntracking and privacy side-effects? This was a problem we faced when we created the WhoTracks.Me\nwebsite. We wanted to have _some_ analytics so that we can measure if we are being successful in\nengaging people with the information we are providing on the site. However, we had a few\nconstraints:\n\n1. No tracking. We [define tracking](../blog/what_is_a_tracker.html) as when a service is able to\ncollect and correlate data across multiple sites. Unfortunately, as server-side aggregation is the\nnorm amongst third-party analytics providers, privacy cannot be guaranteed.\n[Client side alternatives](http://josepmpujol.net/public/papers/big_green_tracker.pdf) have been\nproposed, but unfortunately [the implementation](https://github.com/cliqz-oss/green-analytics) only\nreached a proof-of-concept state. This means we have to roll our own service.\n2. Minimal Ops. WhoTracks.Me is a statically generated site, which is simply hosted on a CDN. This\ndecision was made to minimise costs, make it fast, and eliminate the need to deploy and monitor\nhosting infrastructure. Having done this, it does not make sense to have to deploy infrastructure\nin order to host a [Matomo](https://matomo.org/) or similar service.\n3. Respect Privacy. The system should not store any personal information from users (i.e. IP\naddress), nor be able to correlate visits for an individual user over a long time frame. Apart from\nthe obvious reasons for this, it makes regulatory compliance easy: If we do not hold IP addresses,\nit is not possible for us to extract data on an individual user for data access or deletion\nrequests (as per GDPR).\n\nOur analytics implementation satisfies these three constraints, using probably the oldest technique\non the Internet: server log parsing. Daily analytics for the WhoTracks.Me site are generated as\nfollows:\n\n1. Visits to the site are logged via [CloudFront's logging mechanism](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html).\n2. Each day, a script processes these logs, to obfuscate personal data such as IP addresses. This\nscript generates a random key for the day, and encrypts all IP addresses with this key. The\nanonymised logs are copied to a new bucket, and the key is destroyed once the job completes. This\nmethod allows us to count unique visits from an IP address during a single day, but no day-to-day\ncorrelations can be made, nor can the IP address ever be recovered from the anonymised value.\n3. The original CloudFront logs (with IP addresses) are removed.\n4. We can then parse the clean logs and filter out requests to static resources and those by bots\nin order to see requests to actual pages. We can count unique visitors within single days, using a\ncombination of user-agent and anonymised IP; we can see where incoming traffic is coming from via\nHTTP referrers (which we also strip of potentially revealing parameters) and so on.\n\n\nProcessing of raw CloudFront logs to remove potential personal data.
\n\nThis workflow allows us to keep track of how much traffic we are getting to the WhoTracks.Me\nwebsite. There is also no reason that this method could not be scaled up to more complex use-cases\nwhich services like Google Analytics provides, like conversion counting - provided the time frame\nthat this conversions can occur in are shorter than the time the IP encryption key is used for.\n\nThe method is also safe with respect to privacy regulations and user preferences. As IPs are stored\nfor maximum 1 day (and this is only because CloudFront's logging does not obfuscate IPs for us), no\nother personal information is collected, and message linkage limited to 1 day, there are no\nadditional obligations regarding the usage of this data under GDPR. Furthermore, as tracking is time\nlimited and context limited (this data can only be used for usage on whotracks.me), it respects\n[Do Not Track](https://en.wikipedia.org/wiki/Do_Not_Track) automatically (using the standard's own\n[tracking definition](https://www.w3.org/TR/tracking-dnt/#terminology.activity)).\n\n## Conclusion\n\nWe rolled our own analytics for this site because there was no off-the-shelf solution providing the\n(very basic) analytics we wanted without significant extra overhead, or potential privacy\nimplications for users of the site. Our system leverages CloudFront logging with a data obfuscation\nstep in order to collect privacy-safe server logs which can then be analysed for basic insights.\nThis technique could be extended to provide most of the richer features of existing web analytics\ntools.\n\nThe lack of privacy-preserving tools in the web analytics ecosystem is a worrying trend. Google\nAnalytics dominates as they provide an extremely feature-rich product as zero cost to the webmaster.\nIt is difficult to see how a service can compete with free without selling analytics data. Existing\ncompetitors mostly aim for businesses who will pay for a premium product, and leave bloggers and\nsmaller sites to Google.\n\nWhile increasing use of adblockers is a more fundamental threat to Google's Ad business, a side\neffect may be a loss of trust in Google Analytics, as we measure\n[29%](https://github.com/ghostery/whotracks.me/blob/master/whotracksme/data/assets/2018-03/global/trackers.csv#L2)\nof pages with Google Analytics being affected by blocking. We already see companies which rely on\nanalytics for core business activities (for example advertisers using affiliate schemes) deploying\nmultiple analytics scripts and averaging the results. If the trust in analytics breaks down, then\nthis whole ecosystem may unravel.\n" }, { "path": "blog/static_site.md", "content": "title: Building whotracks.me\nsubtitle: Adding search, data, plots and blog to 1000+ pages of tracker profiles and top domains.\nauthor: privacy team\ntype: article\npublish: False\ndate: 2017-11-03\ntags: tracker-free, lightweight\nheader_img: blog/blog-site.jpg\n+++\n\nAt Cliqz and Ghostery, we [collect anonymous data about trackers](/blog/where_is_the_data_from.html)\nto power our [anti-tracking](blog/how_cliqz_antitracking_protects_users.html) technology.\nWe see our anti-tracking as a community effort and as such we want to share a structured\nrepresentation of this data to cast some light on the tracker landscape.\n\n\nOut of the three main entities involved in a page load: **users**, **websites** and **trackers**,\nwe have data only on the last two. We'll start with:\n\n* Profiles of the [top 500 trackers](/trackers.html)\n* Tracker data on the [top 500 domains](/websites.html).\n\n\nWith these out of the way, a blog space would be needed. This for two\nof reasons. We realised there was a need for a learning space where we\nexplain concepts referred to in the site. We call these **primers**.\nThese define what we call a [tracker](/blog/what_is_a_tracker.html),\nwhat [cookies](/blog/cookies.html) and [fingerprinting](/blog/fingerprinting.html)\nare or [where this data comes from](/blog/where_is_the_data_from.html).\nHopefully over time it will become a space for curious readers\nto be introduced to tracking technologies. The second reason is to have\na space where we'll be writing about particular trackers, technologies, papers,\nengineering, and other interesting topics.\n\n\n\n## Going static\n\nThrough whotracks.me, we want to cast some light on the tracking\nlandscape, but also make a point about trackers and **privacy by design**,\nhence the choice of this being a static site was pretty obvious. This\nmeant that we could build the whole site offline, put it in a folder\nand serve it through CDN.\n\nGiven this will be updated a few times a month, build performance was not really\na big issue for us. But stumbling upon [a discussion](https://news.ycombinator.com/item?id=15507538)\nabout site generators' performance, some comments read:\n\n- *\"with Hugo + Pygments was taking ~20s for ~20 pages at the time\"*\n- *\"92 pages in 1s (full rebuild, No CSS magic tooling though)\"*\n- *\"Rust: ~10k pages in ~60s\"*\n\nThe assumption would be that most of this time is spent parsing the\nmarkdown files. To build this site however, with the exception of the\nblog, the rest of the pages are mainly about instantiating a template,\nplugging some content, and writing to disk. So most likely a comparison between\nsite generators and this would be unfair. At the time of writing,\nwhotracks.me has roughly 1020 pages. On 1000 of these pages there are\noffline generated plots, quite some data and a fair amount of tooling\nwith respect to styling. On a `Thinkpad x230` with an Intel `i3 processor`:\n\n\n```bash\n(venv) ➜ whotracks.me git:(master) ✗ time python build.py site\nHome page ............................... done\nTracker list ............................ done\nWebsite list ............................ done\nBlog List ............................... done\nBlog Posts .............................. done\nWebsite pages ........................... done\nTracker Pages ........................... done\npython build.py site 13.86s user 1.08s system 158% cpu 9.400 total\n```\n\nThis will be a 5 part series dedicated to:\n\n1. [Generating a static site (part 1)](/blog/static_site_generation.html)\n2. [Visualization (part 2)](/blog/static_site_visualization.html)\n3. [Building a blog (part 3)](/blog/static_site_blog.html)\n4. Search, for some definition of search.\n5. No third party trackers and Fast\n\nThe code and data to generate this site is open-sourced at\n[`https://github.com/ghostery/whotracks.me`](https://github.com/ghostery/whotracks.me).\n\n![](\"../static/img/blog/plotting/display_mode_bar.png\")
\nFigure 2: Mode bar on top right corner of plotly plots.
\n\n`include_plotlyjs` is set to `False` to avoid `plotly.js` being loaded inline\nwith the `div` output for every plot. This is not necessary as it is already\nlinked in [`base.html`](https://github.com/ghostery/whotracks.me/blob/master/templates/base.html).\n\n\n## Bar Chart\nOn main page of this site, you will see this:\n\n![](\"../static/img/blog/plotting/bar-chart.png\")
\nFigure 3: Horizontal bar chart on tracking reach of top 10 companies
\n\nThe code to generate this can be found in [`plotting/companies`](https://github.com/ghostery/whotracks.me/blob/master/plotting/companies.py).\nLet's write a simpler function for a horizontal bar plot to get the idea:\n\n\n```python\ndef horizontal_bar_plot(x, y):\n '''\n x: values\n y: names\n '''\n c_purple = \"#A069AB\"\n c_gray = \"#BCC4CE\"\n trace = go.Bar(\n x=x,\n y=y,\n orientation='h'\n marker=dict(\n color=[c_purple]*2 + [c_gray]*8\n ),\n )\n data = [trace]\n layout = go.Layout(\n dict(\n showlegend=False,\n xaxis=dict(\n color=CliqzColors[\"gray_blue\"]\n )\n )\n )\n fig = dict(data=data, layout=layout)\n return div_output(fig)\n```\n\n\n## Tracker Reach - trend Line\nThis chart, as many others, was inspired by Edward Tufte's sparkline [2],\ndrawn without axes or coordinates.\n\n![](\"../static/img/blog/plotting/sparkline.png\")
\nFigure 4: Trend line of tracker reach.
\n\n\n```python\ndef sparkline(ts, t):\n \"\"\"\n Sparkline for plotting line\n Args:\n ts: timeseries data\n t: x-axis (time)\n\n Returns: hmtl output of an interactive timeseries plot\n\n \"\"\"\n y = list(map(lambda x: x * 100, ts)) # scaling percentages\n trace0 = line(\n x=t,\n y=y,\n color=\"#A069AB\" #purple\n )\n trace1 = line(\n x=[t[-1]],\n y=[y[-1]],\n color=\"#A069AB\",\n mode='markers'\n )\n layout = go.Layout(\n dict(\n showlegend=False,\n height=100,\n width=153,\n hoverlabel=dict(\n bgcolor=\"#1A1A25\",\n bordercolor=\"#00000000\", # transparent\n font=dict(\n family=WTMFonts.mono,\n size=13,\n color=\"#BFCBD6\"\n )\n ),\n xaxis=dict(\n autorange=True,\n showgrid=False,\n zeroline=False,\n showline=False,\n autotick=True,\n hoverformat=\"%b %y\",\n ticks='',\n showticklabels=False\n ),\n yaxis=dict(\n # providing some padding for the sparkline\n range=[min(y)*0.90, max(y)*1.05 if max(y) != y[-1] else max(y)*1.15],\n showgrid=False,\n zeroline=False,\n showline=False,\n autotick=True,\n ticks='',\n showticklabels=False\n )\n )\n )\n data = [trace0, trace1]\n fig = dict(data=data, layout=layout)\n return div_output(fig)\n```\nThe code used to plot the sparkline seen in tracker profiles is defined\nin [`plotting/trackers.py`](https://github.com/ghostery/whotracks.me/blob/master/plotting/trackers.py).\n\n\n## Sankey Diagrams\nSankey diagrams are at visualizing flow volume metrics. Sometimes\nthey are found under the name alluvial diagrams, although they originally are\ndifferent types of flow diagrams.\n\n![](\"../static/img/blog/plotting/tracker-map.png\"/)
\nFigure 1: Sankey diagram used to represent a [tracker map](../websites/upornia.com.html)
\n\nIn this site we use sankey diagrams in website profile\npages like [bahn.de](/websites/www.bahn.de.html) to map companies\nand the trackers they operate to the category of the tracker. The thickness\nof the link is a function of the frequency of of appearance of the tracker\nper page load in the given domain. So looking at the diagram above,\nwe know that the dominant tracker category is advertising and Google operates\nthe most trackers and has the highest frequency of appearance.\n\nOur Sankey Diagram function in Python looks like this:\n\n```python\nfrom plotting.utils import div_output\n\ndef sankey_plot(input_data):\n data_trace = dict(\n type='sankey',\n domain=dict(\n x=[0, 1],\n y=[0, 1]\n ),\n hoverinfo=\"none\",\n orientation=\"h\",\n node=dict(\n pad=10,\n thickness=30,\n label=list(map(lambda x: x.replace(\"_\", \" \").capitalize(), input_data['node']['label'])),\n color=input_data['node']['color']\n ),\n link=dict(\n source=input_data['link']['source'],\n target=input_data['link']['target'],\n value=input_data['link']['value'],\n label=input_data['link']['label'],\n color=[\"#dedede\" for _ in range(len(input_data['link']['source']))]\n )\n )\n layout = dict(\n autosize=True,\n font=dict(\n size=12\n )\n )\n fig = dict(data=[data_trace], layout=layout)\n return div_output(fig)\n```\nHaving looked at a lot of examples of sankey plots, we noticed a recurrent\npattern: they do a great job at explaining the plot aesthetics, but\ntake the structure of input data as given. This is a bit of a problem, because\nin most examples the input data is a huge json file, and figuring out the\nstructure of such json file can become tedious.\n\nHere is how `input_data` is structured:\n\n```json\ninput_data = {\n \"node\":{\n \"label\": [],\n \"color\": []\n },\n \"link\": {\n \"source\": [],\n \"target\": [],\n \"value\": [],\n \"label\": [],\n \"color\": []\n }\n}\n```\nAs you notice, input_data has two main parts: node and link:\n\n**NODE**: `input_data[\"node\"]` is responsible for building nodes. In our example these nodes are either\ncategories of trackers or companies that operate them. The atributes of each node are two:\n`label` and `color`. These are both lists of strings. These lists have to have equal length because\nthe mapping of each label to a color is done based on the item's index in the list.\n\n**LINK**: `input_data[\"link\"]` is responsible for linking two nodes together. Each link has\nthe following attributes: `source`, `target`, `value`, `label` and `color`. So here is where the index of\n`input_data[\"node\"][\"label\"]` becomes very important given the way sankey plots have been implemented in\nplotly. The `source` and `target` are lists of equal length, where the index is used to link.\n\n![](\"../static/img/blog/plotting/node_label.png\"/)
\nFigure 5: Node label ilustration
\n\nThe elements in `source` and `target` are in fact the indexes of the source node and target\nnodes in the `input_data[\"node\"][\"label\"]`. So if we were to refer to the illustration in the\nfigure above, to render our sankey diagram we would have:\n\n```python\nsource = [1, 1, 1, ... ]\ntarget = [0, 2, len-2, ... ]\n```\n\nWith that out of the way, the remaining are intuitive: `value` represents how thick the link should be,\n`label` what name it has and `color` its color. All the `link` attributes are lists of equal length, and\nthe matching is done based on index.\n\nFor details, have a look at the actual implementation of the `input_data` generation\nin [`utils/companies.py`](https://github.com/ghostery/whotracks.me/blob/master/utils/companies.py).\n\n\n## References\n[1] [Adding Words to the Brain's Visual Dictionary](http://www.jneurosci.org/content/35/12/4965.short)Figure 1: Distribution of the number of trackers
\n\n\nThere are several differences between our two studies which may explain the increase\nin tracker dominance seen in this study. Firstly, this study’s sample contains the\n500 most popular websites in the US, while our previous study analyzed 144 million\npage loads across more than 12 countries. By only considering the most popular websites\nand neglecting the long tail of more obscure ones, it is not surprising that this study\nsaw a larger proportion of sites with a tracker. Additionally, the data for this study\nwas synthetically generated using a custom crawler, whereas our previous study used data\ngathered from users of the Ghostery browser extension who had opted-in to the collection\nof information about trackers on pages they visit. While the methodologies differ,\nboth studies verify tracker pervasiveness throughout the web.\n\n\n### Trackers and Page Latency\n\nWithout blocking trackers, only 17% of all the pages in the study loaded within\n5 seconds. All other pages loaded much more slowly: it took more than 10 seconds\nto load nearly 60% of the pages, more than 30 seconds for 18% of the pages, and\nnearly 5% of the pages took over a minute to load. This long tail cannot be ignored\nand suggests Internet users waste a lot of time every day simply waiting\nor websites to load.\n\n\nFigure 2: Average time to load trackers
\n\n\nWhile we found that websites are generally slow to load, can any of this page\nlatency be explained by the number of third-party trackers on that site?\nTo answer this question, we calculated the average page load time for\neach tracker count. We excluded both tracker volumes with fewer than\nfive observations and page latency outliers within each tracker\ncount (identified using the interquartile range rule).\n\nTo quantify the relationship between the number of trackers on a website and\nthe average time it took that page to load, we ran a simple linear regression\n(`adj-R2 0.802`) which suggested that each additional tracker adds, on average,\n0.5 seconds to the overall page load. The next model we fitted, which included\na quadratic term (`adj-R2 0.836`), suggests that trackers have an increasing\nimpact on page load times. However, these linear models both exhibit\nheteroscedasticity – uneven variance of the error terms – and thus violate\nlinear regression assumptions.\n\n\nFigure 3: Log Latency as a function of the number of trackers
\n\n\nA Box-Cox test showed that log-transforming the response variable would realize\nthe best fitting model, and also act as a variance-stabilizing transformation.\nThe log-linear model (`adj-R2 0.885`) on the transformed data indicates a\ncompounding effect: if the tracker count increases by 1, we expect the\npage load time to increase by 2.5%.\n\n\n\n### Protection from Trackers\nWe also assessed the difference in page latency when trackers are blocked rather\nthan allowed. The data showed that the average page load time was twice as long\nwhen trackers are not blocked: the mean page latency with no trackers blocked and\nwith all trackers blocked was 19.3 seconds and 8.6 seconds, respectively. These\ntime savings from blocking trackers are even more drastic when only considering the\n10 slowest domains in the sample. We saw that average load times were 10x faster,\nand blocking trackers saved an average of 84 seconds per page load.\n\n\n\nFigure 4: Latencies for certain domains
\n\n\nThe term “piggybacking” describes the practice of one tracker that is placed\ndirectly on a website giving access to other “piggybacking” trackers that are\nnot originally on the site. We observed this phenomenon in our data: page loads\nwere not the only metric significantly reduced when trackers were blocked, there\nwere also fewer trackers detected on the page. We saw significantly more\ntrackers per page when trackers were unblocked compared to blocked, in fact,\namong the domains with the highest average volume of trackers, there were on\naverage 93 fewer trackers present per page load when tracker blocking was enabled.\n\nPiggybacking can create a snowball effect, where trackers bring in more trackers\nthat can then bring in even more trackers; and as suggested above, each additional\ntracker slows down a website more than previous ones. This not only has notable\nperformance implications, but also profound privacy concerns since these trackers\nare not directly on the site, so site owners may not be aware such intrusion\nis occurring.\n\n\n\n## Future Implications\n\nThe data in our study clearly showed the pervasiveness of online tracking,\nas nearly 90% of the most popular sites in the US had at least one third-party\ntracker present. Our study also confirmed the strong, positive link between the\nnumber of trackers on a page and the time it takes that page to load. Generally,\nthe more tracks on a site, the longer the user will have to wait for that site to load.\nQuantifying this relationship depends on the model used, however the optimal model\nwe found shows a compounding effect: for every extra tracker on the page,\nthe time it takes for the page to load increases by 2.5%.\n\nWhile our current study focuses on only the most popular domains in the\nUnited Sates, it would be valuable to apply this framework to other regions\nto see if similar trends persist elsewhere. Additionally, future work may\ninclude measuring additional performance implications of trackers including data\ntransferred. This data transferred, which occurs when trackers make requests to\nother servers, bears real monetary costs to the user, particularly on a mobile\ndevice where data plans are typically based on data used. Expanding this study\nto assess data transfer on mobile could be translated to the out of pocket\nexpense suffered by the user, in addition to the more subjective dollar value\nof the user’s wasted time waiting for pages to load.\n\nOther future work may also include looking at the relationship between\nbounce rates and page load speeds, to calculate a hypothetical tracker\nvalue measure. Given the additional time trackers add to page loads, and\nresearch suggest that slower pages lead to a loss in site traffic, one tracker\nshould provide the same value as this lost site traffic. As bounce rates\nare likely influenced by other factors besides page load speed, like\nfunnel page and domain category, this potential future research involves\nseveral additional considerations.\n\nMoreover, the tracker tax may even have more pronounced implications\nin the United States following the recent repeal of net neutrality.\nIn a time without such net neutrality regulations, users and their browsing\nspeeds may be squeezed from both sides – by the ISP and the online tracking\necosystem. We may then start to see more of a two prong tacker tax: the\ndirect monetary impact imposed by the ISP and the more subjective dollar\nvalue to the user for longer load times, and therefore more unproductive\ntime imposed by trackers.\n\nIn the wake of the net neutrality repeal, now more than ever users must\nconsider the performance implications of browsing online without\nprotection from trackers. The added waiting times incurred by not\nblocking trackers are not trivial, especially as the population is\nspending increasingly more time online. Luckily, various tracker blocking\ntools are available so user can not only protect their privacy, but also\nspeed up their browsing experience by avoiding the tracker tax.\n\n\n\n## References\n[^1]: [Using Passive Measurements to Demystify Online Trackers](https://www.telematica.polito.it/users/mellia/papers/metwalleyComsi.pdf)\n[^2]: [WhoTracks.Me: Monitoring the online tracking landscape at scale](https://arxiv.org/abs/1804.08959)\n[^3]: [Tracking The Trackers](https://pdfs.semanticscholar.org/2bfb/b6b8da453f91f5860ea936588fddef6c80e0.pdf)\n[^4]: [Windows.performance](https://developer.mozilla.org/en-US/docs/Web/API/Window/performance) API\n[^5]: [alexa.com](https://alexa.com)\n[^6]: Ghostery Study: [Tracking the Trackers](https://www.ghostery.com/wp-content/themes/ghostery/images/campaigns/tracker-study/Ghostery_Study_-_Tracking_the_Trackers.pdf)\n" }, { "path": "blog/tracker_categories.md", "content": "title: Tracker Categories\nsubtitle: Definitions for different types of trackers \nauthor: privacy team\ntype: primer\npublish: True\ndate: 2017-07-22\ntags: primer, categories\nheader_img: blog/blog-tracker-categories.jpg\n+++\n\nTrackers differ both in the technologies they use, and the \npurpose they serve. Based on the the service they provide\nto the site owner, we have categorized the trackers in the following: \n\n\nAdvertising\n: Provides advertising or advertising-related services such \n as data collection, behavioral analysis or re-targeting.\n\nComments\n: Enables comments sections for articles and product reviews\n\nCustomer Interaction\n: Includes chat, email messaging, customer support, and other \n interaction tools\n\nEssential\n: Includes tag managers, privacy notices, and technologies \n that are critical to the functionality of a website\n\nPornvertising\n: Delivers advertisements that generally appear on sites with adult content\n\nSite Analytics\n: Collects and analyzes data related to site usage and \n performance.\n\nSocial Media\n: Integrates features related to social media sites\n\nAudio Video Player\n: Enables websites to publish, distribute, and optimize \n video and audio content\n\nCDN (Content Delivery Network)\n: Content delivery network that delivers resources for \n different site utilities and usually for many different customers.\n\nMisc (Miscellaneous)\n: This tracker does not fit in other categories.\n\nHosting\n: This is a service used by the content provider or site owner\n\nUnknown\n: This tracker has either not been labelled yet, or we do not have\n enough information to label it.\n" }, { "path": "blog/trackers-who-steal.md", "content": "title: The Trackers Who Steal\nsubtitle: How WhoTracks.Me caught the trail of the MageCart hackers\nauthor: privacy team\ntype: article\npublish: True\ndate: 2018-11-23\ntags: tracking, hacking\nheader_img: blog/blog-cc-stealing.png\n+++\n\nWe're all aware of the trackers siphoning off information about you as you browse the web. These trackers are mostly doing this for some business intelligence related reason - websites use these services to try to 'better understand' their customers, or to target them in order to attract their attention in a way which will benefit that website owner - be-it increasing the value of products customers put into their shopping cart, or increasing the likelihood that they click an ad.\n\nHowever, there is another kind of tracker which is more nefarious than these. These are hidden scripts placed by hackers on E-commerce sites which try to steal your credit-card details as you enter them. In the last year a string of attacks — dubbed 'Magecart' — have affected major sites, including [British Airways](https://www.riskiq.com/blog/labs/magecart-british-airways-breach/), [Ticketmaster](https://www.riskiq.com/blog/labs/magecart-ticketmaster-breach/), [NewEgg](https://www.riskiq.com/blog/labs/magecart-newegg/) and [VisionDirect](https://twitter.com/troyhunt/status/1064069833967337472); stealing payment information from millions of consumers.\n\nAt WhoTracks.Me we are monitoring the third-parties loaded on millions of pages per day, therefore once we know the domains that these hackers are using to send their stolen data, we can analyse the extent and impact of these operations. In this article we provide a post-analysis of the four big breaches this year, plus some insights our data gives in on-going attacks.\n\n## Four high-profile breaches\n\n### British Airways\n\nIn September 2018, British Airways announced that a security breach had led to a large theft of customer data. [RiskIQ's](https://www.riskiq.com/blog/labs/magecart-british-airways-breach/) write up of the breach explains how the attackers compromised a script on the payment page, such that it would send credit card information to a domain owned by the hackers: `baways.com`.\n\nWith this information we can query our data to look for page loads where `baways.com` was a third-party. This allows us to verify the extent of the breach, and how many users were affected. Our data shows that:\n\n- `www.britishairways.com` was affected between August 22nd and September 5th. 193 pages in our data were affected[^1].\n- We also see two page-loads on `hotline.ba.com` on the 30th August where data was sent to the attackers.\n\nThis data corroborates the [statement by BA](https://www.britishairways.com/en-gb/information/incident/data-theft/latest-information) on the breach, that users entering card details \"between 22:58 BST August 21 2018 until 21:45 BST September 5 2018\" would have been affected.\n\n### Ticketmaster\n\nIn June 2018, Ticketmaster declared a hack of customer information. Again, [RiskIQ's](https://www.riskiq.com/blog/labs/magecart-ticketmaster-breach/) analysis tells us how it happened - this time involving a breach of the third-party supplier Inbenta. Compromised Inbenta scripts were then loaded on ticketmaster payment pages, and these scripts then skimmed credit card data input by customers and sent it to `webfotce.me`.\n\nUnlike the British Airways case, this was not a targeted attack on Ticketmaster, rather a generic hacking program which affected many other sites. We can access the extent of these hacks by looking for the `webfotce.me` domain in our data:\n\n- Ticketmaster's UK, Irish and New Zealand sites were first affected on February 10th. The malicious script remained in place until the June 23rd, and we saw over 2,500 page loads making requests to the hackers during this time.\n- Their German and Australian sites appear to have been affected earlier, with our first observations on December 10th 2017 for ticketmaster.de and December 20th for ticketmaster.com.au. These sites were fixed at the same time as the others.\n\nIn their [disclosure](https://security.ticketmaster.co.uk/), Ticketmaster say that UK customers were affected between February and June 23rd, and international customers could have been affected from September 2017. Again this matches up with our data, though we have no observations for international sites before December 2017.\n\nOur data also shows several other sites affected by this attack:\n\n| Site | \nAffected from | \nAffected to | \nPages | \n
|---|---|---|---|
| otel.com | \n10/12/2017 | \n21/06/2018 | \n125 | \n
| www.cheaperthandirt.com | \n19/01/2018 | \n03/06/2018 | \n42 | \n
| www.printninja.com | \n16/02/2018 | \n20/11/2018 | \n45 | \n
| www.vitacost.com | \n26/02/2018 | \n04/06/2018 | \n35 | \n
| thehungryjpeg.com | \n12/03/2018 | \n12/06/2018 | \n19 | \n
| www.klook.com | \n14/03/2018 | \n12/06/2018 | \n28 | \n
| www.steinmart.com | \n15/03/2018 | \n09/07/2018 | \n12 | \n
| www.marveloptics.com | \n28/03/2018 | \n22/09/2018 | \n15 | \n
Table 1: Sites affected by webfotce.me attack.
\n\nCompared to Ticketmaster, the impact of the breach on these sites is much smaller. Correlations between the dates of infection indicate that these sites were probably infected via a shared third-party (i.e. Inbenta) which was compromised. This shows how hackers can quickly achieve much greater scale by going for third-party services whose scripts will be loaded on many different sites.\n\n### NewEgg\n\nLike British Airways, NewEgg were hit by a [targetted attack](https://www.riskiq.com/blog/labs/magecart-newegg/). In this case the collection server was specific for the target. The hackers registered `neweggstats.com` in order to have a legitimate looking domain so that they could avoid suspicion for as long as possible.\n\nLooking at our data, we see that pages on `secure.newegg.com` were sending requests to `neweggstats.com` for just over a month, between 15th August and 18th September, with 90 pages affected in our dataset[^1].\n\n### VisionDirect\n\nOn the November 19th, VisionDirect, a large UK-based glasses retailer, [announced](https://www.visiondirect.co.uk/customer-data-theft) that their sites had been compromised between the November 3rd and 8th. In this case, a script was injected into the page from `g-analytics.com` which pretending to be a Google Analytics script. The difference is, that it will also send credit-card numbers back when it sees them in the page.\n\nOur analysis shows that VisionDirect's European sites (`.fr`, `.it`, `.es`, `.co.uk`, `.eu` and `.nl`) were all affected from the November 3rd. On the `.nl` and `.ie` sites we still observed pages contacting the attacker's server on the 9th of November, suggesting that the malicious code may not have been completely removed as early as the press release suggests.\n\nCompared to the other collection servers, `g-analytics.com` is currently much more active with 36 sites infected during November. We have, however, observed a shift in traffic since November 20th, with almost all sites which were previously infected with `g-analytics.com`\nswitching to loading a script from `google-analytics.is` instead. This indicates that the\nattacks have ongoing access to these sites, allowing them to update their attack code.\n\n| Collection Server | \nSites Infected | \n
|---|---|
| g-analytics.com | \n36 | \n
| googletagmanager.eu | \n29 | \n
| magento.name | \n19 | \n
| google-analytics.is | \n15 | \n
| trafficanalyzer.biz | \n5 | \n
| web-stats.cc | \n5 | \n
| bandagesplus.com | \n5 | \n
| nearart.com | \n4 | \n
Table 2: Collection servers still active in November 2018
\n\nA full list of sites affected during November is available at the end of this post.\n\n## Breach detection\n\nWhile WhoTracks.Me was originally conceived as a transparency tool to show trackers directly or indirectly placed by site owners, this investigation as opened up another angle on this data. We can now effectively track the spread of malicious code being used to defraud web consumers. This capability can be taken to multiple different directions:\n\n1. Once the collection servers (or drop servers) are known, we can quickly find and notify websites that are compromised. This can reduce the exposure time of websites, and thus reduce the risk to the average web user. (Thanks to the work RiskIQ have done here to collate a list of active drop servers).\n2. We can audit breaches that have occurred, and make sure websites properly notify their users. The GDPR requires that companies notify users and authorities when user data is compromised. This data can be used to hold companies accountable if they try to dodge these responsibilities.\n3. Given the set of collection servers we already know, we can develop algorithms to automatically detect third-parties in pages which are similar. This would then allow us to detect and block these servers even earlier.\n\nWe are very exited to start exploring this direction for our data[^2].\n\n## Third-party scripts: A security liability\n\nIn all of these hacking cases, the entry point has been a malicious script which is loaded in the main document of the page. When this happens on a payment page, the attacker can read all of the information entered: credit card number, CVV, etc. Therefore, any script loaded on a payment page is potentially a critical security weakness.\n\nWith this in mind, we should be critical of the current careless way that scripts of scattered onto what should be secure webpages. In the case of the four big breaches we have outlined here, now standard browser security features could have prevented or limited the amount of data stolen:\n\n- [Subresource Integrity](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity) on script tags would prevent surreptitious changes to first- and third- party scripts (provided the site's own webserver is not also compromised). If a script were to be changed to add the attacker's code the browser will refuse to load it. In the British Airways case one JavaScript file was edited to add the attack payload; for Ticketmaster the attack payload came via a third-party script. This technique provides some protection when loading content for less-trusted origins in pages which require high security.\n- The [Content Security Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy) (CSP) header can be used to prevent requests to unknown origins. In all of these cases the credit card information was sent to a third-party collection server. A CSP header would have prevented this request, thus preventing the malicious script from exfiltrating data.\n\nAs well as these high-profile cases, many of the other sites affected by these attacks are smaller E-Commerce sites using off-the-shelf software to run their business. It is therefore difficult for these sites to deploy these more advanced protection methods - even more so because the loading of 20 or 30 different untrusted third-parties on a webpage has become normalised, so users or even developers would not be able to detect unexpected third-parties appearing on a page.\n\nRelated to this is another tactic employed by these hacking groups: registering domains very similar to common third-party trackers so that developers do not notice that the site is compromised. Some examples:\n\n- `g-analytics.com`, pretending to be [Google Analytics](../trackers/google_analytics.html);\n- `googletagmanager.eu` -> [Google Tag Manager](../trackers/google_tag_manager.html);\n- `slripe.com` -> [Stripe](../trackers/stripe.com.html);\n- `typeklt.com` -> [Adobe Typekit](../trackers/typekit_by_adobe.html);\n- `crtteo.com` -> [Criteo](../trackers/criteo.html);\n- `jsdellvr.com` -> [JSDelivr](../trackers/jsdelivr.html).\n\n## Protecting users\n\nAs we can assume that sites will continue to get hacked, we require a way we can protect users from having their data stolen without relying on site owners. This is where the browser comes in - as the user-agent it should be able to protect the user from attacks like this, much like it already does with phishing and malware sites.\n\nLuckily, as all of these attacks rely on collection servers to receive the stolen data, once we know of a server address we can use blocklists to prevent the browser from contacting these servers. Therefore, even when sites are compromised with malicious Javascript, this code will not be able to contact the hacker's server. For Cliqz and Ghostery users we have already distributed a block-list to block these domains and protect them from credit-card theft.\n\n > For Cliqz and Ghostery users we have already distributed a block-list to block these domains and protect them from credit-card theft.\n\nBlocking is just a reactive measure though. Domains are cheap, and sites are getting hacked all the time, so these hackers could easily turn over their domains faster to mitigate our blocking. Therefore, a more robust solution has to incorporate fast detection of these drop servers in order to minimise the effective lifetime of each attack. We hope to incorporate the WhoTracks.Me data in the hunt for these domains, and to emulate the speed that we are already [able to detect phishing sites](https://cliqz.com/en/whycliqz/anti-phishing).\n\n## Conclusion\n\nIn this post we've shown a new angle on the data we publish on WhoTracks.Me. As well as\nproviding transparency on which companies are tracking you online, we are also able to\nturn this transparency on web criminals who are stealing from web users. This transparency\ncan be used to:\n\n * ensure that breaches are reported promptly when they occur;\n * assess the impact of breaches, in terms of the timespan when sites were affected and how many users may have been affected; and\n * develop new techniques to catch these operations faster and reduce the number of users who\n suffer from them.\n\n [^1]: By \"Pages Affected\" we mean the number of page loads where we saw some third-party call to a server associated with MageCart operations.\n [^2]: Reach out to privacy@cliqz.com if you have suggestions, or would simply like to get in touch.\n\n### Appendix: List of Magecart affected sites during November 2018\n\n| Collection Server | \nSite | \nInfected from | \ninfected to | \nNumber of pages | \n
|---|---|---|---|---|
| google-analytics.is | \nwww.groworganic.com | \n2018-11-22 | \n2018-11-28 | \n64 | \n
| googletagmanager.eu | \nwww.wdrshop.de | \n2018-11-03 | \n2018-11-28 | \n74 | \n
| google-analytics.is | \nwww.directmaterial.com | \n2018-11-28 | \n2018-11-28 | \n1 | \n
| google-analytics.is | \nwww.harriscomm.com | \n2018-11-23 | \n2018-11-28 | \n9 | \n
| google-analytics.is | \nwww.prospin.com.br | \n2018-11-24 | \n2018-11-28 | \n5 | \n
| google-analytics.is | \nwww.electroactiva.com | \n2018-11-28 | \n2018-11-28 | \n1 | \n
| magento.name | \nwww.gamesquest.co.uk | \n2018-11-13 | \n2018-11-28 | \n21 | \n
| google-analytics.is | \nwww.drakegeneralstore.ca | \n2018-11-23 | \n2018-11-28 | \n23 | \n
| google-analytics.is | \nshop.tokidoki.it | \n2018-11-26 | \n2018-11-28 | \n3 | \n
| magento.name | \nstore.curiousinventor.com | \n2018-11-01 | \n2018-11-28 | \n13 | \n
| webfotce.me | \nwww.printninja.com | \n2018-11-05 | \n2018-11-28 | \n7 | \n
| vuserjs.com | \nwww.medelita.com | \n2018-11-01 | \n2018-11-27 | \n63 | \n
| googletagmanager.eu | \nwww.aneros.com | \n2018-11-01 | \n2018-11-27 | \n49 | \n
| magento.name | \nwww.arrazofashion.com.br | \n2018-11-27 | \n2018-11-27 | \n1 | \n
| fastproxycdn.com | \nlessthan10pounds.com | \n2018-11-09 | \n2018-11-27 | \n19 | \n
| magento.name | \nchebdveri.ru | \n2018-11-02 | \n2018-11-27 | \n2 | \n
| googletagmanager.eu | \nwww.onegreekstore.com | \n2018-11-02 | \n2018-11-27 | \n8 | \n
| vmaxjs.com | \nwww.artistsnetwork.com | \n2018-11-01 | \n2018-11-27 | \n105 | \n
| googletagmanager.eu | \nslf24.pl | \n2018-11-08 | \n2018-11-26 | \n26 | \n
| google-analytics.is | \nwww.pvcfittingsonline.com | \n2018-11-22 | \n2018-11-26 | \n42 | \n
| googletagmanager.eu | \nwww.bestkiteboarding.com | \n2018-11-09 | \n2018-11-26 | \n24 | \n
| qsxjs.com | \nvapenw.com | \n2018-11-02 | \n2018-11-26 | \n86 | \n
| magento.name | \nwww.compremake.com.br | \n2018-11-20 | \n2018-11-26 | \n3 | \n
| valdatecode.com | \nwww.carnivalbkk.com | \n2018-11-03 | \n2018-11-26 | \n52 | \n
| googletagmanager.eu | \nwww.mobileparadise.de | \n2018-11-21 | \n2018-11-26 | \n4 | \n
| googletagmanager.eu | \nwww.cht-cottbus.de | \n2018-11-03 | \n2018-11-26 | \n79 | \n
| privatejs.com | \nwww.bydubai.com | \n2018-11-01 | \n2018-11-26 | \n55 | \n
| google-analytics.is | \nwww.scojo.com | \n2018-11-24 | \n2018-11-26 | \n10 | \n
| googletagmanager.eu | \nwww.nordhandel.de | \n2018-11-01 | \n2018-11-25 | \n106 | \n
| alfcdn.com | \nwww.softstarshoes.com | \n2018-11-25 | \n2018-11-25 | \n2 | \n
| magento.name | \nwww.prestigioplaza.com | \n2018-11-25 | \n2018-11-25 | \n3 | \n
| googletagmanager.eu | \nwww.wslstore.com | \n2018-11-18 | \n2018-11-25 | \n2 | \n
| magento.name | \nwww.herve-leger-shop.com | \n2018-11-25 | \n2018-11-25 | \n1 | \n
| googletagmanager.eu | \namsducati.com | \n2018-11-05 | \n2018-11-25 | \n6 | \n
| google-analytics.is | \nwww.ozarksource.com | \n2018-11-24 | \n2018-11-24 | \n1 | \n
| g-analytics.com | \ngeissele.com | \n2018-11-10 | \n2018-11-24 | \n85 | \n
| crtteo.com | \nwww.accessorygeeks.com | \n2018-11-01 | \n2018-11-23 | \n31 | \n
| google-analytics.is | \ndrdennisgross.com | \n2018-11-22 | \n2018-11-23 | \n4 | \n
| googletagmanager.eu | \nwww.everbestshoes.com | \n2018-11-23 | \n2018-11-23 | \n1 | \n
| googletagmanager.eu | \nunitedsalonsupplies.com | \n2018-11-23 | \n2018-11-23 | \n2 | \n
| trafficanalyzer.biz | \nwww.oaknyc.com | \n2018-11-19 | \n2018-11-23 | \n2 | \n
| googletagmanager.eu | \ndampoteket.no | \n2018-11-07 | \n2018-11-23 | \n10 | \n
| magento.name | \nwww.ikonmotorsports.com | \n2018-11-05 | \n2018-11-23 | \n8 | \n
| google-analytics.is | \nwww.dreamduffel.com | \n2018-11-23 | \n2018-11-23 | \n4 | \n
| nearart.com | \nwww.westcottbrand.com | \n2018-11-04 | \n2018-11-23 | \n13 | \n
| magento.name | \noramaoptics.gr | \n2018-11-21 | \n2018-11-23 | \n3 | \n
| google-analytics.is | \nwww.cruyffclassics.com | \n2018-11-23 | \n2018-11-23 | \n3 | \n
| magento.name | \nwww.weldingsuppliesdirect.co.uk | \n2018-11-05 | \n2018-11-22 | \n9 | \n
| googletagmanager.eu | \nhk.ap-nutrition.com | \n2018-11-13 | \n2018-11-22 | \n3 | \n
| nearart.com | \nwww.camillusknives.com | \n2018-11-03 | \n2018-11-22 | \n27 | \n
| google-analytics.is | \nwww.softballfans.com | \n2018-11-22 | \n2018-11-22 | \n3 | \n
| magento.name | \nwww.ammerer.com | \n2018-11-02 | \n2018-11-22 | \n4 | \n
| g-analytics.com | \nwww.candent.ca | \n2018-11-22 | \n2018-11-22 | \n1 | \n
| googletagmanager.eu | \nwww.autosiliconehoses.com | \n2018-11-03 | \n2018-11-21 | \n29 | \n
| google-analytics.is | \ntemptu.com | \n2018-11-21 | \n2018-11-21 | \n4 | \n
| googletagmanager.eu | \nwww.lampen-line.de | \n2018-11-04 | \n2018-11-21 | \n16 | \n
| googletagmanager.eu | \nwww.airagestore.com | \n2018-11-05 | \n2018-11-21 | \n6 | \n
| g-analytics.com | \ndrdennisgross.com | \n2018-11-11 | \n2018-11-20 | \n8 | \n
| g-analytics.com | \npvcpipesupplies.com | \n2018-11-12 | \n2018-11-20 | \n5 | \n
| g-analytics.com | \nwww.cruyffclassics.com | \n2018-11-08 | \n2018-11-20 | \n13 | \n
| g-analytics.com | \nwww.pvcfittingsonline.com | \n2018-11-08 | \n2018-11-20 | \n77 | \n
| g-analytics.com | \nwww.ahmadtea.com | \n2018-11-09 | \n2018-11-20 | \n10 | \n
| g-analytics.com | \nwww.groworganic.com | \n2018-11-04 | \n2018-11-20 | \n87 | \n
| web-stats.cc | \nwww.kingfishertapes.co.uk | \n2018-11-20 | \n2018-11-20 | \n3 | \n
| g-analytics.com | \nwww.fabglassandmirror.com | \n2018-11-10 | \n2018-11-20 | \n9 | \n
| statsdot.eu | \nwww.punkstuff.com | \n2018-11-20 | \n2018-11-20 | \n14 | \n
| onefromeu.com | \nwww.joyfolie.com | \n2018-11-03 | \n2018-11-20 | \n16 | \n
| listrakb.com | \nwww.skistart.com | \n2018-11-02 | \n2018-11-19 | \n4 | \n
| g-analytics.com | \nwww.energymuse.com | \n2018-11-06 | \n2018-11-19 | \n72 | \n
| googletagmanager.eu | \nwww.casinhabonita.com.br | \n2018-11-06 | \n2018-11-19 | \n20 | \n
| g-analytics.com | \nwww.frightprops.com | \n2018-11-15 | \n2018-11-19 | \n3 | \n
| statsdot.eu | \nstoreinfinity.com | \n2018-11-07 | \n2018-11-19 | \n10 | \n
| g-analytics.com | \nwww.especialneeds.com | \n2018-11-12 | \n2018-11-19 | \n21 | \n
| g-analytics.com | \nwww.stmgoods.com.au | \n2018-11-09 | \n2018-11-18 | \n7 | \n
| onefromeu.com | \nwww.poshshop.com | \n2018-11-13 | \n2018-11-18 | \n39 | \n
| googletagmanager.eu | \ndeanzelinsky.com | \n2018-11-07 | \n2018-11-18 | \n11 | \n
| googletagmanager.eu | \nnativetreasuresnm.com | \n2018-11-10 | \n2018-11-18 | \n8 | \n
| g-analytics.com | \nvapage.com | \n2018-11-13 | \n2018-11-18 | \n23 | \n
| magento.name | \nwww.hydraulicsonline.co.uk | \n2018-11-02 | \n2018-11-18 | \n2 | \n
| nearart.com | \nmitchellssalon.com | \n2018-11-18 | \n2018-11-18 | \n1 | \n
| g-analytics.com | \naltheatsupply.com | \n2018-11-14 | \n2018-11-18 | \n5 | \n
| scriptsfyou.com | \nadamspolishes.com | \n2018-11-01 | \n2018-11-17 | \n55 | \n
| googletagmanager.eu | \nwww.recifeingressos.com | \n2018-11-16 | \n2018-11-17 | \n3 | \n
| g-analytics.com | \nwww.stmgoods.com | \n2018-11-10 | \n2018-11-16 | \n12 | \n
| g-analytics.com | \ntemptu.com | \n2018-11-06 | \n2018-11-16 | \n7 | \n
| g-analytics.com | \nwww.drakegeneralstore.ca | \n2018-11-16 | \n2018-11-16 | \n1 | \n
| g-analytics.com | \nshop.tokidoki.it | \n2018-11-15 | \n2018-11-15 | \n3 | \n
| g-analytics.com | \nmedmartonline.com | \n2018-11-13 | \n2018-11-15 | \n4 | \n
| g-analytics.com | \nintl.drdennisgross.com | \n2018-11-15 | \n2018-11-15 | \n2 | \n
| googletagmanager.eu | \nikiegeszitok.hu | \n2018-11-08 | \n2018-11-15 | \n11 | \n
| g-analytics.com | \nwww.weareverincontinence.com | \n2018-11-12 | \n2018-11-14 | \n3 | \n
| cdnscriptx.com | \nwww.cartouchesarabais.com | \n2018-11-11 | \n2018-11-14 | \n14 | \n
| g-analytics.com | \ncig2o.com | \n2018-11-14 | \n2018-11-14 | \n1 | \n
| fastproxycdn.com | \ntilebar.com | \n2018-11-03 | \n2018-11-14 | \n120 | \n
| g-analytics.com | \nwww.curediva.com | \n2018-11-07 | \n2018-11-13 | \n6 | \n
| typeklt.com | \nwww.mariatash.com | \n2018-11-02 | \n2018-11-13 | \n49 | \n
| g-analytics.com | \nwww.lucerooliveoil.com | \n2018-11-13 | \n2018-11-13 | \n5 | \n
| g-analytics.com | \nwww.plumbingsupplynow.com | \n2018-11-13 | \n2018-11-13 | \n1 | \n
| magento.name | \nwww.grafipronto.pt | \n2018-11-12 | \n2018-11-12 | \n1 | \n
| checkercarts.com | \nwww.shambhala.com | \n2018-11-01 | \n2018-11-12 | \n19 | \n
| scriptsenvoir.com | \nwww.heatpressnation.com | \n2018-11-01 | \n2018-11-12 | \n48 | \n
| typeklt.com | \nwww.cabletiesunlimited.com | \n2018-11-09 | \n2018-11-12 | \n6 | \n
| web-stats.cc | \nwww.costway.de | \n2018-11-07 | \n2018-11-10 | \n2 | \n
| g-analytics.com | \nwww.visiondirect.ie | \n2018-11-05 | \n2018-11-09 | \n4 | \n
| web-stats.cc | \nwww.rincondidactico.cl | \n2018-11-09 | \n2018-11-09 | \n1 | \n
| g-analytics.com | \nwww.visiondirect.nl | \n2018-11-04 | \n2018-11-09 | \n41 | \n
| magento.name | \npatbo.com.br | \n2018-11-05 | \n2018-11-09 | \n3 | \n
| googletagmanager.eu | \nprofessional.imageskincare.nl | \n2018-11-09 | \n2018-11-09 | \n2 | \n
| googletagmanager.eu | \nconsument.imageskincare.nl | \n2018-11-09 | \n2018-11-09 | \n2 | \n
| magento.name | \neaccesoriigsm.ro | \n2018-11-08 | \n2018-11-08 | \n1 | \n
| jspoi.com | \nwww.padini.com | \n2018-11-04 | \n2018-11-08 | \n3 | \n
| g-analytics.com | \nwww.visiondirect.co.uk | \n2018-11-03 | \n2018-11-08 | \n112 | \n
| googletagmanager.eu | \nwww.oddbins.com | \n2018-11-01 | \n2018-11-08 | \n9 | \n
| g-analytics.com | \nwww.visiondirect.fr | \n2018-11-03 | \n2018-11-07 | \n53 | \n
| magento.name | \nupmarketpets.com | \n2018-11-07 | \n2018-11-07 | \n1 | \n
| g-analytics.com | \nwww.visiondirect.it | \n2018-11-04 | \n2018-11-07 | \n2 | \n
| g-analytics.com | \nwww.visiondirect.es | \n2018-11-05 | \n2018-11-07 | \n26 | \n
| upgradenstore.com | \nwww.armysurplusworld.com | \n2018-11-06 | \n2018-11-06 | \n1 | \n
| g-analytics.com | \nwww.ozarksource.com | \n2018-11-06 | \n2018-11-06 | \n1 | \n
| upgradenstore.com | \nwww.princesspolly.com | \n2018-11-01 | \n2018-11-06 | \n3 | \n
| locatefyou.com | \nwww.jjroofingsupplies.co.uk | \n2018-11-01 | \n2018-11-06 | \n10 | \n
| g-analytics.com | \nwww.prospin.com.br | \n2018-11-06 | \n2018-11-06 | \n1 | \n
| web-stats.cc | \nwww.baleyo.com | \n2018-11-06 | \n2018-11-06 | \n1 | \n
| maxijs.com | \ncopperlab.com | \n2018-11-05 | \n2018-11-05 | \n9 | \n
| gamacdn.com | \ncsvape.com | \n2018-11-03 | \n2018-11-05 | \n2 | \n
| valdatecode.com | \nwww.pfiwestern.com | \n2018-11-01 | \n2018-11-05 | \n15 | \n
| googletagmanager.eu | \nerecycleronline.com | \n2018-11-05 | \n2018-11-05 | \n1 | \n
| magento.name | \nnicoman.co.uk | \n2018-11-01 | \n2018-11-05 | \n2 | \n
| minifyscripts.com | \nshop.bombingscience.com | \n2018-11-03 | \n2018-11-04 | \n4 | \n
| web-stats.cc | \nshelfadditions.com | \n2018-11-04 | \n2018-11-04 | \n2 | \n
| jspoi.com | \nstore.asqgrp.com | \n2018-11-01 | \n2018-11-04 | \n3 | \n
| trafficanalyzer.biz | \nwww.irishnewsarchive.com | \n2018-11-03 | \n2018-11-03 | \n1 | \n
| magento.name | \nwww.cochesdemetal.es | \n2018-11-01 | \n2018-11-03 | \n2 | \n
| magento.name | \noriginalnye-zapchasti.com | \n2018-11-02 | \n2018-11-02 | \n1 | \n
| googletagmanager.eu | \nwww.exeltek.com.au | \n2018-11-02 | \n2018-11-02 | \n2 | \n
| g-analytics.com | \nwww.hyperparapharmacie.com | \n2018-11-02 | \n2018-11-02 | \n1 | \n
| amasty.biz | \nwww.decantshop.com | \n2018-11-01 | \n2018-11-01 | \n1 | \n
| jspoi.com | \nmassivejoes.com | \n2018-11-01 | \n2018-11-01 | \n4 | \n
| cdnrfv.com | \nwww.versare.com | \n2018-11-01 | \n2018-11-01 | \n18 | \n
| magento.name | \nwww.yourdezire.co.uk | \n2018-11-01 | \n2018-11-01 | \n2 | \n
| allacarts.com | \nwww.plumprettysugar.com | \n2018-11-01 | \n2018-11-01 | \n6 | \n
![\"Trackers](\"../static/img/blog/trackers_on_site/tumblr.png\")
\nFigure 1: Sankey diagram used to represent a [tracker map](../websites/tumblr.com.html)
\n\nSankey diagrams are great at visualizing flow volume metrics. Sometimes\nthey are found under the name alluvial diagrams, although they originally are\ndifferent types of flow diagrams [1].\n\nWe wanted to use the sankey diagram supported in plotly [2], the visualisation\nlibrary of choice used in whotracks.me. The function itself is pretty simple,\nas you will see in a bit when we define `sankey_diagram()`. The challenge to creating\nsankey diagrams with Plotly is understanding the required structure of the input data\nrequired by the plotting function. Hopefully the following example will\nmake it easier for those reading this post, should they ever decide to try\nsankey diagrams.\n\nThe goal here is to show a very small dataset, structured in a way that the\nplotly diagram (and other plotting solutions e.g.: d3.js) understand.\nWe will be mapping cities to the countries they are part of. The value of each link,\nwill be the city population (in millions).\n\n```python\ncity_data = dict(\n nodes = dict(\n label=[\"Germany\", \"Berlin\", \"Munich\", \"Cologne\", \"France\", \"Paris\", \"Lyon\", \"Bordeaux\"],\n color=[\"beige\", \"black\", \"red\", \"yellow\", \"beige\", \"blue\", \"white\", \"red\"]\n ),\n links = dict(\n source=[0, 0, 0, 4, 4, 4],\n target=[1, 2, 3, 5, 6, 7],\n value= [3.5, 1.5, 1, 2.2, 0.5, 0.2],\n label=[\"capital\", \"city\", \"city\", \"capital\", \"city\", \"city\"],\n color=[\"black\", \"red\", \"yellow\", \"blue\", \"whitesmoke\", \"red\"]\n )\n )\n```\n\nNote how there are two keys in the dictionary, `nodes` and `links`, and each has some\nattributes. Let's go over them. Each node has a label (e.g. Germany) and a corresponding\n`color` (in this case `beige`). Note that labels and colors are stored in lists of\nequal length, and the pairing is done based on equality of that index.\n\nLinks contain information about how to link nodes. Each has a `source`, `target`, `value`,\n`label` and `color`. Source contains the index in the list of the source node,\nwhereas target the index in the list of the target node.\nValue determines how thick the link should be (in our case it will be\nthe population of each link, hence each city), Label and color, as the\nname suggests, specify the label and color of the link. Links too, are\npaired based on index.\n\n\n## Plotting a sankey diagram\n\nNow let's write a simple function to plot this data nicely. Most of the work has\nalready been done, given we're feeding the data in a format that's easy to parse.\n\n```python\nfrom plotly.offline import iplot\n\ndef sankey_diagram(sndata, title):\n # First part of a plotly plot is the `trace`\n data_trace = dict(\n type='sankey',\n node=dict(\n pad=10,\n thickness=30,\n # label could easily be equal to sndatap['node]['label']. The following is just cosmetics\n label=list(map(lambda x: x.replace(\"_\", \" \").capitalize(), sndata['nodes']['label'])),\n color=sndata['nodes']['color']\n ),\n link=sndata[\"links\"],\n\n # configuration options for the diagram\n domain=dict(\n x=[0, 1],\n y=[0, 1]\n ),\n hoverinfo=\"none\",\n orientation=\"h\"\n )\n # Second part of a plotly plot is the `layout`\n layout = dict(\n title=title,\n font=dict(\n size=12\n )\n )\n fig = dict(data=[data_trace], layout=layout)\n return iplot(fig)\n```\n\n## Sankey diagram for a few German and French cities\nAll that is left now, is feeding the `city_data` to the `sankey_diagram` function\nand we're done.\n\n![\"Simple](\"../static/img/blog/trackers_on_site/cities.png\")
\nFigure 1: Simple example of a sankey digram for cities
\n\nTrying to create the flags of these countries did not end up being\nsuch an aesthetically good idea.\n\n# From Cities to Trackers\nDoing Sankey diagrams for cities may have been fun. The result of doing the same for\ntrackers on your favourite sites might not be as fun -it may in fact be terrifying.\nWe'll be using public data from whotracks.me to map tracker categories to companies\npresent on a particular site. Each link will be a tracker the company owns.\nThis gives immediate visual insights on who's watching you and why.\n\n## Terse intro to the API\nThe data and API for whotracksme us available on Pypi and you can easily install it\nrunning `pip install whotracksme`.\n\n\n```python\nfrom whotracksme.data.loader import DataSource\nfrom whotracksme.website.plotting.colors import tracker_categoryColors, cliqz_colors\n```\n\nDataSource is a class that provides access to trackers, companies that own them, and\npopular websites. The functionality of DataSource is something we'll be constantly\ntrying to improve and expand. Online tracking is messy enough to analyze,\nso at least the tooling should be as simple as possible.\n\nWe will be looking at Reddit. If you are not familiar with Reddit,\ncheck it out - there are some great communities there. Now we'll look at the tracking\nlandscape in reddit. To do that, we only need to know the reddit `site_id`,\nwhich is `reddit.com`. Each site has a `site_id`, most often its url.\n\n\n## Preparing reddit tracker data for sankey diagram\nHere we will be mapping companies and the trackers they operate to the category\nof the tracker. The thickness of the link is a function of the frequency of\nappearance of the tracker per page load in the given domain.\n\n```python\ndef sankey_data(site_id, data_source):\n\n nodes = []\n link_source = []\n link_target = []\n link_value = []\n link_label = []\n\n for (tracker, category, company) in data_source.sites.trackers_on_site(site_id, data_source.trackers, data_source.companies):\n\n # index of this category in nodes\n if category in nodes:\n cat_idx = nodes.index(category)\n else:\n nodes.append(category)\n cat_idx = len(nodes) - 1\n\n # index of this company in nodes\n if company in nodes:\n com_idx = nodes.index(company)\n else:\n nodes.append(company)\n com_idx = len(nodes) - 1\n\n link_source.append(cat_idx)\n link_target.append(com_idx)\n link_label.append(tracker[\"name\"])\n link_value.append(100.0 * tracker[\"frequency\"])\n\n label_colors = [tracker_categoryColors[l] if l in tracker_category_colors else cliqz_colors[\"purple\"] for l in nodes]\n\n return dict(\n nodes = dict(\n label=nodes,\n color=label_colors\n ),\n links = dict(\n source=link_source,\n target=link_target,\n value=link_value,\n label=link_label,\n color=[\"#dedede\"] * len(link_label)\n )\n )\n```\nNow that we have a function to generate the data in the format we need it, let's\nrun it for reddit and plot the sankey diagram to investigate the tracking landscape:\n\n```python\ninput_data = sankey_data('reddit.com', data_source=DataSource())\nsankey_diagram(input_data, 'Tracker Map on reddit.com')\n```\n\n![\"Reddit](\"../static/img/blog/trackers_on_site/reddit.png\")
\nFigure 1: Tracking landscape on reddit.com
\n\nWe see that most tracking happens for advertising reasons. Although it does not seem like it,\nReddit is keeping the set of advertisers they expose their users somwhat limited compared to\nother portals and news sites. In terms of number of trackers, Google has the most eyes on\nreddit users. For more details on the tracking landscape on reddit, head over to\nreddit's [profile page](https://whotracks.me/websites/reddit.com.html) on whotracks.me.\n\n\n## References\n[[1] Sankey Diagrams](https://en.wikipedia.org/wiki/Sankey_diagram) - Wikipedia![](\"../static/img/blog/tracking_and_ux/banner.png\"/)
\nFigure 1: banner ads are intrusive and distracting. In this example they are placed right next to the content to get the users’ attention.
\n\n### Deception\nAnother reason for annoyance is the use of native \nadvertising: ads that are designed to resemble content \nas much as possible (Figure 2). The main goal is to \nmaximize click-through rates on ads by deliberately \nmisleading users on the nature of the content. Although users are \nless likely to notice the presence of native advertising \ncompared to traditional banner ads [9], it is more \ndifficult for them to distinguish organic content from paid ads. \n\n\n![](\"../static/img/blog/tracking_and_ux/native2.png\"/)
\nFigure 2: native advertising is deceptive as it makes it difficult for users to distinguish organic content from paid ads.
\n\n\n### Page Breakage\nLast but not least, ads and tracking increase page loading \ntimes: users have to wait substantially longer for \ncontent to appear, which degrades the online user \nexperience. The average data usage by trackers \namounts to more than 6MB per page load [2]. In a Mozilla study, \nresearchers further found that the average \nnumber of reported problems with web pages was higher for \nusers with tracker blocking disabled, relative to those \nwith it enabled.[3]. These users reported \nmore often that web pages felt slow, laggy, or \nunresponsive. This is surprising because tracking \nprotection is often the reason for such page breakage.\n\n### Moments of embarrassment\nThe facets discussed so far all relate to functional \nproblems (i.e. web pages do not work, users cannot \ncomplete their tasks, etc.). However, there is also \nanother dimension to the visual effects of tracking: \nsocial implications. \n\nIn order to deliver the most relevant ads to the user, \nthey are often targeted and based on previous online \nbehaviors, such as page visits. For example, a \nuser would see ads for sports shoes on a news page \nafter having searched for them on a shopping \nsite. While the majority of users are opposed to \nbehavioral targeting and are concerned about \ntheir privacy [4, 10], behavioral targeting can \naffect the user experience in a much more direct \nway, in particular when sharing a computer: Imagine \nan online purchase for your loved one popping up \non a web page visited by the future gift recipient — surprise ruined.\nOr imagine a friend looking over your shoulder and getting \na glimpse on an ad about something that you find embarrassing. \n\n\n\n## The Hidden Effects of Tracking\nYet, a large part of tracking takes place behind the scenes of \nthe shiny web surface. It's not obvious that users are \nbeing observed, yet trackers record all their page visits [6]. \nThis is not only a privacy problem, it also affects \nthe user experience.\n\n### Lack of Transparency\nMost users are aware that their searches and interactions \nare recorded when using services like Facebook or Amazon. \nAfter all, they are explicitly registered and logged in. \nUsers understand that such services need to know \ncertain things to provide their services, for example, \nto show interesting posts or to suggest new friends.\n\nHowever, a large part of tracking takes place via \nthird-party trackers, scripts that are embedded \non pages around the web or are part of a browser \nadd-on without the users' awareness. These scripts \ncall home to report on each user's behavior—often \nwithout having asked for permission. It is not \ntransparent to users that their oftentimes personal \ndata is shared, with whom it is shared, and where \nit is stored. For example, users were surprised \nto learn that browsing history is used to target ads [10].\n\n\n### Lack of Control\nEven if users knew about the extent of tracking taking place, \nthere is still a lack of control. Once the data is out \non some servers, users do not have the option to audit \nor delete the data stored about them. Current approaches \nfor giving control to users are not understood by users [10].\n\n### Transparency and Control are Critical\nWhy are transparency and control so important? Data collected \nby trackers reveal more about a person than you might \nthink. One page visit may not tell who you are, but \nthe visit of multiple pages does. Trackers connect \nthese visits through unique identifiers. Suddenly the \nvirtual self turns into a real person: Profile pictures \nfrom social networks reveal the visual appearance, \nlocation sharing exposes home and workplace, and \nshopping behavior hint at personal preferences. \nAll this happens without the awareness of the \nuser—the user experience on the surface does not \nreveal the operating network of trackers underneath \nit.\n\n\n## The UX Challenges\nBoth visible and hidden effects of trackers on user \nexperience are non-trivial to address. Numerous applications \nor add-ons exist to remove ads from web pages or to \nreduce the effects of tracking. Adblock Plus and NoScript \nare two popular examples.\n\n\n![](\"../static/img/blog/tracking_and_ux/adwall2.png\"/)
\nFigure 3: an example of an ad block wall encouraging users to whitelist trackers.
\n\n\nHowever, removing ads leads, similar to ads in the first place, \nto page breakage. News sites, for example, use adblock \ndetection to put up ad block walls, asking users to \nwhitelist ads in order to access the content (Figure 3). \nAd block walls not only degrade the user experience but \nalso reduce traffic to the underlying pages: a recent \nsurvey found that 74% of American adblock users choose \nto leave sites with adblock walls [1]. Another example \nare pages without visible ads, but that use scripts \nfor tracking. Blocking all scripts offers protection, \nbut makes modern web pages unusable as many features \nrely on scripting.\n\nOverall, tracking is a complex topic. Its technical \nfoundation is hard to grasp for most users. Users \nbuild their mental models about how trackers work \nbased on their own experiences. This leads to wrong \nbeliefs, such as that Facebook cannot track users once \nthey are logged out of the platform. On the \nother hand, advertising is often the only revenue \nstream for web sites. Users benefit from it as \nwebsites can run without charging their users. \n\nNonetheless, revenue should never come at the cost of the users’ \nprivacy. It is not easy, but targeted advertising does not \nhave to rely on tracking [5]. Users should always be in \ncontrol over their data. This is the paradigm that Cliqz \nfollows in their products [11]. The challenges, from a user \nexperience point of view, lie in educating users in a simple \nenough way so that they understand the effects of tracking \nand in allowing users to decide which data they want to \nshare or not to share. We love to hear your thoughts on this topic.\n\n\n\n##References\n\n[1] [2017 Adblock Report](https://pagefair.com/blog/2017/adblockreport/)Figure 1: Page loads per country, March 2018
\n\nThis volume of data will also enable us to publish WhoTracks.Me content for individual\ncountries, something we plan to add later this month.\n\n\n## Data restructuring\n\nWe have updated the structure of data which we publish in our [repository](https://github.com/ghostery/whotracks.me/) to make it both easier to use and more\nscalable as we add more data. We now publish CSV files each month for each of the\nfollowing:\n\n * `domains.csv`: Top third-party domains seen tracking.\n * `trackers.csv`: Top trackers - this combines domains known be operated by the same tracker.\n * `companies.csv`: Top companies - aggregates the stats for trackers owned by the same company.\n * `sites.csv`: Stats for number of trackers seen on popular websites.\n * `site_trackers.csv`: Stats for each tracker on each site.\n\nThese files can then be loaded with popular data-analysis tools such as\n[Pandas](https://pandas.pydata.org/). We have also rewritten the code to render the site\nto take advantage of Pandas. We expose the dataframes via the `DataSource` class which\nloads data from all CSV files:\n\n```python\nfrom whotracksme.data.loader import DataSource\ndata = DataSource()\nlen(data.trackers.df)\n>> 7928\n```\n\nWe have also updated the criteria by which we include trackers and sites on the main site.\nWe now 'rollover' entries, so once they have been included once, we will keep publishing\ndata (until they completely disappear from the data). This has the effect of naturally\ngrowing the number of trackers and sites we publish. We currently have data on\n868 trackers and 748 websites published:\n\n```python\npd.DataFrame({\n 'trackers': data.trackers.df.groupby('month').count()['tracker'],\n 'sites': data.sites.df.groupby('month').count()['site']\n}).plot()\n```\n\n\nFigure 2: Growth of trackers and sites
\n\n\n\nThe per-site trend for average number of trackers continues a slightly downward trend,\nalthough the average is still high at 9 trackers per page. There are several possible\nreasons for this, it is not necessarily that sites are using fewer trackers! The\nproportion of data from Ghostery users continues to increase, and these users will\ndisproportionately block many trackers. This has an effect on the average number of\ntrackers, because it prevents the blocked trackers from loading others. The data shows\nalso that the average incidence of blocking for trackers increased to 25% in March, up\nfrom 20% in February.\n\n```python\nsns.boxplot(\n data=data.sites.df[data.sites.df.month >= '2018-01'],\n x='month',\n y='trackers'\n)\n```\n\n\nFigure 3: Average trackers per page since January
\n\n\n```python\n(data.trackers.df[data.trackers.df.month >= '2018-01']\n .groupby('month')\n ['has_blocking'].mean() * 100).plot()\n```\n\n\nFigure 4: Blocking Trend since January
\n\n\nAs in previous months, we look at sites' changing their trackers. [fewo-direct.de](../websites/fewo-direkt.de.html), [brigitte.de](../websites/brigitte.de.html) and [gutefrage.net](../websites/gutefrage.net.html) all had 5 fewer trackers on average per page this month. However, each of these still has over 50 trackers with some kind of presence, showing that this is more likely a side-effect of increased blocking than an active effort to reduce tracking on their sites. [klingel.de](../websites/klingel.de.html) and [informationvine.com](../websites/informationvine.com.html) see the largest increase in tracking of the sites we currently monitor.\n\n| Site | \nTrackers | \nChange since February | \n
|---|---|---|
| informationvine.com | \n18.3 | \n+6.4 | \n
| klingel.de | \n26.7 | \n+5.3 | \n
| gutefrage.net | \n13.0 | \n-5.6 | \n
| brigitte.de | \n19.5 | \n-5.8 | \n
| fewo-direkt.de | \n16.0 | \n-6.6 | \n
Table 1: Websites Tracking Trends
\n\nA side-effect of the filtering we added in this new data pipeline is that the site reach\nfor top trackers has increased. In the previous analysis a long-tail of very rarely\nvisited sites reduced effective site reach. With this factor reduced, we get a real\nsense of the coverage of the largest trackers, with Google Analytics reaching 85% of\npopular sites, and Facebook almost 60%. The data can easily be retrieved\nas shown below:\n\n```python\ndf = data.trackers.get_snapshot().sort_values(by='site_reach', ascending=False).head(10)\ndf['name'] = df.id.apply(func=lambda x: data.app_info[x]['name'])\n```\n\nFigure 5: Reach of top 10 trackers across popular websites
\n\n\nIf you want to delve deeper into our data, it is available on the [WhoTracks.Me Github Repository](https://github.com/ghostery/whotracks.me/tree/master/whotracksme/data), and as a [pip package](https://pypi.python.org/pypi/whotracksme/).\n\n_NB: The code snippets here will not generate the presented plots. Full code snippets\nfor the plots in this post are available in this [Jupyter Notebook](https://nbviewer.jupyter.org/github/ghostery/whotracks.me/blob/master/contrib/wtm_april_update.ipynb)._" }, { "path": "blog/update_dec_2017.md", "content": "title: WhoTracks.me December Update\nsubtitle: New data and trackers in our monthly update.\nauthor: privacy team\ntype: article\npublish: True\ndate: 2017-12-08\ntags: blog, update\nheader_img: blog/blog-data-dec17.png\n+++\n\nWe're happy to update the site today with data from November 2017 - based on data from 100 million page loads. We're also expanding the amount of data we show, up to 600 top websites and 600 top trackers.\n\n## New Trackers in the database\n\nIncreasing the number of trackers displayed meant that we needed to add tracker information for a new batch of tracker domains, as well as new entrants appearing in the top 500. Here are the 3 most interesting entrants:\n\n * [Tru Optik](../trackers/truoptik.html), a company offering targeted advertising for Smart TVs, and claiming 70 Million US households in their 'Household Graph'. Their presence across major German sites suggests they might be using online ad networks in order to harvest user information and link it to active Smart TVs, where they can then push targeted adverts.\n * [Digitrust](../trackers/digitrust.html), a non-profit aiming to reducing the number of third-party requests per page. Their solution, however, is to create a unified user identifier, intended to prevent the need for trackers to synchronise pixels and tracking tokens on each page. Notably, they state that they [do not support](http://www.digitru.st/faqs/) the [Do Not Track](https://en.wikipedia.org/wiki/Do_Not_Track) standard, so their claims to be working in consumers interests are, at best, suspect.\n * [ORC International](../trackers/orc_international.html), the registered owner of the domain `emxdgt.com`, and a subsiduary of [Engine](http://www.enginegroup.com/), an Advertising Agency. Despite only appearing in our data recently, they have quickly risen up to the top 300 trackers, and are listed in the [ads.txt](https://iabtechlab.com/ads-txt/) files as a reseller for several major US publications, such as [The Atlantic](https://www.theatlantic.com/ads.txt) and [CNET](https://www.cnet.com/ads.txt). Their ownership, and policy for the data collection is, however, not transparently disclosed.\n\n## Month-to-month trends\n\nThe average number of trackers on top websites increased to 10, an increase of 3%. [Heine.de](../websites/heine.de.html), [gutefrage.net](../websites/gutefrage.net.html), [sportscheck.com](../websites/sportscheck.com.html) and [bild.de](../websites/bild.de.html) increased their number of trackers the most, each of them added on average 5 more trackers page page load. At the other end of the spectrum, [paket.de](../websites/paket.de.html), [jackpot.de](../websites/jackpot.de.htmwl) and [hurriyet.com.tr](../websites/hurriyet.com.tr.html) had on average 5 fewer trackers per page.\n\nOn the tracker side, the biggest gain was by [pmddby.com](../trackers/pmddby.com.html), which increased its reach by 9 times since October. Its profile is that of Spyware which is injecting ads into webpages for affected users, however at this time we were not able to determine the source - the WHOIS data for the domain is private.\n\n## Additions to the dataset\n\nThis month we added two new signals to the data which attempt to show the effect of ad-blockers on the trackers in our database. These signals are:\n\n * `has_blocking` - the proportion of pages on which this tracker was affected by some kind of blocking.\n * `requests_failed` - the average number of failed requests per page load (for comparison with `requests` to get an idea of how aggressive the blocking is).\n\nThese signals should be able to tell us something about the impact of blocking on different trackers in the ecosystem. For example, we see evidence of blocking 40% of the time for Google Analytics and Facebook, and between 10% and 20% of requests failing. Thus, anyone using these services to measure activity and conversions on their sites must reckon with error rates in these orders.\n\nWe also can see how new entrants can initially avoid the effects of blocking - for [Tru Optik](../trackers/truoptik.html) and [Digitrust](../trackers/digitrust.html) who we mentioned earlier, we measure only 5 and 1% of pages which may be affected by blocking.\n\nThese stats are currently only available in the raw data, but we will be looking at incorporating them in the site in due course.\n" }, { "path": "blog/update_feb_2018.md", "content": "title: February Update - The Tracking Shell Game\nsubtitle: How mergers and acquisitions are hiding who actually is tracking us.\nauthor: privacy team\ntype: article\npublish: True\ndate: 2018-02-06\ntags: blog, update\nheader_img: blog/blog-data-feb18-2.png\n+++\n\n_This post is one of our regular monthly blogs accompanying an update to the data displayed on WhoTracks.Me. In these posts we introduce what data has been added as well as point out interesting trends and case-studies we found in the last month. Previous month's posts can be found here: [January 2018](./update_jan_2018.html), [December 2017](./update_dec_2017.html)._\n\nWe've updated the site today with data collected during January 2018. Due to increased distribution, we have over 115 million page loads this month, an increase of 15% over previous months (see [Where does the data come from?](./where_is_the_data_from.html) for more background on our data collection). The regions from which we are getting data is also diversifying. While 70% of the data still comes from German users, we now have more significant US and international data. We plan to have sufficient data in the coming months in order to provide region-specific tracking breakdowns.\n\n## The tracking shell game\n\nPicking out some of the biggest movers in the rankings this month, we first find [Nexage](../trackers/nexage.html) down 262 places this month, and to one tenth of its reach in May last year. This is probably simply a winding down of the operation which was acquired by Millennial Media in 2014, who were acquired by AOL in 2015, who were acquired by Verizon also in 2015. Their landing page now redirects to [One by AOL](https://www.onebyaol.com/).\n\nOne of the challenges for us on whotracks.me is to make the link between tracker domain names, tracking products, and tracking companies. Nexage is an example of how many trackers lead you down a 'rabbit hole' of mergers and acquisitions until you find the company above it all. If we expand out the web of companies underneath Verizon who are also present on whotracks.me, we find 10 different trackers which can be linked: [Adap.tv](../trackers/adap.tv.html), [ADTECH](../trackers/adtech.html), [Advertising.com](../trackers/advertising.com.html), [alephD](../trackers/alephd.com.html), [Convertro](../trackers/convertro.html), [Nexage](../trackers/nexage.html) and [Vidible](../trackers/vidible.html) under AOL, and [Brightroll](../trackers/brightroll.html) and [Flickr](../trackers/flickr_badge.html) under [Yahoo](../trackers/yahoo.html). Furthermore, Yahoo and AOL both have popular web portals ([yahoo.com](../websites/yahoo.com.html), [aol.com](../websites/aol.com.html) and [aol.de](../websites/aol.de.html)) to drive more traffic which they can track. This leads Verizon to be able to track at least 6% of web traffic, the 11th highest reach of any company in our dataset. You can now check the full list of companies sorted by their trackers' combined reach [here](../companies/reach-chart.html).\n\n![\"Verizon's](\"../static/img/blog/update_feb18/verizon.png\")
\nVerizon's trackers - © WhoTracks.Me 2018
\n\nA new entry at 546, [Smarter Travel Media](../trackers/smarter_travel.html) is another example of a smaller company with giants hiding behind it. The tracker is primarly present on tripadvisor and other travel sites, and infact they are a [Tripadvisor](../trackers/tripadvisor.html) company. Tripadvisor in turn is owned by [Expedia](../trackers/expedia.html). Above all of this stands [InterActiveCorp (IAC)](http://iac.com/) who own several other web brands, including [Vimeo](../trackers/vimeo.html) and [Mindspark](../trackers/mindspark.html).\n\n![\"Verizon's](\"../static/img/blog/update_feb18/iac.png\")
\nIAC - © WhoTracks.Me 2018
\n\nThe final movers we would like to highlight this month are [davebestdeals.com](../trackers/davebestdeals.com.html) and [eshopcomp.com](../trackers/eshopcomp.com.html), up 201 and 194 places respectively. Unfortunately we cannot yet trace the owners of these trackers---they are both registered with PrivacyGuard in [Panama](https://who.is/whois/eshopcomp.com) and have no visible landing page. In fact they are likely operated by the same entity as their domains point to the same CloudFront endpoints, for example on the `istatic` subdomain for both domains.\n\nThe reason for the lack of transparency in this case is that they are malware. Looking at their profile pages we can see that they have a small presence across many sites, including sites which we know for certain would not have trackers like this in the page (e.g. Google sites, which will only ever contain Google's own trackers). These are likely browser extensions which include code to inject their tracking code in all of the pages the user visits, and send this information back to their servers. The user browsing history which they collect can then by sold on. The Web Of Trust extension was [caught doing this](https://www.forbes.com/sites/leemathews/2016/11/07/web-of-trust-browser-add-on-blasted-for-breaking-user-trust/#5029a0a53ef5) in 2015, and our data shows that it is still a common practice (look for the 'Extensions' tracker category on this site).\n\nThis style of user history harvesting also has the advantage that it is not blocked by the majority of Ad-blocking and privacy tools. These domains are not on these blocklists, because the list maintainers will not encounter them - unless they happen to install the malware themselves. Therefore, currently only Cliqz and Ghostery 8's AI anti-tracking are detecting these trackers and preventing them from gathering user sessions - because they are using the same data to find trackers that whotracks.me uses.\n\n## New data points\n\nThis month we add data about the content-types loaded by trackers. This is based on values reported by the [webRequest 'type' property](https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/ResourceType). By reporting these values we can further characterise tracker behaviours, and quantify risks, such as which trackers are being permitted to load scripts on certain pages.\n\nWe add the following new columns for trackers, reported as the proportion of pages where the specific tracker or company loaded particular resource type(s) into the page:\n\n * `script`: Javascript code (via a `\"\n ],\n \"text/vnd.plotly.v1+html\": [\n \"\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n }\n ],\n \"source\": [\n \"# Plotting imports\\n\",\n \"from plotly.offline import init_notebook_mode, iplot\\n\",\n \"init_notebook_mode()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Generating Sankey Data\\n\",\n \"When building the tracker maps that you see on popular site profiles on whotracks.me, sankey diagrams seemed like a good fit to map categories of tracking to companies that own the trackers. Each link would be a tracker, going from a category to a company. \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Given we had decided to use plottly.offline to generate the interactive images, I wanted to use the sankey diagram supported in plotly. The fuction itself is pretty straightforward, as you can see in `sankey_diagram()`, but figuring out how the structure of the input data took a bit. Hopefully the following example will make it easier for those reading this post, should they ever decided to try sankey diagrams.\\n\",\n \"\\n\",\n \"The goal here is to show some very small dataset, structured in a way that the plotly diagram (and other plotting solutions e.g.: d3.js) understand. We will be mapping cities to the countries they are part of. The value of each link, will be the city population (in millions).\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"city_data = dict(\\n\",\n \" nodes = dict(\\n\",\n \" label=[\\\"Germany\\\", \\\"Berlin\\\", \\\"Munich\\\", \\\"Cologne\\\", \\\"France\\\", \\\"Paris\\\", \\\"Lyon\\\", \\\"Bordeaux\\\"],\\n\",\n \" color=[\\\"beige\\\", \\\"black\\\", \\\"red\\\", \\\"yellow\\\", \\\"beige\\\", \\\"blue\\\", \\\"white\\\", \\\"red\\\"]\\n\",\n \" ),\\n\",\n \" links = dict(\\n\",\n \" source=[0, 0, 0, 4, 4, 4],\\n\",\n \" target=[1, 2, 3, 5, 6, 7],\\n\",\n \" value= [3.5, 1.5, 1, 2.2, 0.5, 0.2],\\n\",\n \" label=[\\\"capital\\\", \\\"city\\\", \\\"city\\\", \\\"capital\\\", \\\"city\\\", \\\"city\\\"],\\n\",\n \" color=[\\\"black\\\", \\\"red\\\", \\\"yellow\\\", \\\"blue\\\", \\\"whitesmoke\\\", \\\"red\\\"]\\n\",\n \" )\\n\",\n \" )\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Note how there are two keys in the `dictionary`, `nodes` and `links`, and each has some attributes. Let's go over them. Each node has a label (e.g. `Germany`) and a corresponding color (in this case `beige`). Note than `labels` and `colors` are stored in lists of equal length, and the pairing is done based on the index. \\n\",\n \"\\n\",\n \"Links contain information about how to link nodes. Eeach has a `source`, `target`, `value`, `label` and `color`. Source cointains the index in the list of the source node, whereas target the index in the list of the target node. Value determines how thick the link should be (in our case it will be the population of each link, hence each city), Label and color, as the name suggests, specify the label and color of the link. Links too, are paired based on index.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Plotting a sankey diagram\\n\",\n \"\\n\",\n \"Now let's write a simple function to plot these data nicely. Most of the work has already been done, given we're feeding the data in a format that's easy to parse.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def sankey_diagram(sndata, title):\\n\",\n \" # First part of a plotly plot is the `trace`\\n\",\n \" data_trace = dict(\\n\",\n \" type='sankey',\\n\",\n \" node=dict(\\n\",\n \" pad=10,\\n\",\n \" thickness=30,\\n\",\n \" # label could easily be equal to sndatap['node]['label']. The following is just cosmetics\\n\",\n \" label=list(map(lambda x: x.replace(\\\"_\\\", \\\" \\\").capitalize(), sndata['nodes']['label'])),\\n\",\n \" color=sndata['nodes']['color']\\n\",\n \" ),\\n\",\n \" link=sndata[\\\"links\\\"],\\n\",\n \" \\n\",\n \" # configuration options for the diagram\\n\",\n \" domain=dict(\\n\",\n \" x=[0, 1],\\n\",\n \" y=[0, 1]\\n\",\n \" ),\\n\",\n \" hoverinfo=\\\"none\\\",\\n\",\n \" orientation=\\\"h\\\"\\n\",\n \" )\\n\",\n \" # Second part of a plotly plot is the `layout`\\n\",\n \" layout = dict(\\n\",\n \" title=title,\\n\",\n \" font=dict(\\n\",\n \" size=12\\n\",\n \" )\\n\",\n \" )\\n\",\n \" fig = dict(data=[data_trace], layout=layout)\\n\",\n \" return iplot(fig)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Sankey diagram for a few German and French citites\\n\",\n \"All that is left now, is feeding the city_data to the sankey_diagram function and we're done.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"application/vnd.plotly.v1+json\": {\n \"data\": [\n {\n \"domain\": {\n \"x\": [\n 0,\n 1\n ],\n \"y\": [\n 0,\n 1\n ]\n },\n \"hoverinfo\": \"none\",\n \"link\": {\n \"color\": [\n \"black\",\n \"red\",\n \"yellow\",\n \"blue\",\n \"whitesmoke\",\n \"red\"\n ],\n \"label\": [\n \"capital\",\n \"city\",\n \"city\",\n \"capital\",\n \"city\",\n \"city\"\n ],\n \"source\": [\n 0,\n 0,\n 0,\n 4,\n 4,\n 4\n ],\n \"target\": [\n 1,\n 2,\n 3,\n 5,\n 6,\n 7\n ],\n \"value\": [\n 3.5,\n 1.5,\n 1,\n 2.2,\n 0.5,\n 0.2\n ]\n },\n \"node\": {\n \"color\": [\n \"beige\",\n \"black\",\n \"red\",\n \"yellow\",\n \"beige\",\n \"blue\",\n \"white\",\n \"red\"\n ],\n \"label\": [\n \"Germany\",\n \"Berlin\",\n \"Munich\",\n \"Cologne\",\n \"France\",\n \"Paris\",\n \"Lyon\",\n \"Bordeaux\"\n ],\n \"pad\": 10,\n \"thickness\": 30\n },\n \"orientation\": \"h\",\n \"type\": \"sankey\"\n }\n ],\n \"layout\": {\n \"font\": {\n \"size\": 12\n },\n \"title\": \"A few European Cities\"\n }\n },\n \"text/html\": [\n \"\"\n ],\n \"text/vnd.plotly.v1+html\": [\n \"\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n }\n ],\n \"source\": [\n \"sankey_diagram(city_data, \\\"A few European Cities\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# From Cities to Trackers\\n\",\n \"\\n\",\n \"Doing Sankey diagrams for cities may have been fun. I am not sure the result of doing the same for trackers on your favorite sites will be equally fun. In fact it may be terrifying. We'll be using public data from whotracks.me to map tracker categories to Companies present on a particular site. Each link will be a tracker the company owns. This gives imediate visual insights on who's watching you an why. \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Data Utils from `whotracksme`\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from whotracksme.data.loader import DataSource\\n\",\n \"from whotracksme.website.plotting.colors import tracker_category_colors, cliqz_colors\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Loading Data\\n\",\n \"`DataSource` is a class that provides access to trackers, websites and companies. The functionality of `DataSource` is something we'll be constantly trying to improve and expand. Online tracking is messy enough to analyze, so the tooling should be not.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"DATA = DataSource()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Available entities\\n\",\n \"\\n\",\n \"These entities are loaded into DataSource, but an API is provided for some common operations on each of them. For more details, have a look at `whotracksme.data.loader`. As far as we're concerned, we can load them like this:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"trackers = DATA.trackers\\n\",\n \"sites = DATA.sites\\n\",\n \"companies = DATA.companies\\n\",\n \"\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Looking at reddit.com\\n\",\n \"Most people know what reddit is. For you that don't, check it out - there are some great communities there. Now we'll look at the tracking landscape in reddit. To do that, we only need to know the reddit `site_id`, which is `reddit.com`. Each site has a `site_id`, most often its `url`. \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 8,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"dict_keys(['apps', 'category', 'history', 'name', 'overview', 'subdomains'])\"\n ]\n },\n \"execution_count\": 8,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"reddit_id = \\\"reddit.com\\\"\\n\",\n \"reddit_data = DATA.sites.get_site(reddit_id)\\n\",\n \"\\n\",\n \"# reddit_data is a dictionary. And a site object has the following keys: \\n\",\n \"reddit_data.keys()\\n\",\n \"\\n\",\n \"# apps refers to trackers. Naming is hard, but it'll soon be changed to trackers.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Preparing tracker data for sankey diagram\\n\",\n \"Here we will be mapping the trackers on reddit to the category they belong to (on the left) and to the companies that own them (on the right). This means each link is a tracker, nodes on the left are categories, and nodes on the right are companies. \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 9,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def sankey_data(site_id, data=DATA):\\n\",\n \"\\n\",\n \" nodes = []\\n\",\n \" link_source = []\\n\",\n \" link_target = []\\n\",\n \" link_value = []\\n\",\n \" link_label = []\\n\",\n \"\\n\",\n \" for (tracker, category, company) in data.sites.trackers_on_site(site_id, data.trackers, data.companies):\\n\",\n \"\\n\",\n \" # index of this category in nodes\\n\",\n \" if category in nodes:\\n\",\n \" cat_idx = nodes.index(category)\\n\",\n \" else:\\n\",\n \" nodes.append(category)\\n\",\n \" cat_idx = len(nodes) - 1 \\n\",\n \" \\n\",\n \" # index of this company in nodes\\n\",\n \" if company in nodes:\\n\",\n \" com_idx = nodes.index(company)\\n\",\n \" else:\\n\",\n \" nodes.append(company)\\n\",\n \" com_idx = len(nodes) - 1 \\n\",\n \" \\n\",\n \" link_source.append(cat_idx)\\n\",\n \" link_target.append(com_idx)\\n\",\n \" link_label.append(tracker[\\\"name\\\"])\\n\",\n \" link_value.append(100.0 * tracker[\\\"frequency\\\"])\\n\",\n \"\\n\",\n \" label_colors = [tracker_category_colors[l] if l in tracker_category_colors else cliqz_colors[\\\"purple\\\"] for l in nodes]\\n\",\n \"\\n\",\n \" return dict(\\n\",\n \" nodes = dict(\\n\",\n \" label=nodes,\\n\",\n \" color=label_colors\\n\",\n \" ),\\n\",\n \" links = dict(\\n\",\n \" source=link_source,\\n\",\n \" target=link_target,\\n\",\n \" value=link_value,\\n\",\n \" label=link_label,\\n\",\n \" color=[\\\"#dedede\\\"] * len(link_label)\\n\",\n \" )\\n\",\n \" )\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 10,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"application/vnd.plotly.v1+json\": {\n \"data\": [\n {\n \"domain\": {\n \"x\": [\n 0,\n 1\n ],\n \"y\": [\n 0,\n 1\n ]\n },\n \"hoverinfo\": \"none\",\n \"link\": {\n \"color\": [\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\",\n \"#dedede\"\n ],\n \"label\": [\n \"Reddit\",\n \"Google Tag Manager\",\n \"Google Analytics\",\n \"Amazon Associates\",\n \"Quantcast\",\n \"ScoreCard Research Beacon\",\n \"Google\",\n \"DoubleClick\",\n \"Quantcount\",\n \"Google AdServices\",\n \"Moat\",\n \"Google Syndication\",\n \"Google APIs\",\n \"OpenX\",\n \"Imgur\",\n \"Google CDN\",\n \"YouTube\",\n \"Alexa Metrics\",\n \"WikiMedia\",\n \"Amazon Web Services\",\n \"AppNexus\",\n \"InsightExpress\",\n \"Advertising.com\"\n ],\n \"source\": [\n 0,\n 2,\n 4,\n 5,\n 5,\n 5,\n 5,\n 5,\n 4,\n 5,\n 5,\n 5,\n 10,\n 5,\n 12,\n 10,\n 14,\n 4,\n 10,\n 17,\n 5,\n 4,\n 5\n ],\n \"target\": [\n 1,\n 3,\n 3,\n 6,\n 7,\n 8,\n 3,\n 3,\n 7,\n 3,\n 9,\n 3,\n 3,\n 11,\n 13,\n 3,\n 3,\n 15,\n 16,\n 6,\n 18,\n 19,\n 20\n ],\n \"value\": [\n 99.58731289613593,\n 99.0402000255409,\n 98.2706125110061,\n 91.97209321082666,\n 46.30429960814889,\n 45.09245132106922,\n 44.410240554909564,\n 43.818095052459654,\n 37.652657261343855,\n 32.872476996390674,\n 20.00053770306692,\n 16.99612181662981,\n 16.054469320679388,\n 8.754478058354225,\n 5.226473810499996,\n 4.598705479866381,\n 2.9412357760735577,\n 2.19920554371862,\n 1.9437965869297826,\n 1.2777169127778412,\n 1.2004220969075352,\n 1.1789139742305805,\n 1.1110289620314422\n ]\n },\n \"node\": {\n \"color\": [\n \"#87BCEF\",\n \"#A069AB\",\n \"#FC9834\",\n \"#A069AB\",\n \"#84D7F0\",\n \"#BF90D2\",\n \"#A069AB\",\n \"#A069AB\",\n \"#A069AB\",\n \"#A069AB\",\n \"#C0BB61\",\n \"#A069AB\",\n \"#80C87D\",\n \"#A069AB\",\n \"#F86D4F\",\n \"#A069AB\",\n \"#A069AB\",\n \"#444\",\n \"#A069AB\",\n \"#A069AB\",\n \"#A069AB\"\n ],\n \"label\": [\n \"Social media\",\n \"Reddit\",\n \"Essential\",\n \"Google\",\n \"Site analytics\",\n \"Advertising\",\n \"Amazon\",\n \"Quantcast\",\n \"Comscore\",\n \"Oracle\",\n \"Cdn\",\n \"Openx\",\n \"Misc\",\n \"Imgur\",\n \"Audio video player\",\n \"Alexa\",\n \"Wikimedia\",\n \"Hosting\",\n \"Appnexus\",\n \"Millward brown\",\n \"Aol\"\n ],\n \"pad\": 10,\n \"thickness\": 30\n },\n \"orientation\": \"h\",\n \"type\": \"sankey\"\n }\n ],\n \"layout\": {\n \"font\": {\n \"size\": 12\n },\n \"title\": \"reddit.com\"\n }\n },\n \"text/html\": [\n \"\"\n ],\n \"text/vnd.plotly.v1+html\": [\n \"\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n }\n ],\n \"source\": [\n \"input_data = sankey_data(reddit_id, data=DATA)\\n\",\n \"sankey_diagram(input_data, reddit_id)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Don't forget to check out the article on https://whotracks.me/blog/trackers_in_your_favorite_site.html\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.6.2\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n" }, { "path": "contrib/wtm_april_update.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Whotracks.me April Update\\n\",\n \"\\n\",\n \"This month we have a big update to the site. We have restructured the data we publish to make it easier to use, increased the number of entries we publish, and we have laid the groundwork for internationalised versions of WhoTracks.Me - that means you can see how tracking differs between different countries.\\n\",\n \"\\n\",\n \"Thanks to integration with Ghostery 8 we collected significantly more tracker data this month, covering 360 million page loads. This is spread over countries across the world, with Germany and the USA the most represented.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"\"\n ],\n \"text/vnd.plotly.v1+html\": [\n \"![\"powered](\"https://img.shields.io/badge/ghostery-powered-blue?logo=ghostery&style=flat-square\"){kind=icon}
\n ![\"License](\"https://img.shields.io/github/license/ghostery/whotracks.me?style=flat-square\")