Tracking and Data Collection

Dated Jan 22, 2019; last modified on Sun, 20 Jun 2021

Tracking on the Internet

Screenshot from a 2012 study of third-party online tracking. The red boxes show content (at least the visible ones) served by third parties.

Screenshot from a 2012 study of third-party online tracking. The red boxes show content (at least the visible ones) served by third parties.

When the browser requests a third-party resource embedded on a web page, the snippet below is a subset of the HTTP headers sent by the browser to the third party.

1
2
3
GET         http://youtube.com/watch?v=gHStnhGx1P
Cookie:     id=35c192bcfe0000b1...
Referer:    http://www.nytimes.com/

The combination of the cookie and the referrer makes third-party tracking possible. Incidentally, the HTTP protocol specification misspells ‘referrer’

The Industry

Display Advertising Technology Landscape. Source: LUMA Partners LLC

Display Advertising Technology Landscape. Source: LUMA Partners LLC

Third party online tracking: sites other than the one you’re visiting (typically invisible) compiling profiles of your browsing history.

Behavioral targeting: profiles based on user’s past activity help ad exchanges serve targeted ads based on real-time ad auctions.

There’s still debate on whether the cost of behavioral targeting is worth it. Sometimes the markup over untargeted ads is 500% for a 4% improvement on ROI. Research is limited because behavioral ad giants, e.g. Google and Facebook, are reluctant to open their data for analysis.

Web Tracking Methods

Placing data in the browser

Options: HTTP cookies, HTTP auth, HTTP etags, content cache, IE userdata, HTML5 protocol & content handlers, HTML5 storage, Flash cookies, Silverlight storage, TLS session ID & resume, Browsing history, window.name, HTTP STS, DNS cache, …

Numerous web APIs allow placing data in the browser (directly or indirectly) and all of these can be used for uniquely identifying the user and hence tracking their browsing.

Fingerprinting

Involves observing the browser’s behavior, e.g. user-agent, browser plugins, clock skew, list of installed fonts, cookies enabled?, browser add-ons, screen resolution, …

When these attributes are combined, different devices/browsers will have different fingerprints. Fingerprinting leaves no trace that the user is being tracked. Unlike cookies, users can’t see or control fingerprinting.

Example: Canvas fingerprinting: draw invisible text, then read it back as a sequence of bits. Because of the tiny differences between devices, the bit string acts as a device identifiers.

Scheme flooding: Testing custom URLs schemes like skype:// to discover the apps installed on the device.

Cross-device tracking

Two devices can be linked to the same user if:

  • User logs in with the same credentials from both devices
  • User visits the same/similar set of websites on both devices
  • User travels with two portable devices

Ultrasound beacons

  1. TV ad emits ultrasound (inaudible) binary signal that encodes a unique ID
  2. Viewer’s smartphone app listens in the background
  3. When ultrasound ID is detected, reports that ad has been watched.

Beacon-based tracking is not as widespread as fingerprinting and cross-device tracking. It was implemented by SilverPush and incorporated in a small number of apps.

In recent versions of iOS/Android, apps can’t record audio in the background without user awareness/consent.

Merging online and offline databases

Scenario: retailer wants to target shoppers with ads when they browse online

  1. Consumer shops at retail store, provides email address
  2. Store uploads list of consumer IDs to online advertiser
  3. Consumer logs in to (say) news website using email address
  4. Third-party tracker links the user to retail DB
  5. Ads are served via a cookie that follows the user around

Machine Learning and Inference

Target infamously predicted that a girl was pregnant before her father was aware.

The piece itself is even more interesting:

In major life events, e.g. graduating college, moving to new town, arrival of a baby, consumers change their shopping habits. Bring in a consumer to buy diapers, they’ll probably buy other stuff too. Once birth records are updated, there’s no edge.

So Target built a model to assign the likelihood of pregancy, and it was so good that there was public outcry. Target mixes in other coupons to be less creepy.

Using Facebook likes, one can meaningfully predict sexual orientation, ethnicity, religious and political views, personality traits, intelligence, happiness, use of addictive substances, parental separation, age and gender.

Deep neural networks do better than humans at detecting sexual orientation from facial images.

References

  1. Third-Party Web Tracking: Policy and Technology. Mayer, Jonathan; Mitchell, John. http://cyberlaw.stanford.edu/files/publication/files/trackingsurvey12.pdf . 2012.
  2. Pixel perfect: Fingerprinting canvas in HTML5. Mowery, Keaton; Shacham, Hovav. https://hovav.net/ucsd/dist/canvas.pdf . 2012.
  3. Cross-Device Tracking: Measurement and Disclosures. Brookman, Justin; Rouge, Phoebe; Alva, Aaron; Yeung, Christina. https://petsymposium.org/2017/papers/issue2/paper29-2017-2-source.pdf . 2017.
  4. Private traits and attributes are predictable from digital records of human behavior. Michal Kosinski; David Stillwell; Thore Graepel. https://www.pnas.org/content/pnas/110/15/5802.full.pdf . Feb 13, 2013.
  5. Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Yilun Wang; Michal Kosinski. https://www.semanticscholar.org/paper/Deep-Neural-Networks-Can-Detect-Sexual-Orientation-Wang-Kosinski/af30a2394c620132884bb98c78b6b9e46c791482 . 2017.
  6. The case against behavioral advertising is stacking up. Natasha Lomas. https://techcrunch.com/2019/01/20/dont-be-creepy/ . Jan 20, 2019. Accessed Jun 20, 2021.