Last summer Mozilla conducted a research study in which it set out to investigate how different privacy protections affect website users. A surprising finding was that having Tracking Protection enabled improved user experience.

Mozilla sees its mission is to keep the web open and accessible for everyone. Privacy is therefore a fundamental concern, one aspect of which is striking the correct balance between usability and privacy protection.

The hypotheses Mozilla wanted to test with regard to Firefox were:

It set out to do this with an opt in study which attracted 19,000 users which meant there were over 8,500 active users on the most active day of the study. Participants were assigned to one of eight privacy setting variants or a control group:

To clarify the variants they were:

Over the study period Mozilla counted the number of problems reported by users assigned to each group.

The result that Mozilla reports as “surprising” concerns Tracking Protection, a feature that Firefox has had built into its Private Browsing Mode since 2015. Tracking Protection is not a full-blown content blocker, but does block trackers most of which are from advertising scripts. Specifically it blocks all third-party connections to domains on a block-list maintained by  Disconnect.me, specialists in online privacy and security. Prior to the study it was known that Tracking Protection breaks some websites whose code relies on third-party resources but the surprise was that the group of participants who had Tracking Protection enabled reported the lowest number of problems of any group. Although Mozilla’s report  doesn’t explicitly state the average number of problems per user for entire study, it is around 0.265. The rate for the control group is 0.24, i.e. lower than average. While a higher rate of problems - 0.250 to 0.281 was experienced for the other seven variants, that for Tracking Protection was even lower than for the control at around 0.234.

Looking into comments from the control group revealed the nature of their problems:  “not responsive”, “slow”, “freezing”, “took longer to load”, “not always responding”, “laggy”, “doesn’t load fast” all of which can be summed up in comment:

something on the page is slowing down the loading speed significantly.

The conclusion, according to Mozilla is:

third-party scripts cause a large number of performance problems. Tracking Protection removes them completely, so the number of problems is reduced. So, in a sense, Tracking Protection may actually fix websites by blocking tracking elements that break them, i.e. slow them down).

The second question to be answered was the extent to which performance problems introduced by privacy protection cause users to give up on Firefox. They therefore looked at the percentage of users who disabled the study, assuming that this was because of problems due to the enhanced protection level according to the group they were assigned.

The researchers noted that the percentage of users disabling the study was low across all branches - between a minimum of 5.7% (which  corresponded to originOnlyRefererToThirdParties) and a maximum of 9.7 (which  corresponded to firstPartyIsolation). The percentage of disabling in the control group was 6.2% and with the exception of both firstPartyIsolation groups (with 8) the other group were within the margin of error of the control group.

The way in which websites were broken was also shown to be important:

94% of users reporting screen breakage disabled study

84% of users reporting flash breakage disabled study

82% of users reporting login-failure breakage disabled study

64% of users reporting payment breakage disabled study

After looking at composite breakage scores. a weighted combination of the percentage of users who reported at least one problem, the average number of problems reported per user and the percentage of users who disabled the study the conclusion was that the most promising protections, in terms of lowest overall breakage were:

1. Origin-only Referrer values to third parties
2. Session-only third-party cookies
3. Tracking Protection

Mozilla has already taken action informed by the study in that in Firefox Quantum all users can enable Tracking Protection for their regular browsing. Moreover in Firefox 59+, Private Browsing will default to trimming Referrer values to origins.

More Information

Improving privacy without breaking the web

Related Articles

Firefox Quantum - Fast For Good

Mozilla Looks Into Health of Internet

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info