Author: Rochelle King, Elizabeth F Churchill, and Caitlin Tan
Publisher: O'Reilly
Pages: 370
ISBN: 978-1449334833
Print: 1449334830
Kindle: B06XY9TTN8
Audience: Data developers
Rating: 4
Reviewer: Kay Ewbank

 

This book looks at how you can use data-driven A/B testing for making design decisions in your code.

A/B testing is a statistical technique where you run a controlled experiment where you test two versions of the thing being tested.

The book starts with a discussion of what the increasing amount of data means for developers, and how it can help you come up with better designs. This discussion is followed by a chapter on the ABCs of using data, where A/B testing is introduced.

The authors have created a framework for A/B testing, and this is introduced in the next chapter, along with a set of examples. The framework consists of three stages - definition, execution, and analysis, and each of these is then further explained in its own chapter, with examples and descriptions of how to frame your experiment, how to put it into action, and how to get and interpret the answers from your experiment.

 

 

A chapter on creating the right environment for data-aware design is next, explaining three key principles - shared company culture and values, hiring and growing the right people, and processes to support and align.

The book ends with a concluding chapter looking at ethical considerations - how your experiments might affect people's behavior, the power of suggestion, and general ethics in online experimentation.

This isn't a book about statistical techniques for testing. Instead, it's about things you need to think about when designing tests - how to choose your test users, what you can do to understand your variables. Despite some business guru speak, it is generally quite down to earth and reasonable, and does include some interesting ideas.