Author:Ken W. Collier
Publisher: Addison-Wesley
Pages: 368
ISBN: 978-0321504814
Audience: DW/BI developers
Rating: 5
Reviewer: Sue Gee

This book's snappy two-word title stands in for Agile Data Warehousing, Business Intelligence, and Analytics.

Data warehousing projects are typically lengthy and expensive and statistics suggest that a high proportion fail - time then to investigate an approach that holds out the promise of delivery results more quickly, with economically and with a higher success rate.

This book adapts and applies the principles of the Agile Manifesto to building data warehouses and is intended both for DW/BI practitioners who want to learn more about Agile techniques and for Agile PR actioners who want to know how to apply Agile practices to DW/BI systems. 

As explained in tis preface, the book is motivated by the author's experience of a large data warehousing project, using non-Agile methods, that failed and his subsequent adoption of Agile methods, mentored by Jim Highsmith, one of the founding fathers of the Agile movement. As it progresses the book gives the impression of being rooted in real-world practical experience. This gives confidence in the DW/BI case-study that runs throughout the book as an example of how to turn Agile principles into practice.  

The book is organized into two parts, the first focused on Agile project management techniques and delivery team coordination in the DW/BI context and the second on the technical methods and practices for delivery of working DW/BI systems.

If you are wondering why the book's cover shows a mountaineer on a snow covered rock face, Chapter 1: Introducing Agile Analytics provides the explanation. It compares two types of approach to climbing high mountains such as Everest and Annapurna. The older "siege-style" strategy involved large numbers of porters carrying massive amounts of gear, Such expeditions were lengthy and inflexible. More recently "alpine-style" climbing made it feasible for smaller teams to conquer the high peaks with an approach that had the advantages of responsiveness to uncertain conditions. Agile DW/ BI development is likened to alpine style mountaineering.

Having established this analogy Ken Collier states the way in which he has modified the principles of the Agile Manifesto to make it appropriate in the context of DW/BI and gives a summary of the key characteristics of Agile Analytics. The first of these is its iterative, incremental and evolutionary style of development -characterized by short iterations with frequent user reviews. Later in the chapter Collier dispels some myths and misconceptions emphasising that Agile Analytics is not a wholesale replacement of traditional practices, but that it isn't enough just to break down the traditional waterfall method into short blocks. A section is devoted to answering the question "Why do we need Agile analytics" with three "truths":

1 Building DW/BI systems is hard

2 DW/BI development projects often fail

3 It is best to fail fast and adept

Chapter 2: Agile Project Management then introduces practices for planning, executing and monitoring an Agile Analytics project, contrasting it with the waterfall development approach. It discusses replacing a Plan -> Do model with an Envision -> Explore cycle and advocates "Just Enough Design", as opposed to Big Design Up Front. This translates in practical terms to ideas such as Work in Small Steps; Daily Co-ordination. Timebox Everything and Team Self-Monitoring. 

The focus in Chapter 3 is on two types of collaboration - between team members and with the customer. It establishes the need for a shared-responsibility community model in which the planners, consumers and doers each form an essential part of the project.

Chapter 4: User Stories For BI Systems looks at one of the most fundamental shifts from traditional DW/BI development to Agile Analytics development - the explicit focus on delivering working features to end users and then Chapter 5 looks at the way Agile teams need to be self-organizing by managing their own processes, techniques and outcomes. 

 Click for book details from Informit

In the second part of the book, Collier goes deeper into the technicalities of implementing Agile methods, including SQL code and batch files. Chapter 6, Evolving Excelling Design, has a detailed section on Refactoring before presenting a case study where Collier’s team evolved from traditional architecture to Agile Analytics. Chapter 7 is on the role of testing – test-driven development, integrated testing and test automation, Chapter 8 is introduces version control and Chapter 9 looks at project automation and continuous integration.

Chapter 10: Final Words reiterates an idea from Chapter 1 - Agile Analytics is not a methodology, rather it is a development style based on a set of core values. The author equates Being Agile with success patterns and Doing Agile with anti-patterns and he provides a list of "smells" that are indicative of "Doing Agile". He also describes some of the “gnarly problems” in DW/BI and concludes with a set of adoption strategies.

By the end of the book I came away with the impression that the author wasn’t trying to minimize the problems is adopting Agile methodology - and this made his advice all the more convincing. This is a book that every data warehouse builder should read, whether or not they are already attracted by the ideas of Agile methodology.