Craigslist’s Jeremy Zawodny’s presentation on data storage technologies at Craigslist is available as a video, and gives some fascinating insights into why Craigslist has made various choices of technology.
Zawodny’s presentation was originally given at the 2012 Percona Live MySQL Conference in Santa Clara. The listings website uses both SQL (MySQL) and NoSQL (Redis, MongoDB and memcached), and Zawodny discusses what’s in use, where, why, and why not.
You can view the video below:
Zawodny says MySQL is used for a variety of reasons - it’s the devil they know, very reliable and they have a lot of admin and developer skills in using it.Other reasons for choosing MySQL are durability, replication, and the support.
Memcached makes the grade because, says Zadowny, it’s wickedly fast, stable, virtually zero admin, and easily exists with CPU-intensive services. In answer to the multi-core issue, he says run more instances!
Redis was chosen because of the performance, flexible persistence, and an excellent but simple API. The same solution is put forward for the multi- core question.
The reasons for choosing MongoDB start with the fact it’s schema free. Other reasons include the active community alongside the commercial support. The Perl client is a plus point, but Zawodny reserves his most enthusiastic praise for the ease of scaling (Yay! for built-in sharding support) and fewer single points of failure due to the ‘awesome’ replica sets.
Zadowny also goes through the reasons for choosing Sphinx and Filesystem, and concludes that in answer to the question of relational v non-relational, in practice Craigslist often just doesn’t care, adding that NoSQL is a stupid label.
A data-processing language being developed by the Apache Software Foundation has been elevated to top-level status. Flink is open source, has APIs for Java and Scala and, with specialized APIs for gra [ ... ]
This article, first published in February 2013, proved to be our most popular ever. For those who missed or, or who want to be reminded of it, here it is again. Programmers often say that regular expr [ ... ]