Craigslist’s Jeremy Zawodny’s presentation on data storage technologies at Craigslist is available as a video, and gives some fascinating insights into why Craigslist has made various choices of technology.
Zawodny’s presentation was originally given at the 2012 Percona Live MySQL Conference in Santa Clara. The listings website uses both SQL (MySQL) and NoSQL (Redis, MongoDB and memcached), and Zawodny discusses what’s in use, where, why, and why not.
You can view the video below:
Zawodny says MySQL is used for a variety of reasons - it’s the devil they know, very reliable and they have a lot of admin and developer skills in using it.Other reasons for choosing MySQL are durability, replication, and the support.
Memcached makes the grade because, says Zadowny, it’s wickedly fast, stable, virtually zero admin, and easily exists with CPU-intensive services. In answer to the multi-core issue, he says run more instances!
Redis was chosen because of the performance, flexible persistence, and an excellent but simple API. The same solution is put forward for the multi- core question.
The reasons for choosing MongoDB start with the fact it’s schema free. Other reasons include the active community alongside the commercial support. The Perl client is a plus point, but Zawodny reserves his most enthusiastic praise for the ease of scaling (Yay! for built-in sharding support) and fewer single points of failure due to the ‘awesome’ replica sets.
Zadowny also goes through the reasons for choosing Sphinx and Filesystem, and concludes that in answer to the question of relational v non-relational, in practice Craigslist often just doesn’t care, adding that NoSQL is a stupid label.
When OpenAI, an open source AI initiative backed by Elon Musk, Sam Altman and Ilya Sutskever, was announced earlier in the year, I doubt anyone expected anything to be produced so quickly and certainl [ ... ]