Spotify: Product Area Lead, Data Infrastructure
Spotify launched in Sweden 2008 and has since spread to 55 markets. Our data infrastructure is now used by hundreds of engineers and analysts on a daily basis as part of their daily work. The decision to platformize our data infrastructure was a key decision to enable our organization to scale and enable easy access to data. Coming from humble beginnings of trying to move our reporting to Hadoop, we now power our entire business with data from Hadoop. Over the last few years we have continuously invested in our data infrastructure, by which we typically mean the systems that collect, store, process and visualize data. By enabling access to our data and putting it in the hands of our analysts we allow Spotify to make more well-informed decisions and make it possible to validate assumptions. We are also using our data to enrich the experience of our users by personalizing their product experience and providing cool product features. This presentation will give an overview of Spotify's data infrastructure and explain our architectural and organizational transformation by looking at it with a historical context. We will share some of the analytics we do by zooming in on a few key use-cases and looking at what infrastructure is required. We will also share some of the problems we have faced over the last few years and a few of the lessons learned.
Spotify brings you the right music for every moment. WIth over 24 million users and 20 million songs, Spotify has to deal with vast amounts of data everyday. Over the last few years Spotify has seen a small hadoop cluster for reporting grow into a platform that feeds the entire organization with data. Fabian Alenius is a Product Area Lead for Data Infrastructure at Spotify and has seen the transition firsthand.