JET write-through cache

2 years ago

This project pushed us into a completely different league of load. We were already used to big campaigns, but this was the whole UK market arriving at once. Just Eat Takeaway users received a link to our activation to win prizes, and they all showed up at dinner time, exactly when people order their food. That created an enormous, concentrated spike of traffic.

Our usual setup, a regular API with the database tricks we normally rely on, simply could not hold that load.

So for the first time we rebuilt the application around a write-through cache. Everything was written to a Redis cache, and through a pub/sub system that same data was also persisted to the database. Reads stayed lightning fast because they never had to hit the database directly.

Because Redis only gave us keys and hashes to work with, it really challenged me to think differently about how to model and solve problems in code. It was a different way of programming than I was used to, and I learned a lot from it.

On top of that we built a complex dynamic win-chance system. The win chance changed live during the campaign based on how many prizes had already been won in a given period, automatically scaling up or down to keep the prize distribution even. With so many prizes being won in such a short window, that live balancing was essential.

For the very first campaign that went live, from the 3rd to the 23rd of December, we handled around 2 million visitors in a short and intense period. All in all it was a great learning experience for me and the team.