At Delivery Hero, personalisation of the user experience is at the very core. Right from identifying the right bakery for your dessert cravings to your favourite curry place for lunch, we get it all to your fingertips. Let us see what goes behind the scenes to make this all happen.
More often than not, when we talk about a recommendation system, the spotlight is on the algorithm used. However, in the last few years while working on our own recommendation and personalization system we discovered that there is more to it than meets the eye. Like any other complex and sophisticated system around us, recommendation systems are composed of a number of building blocks and moving parts. Not paying attention to each of these components can cause even the best algorithm to fail.
In this post, we will discuss the building blocks of our personalisation system along with our learnings. Let’s get started!
A Team of Heroes
The first and the most important building block of this overall setup is a highly-skilled, motivated cross-functional team. No wonder we call them “Heroes” @DH. In our experience, having a cross-functional data focussed team that has the autonomy to take independent decisions about the system being developed is very important. The setup that has worked for us, is a good mix of broad and deep expertise across roles and levels. To envision the system and plan, we have our own strategist (product manager). A highly skilled set of Data Scientists from varying backgrounds such as engineering, mathematics and physics keeps us grounded and enables us to challenge the status quo of ML-first approaches. A quick and talented set of investigators (data analysts) who love to dig into things and are curious to the core. To make all the ML/AI magic and plumbing work seamlessly (data pipelines, APIs, storage and more), an ingenious set of software and data engineers. And finally, a business owner to help unblock a lot of operational challenges keeping the team aligned to the vision.
Interesting Fact: Even though our team is spread between Berlin and Singapore Offices, we have been able to work closely in a highly collaborative and supportive environment.
Vision, Strategy and Roadmap
Now, this might sound a bit superficial and flashy but it all begins with a goal or a mission statement. A goal to answer open questions and solve for gaps in offerings. A high-level goal/need for a team’s existence and its future are typically summarized as Mission and Vision statements.
First and foremost, you need to set up the right vision and strategy for the team. It is important to spend some time understanding the current state of the system. In our case, our ranking system was a rule based system a few years back and there was very little personalization. Our aim was to strive for an improved customer experience. To improve the experience, personalization was the obvious next step. Our main hypothesis was that by learning more about our users, understanding their tastes and needs we should be able to improve the relevance for our end customers.
One important thing to think about is your north star metric/KPI. The driving question to ask is “why are we doing this?”. What does it mean to improve relevance? How will we be able to measure it and what tradeoffs are there to consider? In our case, our focus was towards measuring our KPIs through A/B tests, results of which can be attributed to our system.
The mission and vision statements are what drive a team, help in aligning stakeholders and provide guard-rails throughout the life-cycle of such teams. Make sure you give it enough thought, challenge the plan and vet it multiple times before the actual work begins.
The Architecture
Every product is like an organism that grows and improves as it progresses in its lifecycle. A recommendation system is no different. The following figure introduces a high-level schematic of our recommendation system (simplified for ease of understanding and discussion)
We will go into a lot more detail in a later blog where we plan to explain each of these systems. For now, the following is a gist of some learnings which we discovered over time:
- The ability to serve multiple models in parallel is very important to empower research and improve the speed of experimentation.
- A configurable re-ranking layer can enable the team to test quite a few hypotheses without your data scientists spending a lot of time on them.
- A feature store which can be used across multiple models not only helps you bring standardization but also makes model generation faster.
Automation is super important. Lesser human intervention at each step means faster time to market and also lesser errors. It is no surprise that a system of such scale and complexity immensely benefits from CI/CD pipelines and monitoring/alerting mechanisms.
Conclusion
We hope we were able to intrigue you enough about recommendation systems. Building one such system of our own has been an exciting and insightful experience. Our aim was to highlight the learnings and share them. We discussed composition of the team, vision-mission and strategic stuff and finally a glimpse of the overall architecture. Stay tuned, we have a number of detailed follow-ups coming shortly covering aspects of understanding customers, modeling for new and existing users, setting up online AB experiments, offline evaluation framework and more.
If you like what you’ve read and you’re someone who wants to work on open, interesting projects in a caring environment, we’re on the lookout for Data Engineers, Data Scientists, Data Science Managers, and Data Engineering Managers. Check out our full list of open roles here – from Backend to Frontend and everything in between. We’d love to have you on board for an amazing journey ahead.
