Media and content / Online video
Netflix: Connecting a shared recommendation model to many services
- Company
- Netflix
- Country
- United States
- Adoption stage
- In operation
- Source published
- Date basis
- The date the source was published. It can differ from the date adoption started.
- How the source was checked
- Read the full source text
The work problem
The Netflix home screen is split among several specialised models. Maintaining and improving each of them took a lot of time and resources. So the company decided to gather member taste learning in one place and let one large model share what it learns with the other models. Material covering how to fit a large model into a production system that is already running was scarce.
Technology and data
Netflix tried three ways of connecting and now uses all of them depending on the purpose. The first is embeddings. The hidden state of the last user event is taken as the profile embedding and the weights of the item tower as the item embedding. The Foundation Model is pre-trained from scratch every month and fine-tuned daily on the latest data. Batch inference regenerates the embeddings and uploads them to the Embedding Store. Because the embedding space changes with each new training run, a stabilisation technique aligns it to the same space. The second is subgraphs. The decoder stack of the Foundation Model is placed inside the application model graph so raw user behaviour sequences are processed directly. The third is fine-tuning. Screens where recent behaviour matters more, such as the Trending now row, use a model fine-tuned on product-specific data directly.
Results
Netflix said all three approaches are used in live recommendation services. The embedding approach costs little and has a large effect, so most work starts there. The team regards this foundation as important and additionally built a near-real-time embedding generation system so embeddings can be updated from behaviour within a session.
Limits and open questions
The source has no service metrics such as a rise in viewing. Netflix also stated the downsides together: embeddings becoming less current, subgraphs increasing model size and inference time, and fine-tuning increasing the number of models to maintain.
Sources
- integrating netflixs foundation model into personalization applicationsnetflixtechblog.medium.com, Accessed
Compiled from public sources. These are not results from ATF Works customers.