Media and content / Online advertising
Meta: Upgrading the ad recommendation model
- Company
- Meta
- Country
- United States
- Adoption stage
- In operation
- Source published
- Date basis
- The date the source was published. It can differ from the date adoption started.
- How the source was checked
- Read the full source text
The work problem
Billions of interactions between users and ads build up across Meta's apps each day. Among them, meaningful signals such as clicks or conversions are rare. Material of different kinds has to be handled together: advertiser objectives, creative formats, measurement signals and user behaviour. The existing approach compressed long behaviour histories into short vectors and lost important signals in the process.
Technology and data
GEM is a foundation model for ad recommendation. It was designed in the manner of an LLM and trained on thousands of GPUs. The training material is ad content and user interaction records, and it uses reactions to ordinary posts as well as ads. Features are split into ordered behaviour records and unordered attributes. Attention suited to each is applied, and the relationship between the two groups is learned as well. Once training is done, the knowledge is transferred to downstream models. GEM sometimes passes it straight to the main models and sometimes through domain-specific foundation models. In distillation, a Student Adapter corrects the teacher model's output with the latest real outcomes.
Results
Meta said it released GEM earlier this year and applied it to Facebook and Instagram. It announced that as a result, ad conversions on Instagram rose 5 percent and ad conversions in Facebook Feed rose 3 percent in the second quarter. The company explained that the new architecture is four times as efficient as the previous ad ranking model on the same data and compute. It said the knowledge transfer method is twice as effective as ordinary distillation. It wrote that in the third quarter it revised the architecture and doubled the performance gain obtained from the same data and compute.
Limits and open questions
The conversion growth rates are Meta's own assessment. This is different from a case of generating ad creative automatically, and the figures are not revenue or net profit growth rates.
Sources
Compiled from public sources. These are not results from ATF Works customers.