REINFORCEMENT LEARNING

RANKING & RECOMMENDATIONS

Reinforcement Learning-based Product Delivery Frequency Control

December 18, 2020

Abstract

Frequency control is an important problem in modern recommender systems. It dictates the delivery frequency of recommendations to maintain product quality and efficiency. For example, the frequency of delivering promotional notifications impacts daily metrics as well as the infrastructure resource consumption (e.g. CPU and memory usage). There remain open questions on what objective we should optimize to represent business values in the long term best, and how we should balance between daily metrics and resource consumption in a dynamically fluctuating environment. We propose a personalized methodology for the frequency control problem, which combines long-term value optimization using reinforcement learning (RL) with a robust volume control technique we termed "Effective Factor". We demonstrate statistically significant improvement in daily metrics and resource efficiency by our method in several notification applications at a scale of billions of users. To our best knowledge, our study represents the first deep RL application on the frequency control problem at such an industrial scale.

Download the Paper

AUTHORS

Written by

Yang Liu

Zhengxing Chen

Kittipat Virochsiri

Juan Wang

Jiahao Wu

Feng Liang

Publisher

Conference on Artificial Intelligence (AAAI)

Research Topics

Ranking and Recommendations

Reinforcement Learning

Related Publications

May 03, 2019

RANKING & RECOMMENDATIONS

Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search | Facebook AI Research

Jinfeng Rao, Wei Yang, Yuhao Zhang, Ferhan Ture, Jimmy Lin

May 03, 2019

November 01, 2018

RANKING & RECOMMENDATIONS

Horizon: Facebook's Open Source Applied Reinforcement Learning Platform | Facebook AI Research

Jason Gauci, Edoardo Conti, Yitao Liang, Kittipat Virochsiri, Yuchen He, Zachary Kaden, Vivek Narayanan, Xiaohui Ye

November 01, 2018

December 03, 2018

RANKING & RECOMMENDATIONS

NLP

Training with Low-precision Embedding Tables | Facebook AI Research

Jian Zhang, Jiyan Yang, Hector Yuen

December 03, 2018

December 18, 2020

RANKING & RECOMMENDATIONS

REINFORCEMENT LEARNING

Reinforcement Learning-based Product Delivery Frequency Control

Yang Liu, Zhengxing Chen, Kittipat Virochsiri, Juan Wang, Jiahao Wu, Feng Liang

December 18, 2020

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.