REINFORCEMENT LEARNING

RANKING & RECOMMENDATIONS

Reinforcement Learning-based Product Delivery Frequency Control

December 18, 2020

Abstract

Frequency control is an important problem in modern recommender systems. It dictates the delivery frequency of recommendations to maintain product quality and efficiency. For example, the frequency of delivering promotional notifications impacts daily metrics as well as the infrastructure resource consumption (e.g. CPU and memory usage). There remain open questions on what objective we should optimize to represent business values in the long term best, and how we should balance between daily metrics and resource consumption in a dynamically fluctuating environment. We propose a personalized methodology for the frequency control problem, which combines long-term value optimization using reinforcement learning (RL) with a robust volume control technique we termed "Effective Factor". We demonstrate statistically significant improvement in daily metrics and resource efficiency by our method in several notification applications at a scale of billions of users. To our best knowledge, our study represents the first deep RL application on the frequency control problem at such an industrial scale.

Download the Paper

AUTHORS

Written by

Yang Liu

Zhengxing Chen

Kittipat Virochsiri

Juan Wang

Jiahao Wu

Feng Liang

Publisher

Conference on Artificial Intelligence (AAAI)

Research Topics

Ranking and Recommendations

Reinforcement Learning

Related Publications

May 03, 2019

RANKING & RECOMMENDATIONS

Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search | Facebook AI Research

Despite substantial interest in applications of neural networks to information retrieval, neural ranking models have mostly been applied to “standard” ad hoc retrieval tasks over web pages and newswire articles. This paper proposes MP-HCNN…

Jinfeng Rao, Wei Yang, Yuhao Zhang, Ferhan Ture, Jimmy Lin

May 03, 2019

November 01, 2018

RANKING & RECOMMENDATIONS

Horizon: Facebook's Open Source Applied Reinforcement Learning Platform | Facebook AI Research

In this paper we present Horizon, Facebook’s open source applied reinforcement learning (RL) platform. Horizon is an end-to-end platform designed to solve industry applied RL problems where datasets are large (millions to billions of…

Jason Gauci, Edoardo Conti, Yitao Liang, Kittipat Virochsiri, Yuchen He, Zachary Kaden, Vivek Narayanan, Xiaohui Ye

November 01, 2018

December 03, 2018

RANKING & RECOMMENDATIONS

NLP

Training with Low-precision Embedding Tables | Facebook AI Research

Starting from the success of Glove and Word2Vec in natural language processing, continuous representations are widely deployed in many other domain of applications. These applications span over encoding textual information to modeling user and…

Jian Zhang, Jiyan Yang, Hector Yuen

December 03, 2018

December 18, 2020

RANKING & RECOMMENDATIONS

REINFORCEMENT LEARNING

Reinforcement Learning-based Product Delivery Frequency Control

Frequency control is an important problem in modern recommender systems. It dictates the delivery frequency of recommendations to maintain product quality and efficiency.…

Yang Liu, Zhengxing Chen, Kittipat Virochsiri, Juan Wang, Jiahao Wu, Feng Liang

December 18, 2020

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.