Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data
Yanru Qu, Bohui Fang, Wei-Nan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, Xiuqiang He
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/Atomu2014/product-nets-distributedOfficialIn papertf★ 0
- github.com/shenweichen/DeepCTRtf★ 8,006
- github.com/xue-pai/FuxiCTRpytorch★ 1,379
- github.com/UlionTse/mlgbpytorch★ 1,049
- github.com/Atomu2014/product-netstf★ 0
- github.com/MindSpore-scientific/code-13/tree/main/PR_Productmindspore★ 0
- github.com/MindSpore-scientific-2/code-4/tree/main/PR_Productmindspore★ 0
- github.com/DataCanvasIO/DeepTablestf★ 0
- github.com/shenweichen/DeepCTR-PyTorchpytorch★ 0
- github.com/jccarles/product-netstf★ 0
Abstract
User response prediction is a crucial component for personalized information retrieval and filtering scenarios, such as recommender system and web search. The data in user response prediction is mostly in a multi-field categorical format and transformed into sparse representations via one-hot encoding. Due to the sparsity problems in representation and optimization, most research focuses on feature engineering and shallow modeling. Recently, deep neural networks have attracted research attention on such a problem for their high capacity and end-to-end training scheme. In this paper, we study user response prediction in the scenario of click prediction. We first analyze a coupled gradient issue in latent vector-based models and propose kernel product to learn field-aware feature interactions. Then we discuss an insensitive gradient issue in DNN-based models and propose Product-based Neural Network (PNN) which adopts a feature extractor to explore feature interactions. Generalizing the kernel product to a net-in-net architecture, we further propose Product-network In Network (PIN) which can generalize previous models. Extensive experiments on 4 industrial datasets and 1 contest dataset demonstrate that our models consistently outperform 8 baselines on both AUC and log loss. Besides, PIN makes great CTR improvement (relatively 34.67%) in online A/B test.