新加坡国立大学：Adversarial Personalized Ranking for Recommendation

NUS崖, S|G|R2018 National University Lab for Media Search Adversarial Personalized Ranking for Recommendation Xiangnan He, Zhankui He, Xiaoyu du, Tat-Seng Chua School of Computing National University of Singapore

Adversarial Personalized Ranking for Recommendation Xiangnan He, Zhankui He, Xiaoyu Du, Tat-Seng Chua School of Computing National University of Singapore 1 SIGIR 2018

DeNUS Motivation ational University The core of ir tasks is ranking Search Given a query, ranking documents Recommendation Given a user, ranking items a personalized ranking task Ranking is usually supported by the underlying scoring model Linear. Probabilistic, Neural network models etc Model parameters are learned by optimizing learning- to-rank loss Question: is the learned model robust in ranking? Will small change on inputs/parameters lead to big change on the ranking result? This concerns model generalization ability

Motivation • The core of IR tasks is ranking. • Search: Given a query, ranking documents • Recommendation: Given a user, ranking items – A personalized ranking task • Ranking is usually supported by the underlying scoring model. – Linear, Probabilistic, Neural network models etc. – Model parameters are learned by optimizing learning-to-rank loss • Question: is the learned model robust in ranking? – Will small change on inputs/parameters lead to big change on the ranking result? – This concerns model generalization ability. 2

Adversarial Examples on DeNUS ational University Classification ( Goodfellow et al, ICLR' 15 Recent efforts on adversarial machine learning show many well-trained classitiers sutter from adversarial examples: +.007× panda nematode” gibbon 57.7 confidence 8. 2% confidence 99.3 confidence This implies weak generalization ability of the classifier Question: do such adversarial examples also exist for IR ranking methods?

Adversarial Examples on Classification (Goodfellow et al, ICLR’15) • Recent efforts on adversarial machine learning show many well-trained classifiers suffer from adversarial examples: – This implies weak generalization ability of the classifier • Question: do such adversarial examples also exist for IR ranking methods? 3

Adversarial Examples on DeNUS ational University Personalized Ranking We train visually-aware BPR(He et al, AAAl16 on a user image interaction dataset for visualization VBPR is a pairwise learning-to-rank method Effect of adversarial examples on personalized ranking Top-4 image ranking 360 5.5 +0.007 of a sampled user. o0a0o05 3.50 +0.007x before vs. after adversarial noise: 49 +0.007 翻 34 测+000 4.13 ariginal Images Perturbed Ir Small adversarial noises on images ( noise level e= 0.007)leads to big change on ranking

Adversarial Examples on Personalized Ranking • We train Visually-aware BPR (He et al, AAAI’16) on a user￾image interaction dataset for visualization. – VBPR is a pairwise learning-to-rank method • Effect of adversarial examples on personalized ranking: 4 Small adversarial noises on images (noise level ϵ = 0.007)leads to big change on ranking. Ranking scores (before) Ranking scores (after) Top-4 image ranking of a sampled user. before vs. after adversarial noise:

Quantitative Analysis on DeNUS ational University Adversarial Attacks We train matrix factorization(Mf)with BPR loss MF is a widely used model in recommendation BPR is a standard pairwise loss for personalized ranking We add noises on model parameters of mf Random noise vs. Adversarial noise Performance change w.r.t. different noise levels E(i. e, L, norm Conclusion: 0.16 MF-BPR is robust to 0.06 0.1 random noise but not 0.08 for adversarial noise! ● Adversarial Noise 004 Adversarial Noise Random Noise 0 040.60.8 (a) Testing NDCG vs.∈ (c) Testing NDCG vs.∈

Quantitative Analysis on Adversarial Attacks • We train matrix factorization (MF) with BPR loss – MF is a widely used model in recommendation – BPR is a standard pairwise loss for personalized ranking • We add noises on model parameters of MF – Random noise vs. Adversarial noise – Performance change w.r.t. different noise levels ε (i.e., L2 norm): 5 Conclusion: MF-BPR is robust to random noise, but not for adversarial noise!

DeNUS Outline ational University Introduction motivation Method Recap bpr Bayesian Personalized Ranking APR: Adversarial training for BPR Experiments Conclusion

Outline • Introduction & Motivation • Method – Recap BPR (Bayesian Personalized Ranking) – APR: Adversarial Training for BPR • Experiments • Conclusion 6

DeNUS Recap BPR ational University BPR aims to maximize the margin between an ordered example paIr. sigmoid Positive prediction Negative prediction LBPR(O⊙) In oyu(o)-yuj(e)+hellOll (,i,j)∈分 Pairwise training examples: u prefers i over j An example of using bpr to optimize mf model BPR Objective In al.u- 9u). Training Minimizer yui=Puq redictions Puqi Embeddin qp q (positive item) user (negative item [Rendle et al, UAI09

Recap BPR 7 • BPR aims to maximize the margin between an ordered example pair. • An example of using BPR to optimize MF model: Pairwise training examples: u prefers i over j sigmoid Positive prediction Negative prediction [Rendle et al, UAI’09]

Our Method APR: Adversarial DeNUS ational University Personalized Ranking The aim is to improve the robustness of model trained for personalized ranking ·|dea: 1 Construct an adversary to generate noise on bPr during training 2)train the model to make it perform well even under noise Original BPR Loss Perturbed bpr loss BPR(O)⊙)+|LBR(D|e+△ Generate additive noise by Minimize maximizing BPr loss Learner Adversary

Our Method APR: Adversarial Personalized Ranking • The aim is to improve the robustness of model trained for personalized ranking. • Idea: 1) Construct an adversary to generate noise on BPR during training 2) Train the model to make it perform well even under noise. 8 Learner Original BPR Loss Perturbed BPR Loss + Minimize Adversary Generate additive noise by maximizing BPR loss

DeNUS APR Formulation ational University earning objective of apr (to be minimized Adversarial noise LAPR(⊙)=LBPR(Oe)+LBPR(O|e+△a Original BPR Loss Perturbed bpr loss Where the adversarial noise tries to maximize bpr loss db= arg max LBp(同+△) E Current model parameters Control magnitude of noise(avoid trivial solution that simply increases value Can be seen as adding an adaptive regularizer to BPr training Dynamically change during training n controls strength of regularization

APR Formulation • Learning objective of APR (to be minimized): where the adversarial noise tries to maximize BPR loss: • Can be seen as adding an adaptive regularizer to BPR training – Dynamically change during training – λ controls strength of regularization 9 Original BPR Loss Perturbed BPR Loss Adversarial noise Control magnitude of noise (avoid trivial solution that simply increases value) Current model parameters

DeNUS APR Formulation ational University Overall formulation is solving a mini-max problem e,△*= arg min, max LBPR(O)+LBPR(⊙+△ e△,A‖|≤e Model Learning mInl-max game Minimize ran king loss Adversary Learning --- Maximize ranking loss i adversary loss Next: Iterative two-step solution for apr learning 1. Generate Adversarial Noise(maximizing player) 2. Update Model Parameters(minimizing player Until a convergence state is reached

APR Formulation • Overall formulation is solving a mini-max problem: • Next: Iterative two-step solution for APR learning: 1. Generate Adversarial Noise (maximizing player) 2. Update Model Parameters (minimizing player) – Until a convergence state isreached 10 Model Learning Minimize ranking loss + adversary loss Adversary Learning Maximize ranking loss mini-max game