July 14–18, 2024, Washington D.C., USA | Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
This paper addresses the issue of adversarial ranking attacks on neural ranking models (NRMs) by introducing a multi-granular adversarial attack method. Traditional single-granular attacks, which perturb documents at either word, phrase, or sentence levels, have limitations in their effectiveness and flexibility. To overcome these limitations, the authors propose a multi-granular approach that incorporates perturbations at multiple levels of granularity, enhancing the attack's effectiveness and naturalness.
The authors formulate the multi-granular attack problem as a sequential decision-making process, where the attacker sequentially introduces perturbations at different granularities, guided by a surrogate ranking model and an advanced language model (LLM). The surrogate model simulates the behavior of the target NRM, while the LLM evaluates the naturalness of the perturbed documents. The multi-granular attacker consists of two agents: a sub-agent that identifies vulnerable positions and a meta-agent that generates and organizes perturbations.
Experimental results on two benchmark datasets, MS MARCO and ClueWeb09-B, demonstrate that the proposed RL-MARA framework significantly outperforms existing single-granular attack methods in terms of attack effectiveness and imperceptibility. The framework's ability to balance attack effectiveness and naturalness is highlighted, and its robustness in different scenarios, including white-box and out-of-distribution settings, is evaluated. The study also assesses the naturalness of the generated adversarial examples, showing that RL-MARA produces more fluent and less suspicious perturbations compared to baseline methods.This paper addresses the issue of adversarial ranking attacks on neural ranking models (NRMs) by introducing a multi-granular adversarial attack method. Traditional single-granular attacks, which perturb documents at either word, phrase, or sentence levels, have limitations in their effectiveness and flexibility. To overcome these limitations, the authors propose a multi-granular approach that incorporates perturbations at multiple levels of granularity, enhancing the attack's effectiveness and naturalness.
The authors formulate the multi-granular attack problem as a sequential decision-making process, where the attacker sequentially introduces perturbations at different granularities, guided by a surrogate ranking model and an advanced language model (LLM). The surrogate model simulates the behavior of the target NRM, while the LLM evaluates the naturalness of the perturbed documents. The multi-granular attacker consists of two agents: a sub-agent that identifies vulnerable positions and a meta-agent that generates and organizes perturbations.
Experimental results on two benchmark datasets, MS MARCO and ClueWeb09-B, demonstrate that the proposed RL-MARA framework significantly outperforms existing single-granular attack methods in terms of attack effectiveness and imperceptibility. The framework's ability to balance attack effectiveness and naturalness is highlighted, and its robustness in different scenarios, including white-box and out-of-distribution settings, is evaluated. The study also assesses the naturalness of the generated adversarial examples, showing that RL-MARA produces more fluent and less suspicious perturbations compared to baseline methods.