Adversarial Machine Learning: A Systematic Survey from the Lifecycle Perspective

Adversarial machine learning (AML) investigates vulnerabilities that cause machine learning systems to produce predictions deviating from human expectations. We unify the landscape of attacks and defenses through a lifecycle-aware perspective spanning pre-training, training, post-training, deployment, and inference. On the attack side, we synthesize three primary paradigms—backdoor attacks (arising through data poisoning or training control, activated at inference), weight attacks (injected post-training or at deployment via bit-flips), and adversarial examples (crafted at inference)—and clarify their connections and differences across stages. On the defense side, we present a coherent taxonomy aligned to the same lifecycle: pre-training (robust model architectures; poisoned data detection), training (adversarial training; secure centralized and decentralized training), post-training (backdoor detection, target-class detection, and mitigation/removal), deployment (model enhancement and fingerprint verification against weight attacks), and inference (poisoned-query detection, robust prediction, adversarial-query/sequence detection, and randomized/dynamic inference). To facilitate reproducible evaluation under this unified perspective, we further release two open-source benchmarks: BackdoorBench and BlackBoxBench that provide standardized datasets, protocols, and metrics mapped to the lifecycle stages, enabling apples-to-apples comparisons and stress-testing of cross-stage strategies.

These works bridge this gap through three key contributions: (1) offers formal, lifecycle-aware definitions of AML threats, (2) organizes both attack methodologies and stage-specific defenses into hierarchical taxonomies while elucidating inter-paradigm relationships, and (3) extends analysis to generative and multimodal models as well as beneficial applications.

Our findings emphasize that piecemeal, single-stage defenses are insufficient against coordinated, multi-stage attacks; instead, holistic, cross-stage strategies and lifecycle-aware monitoring are essential. We distill comparative insights and trade-offs (effectiveness, cost, clean-accuracy impact, scalability, generality, evasion difficulty), highlight practical considerations in safety-critical domains, and chart concrete research directions toward adaptive, automated, and comprehensive AML defenses.