On the Path Dependence of Gradient Ascent-Based Unlearning

Machine unlearning enables selective erasure of knowledge associated with specific data points from trained models. In this work, we show that gradeint ascent-based machine unlearning is fundamentally path-dependent, i.e, under identical optimization budgets, different orderings of the same forget samples can induce qualitatively different forgetting vs retention trade-offs. Through experiments on CIFAR-10 and TinyImageNet, we demonstrate that structured sample orderings systematically steer unlearning toward distinct extremal regimes, whereas random ordering explores these outcomes only stochastically and with high variance. We further show that this behavior is robust across multiple ordering proxies, including epistemic uncertainty, loss, and gradient norm, indicating that the ordering itself, rather than a specific heuristic are the primary driver of trajectory divergence. These findings reveal sample ordering as a previously overlooked but practically significant degree of freedom in machine unlearning and motivate trajectory-aware design and evaluation of optimization-based unlearning methods.