RKLU: Redistributive KL Distillation for Efficient Retain-Free Machine Unlearning

Machine unlearning aims to remove the influence of specific training samples from model, motivated by privacy regulations and data revocation requirements. Existing approaches often depend on retain data, which compromises privacy and becomes computationally expensive. To address this challenge, we propose RKLU, a novel retain-data-free unlearning method that fine-tunes models by minimizing the KL divergence between their outputs and a target distribution that suppresses the probabilities of the samples to be forgotten. RKLU achieves near-perfect unlearning with minimal utility loss on diverse vision and text classification benchmarks, offering a privacy-preserving and efficient alternative.