Efficient Knowledge Deletion from Trained Models Through Layer-wise Partial Machine Unlearning

Machine unlearning has garnered significant attention due to its ability to selectively eraseknowledge obtained from specific training data samples in an already trained machinelearning model. This capability enables data holders to adhere strictly to data protectionregulations. However, existing unlearning techniques face practical constraints, often causingperformance degradation, demanding brief fine-tuning post unlearning, and requiringsignificant storage. In response, this paper introduces a novel class of layer-wise partial machineunlearning algorithms that enable selective and controlled erasure of targeted knowledge.Of these, partial amnesiac unlearning integrates layer-wise selective pruning with thestate-of-the-art amnesiac unlearning. This method selectively prunes and stores updatesmade to the model during training, enabling the targeted removal of specific data from thetrained model. Other methods assimilates layer-wise partial-updates into label-flipping andoptimization-based unlearning, thereby mitigating the adverse effects of specific knowledgedeletion on model efficacy. Through a detailed experimental evaluation, we showcase theeffectiveness of proposed unlearning methods. Experimental results highlight that the partialamnesiac unlearning not only preserves model efficacy but also eliminates the necessityfor brief fine-tuning post unlearning, unlike conventional amnesiac unlearning. Further,employing layer-wise partial updates in label-flipping and optimization-based unlearningtechniques demonstrates superiority in preserving model efficacy compared to their naivecounterparts.