This paper investigates the factors that make machine unlearning difficult and proposes a framework to improve unlearning performance. Machine unlearning aims to remove the influence of a subset of training data (the "forget set") from a trained model, often to comply with user data deletion requests or to remove problematic data. The study identifies two key factors affecting unlearning difficulty: the entanglement between the forget and retain sets in the embedding space, and the memorization of the forget set examples. The more entangled the sets are, and the more memorized the forget set examples are, the harder unlearning becomes. The paper evaluates the performance of various unlearning algorithms on different forget sets and finds that different algorithms perform better under different conditions. For example, relabelling-based algorithms perform poorly on highly entangled sets but better on highly memorized sets. The study also reveals that unlearning algorithms can fail in unexpected ways when dealing with complex data. To address these challenges, the authors propose the Refined-Unlearning Meta-algorithm (RUM), which involves two steps: first, refining the forget set into homogeneous subsets based on relevant factors, and second, applying different unlearning algorithms to each subset and combining the results. RUM significantly improves the performance of state-of-the-art unlearning algorithms by leveraging the strengths of different algorithms for different types of forget sets. The paper also introduces a new metric, ToW (tug-of-war), to evaluate unlearning performance by balancing the accuracy on the forget, retain, and test sets. The results show that RUM outperforms existing methods, especially when applied to homogenized subsets of the forget set. Additionally, the study finds that using a compute-efficient proxy for memorization, called C-proxy, can achieve similar performance gains without the high computational cost of calculating memorization scores. The paper concludes that understanding the factors affecting unlearning difficulty is crucial for improving unlearning algorithms and developing effective evaluation metrics. The proposed RUM framework provides a new approach to unlearning by leveraging the strengths of different algorithms for different types of forget sets, and it has the potential to significantly improve the state-of-the-art in unlearning research.This paper investigates the factors that make machine unlearning difficult and proposes a framework to improve unlearning performance. Machine unlearning aims to remove the influence of a subset of training data (the "forget set") from a trained model, often to comply with user data deletion requests or to remove problematic data. The study identifies two key factors affecting unlearning difficulty: the entanglement between the forget and retain sets in the embedding space, and the memorization of the forget set examples. The more entangled the sets are, and the more memorized the forget set examples are, the harder unlearning becomes. The paper evaluates the performance of various unlearning algorithms on different forget sets and finds that different algorithms perform better under different conditions. For example, relabelling-based algorithms perform poorly on highly entangled sets but better on highly memorized sets. The study also reveals that unlearning algorithms can fail in unexpected ways when dealing with complex data. To address these challenges, the authors propose the Refined-Unlearning Meta-algorithm (RUM), which involves two steps: first, refining the forget set into homogeneous subsets based on relevant factors, and second, applying different unlearning algorithms to each subset and combining the results. RUM significantly improves the performance of state-of-the-art unlearning algorithms by leveraging the strengths of different algorithms for different types of forget sets. The paper also introduces a new metric, ToW (tug-of-war), to evaluate unlearning performance by balancing the accuracy on the forget, retain, and test sets. The results show that RUM outperforms existing methods, especially when applied to homogenized subsets of the forget set. Additionally, the study finds that using a compute-efficient proxy for memorization, called C-proxy, can achieve similar performance gains without the high computational cost of calculating memorization scores. The paper concludes that understanding the factors affecting unlearning difficulty is crucial for improving unlearning algorithms and developing effective evaluation metrics. The proposed RUM framework provides a new approach to unlearning by leveraging the strengths of different algorithms for different types of forget sets, and it has the potential to significantly improve the state-of-the-art in unlearning research.