This paper explores various fine-tuning methods for BERT (Bidirectional Encoder Representations from Transformers) to enhance its performance in text classification tasks. The authors conduct extensive experiments to investigate different strategies, including further pre-training on task-specific or in-domain data, multi-task learning, and layer selection. They propose a general solution for fine-tuning BERT, which involves three steps: further pre-training on task-specific data, optional multi-task learning, and fine-tuning for the target task. The proposed method achieves state-of-the-art results on eight widely-studied text classification datasets, both in English and Chinese. Key findings include the effectiveness of using the top layer of BERT, the benefits of appropriate layer-wise learning rates to overcome catastrophic forgetting, and the advantages of within-task and in-domain pre-training. The paper also demonstrates that BERT can improve performance on small datasets and provides insights into how BERT works in text classification tasks.This paper explores various fine-tuning methods for BERT (Bidirectional Encoder Representations from Transformers) to enhance its performance in text classification tasks. The authors conduct extensive experiments to investigate different strategies, including further pre-training on task-specific or in-domain data, multi-task learning, and layer selection. They propose a general solution for fine-tuning BERT, which involves three steps: further pre-training on task-specific data, optional multi-task learning, and fine-tuning for the target task. The proposed method achieves state-of-the-art results on eight widely-studied text classification datasets, both in English and Chinese. Key findings include the effectiveness of using the top layer of BERT, the benefits of appropriate layer-wise learning rates to overcome catastrophic forgetting, and the advantages of within-task and in-domain pre-training. The paper also demonstrates that BERT can improve performance on small datasets and provides insights into how BERT works in text classification tasks.