This paper presents a comprehensive study on code summarization in the era of large language models (LLMs). The study investigates the effectiveness of various prompting techniques, model settings, and programming languages in code summarization tasks. The research evaluates the quality of summaries generated by LLMs using multiple automated evaluation methods, including text and semantic similarity metrics, and compares them with human evaluations. The study finds that GPT-4-based evaluation methods have the strongest correlation with human evaluations. It also reveals that advanced prompting techniques may not always outperform simple zero-shot prompting. The study further explores the impact of model settings such as top_p and temperature on the quality of generated summaries, finding that their effects vary depending on the base LLM and programming language. Additionally, the study shows that LLMs perform suboptimally when summarizing code written in logic programming languages compared to other language types. The study also investigates the ability of LLMs to generate summaries of different categories, such as What, Why, How-to-use-it, How-it-is-done, Property, and Others, and finds that different LLMs excel in different categories. The study concludes that CodeLlama-Instruct with 7B parameters can outperform advanced GPT-4 in generating summaries describing code implementation details and asserting code properties. The study also makes its dataset and source code publicly available for further research. The findings of this study provide a comprehensive understanding of code summarization in the era of LLMs and can assist researchers in designing advanced LLM-based code summarization techniques for specific fields.This paper presents a comprehensive study on code summarization in the era of large language models (LLMs). The study investigates the effectiveness of various prompting techniques, model settings, and programming languages in code summarization tasks. The research evaluates the quality of summaries generated by LLMs using multiple automated evaluation methods, including text and semantic similarity metrics, and compares them with human evaluations. The study finds that GPT-4-based evaluation methods have the strongest correlation with human evaluations. It also reveals that advanced prompting techniques may not always outperform simple zero-shot prompting. The study further explores the impact of model settings such as top_p and temperature on the quality of generated summaries, finding that their effects vary depending on the base LLM and programming language. Additionally, the study shows that LLMs perform suboptimally when summarizing code written in logic programming languages compared to other language types. The study also investigates the ability of LLMs to generate summaries of different categories, such as What, Why, How-to-use-it, How-it-is-done, Property, and Others, and finds that different LLMs excel in different categories. The study concludes that CodeLlama-Instruct with 7B parameters can outperform advanced GPT-4 in generating summaries describing code implementation details and asserting code properties. The study also makes its dataset and source code publicly available for further research. The findings of this study provide a comprehensive understanding of code summarization in the era of LLMs and can assist researchers in designing advanced LLM-based code summarization techniques for specific fields.