This paper explores the evolving landscape of interpretable machine learning (IML) in the context of large language models (LLMs). While IML has gained traction due to advancements in data and neural networks, LLMs offer new opportunities for interpretability through their natural language generation capabilities. However, they also present challenges such as hallucinated explanations and high computational costs. The paper reviews existing methods for evaluating LLM interpretation, emphasizing the potential of LLMs to redefine interpretability with a broader scope, including auditing LLMs themselves. Two emerging research priorities are highlighted: using LLMs to analyze new datasets and generate interactive explanations. LLMs can provide more elaborate explanations than traditional IML techniques, communicating in natural language to address complex patterns. This allows for targeted queries and immediate, relevant responses. However, challenges such as hallucination and the opacity of LLMs must be addressed. The paper discusses techniques for explaining LLMs, including local explanations (e.g., feature attributions, attention mechanisms) and global explanations (e.g., mechanistic interpretations, probing). It also explores methods for explaining datasets, including using LLMs to analyze tabular and text data, and generating natural language explanations. The paper highlights the potential of LLMs in dataset explanation, enabling data analysis, knowledge discovery, and scientific applications. It also discusses the importance of interactive explanations, which allow users to engage with models through dialogues and follow-up questions. The paper concludes that the integration of LLMs into interpretative processes represents a transformative shift in IML, with future research focusing on improving explanation reliability, advancing dataset interpretation, and developing more user-centric, interactive explanations. The ultimate goal is to enable LLMs to provide reliable, complex explanations that enhance understanding and trust in AI systems.This paper explores the evolving landscape of interpretable machine learning (IML) in the context of large language models (LLMs). While IML has gained traction due to advancements in data and neural networks, LLMs offer new opportunities for interpretability through their natural language generation capabilities. However, they also present challenges such as hallucinated explanations and high computational costs. The paper reviews existing methods for evaluating LLM interpretation, emphasizing the potential of LLMs to redefine interpretability with a broader scope, including auditing LLMs themselves. Two emerging research priorities are highlighted: using LLMs to analyze new datasets and generate interactive explanations. LLMs can provide more elaborate explanations than traditional IML techniques, communicating in natural language to address complex patterns. This allows for targeted queries and immediate, relevant responses. However, challenges such as hallucination and the opacity of LLMs must be addressed. The paper discusses techniques for explaining LLMs, including local explanations (e.g., feature attributions, attention mechanisms) and global explanations (e.g., mechanistic interpretations, probing). It also explores methods for explaining datasets, including using LLMs to analyze tabular and text data, and generating natural language explanations. The paper highlights the potential of LLMs in dataset explanation, enabling data analysis, knowledge discovery, and scientific applications. It also discusses the importance of interactive explanations, which allow users to engage with models through dialogues and follow-up questions. The paper concludes that the integration of LLMs into interpretative processes represents a transformative shift in IML, with future research focusing on improving explanation reliability, advancing dataset interpretation, and developing more user-centric, interactive explanations. The ultimate goal is to enable LLMs to provide reliable, complex explanations that enhance understanding and trust in AI systems.