This paper introduces a novel approach to document chunking for Retrieval Augmented Generation (RAG) that focuses on structural elements of documents rather than just paragraph-level chunking. The proposed method enhances the accuracy and relevance of information retrieval by leveraging document structure, such as headings, paragraphs, and tables, to create more contextually relevant chunks. This approach is particularly effective for financial reporting, where documents have complex structures and contain a variety of tabular information. The study evaluates various chunking strategies, including basic token-based chunking and element-based chunking, using the FinanceBench dataset. Results show that element-based chunking significantly improves retrieval accuracy and question-answering performance compared to traditional methods. The method also offers the advantage of being generalizable across different document types without requiring hyperparameter tuning. The study highlights the importance of considering document structure in chunking strategies to enhance the effectiveness of RAG systems in financial domains. The findings suggest that element-based chunking is more efficient and effective, reducing indexing costs and improving query latency. The research contributes to the field of document understanding and RAG by providing a systematic approach to chunking that enhances the accuracy and relevance of information retrieval.This paper introduces a novel approach to document chunking for Retrieval Augmented Generation (RAG) that focuses on structural elements of documents rather than just paragraph-level chunking. The proposed method enhances the accuracy and relevance of information retrieval by leveraging document structure, such as headings, paragraphs, and tables, to create more contextually relevant chunks. This approach is particularly effective for financial reporting, where documents have complex structures and contain a variety of tabular information. The study evaluates various chunking strategies, including basic token-based chunking and element-based chunking, using the FinanceBench dataset. Results show that element-based chunking significantly improves retrieval accuracy and question-answering performance compared to traditional methods. The method also offers the advantage of being generalizable across different document types without requiring hyperparameter tuning. The study highlights the importance of considering document structure in chunking strategies to enhance the effectiveness of RAG systems in financial domains. The findings suggest that element-based chunking is more efficient and effective, reducing indexing costs and improving query latency. The research contributes to the field of document understanding and RAG by providing a systematic approach to chunking that enhances the accuracy and relevance of information retrieval.