5 Jan 2024 | Gabriel Lino Garcia, Pedro Henrique Paiola, Luis Henrique Morelli, Giovani Candido, Arnaldo Cândido Júnior, Danilo Samuel Jodas, Luis C. S. Afonso, Ivan Rizzo Guilherme, Bruno Elias Penteado, João Paulo Papa
This paper introduces Bode, a fine-tuned large language model (LLM) for Portuguese prompt-based tasks. Bode is based on the LLaMA 2 architecture and is available in two versions: 7B and 13B. The model was fine-tuned using a publicly translated dataset of instruction-following tasks to improve its performance in Portuguese language tasks. The paper evaluates Bode's performance in classification tasks using a zero-shot approach with in-context learning and compares it with other LLMs. The main contribution of this work is to provide an LLM that performs well in Portuguese, as well as a model that is free for research or commercial use.
The paper discusses the theoretical background of LLaMA models, Mistral 7B, Falcon 7B, and Low-Rank Adaptation (LoRA). It also reviews related works, including Sabiá and openCabrita, which are Portuguese LLMs. The proposed model, Bode, is designed to address the challenges of instruction-following tasks in Portuguese by adapting the LLaMA 2 architecture for Portuguese instruction-following responses. The model was trained on a Portuguese dataset and fine-tuned using LoRA with specific hyperparameters.
The methodology involves evaluating Bode's performance in classification tasks using zero-shot and in-context learning approaches. The experiments were conducted on three Portuguese datasets: TweetSentBr, AGNews, and FakeRecogna. The results show that Bode outperforms other LLMs in sentiment analysis, news classification, and fake news detection tasks. The model achieves high accuracy and F1-score values, demonstrating its effectiveness in Portuguese language tasks.
The paper concludes that Bode is a promising model for Portuguese language tasks and has the potential to enhance various NLP applications in Portuguese. The model is available for research and commercial use, and further development is recommended to improve its performance across a range of tasks.This paper introduces Bode, a fine-tuned large language model (LLM) for Portuguese prompt-based tasks. Bode is based on the LLaMA 2 architecture and is available in two versions: 7B and 13B. The model was fine-tuned using a publicly translated dataset of instruction-following tasks to improve its performance in Portuguese language tasks. The paper evaluates Bode's performance in classification tasks using a zero-shot approach with in-context learning and compares it with other LLMs. The main contribution of this work is to provide an LLM that performs well in Portuguese, as well as a model that is free for research or commercial use.
The paper discusses the theoretical background of LLaMA models, Mistral 7B, Falcon 7B, and Low-Rank Adaptation (LoRA). It also reviews related works, including Sabiá and openCabrita, which are Portuguese LLMs. The proposed model, Bode, is designed to address the challenges of instruction-following tasks in Portuguese by adapting the LLaMA 2 architecture for Portuguese instruction-following responses. The model was trained on a Portuguese dataset and fine-tuned using LoRA with specific hyperparameters.
The methodology involves evaluating Bode's performance in classification tasks using zero-shot and in-context learning approaches. The experiments were conducted on three Portuguese datasets: TweetSentBr, AGNews, and FakeRecogna. The results show that Bode outperforms other LLMs in sentiment analysis, news classification, and fake news detection tasks. The model achieves high accuracy and F1-score values, demonstrating its effectiveness in Portuguese language tasks.
The paper concludes that Bode is a promising model for Portuguese language tasks and has the potential to enhance various NLP applications in Portuguese. The model is available for research and commercial use, and further development is recommended to improve its performance across a range of tasks.