27 Jun 2024 | Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, GP Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachin Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Meza Soria, Dheeraj Sreedhar, Praveen Venkateswaran, Merve Unuvar, David Cox, Salim Roukos, Luis Lastras, Pavan Kapanipathi
The Granite-Function Calling Model introduces function calling abilities to large language models (LLMs) through multi-task learning on granular tasks. The model, GRANITE-20B-FUNCTIONCALLING, is trained on seven fundamental tasks related to function calling, including nested function calling, function chaining, parallel functions, function name detection, parameter-value pair detection, next-best function, and response generation. It is trained using a multi-task approach on diverse datasets such as API-BLEND and Glaive-V2, and is released under an Apache 2.0 license. The model is evaluated on multiple out-of-domain datasets and benchmarks, showing strong performance compared to other open and proprietary models. GRANITE-20B-FUNCTIONCALLING achieves the best performance among open models on the Berkeley Function Calling Leaderboard and fourth overall. It demonstrates good generalizability across various tasks and datasets, outperforming many proprietary models with larger parameters. The model is instruct-tuned using a mixture of training data, and is evaluated on multiple metrics including F1 scores, LCS, exact match, and language generation metrics like BERTScore, ROUGE-L, and BLEU. The model also shows strong performance in response generation tasks, with GRANITE-20B-FUNCTIONCALLING coming close to the top-performing model, Meta-Llama-3-70B-Instruct, in several metrics. The model is designed to handle complex tasks by leveraging function calling and is capable of performing well in both open and proprietary settings.The Granite-Function Calling Model introduces function calling abilities to large language models (LLMs) through multi-task learning on granular tasks. The model, GRANITE-20B-FUNCTIONCALLING, is trained on seven fundamental tasks related to function calling, including nested function calling, function chaining, parallel functions, function name detection, parameter-value pair detection, next-best function, and response generation. It is trained using a multi-task approach on diverse datasets such as API-BLEND and Glaive-V2, and is released under an Apache 2.0 license. The model is evaluated on multiple out-of-domain datasets and benchmarks, showing strong performance compared to other open and proprietary models. GRANITE-20B-FUNCTIONCALLING achieves the best performance among open models on the Berkeley Function Calling Leaderboard and fourth overall. It demonstrates good generalizability across various tasks and datasets, outperforming many proprietary models with larger parameters. The model is instruct-tuned using a mixture of training data, and is evaluated on multiple metrics including F1 scores, LCS, exact match, and language generation metrics like BERTScore, ROUGE-L, and BLEU. The model also shows strong performance in response generation tasks, with GRANITE-20B-FUNCTIONCALLING coming close to the top-performing model, Meta-Llama-3-70B-Instruct, in several metrics. The model is designed to handle complex tasks by leveraging function calling and is capable of performing well in both open and proprietary settings.