2024 | Mubashara Akhtar, Omar Benjelloun, Costanza Conforti, Pieter Gijsbers, Joan Giner-Miguelez, Nitisha Jain, Michael Kuchnik, Quentin Lhoest, Pierre Marcenac, Manil Maskey, Peter Mattson, Luis Oala, Pierre Ruysen, Rajat Shinde, Elena Simperl, Geffry Thomas, Slava Tykhonov, Joaquin Vanschoren, Jos van der Velde, Steffen Vogler, Carole-Jean Wu
Croissant is a metadata format designed to make datasets "ML-ready" by simplifying their use in machine learning (ML) tools and frameworks. It enhances dataset discoverability, portability, and interoperability, addressing key challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, including Hugging Face, Kaggle, and OpenML, with over 400,000 datasets available in the format. The format is open-source and developed through an open community process as part of ML Commons.
Croissant is organized into four layers: Dataset Metadata, Resources, Structure, and Semantic. The Dataset Metadata layer contains general information about the dataset, such as its name, description, and license. The Resources layer describes the source data included in the dataset, using concepts like FileObject and FileSet. The Structure layer describes and organizes the structure of the resources, using RecordSets to represent data as a set of records. The Semantic layer applies ML-specific data interpretations, including custom data types and dataset organization methods.
Croissant supports Responsible AI (RAI) via dataset documentation, in line with existing RAI initiatives. It provides a dedicated RAI extension to cover key use cases such as data lifecycle, labeling, safety, fairness, traceability, regulatory compliance, and inclusion. Croissant also supports semantic typing of Fields and RecordSets, linking data to known vocabularies and identifiers. This enables ML tools to describe important aspects of datasets, such as splits for test, training, and validation, as well as label information.
Croissant has been integrated into three dataset repositories: Hugging Face, Kaggle, and OpenML. It is also supported by Google Dataset Search, allowing users to search for Croissant datasets across data repositories and the web. The Croissant Editor is a tool that lets users visually create and modify Croissant datasets, providing form-based editing and validation. The editor also integrates with the Croissant Responsible AI extension, guiding users in describing RAI aspects of their datasets.
Croissant is expected to evolve based on user feedback and emerging needs in the rapidly evolving field of machine learning. The format is designed to be adaptable and relevant in various applications, and it may benefit other fields given the broad range of datasets it can represent. Additional features required by specific domains may be developed as Croissant extensions, similar to the one developed for Responsible AI.Croissant is a metadata format designed to make datasets "ML-ready" by simplifying their use in machine learning (ML) tools and frameworks. It enhances dataset discoverability, portability, and interoperability, addressing key challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, including Hugging Face, Kaggle, and OpenML, with over 400,000 datasets available in the format. The format is open-source and developed through an open community process as part of ML Commons.
Croissant is organized into four layers: Dataset Metadata, Resources, Structure, and Semantic. The Dataset Metadata layer contains general information about the dataset, such as its name, description, and license. The Resources layer describes the source data included in the dataset, using concepts like FileObject and FileSet. The Structure layer describes and organizes the structure of the resources, using RecordSets to represent data as a set of records. The Semantic layer applies ML-specific data interpretations, including custom data types and dataset organization methods.
Croissant supports Responsible AI (RAI) via dataset documentation, in line with existing RAI initiatives. It provides a dedicated RAI extension to cover key use cases such as data lifecycle, labeling, safety, fairness, traceability, regulatory compliance, and inclusion. Croissant also supports semantic typing of Fields and RecordSets, linking data to known vocabularies and identifiers. This enables ML tools to describe important aspects of datasets, such as splits for test, training, and validation, as well as label information.
Croissant has been integrated into three dataset repositories: Hugging Face, Kaggle, and OpenML. It is also supported by Google Dataset Search, allowing users to search for Croissant datasets across data repositories and the web. The Croissant Editor is a tool that lets users visually create and modify Croissant datasets, providing form-based editing and validation. The editor also integrates with the Croissant Responsible AI extension, guiding users in describing RAI aspects of their datasets.
Croissant is expected to evolve based on user feedback and emerging needs in the rapidly evolving field of machine learning. The format is designed to be adaptable and relevant in various applications, and it may benefit other fields given the broad range of datasets it can represent. Additional features required by specific domains may be developed as Croissant extensions, similar to the one developed for Responsible AI.