TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks

TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks

19 Mar 2024 | Benjamin Feuer, Robin Tibor Schirrmeister, Valeriia Cherepanova, Chinmay Hegde, Frank Hutter, Micah Goldblum, Niv Cohen, Colin White
The paper introduces TuneTables, a novel prompt-tuning technique for prior-data fitted networks (PFNs), which are a recent breakthrough in tabular classification that achieve strong performance on new tasks with a single forward pass. PFNs, particularly TabPFN, have limitations that restrict their widespread adoption, such as being limited to small datasets with a maximum of 1000 training samples, 100 features, and 10 classes. TuneTables overcomes these limitations by compressing large datasets into a smaller learned context, making PFNs competitive with state-of-the-art tabular classification methods on larger datasets. The technique trades increased training time for lower inference time compared to TabPFN. Additionally, TuneTables can be used for multi-objective optimization, such as balancing accuracy and fairness, and as an interpretability tool. The paper includes extensive experiments demonstrating TuneTables' effectiveness on various datasets and its ability to mitigate bias.The paper introduces TuneTables, a novel prompt-tuning technique for prior-data fitted networks (PFNs), which are a recent breakthrough in tabular classification that achieve strong performance on new tasks with a single forward pass. PFNs, particularly TabPFN, have limitations that restrict their widespread adoption, such as being limited to small datasets with a maximum of 1000 training samples, 100 features, and 10 classes. TuneTables overcomes these limitations by compressing large datasets into a smaller learned context, making PFNs competitive with state-of-the-art tabular classification methods on larger datasets. The technique trades increased training time for lower inference time compared to TabPFN. Additionally, TuneTables can be used for multi-objective optimization, such as balancing accuracy and fairness, and as an interpretability tool. The paper includes extensive experiments demonstrating TuneTables' effectiveness on various datasets and its ability to mitigate bias.
Reach us at info@study.space