This paper introduces information-based objective functions for active data selection in Bayesian learning. The goal is to select data points that maximize the expected informativeness for learning. Three different criteria for data selection are derived based on different objectives: maximizing information about model parameters, predicting values in a specific region, and distinguishing between models. These criteria depend on the assumption that the hypothesis space is correct, which may be a weakness. The paper discusses how to select data points for interpolation models, using a Bayesian framework. It shows that the expected information gain can be measured using entropy changes and that two information measures, entropy change and cross-entropy, are equivalent in expectation. The first task is to select data points where the error bars on the interpolant are largest, which aligns with the D-optimal and minimax design criteria. The second task involves maximizing information about the interpolant in a specific region. This is achieved by measuring at points where the sensitivity of the interpolant is highest. The third task focuses on distinguishing between models, where the most informative data points are those that maximize the difference in model predictions. The paper also addresses computational complexity, showing that the suggested objective functions are computationally feasible once the inverse Hessian is available. It highlights that these methods may fail if the model is incorrect, as they assume the model is accurate. The paper concludes that while these methods are promising, further research is needed to develop more robust data utility measures.This paper introduces information-based objective functions for active data selection in Bayesian learning. The goal is to select data points that maximize the expected informativeness for learning. Three different criteria for data selection are derived based on different objectives: maximizing information about model parameters, predicting values in a specific region, and distinguishing between models. These criteria depend on the assumption that the hypothesis space is correct, which may be a weakness. The paper discusses how to select data points for interpolation models, using a Bayesian framework. It shows that the expected information gain can be measured using entropy changes and that two information measures, entropy change and cross-entropy, are equivalent in expectation. The first task is to select data points where the error bars on the interpolant are largest, which aligns with the D-optimal and minimax design criteria. The second task involves maximizing information about the interpolant in a specific region. This is achieved by measuring at points where the sensitivity of the interpolant is highest. The third task focuses on distinguishing between models, where the most informative data points are those that maximize the difference in model predictions. The paper also addresses computational complexity, showing that the suggested objective functions are computationally feasible once the inverse Hessian is available. It highlights that these methods may fail if the model is incorrect, as they assume the model is accurate. The paper concludes that while these methods are promising, further research is needed to develop more robust data utility measures.