This paper proposes a novel framework for labelling problems that combines multiple segmentations in a principled manner. The method is based on higher order conditional random fields (CRFs) and uses potentials defined on image segments generated by unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and generalize the commonly used pairwise contrast sensitive smoothness potentials. The higher order potentials used in the framework are based on the Robust $ P^{n} $ model, which is more general than the $ P^{n} $ Potts model. The optimal swap and expansion moves for energy functions composed of these potentials can be computed by solving a st-mincut problem, enabling the use of powerful graph cut based algorithms for inference. The method is tested on multi-class object segmentation, where higher order potentials defined on image regions improve results significantly. The paper also introduces a new family of higher order potentials, the Robust $ P^{n} $ model, which is a generalization of the $ P^{n} $ Potts model. These potentials are more robust and can handle inaccurate segments. The paper discusses the use of higher order CRFs for object segmentation and recognition, integrating these potentials with conventional unary and pairwise potentials. The framework is tested on challenging data sets, showing improved results in terms of object boundary definition. The paper also discusses the use of graph cuts for minimizing higher order move functions, transforming them into submodular quadratic functions. The method is shown to be efficient and effective for large cliques. The paper concludes that the proposed method can be used to improve many other labelling problems.This paper proposes a novel framework for labelling problems that combines multiple segmentations in a principled manner. The method is based on higher order conditional random fields (CRFs) and uses potentials defined on image segments generated by unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and generalize the commonly used pairwise contrast sensitive smoothness potentials. The higher order potentials used in the framework are based on the Robust $ P^{n} $ model, which is more general than the $ P^{n} $ Potts model. The optimal swap and expansion moves for energy functions composed of these potentials can be computed by solving a st-mincut problem, enabling the use of powerful graph cut based algorithms for inference. The method is tested on multi-class object segmentation, where higher order potentials defined on image regions improve results significantly. The paper also introduces a new family of higher order potentials, the Robust $ P^{n} $ model, which is a generalization of the $ P^{n} $ Potts model. These potentials are more robust and can handle inaccurate segments. The paper discusses the use of higher order CRFs for object segmentation and recognition, integrating these potentials with conventional unary and pairwise potentials. The framework is tested on challenging data sets, showing improved results in terms of object boundary definition. The paper also discusses the use of graph cuts for minimizing higher order move functions, transforming them into submodular quadratic functions. The method is shown to be efficient and effective for large cliques. The paper concludes that the proposed method can be used to improve many other labelling problems.