The paper introduces DEEPEN, a training-free ensemble framework for heterogeneous large language models (LLMs). DEEPEN addresses the challenge of vocabulary discrepancy between different LLMs by mapping their probability distributions to a universal relative space using relative representation theory. This approach allows for the aggregation of informative probability distributions from various LLMs at each decoding step. The aggregated result is then transformed back to the probability space of one of the ensembled LLMs to determine the next token. Extensive experiments on six benchmarks demonstrate that DEEPEN achieves consistent improvements over individual models and complements other ensemble methods such as voting. The method also shows stability and complementary strengths, making it a promising approach for LLM collaboration and generalization to unseen data distributions.The paper introduces DEEPEN, a training-free ensemble framework for heterogeneous large language models (LLMs). DEEPEN addresses the challenge of vocabulary discrepancy between different LLMs by mapping their probability distributions to a universal relative space using relative representation theory. This approach allows for the aggregation of informative probability distributions from various LLMs at each decoding step. The aggregated result is then transformed back to the probability space of one of the ensembled LLMs to determine the next token. Extensive experiments on six benchmarks demonstrate that DEEPEN achieves consistent improvements over individual models and complements other ensemble methods such as voting. The method also shows stability and complementary strengths, making it a promising approach for LLM collaboration and generalization to unseen data distributions.