LANGBRIDGE is a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. It bridges two models: one specialized in understanding multiple languages (e.g., mT5 encoder) and one specialized in reasoning (e.g., Orca 2). By introducing minimal trainable parameters between them, LANGBRIDGE enhances the performance of language models on low-resource languages across mathematical reasoning, code completion, logical reasoning, and commonsense reasoning. The effectiveness of LANGBRIDGE stems from the language-agnostic characteristics of multilingual representations. The method leverages the encoder from mT5 and introduces a small number of trainable parameters between the encoder and the target LM. LANGBRIDGE does not require multilingual supervision and solely relies on English data while generalizing to multiple languages during test time, resembling zero-shot cross-lingual transfer. The paper demonstrates the effectiveness of LANGBRIDGE by applying it to LMs specialized in diverse reasoning tasks. Empirical results show that LANGBRIDGE substantially enhances the multilingual reasoning performance of LMs. For example, applying LANGBRIDGE to MetaMath-13B with mT5-XL encoder boosts the average accuracy on MGSM from 40.5% to 53.5%, matching the performance of PaLM-540B. LANGBRIDGE also significantly boosts LM performance on reasoning datasets that require intrinsic linguistic understanding. The hypothesis is that the effectiveness of LANGBRIDGE is anchored in the language-agnostic characteristics of multilingual representations. The paper also discusses related work, including English-centric language models, zero-shot cross-lingual transfer, and aligning pretrained representations. The method is evaluated on four task categories: mathematical reasoning, code completion, logical reasoning, and commonsense reasoning. The results show that LANGBRIDGE models outperform similar-sized multilingual models, establishing LANGBRIDGE as a viable approach for developing mathematical reasoning models for low-resource languages. The paper also discusses limitations, ethical considerations, and future research directions.LANGBRIDGE is a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. It bridges two models: one specialized in understanding multiple languages (e.g., mT5 encoder) and one specialized in reasoning (e.g., Orca 2). By introducing minimal trainable parameters between them, LANGBRIDGE enhances the performance of language models on low-resource languages across mathematical reasoning, code completion, logical reasoning, and commonsense reasoning. The effectiveness of LANGBRIDGE stems from the language-agnostic characteristics of multilingual representations. The method leverages the encoder from mT5 and introduces a small number of trainable parameters between the encoder and the target LM. LANGBRIDGE does not require multilingual supervision and solely relies on English data while generalizing to multiple languages during test time, resembling zero-shot cross-lingual transfer. The paper demonstrates the effectiveness of LANGBRIDGE by applying it to LMs specialized in diverse reasoning tasks. Empirical results show that LANGBRIDGE substantially enhances the multilingual reasoning performance of LMs. For example, applying LANGBRIDGE to MetaMath-13B with mT5-XL encoder boosts the average accuracy on MGSM from 40.5% to 53.5%, matching the performance of PaLM-540B. LANGBRIDGE also significantly boosts LM performance on reasoning datasets that require intrinsic linguistic understanding. The hypothesis is that the effectiveness of LANGBRIDGE is anchored in the language-agnostic characteristics of multilingual representations. The paper also discusses related work, including English-centric language models, zero-shot cross-lingual transfer, and aligning pretrained representations. The method is evaluated on four task categories: mathematical reasoning, code completion, logical reasoning, and commonsense reasoning. The results show that LANGBRIDGE models outperform similar-sized multilingual models, establishing LANGBRIDGE as a viable approach for developing mathematical reasoning models for low-resource languages. The paper also discusses limitations, ethical considerations, and future research directions.