This paper explores the application of nonlinear embedding algorithms, commonly used in shallow semi-supervised learning techniques, to deep multi-layer architectures. The authors propose three modes for incorporating these embedding algorithms into deep learning: (1) as a regularizer at the output layer, (2) directly on each layer of the architecture, and (3) by creating an auxiliary network that shares the first few layers with the original network. This approach leverages existing ideas from shallow semi-supervised algorithms to improve the performance of deep learning models. The paper demonstrates the effectiveness of this method through experimental evaluations on various datasets, including small-scale and large-scale benchmarks, and a case study on semantic role labeling of English sentences. The results show that the proposed method can achieve competitive error rates compared to both shallow semi-supervised techniques and existing deep learning methods.This paper explores the application of nonlinear embedding algorithms, commonly used in shallow semi-supervised learning techniques, to deep multi-layer architectures. The authors propose three modes for incorporating these embedding algorithms into deep learning: (1) as a regularizer at the output layer, (2) directly on each layer of the architecture, and (3) by creating an auxiliary network that shares the first few layers with the original network. This approach leverages existing ideas from shallow semi-supervised algorithms to improve the performance of deep learning models. The paper demonstrates the effectiveness of this method through experimental evaluations on various datasets, including small-scale and large-scale benchmarks, and a case study on semantic role labeling of English sentences. The results show that the proposed method can achieve competitive error rates compared to both shallow semi-supervised techniques and existing deep learning methods.