This study proposes a high-security learning-based optical encryption system using a spin-multiplexing disordered metasurface (DM). The system employs a DM as a scattering medium, which provides multiple polarization encryption channels and enhances security through a double-secure procedure. The DM, designed with titanium dioxide meta-pillars on a fused silica substrate, offers superior stability and a large memory effect, making it less sensitive to environmental perturbations compared to conventional scattering media (CSM). The system uses a double-secure approach where the plaintext is encrypted with a security key (QR code) and then transmitted through the DM to generate a ciphertext (speckle pattern). The decryption process involves a neural network that requires both the ciphertext and the correct security key with matched polarization to recover the plaintext. The DM supports multiple polarization channels, allowing for enhanced information capacity and security. The system demonstrates excellent decryption efficiency even in noisy environments and can recover to its initial state after perturbations, thanks to its large memory effect. The proposed system is robust and practical, with the ability to handle new data without additional training. The system's stability and security are validated through extensive testing, showing that it outperforms conventional CSM-based systems in terms of long-term performance and resistance to attacks. The use of a DM enables a versatile platform for speckle-based optical encryption with potential applications in various real-world scenarios.This study proposes a high-security learning-based optical encryption system using a spin-multiplexing disordered metasurface (DM). The system employs a DM as a scattering medium, which provides multiple polarization encryption channels and enhances security through a double-secure procedure. The DM, designed with titanium dioxide meta-pillars on a fused silica substrate, offers superior stability and a large memory effect, making it less sensitive to environmental perturbations compared to conventional scattering media (CSM). The system uses a double-secure approach where the plaintext is encrypted with a security key (QR code) and then transmitted through the DM to generate a ciphertext (speckle pattern). The decryption process involves a neural network that requires both the ciphertext and the correct security key with matched polarization to recover the plaintext. The DM supports multiple polarization channels, allowing for enhanced information capacity and security. The system demonstrates excellent decryption efficiency even in noisy environments and can recover to its initial state after perturbations, thanks to its large memory effect. The proposed system is robust and practical, with the ability to handle new data without additional training. The system's stability and security are validated through extensive testing, showing that it outperforms conventional CSM-based systems in terms of long-term performance and resistance to attacks. The use of a DM enables a versatile platform for speckle-based optical encryption with potential applications in various real-world scenarios.