This paper presents a novel method for recovering 3D non-rigid shape models from 2D image sequences. The approach is based on a non-rigid model where the 3D shape in each frame is a linear combination of a set of basis shapes. The tracking matrix is of higher rank and can be factored into three steps to yield pose, configuration, and shape. The method is demonstrated on video sequences of people and animals, allowing high accuracy recovery of 3D non-rigid facial models. The paper discusses previous work on structure-from-motion techniques, which typically assume rigid objects. These methods are limited to rigid objects and cannot be applied to non-rigid deforming objects. The proposed method uses scaled orthographic projection and factorization to recover 3D non-rigid shape models. The 3D shape in each frame is a linear combination of a set of basis shapes, and the 2D tracking matrix is of rank 3K, which can be factored into 3D pose, object configuration, and 3D basis shapes using SVD. The algorithm is described in three steps: basis shape factorization, factoring pose from configuration, and adjusting pose and shape. The method is tested on several video sequences, including human faces and animals. The results show that the method can accurately recover 3D non-rigid shape models, even with limited data. The technique is not limited to facial animation and can be applied to other domains. The paper also discusses future work, including handling occluded feature tracks and extending the technique to track longer sequences with more view angles. The results on the three video databases are very encouraging, showing the potential of the method for various applications.This paper presents a novel method for recovering 3D non-rigid shape models from 2D image sequences. The approach is based on a non-rigid model where the 3D shape in each frame is a linear combination of a set of basis shapes. The tracking matrix is of higher rank and can be factored into three steps to yield pose, configuration, and shape. The method is demonstrated on video sequences of people and animals, allowing high accuracy recovery of 3D non-rigid facial models. The paper discusses previous work on structure-from-motion techniques, which typically assume rigid objects. These methods are limited to rigid objects and cannot be applied to non-rigid deforming objects. The proposed method uses scaled orthographic projection and factorization to recover 3D non-rigid shape models. The 3D shape in each frame is a linear combination of a set of basis shapes, and the 2D tracking matrix is of rank 3K, which can be factored into 3D pose, object configuration, and 3D basis shapes using SVD. The algorithm is described in three steps: basis shape factorization, factoring pose from configuration, and adjusting pose and shape. The method is tested on several video sequences, including human faces and animals. The results show that the method can accurately recover 3D non-rigid shape models, even with limited data. The technique is not limited to facial animation and can be applied to other domains. The paper also discusses future work, including handling occluded feature tracks and extending the technique to track longer sequences with more view angles. The results on the three video databases are very encouraging, showing the potential of the method for various applications.