This paper presents a method for evaluating multiple feature spaces while tracking and for adjusting the set of features used to improve tracking performance. The hypothesis is that the features that best discriminate between object and background are also best for tracking. An on-line feature ranking mechanism based on the two-class variance ratio measure is developed, applied to log likelihood values computed from empirical distributions of object and background pixels with respect to a given feature. This mechanism is embedded in a tracking system that adaptively selects the top-ranked discriminative features for tracking. Examples illustrate how the method adapts to changing appearances of both tracked object and scene background. The paper discusses the importance of feature selection in tracking, noting that most tracking applications use a fixed set of features. However, the ability to distinguish between object and background is crucial, and both foreground and background appearances change as the target moves. Therefore, tracking features must adapt. The paper introduces an on-line, adaptive feature selection method that continuously evaluates and updates the set of features used for tracking. The method uses a two-class variance ratio measure to rank features based on their discriminative power. The best features are used to label pixels in a new video frame with the likelihood that they correspond to either object or background. Discriminative features produce likelihood maps where object pixels have high values, and background pixels have low values. The mean-shift algorithm is used to find the nearest local mode of this likelihood surface, estimating the 2D location of the object in the image. The paper presents experiments showing the effectiveness of the method in tracking a car through varying lighting conditions and a flag in changing background conditions. The results demonstrate that the method successfully adapts to changing appearances of the object and background, leading to improved tracking performance. The method is shown to outperform traditional tracking methods in handling challenging scenarios, such as when the color of the background is similar to the object. The paper concludes that the method provides a robust and adaptive approach to tracking, allowing the features used for tracking and the appearance models of object and background to co-evolve over time.This paper presents a method for evaluating multiple feature spaces while tracking and for adjusting the set of features used to improve tracking performance. The hypothesis is that the features that best discriminate between object and background are also best for tracking. An on-line feature ranking mechanism based on the two-class variance ratio measure is developed, applied to log likelihood values computed from empirical distributions of object and background pixels with respect to a given feature. This mechanism is embedded in a tracking system that adaptively selects the top-ranked discriminative features for tracking. Examples illustrate how the method adapts to changing appearances of both tracked object and scene background. The paper discusses the importance of feature selection in tracking, noting that most tracking applications use a fixed set of features. However, the ability to distinguish between object and background is crucial, and both foreground and background appearances change as the target moves. Therefore, tracking features must adapt. The paper introduces an on-line, adaptive feature selection method that continuously evaluates and updates the set of features used for tracking. The method uses a two-class variance ratio measure to rank features based on their discriminative power. The best features are used to label pixels in a new video frame with the likelihood that they correspond to either object or background. Discriminative features produce likelihood maps where object pixels have high values, and background pixels have low values. The mean-shift algorithm is used to find the nearest local mode of this likelihood surface, estimating the 2D location of the object in the image. The paper presents experiments showing the effectiveness of the method in tracking a car through varying lighting conditions and a flag in changing background conditions. The results demonstrate that the method successfully adapts to changing appearances of the object and background, leading to improved tracking performance. The method is shown to outperform traditional tracking methods in handling challenging scenarios, such as when the color of the background is similar to the object. The paper concludes that the method provides a robust and adaptive approach to tracking, allowing the features used for tracking and the appearance models of object and background to co-evolve over time.