**MoodCapture: Depression Detection Using In-the-Wild Smartphone Images** **Authors:** Subigya Nepal, Arvind Pillai, Weichen Wang, Tess Griffin, Amanda C. Collins, Michael Heinz, Damien Lekkas, Shayan Mirjafar, Matthew Nemesure, George Price, Nicholas C. Jacobson, Andrew T. Campbell **Abstract:** MoodCapture presents a novel approach to assess depression using images automatically captured from the front-facing camera of smartphones in natural, everyday environments. The study collected over 125,000 photos from 177 participants diagnosed with major depressive disorder over 90 days. Images were captured while participants responded to the PHQ-8 depression survey. The analysis explores image attributes such as angle, dominant colors, location, objects, and lighting. A random forest trained with face landmarks effectively classified samples as depressed or non-depressed and predicted raw PHQ-8 scores. Post-hoc analysis includes an ablation study, feature importance analysis, and bias assessment. The study also addresses user concerns about privacy and photo sharing for mental health assessment. **Key Contributions:** - Development of MoodCapture, a passive-sensing mobile app that collects in-the-wild smartphone images. - Analysis of various image characteristics, providing insights into visual properties. - Evaluation of machine and deep learning models for depression detection and PHQ-8 score prediction. - User acceptance study on comfort levels and privacy concerns. **Methodology:** - Participants were recruited and diagnosed with MDD. - The app collected Ecological Momentary Assessments (EMAs) and captured images when participants responded to the PHQ-8. - Image characteristics were analyzed using the BLIP VQA model. - Machine learning and deep learning models were trained for classification and regression tasks. **Results:** - The random forest model trained with 3D face landmarks achieved balanced accuracy of 0.60, MCC of 0.14, and MAE of 130.31. - Ablation study identified the importance of specific OpenFace feature sets. - Ethical considerations and user acceptance were addressed, ensuring privacy and transparency. **Conclusion:** MoodCapture demonstrates the feasibility of using in-the-wild smartphone images for depression detection, providing valuable insights for future research and tool development.**MoodCapture: Depression Detection Using In-the-Wild Smartphone Images** **Authors:** Subigya Nepal, Arvind Pillai, Weichen Wang, Tess Griffin, Amanda C. Collins, Michael Heinz, Damien Lekkas, Shayan Mirjafar, Matthew Nemesure, George Price, Nicholas C. Jacobson, Andrew T. Campbell **Abstract:** MoodCapture presents a novel approach to assess depression using images automatically captured from the front-facing camera of smartphones in natural, everyday environments. The study collected over 125,000 photos from 177 participants diagnosed with major depressive disorder over 90 days. Images were captured while participants responded to the PHQ-8 depression survey. The analysis explores image attributes such as angle, dominant colors, location, objects, and lighting. A random forest trained with face landmarks effectively classified samples as depressed or non-depressed and predicted raw PHQ-8 scores. Post-hoc analysis includes an ablation study, feature importance analysis, and bias assessment. The study also addresses user concerns about privacy and photo sharing for mental health assessment. **Key Contributions:** - Development of MoodCapture, a passive-sensing mobile app that collects in-the-wild smartphone images. - Analysis of various image characteristics, providing insights into visual properties. - Evaluation of machine and deep learning models for depression detection and PHQ-8 score prediction. - User acceptance study on comfort levels and privacy concerns. **Methodology:** - Participants were recruited and diagnosed with MDD. - The app collected Ecological Momentary Assessments (EMAs) and captured images when participants responded to the PHQ-8. - Image characteristics were analyzed using the BLIP VQA model. - Machine learning and deep learning models were trained for classification and regression tasks. **Results:** - The random forest model trained with 3D face landmarks achieved balanced accuracy of 0.60, MCC of 0.14, and MAE of 130.31. - Ablation study identified the importance of specific OpenFace feature sets. - Ethical considerations and user acceptance were addressed, ensuring privacy and transparency. **Conclusion:** MoodCapture demonstrates the feasibility of using in-the-wild smartphone images for depression detection, providing valuable insights for future research and tool development.