Multimodal Sentiment Analysis for Depression Detection on TikTok: Combining Generative AI Annotation, DSM-5 Criteria and Public Health Trends

Modern social media platforms like TikTok have become popular spaces where individuals share feelings and struggles more openly than they might in the physical world. Given that recent studies demonstrate that mental health trends are increasing worldwide, with high prevalence numbers for depression, this original study looks at whether content from social videos can be leveraged to identify depressive. Specifically, publicly available TikTok and Youtube videos (100 in total) were collected and processed to extract features across different modalities. These were integrated into a structured multimodal dataset. Then, we used Generative AI technologies to classify the dataset for depression via the nine DSM-5 symptoms. A subset of the samples was validated by mental health professionals to assess reliability (25% of the total dataset). The dataset was also associated with epidemiological data on depression prevalence to explore broader correlations and support the need for early diagnoses of the studied diseases. The annotation process produced a multimodally-labelled dataset with promising coverage of DSM-5 symptoms across all modalities. The AI-assisted annotation approach proved to be both efficient and reliable, capable of processing data at a scale that would not be feasible through manual review. Overall, this research shows that multimodal analysis of short-form video content, enriched with AI assisted annotation, can serve as a scalable and reliable method for identifying depressive indicators in social media data, contributing a novel dataset and a predictive model to the field of computational mental health, supporting the push for earlier, data-driven interventions and technologies. Initial results indicate a Cohen’s κ = -0.129. We emphasize that the results presented are not based on clinical-level validation nor does it aim to replace professional services. All video data utilized in this study was sourced from publicly accessible platforms, and have been fully anonymized prior to analysis.