Abstract:
Depression is a highly prevalent mental health disorder, and its diagnosis still relies mainly on self-report questionnaires and clinical interviews. The purpose of this work is to provide a multimodal system for automatic depression detection from spontaneous speech, combining acoustic, linguistic and emotional representations through a late-fusion meta-learning strategy. Acoustic embeddings are extracted with Whisper and linguistic embeddings with XLM-RoBERTa, while affective information is obtained from an Italian fine-tuned wav2vec 2.0 model and from FEEL-IT applied to the transcribed text. Emotion probabilities are combined with the base-classifier outputs into a feature vector fed into a meta-model (logistic regression or XGBoost). The system is evaluated on the Androids Corpus using patient-level stratified cross-validation, and the best configuration achieves an F1-score of 0.932 and an AUC-ROC of 0.980. A SHAP-based interpretability analysis further shows that emotional features contribute to refining the model’s decisions alongside the linguistic probability, consistent with clinical evidence linking depression to impaired emotional regulation. These results support the hypothesis that affective information improves speech-based depression detection and provides an explainable pathway for clinical decision support.