Pali: Parallel aligned logits integration for robust multimodal sentiment analysis under real-world missing modality scenarios

Abstract

Multimodal sentiment analysis has gained significant attention due to its ability to integrate multiple modalities for richer understanding of human emotions. However, existing approaches often fail when one or more modalities are missing and this leads to degraded performance. Real-world applications often suffer from missing modalities due to sensor failures, network issues, high data costs or data corruption. Existing approaches treat robustness as a fusion problem. This paper reframed the challenge as a representation learning problem and proposes a new methodological framework called Parallel Aligned Logits Integration (PALI). Unlike traditional fusion or imputation methods, PALI operates in the logit space by training modality-specific encoders in parallel and aligning their output distributions through a contrastive alignment loss. During inference, missing modalities are handled by a dynamic logit weighting mechanism that adaptively scales contributions based on estimated modality reliability. PALI is presented both as a model‑output integration strategy (named PALI strategy) and as a comprehensive methodological framework that applies this strategy (named PALI methodology) to combine model outputs for robust multimodal sentiment analysis. Empirical evaluation shows that PALI outperforms several popular fusion baselines on an “MVSA single” benchmark dataset in both complete-modality and missing-modality settings by achieving an accuracy of 82.11% 82.11%. This accuracy rate represents a substantial improvement over existing state-of-the-art models, whose performance ranges between 73.77% and 75.78%. An improvement of more than six percentage points over the best performing benchmark model (MSACA at 75.78%) is practically significant. The contribution of this paper is  methodological. It offers a modular and interpretable integration framework for robust multimodal sentiment analysis.