Multimodal Deep Learning for Non-Contact Automated Pain Intensity Estimation in Infants via Remote Optical Sensing

Introduction: Neonatal pain assessment remains a critical challenge in clinical practice, as infants cannot self-report pain, resulting in reliance on subjective observational scales prone to inter-observer variability. Untreated or undertreated pain can lead to adverse long-term neurodevelopmental outcomes, including altered pain sensitivity and cognitive impairments. Non-contact methods leveraging remote optical sensing—analogous to passive remote sensing techniques—provide an objective, continuous monitoring solution using standard cameras, minimizing disturbance to fragile infants in neonatal intensive care units (NICUs). Just as passive remote sensing acquires surface information from a distance without physical contact or active signal emission, our framework captures infant physiological and behavioral cues—facial micro-expressions, skin color variations, and vocal acoustics—non-invasively from ambient-light video streams, establishing a direct methodological bridge between Earth observation paradigms and clinical optical biomarker extraction.

Methods: This study introduces a multimodal deep learning framework for automated pain intensity estimation from remote infant videos. RGB video sequences capture facial expressions non-invasively, fused with complementary modalities such as cry acoustics and physiological signals. Spatio-temporal features are extracted via CNNs for facial dynamics and LSTMs for temporal dependencies, with early, late, or hybrid fusion strategies. The framework is validated on clinical datasets annotated with standardized scales (NFCS, NIPS).

Results: The system achieves over 90% accuracy in binary pain/no-pain classification and strong Pearson correlation (r > 0.85) with clinical scores. Multimodal fusion yields 10–15% performance gains over unimodal baselines, with robustness to variable poses, partial occlusion, and illumination. Real-time operation (30 fps) enables continuous monitoring without dedicated clinical staff.

Conclusions: This AI-enhanced, non-contact approach bridges remote sensing methodologies with healthcare, offering objective, real-time pain monitoring via ubiquitous cameras. It holds potential to mitigate long-term neurodevelopmental risks from unmanaged pain, particularly in resource-limited settings, and paves the way for broader AI applications in remote physiological assessment.