ADAPTIVE MULTIMODAL LSTM WITH ONLINE LEARNING FOR EVOLVING IOT DATA STREAMS

One of the major challenges in the Internet of Things (IoT) is processing complex data. Data sourced from numerous sensors, cameras, and network logs are continuously evolving and are also multimodal. Having multiple data sources and a dynamic environment gives rise to concept drift. The precision and reliability of Static Machine Learning Models are impacted by concept drift. This is where our framework, Adaptive Multimodal Long Short-Term Memory (AM-LSTM), comes in. It uses a unique technique that combines online learning with smart fusion to ensure accurate and robust implementation in a dynamic system. The system learns relentlessly, adapting to all inputs, and constantly ensures the data stays relevant. To implement this, the following methodologies were adopted: First, dedicated LSTMs help isolate and learn the temporal patterns in each data modality. Secondly, the attention-based fusion mechanism dynamically selects the most appropriate information across these modalities and is tolerant to missing data issues. Thirdly, the concept drift was addressed through the window technique. The method used makes ongoing assessments of the prediction errors, and when a substantial change is observed, a relevant retraining cycle is detected. The AM-LSTM model was assessed using UNSW-NB15 and Edge-IoT dataset benchmarks and was found to function effectively. The model yielded a score of 88.7% and an F1 score of 0.85. It was reactive to concept drift by adapting to changes after just 620 samples, which outperformed all benchmark models. The 47 milliseconds delay in every batch update indicates the performance and robustness of real-time IoT systems.