An Automated Open-Source Pipeline for Near-Real-Time Atmospheric Anomaly Detection and Visualization Using Sentinel-5P TROPOMI Data

The current GIS methods for atmospheric monitoring rely on manual data retrieval and static desktop processing, which causes significant delays in the identification of industrial greenhouse gas emissions. We present a dynamic, reproducible “satellite-to-dashboard” pipeline for the continuous monitoring of methane (CH4) and carbon monoxide (CO). The architecture programmatically ingests daily sentinel-5PTROPOMI Level-2 data from the Copernicus API, filtered by geographic bounding boxes and cloud cover less than 20%. The pipeline applies a pixel-wise Z-score anomaly detection algorithm against historical baselines using xarray and NumPy. We identify pixels where Z>3 as industrial plumes. For web performance, the raw multi-megabyte NetCDF4 files are cleaned and compressed into a lightweight Apache Parquet metadata store, extracting only key spatial, temporal and concentration attributes. The frontend is developed using Streamlit and Plotly to visualize interactive density map overlays and temporal trend charts. This framework takes remote sensing from the retrospective to the operational, automated framework and provides stakeholders with a scalable tool for near-real-time environmental oversight. The automated application of historical pattern and then predicate future will be considered. It explicitly outlines the data ingestion, mathematical filtering (Z-score), data reduction and final visualization layers, making it highly competitive for a technical modeling session. The benefits of such an approach help in data retention, data reduction, storage efficiency and algorithmic innovations.