The detection of fetal brain abnormalities at an early stage has a significant impact with prenatal health care. The brain abnormalities arise a major concern for the lifelong problem in newborns. Early detection of fetal brain abnormalities helps clinicians to give extra care during pregnancy. They can plan the future treatment based on it. They can go for several tests, like fetal MRI to prepare for the appropriate treatment after birth. If these can be detected early and cure the impact these abnormalities can bring when they are grown-up. These abnormalities include cerebral palsy, developmental delays, and cognitive impairments. The existing methods for the detection of the fetal abnormalities at an early stage have less accuracy and are time consuming complex processes. Here we propose, a feasible multisensory framework-based system that helps to detect preliminary fetal brain abnormalities. The system involves sensors like Doppler Ultrasound, Fetal Electroencephalography (fEEG), Near-Infrared Spectroscopy (NIRS) and other imaging systems combined together with the multimodal approach to provide an insight on the brain growth and status. The Doppler Ultrasound sensor is used to identify fetal heart rate patterns, NIRS is used to measures oxygen levels in the brain, helping to detect low oxygen conditions that may harm brain development. fEEG is used to monitor brain activity non-invasively by capturing magnetic signals from the fetal brain by giving high-resolution insights on the neurological function. Other ultrasound imaging techniques are used to detect the physical abnormalities like ventriculomegaly, corpus callosum agenesis, and hydrocephalus. This proposed system uses AI models that work as an ensembled method which comprises of Convolutional Neural Network (CNN) and Long Short-Term Memory network (LSTM) for identifying the structural brain abnormalities. The system has been validated against sourced sample data sets and proved to provide a comparatively higher accuracy and better performance.