The prompt and precise detection of breast cancer, a substantial worldwide health issue, is essential for enhancing patient results. Nevertheless, conventional techniques such as mammography encounter constraints in terms of sensitivity and specificity, resulting in overlooked diagnoses and unwarranted biopsies. Deep learning (DL) is a state-of-the-art technique that can improve breast cancer detection by effectively extracting complex characteristics from medical images and enabling precise categorization. This research introduces a novel framework designed for the nuanced categorization of breast cancer by analyzing histopathology images using DL techniques. The journey begins with the original histopathological images employing a substantial dataset comprising approximately 150,408 image patches of two separate categories based on the presence or absence of invasive ductal carcinoma (IDC), each with dimensions of 50x50 pixels and RGB colour representation. The system underwent rigorous pre-processing steps and employed data augmentation to reduce overfitting. These augmented images serve as the input for the fine-tuned DL models, a repertoire that includes Custom CNN, ResNet50, DenseNet201, and VGG16, all orchestrated for meticulous training, testing, and validation. After carefully analyzing the pre-trained models and custom CNN, we found that the fine-tuned VGG16 model had an exceptional performance, obtaining an accuracy rate of 96%. The proposed approach was subjected to thorough examination, confirming its efficacy and ability to reduce diagnostic errors caused by human factors.