The Multi-Class Classification of diseases on thoracic area using Chest X-Ray (CXR) Images is difficult owing to visual similarities between the different diseases such as Tuberculosis (TB), COVID-19, and Pneumonia. In the existing studies, the use of End-to-end Deep Learning Networks has been made for classifying the different diseases. Such an approach would restrict the extraction of discriminatory features between the classes. In order to overcome this limitation, in this paper, MEDXPERT (Graph-Guided Collaborative Multi-Expert Framework with Adaptive Expert Memory) for explainable multi-class chest X-ray diagnosis is proposed. The concept in MEDXPERT lies in the fact that adaptive disease routing is used to select the right experts on the basis of input images. Experts learn discriminative disease features whereas the adaptive expert memory evolves disease prototypes. Graph-guided collaboration helps in modeling the relationship between disease classes and knowledge transfer among experts prior to prediction. Uncertainty estimation helps in identifying low confidence predictions while Explainability provides visual attention maps and diagnostic evidence. The evaluation of MEDXPERT is done on the publicly available Tuberculosis-tb-chest-xray-dataset, which consists of CXR images from the categories of tuberculosis, COVID-19, pneumonia, and normal. A dataset having 10,000 CXR images with a balanced number of images in each category, where 2,500 images are selected from each class and 8,000 images are used for training while the remaining 2,000 images are for testing. The performance of MEDXPERT is assessed in terms of Accuracy, Precision, Recall, F1-score, AUC, and calibration metrics, where ResNet50, DenseNet121, EfficientNet-B3, Vision Transformer, and Swin Transformer have been taken into consideration as baseline approaches. MEDXPERT obtains an Accuracy of 97.35%, Precision of 97.35%, Recall of 97.35%, F1-score of 97.34%, and AUC of 99.89%.