The identification of olive tree cultivars is essential to support agricultural sustainability, improve biodiversity conservation, and certify the traceability and the quality of olive oil. This is particularly important in regions such as Trás-os-Montes e Alto Douro in Portugal, where “Cobrançosa”, “Madural”, and “Verdeal de Trás-os-Montes” are predominant olive cultivars, being crucial in the agricultural economy, sustainability, and cultural heritage. As the world’s seventh largest producer of olive oil, Portugal depends on its olive-growing regions, with this region ranking as the second most important contributor to national production. Traditional methods for identifying olive cultivars rely on cataloging and using germplasm collections of cultivars and accessions, achieved through the application of both morphological and molecular markers. Nevertheless, these methods are laborious and time-consuming, limiting their scalability. Spectral reflectance analysis, using spectroscopic instruments to measure light interactions with plant surfaces at a specific wavelength, presents an efficient and reliable method for cultivar identification. In this study, spectral reflectance data from the leaves of “Cobrançosa”, “Madural”, and “Verdeal de Trás-os-Montes” olive tree cultivars were collected using a spectroradiometer (500–900 nm). To address the high dimensionality of the dataset, principal component analysis (PCA) was applied to retain essential information while reducing complexity, creating a dataset with 50 features and 432 samples (144 for each cultivar). Different machine learning algorithms—eXtreme Gradient Boosting (XGBoost), Random Forest (RF), Support Vector Classifier (SVC) and Decision Tree (DT)—were then trained to classify the olive tree cultivars, with XGBoost achieving the highest classification accuracy of 93.1%, while DT showed the lowest accuracy of 80%. The results also revealed a variation in the effectiveness of differentiating the three olive tree cultivars. The cultivar “Madural” demonstrated the highest F1-score (0.953), indicating clear spectral distinction from the others. In contrast, the cultivar “Cobrançosa” showed the least differentiation (F1-score of 0.918) due to a greater spectral overlap with the cultivar “Verdeal de Trás-os-Montes”. Thus, the results suggest the need for further refinement of the approach to address higher intra-class variability. This refinement would be particularly advantageous when incorporating new cultivars with spectral signatures that may increase interclass similarity. This study shows the potential of spectral reflectance and machine learning for precision agriculture. The results provide a methodology, even in preliminary stages, for applying spectroscopy to sustainable crop management. Future research could expand the dataset to include additional cultivars while exploring advanced machine learning techniques to further improve classification performance.