Potato (Solanum tuberosum L.) is one of the most widely produced and consumed crops worldwide and can easily be cultivated in different locations and climates. Given its importance for food and nutritional security, an experimental trial was conducted to evaluate the impact on the phenological development and accumulation of potentially contaminating metals in the vegetative organs of cultivating three varieties (Red Lady, Agria, and Désirée) in substrates containing slag and waste from the Caveira polymetallic sulfide mine, located in Grândola (Beja, Portugal), at proportions of 10:0, 10:1, 10:2, and 10:3. The physicochemical parameters of the substrates were characterised before and after harvest (pH, electrical conductivity, organic matter, Cu, Hg, As, and Pb) and also in the vegetative organs of S. tuberosum after harvest (Cu, As and Pb). A physicochemical analysis of the mine leachate was also conducted, presenting a pH of 3.03 and an electric conductivity of 1240 mS/cm. The produced tubers were analysed in terms of their dry weight, number, size, and metal concentration. The tubers revealed the presence of Cu, Pb, and As, varying between 17.3 and 32, <5 and 27.6, and <5 and 14.8 mg/kg, respectively, considering the three varieties and all the substrates. The cultivation substrate impacted the mobilisation and accumulation of Cu, As, and Pb in the different vegetative organs and was dependent on the variety and substrate composition. Regarding the phenological development, no influence was observed. In the three varieties, the number of tubers per plant increased in the substrates that contained mine waste compared to the control substrate. Pearson's and Spearman's correlation coefficients were used to determine the correlations between the metal concentrations in the substrates and those in the vegetative organs of the plants. It was found that potato production in contaminated soils can cause a risk to human health, with a lower risk for the Agria variety, as lower levels of contaminating metals accumulate in these tubers.