Olive leaves are a source of valuable compounds, including phenolic compounds and triterpenic acids, with applications in the phytopharmacy sector, among others. Nevertheless, there are untapped olive resources that can give new clues for the discovery of natural bioactive compounds. Therefore, in this study the antioxidant composition of olive mill leaves (OML), extracted olive pomace (EOP) and a new byproduct, the residual fraction from olive pit cleaning (RFOPC), was characterized and compared to olive leaves, which have been extensively studied as a source of bioactive compounds. The chemical characterization showed that all these byproducts contain a high amount of extractives; in all cases it was higher than 26%. This is interesting since the extractive fraction contains non-structural compounds, such as phenolic compounds and triterpenic acids. Then, ultrasound-assisted extraction was applied to recover phenolic compounds from this fraction using an aqueous-ethanolic solution. All the extracts showed antioxidant properties and the total phenolic content ranged from 69 (RFOPC) to 140 (EOP) g of gallic acid equivalents/kg dry weight. The profile obtained by liquid chromatography-mass spectrometry was different, suggesting some chemical differences in the phenolic composition. In fact, while olive leafy biomass and EOP contained hydroxytyrosol derivatives, RFOPC was a source of novel trilignols. Other bioactive compounds were found in the extracts, including triterpenic acids, which could give an extra value due to their cardioprotective properties.