The application of herbicides in agricultural systems raises concerns regarding their non-target effects on soil microbial communities, particularly symbiotic nitrogen-fixing rhizobia. This study investigated the effect of herbicide-treated soil inoculum on rhizobia nodulation in cowpea and soybean. A glasshouse experiment was conducted with 15 herbicide-treated and 15 untreated soils collected from farmers’ fields. Seeds were surface-sterilized (70% ethanol and sodium hypochlorite) and sown at 2–3 cm depth. Seedlings were inoculated with soil suspensions (5–10 g soil in 300 mL distilled water; 100 mL per pot), with three replicates per treatment. Plants were irrigated with nitrogen-free nutrient solution and harvested at flowering (65 days after planting, DAP). Nodulation and growth parameters—including nodule number, nodule DM, shoots DM, and roots—were quantified to assess herbicide-induced effects. The results revealed significant (p ≤ 0.05) differences in nodulation, plant growth, and nitrogen fixation in herbicide-treated soils compared to untreated controls, with variability influenced by crop species and soil origin. Untreated soils increased whole-plant dry matter (DM), particularly in cowpea, at locations Nator (6.05 g/plant) and McCoy (5.87 g/plant). Soybean exhibited notable nodule reduction in herbicide-treated soils, except in Dapuori, suggesting potential herbicide tolerance in local rhizobial strains. Conversely, untreated soil from Lambussie recorded an increase in soybean nodulation (19.00 nodules/plant). Cowpea exhibited greater sensitivity to herbicide residues, with pronounced declines in nodule formation, while soybean demonstrated moderate resilience. These findings underscore that herbicides significantly impair nodulation, with cowpea being more adversely affected than soybean. Untreated soil inoculum promoted superior nodulation and biomass, emphasizing the necessity for herbicide selection based on soil-microbe compatibility. Site-specific variations suggest the existence of herbicide-tolerant rhizobial strains in certain soils. Prolonged herbicide use may compromise soil fertility and sustainable crop production, warranting further investigation into microbial adaptation strategies.