A low-carbon bainitic drilled steel exhibits high hardness after hot rolling, which is not conducive to machining. In order to soften this type of drilled steel less than 260 HB and accelerate the subsequent soft annealing, a pre-austenitizing was designed based on thermodynamic calculations of phase stability.Different initial microstructures were prepared with three austenitizing temperatures (680 ^oC, 850 ^oC, 1000 ^oC) and three cooling methods (water quenching, oil quenching, and air cooling). The effects of initial microstructure during annealing with different temperatures and times on microstructures and mechanical properties were studied. The softening equations as a function of λ-value was established for different initial microstructures, and the relationships between annealing temperature, annealing time, activation energy and hardness were explored. The predicted hardness were consistent with the measured values. The initial microstructures affect activation energy, i.e., the activation energy for diffusion with respect to the martensitic structure was less than that of the bainitic structure, and the corresponding softening rate with the martensitic initial structure was greater. In addition, the higher the carbide content in the bainitic structure, the greater the proportion of martensite in the martensite-retained austenite (M/A) structure, the more lath-shaped M/A and the less massive M/A, the smaller the activation energy tended to be.