Forty-eight goats were selected at eight-time points (1, 7, 14, 28, 42, 56, 70, and 84 days of age; 6 goats per group) and slaughtered for the evaluation of the anatomical and morphological development of gastrointestinal tract. The newborn goats fed maternal colostrum (MC) from birth to 7 d, maternal milk (MM) until 28 d, and maternal milk plus solid diet (MMSD) later. While at 60 d, the goats were weaned and offered only solid diet (SD). Body weight (BW) and carcass weight (CW) showed significant increase with age (p < 0.01). The absolute growth rate (AGR) of BW and CW were higher (p < 0.05) at MMSD phase than MM phase. Internal and external organs as a percentage of BW decreased with age (p < 0.01). However, the complex stomach percentage increased (p < 0.01) with age. The rumen and omasum weight experienced synchronous AGR with age (p < 0.01), especially at the SD phase. In contrast, the AGR of reticulum and abomasum were the highest (p < 0.01) at MMSD and MC, respectively. After weaning, the goats reared on SD showed the highest (p < 0.05) papillae height, lamina propria, muscle layer thickness (MLT), and epithelial thickness (ET). Compared the MMSD phase, the SD phase showed the highest (p < 0.01) colonic mucosa thickness and ileal MLT and villus height. However, the crypt depth (CD) was higher (p < 0.01) at MMSD phase than MM phase. Moreover, the CD and MLT of the jejunum increased (p < 0.01) with age. Furthermore, duodenal CD, MLT, and ET increased (p < 0.01) in SD phase. Overall, these findings contribute to a better understanding of the effective development pattern of the rumen and ultimately contribute to optimal feeding management, offset weaning challenges, and avoiding digestive disorders in newborn goats.