Many strawberry growers in certain regions of the United States rely on customers to pick the fruits during the peak harvest months. Unfavorable weather conditions such as high humidity and excessive rainfall can quickly promote fruit rot and diseases. In this study, an elementary farm information system was established with the goal to demonstrate timely information on the farm and fruit conditions (ripe, unripe or rotten) to the grower. The information system processes a video clip or a sequence of images from a camera to provide a map with estimated quantities of strawberries at different stages of ripeness to the growers. The farm map is built by the state-of-the-art vision-based simultaneous localization and mapping (SLAM) techniques, which can generate the map as well as track the motion trajectory using image features. In addition, the input images will pass through a semantic segmentation process using a learning-based approach to identify the conditions of the berries. An encoder-decoder neural network model is trained by a set of labeled images first, and then the trained model is used to identify the fruit conditions from the incoming images. Finally, the quantities of berries in different conditions are estimated using the segmentation results and demonstrated in the system. Generating this information can aid the growers’ decision-making process in regard to directing consumer traffic or farm labor to specific strawberry locations within a farm where fruits need to be picked, where rotten berries need to be removed or where pesticide applications could be targeted. The obtained system can help reduce farm revenue loss and promote sustainable crop production.