Deep learning models are applied in precision agriculture for site-specific weed management. This is done by classifying and detecting weeds in farmlands. However, due to their largeness in size, they are rarely adopted in resource-constrained devices (like edge devices) used in precision agriculture. In this study, we propose a lightweight deep learning model for detecting weeds in corn and soybean plants. We used transfer learning to train a MobilenetV2 model for detecting weeds in corn and soybean. The dataset used consists of 5773 samples of corn, soybean, and weeds. The model reached a classification accuracy of 96% but the size of it. We then applied the quantization technique to reduce the size of the model, and also increase latency so it can be deployed on edge devices. The MobilenetV2 model achieved an accuracy of 96% while the quantized model has an accuracy of 90%. Also, the quantized model is three times lesser in size than the original MobilenetV2 model. The results show that quantization can be used to reduce the size of a model, yet maintain a reasonable amount of its accuracy.