Nowadays, artificial intelligence (AI) has become very prominent and impactful owing to its proficiency in accomplishing a wide variety of tasks with high levels of effectiveness and efficiency. Some of the areas where AI has demonstrated its capabilities include, but are not restricted to, visual recognition tasks like image classification, object detection, sensor data and natural language processing. Deep learning is an advanced sub-discipline of machine learning that emphasizes on refining artificial neural networks with multiple layers to apprehend intricate representations of data. It can learn useful things from raw data without manual feature engineering. In contrast, the advent of Internet of Things devices having inbuilt sensors opens up novel prospects for implementing convolutional neural networks (CNNs) directly on resource-limited devices. However, these devices have limited memory, storage, and computing power, making extensive, complex CNNs infeasible. Implementing compact CNNs with smaller models and computational needs on IoT devices enables localized AI capabilities like object recognition without relying on the cloud. This reduces latency while improving privacy and reliability. The goal of this paper is to thoroughly evaluate various compact CNN architectures for object recognition trained on a small resource-constrained platform, the NVIDIA Jetson Xavier. Rigorous experimentation identifies the best compact CNN models that balance accuracy and speed on embedded IoT devices. The key objectives are to analyze resource usage such as CPU/GPU and RAM used to train models, the performance of the CNNs, identify trade-offs, and find optimized deep learning solutions tailored for training and real-time inferencing on edge devices with tight resource constraints.