This study presents TomatoBot, a semi-autonomous agricultural robot designed for tomato detection, navigation, and harvesting. The system integrates computer vision, semantic understanding, and robotic manipulation to address labor-intensive tomato harvesting tasks. TomatoBot detects and classifies tomatoes into six categories using a custom-trained YOLOv8 model and performs navigation by identifying crop lanes and following a computed centerline. Environmental understanding is achieved through semantic segmentation of soil, crop, and background classes using a U-Net with FCN and a ResNet50 backbone. Lane centerlines are estimated using a Deep Hough Transformer with a MobileNetV2 backbone, where geometric interpolation is applied to generate a stable navigation path. The robot is controlled by a Raspberry Pi 4 and equipped with a 6-DOF robotic arm driven by inverse kinematics for tomato plucking. A mobile application enables real-time monitoring and semi-manual interaction. Experimental results demonstrate a mean average precision (mAP@50) of 88.1% for tomato detection, an overall pixel accuracy of 96.11% for semantic segmentation, and an F-measure of 90.45% for semantic line detection, resulting in improved navigation stability. Overall, TomatoBot demonstrates the feasibility of combining lightweight AI models with robotic manipulation for precision farming and provides a scalable foundation for future autonomous agricultural systems.