greenhouse

This repository contains code for detecting heat pipes in the greenhouse as well as estimating the pose of the pipes.

Platform: ROS 2, Humble, Ubuntu 22.04

How to build the workspace?

How to install dependencies??

This section explains what each module is responsible for.

pipe_msgs

find-pose

yolov3_ros

yolov3

This module contains code for yolov3. This method is being used to train the model to detect pipes in the greenhouse.

• Ros bag was converted from ros 1 to ros 2 using rosbags module

• Ros 2 bag was played, convert_2_img was used to subscribe to images and all the images were saved as .jpeg in a folder.

• Images were labeled used labelImg.

• Create a custom-yolov3.yaml file that has information about number of classes, location for labels,images.

• Used google colab to run the yolov3 training.

• Upload the label and images to drive and Mount the google drive in python notebook

from google.colab import drive
drive.mount("/content/gdrive")

• Upload the yolov3 code and cd into the location of code

%cd /content/gdrive/MyDrive/yolov3

• Run the training script

!python train.py --img 1280 --batch 16 --epochs 300 --data data/custom-yolov3.yaml --weights '' --cfg yolov3.yaml

• Important to note that we are not using any pre-trained weights.
• Once training is finished, a run folder is created with exp number that stores the best weights (a .pt file).
• This file is then used by detection to node to detect on new data. Update the image path and weights path to run detection.

!python detect.py --img 1280 --source ../pipe-dataset/validate/images/stereo_image103.jpeg --weights runs/train/exp14/weights/best.pt

• Once detect.py is finished, it create a new folder called detect inside runs folder that store the image with bounding box of detected object. More details: https://github.com/ultralytics/yolov3

convert_2_img

This ROS module converts the rosbag data to images for yolo training purpose etc.

• Make sure you have this module inside a ros workspace.
• Create folder called stereo inside convert_2_img module.
• Run following command to launch the node. Currently, this node listens for /camera/color/image_raw topic.

$ cd ros2_ws/src/convert_2_img
$ python3 convert_2_img/convert_to_img.py

• Play the rosbag in another terminal

$ ros2 bag play bag/bag.db

• Once bag has finished playing, the images will be stored inside stereo folder.

labelImg

This module is used to label images for yolo. The pre-defined custom classes file was changed to use new labels. This file is stored in cd labelImg/data/predefined_classes.txt To launch the gui, run

$ cd labelImg
$ python3 labelImg.py 

More details: https://github.com/heartexlabs/labelImg

rosbags

This module convert rosbags from ros 1 to ros 2.

git clone https://gitlab.com/ternaris/rosbags.git
cd rosbags
python -m venv venv
. venv/bin/activate

pip install -r requirements-dev.txt
pip install -e .

rosbags-convert ../single_depth_color_640x480.bag

pcl_conversions

pcl_ros

flann_based

Module that uses 3D model of the pipe to estimate pose. This method was not successful.

yolov7

This module contains code for yolov7. This method was too heavy and didn't produce great results for detection.

darknet_ros2

This module was provides interface to run neural network as rosnodes. The purpose was to use yolo model as ros node but this method was not successful

darknet_vendor

This module was needed to build darknet_ros2.