greenhouse/README.md

# greenhouse

This repository contains code for detecting heat pipes in the greenhouse as well as estimating the pose of the pipes.

Platform: ROS 2, Humble, Ubuntu 22.04

How to build the workspace?

How to install dependencies??


This section explains what each module is responsible for.

## pipe_msgs

## find-pose

## yolov3_ros

## yolov3
This module contains code for yolov3. This method is being used to train the model to detect pipes in the greenhouse.
- Ros bag was converted from ros 1 to ros 2 using rosbags module
- Ros 2 bag was played, convert_2_img was used to subscribe to images and all the images were saved as .jpeg in a folder.
- Images were labeled used labelImg.
- Create a `custom-yolov3.yaml` file that has information about number of classes, location for labels,images.

- Used google colab to run the yolov3 training.
- Upload the label and images to drive and Mount the google drive in python notebook
```
from google.colab import drive
drive.mount("/content/gdrive")
```
- Upload the yolov3 code and cd into the location of code
```
%cd /content/gdrive/MyDrive/yolov3
```
- Run the training script
```
!python train.py --img 1280 --batch 16 --epochs 300 --data data/custom-yolov3.yaml --weights '' --cfg yolov3.yaml
```
- Important to note that we are not using any pre-trained weights.
- Once training is finished, a `run` folder is created with exp number that stores the best weights (a .pt file).
- This file is then used by detection to node to detect on new data. Update the image path and weights path to run detection.
```
!python detect.py --img 1280 --source ../pipe-dataset/validate/images/stereo_image103.jpeg --weights runs/train/exp14/weights/best.pt
```
- Once detect.py is finished, it create a new folder called `detect` inside `runs` folder that store the image with bounding box of detected object.
More details: https://github.com/ultralytics/yolov3

## convert_2_img
This ROS module converts the rosbag data to images for yolo training purpose etc.
- Make sure you have this module inside a ros workspace.
- Create folder called `stereo` inside convert_2_img module.
- Run following command to launch the node. Currently, this node listens for `/camera/color/image_raw` topic.
```
$ cd ros2_ws/src/convert_2_img
$ python3 convert_2_img/convert_to_img.py
```
- Play the rosbag in another terminal
```
$ ros2 bag play bag/bag.db
```
- Once bag has finished playing, the images will be stored inside `stereo` folder.

## labelImg
This module is used to label images for yolo. The pre-defined custom classes file was changed to use new labels. This file is stored in `cd labelImg/data/predefined_classes.txt`
To launch the gui, run
```
$ cd labelImg
$ python3 labelImg.py 
```
More details: https://github.com/heartexlabs/labelImg

## rosbags
This module convert rosbags from ros 1 to ros 2. 
```
git clone https://gitlab.com/ternaris/rosbags.git
cd rosbags
python -m venv venv
. venv/bin/activate

pip install -r requirements-dev.txt
pip install -e .

rosbags-convert ../single_depth_color_640x480.bag
```

## pcl_conversions

## pcl_ros


## flann_based
Module that uses 3D model of the pipe to estimate pose. This method was not successful.

## yolov7
This module contains code for yolov7. This method was too heavy and didn't produce great results for detection.

## darknet_ros2
This module was provides interface to run neural network as rosnodes. The purpose was to use yolo model as ros node but this method was not successful

## darknet_vendor
This module was needed to build darknet_ros2.