- Once detect.py is finished, it create a new folder called `detect` inside `runs` folder that store the image with bounding box of detected object.
More details: https://github.com/ultralytics/yolov3
## convert_2_img
This ROS module converts the rosbag data to images for yolo training purpose etc.
- Make sure you have this module inside a ros workspace.
- Create folder called `stereo` inside convert_2_img module.
- Run following command to launch the node. Currently, this node listens for `/camera/color/image_raw` topic.
```
$ cd ros2_ws/src/convert_2_img
$ python3 convert_2_img/convert_to_img.py
```
- Play the rosbag in another terminal
```
$ ros2 bag play bag/bag.db
```
- Once bag has finished playing, the images will be stored inside `stereo` folder.
## labelImg
This module is used to label images for yolo. The pre-defined custom classes file was changed to use new labels. This file is stored in `cd labelImg/data/predefined_classes.txt`
To launch the gui, run
```
$ cd labelImg
$ python3 labelImg.py
```
More details: https://github.com/heartexlabs/labelImg
## rosbags
This module convert rosbags from ros 1 to ros 2.
```
git clone https://gitlab.com/ternaris/rosbags.git
cd rosbags
python -m venv venv
. venv/bin/activate
pip install -r requirements-dev.txt
pip install -e .
rosbags-convert ../single_depth_color_640x480.bag
```
## flann_based
Module that uses 3D model of the pipe to estimate pose. This method was not successful.
## yolov7
This module contains code for yolov7. This method was too heavy and didn't produce great results for detection.
## darknet_ros2
This module was provides interface to run neural network as rosnodes. The purpose was to use yolo model as ros node but this method was not successful
Once you have succefully built the ROS Modules specifically, `yolov3_ros` and `find-pose` along with dependencies like `pipe_msgs` and `perception_ros`, `perception_pcl`,you can do the following:
Assumption:
You have the following required data in forms of ros topic:
- /camera/color/image_raw: RGB Image topic
- /camera/aligned_depth_to_color/image_raw: Aligned depth image topic (aligned with rgb image)
- /camera/color/camera_info: Camera calibration parameters topic
How do you get this data?
- This data can come from a pre recorded Rosbag. If this is the case, do following:
```
$ ros2 bag play bag-folder/bag-name.db3
```
- This data can come directly from camera's (D455/ZED2I) ROS Node: Launch your node in a terminal.
- If your camera topics have different names than the default topic names mentioned above, update/remap ONLY the `pipe_detection.launch.py` script stored in `ros2_ws/src/yolov3_ros/launch` folder.
- Once you have updated the launch file, build the code again using `colcon build --packages-select yolov3_ros` and proceedas mentioned below.
2. Run `$ cd greenhouse/ros2_ws/` and `$ . install/setup.bash`.
3. Launch node for object detection:
```
$ ros2 launch yolov3_ros pipe_detection.launch.py
```
This node will output two topics: `/bboxes` and `/detection_image`.
4. Launch node for pose estimation:
```
$ ros2 launch find-pose find-pose-node.launch.py
```
This node will output TF topics between `/camera_link` and `/${detected_object_name}`.
5. Open RVIZ by running `$ rviz2`. Change `fixed frame` from `map` to `camera_link`.
5a. Go to `Add` button. Under `By Display type`, select `tf`. Once tf is added, select required frames like `camera_link` and `l_trail` to see the tfs.
5b. To see the current image with detected object, Go to `Add` Button. Under `By Topic`, select topci called `detection_image`.
You can add other topics as per the need and topic names.
6. You can open launch files to update/remap topic name if different camera is being used.
7. You can also update ros parameter from launch file. Currently, the `pipe_weights.pt` file is the one used. This file can be changed and you can update the parameter name `best_weights` inside the `pipe_detection.launch.py` file.
