Compare commits

...

6 Commits

Author SHA1 Message Date
gautam 79e53e788a Update 'README.md' 2023-04-13 14:36:49 +02:00
gautam cd5aade200 Update 'README.md' 2023-04-12 16:28:40 +02:00
apoorva 303d8e7742 cleanup tabs 2023-04-05 15:35:07 +05:30
apoorva db318cf66c cleanup 2023-04-05 15:34:19 +05:30
apoorva 7e8d3fb2a8 updated with dependencies 2023-04-05 15:31:08 +05:30
apoorva 3c49096d39 one more update to readme 2023-04-05 15:31:08 +05:30
+81 -45
View File
@@ -4,38 +4,72 @@ This repository contains code for detecting heat pipes in the greenhouse as well
Platform: ROS 2, Humble, Ubuntu 22.04 Platform: ROS 2, Humble, Ubuntu 22.04
## How to build the workspace?
```
$ git clone https://tea.der-space.de/apoorva/greenhouse.git
```
## How to install dependencies?? ## How to install dependencies??
- Install `git`
``` ```
sudo apt install git
sudo apt-get install ros-humble-$(PACKAGE_NAME)
``` ```
- Install `ROS 2 Humble` on `Ubuntu 22.04` by following https://docs.ros.org/en/foxy/Installation/Ubuntu-Install-Debians.html
- Install `colcon build` by following https://docs.ros.org/en/foxy/Tutorials/Beginner-CLI-Tools/Configuring-ROS2-Environment.html
```
sudo apt install python3-colcon-common-extensions
```
- Instal `pip` for python packages
```
sudo apt install python3-pip
```
- Now clone the repository
```
git clone https://tea.der-space.de/apoorva/greenhouse.git
```
- Install `ultralytics` for yolov3 package
```
pip install ultralytics
```
- For `yolov3_ros`, there are a bunch of other requirements. Go to `yolov3` folder and install using following commands:
```
cd ~/greenhouse/yolov3
pip install -r requirements.txt
```
- Go to `ros2_ws` inside `greenhouse`. Make sure you only have `src` folder.
```
cd ~/greenhouse/ros2_ws
ls
src
```
- Inside `ros2_ws` folder, start building individual packages in the below sequence to avoid errors.
```
colcon build --packages-select pipe_msgs
colcon build --packages-select pcl_ros
colcon build --allow-overriding pcl_ros
colcon build --packages-select pcl_conversions
colcon build --allow-overriding pcl_conversions
colcon build --packages-select find-pose
colcon build --packages-select yolov3_ros
. install/setup.bash
```
- The code should be ready to launch as explained in [How to run Live Detection?](#how-to-run-live-detection)
This section explains what each module is responsible for. This section explains what each module is responsible for.
## perception_pcl ## perception_pcl
This module is responsible for providing `pcl_conversions` and `pcl_ros` modules in `ros 2`. This module is responsible for providing `pcl_conversions` and `pcl_ros` modules in `ros 2`.
To build, run the following command: To build, run the following command:
``` ```
$ cd ros2_ws/ cd ros2_ws/
$ colcon build --packages-select perception_pcl colcon build --packages-select perception_pcl
$ . install/setup.bash . install/setup.bash
``` ```
## pipe_msgs ## pipe_msgs
This module contains ros msgs for storing information about the detected object's bounding box. This module contains ros msgs for storing information about the detected object's bounding box.
``` ```
$ cd ros2_ws/ cd ros2_ws/
$ colcon build --packages-select pipe_msgs colcon build --packages-select pipe_msgs
$ . install/setup.bash . install/setup.bash
``` ```
To check if msgs are built properly, run following command To check if msgs are built properly, run following command
``` ```
$ ros2 interface show pipe_msgs/msg/BoundingBox ros2 interface show pipe_msgs/msg/BoundingBox
```
The output will be: The output will be:
float64 probability float64 probability
int64 xmin int64 xmin
@@ -44,7 +78,7 @@ int64 xmax
int64 ymax int64 ymax
int16 id int16 id
string class_id string class_id
```
## find-pose ## find-pose
This ROS module is responsible for determining the position of the detected objects. This ROS module is responsible for determining the position of the detected objects.
@@ -58,9 +92,9 @@ Output:
How to build and run? How to build and run?
This package is dependent on custom `pcl_conversion` and `pcl_ros` module. Make sure you have built those before building this package. This package is dependent on custom `pcl_conversion` and `pcl_ros` module. Make sure you have built those before building this package.
``` ```
$ colcon build --packages-select find-pose colcon build --packages-select find-pose
$ . install/setup.bash . install/setup.bash
$ ros2 launch find-pose find-pose-node.launch.py ros2 launch find-pose find-pose-node.launch.py
``` ```
All the topics can be remapped in the launch file. All the topics can be remapped in the launch file.
@@ -73,7 +107,7 @@ The following input/ros topics are needed:
ROS Paramater Input: ROS Paramater Input:
- best_weights: String that is the path to the best weights file of yolov3 detection - best_weights: String that is the path to the best weights file of yolov3 detection
Defaults: `'src/pipe_weights.pt'` Defaults: `'src/pipe_weights.pt'` inside `ros2_ws` folder
The following are the output topics: The following are the output topics:
- /detection_image: The RGB Image topic with bounding box drawn on it for visualization and debugging purpose - /detection_image: The RGB Image topic with bounding box drawn on it for visualization and debugging purpose
@@ -84,9 +118,10 @@ The following are the output topics:
How to build and run? How to build and run?
``` ```
$ colcon build --packages-select yolov3_ros colcon build --packages-select yolov3_ros
$ . install/setup.bash . install/setup.bash
$ ros2 launch yolov3_ros pipe_detection.launch.py cd greenhouse/ros2_ws/
ros2 launch yolov3_ros pipe_detection.launch.py
``` ```
All the topics can be remapped in the launch file. The path to best_weights can also be changed inside the launch file. All the topics can be remapped in the launch file. The path to best_weights can also be changed inside the launch file.
Launch file is stored in `yolov3_ros/launch/`. Launch file is stored in `yolov3_ros/launch/`.
@@ -106,7 +141,7 @@ drive.mount("/content/gdrive")
``` ```
- Upload the yolov3 code and cd into the location of code - Upload the yolov3 code and cd into the location of code
``` ```
%cd /content/gdrive/MyDrive/yolov3 cd /content/gdrive/MyDrive/yolov3
``` ```
- Run the training script - Run the training script
``` ```
@@ -127,12 +162,12 @@ This ROS module converts the rosbag data to images for yolo training purpose etc
- Create folder called `stereo` inside convert_2_img module. - Create folder called `stereo` inside convert_2_img module.
- Run following command to launch the node. Currently, this node listens for `/camera/color/image_raw` topic. - Run following command to launch the node. Currently, this node listens for `/camera/color/image_raw` topic.
``` ```
$ cd ros2_ws/src/convert_2_img cd ros2_ws/src/convert_2_img
$ python3 convert_2_img/convert_to_img.py python3 convert_2_img/convert_to_img.py
``` ```
- Play the rosbag in another terminal - Play the rosbag in another terminal
``` ```
$ ros2 bag play bag/bag.db ros2 bag play bag/bag.db
``` ```
- Once bag has finished playing, the images will be stored inside `stereo` folder. - Once bag has finished playing, the images will be stored inside `stereo` folder.
@@ -140,8 +175,8 @@ $ ros2 bag play bag/bag.db
This module is used to label images for yolo. The pre-defined custom classes file was changed to use new labels. This file is stored in `cd labelImg/data/predefined_classes.txt` This module is used to label images for yolo. The pre-defined custom classes file was changed to use new labels. This file is stored in `cd labelImg/data/predefined_classes.txt`
To launch the gui, run To launch the gui, run
``` ```
$ cd labelImg cd labelImg
$ python3 labelImg.py python3 labelImg.py
``` ```
More details: https://github.com/heartexlabs/labelImg More details: https://github.com/heartexlabs/labelImg
@@ -181,28 +216,29 @@ You have the following required data in forms of ros topic:
How do you get this data? How do you get this data?
- This data can come from a pre recorded Rosbag. If this is the case, do following: - This data can come from a pre recorded Rosbag. If this is the case, do following:
``` ```
$ ros2 bag play bag-folder/bag-name.db3 ros2 bag play bag-folder/bag-name.db3
``` ```
- This data can come directly from camera's (D455/ZED2I) ROS Node: Launch your node in a terminal. - This data can come directly from camera's (D455/ZED2I) ROS Node: Launch your node in a terminal.
- If your camera topics have different names than the default topic names mentioned above, update/remap ONLY the `pipe_detection.launch.py` script stored in `ros2_ws/src/yolov3_ros/launch` folder. - If your camera topics have different names than the default topic names mentioned above, update/remap ONLY the `pipe_detection.launch.py` script stored in `ros2_ws/src/yolov3_ros/launch` folder.
- Once you have updated the launch file, build the code again using `colcon build --packages-select yolov3_ros` and proceedas mentioned below. - Once you have updated the launch file, build the code again using `colcon build --packages-select yolov3_ros` and proceedas mentioned below.
Steps to run: Steps to run:
1. Open a terminal. - Open a terminal.
2. Run `$ cd greenhouse/ros2_ws/` and `$ . install/setup.bash`. - Run `$ cd greenhouse/ros2_ws/` and `$ . install/setup.bash`.
3. Launch node for object detection: - Launch node for object detection:
``` ```
$ ros2 launch yolov3_ros pipe_detection.launch.py ros2 launch yolov3_ros pipe_detection.launch.py
``` ```
This node will output two topics: `/bboxes` and `/detection_image`. - This node will output two topics: `/bboxes` and `/detection_image`.
4. Launch node for pose estimation: - Make sure that you run this launch file from `greenhouse/ros2_ws/` folder since the path of weights is relative (`/src/pipe_weights.pt`) inside the launch file.
- Launch node for pose estimation:
``` ```
$ ros2 launch find-pose find-pose-node.launch.py ros2 launch find-pose find-pose-node.launch.py
``` ```
This node will output TF topics between `/camera_link` and `/${detected_object_name}`. - This node will output TF topics between `/camera_link` and `/${detected_object_name}`.
5. Open RVIZ by running `$ rviz2`. Change `fixed frame` from `map` to `camera_link`. - Open RVIZ by running `$ rviz2`. Change `fixed frame` from `map` to `camera_link`.
5a. Go to `Add` button. Under `By Display type`, select `tf`. Once tf is added, select required frames like `camera_link` and `l_trail` to see the tfs. - Go to `Add` button. Under `By Display type`, select `tf`. Once tf is added, select required frames like `camera_link` and `l_trail` to see the tfs.
5b. To see the current image with detected object, Go to `Add` Button. Under `By Topic`, select topci called `detection_image`. - To see the current image with detected object, Go to `Add` Button. Under `By Topic`, select topci called `detection_image`.
You can add other topics as per the need and topic names. -You can add other topics as per the need and topic names.
6. You can open launch files to update/remap topic name if different camera is being used. - You can open launch files to update/remap topic name if different camera is being used.
7. You can also update ros parameter from launch file. Currently, the `pipe_weights.pt` file is the one used. This file can be changed and you can update the parameter name `best_weights` inside the `pipe_detection.launch.py` file. - You can also update ros parameter from launch file. Currently, the `pipe_weights.pt` file is the one used. This file can be changed and you can update the parameter name `best_weights` inside the `pipe_detection.launch.py` file.