Update 'README.md'

README.md

@@ -4,18 +4,127 @@ This repository contains code for detecting heat pipes in the greenhouse as well
 
 Platform: ROS 2, Humble, Ubuntu 22.04
 
-How to build the workspace?
+## How to install dependencies??
-
+- Install `git`
-How to install dependencies??
+```
-
+sudo apt install git
-
+```
+- Install `ROS 2 Humble` on `Ubuntu 22.04` by following https://docs.ros.org/en/foxy/Installation/Ubuntu-Install-Debians.html
+- Install `colcon build` by following https://docs.ros.org/en/foxy/Tutorials/Beginner-CLI-Tools/Configuring-ROS2-Environment.html
+```
+sudo apt install python3-colcon-common-extensions
+```
+- Instal `pip` for python packages
+```
+sudo apt install python3-pip
+```
+- Now clone the repository
+```
+git clone https://tea.der-space.de/apoorva/greenhouse.git
+```
+- Install `ultralytics` for yolov3 package
+```
+pip install ultralytics
+```
+- For `yolov3_ros`, there are a bunch of other requirements. Go to `yolov3` folder and install using following commands:
+```
+cd ~/greenhouse/yolov3
+pip install -r requirements.txt
+```
+- Go to `ros2_ws` inside `greenhouse`. Make sure you only have `src` folder.
+```
+cd ~/greenhouse/ros2_ws
+ls
+src
+```
+- Inside `ros2_ws` folder, start building individual packages in the below sequence to avoid errors.
+```
+colcon build --packages-select pipe_msgs
+colcon build --packages-select pcl_ros
+colcon build --allow-overriding pcl_ros
+colcon build --packages-select pcl_conversions
+colcon build --allow-overriding pcl_conversions
+colcon build --packages-select find-pose
+colcon build --packages-select yolov3_ros
+. install/setup.bash
+```
+- The code should be ready to launch as explained in [How to run Live Detection?](#how-to-run-live-detection)
 This section explains what each module is responsible for.
+## perception_pcl
+This module is responsible for providing `pcl_conversions` and `pcl_ros` modules in `ros 2`.
+To build, run the following command:
+```
+cd ros2_ws/
+colcon build --packages-select perception_pcl
+. install/setup.bash
+```
 
 ## pipe_msgs
+This module contains ros msgs for storing information about the detected object's bounding box.
+```
+cd ros2_ws/
+colcon build --packages-select pipe_msgs
+. install/setup.bash
+```
+To check if msgs are built properly, run following command
+```
+ros2 interface show pipe_msgs/msg/BoundingBox
+```
+The output will be:
+float64 probability
+int64 xmin
+int64 ymin
+int64 xmax
+int64 ymax
+int16 id
+string class_id
+
 
 ## find-pose
+This ROS module is responsible for determining the position of the detected objects.
+The following input/ros topics are needed:
+- /rgb_img: RGB Image topic
+- /camera_info: Camera calibration parameters topic
+- /depth_img: Aligned depth image topic (aligned with rgb image)
+- /bboxes: Bounding box of each detected object. (Comes from yolov3 detection module)
+Output:
+- TF: Transform between camera_link and detected_object frame.
+How to build and run?
+This package is dependent on custom `pcl_conversion` and `pcl_ros` module. Make sure you have built those before building this package.
+```
+colcon build --packages-select find-pose
+. install/setup.bash
+ros2 launch find-pose find-pose-node.launch.py
+```
+All the topics can be remapped in the launch file.
 
 ## yolov3_ros
+This ROS module is responsible for detecting the pipes from rgb image topic and also syncing the depth, rgb and camera_info topics.
+The following input/ros topics are needed:
+- /camera/color/image_raw: RGB Image topic
+- /camera/aligned_depth_to_color/image_raw: Aligned depth image topic (aligned with rgb image)
+- /camera/color/camera_info: Camera calibration parameters topic
+
+ROS Paramater Input:
+- best_weights: String that is the path to the best weights file of yolov3 detection
+Defaults: `'src/pipe_weights.pt'` inside `ros2_ws` folder
+
+The following are the output topics:
+- /detection_image: The RGB Image topic with bounding box drawn on it for visualization and debugging purpose
+- /bboxes: Bounding box of each detected object.
+- /rgb_img: Time Synced RGB Image topic
+- /camera_info: Time Synced Camera calibration parameters topic
+- /depth_img: Time Synced Aligned depth image topic (aligned with rgb image)
+
+How to build and run?
+```
+colcon build --packages-select yolov3_ros
+. install/setup.bash
+cd greenhouse/ros2_ws/
+ros2 launch yolov3_ros pipe_detection.launch.py
+```
+All the topics can be remapped in the launch file. The path to best_weights can also be changed inside the launch file.
+Launch file is stored in `yolov3_ros/launch/`.
 
 ## yolov3
 This module contains code for yolov3. This method is being used to train the model to detect pipes in the greenhouse.
@@ -32,7 +141,7 @@ drive.mount("/content/gdrive")
 ```
 - Upload the yolov3 code and cd into the location of code
 ```
-%cd /content/gdrive/MyDrive/yolov3
+cd /content/gdrive/MyDrive/yolov3
 ```
 - Run the training script
 ```
@@ -53,12 +162,12 @@ This ROS module converts the rosbag data to images for yolo training purpose etc
 - Create folder called `stereo` inside convert_2_img module.
 - Run following command to launch the node. Currently, this node listens for `/camera/color/image_raw` topic.
 ```
-$ cd ros2_ws/src/convert_2_img
+cd ros2_ws/src/convert_2_img
-$ python3 convert_2_img/convert_to_img.py
+python3 convert_2_img/convert_to_img.py
 ```
 - Play the rosbag in another terminal
 ```
-$ ros2 bag play bag/bag.db
+ros2 bag play bag/bag.db
 ```
 - Once bag has finished playing, the images will be stored inside `stereo` folder.
 
@@ -66,8 +175,8 @@ $ ros2 bag play bag/bag.db
 This module is used to label images for yolo. The pre-defined custom classes file was changed to use new labels. This file is stored in `cd labelImg/data/predefined_classes.txt`
 To launch the gui, run
 ```
-$ cd labelImg
+cd labelImg
-$ python3 labelImg.py 
+python3 labelImg.py 
 ```
 More details: https://github.com/heartexlabs/labelImg
 
@@ -85,11 +194,6 @@ pip install -e .
 rosbags-convert ../single_depth_color_640x480.bag
 ```
 
-## pcl_conversions
-
-## pcl_ros
-
-
 ## flann_based
 Module that uses 3D model of the pipe to estimate pose. This method was not successful.
 
@@ -102,6 +206,39 @@ This module was provides interface to run neural network as rosnodes. The purpos
 ## darknet_vendor
 This module was needed to build darknet_ros2.
 
+# How to run live detection?
+Once you have succefully built the ROS Modules specifically, `yolov3_ros` and `find-pose` along with dependencies like `pipe_msgs` and `perception_ros`, `perception_pcl`,you can do the following: 
+Assumption:
+You have the following required data in forms of ros topic:
+- /camera/color/image_raw: RGB Image topic
+- /camera/aligned_depth_to_color/image_raw: Aligned depth image topic (aligned with rgb image)
+- /camera/color/camera_info: Camera calibration parameters topic
+How do you get this data?
+- This data can come from a pre recorded Rosbag. If this is the case, do following:
+```
+ros2 bag play bag-folder/bag-name.db3
+```
+- This data can come directly from camera's (D455/ZED2I) ROS Node: Launch your node in a terminal. 
+- If your camera topics have different names than the default topic names mentioned above, update/remap ONLY the `pipe_detection.launch.py` script stored in `ros2_ws/src/yolov3_ros/launch` folder.
+- Once you have updated the launch file, build the code again using `colcon build --packages-select yolov3_ros` and proceedas mentioned below.
 
-
+Steps to run:
-
+- Open a terminal.
+- Run `$ cd greenhouse/ros2_ws/` and `$ . install/setup.bash`.
+- Launch node for object detection:
+```
+ros2 launch yolov3_ros pipe_detection.launch.py
+```
+- This node will output two topics: `/bboxes` and `/detection_image`.
+- Make sure that you run this launch file from `greenhouse/ros2_ws/` folder since the path of weights is relative (`/src/pipe_weights.pt`) inside the launch file.
+- Launch node for pose estimation:
+```
+ros2 launch find-pose find-pose-node.launch.py
+```
+- This node will output TF topics between `/camera_link` and `/${detected_object_name}`.
+- Open RVIZ by running `$ rviz2`. Change `fixed frame` from `map` to `camera_link`.
+    - Go to `Add` button. Under `By Display type`, select `tf`. Once tf is added, select required frames like `camera_link` and `l_trail` to see the tfs.
+    - To see the current image with detected object, Go to `Add` Button. Under `By Topic`, select topci called `detection_image`.
+    -You can add other topics as per the need and topic names. 
+- You can open launch files to update/remap topic name if different camera is being used. 
+- You can also update ros parameter from launch file. Currently, the `pipe_weights.pt` file is the one used. This file can be changed and you can update the parameter name `best_weights` inside the `pipe_detection.launch.py` file.