# bop_toolkit

**Repository Path**: cuge1995/bop_toolkit

## Basic Information

- **Project Name**: bop_toolkit
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-11-09
- **Last Updated**: 2021-11-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# BOP Toolkit

A Python toolkit of the BOP benchmark for 6D object pose estimation
(http://bop.felk.cvut.cz).

- **bop_toolkit_lib** - The core Python library for i/o operations, calculation
  of pose errors, Python based rendering etc.
- **docs** - Documentation and conventions.
- **scripts** - Scripts for evaluation, rendering of training images,
  visualization of 6D object poses etc.

## Installation

### Python Dependencies

To install the required python libraries, run:
```
pip install -r requirements.txt
```

In the case of problems, try to first run: ```pip install --upgrade pip setuptools```

### Python Renderer

The Python based renderer is implemented using
[Glumpy](https://glumpy.github.io/) which depends on
[freetype](https://www.freetype.org/) and [GLFW](https://www.glfw.org/).
Glumpy is installed using the pip command above. On Linux, freetype and GLFW can
be installed by:

```
apt-get install freetype
apt-get install libglfw3
```

To install freetype and GLFW on Windows, follow [these instructions](https://glumpy.readthedocs.io/en/latest/installation.html#step-by-step-install-for-x64-bit-windows-7-8-and-10).

GLFW serves as a backend of Glumpy. [Another backend](https://glumpy.readthedocs.io/en/latest/api/app-backends.html)
can be used but were not tested with our code.

### C++ Renderer

For fast CPU-based rendering on a headless server, we recommend installing [bop_renderer](https://github.com/thodan/bop_renderer),
an off-screen C++ renderer with Python bindings.

## Usage

### 1. Get the BOP datasets

Download the BOP datasets and make sure they are in the [expected folder structure](https://bop.felk.cvut.cz/datasets/).

### 2. Run your method

Estimate poses and save them in one .csv file per dataset ([format description](https://bop.felk.cvut.cz/challenges/bop-challenge-2020/#howtoparticipate)).

### 3. Configure the BOP Toolkit

In [bop_toolkit_lib/config.py](https://github.com/thodan/bop_toolkit/blob/master/bop_toolkit_lib/config.py), set paths to the BOP datasets, to a folder with results to be evaluated, and to a folder for the evaluation output. The other parameters are necessary only if you want to visualize results or run the C++ Renderer.

### 4. Evaluate the pose estimates
```
python scripts/eval_bop19.py --renderer_type=python --result_filenames=NAME_OF_CSV_WITH_RESULTS
```
--renderer_type: Either "python" or "cpp" (you need to install the C++ Renderer for the latter).
--result_filenames: Comma-separated filenames with pose estimates in .csv ([examples](http://ptak.felk.cvut.cz/6DB/public/bop_sample_results)).

## Convert BOP to COCO format

```
python scripts/calc_gt_coco.py
```

Set the dataset and split parameters in the top section of the script.



```
def draw_detections_pose(img, intrinsics, pred_sRT, pred_size, pred_class_ids):
    """ Visualize pose predictions.
    """

    for i in range(pred_sRT.shape[0]):
        sRT = pred_sRT[i]
        #if pred_class_ids[i] in [1, 2, 4]:
        #    sRT = align_rotation(pred_sRT[i, :, :])
        axis = np.array([[0,0,0], [1.0, 0, 0], [0, 1.0, 0], [0, 0, 1.0]]).T #3,4
        s = np.cbrt(np.linalg.det(sRT[:3, :3]))
        RT = sRT[:3,:3]/s
        axis = RT[:3,:3].dot(axis * 0.1) + sRT[:3, 3].reshape(3,1)
        axis = intrinsics.dot(axis).T
        axis = axis[:,:2]/axis[:,2].reshape(-1,1)
        axis = axis.astype(np.int32)
        cv2.line(img, tuple(axis[0].tolist()), tuple(axis[1].tolist()), ( 0, 0, 255), 2)
        cv2.line(img, tuple(axis[0].tolist()), tuple(axis[2].tolist()), ( 0, 255, 0), 2)
        cv2.line(img, tuple(axis[0].tolist()), tuple(axis[3].tolist()), ( 255, 0, 0), 2)

    return img



```


### for pvnet
```
    def draw_detections_pose(img, intrinsics, pred_sRT):
        """ Visualize pose predictions.
        """

        for i in range(pred_sRT.shape[0]):
            sRT = pred_sRT[i]
            #if pred_class_ids[i] in [1, 2, 4]:
            #    sRT = align_rotation(pred_sRT[i, :, :])
            axis = np.array([[0,0,0], [1.0, 0, 0], [0, 1.0, 0], [0, 0, 1.0]]).T #3,4
            s = np.cbrt(np.linalg.det(sRT[:3, :3]))
            RT = sRT[:3,:3]/s
            axis = RT[:3,:3].dot(axis * 0.1) + sRT[:3, 3].reshape(3,1)
            axis = intrinsics.dot(axis).T
            axis = axis[:,:2]/axis[:,2].reshape(-1,1)
            axis = axis.astype(np.int32)
            cv2.line(img, tuple(axis[0].tolist()), tuple(axis[1].tolist()), ( 0, 0, 255), 2)
            cv2.line(img, tuple(axis[0].tolist()), tuple(axis[2].tolist()), ( 0, 255, 0), 2)
            cv2.line(img, tuple(axis[0].tolist()), tuple(axis[3].tolist()), ( 255, 0, 0), 2)

        return img
    
    def visualize_axis(self, output, batch):
        inp = img_utils.unnormalize_img(batch['inp'][0], mean, std).permute(1, 2, 0)
        kpt_2d = output['kpt_2d'][0].detach().cpu().numpy()

        img_id = int(batch['img_id'][0])
        anno = self.coco.loadAnns(self.coco.getAnnIds(imgIds=img_id))[0]
        kpt_3d = np.concatenate([anno['fps_3d'], [anno['center_3d']]], axis=0)
        K = np.array(anno['K'])

        pose_gt = np.array(anno['pose'])
        pose_pred = pvnet_pose_utils.pnp(kpt_3d, kpt_2d, K)

        inp = draw_detections_pose(inp, K, pose_pred)

        # corner_3d = np.array(anno['corner_3d'])
        # corner_2d_gt = pvnet_pose_utils.project(corner_3d, K, pose_gt)
        # corner_2d_pred = pvnet_pose_utils.project(corner_3d, K, pose_pred)

        _, ax = plt.subplots(1)
        ax.imshow(inp)
        plt.savefig('test.jpg')

```