# CANN hw acceleration for libValkka

**Repository Path**: ElSampsa/valkka_cann

## Basic Information

- **Project Name**: CANN hw acceleration for libValkka
- **Description**: CANN hw acceleration for libValkka
- **Primary Language**: Unknown
- **License**: WTFPL
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-04-20
- **Last Updated**: 2021-08-17

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

 
# CANN Decoding and Inference for libValkka

CANN is an interface to Huawei's hardware-accelerated video decoding and
neural-net model inference.

LibValkka is a python media-streaming framework.

This module implements CANN hardware acceleration to libValkka as a separate module.

## Installing

### 1. Install libValkka

This is a libValkka extension module, so you need libValkka installed.  Please see [the valkka main page](https://elsampsa.github.io/valkka-examples/_build/html/index.html).

WARNING: (**) 

libValkka that is installed in this standard way, has been precompiled and linked with your **Ubuntu's default/standard python version**.

You might have installed a different python version as the default "python3" into your systems (for sure, if you have followed Huawei's instructions to setup the libraries).

After installing libValkka, try this:
```
python3 -c 'from valkka import core'
```

Or, if it doesn't work, you must explicitly define your system's **default python version**:
```
python3.6 -c 'from valkka import core'
```

Please be aware which is your system's **default python version** (as that is the version used by libValkka) and use that when running libValkka-based programs.

### 2. Install libACL/CANN drivers

Drivers for your specific hw are available [here](https://ascend.huawei.com/#/hardware/firmware-drivers).

For a concise installation scheme of the drivers, please look at [our installation guidelines](cann_guidelines.md)

### 3. Build

```
mkdir build_dir
cd build_dir
```

Assuming that .so files and headers are at ``/usr/lib/ascend/acllib`` (as explained [here](cann_guidelines.md)):
```
cmake ..
```

If they are somewhere else, you need to specify the directory, for example:
```
cmake -Dacl_root=/usr/local/Ascend/ascend-toolkit/latest/x86_64-linux_gcc7.3.0/acllib ..
```

If you have messed up your system's python installation with various python versions (as explained above), you should tell cmake, against which python version it should generate the makefiles (please use your system's **default python version**), for example:
```
cmake -Dpython_command=python3.6 ..
```

### 4. Production Install

Create a debian package and install it:
```
make package
sudo dpkg -i valkka_acl-*
```
## Usage

### A. Video Decoder

All python bindings appear under ``valkka.acl`` namespace.  Test that everything is ok with:
```
python3 -c 'from valkka.acl import *'
```
Or (**) define explicitly your python version:
```
python3.6 -c 'from valkka.acl import *'
```

CANN accelerated decoding is performed with ``ACLThread``:
```
from valkka.acl.core import ACLThread
```

``ACLThread`` is a drop-in replacement for ``AVThread``, with which you should be very familiar if you have done your [libValkka tutorial](https://elsampsa.github.io/valkka-examples/_build/html/lesson_1.html).

``ACLThread`` tries to use the CANN video decoders.  If it fails for some reason, it defaults back to the normal ffmpeg/libav-based decoder.

For a simple example using an rtsp camera, look [here](examples/video).

### B. Inference

Using CANN inference works now like this:
```
from valkka.acl.api2 import ACLPredictor

p = ACLPredictor("yolov3_caffe_416_no_csc.om")
output_list = p.getArrayList() # model output appears here

a1 = img # input RGB24 image as 1D byte buffer
a2 = np.frombuffer(np.array([ # four floats
    416, 416, 416, 416
    ],dtype=np.single).tobytes(), dtype=np.uint8)

success = p.predict([a1, a2])

# results are now in these numpy arrays:
r1 = output_list[0]
r2 = output_list[1]
# user needs to transform byte buffers to sensible results (floats, etc.)
```

As you can see, in this API, the user has to worry *only* about the correct formatting of the input and output ACL byte buffers at the python side.  All dirty details run at the C++ level.

It get's even more neat once you subclass ``valkka.acl.api2.ACLPredictor`` and define the data transformations your code will look like this:
```
predictor = MyPredictor("your_file.om")
results = predictor(img)
```

For an example case, please see [this folder](examples/predictor/yolov3).

## Developer Zone

*only if you need to do development on this module*

### Development Install

First, copy some files into the ``build_dir`` with
```
cp ../tools/build/test_env.bash .
cp ../tools/build/set_test_streams.bash .
```

Edit the file ``set_test_streams.bash`` to include your default test rtsp cameras.  Then run:
```
source test_env.bash
source set_test_streams.bash
```

Now you can run the test programs:
```
bin/dectest 1 1
```

Or even better:
```
valgrind bin/dectest 1 1
```

To test the python interface, 
```
cd ../python
source test_env.bash
python3 quicktest.py
```
(or ``python3.6 quicktest.py`` or similar as discussed above).

# Authors

Tianyu Zhou

Sampsa Riikonen

# Maintainer

Tianyu Zhou

# Copyright

(c) Copyright 2021 Tianyu Zhou

# License

MIT