# LiteRT-LM

**Repository Path**: mirrors/LiteRT-LM

## Basic Information

- **Project Name**: LiteRT-LM
- **Description**:  LiteRT-LM  是用于在边缘平台上高效运行语言模型的 C++ 库
- **Primary Language**: C/C++
- **License**: Apache-2.0
- **Default Branch**: aaronndx-patch-2
- **Homepage**: https://www.oschina.net/p/litert-lm
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2025-06-13
- **Last Updated**: 2025-09-06

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# LiteRT LM
GitHub repository for Google's open-source high-performance runtime for
on-device LLM.

## How to build and run

### Linux

`clang` is used to build LiteRT LM on linux. Build `litert_lm_main`, a CLI
executable and run models on CPU.

```
bazel build //runtime/engine:litert_lm_main

bazel-bin/runtime/engine/litert_lm_main \
    --backend=cpu \
    --model_path=<model task file>
```

### Android

Install NDK r28b or newer from https://developer.android.com/ndk/downloads.
To run on CPU, only `litert_lm_main` is required.

```
export ANDROID_NDK_HOME=<ndk dir>

bazel build --config=android_arm64 //runtime/engine:litert_lm_main

adb push bazel-bin/runtime/engine/litert_lm_main /data/local/tmp

adb shell /data/local/tmp/litert_lm_main \
    --backend=cpu \
    --model_path=<model task file>
```

To run on GPU, `libLiteRtGpuAccelerator.so` is required. Download the maven
package from [TBD]()

```
adb push libLiteRtGpuAccelerator.so /data/local/tmp

adb shell LD_LIBRARY_PATH=/data/local/tmp \
    /data/local/tmp/litert_lm_main \
    --backend=gpu \
    --model_path=<model task file>
```

To run on Qualcomm NPU, `libLiteRtDispatch_Qualcomm.so` in the maven package
from [TBD]() and Qualcomm's runtime shared libraries downloaded from [TBD]()
are required. Model file should be compiled with LiteRt compiler with Qualcomm's
plugin.

```
adb push libLiteRtDispatch_Qualcomm.so /data/local/tmp
adb push <qualcomm runtime shlibs> /data/local/tmp

adb shell LD_LIBRARY_PATH=/data/local/tmp \
    /data/local/tmp/litert_lm_main \
    --backend=qnn \
    --model_path=<model tflite file>
```