# Janus-Pro-1B-ACL-Collaboration **Repository Path**: tanlonghui/JanusPro1B-ACL ## Basic Information - **Project Name**: Janus-Pro-1B-ACL-Collaboration - **Description**: No description available - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 1 - **Created**: 2025-03-31 - **Last Updated**: 2025-03-31 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Janus-Pro-1B-ACL-Collaboration ## 介绍 本仓在香橙派AIPro设备基于ACL离线推理实现Janus-Pro-1B的功能适配,欢迎大家一起进行功能的完善和性能优化 tips:以下运行命令基于root用户 ## 版本配套 | 组件 | 版本 | | - | - | | CANN | 8.1.RC1 | | Python | 3.9 | | HDK | 23.0.0 | | 硬件 | OrangePi AiPro 20T(24G)/OrangePi AiPro 8T(16G) | ## 一、准备运行环境 ### 1.1 CANN安装 ```shell # CANN包卸载 rm -rf /usr/local/Ascend/ascend-toolkit/* # CANN包下载 wget https://mindie.obs.cn-north-4.myhuaweicloud.com/xiangchengpai_20250211/Ascend-cann-toolkit_8.1.RC1_linux-aarch64.run wget https://mindie.obs.cn-north-4.myhuaweicloud.com/xiangchengpai_20250211/Ascend-cann-kernels-310b_8.1.RC1_linux-aarch64.run # 增加软件包可执行权限 chmod +x ./Ascend-cann-toolkit_8.1.RC1_linux-aarch64.run chmod +x ./Ascend-cann-kernels-310b_8.1.RC1_linux-aarch64.run # 安装 ./Ascend-cann-toolkit_8.1.RC1_linux-aarch64.run --install --force ./Ascend-cann-kernels-310b_8.1.RC1_linux-aarch64.run --install # 设置环境变量 source /usr/local/Ascend/ascend-toolkit/set_env.sh ``` ### 1.2 Janus库安装 ``` git clone https://gitee.com/tanlonghui/janus.git #Janus 安装 cd janus pip3 install -e . ``` ### 1.3 ais_bench安装 ```shell wget https://aisbench.obs.myhuaweicloud.com/packet/ais_bench_infer/0.0.2/ait/aclruntime-0.0.2-cp39-cp39-linux_aarch64.whl wget https://aisbench.obs.myhuaweicloud.com/packet/ais_bench_infer/0.0.2/ait/ais_bench-0.0.2-py3-none-any.whl pip3 install aclruntime-0.0.2-cp39-cp39-linux_aarch64.whl pip3 install ais_bench-0.0.2-py3-none-any.whl ``` Tips:非Python3.9版本请前往[ais_bench工具](https://gitee.com/ascend/tools/tree/master/ais-bench_workload/tool/ais_bench#https://gitee.com/link?target=https%3A%2F%2Faisbench.obs.myhuaweicloud.com%2Fpacket%2Fais_bench_infer%2F0.0.2%2Fait%2Fais_bench-0.0.2-py3-none-any.whl)页面自行下载 ### 1.4 相关依赖安装 安装本仓库目录下的依赖 ```shell pip3 install -r ${此代码仓库所在路径}/JanusPro1B-ACL/requirements.txt ``` ### 1.5 自定义算子包编译 ``` cd {$此代码仓库路径}/JanusPro1B-ACL/AddCustom ./build.sh cd build_out ./custom_opp_ubuntu_aarch64.run ``` ## 二、模型获取 ### 2.1 下载ONNX模型 ```shell cd {$此代码仓库路径}/JanusPro1B-ACL/models wget https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/orangepi/JanusPro1B/vision.onnx wget https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/orangepi/JanusPro1B/Janus-LLM.onnx wget https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/orangepi/JanusPro1B/Janus-Embedding.onnx ``` ONNX的具体导出步骤可参考[ONNX导出(基于300I Pro)](export/README.md) ### 2.2 环境变量设置 ``` export MAX_COMPILE_CORE_NUMBER=1 export TE_PARALLEL_COMPILER=1 ``` ### 2.3 ATC转换 以下命令为OrangePi AiPro 20T(24G)转换命令,OrangePi AiPro 8T(16G)设备请更改soc_version=Ascend310B4 ```shell atc --model Janus-LLM.onnx --soc_version=Ascend310B1 --output Janus-LLM --framework 5 --input_shape="attention_mask:1,-1;position_ids:1,-1;past_key_values:24,2,1,16,-1,128;input_embeds:1,-1,2048" --dynamic_dims="1025,1,1024,1;1600,576,1024,576" --precision_mode=must_keep_origin_dtype --input_format=ND ``` ``` shell atc --model Janus-Embedding.onnx --framework 5 --soc_version Ascend310B1 --output Janus-Embedding --input_shape "input_ids:1,-1" --precision_mode=must_keep_origin_dtype ``` ```shell atc --model=vision.onnx --framework=5 --soc_version=Ascend310B1 --output=vision ``` ## 三、执行推理 ### 3.1 推理脚本启动 ``` python3 main.py cli ``` 若如下图提示出现init success ,则启动成功 ![输入图片说明](images/init_success.png) ### 3.2 输入文字和图片路径 ``` # 输入文本 what's this picture describe? # 输入图片路径（绝对路径） images/doge.png ```