From e5ee1e6fed33875f9439e1a9939275d3f4780ca5 Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Wed, 1 Feb 2023 16:28:22 +0800
Subject: [PATCH 1/7] Match-id-1d86bb5297856d42a687407edacf5e3019d28d5a

---
 README.md | 45 +++++++++++++++++++++++++--------------------
 1 file changed, 25 insertions(+), 20 deletions(-)

diff --git a/README.md b/README.md
index c95a1f1..264f758 100644
--- a/README.md
+++ b/README.md
@@ -1,17 +1,17 @@
 # Ascend Docker Runtime.zh
-[组件介绍](##"组件介绍")
-[编译Ascend Docker Runtime](##"编译Ascend Docker Runtime")
-[组件安装](##"组件安装")
-[更新日志](##"更新日志")
+-  **[组件介绍](#"组件介绍.md")**
+-  **[编译Ascend Docker Runtime](#"编译Ascend-Docker-Runtime.md")**
+-  **[组件安装](#"组件安装")**
+-  **[更新日志](#"更新日志")**
 
-## 组件介绍
+<h2 id="组件介绍.md">组件介绍</h2>
 容器引擎插件（Ascend Docker，又叫昇腾容器）是CANN的基础组件，为所有的AI训练/推理作业提供Ascend NPU（昇腾处理器）容器化支持，使用户AI作业能够以Docker容器的方式平滑运行在昇腾设备之上，如图1-1所示。
 
 图1-1 Ascend Docker
 
 ![image](assets/20210329102949456.png)
 
-### Ascend Docker设计简介
+<h3 id="Ascend Docker设计简介.md">Ascend Docker设计简介</h3>
 
 Ascend Docker本质上是基于OCI标准实现的Docker Runtime，不修改Docker引擎，对Docker以插件方式提供Ascend NPU适配功能。
 如图1-2所示，Ascend Docker通过OCI接口与原生Docker对接。在原生Docker的runc启动容器过程中，会调用prestart-hook对容器进行配置管理。
@@ -26,37 +26,42 @@ Ascend Docker在prestart-hook这个钩子函数中，对容器做了以下配置
 2.在Host上配置该容器的device cgroup，确保该容器只可以使用指定的NPU，保证设备的隔离。
 3.将Host上的CANN Runtime Library挂载到容器的namespace。
 
-***
-## 编译Ascend Docker Runtime
+<h2 id="编译Ascend-Docker-Runtime.md">编译Ascend Docker Runtime</h2>
 执行以下步骤进行编译
-```shell
-# 1、下载Ascend Docker Runtime源码
-git clone https://gitee.com/ascend/ascend-docker-runtime.git
 
-# 2、下载安全函数库
-cd ascend-docker-runtime/platform
-git clone https://gitee.com/openeuler/libboundscheck.git
+ 1、下载源码包，获得ascend-docker-runtime
+ 
+示例：源码放在/home/test/ascend-docker-runtime目录下
+
+ 2、下载tag为v1.1.10的安全函数库
+````shell
+cd /home/test/ascend-docker-runtime
+git clone -b v1.1.10 https://gitee.com/openeuler/libboundscheck.git
+````
 
-# 3、下载makeself
+3、下载makeself
+```shell
 cd ../opensource
 git clone https://gitee.com/src-openeuler/makeself.git
 tar -zxvf makeself/makeself-2.4.2.tar.gz
-
-# 4、编译
+```
+ 4、编译
+```shell
 cd ../build
 bash build.sh
 ```
 编译完成后，会在output文件夹看到相应的二进制run包
 ```shell
-root@#:ascend-docker-runtime/output# ll
+root@#:/home/test/ascend-docker-runtime/output# ll
 ...
 -rwxr-xr-x  ... Ascend-docker-runtime_3.0.0_linux-x86_64.run*
 ```
 
-## 组件安装
+<h2 id="组件安装.md">组件安装</h2>
 请参考[《MindX DL用户指南》](https://www.hiascend.com/software/mindx-dl)中的“集群调度用户指南 > 安装部署指导 > 安装集群调度组件 > 典型安装场景 > 集群调度场景”进行。
 
-## 更新日志
+<h2 id="更新日志.md">更新日志</h2>
+
 |   版本   | 发布日期 | 修改说明  |
 |:------:|:----:|:-----:|
 | v3.0.0 | 2023-01-18 | 第一次发布 |
-- 
Gitee


From 672fc29003c08ed19eb7e9dc756510d4ac4020c5 Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Mon, 6 Feb 2023 10:48:03 +0800
Subject: [PATCH 2/7] Match-id-1893aa1122a06aff80c10e76177513729a03ece2

---
 README.md | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/README.md b/README.md
index 264f758..4fa9122 100644
--- a/README.md
+++ b/README.md
@@ -1,19 +1,19 @@
 # Ascend Docker Runtime.zh
--  **[组件介绍](#"组件介绍.md")**
--  **[编译Ascend Docker Runtime](#"编译Ascend-Docker-Runtime.md")**
--  **[组件安装](#"组件安装")**
--  **[更新日志](#"更新日志")**
+-  **[组件介绍](#组件介绍)**
+-  **[编译Ascend-Docker-Runtime](#编译Ascend-Docker-Runtime)**
+-  **[组件安装](#组件安装)**
+-  **[更新日志](#更新日志)**
 
-<h2 id="组件介绍.md">组件介绍</h2>
+# 组件介绍
 容器引擎插件（Ascend Docker，又叫昇腾容器）是CANN的基础组件，为所有的AI训练/推理作业提供Ascend NPU（昇腾处理器）容器化支持，使用户AI作业能够以Docker容器的方式平滑运行在昇腾设备之上，如图1-1所示。
 
 图1-1 Ascend Docker
 
 ![image](assets/20210329102949456.png)
 
-<h3 id="Ascend Docker设计简介.md">Ascend Docker设计简介</h3>
+## 设计简介
 
-Ascend Docker本质上是基于OCI标准实现的Docker Runtime，不修改Docker引擎，对Docker以插件方式提供Ascend NPU适配功能。
+Ascend Docker Runtime本质上是基于OCI标准实现的Docker Runtime，不修改Docker引擎，对Docker以插件方式提供Ascend NPU适配功能。
 如图1-2所示，Ascend Docker通过OCI接口与原生Docker对接。在原生Docker的runc启动容器过程中，会调用prestart-hook对容器进行配置管理。
 
 图1-2 Docker适配原理
@@ -26,12 +26,15 @@ Ascend Docker在prestart-hook这个钩子函数中，对容器做了以下配置
 2.在Host上配置该容器的device cgroup，确保该容器只可以使用指定的NPU，保证设备的隔离。
 3.将Host上的CANN Runtime Library挂载到容器的namespace。
 
-<h2 id="编译Ascend-Docker-Runtime.md">编译Ascend Docker Runtime</h2>
+# 编译Ascend-Docker-Runtime
 执行以下步骤进行编译
 
- 1、下载源码包，获得ascend-docker-runtime
+ 1、下载tag为sync-dev的源码包，获得ascend-docker-runtime
  
 示例：源码放在/home/test/ascend-docker-runtime目录下
+```shell
+git clone -b sync-dev https://gitee.com/ascend/ascend-docker-runtime.git
+```
 
  2、下载tag为v1.1.10的安全函数库
 ````shell
@@ -57,10 +60,10 @@ root@#:/home/test/ascend-docker-runtime/output# ll
 -rwxr-xr-x  ... Ascend-docker-runtime_3.0.0_linux-x86_64.run*
 ```
 
-<h2 id="组件安装.md">组件安装</h2>
+# 组件安装
 请参考[《MindX DL用户指南》](https://www.hiascend.com/software/mindx-dl)中的“集群调度用户指南 > 安装部署指导 > 安装集群调度组件 > 典型安装场景 > 集群调度场景”进行。
 
-<h2 id="更新日志.md">更新日志</h2>
+# 更新日志
 
 |   版本   | 发布日期 | 修改说明  |
 |:------:|:----:|:-----:|
-- 
Gitee


From 34ab0092af08c17fd8f0f5700ee315d524516725 Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Mon, 9 Jan 2023 21:05:30 +0800
Subject: [PATCH 3/7] Match-id-85d28edeface7978c7aa5ca4cedd283b38de5246

---
 build/scripts/help.info    |  3 ++-
 build/scripts/run_main.sh  | 18 +++++++++++++++++-
 build/scripts/uninstall.sh |  9 ++++++++-
 runtime/main.go            |  7 +++++++
 4 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/build/scripts/help.info b/build/scripts/help.info
index 31fe6e0..97e5e23 100644
--- a/build/scripts/help.info
+++ b/build/scripts/help.info
@@ -2,4 +2,5 @@
   --install-path                    Specify the installation path (default: /usr/local/Ascend/Ascend-Docker-Runtime)
   --uninstall                       Uninstall the installed ascend-docker-runtime tool
   --upgrade                         Upgrade the installed ascend-docker-runtime tool
-  --install-type=<type>             Only A500, A200ISoC and A200 need to specify the installation type of Ascend-docker-runtime (eg: --install-type=A500)
\ No newline at end of file
+  --install-type=<type>             Only A500, A200ISoC and A200 need to specify the installation type of Ascend-docker-runtime (eg: --install-type=A500)
+  --ce=<ce>                         Only iSula need to specify the container engine(eg: --ce=isula)
\ No newline at end of file
diff --git a/build/scripts/run_main.sh b/build/scripts/run_main.sh
index 1e6ea5f..062272b 100644
--- a/build/scripts/run_main.sh
+++ b/build/scripts/run_main.sh
@@ -123,7 +123,7 @@ function uninstall()
         exit 0
     fi
 
-    ${INSTALL_PATH}/script/uninstall.sh
+    ${INSTALL_PATH}/script/uninstall.sh ${ISULA}
     echo 'remove daemon.json setting success'
 
     [ -n "${INSTALL_PATH}" ] && rm -rf ${INSTALL_PATH}
@@ -191,6 +191,7 @@ a500=n
 a200=n
 a200isoc=n
 quiet_flag=n
+ISULA=n
 
 while true
 do
@@ -236,6 +237,21 @@ do
             UPGRADE_FLAG=y
             shift
             ;;
+        --ce=*)
+            if [ "${ISULA}" == "y" ]; then
+              echo "warning: Repeat parameter!"
+              exit 1
+            fi
+
+            if [ "$3" == "--ce=isula" ]; then
+              DOCKER_CONFIG_DIR="/etc/isulad"
+              ISULA=y
+            else
+              echo "ERROR :Please check the parameter of --ce=<ce>"
+              exit 1
+            fi
+            shift
+            ;;
         --install-type=*)
             if [ "${a500}" == "y" ] || [ "${a200}" == "y" ] || [ "${a200isoc}" == "y" ]; then
                 echo "warning :Repeat parameter!"
diff --git a/build/scripts/uninstall.sh b/build/scripts/uninstall.sh
index 4c320d5..c32723f 100644
--- a/build/scripts/uninstall.sh
+++ b/build/scripts/uninstall.sh
@@ -20,7 +20,14 @@ LOG_FILE="/var/log/ascend_seclog/ascend_toolbox_install.log"
 echo "Ascend-Docker-Runtime" $(date +%Y%m%d-%H:%M:%S) "start uninstall"
 echo "Ascend-Docker-Runtime" $(date +%Y%m%d-%H:%M:%S) "start uninstall"  >>${LOG_FILE}
 ROOT=$(cd $(dirname $0); pwd)/..
-DST='/etc/docker/daemon.json'
+if [ "$*" == "y" ] ; then
+  DST='/etc/isulad/daemon.json'
+  echo "[INFO]: You will recover iSula's daemon"
+else
+  DST='/etc/docker/daemon.json'
+  echo "[INFO]: You will recover Docker's daemon"
+fi
+
 SRC="${DST}.${PPID}"
 ASCEND_RUNTIME_CONFIG_DIR=/etc/ascend-docker-runtime.d
 
diff --git a/runtime/main.go b/runtime/main.go
index d173a67..e78bc19 100644
--- a/runtime/main.go
+++ b/runtime/main.go
@@ -249,6 +249,13 @@ func modifySpecFile(path string) error {
 		return fmt.Errorf("failed to read oci spec file %s: %v", path, err)
 	}
 
+	if err = jsonFile.Truncate(0); err != nil {
+		return fmt.Errorf("failed to truncate: %v", err)
+	}
+	if _, err = jsonFile.Seek(0, 0); err != nil {
+		return fmt.Errorf("failed to seek: %v", err)
+	}
+
 	var spec specs.Spec
 	if err = json.Unmarshal(jsonContent, &spec); err != nil {
 		return fmt.Errorf("failed to unmarshal oci spec file %s: %v", path, err)
-- 
Gitee


From abc486665348cef8af044026a949e60ba3f62e7f Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Sat, 11 Feb 2023 15:37:31 +0800
Subject: [PATCH 4/7] Match-id-d63746a2593d23d7ab3b338e979ef38cb1963697

---
 runtime/main.go | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/runtime/main.go b/runtime/main.go
index df1f5f9..76671fe 100644
--- a/runtime/main.go
+++ b/runtime/main.go
@@ -46,6 +46,10 @@ const (
 	dockerRuncFile      = "docker-runc"
 	runcFile            = "runc"
 	envLength           = 2
+
+	// ENV for device-plugin to identify ascend-docker-runtime
+	useAscendDocker = "ASCEND_DOCKER_RUNTIME=True"
+	devicePlugin    = "ascend-device-plugin"
 )
 
 var (
@@ -134,6 +138,21 @@ var execRunc = func() error {
 	return nil
 }
 
+func addEnvToDevicePlugin(spec *specs.Spec) {
+	if spec.Process.Env == nil {
+		return
+	}
+
+	for _, line := range spec.Process.Env {
+		words := strings.Split(line, "=")
+		if len(words) == envLength && strings.TrimSpace(words[0]) == "HOSTNAME" &&
+			strings.Contains(words[1], devicePlugin) {
+			spec.Process.Env = append(spec.Process.Env, useAscendDocker)
+			break
+		}
+	}
+}
+
 func addHook(spec *specs.Spec) error {
 	currentExecPath, err := os.Executable()
 	if err != nil {
@@ -258,6 +277,8 @@ func modifySpecFile(path string) error {
 		return fmt.Errorf("failed to inject hook: %v", err)
 	}
 
+	addEnvToDevicePlugin(&spec)
+
 	jsonOutput, err := json.Marshal(spec)
 	if err != nil {
 		return fmt.Errorf("failed to marshal OCI spec file: %v", err)
-- 
Gitee


From 206bf0a3968e372bbdda60e217faba8d6a93f5f0 Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Thu, 16 Feb 2023 19:45:49 +0800
Subject: [PATCH 5/7] Match-id-c911fbb442ba6616d9e4d439886b7f64268e3b40

---
 README.md                  | 6 ++++--
 build/build.sh             | 9 ++++-----
 build/scripts/run_main.sh  | 8 ++++----
 build/scripts/uninstall.sh | 2 +-
 4 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/README.md b/README.md
index 4fa9122..6a8604f 100644
--- a/README.md
+++ b/README.md
@@ -38,15 +38,17 @@ git clone -b sync-dev https://gitee.com/ascend/ascend-docker-runtime.git
 
  2、下载tag为v1.1.10的安全函数库
 ````shell
-cd /home/test/ascend-docker-runtime
+cd /home/test/ascend-docker-runtime/platform
 git clone -b v1.1.10 https://gitee.com/openeuler/libboundscheck.git
 ````
 
 3、下载makeself
 ```shell
 cd ../opensource
-git clone https://gitee.com/src-openeuler/makeself.git
+git clone -b openEuler-22.03-LTS https://gitee.com/src-openeuler/makeself.git
 tar -zxvf makeself/makeself-2.4.2.tar.gz
+git clone https://gitee.com/song-xiangbing/makeself-header.git
+cp makeself-header/makeself-header.sh makeself-release-2.4.2/
 ```
  4、编译
 ```shell
diff --git a/build/build.sh b/build/build.sh
index 6f4bbb3..7ee6891 100644
--- a/build/build.sh
+++ b/build/build.sh
@@ -18,7 +18,6 @@
 set -ex
 
 ROOT=$(cd $(dirname $0); pwd)/..
-TOP_DIR=$ROOT/..
 
 OPENSRC=${ROOT}/opensource
 PLATFORM=${ROOT}/platform
@@ -51,12 +50,12 @@ RUNTIMESRCDIR=${RUNTIMESRCPATH%/${RUNTIMESRCNAME}}
 
 PACKAGENAME="Ascend-docker-runtime"
 
-VERSION="v3.0.0"
-version_file="${TOP_DIR}"/service_config.ini
+VERSION="5.0.RC1"
+version_file="${ROOT}"/service_config.ini
 if  [ -f "$version_file" ]; then
-  line=$(sed -n '4p' "$version_file" 2>&1)
+  line=$(sed -n '1p' "$version_file" 2>&1)
   #cut the chars after ':'
-  VERSION=${line#*:}
+  VERSION=${line#*=}
 fi
 
 CPUARCH=$(uname -m)
diff --git a/build/scripts/run_main.sh b/build/scripts/run_main.sh
index 95673c5..87d60a0 100644
--- a/build/scripts/run_main.sh
+++ b/build/scripts/run_main.sh
@@ -27,7 +27,7 @@ function save_install_args() {
         rm "${INSTALL_PATH}"/ascend_docker_runtime_install.info
     fi
     {
-      echo -e "version=${PACKAGE_VERSION}"
+      echo -e "version=v${PACKAGE_VERSION}"
       echo -e "arch=$(uname -m)"
       echo -e "os=linux"
       echo -e "path=${INSTALL_PATH}"
@@ -109,13 +109,13 @@ function install()
     echo 'create damom.json success'
     save_install_args
     echo "[INFO]: Ascend Docker Runtime has been installed in: ${INSTALL_PATH}"
-    echo "[INFO]: The version of Ascend Docker Runtime is: ${PACKAGE_VERSION}"
+    echo "[INFO]: The version of Ascend Docker Runtime is: v${PACKAGE_VERSION}"
     echo 'please reboot docker daemon to take effect'
 }
 
 function uninstall()
 {
-    echo "[INFO]: Uninstalling ascend docker runtime ${PACKAGE_VERSION}"
+    echo "[INFO]: Uninstalling ascend docker runtime v${PACKAGE_VERSION}"
 
     if [ ! -d "${INSTALL_PATH}" ]; then
         echo 'WARNING: the specified install path does not exist, skipping'
@@ -179,7 +179,7 @@ function upgrade()
 
     echo "[INFO]: Ascend Docker Runtime has been installed in: ${INSTALL_PATH}"
     echo '[INFO]: upgrade ascend docker runtime success'
-    echo "[INFO]: The version of Ascend Docker Runtime is: ${PACKAGE_VERSION}"
+    echo "[INFO]: The version of Ascend Docker Runtime is: v${PACKAGE_VERSION}"
 }
 
 INSTALL_FLAG=n
diff --git a/build/scripts/uninstall.sh b/build/scripts/uninstall.sh
index 4c320d5..6ba1d46 100644
--- a/build/scripts/uninstall.sh
+++ b/build/scripts/uninstall.sh
@@ -16,7 +16,7 @@
 # ============================================================================
 
 set -e
-LOG_FILE="/var/log/ascend_seclog/ascend_toolbox_install.log"
+LOG_FILE="/var/log/ascend-docker-runtime/installer.log"
 echo "Ascend-Docker-Runtime" $(date +%Y%m%d-%H:%M:%S) "start uninstall"
 echo "Ascend-Docker-Runtime" $(date +%Y%m%d-%H:%M:%S) "start uninstall"  >>${LOG_FILE}
 ROOT=$(cd $(dirname $0); pwd)/..
-- 
Gitee


From 8c14acc04ae233d952860132d9fe61d68272f0a5 Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Mon, 20 Feb 2023 10:39:40 +0800
Subject: [PATCH 6/7] Match-id-de738d3dd39ac2980d55310ebd94554258329573

---
 build/scripts/run_main.sh  | 6 +++---
 build/scripts/uninstall.sh | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/build/scripts/run_main.sh b/build/scripts/run_main.sh
index d863f0b..87025d1 100644
--- a/build/scripts/run_main.sh
+++ b/build/scripts/run_main.sh
@@ -190,7 +190,7 @@ a500=n
 a200=n
 a200isoc=n
 quiet_flag=n
-ISULA=n
+ISULA=none
 
 while true
 do
@@ -237,14 +237,14 @@ do
             shift
             ;;
         --ce=*)
-            if [ "${ISULA}" == "y" ]; then
+            if [ "${ISULA}" == "isula" ]; then
               echo "warning: Repeat parameter!"
               exit 1
             fi
 
             if [ "$3" == "--ce=isula" ]; then
               DOCKER_CONFIG_DIR="/etc/isulad"
-              ISULA=y
+              ISULA=isula
             else
               echo "ERROR :Please check the parameter of --ce=<ce>"
               exit 1
diff --git a/build/scripts/uninstall.sh b/build/scripts/uninstall.sh
index 2aa506d..058f1c6 100644
--- a/build/scripts/uninstall.sh
+++ b/build/scripts/uninstall.sh
@@ -20,7 +20,7 @@ LOG_FILE="/var/log/ascend-docker-runtime/installer.log"
 echo "Ascend-Docker-Runtime" $(date +%Y%m%d-%H:%M:%S) "start uninstall"
 echo "Ascend-Docker-Runtime" $(date +%Y%m%d-%H:%M:%S) "start uninstall"  >>${LOG_FILE}
 ROOT=$(cd $(dirname $0); pwd)/..
-if [ "$*" == "y" ] ; then
+if [ "$*" == "isula" ] ; then
   DST='/etc/isulad/daemon.json'
   echo "[INFO]: You will recover iSula's daemon"
 else
-- 
Gitee


From 8a75a4851439f5b8bc7abf7bbca7581fa419f7d7 Mon Sep 17 00:00:00 2001
From: BianTanggui <songxiangbing@huawei.com>
Date: Mon, 20 Feb 2023 16:08:03 +0800
Subject: [PATCH 7/7] Match-id-8afa2c0dd1a37e590464eadfacdd152e559a64ef

---
 build/scripts/run_main.sh | 2 +-
 cli/src/utils.c           | 3 +--
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/build/scripts/run_main.sh b/build/scripts/run_main.sh
index 87025d1..110ad24 100644
--- a/build/scripts/run_main.sh
+++ b/build/scripts/run_main.sh
@@ -110,7 +110,7 @@ function install()
     save_install_args
     echo "[INFO]: Ascend Docker Runtime has been installed in: ${INSTALL_PATH}"
     echo "[INFO]: The version of Ascend Docker Runtime is: v${PACKAGE_VERSION}"
-    echo 'please reboot docker daemon to take effect'
+    echo 'please reboot daemon and container engine to take effect'
 }
 
 function uninstall()
diff --git a/cli/src/utils.c b/cli/src/utils.c
index 605e381..94c93e0 100644
--- a/cli/src/utils.c
+++ b/cli/src/utils.c
@@ -195,8 +195,7 @@ int MakeMountPoints(const char *path, mode_t mode)
 
 static bool ShowExceptionInfo(const char* exceptionInfo)
 {
-    (void)fprintf(stderr, exceptionInfo);
-    (void)fprintf(stderr, "\n");
+    (void)fprintf(stderr, "%s\n", exceptionInfo);
     return false;
 }
 
-- 
Gitee