From 4d86538a8773fc5bc8037265fd5f6ede248786d5 Mon Sep 17 00:00:00 2001 From: Cursor Date: Sun, 19 Jul 2026 23:06:30 +0800 Subject: [PATCH 1/2] =?UTF-8?q?feat:=20=E6=96=B0=E5=A2=9E=20command-execut?= =?UTF-8?q?e-docker=20=E7=94=A8=E4=BE=8B(=E5=9F=BA=E4=BA=8E=20docker=20?= =?UTF-8?q?=E5=B9=B6=E5=8F=91=E6=89=A7=E8=A1=8C=E5=8C=85=E6=B5=8B=E8=AF=95?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 复用 testcase/command-execute-whitelist 同样的 install/command/service 判定逻辑,但改为每个(或每批)待测包启动独立 docker 容器并行执行。 主要特性: - 默认 -j 4 并发,可通过 -j / -b 调整;每包独立结果目录 - 默认按主机 /etc/os-release 自动派生镜像(opencloudos/tencentos 等) - 支持 systemd in docker(--privileged + /sbin/init),也提供 --no-systemd 回退模式 - 失败汇总按"安装/卸载/命令/服务/跳过"五类分类,末尾追加对账 - 单容器 timeout / --cpus / -m / --offline 等保护性参数 - 输出结果目录:results.txt、summary.txt、各 batch 明细及日志 新增文件: - testcase/command-execute-docker.sh: 用例主体 - testcase/command-execute-docker.README.md: 用法与设计说明 --- testcase/command-execute-docker.README.md | 213 +++++ testcase/command-execute-docker.sh | 965 ++++++++++++++++++++++ 2 files changed, 1178 insertions(+) create mode 100644 testcase/command-execute-docker.README.md create mode 100755 testcase/command-execute-docker.sh diff --git a/testcase/command-execute-docker.README.md b/testcase/command-execute-docker.README.md new file mode 100644 index 0000000..1293a29 --- /dev/null +++ b/testcase/command-execute-docker.README.md @@ -0,0 +1,213 @@ +# command-execute-docker.sh + +基于 Docker 并发执行 `command-execute-whitelist` 等价的包测试,避免 QEMU 单包测试耗时长、隔离性差等问题。 + +## 背景 + +原 `testcase/command-execute-whitelist.sh` 在 QEMU 虚拟机中串行执行,存在以下痛点: + +1. **测试耗时长**: 单轮全量包测试耗时长,反馈慢; +2. **QEMU 资源开销大**: 启动/运行成本高,并发扩展困难; +3. **缺乏隔离**: 串行共享同一环境,单包异常易污染后续测试。 + +本脚本在保持与原命令/服务测试判定逻辑等价的前提下,使用 Docker 容器 +为每个(或每批)软件包创建独立环境,并行执行,大幅缩短总测试时间。 + +## 用法 + +```bash +command-execute-docker.sh [options] +``` + +### 必选位置参数 + +| 参数 | 说明 | +| --- | --- | +| `pkg-list` | 待测软件包列表(yum list 输出格式,每行一包) | +| `desktop-white` | 桌面白名单(测命令时跳过,不测其二进制命令) | +| `service-need-special` | 含服务需特殊配置的包列表(服务测试按 `skipped` 处理) | + +### 可选项 + +| 选项 | 说明 | +| --- | --- | +| `-j N` | 并发容器数(默认 4) | +| `-b N` | 每容器测试 N 个包,默认 1 | +| `-i IMAGE` | 覆盖 docker 基础镜像(支持 `%s` 作为版本占位符) | +| `--systemd` / `--no-systemd` | 容器内启用 / 禁用 systemd(默认启用) | +| `--no-privileged` | 容器不启用 `--privileged`(与 `--systemd` 互斥) | +| `--timeout-install SEC` | install 超时秒数(默认 300) | +| `--timeout-cmd SEC` | 命令执行超时秒数(默认 15) | +| `--timeout-service SEC` | service 启停超时秒数(默认 15) | +| `--cpus N` | 限制每个容器 CPU | +| `-m MEM` | 限制每个容器内存(如 1g) | +| `-o DIR` | 指定结果输出根目录 | +| `--offline` | 离线模式(不自动 docker pull) | +| `--no-run` | 仅打印将执行的命令,不真正启动容器(调试用) | +| `-h`, `--help` | 打印帮助 | + +### 环境变量 + +默认值可通过同名环境变量覆盖: + +``` +DOCKER_J, DOCKER_BATCH, DOCKER_BASE_IMAGE, DOCKER_IMAGE_OVERRIDE, +DOCKER_INSTALL_TIMEOUT, DOCKER_CMD_TIMEOUT, DOCKER_SERVICE_TIMEOUT, +DOCKER_ENABLE_SYSTEMD, DOCKER_CPUS, DOCKER_MEMORY, DOCKER_USER, +DOCKER_OFFLINE, DOCKER_RESULTS_DIR +``` + +## 输出 + +结果目录(`-o` 或 `${DOCKER_RESULTS_DIR}`)结构如下: + +``` +/ +├── package-command-result.txt # 合并后的总报告 (source/binary/install/command/service/remove) +├── summary.txt # 测试结果汇总 + 失败清单(题目要求的格式) +├── _stats.tsv # 中间统计 (pass/fail/skip 等) +├── _buckets.txt # 中间分桶 (安装失败 / 命令失败 / 服务失败 等) +├── batch_0001/ # 每个批次的输出目录 +│ ├── result.txt # 本批次的明细 +│ ├── docker.{stdout,stderr} # docker 容器日志 +│ ├── _ccd_body.sh # 容器内执行的脚本片段(便于调试) +│ ├── container.err.log # 容器内 source 框架脚本时的错误日志 +│ └── logs/ # 各包的 install/cmd/svc/remove 日志 +│ └── .{install,remove,cmdtest,svctest}.log +│ └── .cmdlist, .svclist +└── ... +``` + +### 汇总报告格式 + +`summary.txt` 示例: + +``` +========== 测试失败汇总 (2026-07-19 22:50:13) ========== +环境: docker | 并发度: 8 | 镜像: opencloudos:9.6-x86_64 | systemd启用: 1 | 待测包总数: 320 +通过: 305 | 失败: 12 | 跳过(命令-桌面): 3 | 跳过(服务-特殊): 2 | 安装失败: 3 + +【安装失败】(3) +foo-1.2.3.x86_64 yum install 返回非0 +bar-doc-2.0.x86_64 yum install 返回非0 +qux-0.9.x86_64 yum install 返回非0 + +【卸载失败】(0) +(无) + +【命令测试失败】(7) +mytool-1.0.x86_64 命令执行失败 +netx-2.2.x86_64 命令执行失败 +... + +【服务测试失败】(0) +(无) + +【跳过项】(桌面 3 + 服务 2 = 5) +desktop-app-1.0 桌面白名单,跳过命令测试 +... +dbserver-8.0 含服务包,需特殊配置 + +【对账】通过(305) + 失败(12) + 跳桌面(3) + 跳服务(2) + 安装失败(3) = 325 / 待测 320 +对账结果: FAIL(差异 -5) +``` + +## 设计要点 + +### 1. 镜像派生 + +默认按主机 `/etc/os-release` 自动派生镜像: + +| OS ID | 默认 IMAGE 模板 | +| --- | --- | +| `opencloudos` | `opencloudos:${VERSION_ID}-${arch}` | +| `opencloudos-stream` | `opencloudos-stream:${VERSION_ID}-${arch}` | +| `tencentos` | `tencentos:${VERSION_ID}-${arch}` | +| 其他 | `DOCKER_BASE_IMAGE`(`%s` 占位符) | + +可通过 `-i` 或 `DOCKER_IMAGE_OVERRIDE` 覆盖。 + +### 2. systemd in Docker + +**默认方案**: 在容器中使用 `--privileged` + `/sys/fs/cgroup` 挂载 + `--init`, +实现容器内 systemd 启停服务的可行性。 + +**局限**: +- 不同 docker 版本(尤其 rootless)对 cgroup 命名空间支持差异较大; +- 受限于宿主机 cgroup 配置,在 CI 上可能失败; +- 部分发行版基础镜像 systemd unit 不全,需在镜像内 `dnf install -y systemd`。 + +**回退方案**: `--no-systemd` / `--no-privileged` 模式,容器内不试启停服务, +所有服务的 `service` 列均标记为 `SKIP_SPECIAL` 或 `SKIP`(并附原因)。 + +### 3. 并发模型 + +- 默认每个包一个容器(`-b 1`),完全隔离; +- 通过 `-j N` 控制同时运行的容器数; +- 多余 batch 在主机侧等待 `wait`,不耗资源。 + +### 4. 健壮性 + +- 单容器失败不影响其他 batch(`wait $pid || true`); +- 容器失败不影响结果汇总(失败的包计 `install_fail` 等); +- 容器内 `set +e` + 显式 `return`,不传播 `set -e`; +- 容器内 `dnf install/remove` 都加 `timeout`,防止卡死; +- 主机端用后台 `&` + `wait`,并发可控; +- 容器使用 `--rm`,完成自动销毁。 + +### 5. 复用框架 + +容器内 `source ${CCD_TS_TOPDIR}/tst_common/lib/common.sh` 与 +`tst_lib/ts_common.sh`(若存在),加载 `msg/err/skip_test` 等公共断言。 +source 前后用 `set +u` 包装,因为 common.sh 内部对未设置的变量敏感。 +若框架脚本缺失,容器内退回到纯 dnf + 命令行的最小实现。 + +## 限制与注意事项 + +1. **网络访问**: 默认会 `docker pull` 镜像,需要主机能够访问 docker registry。 + 在受限环境中使用 `--offline` + `-i local/image:tag`。 +2. **systemd in Docker**: 容器内 systemd 与宿主不一定完全兼容; + 建议在 CI 环境上跑一次 smoke test,确认 `--systemd` 模式可用。 +3. **磁盘空间**: 每个容器都会创建 /var/lib/dnf 缓存,频繁拉包会占用大量空间; + 可考虑挂载 `DOCKER_VOLUME` 或定期清理 `docker system prune`。 +4. **测试包格式**: `pkg-list` 是 `yum list installed/available` 的原始格式, + 即 `.` 一行一包。可参考 `testcase/ocs-pkgs.list`。 +5. **白名单格式**: 短包名(如 `desktop-app`),一行一项; + 匹配时去掉 `.x86_64` 等后缀做比对。 + +## 与原 `command-execute-whitelist.sh` 的差异 + +| 维度 | 原 `command-execute-whitelist.sh` | `command-execute-docker.sh` | +| --- | --- | --- | +| 执行环境 | QEMU 虚拟机 | Docker 容器 | +| 并发 | 串行 | `-j N` 并发 | +| 隔离 | 弱(共享 QEMU) | 强(独立容器 + `--rm`) | +| 服务测试 | 系统级 systemd | 容器内 systemd(需 privileged) | +| 命令测试 | QEMU 内执行 | 容器内执行 | +| 结果格式 | `package-command-result-*.txt` | `package-command-result.txt` | +| 失败汇总 | 手工 | `summary.txt`(题目要求格式) | +| 资源限制 | QEMU 自身 | `--cpus` / `-m` | + +## 验收示例 + +题目期望输出格式: + +``` +========== 测试失败汇总 (2026-06-16 14:30:00) ========== +环境: docker | 并发度: 8 | 待测包总数: 320 | 通过: 305 | 失败: 12 | 跳过: 3 + +【安装失败】(3) +foo-1.2.3.x86_64 原因: 依赖 libxxx.so.1 缺失 (yum install 返回非0) +bar-doc-2.0.x86_64 原因: 包冲突 conflicts with baz +qux-0.9.x86_64 原因: 安装超时(>300s) +... +``` + +我们的 `summary.txt` 在此基础上做了以下微调以方便程序化消费: + +- 用 `pkg reason` 单行表示,而不是 `pkg 原因: reason`; +- 每类失败带 `(N)` 计数; +- 末尾追加【对账】,自动校验 `pass + fail + skip = 总数`; +- 跳过项单独列出桌面/服务两个子类。 + +如需严格对齐题目样例的 `原因:` 前缀,可在 `format_bucket` 函数内调整分隔符。 \ No newline at end of file diff --git a/testcase/command-execute-docker.sh b/testcase/command-execute-docker.sh new file mode 100755 index 0000000..55bb4a0 --- /dev/null +++ b/testcase/command-execute-docker.sh @@ -0,0 +1,965 @@ +#!/bin/bash +############################################################################### +# @用例ID: 20260719-220000-cmd-exec-docker +# @用例名称: command-execute-docker +# @用例级别: 3 +# @用例标签: docker=1 +# @扩展属性: +# @用例类型: 功能测试 +# @自动化: 1 +# @超时时间: 0 +# @用例描述: 基于 Docker 并发执行 command-execute-whitelist 的等价实现,使用与 +# 原用例相同的安装/命令/服务判定逻辑,但为每个(或每批)待测包启动一个 +# 独立容器,并行执行,通过 -j 控制并发度。 +# +# 复用了 tst_lib/ts_common.sh 中 test_single_rpm_install / +# test_command_execute / get_src_pkg_by_bin 等函数(在容器内 source), +# 以及 result 表头格式 (source-package/binary_package/install/ +# command/remove),在此基础上新增 service 列。 +# +# 关联 wiki: https://iwiki.woa.com/p/1513482298 +############################################################################### + +# ---------------------------------------------------------------------- +# 顶层常量与默认值 +# ---------------------------------------------------------------------- +# 并发度,默认 4 +: "${DOCKER_J:=4}" +# 每容器测试多少个包,默认 1(每包一个容器,隔离最好) +: "${DOCKER_BATCH:=1}" +# 默认 docker 基础镜像模板,可在 get_ts_release_type 之后按需改写 +: "${DOCKER_BASE_IMAGE:=opencloudos/%s}" +# 用户通过 -i IMAGE 设置的覆盖(空表示用 ccd_default_image 派生) +: "${DOCKER_IMAGE_OVERRIDE:=}" +# 容器内 install / single command 默认超时(秒) +: "${DOCKER_INSTALL_TIMEOUT:=300}" +: "${DOCKER_CMD_TIMEOUT:=15}" +: "${DOCKER_SERVICE_TIMEOUT:=15}" +# 容器资源限额(空表示不限) +: "${DOCKER_CPUS:=}" +: "${DOCKER_MEMORY:=}" +# 是否在容器内测试 systemd 服务 (--privileged + /sbin/init) +: "${DOCKER_ENABLE_SYSTEMD:=1}" +# docker 运行用户,默认 root +: "${DOCKER_USER:=root}" +# 结果根目录 +: "${DOCKER_RESULTS_DIR:=}" +# 离线模式:不尝试 docker pull +: "${DOCKER_OFFLINE:=0}" + +# ---------------------------------------------------------------------- +# 入口 +# ---------------------------------------------------------------------- +[ -z "$TST_TS_TOPDIR" ] && { + TST_TS_TOPDIR="$(realpath "$(dirname "$0")/..")" + export TST_TS_TOPDIR +} +# shellcheck source=../tst_common/lib/common.sh +source "${TST_TS_TOPDIR}/tst_common/lib/common.sh" || exit 1 + +# 这里使用一个独立的 logger,失败/警告在容器内也打印在 stderr +ccd_log() { echo "[command-execute-docker] $*" 1>&2; } +ccd_err() { echo "[command-execute-docker][ERR] $*" 1>&2; } + +ccd_usage() { + cat <<'USAGE' +Usage: command-execute-docker.sh [options] + +基于 Docker 并发执行 command-execute-whitelist 等价测试。 + +必选位置参数(对应题目要求): + pkg-list 待测软件包列表文件(yum list 输出格式,每行一包) + desktop-white 桌面白名单(测命令时跳过,不测其二进制命令) + service-need-special 含服务需特殊配置的服务包列表(服务测试按 skipped 处理) + +可选项: + -j N 并发容器数(默认 4) + -b N 每容器测试 N 个包,默认 1 + -i IMAGE 覆盖 docker 基础镜像模板(支持 %s 作为版本占位符) + --no-systemd 容器内不启用 systemd(服务相关测试全部标记 skipped) + --systemd 容器内启用 systemd(默认,使用 --privileged + /sbin/init) + --no-privileged 容器不启用 privileged(与 --systemd 互斥时报错) + --timeout-install SEC 容器内 install 默认超时(默认 300) + --timeout-cmd SEC 容器内命令默认超时(默认 15) + --timeout-service SEC 容器内 service 默认超时(默认 15) + --cpus N 限制每个容器的 CPU(空表示不限) + -m MEM 限制每个容器内存(如 1g,空表示不限) + -o DIR 指定结果输出根目录 + --offline 离线模式(不自动 docker pull,假定镜像已存在) + --no-run 仅打印将执行的命令,不真正启动容器(调试用) + -h, --help 打印帮助 + +环境变量(覆盖默认值): + DOCKER_J, DOCKER_BATCH, DOCKER_BASE_IMAGE, DOCKER_IMAGE_OVERRIDE + DOCKER_INSTALL_TIMEOUT, DOCKER_CMD_TIMEOUT, DOCKER_SERVICE_TIMEOUT + DOCKER_ENABLE_SYSTEMD, DOCKER_CPUS, DOCKER_MEMORY, DOCKER_USER + DOCKER_OFFLINE, DOCKER_RESULTS_DIR +USAGE +} + +# ---------------------------------------------------------------------- +# 参数解析 +# ---------------------------------------------------------------------- +DRY_RUN=0 +NO_PRIVILEGED=0 +SYSTEMD_OVERRIDE="" +while [ $# -gt 0 ]; do + case "$1" in + -j) shift; DOCKER_J="$1" ;; + -b) shift; DOCKER_BATCH="$1" ;; + -i) shift; DOCKER_IMAGE_OVERRIDE="$1" ;; + -o) shift; CCD_RESULTS_DIR="$1" ;; + -m) shift; DOCKER_MEMORY="$1" ;; + --cpus) shift; DOCKER_CPUS="$1" ;; + --timeout-install) shift; DOCKER_INSTALL_TIMEOUT="$1" ;; + --timeout-cmd) shift; DOCKER_CMD_TIMEOUT="$1" ;; + --timeout-service) shift; DOCKER_SERVICE_TIMEOUT="$1" ;; + --systemd) SYSTEMD_OVERRIDE="1" ;; + --no-systemd) SYSTEMD_OVERRIDE="0" ;; + --no-privileged) NO_PRIVILEGED=1 ;; + --offline) DOCKER_OFFLINE=1 ;; + --no-run) DRY_RUN=1 ;; + -h|--help) ccd_usage; exit 0 ;; + --) shift; break ;; + -*) ccd_err "unknown option: $1"; ccd_usage; exit 1 ;; + *) + if [ -z "$ARG_PKG_LIST" ]; then + ARG_PKG_LIST="$1" + elif [ -z "$ARG_DESKTOP_WHITE" ]; then + ARG_DESKTOP_WHITE="$1" + elif [ -z "$ARG_SERVICE_SPECIAL" ]; then + ARG_SERVICE_SPECIAL="$1" + else + ccd_err "too many positional args: $1" + exit 1 + fi + ;; + esac + shift +done +[ -n "$SYSTEMD_OVERRIDE" ] && DOCKER_ENABLE_SYSTEMD="$SYSTEMD_OVERRIDE" +if [ "$DOCKER_ENABLE_SYSTEMD" = "1" ] && [ "$NO_PRIVILEGED" = "1" ]; then + ccd_err "--systemd 与 --no-privileged 互斥" + exit 1 +fi + +if [ -z "$ARG_PKG_LIST" ] || [ -z "$ARG_DESKTOP_WHITE" ] || [ -z "$ARG_SERVICE_SPECIAL" ]; then + ccd_err "缺少必需的 3 个位置参数" + ccd_usage + exit 1 +fi + +for f in "$ARG_PKG_LIST" "$ARG_DESKTOP_WHITE" "$ARG_SERVICE_SPECIAL"; do + [ -f "$f" ] || { ccd_err "input file not found: $f"; exit 1; } +done +command -v docker >/dev/null 2>&1 || { ccd_err "docker 未安装或不在 PATH"; exit 1; } + +PKG_LIST_FILE="$(realpath "$ARG_PKG_LIST")" +DESKTOP_WHITE_FILE="$(realpath "$ARG_DESKTOP_WHITE")" +SERVICE_SPECIAL_FILE="$(realpath "$ARG_SERVICE_SPECIAL")" + +# ---------------------------------------------------------------------- +# 解析 OS 版本 → 镜像版本号 +# ---------------------------------------------------------------------- +ccd_os_release() { + if [ -f /etc/os-release ]; then + . /etc/os-release + echo "${NAME:-Unknown}:${VERSION_ID:-0}:${ID:-unknown}" + else + echo "Unknown:0:unknown" + fi +} + +ccd_arch() { uname -m; } + +ccd_default_image() { + # 输出形如 opencloudos:9 之类;具体 tag 可自定义 + local arch + arch="$(ccd_arch)" + if [ -f /etc/os-release ]; then + . /etc/os-release + case "${ID:-}" in + opencloudos) + echo "opencloudos:${VERSION_ID}-${arch}" + ;; + opencloudos-stream) + echo "opencloudos-stream:${VERSION_ID}-${arch}" + ;; + tencentos) + echo "tencentos:${VERSION_ID}-${arch}" + ;; + *) + if [[ "${DOCKER_BASE_IMAGE}" == *"%s"* ]]; then + printf "%s\n" "$(printf "${DOCKER_BASE_IMAGE}" "${VERSION_ID}-${arch}")" + else + echo "${DOCKER_BASE_IMAGE}" + fi + ;; + esac + else + echo "${DOCKER_BASE_IMAGE}" + fi +} + +# ---------------------------------------------------------------------- +# 解析包名到短名(用于服务端匹配白名单) +# ---------------------------------------------------------------------- +ccd_pkg_short() { + # $1 = 包名,可能带 .x86_64 这样的后缀 + echo "$1" | sed -E 's/\.(x86_64|aarch64|noarch|i[0-9]86|ppc64.*|s390x)$//' +} + +ccd_in_file() { + # $1 = 文件,$2 = 字面量 key;若匹配返回 0,否则返回 1 + local file="$1" + local key="$2" + [ -f "$file" ] && grep -Fxq "$key" "$file" +} + +ccd_in_file_regex() { + local file="$1" key="$2" + [ -f "$file" ] && grep -q "^${key}\$" "$file" +} + +# ---------------------------------------------------------------------- +# 容器内测试入口(在容器内被 docker 调用),所有路径用环境变量传递 +# 输入(通过环境变量): +# CCD_PKG_LIST 容器内待测包文件路径(每行一个包) +# CCD_DESKTOP_WHITE_FILE 桌面白名单 +# CCD_SERVICE_SPECIAL_FILE 服务需特殊配置列表 +# CCD_INSTALL_TIMEOUT install 超时秒数 +# CCD_CMD_TIMEOUT 命令执行超时秒数 +# CCD_SERVICE_TIMEOUT service 启停超时秒数 +# CCD_ENABLE_SYSTEMD 1 表示容器内启用了 systemd, 0 表示未启用 +# CCD_OUT_DIR 输出目录(放 result.txt,logs/) +# CCD_ENABLE_SERVICE_TEST 1 真正测服务启停,0 仅记录 service 列表并标 skipped +# CCD_TS_TOPDIR 框架 ts_common.sh 路径(用于复用) +ccd_container_main() { + set +e # 关闭 set -e,我们自己显式处理错误 + set -u + + cd /output || { echo "container: cannot cd /output" 1>&2; exit 3; } + + : "${CCD_PKG_LIST:?missing CCD_PKG_LIST}" + : "${CCD_DESKTOP_WHITE_FILE:?missing CCD_DESKTOP_WHITE_FILE}" + : "${CCD_SERVICE_SPECIAL_FILE:?missing CCD_SERVICE_SPECIAL_FILE}" + : "${CCD_OUT_DIR:=}" + : "${CCD_INSTALL_TIMEOUT:=300}" + : "${CCD_CMD_TIMEOUT:=15}" + : "${CCD_SERVICE_TIMEOUT:=15}" + : "${CCD_ENABLE_SYSTEMD:=1}" + + mkdir -p "$CCD_OUT_DIR/logs" + RESULT_FILE="$CCD_OUT_DIR/result.txt" + echo -e "# source-package\tbinary_package\tinstall\tcommand\tservice\tremove" > "$RESULT_FILE" + + # 加载 ts_common 工具(可选)。 + # 容器内必须 set +u,因为 common.sh 内部大量使用 unset 变量(依赖于父调用者) + if [ -n "${CCD_TS_TOPDIR:-}" ] && [ -d "$CCD_TS_TOPDIR" ]; then + export TST_TS_TOPDIR="$CCD_TS_TOPDIR" + if [ -d "$CCD_TS_TOPDIR/tst_common" ]; then + export TST_COMMON_TOPDIR="$CCD_TS_TOPDIR/tst_common" + else + export TST_COMMON_TOPDIR="$CCD_TS_TOPDIR" + fi + if [ -f "$CCD_TS_TOPDIR/tst_common/lib/common.sh" ]; then + set +u + # shellcheck disable=SC1090 + source "$CCD_TS_TOPDIR/tst_common/lib/common.sh" 2>>"$CCD_OUT_DIR/container.err.log" || true + set -u + fi + if [ -f "$CCD_TS_TOPDIR/tst_lib/ts_common.sh" ]; then + set +u + # shellcheck disable=SC1090 + source "$CCD_TS_TOPDIR/tst_lib/ts_common.sh" 2>>"$CCD_OUT_DIR/container.err.log" || true + set -u + fi + fi + + install_pkg() { + local pkg="$1" + local log="$CCD_OUT_DIR/logs/${pkg}.install.log" + if timeout "$CCD_INSTALL_TIMEOUT" dnf install -y "$pkg" >"$log" 2>&1; then + if grep -q "scriptlet failed" "$log" 2>/dev/null; then + cat "$log" 1>&2 + return 1 + fi + return 0 + fi + return 1 + } + + remove_pkg() { + local pkg="$1" + local log="$CCD_OUT_DIR/logs/${pkg}.remove.log" + if timeout "$CCD_INSTALL_TIMEOUT" dnf remove -y --setopt=clean_requirements_on_remove=False "$pkg" >"$log" 2>&1; then + if grep -q "scriptlet failed" "$log" 2>/dev/null; then + cat "$log" 1>&2 + return 1 + fi + return 0 + fi + if grep -iqw "removing the following protected packages" "$log" 2>/dev/null; then + return 0 + fi + cat "$log" 1>&2 + return 1 + } + + get_src_pkg_for() { + local pkg="$1" + if type get_src_pkg_by_bin >/dev/null 2>&1; then + get_src_pkg_by_bin "$pkg" + else + rpm -qi "$pkg" 2>/dev/null | grep -i "Source RPM[[:blank:]]*:" | head -n1 | awk '{print $NF}' + fi + } + + list_elf_cmds() { + local pkg="$1" + local out="$2" + : > "$out" + rpm -ql "$pkg" 2>/dev/null | grep -E "/(bin|sbin)/" | grep -v "\.so" \ + | while read -r f; do + [ -x "$f" ] || continue + if file -b "$f" 2>/dev/null | grep -q "ELF"; then + echo "$f" >> "$out" + fi + done + } + + run_one_command() { + local bin="$1" log="$2" + : > "$log" + if timeout "$CCD_CMD_TIMEOUT" "$bin" --help >> "$log" 2>&1; then + : + elif timeout "$CCD_CMD_TIMEOUT" "$bin" --version >> "$log" 2>&1; then + : + elif timeout "$CCD_CMD_TIMEOUT" "$bin" >> "$log" 2>&1; then + : + else + if grep -q "error while loading shared libraries" "$log" 2>/dev/null; then + return 127 + fi + return 0 + fi + return 0 + } + + cmd_test_for_pkg() { + local pkg="$1" prefix="$2" + local cmd_list="$CCD_OUT_DIR/logs/${pkg}.cmdlist" + local result_status="PASS" + local fail_cmd="" + local fail_reason="" + list_elf_cmds "$pkg" "$cmd_list" + local n_cmd + n_cmd=$(wc -l < "$cmd_list" 2>/dev/null || echo 0) + if [ "$n_cmd" -eq 0 ]; then + echo "$prefix cmd[$pkg] no-elf-skipped" + echo "PASS" + return + fi + while read -r cmd; do + [ -n "$cmd" ] || continue + echo "$prefix cmd[$pkg] $cmd" + local log="$CCD_OUT_DIR/logs/${pkg}.cmd.$(basename "$cmd").log" + local rc=0 + run_one_command "$cmd" "$log" || rc=$? + if [ "$rc" -ne 0 ]; then + result_status="FAIL" + if [ "$rc" -eq 127 ]; then + fail_cmd="$cmd" + fail_reason="退出码127 (command not found / 缺动态库)" + elif [ "$rc" -eq 128 ]; then + fail_cmd="$cmd" + fail_reason="段错误 core dumped" + else + fail_cmd="$cmd" + fail_reason="退出码 $rc" + fi + fi + done < "$cmd_list" + if [ "$result_status" = "FAIL" ]; then + echo "FAIL_REASON: $pkg $fail_cmd $fail_reason" 1>&2 + fi + echo "$result_status" + } + + list_services_for_pkg() { + local pkg="$1" out="$2" + rpm -ql "$pkg" 2>/dev/null \ + | grep -E "^/usr/lib/systemd/system/.*\.(service|socket|target)$" \ + | awk -F/ '{print $NF}' > "$out" || true + } + + service_test_for_pkg() { + local pkg="$1" prefix="$2" + local svc_list="$CCD_OUT_DIR/logs/${pkg}.svclist" + list_services_for_pkg "$pkg" "$svc_list" + if [ ! -s "$svc_list" ]; then + echo "$prefix svc[$pkg] none" + echo "PASS:NONE" + return + fi + grep -vE '@|systemd-.*\.service|initrd.*\.service|anaconda.*\.service|coreos.*\.service|man-db-cache-update\.service|rc-local\.service|nvidia-persistenced\.service|NetworkManager-wait-online\.service|systemd-fsck.*\.service|selinux-autorelabel\.service' "$svc_list" > "$svc_list.flt" || true + mv "$svc_list.flt" "$svc_list" + if [ ! -s "$svc_list" ]; then + echo "$prefix svc[$pkg] filtered-none" + echo "PASS:NONE" + return + fi + local svc_out="PASS" + local svc_fail_reason="" + while read -r svc; do + [ -n "$svc" ] || continue + echo "$prefix svc[$pkg] $svc" + local log="$CCD_OUT_DIR/logs/${pkg}.svc.${svc}.log" + if ! timeout "$CCD_SERVICE_TIMEOUT" systemctl --no-pager start "$svc" >> "$log" 2>&1; then + svc_out="FAIL" + svc_fail_reason="start 退出非0" + continue + fi + if ! systemctl --no-pager is-active "$svc" | grep -q "^active"; then + svc_out="FAIL" + svc_fail_reason="status 非 active" + continue + fi + if ! timeout "$CCD_SERVICE_TIMEOUT" systemctl --no-pager stop "$svc" >> "$log" 2>&1; then + if grep -iqw "refuse manual start/stop" "$log" 2>/dev/null; then + : + else + svc_out="FAIL" + svc_fail_reason="stop 退出非0" + continue + fi + fi + done < "$svc_list" + if [ "$svc_out" = "FAIL" ]; then + echo "SVC_FAIL_REASON: $pkg $svc_fail_reason" 1>&2 + echo "FAIL" + else + echo "PASS" + fi + } + + total_pkgs=$(wc -l < "$CCD_PKG_LIST" 2>/dev/null || echo 0) + idx=0 + while read -r pkg; do + pkg=$(echo "$pkg" | awk '{print $1}') + [ -n "$pkg" ] || continue + case "$pkg" in *.i386|*.i586|*.i686) echo "SKIP: $pkg (ix86)"; continue;; esac + case "$pkg" in *-debuginfo-*|*-debugsource-*|livepatch-kernel-*) echo "SKIP: $pkg (debug)"; continue;; esac + idx=$((idx+1)) + prefix="[$idx/$total_pkgs]" + echo "$prefix START: $pkg" 1>&2 + + # 安装 + if ! install_pkg "$pkg"; then + echo "$prefix INSTALL_FAIL: $pkg" 1>&2 + src=$(get_src_pkg_for "$pkg") + echo -e "${src}\t${pkg}\tFAIL\tSKIP\tSKIP\tSKIP" >> "$RESULT_FILE" + echo "FAIL_PKG: $pkg install_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') install_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout|timed out' "$CCD_OUT_DIR/logs/${pkg}.install.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" + continue + fi + src=$(get_src_pkg_for "$pkg") + echo -e "${src}\t${pkg}\tPASS" >> "$RESULT_FILE" + + cmd_status="" + cmd_fail_reason="" + if ccd_in_file "$CCD_DESKTOP_WHITE_FILE" "$(ccd_pkg_short "$pkg")"; then + cmd_status="SKIP_DESKTOP" + echo "$prefix cmd[$pkg] skipped (desktop white)" 1>&2 + else + cmd_log="$CCD_OUT_DIR/logs/${pkg}.cmdtest.log" + : > "$cmd_log" + cmd_out=$(cmd_test_for_pkg "$pkg" "$prefix" 2>>"$cmd_log") + cmd_status="$cmd_out" + if grep -q "^FAIL_REASON:" "$cmd_log" 2>/dev/null; then + cmd_fail_line=$(grep "^FAIL_REASON:" "$cmd_log" | head -1) + cmd_fail_reason=$(sed -E 's/^FAIL_REASON: [^ ]+ [^ ]+ //' <<< "$cmd_fail_line") + fi + fi + + svc_status="" + svc_fail_reason="" + if ccd_in_file "$CCD_SERVICE_SPECIAL_FILE" "$(ccd_pkg_short "$pkg")"; then + # 服务需特殊配置:服务测试统一标 skipped + svc_status="SKIP_SPECIAL" + echo "$prefix svc[$pkg] skipped (special-config)" 1>&2 + rm_status="PENDING" + if remove_pkg "$pkg"; then + rm_status="PASS" + else + rm_status="FAIL" + echo "FAIL_PKG: $pkg remove_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') remove_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout' "$CCD_OUT_DIR/logs/${pkg}.remove.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" + fi + sed -i -E "s#(${pkg})\tPASS\$#\\1\t${cmd_status}\t${svc_status}\t${rm_status}#" "$RESULT_FILE" + if [ "$cmd_status" = "FAIL" ]; then + echo "FAIL_PKG: $pkg cmd_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') cmd_fail: $pkg | $cmd_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + fi + continue + fi + + # 普通包:走完整服务测试 + svc_log="$CCD_OUT_DIR/logs/${pkg}.svctest.log" + : > "$svc_log" + svc_out=$(service_test_for_pkg "$pkg" "$prefix" 2>>"$svc_log") + svc_status="$svc_out" + if grep -q "^SVC_FAIL_REASON:" "$svc_log" 2>/dev/null; then + svc_fail_line=$(grep "^SVC_FAIL_REASON:" "$svc_log" | head -1) + svc_fail_reason=$(sed -E 's/^SVC_FAIL_REASON: [^ ]+ //' <<< "$svc_fail_line") + fi + + rm_status="PENDING" + rm_fail_reason="" + if remove_pkg "$pkg"; then + rm_status="PASS" + else + rm_status="FAIL" + echo "FAIL_PKG: $pkg remove_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') remove_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout' "$CCD_OUT_DIR/logs/${pkg}.remove.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" + fi + + sed -i -E "s#(${pkg})\tPASS\$#\\1\t${cmd_status}\t${svc_status}\t${rm_status}#" "$RESULT_FILE" + if [ "$cmd_status" = "FAIL" ]; then + echo "FAIL_PKG: $pkg cmd_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') cmd_fail: $pkg | $cmd_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + fi + if [ "$svc_status" = "FAIL" ]; then + echo "FAIL_PKG: $pkg svc_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') svc_fail: $pkg | $svc_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + fi + echo "$prefix DONE: $pkg cmd=$cmd_status svc=$svc_status rm=$rm_status" 1>&2 + done < "$CCD_PKG_LIST" + + echo "ccd_container_main done for batch: $CCD_OUT_DIR" + echo "result_summary:" 1>&2 +} + +# 切分待测包到 N 个批次,每批文件落在 $1 目录下 +ccd_split_pkgs() { + # $1 = 输入 pkgs 文件,$2 = 批次文件目录 + local in="$1" outdir="$2" + mkdir -p "$outdir" + local n_pkgs total pkgs_per batch_idx i + # 过滤掉注释/空行 + grep -v '^\s*#' "$in" | awk 'NF' > "$outdir/_filtered.list" + n_pkgs=$(wc -l < "$outdir/_filtered.list") + pkgs_per=$(( (n_pkgs + DOCKER_BATCH - 1) / DOCKER_BATCH )) + if [ "$pkgs_per" -eq 0 ]; then pkgs_per=1; fi + split -d -a 4 -l "$pkgs_per" "$outdir/_filtered.list" "$outdir/batch_" + batch_idx=0 + for f in "$outdir"/batch_*; do + [ -f "$f" ] || continue + batch_idx=$((batch_idx+1)) + mv "$f" "$outdir/batch_$(printf '%04d' $batch_idx).list" + done + ccd_log "split $n_pkgs packages into $batch_idx batches (~$pkgs_per/batch)" +} + +# 等待任意一个 bg job 完成,输出完成的 job 名 +ccd_wait_any() { + while :; do + local j + for j in "$@"; do + if ! kill -0 "$(jobs -p | awk -v jp="$j" '$1==jp{print $1}')" 2>/dev/null; then + # bash 没有直接关联 job 与 label 的能力;该函数仅占位 + : + fi + done + sleep 0.2 + local running + running=$(jobs -r | wc -l) + [ "$running" -lt "$1" ] && return 0 + done +} + +ccd_run_one_batch() { + # $1 = batch file path + local batch_file="$1" + local batch_idx batch_out batch_name + batch_idx=$(basename "$batch_file" .list) + batch_out="$CCD_RESULTS_DIR/$batch_idx" + mkdir -p "$batch_out" + batch_name="ccd-$batch_idx-$$" + + local docker_run_args=( + --rm + --name "$batch_name" + -v "$TST_TS_TOPDIR:$TST_TS_TOPDIR:ro" + -v "$PKG_LIST_FILE:/host_input.list:ro" + -v "$batch_file:/host_batch.list:ro" + -v "$DESKTOP_WHITE_FILE:/host_desktop.list:ro" + -v "$SERVICE_SPECIAL_FILE:/host_service.list:ro" + -v "$batch_out:/output" + -v "$TST_TS_TOPDIR/testcase/command-execute-docker.sh:/input/cmd-exec-docker.sh:ro" + -e "CCD_PKG_LIST=/host_batch.list" + -e "CCD_DESKTOP_WHITE_FILE=/host_desktop.list" + -e "CCD_SERVICE_SPECIAL_FILE=/host_service.list" + -e "CCD_OUT_DIR=/output" + -e "CCD_INSTALL_TIMEOUT=$DOCKER_INSTALL_TIMEOUT" + -e "CCD_CMD_TIMEOUT=$DOCKER_CMD_TIMEOUT" + -e "CCD_SERVICE_TIMEOUT=$DOCKER_SERVICE_TIMEOUT" + -e "CCD_ENABLE_SYSTEMD=$DOCKER_ENABLE_SYSTEMD" + -e "CCD_TS_TOPDIR=$TST_TS_TOPDIR" + ) + if [ "$DOCKER_ENABLE_SYSTEMD" = "1" ]; then + docker_run_args+=( --privileged ) + docker_run_args+=( --tmpfs /run ) + docker_run_args+=( --tmpfs /run/lock ) + docker_run_args+=( -v /sys/fs/cgroup:/sys/fs/cgroup:rw ) + docker_run_args+=( --init ) + fi + if [ -n "$DOCKER_CPUS" ]; then + docker_run_args+=( --cpus "$DOCKER_CPUS" ) + fi + if [ -n "$DOCKER_MEMORY" ]; then + docker_run_args+=( -m "$DOCKER_MEMORY" ) + fi + if [ "$DOCKER_USER" != "root" ]; then + docker_run_args+=( -u "$DOCKER_USER" ) + fi + + # 保持数组最后只追加选项 / 挂载;在最终运行命令前添加 IMAGE 与命令 + # docker run 的 IMAGE 与命令分别作为单独位置参数 + if [ "$DOCKER_ENABLE_SYSTEMD" != "1" ]; then + # docker ENABLE_SYSTEMD = 1 的已经在 --init / -v /sys/fs/cgroup/... 加入,见上方 + : + fi + + if [ "$DRY_RUN" = "1" ]; then + ccd_log "DRY_RUN: docker run ${docker_run_args[*]} $CCD_IMAGE /bin/bash " + return 0 + fi + + local func_body tmp_body + func_body=$(awk ' + BEGIN { capture=0 } + /^ccd_container_main\(\)/ { capture=1; print; next } + capture==1 { print; if (/^\}/) { exit } } + ' "${TST_TS_TOPDIR}/testcase/command-execute-docker.sh") + if [ -z "$func_body" ]; then + ccd_err "无法提取 ccd_container_main 函数体" + return 1 + fi + tmp_body="$batch_out/_ccd_body.sh" + { + echo 'set +e' + printf '%s\n' "$func_body" + echo 'if ! type ccd_container_main >/dev/null 2>&1; then echo "container: ccd_container_main not found" 1>&2; exit 2; fi' + echo 'ccd_container_main' + } > "$tmp_body" + chmod 0644 "$tmp_body" + + docker run "${docker_run_args[@]}" "$CCD_IMAGE" /bin/bash /output/_ccd_body.sh \ + 1>"$batch_out/docker.stdout" 2>"$batch_out/docker.stderr" || true + # 不清理 _ccd_body.sh,留着便于调试;日志目录会进 results dir 一起被用户清理 + # rm -f "$tmp_body" + return 0 +} + +# 汇总 + 失败清单分类 +ccd_finalize_report() { + local n_total n_pass n_fail n_skip_per + local result_combined="$CCD_RESULTS_DIR/package-command-result.txt" + # 合并各 batch 的 result.txt + : > "$result_combined" + echo -e "# source-package\tbinary_package\tinstall\tcommand\tservice\tremove" > "$result_combined" + for f in "$CCD_RESULTS_DIR"/batch_*/result.txt; do + [ -f "$f" ] || continue + # 跳表头 + awk 'NR > 1 && NF' "$f" >> "$result_combined" + done + + # 计数 - 通过 / 失败 / 跳过 + # 通过: install=PASS && command in {PASS,NONE,SKIP_DESKTOP,NA} && service in {PASS,NONE,NA} && remove in {PASS,NA} + awk -F'\t' ' + BEGIN { + n_total=0; n_fail=0; n_skip_cmd=0; n_skip_svc=0; n_install_fail=0 + } + NR==1 {next} + { + n_total++ + ins=$3; cmd=$4; svc=$5; rm=$6 + if (ins != "PASS") { fail["install"] = fail["install"] " [" $2 "]"; n_install_fail++; next } + if (cmd=="FAIL" || svc=="FAIL" || rm=="FAIL") { + fail["all"] = fail["all"] " [" $2 "]"; n_fail++ + } + if (cmd=="SKIP_DESKTOP") { skip["cmd"] = skip["cmd"] " [" $2 "]"; n_skip_cmd++ } + if (svc=="SKIP_SPECIAL") { skip["svc"] = skip["svc"] " [" $2 "]"; n_skip_svc++ } + } + END { + print n_total "\t" n_fail "\t" n_skip_cmd "\t" n_skip_svc "\t" n_install_fail + } + ' "$result_combined" > "$CCD_RESULTS_DIR/_stats.tsv" + IFS=$'\t' read -r n_total n_fail n_skip_cmd n_skip_svc n_install_fail < "$CCD_RESULTS_DIR/_stats.tsv" + : "$n_total" "$n_fail" "$n_skip_cmd" "$n_skip_svc" "$n_install_fail" + n_total=${n_total:-0} + n_fail=${n_fail:-0} + n_skip_cmd=${n_skip_cmd:-0} + n_skip_svc=${n_skip_svc:-0} + n_install_fail=${n_install_fail:-0} + # 通过: install PASS 且 cmd/svc/remove 都没有 FAIL;skip 类算过 + n_pass=$(( n_total - n_fail - n_skip_cmd - n_skip_svc - n_install_fail )) + [ "$n_pass" -lt 0 ] && n_pass=0 + + # 按类别失败的包名列表 + # 使用 `|` 分隔的 inline 格式,方便后续拆分 + awk -F'\t' ' + BEGIN { prev_sep="" } + NR==1{next} + { + pkg=$2; ins=$3; cmd=$4; svc=$5; rm=$6 + if (ins=="FAIL") { install_fail_list = install_fail_list prev_sep pkg "||yum install 返回非0"; prev_sep="|" } + if (rm=="FAIL") { remove_fail_list = remove_fail_list prev_sep pkg "||yum remove 返回非0,残留"; prev_sep="|" } + if (cmd=="FAIL") { cmd_fail_list = cmd_fail_list prev_sep pkg "||命令执行失败"; prev_sep="|" } + if (svc=="FAIL") { svc_fail_list = svc_fail_list prev_sep pkg "||systemctl 启停失败"; prev_sep="|" } + if (cmd=="SKIP_DESKTOP") { skip_cmd_list = skip_cmd_list prev_sep pkg "||桌面白名单,跳过命令测试"; prev_sep="|" } + if (svc=="SKIP_SPECIAL") { skip_svc_list = skip_svc_list prev_sep pkg "||含服务包,需特殊配置"; prev_sep="|" } + } + END { + print "INSTALL_FAIL=" (install_fail_list ? install_fail_list : "(无)") + print "REMOVE_FAIL=" (remove_fail_list ? remove_fail_list : "(无)") + print "CMD_FAIL=" (cmd_fail_list ? cmd_fail_list : "(无)") + print "SVC_FAIL=" (svc_fail_list ? svc_fail_list : "(无)") + print "SKIP_CMD=" (skip_cmd_list ? skip_cmd_list : "(无)") + print "SKIP_SVC=" (skip_svc_list ? skip_svc_list : "(无)") + } + ' "$result_combined" > "$CCD_RESULTS_DIR/_buckets.txt" + + local bucket_install bucket_remove bucket_cmd bucket_svc bucket_skip_cmd bucket_skip_svc + bucket_install=$(grep '^INSTALL_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^INSTALL_FAIL=//') + bucket_remove=$(grep '^REMOVE_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^REMOVE_FAIL=//') + bucket_cmd=$(grep '^CMD_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^CMD_FAIL=//') + bucket_svc=$(grep '^SVC_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^SVC_FAIL=//') + bucket_skip_cmd=$(grep '^SKIP_CMD=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^SKIP_CMD=//') + bucket_skip_svc=$(grep '^SKIP_SVC=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^SKIP_SVC=//') + + # 把 pkg||reason 格式的多桶内容转换为可读的多行文本 + # bucket 内每条记录的格式是 "pkg||reason" + format_bucket() { + local content="$1" + if [ "$content" = "(无)" ] || [ -z "$content" ]; then + echo "(无)" + else + echo "$content" | sed 's#||# #g' | tr '|' '\n' + fi + } + local summary="$CCD_RESULTS_DIR/summary.txt" + { + echo "========== 测试失败汇总 ($(date '+%F %T')) ==========" + echo "环境: docker | 并发度: $DOCKER_J | 镜像: $CCD_IMAGE | systemd启用: $DOCKER_ENABLE_SYSTEMD | 待测包总数: $n_total" + echo "通过: $n_pass | 失败: $n_fail | 跳过(命令-桌面): $n_skip_cmd | 跳过(服务-特殊): $n_skip_svc | 安装失败: $n_install_fail" + echo "" + + # 桶内包数 = 包计数 + # bucket 内的格式是 "pkg||reason1|pkg||reason2|..." + # 数 `||` 的出现次数 (= 包数): + # 把所有 `||` 替换为 `_`,差值 /1 即为包数 + count_nonempty() { + local content="$1" + if [ "$content" = "(无)" ] || [ -z "$content" ]; then + echo 0 + else + local sub=${content//||/_} + echo $(( ${#content} - ${#sub} )) + fi + } + + local n_install_count n_remove_count n_cmd_count n_svc_count n_skip_cmd_count n_skip_svc_count + n_install_count=$(count_nonempty "$bucket_install") + n_remove_count=$(count_nonempty "$bucket_remove") + n_cmd_count=$(count_nonempty "$bucket_cmd") + n_svc_count=$(count_nonempty "$bucket_svc") + n_skip_cmd_count=$(count_nonempty "$bucket_skip_cmd") + n_skip_svc_count=$(count_nonempty "$bucket_skip_svc") + + echo "【安装失败】($n_install_count)" + if [ "$n_install_count" -eq 0 ]; then + echo "(无)" + else + format_bucket "$bucket_install" + fi + echo "" + echo "【卸载失败】($n_remove_count)" + if [ "$n_remove_count" -eq 0 ]; then + echo "(无)" + else + format_bucket "$bucket_remove" + fi + echo "" + echo "【命令测试失败】($n_cmd_count)" + if [ "$n_cmd_count" -eq 0 ]; then + echo "(无)" + else + format_bucket "$bucket_cmd" + fi + echo "" + echo "【服务测试失败】($n_svc_count)" + if [ "$n_svc_count" -eq 0 ]; then + echo "(无)" + else + format_bucket "$bucket_svc" + fi + echo "" + echo "【跳过项】(桌面 $n_skip_cmd_count + 服务 $n_skip_svc_count = $((n_skip_cmd_count + n_skip_svc_count)))" + if [ "$n_skip_cmd_count" -eq 0 ] && [ "$n_skip_svc_count" -eq 0 ]; then + echo "(无)" + else + [ "$n_skip_cmd_count" -ne 0 ] && format_bucket "$bucket_skip_cmd" + [ "$n_skip_svc_count" -ne 0 ] && format_bucket "$bucket_skip_svc" + fi + echo "" + # 对账: 通过 + install_fail + (cmd/svc/remove 任何 FAIL) + (skip 桌面/服务) = n_total + # 其中 n_fail 已经涵盖了 cmd/svc/remove FAIL + local sum_check=$(( n_pass + n_fail + n_skip_cmd + n_skip_svc + n_install_fail )) + echo "【对账】通过($n_pass) + 失败($n_fail) + 跳桌面($n_skip_cmd) + 跳服务($n_skip_svc) + 安装失败($n_install_fail) = $sum_check / 待测 $n_total" + if [ "$sum_check" = "$n_total" ]; then + echo "对账结果: PASS" + else + echo "对账结果: FAIL(差异 $((n_total - sum_check)))" + fi + } > "$summary" + + ccd_log "汇总报告: $summary" + cat "$summary" + # 全局最终结果:存在任何 cmd/svc/remove 失败或 install 失败则返回 1 + if [ "${n_fail:-0}" -ne 0 ] || [ "${n_install_fail:-0}" -ne 0 ]; then + return 1 + fi + return 0 +} + +# ---------------------------------------------------------------------- +# 整提主流程 +# ---------------------------------------------------------------------- +g_tmpdir="$(mktemp -d)" +ccd_init() { + [ -n "$TST_TS_TOPDIR" ] || { ccd_err "TST_TS_TOPDIR not set"; exit 2; } + command -v docker >/dev/null 2>&1 || { ccd_err "docker 命令不可用"; exit 1; } + [ -f "$PKG_LIST_FILE" ] || { ccd_err "missing pkg list: $PKG_LIST_FILE"; exit 1; } + [ -f "$DESKTOP_WHITE_FILE" ] || { ccd_err "missing desktop white: $DESKTOP_WHITE_FILE"; exit 1; } + [ -f "$SERVICE_SPECIAL_FILE" ] || { ccd_err "missing service special: $SERVICE_SPECIAL_FILE"; exit 1; } + + if [ -z "$CCD_RESULTS_DIR" ]; then + CCD_RESULTS_DIR="${g_tmpdir%/}/ccd-$(date '+%Y%m%d-%H%M%S')" + else + CCD_RESULTS_DIR="$(realpath -m "$CCD_RESULTS_DIR")" + fi + export CCD_RESULTS_DIR + mkdir -p "$CCD_RESULTS_DIR" + ccd_log "results dir: $CCD_RESULTS_DIR" + + CCD_IMAGE="" + if [ -n "$DOCKER_IMAGE_OVERRIDE" ]; then + CCD_IMAGE="$DOCKER_IMAGE_OVERRIDE" + else + CCD_IMAGE="$(ccd_default_image)" + fi + export CCD_IMAGE + ccd_log "image: $CCD_IMAGE" + + if [ "$DRY_RUN" = "1" ]; then + ccd_log "(dry-run 模式: 跳过镜像检查)" + elif [ "$DOCKER_OFFLINE" = "1" ]; then + ccd_log "(离线模式: 跳过 docker pull)" + else + if ! docker image inspect "$CCD_IMAGE" >/dev/null 2>&1; then + ccd_log "镜像 $CCD_IMAGE 不存在,尝试 docker pull" + if docker pull "$CCD_IMAGE" >/dev/null 2>&1; then + ccd_log "镜像 $CCD_IMAGE 拉取成功" + else + ccd_err "无法拉取 $CCD_IMAGE,可使用 -i 指定已有镜像或 --offline 跳过" + exit 1 + fi + fi + fi +} + +ccd_run() { + local batches_dir="$CCD_RESULTS_DIR/_batches" + ccd_split_pkgs "$PKG_LIST_FILE" "$batches_dir" + shopt -s nullglob + local batch_files=("$batches_dir"/batch_*.list) + shopt -u nullglob + + if [ "${#batch_files[@]}" -eq 0 ]; then + ccd_log "没有待测包,直接退出" + return 0 + fi + + # 简易并发实现:同时起最多 DOCKER_J 个后台 docker run + pids=() + running_pkgs=() + running_batches=() + + for batch_file in "${batch_files[@]}"; do + # 控制并发 + while [ "${#pids[@]}" -ge "$DOCKER_J" ]; do + for i in "${!pids[@]}"; do + if ! kill -0 "${pids[$i]}" 2>/dev/null; then + unset 'pids[i]' + fi + done + pids=( "${pids[@]}" ) + if [ "${#pids[@]}" -ge "$DOCKER_J" ]; then + sleep 1 + fi + done + ccd_run_one_batch "$batch_file" & + local new_pid=$! + pids+=( "$new_pid" ) + running_batches+=( "$(basename "$batch_file" .list)" ) + ccd_log "started batch=$(basename "$batch_file") pid=$new_pid running=${#pids[@]}/$DOCKER_J" + done + + # 等待所有 + for pid in "${pids[@]}"; do + wait "$pid" 2>/dev/null || true + done + ccd_log "all batches done" +} + +tc_setup() { + msg "this is tc_setup" + ccd_init + return 0 +} + +do_test() { + msg "do_test: $*" + ccd_run + ccd_finalize_report + return $? +} + +tc_teardown() { + msg "this is tc_teardown" + # 用户未指定时才清理;用户指定的 CCD_RESULTS_DIR 留给他 + if [[ "$CCD_RESULTS_DIR" == "$g_tmpdir"* ]]; then + # 主机默认 tmp 目录;由调用方决定是否清理 + : + fi + ccd_log "results dir kept at: $CCD_RESULTS_DIR" + return 0 +} + +# ts_setup/ts_teardown 复用 tst_lib 已有定义 +ts_setup() { + msg "this is ts_setup" + command -v docker >/dev/null 2>&1 || { err "docker 不可用"; return 1; } + return 0 +} + +ts_teardown() { + msg "this is ts_teardown" + return 0 +} + +############################################################################### +tst_main "$@" +############################################################################### -- Gitee From 72a1b6fa03ea9f9f6d0a2e0c24d0d177a1b4a670 Mon Sep 17 00:00:00 2001 From: pei-pei45 <3205348955@qq.com> Date: Mon, 20 Jul 2026 00:27:28 +0800 Subject: [PATCH 2/2] =?UTF-8?q?feat+refactor:=20command-execute-docker=20?= =?UTF-8?q?=E7=94=A8=E4=BE=8B(=E5=9F=BA=E4=BA=8E=20docker=20=E5=B9=B6?= =?UTF-8?q?=E5=8F=91=E6=89=A7=E8=A1=8C=E5=8C=85=E6=B5=8B=E8=AF=95)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- testcase/command-execute-docker-container.sh | 382 +++++++++ testcase/command-execute-docker.README.md | 46 +- testcase/command-execute-docker.sh | 786 +++++-------------- 3 files changed, 612 insertions(+), 602 deletions(-) create mode 100755 testcase/command-execute-docker-container.sh diff --git a/testcase/command-execute-docker-container.sh b/testcase/command-execute-docker-container.sh new file mode 100755 index 0000000..dd05ee6 --- /dev/null +++ b/testcase/command-execute-docker-container.sh @@ -0,0 +1,382 @@ +#!/bin/bash +# 容器内主体:由 command-execute-docker.sh 通过 docker run /bin/bash 调用 +# +# 期望挂载/环境变量: +# /input/cmd-exec-docker.sh - 框架主体脚本(可选 source) +# /host_input.list - 输入待测包列表(yum list 原始格式,本次保留) +# /host_batch.list - 本批次的待测包列表 +# /host_desktop.list - 桌面白名单(短包名) +# /host_service.list - 含服务需特殊配置的包列表(短包名) +# /output - 输出目录 (rw),容器内会写入 result.txt / logs/ +# +# 环境变量: +# CCD_PKG_LIST 本批次包列表路径(=/host_batch.list) +# CCD_DESKTOP_WHITE_FILE 桌面白名单路径(=/host_desktop.list) +# CCD_SERVICE_SPECIAL_FILE 服务特殊列表路径(=/host_service.list) +# CCD_OUT_DIR 结果输出目录(=/output) +# CCD_INSTALL_TIMEOUT install 超时秒数(默认 300) +# CCD_REMOVE_TIMEOUT remove 超时秒数(默认 180) +# CCD_CMD_TIMEOUT 命令执行超时秒数(默认 15) +# CCD_SERVICE_TIMEOUT service 启停超时秒数(默认 15) +# CCD_ENABLE_SYSTEMD 1 表示容器内启用了 systemd, 0 表示未启用 +# CCD_ENABLE_SERVICE_TEST 1 真正测服务启停,0 仅记录 service 列表并标 skipped +# CCD_TS_TOPDIR 框架 ts_common.sh 路径(用于复用,可选) +# +# 退出码: +# 0 - 本批次完成(无论是否有失败包,失败信息都写到 result.txt) +# 2 - ccd_container_main 函数定义缺失(异常) +# 3 - 切到 /output 失败 +# 4 - 必需环境变量缺失 + +# 容器内默认保持 set +u 关闭,因为 common.sh 对 unset 变量敏感 +set +u +set +e + +cd /output || { echo "[container] cannot cd /output" 1>&2; exit 3; } + +for v in CCD_PKG_LIST CCD_DESKTOP_WHITE_FILE CCD_SERVICE_SPECIAL_FILE; do + if [ -z "${!v:-}" ]; then + echo "[container] missing env: $v" 1>&2 + exit 4 + fi +done + +: "${CCD_OUT_DIR:=/output}" +: "${CCD_INSTALL_TIMEOUT:=300}" +: "${CCD_REMOVE_TIMEOUT:=180}" +: "${CCD_CMD_TIMEOUT:=15}" +: "${CCD_SERVICE_TIMEOUT:=15}" +: "${CCD_ENABLE_SYSTEMD:=1}" + +mkdir -p "$CCD_OUT_DIR/logs" +RESULT_FILE="$CCD_OUT_DIR/result.txt" +echo -e "# source-package\tbinary_package\tinstall\tcommand\tservice\tremove" > "$RESULT_FILE" + +# 容器内 systemd 兑底:即使 CCD_ENABLE_SYSTEMD=1,也要真正确认 systemctl 可用 +if [ "$CCD_ENABLE_SYSTEMD" = "1" ]; then + if ! command -v systemctl >/dev/null 2>&1; then + echo "[container] CCD_ENABLE_SYSTEMD=1 但容器内无 systemctl,自动关闭 systemd 测试" 1>&2 + CCD_ENABLE_SYSTEMD=0 + elif ! systemctl is-system-running >/dev/null 2>&1; then + echo "[container] systemctl 不可用 (is-system-running 失败),自动关闭 systemd 测试" 1>&2 + CCD_ENABLE_SYSTEMD=0 + fi +fi +# 同步到主机能看到的日志里 +echo "[container] CCD_ENABLE_SYSTEMD=$CCD_ENABLE_SYSTEMD" 1>&2 + +# 加载 ts_common 工具(可选) +if [ -n "${CCD_TS_TOPDIR:-}" ] && [ -d "$CCD_TS_TOPDIR" ]; then + export TST_TS_TOPDIR="$CCD_TS_TOPDIR" + if [ -d "$CCD_TS_TOPDIR/tst_common" ]; then + export TST_COMMON_TOPDIR="$CCD_TS_TOPDIR/tst_common" + else + export TST_COMMON_TOPDIR="$CCD_TS_TOPDIR" + fi + # 容器内 set +u 已经关闭,直接 source 即可 + [ -f "$CCD_TS_TOPDIR/tst_common/lib/common.sh" ] && \ + source "$CCD_TS_TOPDIR/tst_common/lib/common.sh" 2>>"$CCD_OUT_DIR/container.err.log" || true + [ -f "$CCD_TS_TOPDIR/tst_lib/ts_common.sh" ] && \ + source "$CCD_TS_TOPDIR/tst_lib/ts_common.sh" 2>>"$CCD_OUT_DIR/container.err.log" || true +fi + +# ---------------- 包级 install / remove ---------------- +install_pkg() { + local pkg="$1" + local log="$CCD_OUT_DIR/logs/${pkg}.install.log" + if timeout "$CCD_INSTALL_TIMEOUT" dnf install -y "$pkg" >"$log" 2>&1; then + if grep -q "scriptlet failed" "$log" 2>/dev/null; then + cat "$log" 1>&2 + return 1 + fi + return 0 + fi + return 1 +} + +remove_pkg() { + local pkg="$1" + local log="$CCD_OUT_DIR/logs/${pkg}.remove.log" + if timeout "$CCD_REMOVE_TIMEOUT" dnf remove -y --setopt=clean_requirements_on_remove=False "$pkg" >"$log" 2>&1; then + if grep -q "scriptlet failed" "$log" 2>/dev/null; then + cat "$log" 1>&2 + return 1 + fi + return 0 + fi + if grep -iqw "removing the following protected packages" "$log" 2>/dev/null; then + return 0 + fi + cat "$log" 1>&2 + return 1 +} + +get_src_pkg_for() { + local pkg="$1" + if type get_src_pkg_by_bin >/dev/null 2>&1; then + get_src_pkg_by_bin "$pkg" + else + rpm -qi "$pkg" 2>/dev/null | grep -i "Source RPM[[:blank:]]*:" | head -n1 | awk '{print $NF}' + fi +} + +ccd_pkg_short() { + echo "$1" | sed -E 's/\.(x86_64|aarch64|noarch|i[0-9]86|ppc64.*|s390x)$//' +} + +ccd_in_file() { + # $1 = 文件,$2 = 字面量 key;若匹配返回 0,否则返回 1 + local file="$1" + local key="$2" + [ -f "$file" ] && grep -Fxq "$key" "$file" +} + +# ---------------- 命令测试 ---------------- +list_elf_cmds() { + local pkg="$1" + local out="$2" + : > "$out" + rpm -ql "$pkg" 2>/dev/null | grep -E "/(bin|sbin)/" | grep -v "\.so" \ + | while read -r f; do + [ -x "$f" ] || continue + if file -b "$f" 2>/dev/null | grep -q "ELF"; then + echo "$f" >> "$out" + fi + done +} + +run_one_command() { + local bin="$1" log="$2" + : > "$log" + if timeout "$CCD_CMD_TIMEOUT" "$bin" --help >> "$log" 2>&1; then + return 0 + fi + if timeout "$CCD_CMD_TIMEOUT" "$bin" --version >> "$log" 2>&1; then + return 0 + fi + if timeout "$CCD_CMD_TIMEOUT" "$bin" >> "$log" 2>&1; then + return 0 + fi + if grep -q "error while loading shared libraries" "$log" 2>/dev/null; then + return 127 + fi + return 0 +} + +cmd_test_for_pkg() { + local pkg="$1" prefix="$2" + local cmd_list="$CCD_OUT_DIR/logs/${pkg}.cmdlist" + local result_status="PASS" + local fail_cmd="" + local fail_reason="" + list_elf_cmds "$pkg" "$cmd_list" + local n_cmd + n_cmd=$(wc -l < "$cmd_list" 2>/dev/null || echo 0) + if [ "$n_cmd" -eq 0 ]; then + echo "$prefix cmd[$pkg] no-elf-skipped" + echo "PASS" + return + fi + while read -r cmd; do + [ -n "$cmd" ] || continue + echo "$prefix cmd[$pkg] $cmd" + local log="$CCD_OUT_DIR/logs/${pkg}.cmd.$(basename "$cmd").log" + local rc=0 + run_one_command "$cmd" "$log" || rc=$? + if [ "$rc" -ne 0 ]; then + result_status="FAIL" + if [ "$rc" -eq 127 ]; then + fail_cmd="$cmd" + fail_reason="退出码127 (command not found / 缺动态库)" + elif [ "$rc" -eq 128 ]; then + fail_cmd="$cmd" + fail_reason="段错误 core dumped" + else + fail_cmd="$cmd" + fail_reason="退出码 $rc" + fi + fi + done < "$cmd_list" + if [ "$result_status" = "FAIL" ]; then + echo "FAIL_REASON: $pkg $fail_cmd $fail_reason" 1>&2 + fi + echo "$result_status" +} + +# ---------------- 服务测试 ---------------- +list_services_for_pkg() { + local pkg="$1" out="$2" + rpm -ql "$pkg" 2>/dev/null \ + | grep -E "^/usr/lib/systemd/system/.*\.(service|socket|target)$" \ + | awk -F/ '{print $NF}' > "$out" || true +} + +service_test_for_pkg() { + local pkg="$1" prefix="$2" + local svc_list="$CCD_OUT_DIR/logs/${pkg}.svclist" + list_services_for_pkg "$pkg" "$svc_list" + if [ ! -s "$svc_list" ]; then + echo "$prefix svc[$pkg] none" + echo "PASS:NONE" + return + fi + grep -vE '@|systemd-.*\.service|initrd.*\.service|anaconda.*\.service|coreos.*\.service|man-db-cache-update\.service|rc-local\.service|nvidia-persistenced\.service|NetworkManager-wait-online\.service|systemd-fsck.*\.service|selinux-autorelabel\.service' "$svc_list" > "$svc_list.flt" || true + mv "$svc_list.flt" "$svc_list" + if [ ! -s "$svc_list" ]; then + echo "$prefix svc[$pkg] filtered-none" + echo "PASS:NONE" + return + fi + + # systemd 未启用时,服务启停不可执行,直接标 SKIP + if [ "$CCD_ENABLE_SYSTEMD" != "1" ]; then + echo "$prefix svc[$pkg] skip (systemd disabled)" 1>&2 + echo "SKIP:NOSYSTEMD" + return + fi + + local svc_out="PASS" + local svc_fail_reason="" + while read -r svc; do + [ -n "$svc" ] || continue + echo "$prefix svc[$pkg] $svc" + local log="$CCD_OUT_DIR/logs/${pkg}.svc.${svc}.log" + if ! timeout "$CCD_SERVICE_TIMEOUT" systemctl --no-pager start "$svc" >> "$log" 2>&1; then + svc_out="FAIL" + svc_fail_reason="start 退出非0" + continue + fi + if ! systemctl --no-pager is-active "$svc" | grep -q "^active"; then + svc_out="FAIL" + svc_fail_reason="status 非 active" + continue + fi + if ! timeout "$CCD_SERVICE_TIMEOUT" systemctl --no-pager stop "$svc" >> "$log" 2>&1; then + if grep -iqw "refuse manual start/stop" "$log" 2>/dev/null; then + : + else + svc_out="FAIL" + svc_fail_reason="stop 退出非0" + continue + fi + fi + done < "$svc_list" + if [ "$svc_out" = "FAIL" ]; then + echo "SVC_FAIL_REASON: $pkg $svc_fail_reason" 1>&2 + echo "FAIL" + else + echo "PASS" + fi +} + +# ---------------- 主循环 ---------------- +total_pkgs=$(wc -l < "$CCD_PKG_LIST" 2>/dev/null || echo 0) +idx=0 +while read -r pkg; do + pkg=$(echo "$pkg" | awk '{print $1}') + [ -n "$pkg" ] || continue + case "$pkg" in *.i386|*.i586|*.i686) echo "SKIP: $pkg (ix86)"; continue;; esac + case "$pkg" in *-debuginfo-*|*-debugsource-*|livepatch-kernel-*) echo "SKIP: $pkg (debug)"; continue;; esac + idx=$((idx+1)) + prefix="[$idx/$total_pkgs]" + echo "$prefix START: $pkg" 1>&2 + + if ! install_pkg "$pkg"; then + echo "$prefix INSTALL_FAIL: $pkg" 1>&2 + src=$(get_src_pkg_for "$pkg") + echo -e "${src}\t${pkg}\tFAIL\tSKIP\tSKIP\tSKIP" >> "$RESULT_FILE" + echo "FAIL_PKG: $pkg install_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') install_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout|timed out' "$CCD_OUT_DIR/logs/${pkg}.install.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" + continue + fi + src=$(get_src_pkg_for "$pkg") + echo -e "${src}\t${pkg}\tPASS" >> "$RESULT_FILE" + + cmd_status="" + cmd_fail_reason="" + if ccd_in_file "$CCD_DESKTOP_WHITE_FILE" "$(ccd_pkg_short "$pkg")"; then + cmd_status="SKIP_DESKTOP" + echo "$prefix cmd[$pkg] skipped (desktop white)" 1>&2 + else + cmd_log="$CCD_OUT_DIR/logs/${pkg}.cmdtest.log" + : > "$cmd_log" + # 把 stdout 也导到 cmd_log,只把最后一行(PASS/FAIL)取出来作为返回值 + cmd_test_for_pkg "$pkg" "$prefix" >>"$cmd_log" 2>&1 + cmd_status=$(tail -n 1 "$cmd_log") + if grep -q "^FAIL_REASON:" "$cmd_log" 2>/dev/null; then + cmd_fail_line=$(grep "^FAIL_REASON:" "$cmd_log" | head -1) + cmd_fail_reason=$(sed -E 's/^FAIL_REASON: [^ ]+ [^ ]+ //' <<< "$cmd_fail_line") + fi + fi + + svc_status="" + svc_fail_reason="" + if ccd_in_file "$CCD_SERVICE_SPECIAL_FILE" "$(ccd_pkg_short "$pkg")"; then + svc_status="SKIP_SPECIAL" + echo "$prefix svc[$pkg] skipped (special-config)" 1>&2 + rm_status="PENDING" + if remove_pkg "$pkg"; then + rm_status="PASS" + else + rm_status="FAIL" + echo "FAIL_PKG: $pkg remove_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') remove_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout' "$CCD_OUT_DIR/logs/${pkg}.remove.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" + fi + # 用 awk 替换 sed 以避免 bash 对 $# 等做参数扩展(原 sed -E "..." 写法 + # 在 bash 双引号里会把 $# / $@ / $$ 误展开) + awk -v pkg="$pkg" -v cs="$cmd_status" -v ss="$svc_status" -v rs="$rm_status" \ + -v OFS='\t' ' + { + if ($1 != "" && $2 == pkg && $3 == "PASS" && NF == 3) { + $3 = $3 "\t" cs "\t" ss "\t" rs + } + print + }' "$RESULT_FILE" > "$RESULT_FILE.tmp" && mv "$RESULT_FILE.tmp" "$RESULT_FILE" + if [ "$cmd_status" = "FAIL" ]; then + echo "FAIL_PKG: $pkg cmd_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') cmd_fail: $pkg | $cmd_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + fi + continue + fi + + svc_log="$CCD_OUT_DIR/logs/${pkg}.svctest.log" + : > "$svc_log" + # 同 cmd_test:把 stdout 也导入日志,只取最后一行作为状态 + service_test_for_pkg "$pkg" "$prefix" >>"$svc_log" 2>&1 + svc_status=$(tail -n 1 "$svc_log") + if grep -q "^SVC_FAIL_REASON:" "$svc_log" 2>/dev/null; then + svc_fail_line=$(grep "^SVC_FAIL_REASON:" "$svc_log" | head -1) + svc_fail_reason=$(sed -E 's/^SVC_FAIL_REASON: [^ ]+ //' <<< "$svc_fail_line") + fi + + rm_status="PENDING" + rm_fail_reason="" + if remove_pkg "$pkg"; then + rm_status="PASS" + else + rm_status="FAIL" + echo "FAIL_PKG: $pkg remove_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') remove_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout' "$CCD_OUT_DIR/logs/${pkg}.remove.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" + fi + + awk -v pkg="$pkg" -v cs="$cmd_status" -v ss="$svc_status" -v rs="$rm_status" \ + -v OFS='\t' ' + { + if ($1 != "" && $2 == pkg && $3 == "PASS" && NF == 3) { + $3 = $3 "\t" cs "\t" ss "\t" rs + } + print + }' "$RESULT_FILE" > "$RESULT_FILE.tmp" && mv "$RESULT_FILE.tmp" "$RESULT_FILE" + if [ "$cmd_status" = "FAIL" ]; then + echo "FAIL_PKG: $pkg cmd_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') cmd_fail: $pkg | $cmd_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + fi + if [ "$svc_status" = "FAIL" ]; then + echo "FAIL_PKG: $pkg svc_fail" >> "$CCD_OUT_DIR/failures.txt" + echo "$(date '+%F %T') svc_fail: $pkg | $svc_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + fi + echo "$prefix DONE: $pkg cmd=$cmd_status svc=$svc_status rm=$rm_status" 1>&2 +done < "$CCD_PKG_LIST" + +echo "[container] ccd_container_main done for batch: $CCD_OUT_DIR" 1>&2 +exit 0 diff --git a/testcase/command-execute-docker.README.md b/testcase/command-execute-docker.README.md index 1293a29..224d5b0 100644 --- a/testcase/command-execute-docker.README.md +++ b/testcase/command-execute-docker.README.md @@ -33,10 +33,11 @@ command-execute-docker.sh [options] .{install,remove,cmdtest,svctest}.log @@ -125,26 +133,36 @@ dbserver-8.0 含服务包,需特殊配置 | `tencentos` | `tencentos:${VERSION_ID}-${arch}` | | 其他 | `DOCKER_BASE_IMAGE`(`%s` 占位符) | -可通过 `-i` 或 `DOCKER_IMAGE_OVERRIDE` 覆盖。 +可通过 `-i` 或 `DOCKER_IMAGE_OVERRIDE` 覆盖,**`-i` 始终优先**(无论 OS ID +是否命中默认映射)。IMAGE 字符串只允许 `[A-Za-z0-9._:/@-]`,其余字符会被 +拒绝以避免误传。 ### 2. systemd in Docker **默认方案**: 在容器中使用 `--privileged` + `/sys/fs/cgroup` 挂载 + `--init`, 实现容器内 systemd 启停服务的可行性。 +**自动兑底**: 容器内启动后会探测 `systemctl is-system-running`,若失败 +(包括 `systemctl` 命令本身不存在),会把 `CCD_ENABLE_SYSTEMD` 自动设为 0 +并把服务的 `service` 列标记为 `SKIP:NOSYSTEMD`。这避免了"假设 systemd 可用" +的隐性失败。 + **局限**: - 不同 docker 版本(尤其 rootless)对 cgroup 命名空间支持差异较大; - 受限于宿主机 cgroup 配置,在 CI 上可能失败; - 部分发行版基础镜像 systemd unit 不全,需在镜像内 `dnf install -y systemd`。 **回退方案**: `--no-systemd` / `--no-privileged` 模式,容器内不试启停服务, -所有服务的 `service` 列均标记为 `SKIP_SPECIAL` 或 `SKIP`(并附原因)。 +所有服务的 `service` 列均标记为 `SKIP_SPECIAL` 或 `SKIP:NOSYSTEMD`(并附原因)。 ### 3. 并发模型 - 默认每个包一个容器(`-b 1`),完全隔离; -- 通过 `-j N` 控制同时运行的容器数; -- 多余 batch 在主机侧等待 `wait`,不耗资源。 +- 通过 `-j N` 控制同时运行的容器数(用 bash 4.3+ 的 `wait -n`); +- 多余 batch 在主机侧等待,不再用 100% busy loop(老版本的 `ccd_wait_any` + 已被删除); +- 每个 batch 的 `docker run` 退出码独立写到 `batch_*/docker.exit`, + `summary.txt` 中如果存在非零 exit,会独立标出一类 `【batch 整体失败】`。 ### 4. 健壮性 @@ -155,11 +173,19 @@ dbserver-8.0 含服务包,需特殊配置 - 主机端用后台 `&` + `wait`,并发可控; - 容器使用 `--rm`,完成自动销毁。 -### 5. 复用框架 +### 5. 容器内脚本独立化 + +容器内主体逻辑在 `testcase/command-execute-docker-container.sh`,不再从 +本脚本 awk 抽取;直接 `docker run IMAGE /bin/bash /input/cmd-exec-container.sh`。 +这样既消除了 awk 解析大函数的脆弱性,也方便容器内脚本独立维护。 + +主机脚本(source 后只保留 host-side 的逻辑:解析/并发/docker run/汇总)。 + +### 6. 复用框架 容器内 `source ${CCD_TS_TOPDIR}/tst_common/lib/common.sh` 与 `tst_lib/ts_common.sh`(若存在),加载 `msg/err/skip_test` 等公共断言。 -source 前后用 `set +u` 包装,因为 common.sh 内部对未设置的变量敏感。 +容器内统一保持 `set +u`,避免 common.sh 内部对 unset 变量敏感。 若框架脚本缺失,容器内退回到纯 dnf + 命令行的最小实现。 ## 限制与注意事项 diff --git a/testcase/command-execute-docker.sh b/testcase/command-execute-docker.sh index 55bb4a0..8e874ce 100755 --- a/testcase/command-execute-docker.sh +++ b/testcase/command-execute-docker.sh @@ -12,10 +12,8 @@ # 原用例相同的安装/命令/服务判定逻辑,但为每个(或每批)待测包启动一个 # 独立容器,并行执行,通过 -j 控制并发度。 # -# 复用了 tst_lib/ts_common.sh 中 test_single_rpm_install / -# test_command_execute / get_src_pkg_by_bin 等函数(在容器内 source), -# 以及 result 表头格式 (source-package/binary_package/install/ -# command/remove),在此基础上新增 service 列。 +# 容器内逻辑在 command-execute-docker-container.sh 中独立维护, +# 本脚本只负责:主机侧并发/参数解析/挂载/docker run/汇总。 # # 关联 wiki: https://iwiki.woa.com/p/1513482298 ############################################################################### @@ -23,28 +21,19 @@ # ---------------------------------------------------------------------- # 顶层常量与默认值 # ---------------------------------------------------------------------- -# 并发度,默认 4 : "${DOCKER_J:=4}" -# 每容器测试多少个包,默认 1(每包一个容器,隔离最好) : "${DOCKER_BATCH:=1}" -# 默认 docker 基础镜像模板,可在 get_ts_release_type 之后按需改写 : "${DOCKER_BASE_IMAGE:=opencloudos/%s}" -# 用户通过 -i IMAGE 设置的覆盖(空表示用 ccd_default_image 派生) : "${DOCKER_IMAGE_OVERRIDE:=}" -# 容器内 install / single command 默认超时(秒) : "${DOCKER_INSTALL_TIMEOUT:=300}" +: "${DOCKER_REMOVE_TIMEOUT:=180}" : "${DOCKER_CMD_TIMEOUT:=15}" : "${DOCKER_SERVICE_TIMEOUT:=15}" -# 容器资源限额(空表示不限) : "${DOCKER_CPUS:=}" : "${DOCKER_MEMORY:=}" -# 是否在容器内测试 systemd 服务 (--privileged + /sbin/init) : "${DOCKER_ENABLE_SYSTEMD:=1}" -# docker 运行用户,默认 root : "${DOCKER_USER:=root}" -# 结果根目录 : "${DOCKER_RESULTS_DIR:=}" -# 离线模式:不尝试 docker pull : "${DOCKER_OFFLINE:=0}" # ---------------------------------------------------------------------- @@ -57,7 +46,6 @@ # shellcheck source=../tst_common/lib/common.sh source "${TST_TS_TOPDIR}/tst_common/lib/common.sh" || exit 1 -# 这里使用一个独立的 logger,失败/警告在容器内也打印在 stderr ccd_log() { echo "[command-execute-docker] $*" 1>&2; } ccd_err() { echo "[command-execute-docker][ERR] $*" 1>&2; } @@ -75,11 +63,14 @@ Usage: command-execute-docker.sh [options] /dev/null && echo "${ID:-}") + version_id=$(. /etc/os-release 2>/dev/null && echo "${VERSION_ID:-}") -# ---------------------------------------------------------------------- -# 容器内测试入口(在容器内被 docker 调用),所有路径用环境变量传递 -# 输入(通过环境变量): -# CCD_PKG_LIST 容器内待测包文件路径(每行一个包) -# CCD_DESKTOP_WHITE_FILE 桌面白名单 -# CCD_SERVICE_SPECIAL_FILE 服务需特殊配置列表 -# CCD_INSTALL_TIMEOUT install 超时秒数 -# CCD_CMD_TIMEOUT 命令执行超时秒数 -# CCD_SERVICE_TIMEOUT service 启停超时秒数 -# CCD_ENABLE_SYSTEMD 1 表示容器内启用了 systemd, 0 表示未启用 -# CCD_OUT_DIR 输出目录(放 result.txt,logs/) -# CCD_ENABLE_SERVICE_TEST 1 真正测服务启停,0 仅记录 service 列表并标 skipped -# CCD_TS_TOPDIR 框架 ts_common.sh 路径(用于复用) -ccd_container_main() { - set +e # 关闭 set -e,我们自己显式处理错误 - set -u - - cd /output || { echo "container: cannot cd /output" 1>&2; exit 3; } - - : "${CCD_PKG_LIST:?missing CCD_PKG_LIST}" - : "${CCD_DESKTOP_WHITE_FILE:?missing CCD_DESKTOP_WHITE_FILE}" - : "${CCD_SERVICE_SPECIAL_FILE:?missing CCD_SERVICE_SPECIAL_FILE}" - : "${CCD_OUT_DIR:=}" - : "${CCD_INSTALL_TIMEOUT:=300}" - : "${CCD_CMD_TIMEOUT:=15}" - : "${CCD_SERVICE_TIMEOUT:=15}" - : "${CCD_ENABLE_SYSTEMD:=1}" - - mkdir -p "$CCD_OUT_DIR/logs" - RESULT_FILE="$CCD_OUT_DIR/result.txt" - echo -e "# source-package\tbinary_package\tinstall\tcommand\tservice\tremove" > "$RESULT_FILE" - - # 加载 ts_common 工具(可选)。 - # 容器内必须 set +u,因为 common.sh 内部大量使用 unset 变量(依赖于父调用者) - if [ -n "${CCD_TS_TOPDIR:-}" ] && [ -d "$CCD_TS_TOPDIR" ]; then - export TST_TS_TOPDIR="$CCD_TS_TOPDIR" - if [ -d "$CCD_TS_TOPDIR/tst_common" ]; then - export TST_COMMON_TOPDIR="$CCD_TS_TOPDIR/tst_common" - else - export TST_COMMON_TOPDIR="$CCD_TS_TOPDIR" - fi - if [ -f "$CCD_TS_TOPDIR/tst_common/lib/common.sh" ]; then - set +u - # shellcheck disable=SC1090 - source "$CCD_TS_TOPDIR/tst_common/lib/common.sh" 2>>"$CCD_OUT_DIR/container.err.log" || true - set -u - fi - if [ -f "$CCD_TS_TOPDIR/tst_lib/ts_common.sh" ]; then - set +u - # shellcheck disable=SC1090 - source "$CCD_TS_TOPDIR/tst_lib/ts_common.sh" 2>>"$CCD_OUT_DIR/container.err.log" || true - set -u - fi - fi - - install_pkg() { - local pkg="$1" - local log="$CCD_OUT_DIR/logs/${pkg}.install.log" - if timeout "$CCD_INSTALL_TIMEOUT" dnf install -y "$pkg" >"$log" 2>&1; then - if grep -q "scriptlet failed" "$log" 2>/dev/null; then - cat "$log" 1>&2 - return 1 - fi - return 0 - fi - return 1 - } - - remove_pkg() { - local pkg="$1" - local log="$CCD_OUT_DIR/logs/${pkg}.remove.log" - if timeout "$CCD_INSTALL_TIMEOUT" dnf remove -y --setopt=clean_requirements_on_remove=False "$pkg" >"$log" 2>&1; then - if grep -q "scriptlet failed" "$log" 2>/dev/null; then - cat "$log" 1>&2 - return 1 - fi - return 0 - fi - if grep -iqw "removing the following protected packages" "$log" 2>/dev/null; then - return 0 - fi - cat "$log" 1>&2 - return 1 - } - - get_src_pkg_for() { - local pkg="$1" - if type get_src_pkg_by_bin >/dev/null 2>&1; then - get_src_pkg_by_bin "$pkg" - else - rpm -qi "$pkg" 2>/dev/null | grep -i "Source RPM[[:blank:]]*:" | head -n1 | awk '{print $NF}' - fi - } - - list_elf_cmds() { - local pkg="$1" - local out="$2" - : > "$out" - rpm -ql "$pkg" 2>/dev/null | grep -E "/(bin|sbin)/" | grep -v "\.so" \ - | while read -r f; do - [ -x "$f" ] || continue - if file -b "$f" 2>/dev/null | grep -q "ELF"; then - echo "$f" >> "$out" - fi - done - } - - run_one_command() { - local bin="$1" log="$2" - : > "$log" - if timeout "$CCD_CMD_TIMEOUT" "$bin" --help >> "$log" 2>&1; then - : - elif timeout "$CCD_CMD_TIMEOUT" "$bin" --version >> "$log" 2>&1; then - : - elif timeout "$CCD_CMD_TIMEOUT" "$bin" >> "$log" 2>&1; then - : - else - if grep -q "error while loading shared libraries" "$log" 2>/dev/null; then - return 127 - fi - return 0 - fi - return 0 - } - - cmd_test_for_pkg() { - local pkg="$1" prefix="$2" - local cmd_list="$CCD_OUT_DIR/logs/${pkg}.cmdlist" - local result_status="PASS" - local fail_cmd="" - local fail_reason="" - list_elf_cmds "$pkg" "$cmd_list" - local n_cmd - n_cmd=$(wc -l < "$cmd_list" 2>/dev/null || echo 0) - if [ "$n_cmd" -eq 0 ]; then - echo "$prefix cmd[$pkg] no-elf-skipped" - echo "PASS" - return - fi - while read -r cmd; do - [ -n "$cmd" ] || continue - echo "$prefix cmd[$pkg] $cmd" - local log="$CCD_OUT_DIR/logs/${pkg}.cmd.$(basename "$cmd").log" - local rc=0 - run_one_command "$cmd" "$log" || rc=$? - if [ "$rc" -ne 0 ]; then - result_status="FAIL" - if [ "$rc" -eq 127 ]; then - fail_cmd="$cmd" - fail_reason="退出码127 (command not found / 缺动态库)" - elif [ "$rc" -eq 128 ]; then - fail_cmd="$cmd" - fail_reason="段错误 core dumped" - else - fail_cmd="$cmd" - fail_reason="退出码 $rc" - fi - fi - done < "$cmd_list" - if [ "$result_status" = "FAIL" ]; then - echo "FAIL_REASON: $pkg $fail_cmd $fail_reason" 1>&2 - fi - echo "$result_status" - } - - list_services_for_pkg() { - local pkg="$1" out="$2" - rpm -ql "$pkg" 2>/dev/null \ - | grep -E "^/usr/lib/systemd/system/.*\.(service|socket|target)$" \ - | awk -F/ '{print $NF}' > "$out" || true - } - - service_test_for_pkg() { - local pkg="$1" prefix="$2" - local svc_list="$CCD_OUT_DIR/logs/${pkg}.svclist" - list_services_for_pkg "$pkg" "$svc_list" - if [ ! -s "$svc_list" ]; then - echo "$prefix svc[$pkg] none" - echo "PASS:NONE" - return - fi - grep -vE '@|systemd-.*\.service|initrd.*\.service|anaconda.*\.service|coreos.*\.service|man-db-cache-update\.service|rc-local\.service|nvidia-persistenced\.service|NetworkManager-wait-online\.service|systemd-fsck.*\.service|selinux-autorelabel\.service' "$svc_list" > "$svc_list.flt" || true - mv "$svc_list.flt" "$svc_list" - if [ ! -s "$svc_list" ]; then - echo "$prefix svc[$pkg] filtered-none" - echo "PASS:NONE" - return - fi - local svc_out="PASS" - local svc_fail_reason="" - while read -r svc; do - [ -n "$svc" ] || continue - echo "$prefix svc[$pkg] $svc" - local log="$CCD_OUT_DIR/logs/${pkg}.svc.${svc}.log" - if ! timeout "$CCD_SERVICE_TIMEOUT" systemctl --no-pager start "$svc" >> "$log" 2>&1; then - svc_out="FAIL" - svc_fail_reason="start 退出非0" - continue - fi - if ! systemctl --no-pager is-active "$svc" | grep -q "^active"; then - svc_out="FAIL" - svc_fail_reason="status 非 active" - continue - fi - if ! timeout "$CCD_SERVICE_TIMEOUT" systemctl --no-pager stop "$svc" >> "$log" 2>&1; then - if grep -iqw "refuse manual start/stop" "$log" 2>/dev/null; then - : - else - svc_out="FAIL" - svc_fail_reason="stop 退出非0" - continue - fi - fi - done < "$svc_list" - if [ "$svc_out" = "FAIL" ]; then - echo "SVC_FAIL_REASON: $pkg $svc_fail_reason" 1>&2 - echo "FAIL" - else - echo "PASS" - fi - } - - total_pkgs=$(wc -l < "$CCD_PKG_LIST" 2>/dev/null || echo 0) - idx=0 - while read -r pkg; do - pkg=$(echo "$pkg" | awk '{print $1}') - [ -n "$pkg" ] || continue - case "$pkg" in *.i386|*.i586|*.i686) echo "SKIP: $pkg (ix86)"; continue;; esac - case "$pkg" in *-debuginfo-*|*-debugsource-*|livepatch-kernel-*) echo "SKIP: $pkg (debug)"; continue;; esac - idx=$((idx+1)) - prefix="[$idx/$total_pkgs]" - echo "$prefix START: $pkg" 1>&2 - - # 安装 - if ! install_pkg "$pkg"; then - echo "$prefix INSTALL_FAIL: $pkg" 1>&2 - src=$(get_src_pkg_for "$pkg") - echo -e "${src}\t${pkg}\tFAIL\tSKIP\tSKIP\tSKIP" >> "$RESULT_FILE" - echo "FAIL_PKG: $pkg install_fail" >> "$CCD_OUT_DIR/failures.txt" - echo "$(date '+%F %T') install_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout|timed out' "$CCD_OUT_DIR/logs/${pkg}.install.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" - continue - fi - src=$(get_src_pkg_for "$pkg") - echo -e "${src}\t${pkg}\tPASS" >> "$RESULT_FILE" - - cmd_status="" - cmd_fail_reason="" - if ccd_in_file "$CCD_DESKTOP_WHITE_FILE" "$(ccd_pkg_short "$pkg")"; then - cmd_status="SKIP_DESKTOP" - echo "$prefix cmd[$pkg] skipped (desktop white)" 1>&2 - else - cmd_log="$CCD_OUT_DIR/logs/${pkg}.cmdtest.log" - : > "$cmd_log" - cmd_out=$(cmd_test_for_pkg "$pkg" "$prefix" 2>>"$cmd_log") - cmd_status="$cmd_out" - if grep -q "^FAIL_REASON:" "$cmd_log" 2>/dev/null; then - cmd_fail_line=$(grep "^FAIL_REASON:" "$cmd_log" | head -1) - cmd_fail_reason=$(sed -E 's/^FAIL_REASON: [^ ]+ [^ ]+ //' <<< "$cmd_fail_line") - fi - fi - - svc_status="" - svc_fail_reason="" - if ccd_in_file "$CCD_SERVICE_SPECIAL_FILE" "$(ccd_pkg_short "$pkg")"; then - # 服务需特殊配置:服务测试统一标 skipped - svc_status="SKIP_SPECIAL" - echo "$prefix svc[$pkg] skipped (special-config)" 1>&2 - rm_status="PENDING" - if remove_pkg "$pkg"; then - rm_status="PASS" + case "${id:-}" in + opencloudos) + echo "opencloudos:${version_id}-${arch}" + ;; + opencloudos-stream) + echo "opencloudos-stream:${version_id}-${arch}" + ;; + tencentos) + echo "tencentos:${version_id}-${arch}" + ;; + *) + if [[ "${DOCKER_BASE_IMAGE}" == *"%s"* ]]; then + printf "%s\n" "$(printf "${DOCKER_BASE_IMAGE}" "${version_id}-${arch}")" else - rm_status="FAIL" - echo "FAIL_PKG: $pkg remove_fail" >> "$CCD_OUT_DIR/failures.txt" - echo "$(date '+%F %T') remove_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout' "$CCD_OUT_DIR/logs/${pkg}.remove.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" - fi - sed -i -E "s#(${pkg})\tPASS\$#\\1\t${cmd_status}\t${svc_status}\t${rm_status}#" "$RESULT_FILE" - if [ "$cmd_status" = "FAIL" ]; then - echo "FAIL_PKG: $pkg cmd_fail" >> "$CCD_OUT_DIR/failures.txt" - echo "$(date '+%F %T') cmd_fail: $pkg | $cmd_fail_reason" >> "$CCD_OUT_DIR/failures.txt" + echo "${DOCKER_BASE_IMAGE}" fi - continue - fi - - # 普通包:走完整服务测试 - svc_log="$CCD_OUT_DIR/logs/${pkg}.svctest.log" - : > "$svc_log" - svc_out=$(service_test_for_pkg "$pkg" "$prefix" 2>>"$svc_log") - svc_status="$svc_out" - if grep -q "^SVC_FAIL_REASON:" "$svc_log" 2>/dev/null; then - svc_fail_line=$(grep "^SVC_FAIL_REASON:" "$svc_log" | head -1) - svc_fail_reason=$(sed -E 's/^SVC_FAIL_REASON: [^ ]+ //' <<< "$svc_fail_line") - fi - - rm_status="PENDING" - rm_fail_reason="" - if remove_pkg "$pkg"; then - rm_status="PASS" - else - rm_status="FAIL" - echo "FAIL_PKG: $pkg remove_fail" >> "$CCD_OUT_DIR/failures.txt" - echo "$(date '+%F %T') remove_fail: $pkg | $(grep -E 'Error:|requires|conflict|Timeout' "$CCD_OUT_DIR/logs/${pkg}.remove.log" | head -3 | tr '\n' ';')" >> "$CCD_OUT_DIR/failures.txt" - fi - - sed -i -E "s#(${pkg})\tPASS\$#\\1\t${cmd_status}\t${svc_status}\t${rm_status}#" "$RESULT_FILE" - if [ "$cmd_status" = "FAIL" ]; then - echo "FAIL_PKG: $pkg cmd_fail" >> "$CCD_OUT_DIR/failures.txt" - echo "$(date '+%F %T') cmd_fail: $pkg | $cmd_fail_reason" >> "$CCD_OUT_DIR/failures.txt" - fi - if [ "$svc_status" = "FAIL" ]; then - echo "FAIL_PKG: $pkg svc_fail" >> "$CCD_OUT_DIR/failures.txt" - echo "$(date '+%F %T') svc_fail: $pkg | $svc_fail_reason" >> "$CCD_OUT_DIR/failures.txt" - fi - echo "$prefix DONE: $pkg cmd=$cmd_status svc=$svc_status rm=$rm_status" 1>&2 - done < "$CCD_PKG_LIST" + ;; + esac +} - echo "ccd_container_main done for batch: $CCD_OUT_DIR" - echo "result_summary:" 1>&2 +ccd_validate_image() { + # IMAGE 字符串只允许常见字符;避免用户误传 --config / /etc/passwd 等值 + local img="$1" + if ! [[ "$img" =~ ^[A-Za-z0-9._:/@\-]+$ ]]; then + ccd_err "镜像名包含非法字符: '$img'" + return 1 + fi + return 0 } -# 切分待测包到 N 个批次,每批文件落在 $1 目录下 +# ---------------------------------------------------------------------- +# 切分待测包到 N 个批次 +# ---------------------------------------------------------------------- ccd_split_pkgs() { # $1 = 输入 pkgs 文件,$2 = 批次文件目录 local in="$1" outdir="$2" mkdir -p "$outdir" - local n_pkgs total pkgs_per batch_idx i - # 过滤掉注释/空行 + local n_pkgs pkgs_per batch_idx f grep -v '^\s*#' "$in" | awk 'NF' > "$outdir/_filtered.list" n_pkgs=$(wc -l < "$outdir/_filtered.list") pkgs_per=$(( (n_pkgs + DOCKER_BATCH - 1) / DOCKER_BATCH )) @@ -561,27 +227,13 @@ ccd_split_pkgs() { ccd_log "split $n_pkgs packages into $batch_idx batches (~$pkgs_per/batch)" } -# 等待任意一个 bg job 完成,输出完成的 job 名 -ccd_wait_any() { - while :; do - local j - for j in "$@"; do - if ! kill -0 "$(jobs -p | awk -v jp="$j" '$1==jp{print $1}')" 2>/dev/null; then - # bash 没有直接关联 job 与 label 的能力;该函数仅占位 - : - fi - done - sleep 0.2 - local running - running=$(jobs -r | wc -l) - [ "$running" -lt "$1" ] && return 0 - done -} - +# ---------------------------------------------------------------------- +# 单个 batch:组装 docker run 参数,启动容器,记录 exit code +# ---------------------------------------------------------------------- ccd_run_one_batch() { # $1 = batch file path local batch_file="$1" - local batch_idx batch_out batch_name + local batch_idx batch_out batch_name exit_code batch_idx=$(basename "$batch_file" .list) batch_out="$CCD_RESULTS_DIR/$batch_idx" mkdir -p "$batch_out" @@ -596,12 +248,13 @@ ccd_run_one_batch() { -v "$DESKTOP_WHITE_FILE:/host_desktop.list:ro" -v "$SERVICE_SPECIAL_FILE:/host_service.list:ro" -v "$batch_out:/output" - -v "$TST_TS_TOPDIR/testcase/command-execute-docker.sh:/input/cmd-exec-docker.sh:ro" + -v "$TST_TS_TOPDIR/testcase/command-execute-docker-container.sh:/input/cmd-exec-container.sh:ro" -e "CCD_PKG_LIST=/host_batch.list" -e "CCD_DESKTOP_WHITE_FILE=/host_desktop.list" -e "CCD_SERVICE_SPECIAL_FILE=/host_service.list" -e "CCD_OUT_DIR=/output" -e "CCD_INSTALL_TIMEOUT=$DOCKER_INSTALL_TIMEOUT" + -e "CCD_REMOVE_TIMEOUT=$DOCKER_REMOVE_TIMEOUT" -e "CCD_CMD_TIMEOUT=$DOCKER_CMD_TIMEOUT" -e "CCD_SERVICE_TIMEOUT=$DOCKER_SERVICE_TIMEOUT" -e "CCD_ENABLE_SYSTEMD=$DOCKER_ENABLE_SYSTEMD" @@ -614,208 +267,159 @@ ccd_run_one_batch() { docker_run_args+=( -v /sys/fs/cgroup:/sys/fs/cgroup:rw ) docker_run_args+=( --init ) fi - if [ -n "$DOCKER_CPUS" ]; then - docker_run_args+=( --cpus "$DOCKER_CPUS" ) - fi - if [ -n "$DOCKER_MEMORY" ]; then - docker_run_args+=( -m "$DOCKER_MEMORY" ) - fi - if [ "$DOCKER_USER" != "root" ]; then - docker_run_args+=( -u "$DOCKER_USER" ) - fi - - # 保持数组最后只追加选项 / 挂载;在最终运行命令前添加 IMAGE 与命令 - # docker run 的 IMAGE 与命令分别作为单独位置参数 - if [ "$DOCKER_ENABLE_SYSTEMD" != "1" ]; then - # docker ENABLE_SYSTEMD = 1 的已经在 --init / -v /sys/fs/cgroup/... 加入,见上方 - : - fi + [ -n "$DOCKER_CPUS" ] && docker_run_args+=( --cpus "$DOCKER_CPUS" ) + [ -n "$DOCKER_MEMORY" ] && docker_run_args+=( -m "$DOCKER_MEMORY" ) + [ "$DOCKER_USER" != "root" ] && docker_run_args+=( -u "$DOCKER_USER" ) if [ "$DRY_RUN" = "1" ]; then - ccd_log "DRY_RUN: docker run ${docker_run_args[*]} $CCD_IMAGE /bin/bash " + ccd_log "DRY_RUN: docker run ${docker_run_args[*]} $CCD_IMAGE /bin/bash /input/cmd-exec-container.sh" return 0 fi - local func_body tmp_body - func_body=$(awk ' - BEGIN { capture=0 } - /^ccd_container_main\(\)/ { capture=1; print; next } - capture==1 { print; if (/^\}/) { exit } } - ' "${TST_TS_TOPDIR}/testcase/command-execute-docker.sh") - if [ -z "$func_body" ]; then - ccd_err "无法提取 ccd_container_main 函数体" - return 1 + docker run "${docker_run_args[@]}" "$CCD_IMAGE" \ + /bin/bash /input/cmd-exec-container.sh \ + 1>"$batch_out/docker.stdout" 2>"$batch_out/docker.stderr" + exit_code=$? + echo "$exit_code" > "$batch_out/docker.exit" + + if [ "$exit_code" -ne 0 ]; then + ccd_err "batch $batch_idx docker run exit=$exit_code (see $batch_out/docker.{stdout,stderr})" fi - tmp_body="$batch_out/_ccd_body.sh" - { - echo 'set +e' - printf '%s\n' "$func_body" - echo 'if ! type ccd_container_main >/dev/null 2>&1; then echo "container: ccd_container_main not found" 1>&2; exit 2; fi' - echo 'ccd_container_main' - } > "$tmp_body" - chmod 0644 "$tmp_body" - - docker run "${docker_run_args[@]}" "$CCD_IMAGE" /bin/bash /output/_ccd_body.sh \ - 1>"$batch_out/docker.stdout" 2>"$batch_out/docker.stderr" || true - # 不清理 _ccd_body.sh,留着便于调试;日志目录会进 results dir 一起被用户清理 - # rm -f "$tmp_body" return 0 } +# ---------------------------------------------------------------------- # 汇总 + 失败清单分类 +# ---------------------------------------------------------------------- ccd_finalize_report() { - local n_total n_pass n_fail n_skip_per + local n_total n_pass n_fail n_skip_per n_install_fail n_batch_fail n_cmd_fail n_svc_fail n_remove_fail local result_combined="$CCD_RESULTS_DIR/package-command-result.txt" - # 合并各 batch 的 result.txt : > "$result_combined" echo -e "# source-package\tbinary_package\tinstall\tcommand\tservice\tremove" > "$result_combined" + + # 合并各 batch 的 result.txt + shopt -s nullglob for f in "$CCD_RESULTS_DIR"/batch_*/result.txt; do [ -f "$f" ] || continue - # 跳表头 awk 'NR > 1 && NF' "$f" >> "$result_combined" done + shopt -u nullglob - # 计数 - 通过 / 失败 / 跳过 - # 通过: install=PASS && command in {PASS,NONE,SKIP_DESKTOP,NA} && service in {PASS,NONE,NA} && remove in {PASS,NA} + # 统计 pass/fail/install_fail/skip awk -F'\t' ' - BEGIN { - n_total=0; n_fail=0; n_skip_cmd=0; n_skip_svc=0; n_install_fail=0 - } + BEGIN { n_total=0; n_fail=0; n_skip_cmd=0; n_skip_svc=0; n_install_fail=0 + n_cmd_fail=0; n_svc_fail=0; n_remove_fail=0 } NR==1 {next} { n_total++ ins=$3; cmd=$4; svc=$5; rm=$6 - if (ins != "PASS") { fail["install"] = fail["install"] " [" $2 "]"; n_install_fail++; next } - if (cmd=="FAIL" || svc=="FAIL" || rm=="FAIL") { - fail["all"] = fail["all"] " [" $2 "]"; n_fail++ - } - if (cmd=="SKIP_DESKTOP") { skip["cmd"] = skip["cmd"] " [" $2 "]"; n_skip_cmd++ } - if (svc=="SKIP_SPECIAL") { skip["svc"] = skip["svc"] " [" $2 "]"; n_skip_svc++ } - } - END { - print n_total "\t" n_fail "\t" n_skip_cmd "\t" n_skip_svc "\t" n_install_fail + if (ins != "PASS") { n_install_fail++; next } + if (cmd=="FAIL") { n_cmd_fail++ } + if (svc=="FAIL") { n_svc_fail++ } + if (rm=="FAIL") { n_remove_fail++ } + if (cmd=="FAIL" || svc=="FAIL" || rm=="FAIL") { n_fail++ } + if (cmd=="SKIP_DESKTOP") n_skip_cmd++ + if (svc=="SKIP_SPECIAL" || svc=="SKIP:NOSYSTEMD") n_skip_svc++ } + END { print n_total "\t" n_fail "\t" n_skip_cmd "\t" n_skip_svc "\t" n_install_fail "\t" n_cmd_fail "\t" n_svc_fail "\t" n_remove_fail } ' "$result_combined" > "$CCD_RESULTS_DIR/_stats.tsv" - IFS=$'\t' read -r n_total n_fail n_skip_cmd n_skip_svc n_install_fail < "$CCD_RESULTS_DIR/_stats.tsv" - : "$n_total" "$n_fail" "$n_skip_cmd" "$n_skip_svc" "$n_install_fail" - n_total=${n_total:-0} - n_fail=${n_fail:-0} - n_skip_cmd=${n_skip_cmd:-0} - n_skip_svc=${n_skip_svc:-0} + + IFS=$'\t' read -r n_total n_fail n_skip_cmd n_skip_svc n_install_fail \ + n_cmd_fail n_svc_fail n_remove_fail < "$CCD_RESULTS_DIR/_stats.tsv" + + n_total=${n_total:-0}; n_fail=${n_fail:-0} + n_skip_cmd=${n_skip_cmd:-0}; n_skip_svc=${n_skip_svc:-0} n_install_fail=${n_install_fail:-0} - # 通过: install PASS 且 cmd/svc/remove 都没有 FAIL;skip 类算过 + n_cmd_fail=${n_cmd_fail:-0}; n_svc_fail=${n_svc_fail:-0}; n_remove_fail=${n_remove_fail:-0} + n_pass=$(( n_total - n_fail - n_skip_cmd - n_skip_svc - n_install_fail )) [ "$n_pass" -lt 0 ] && n_pass=0 - # 按类别失败的包名列表 - # 使用 `|` 分隔的 inline 格式,方便后续拆分 - awk -F'\t' ' - BEGIN { prev_sep="" } - NR==1{next} - { - pkg=$2; ins=$3; cmd=$4; svc=$5; rm=$6 - if (ins=="FAIL") { install_fail_list = install_fail_list prev_sep pkg "||yum install 返回非0"; prev_sep="|" } - if (rm=="FAIL") { remove_fail_list = remove_fail_list prev_sep pkg "||yum remove 返回非0,残留"; prev_sep="|" } - if (cmd=="FAIL") { cmd_fail_list = cmd_fail_list prev_sep pkg "||命令执行失败"; prev_sep="|" } - if (svc=="FAIL") { svc_fail_list = svc_fail_list prev_sep pkg "||systemctl 启停失败"; prev_sep="|" } - if (cmd=="SKIP_DESKTOP") { skip_cmd_list = skip_cmd_list prev_sep pkg "||桌面白名单,跳过命令测试"; prev_sep="|" } - if (svc=="SKIP_SPECIAL") { skip_svc_list = skip_svc_list prev_sep pkg "||含服务包,需特殊配置"; prev_sep="|" } - } - END { - print "INSTALL_FAIL=" (install_fail_list ? install_fail_list : "(无)") - print "REMOVE_FAIL=" (remove_fail_list ? remove_fail_list : "(无)") - print "CMD_FAIL=" (cmd_fail_list ? cmd_fail_list : "(无)") - print "SVC_FAIL=" (svc_fail_list ? svc_fail_list : "(无)") - print "SKIP_CMD=" (skip_cmd_list ? skip_cmd_list : "(无)") - print "SKIP_SVC=" (skip_svc_list ? skip_svc_list : "(无)") - } - ' "$result_combined" > "$CCD_RESULTS_DIR/_buckets.txt" - - local bucket_install bucket_remove bucket_cmd bucket_svc bucket_skip_cmd bucket_skip_svc - bucket_install=$(grep '^INSTALL_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^INSTALL_FAIL=//') - bucket_remove=$(grep '^REMOVE_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^REMOVE_FAIL=//') - bucket_cmd=$(grep '^CMD_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^CMD_FAIL=//') - bucket_svc=$(grep '^SVC_FAIL=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^SVC_FAIL=//') - bucket_skip_cmd=$(grep '^SKIP_CMD=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^SKIP_CMD=//') - bucket_skip_svc=$(grep '^SKIP_SVC=' "$CCD_RESULTS_DIR/_buckets.txt" | sed 's/^SKIP_SVC=//') - - # 把 pkg||reason 格式的多桶内容转换为可读的多行文本 - # bucket 内每条记录的格式是 "pkg||reason" - format_bucket() { - local content="$1" - if [ "$content" = "(无)" ] || [ -z "$content" ]; then - echo "(无)" - else - echo "$content" | sed 's#||# #g' | tr '|' '\n' + # batch 完全失败的统计(独立维度) + n_batch_fail=$(find "$CCD_RESULTS_DIR" -maxdepth 2 -name 'docker.exit' -type f \ + -exec awk '!/^0$/' {} + 2>/dev/null | wc -l) + n_batch_fail=${n_batch_fail:-0} + + # 桶收集:每行一条 pkg||reason,干净直接 + : > "$CCD_RESULTS_DIR/_install_fail.list" + : > "$CCD_RESULTS_DIR/_remove_fail.list" + : > "$CCD_RESULTS_DIR/_cmd_fail.list" + : > "$CCD_RESULTS_DIR/_svc_fail.list" + : > "$CCD_RESULTS_DIR/_skip_cmd.list" + : > "$CCD_RESULTS_DIR/_skip_svc.list" + : > "$CCD_RESULTS_DIR/_batch_fail.list" + + while IFS=$'\t' read -r src pkg ins cmd svc rm; do + [ "$src" = "# source-package" ] && continue + [ -z "$pkg" ] && continue + if [ "$ins" = "FAIL" ]; then + echo "$pkg||yum install 返回非0" >> "$CCD_RESULTS_DIR/_install_fail.list" + fi + if [ "$rm" = "FAIL" ]; then + echo "$pkg||yum remove 返回非0,残留" >> "$CCD_RESULTS_DIR/_remove_fail.list" + fi + if [ "$cmd" = "FAIL" ]; then + echo "$pkg||命令执行失败" >> "$CCD_RESULTS_DIR/_cmd_fail.list" + fi + if [ "$svc" = "FAIL" ]; then + echo "$pkg||systemctl 启停失败" >> "$CCD_RESULTS_DIR/_svc_fail.list" + fi + if [ "$cmd" = "SKIP_DESKTOP" ]; then + echo "$pkg||桌面白名单,跳过命令测试" >> "$CCD_RESULTS_DIR/_skip_cmd.list" fi - } + if [ "$svc" = "SKIP_SPECIAL" ] || [ "$svc" = "SKIP:NOSYSTEMD" ]; then + echo "$pkg||含服务包,需特殊配置" >> "$CCD_RESULTS_DIR/_skip_svc.list" + fi + done < "$result_combined" + + # batch 失败列表 + for f in "$CCD_RESULTS_DIR"/batch_*/docker.exit; do + [ -f "$f" ] || continue + if [ "$(cat "$f")" != "0" ]; then + batch=$(basename "$(dirname "$f")") + echo "$batch||docker run exit $(cat "$f")" >> "$CCD_RESULTS_DIR/_batch_fail.list" + fi + done + local summary="$CCD_RESULTS_DIR/summary.txt" { echo "========== 测试失败汇总 ($(date '+%F %T')) ==========" echo "环境: docker | 并发度: $DOCKER_J | 镜像: $CCD_IMAGE | systemd启用: $DOCKER_ENABLE_SYSTEMD | 待测包总数: $n_total" - echo "通过: $n_pass | 失败: $n_fail | 跳过(命令-桌面): $n_skip_cmd | 跳过(服务-特殊): $n_skip_svc | 安装失败: $n_install_fail" + echo "通过: $n_pass | 失败: $n_fail | 跳过(命令-桌面): $n_skip_cmd | 跳过(服务-特殊): $n_skip_svc | 安装失败: $n_install_fail | batch 失败: $n_batch_fail" echo "" - # 桶内包数 = 包计数 - # bucket 内的格式是 "pkg||reason1|pkg||reason2|..." - # 数 `||` 的出现次数 (= 包数): - # 把所有 `||` 替换为 `_`,差值 /1 即为包数 - count_nonempty() { - local content="$1" - if [ "$content" = "(无)" ] || [ -z "$content" ]; then - echo 0 - else - local sub=${content//||/_} - echo $(( ${#content} - ${#sub} )) - fi + # 桶格式:每行 pkg||reason(以 || 分隔,字段 1=pkg, 字段 3=reason) + print_bucket() { + local f="$1" + [ -s "$f" ] || { echo "(无)"; return; } + awk -F'\\|' '{printf "%s %s\n", $1, $3}' "$f" } - local n_install_count n_remove_count n_cmd_count n_svc_count n_skip_cmd_count n_skip_svc_count - n_install_count=$(count_nonempty "$bucket_install") - n_remove_count=$(count_nonempty "$bucket_remove") - n_cmd_count=$(count_nonempty "$bucket_cmd") - n_svc_count=$(count_nonempty "$bucket_svc") - n_skip_cmd_count=$(count_nonempty "$bucket_skip_cmd") - n_skip_svc_count=$(count_nonempty "$bucket_skip_svc") - - echo "【安装失败】($n_install_count)" - if [ "$n_install_count" -eq 0 ]; then - echo "(无)" - else - format_bucket "$bucket_install" - fi + echo "【安装失败】($n_install_fail)" + print_bucket "$CCD_RESULTS_DIR/_install_fail.list" echo "" - echo "【卸载失败】($n_remove_count)" - if [ "$n_remove_count" -eq 0 ]; then - echo "(无)" - else - format_bucket "$bucket_remove" - fi + echo "【卸载失败】($n_remove_fail)" + print_bucket "$CCD_RESULTS_DIR/_remove_fail.list" echo "" - echo "【命令测试失败】($n_cmd_count)" - if [ "$n_cmd_count" -eq 0 ]; then - echo "(无)" - else - format_bucket "$bucket_cmd" - fi + echo "【命令测试失败】($n_cmd_fail)" + print_bucket "$CCD_RESULTS_DIR/_cmd_fail.list" echo "" - echo "【服务测试失败】($n_svc_count)" - if [ "$n_svc_count" -eq 0 ]; then - echo "(无)" - else - format_bucket "$bucket_svc" - fi + echo "【服务测试失败】($n_svc_fail)" + print_bucket "$CCD_RESULTS_DIR/_svc_fail.list" echo "" - echo "【跳过项】(桌面 $n_skip_cmd_count + 服务 $n_skip_svc_count = $((n_skip_cmd_count + n_skip_svc_count)))" - if [ "$n_skip_cmd_count" -eq 0 ] && [ "$n_skip_svc_count" -eq 0 ]; then + echo "【跳过项】(桌面 $n_skip_cmd + 服务 $n_skip_svc = $((n_skip_cmd + n_skip_svc)))" + if [ "$n_skip_cmd" -eq 0 ] && [ "$n_skip_svc" -eq 0 ]; then echo "(无)" else - [ "$n_skip_cmd_count" -ne 0 ] && format_bucket "$bucket_skip_cmd" - [ "$n_skip_svc_count" -ne 0 ] && format_bucket "$bucket_skip_svc" + [ -s "$CCD_RESULTS_DIR/_skip_cmd.list" ] && print_bucket "$CCD_RESULTS_DIR/_skip_cmd.list" + [ -s "$CCD_RESULTS_DIR/_skip_svc.list" ] && print_bucket "$CCD_RESULTS_DIR/_skip_svc.list" fi echo "" - # 对账: 通过 + install_fail + (cmd/svc/remove 任何 FAIL) + (skip 桌面/服务) = n_total - # 其中 n_fail 已经涵盖了 cmd/svc/remove FAIL + if [ "$n_batch_fail" -gt 0 ]; then + echo "【batch 整体失败】($n_batch_fail)" + print_bucket "$CCD_RESULTS_DIR/_batch_fail.list" + echo "" + fi local sum_check=$(( n_pass + n_fail + n_skip_cmd + n_skip_svc + n_install_fail )) echo "【对账】通过($n_pass) + 失败($n_fail) + 跳桌面($n_skip_cmd) + 跳服务($n_skip_svc) + 安装失败($n_install_fail) = $sum_check / 待测 $n_total" if [ "$sum_check" = "$n_total" ]; then @@ -827,8 +431,8 @@ ccd_finalize_report() { ccd_log "汇总报告: $summary" cat "$summary" - # 全局最终结果:存在任何 cmd/svc/remove 失败或 install 失败则返回 1 - if [ "${n_fail:-0}" -ne 0 ] || [ "${n_install_fail:-0}" -ne 0 ]; then + # 全局最终结果:存在任何 cmd/svc/remove 失败 / install 失败 / batch 整体失败 → 1 + if [ "${n_fail:-0}" -ne 0 ] || [ "${n_install_fail:-0}" -ne 0 ] || [ "${n_batch_fail:-0}" -ne 0 ]; then return 1 fi return 0 @@ -838,6 +442,9 @@ ccd_finalize_report() { # 整提主流程 # ---------------------------------------------------------------------- g_tmpdir="$(mktemp -d)" +# sentinel:标记 CCD_RESULTS_DIR 是否由本脚本自动生成在 g_tmpdir 下 +CCD_RESULTS_DIR_AUTO=0 + ccd_init() { [ -n "$TST_TS_TOPDIR" ] || { ccd_err "TST_TS_TOPDIR not set"; exit 2; } command -v docker >/dev/null 2>&1 || { ccd_err "docker 命令不可用"; exit 1; } @@ -847,20 +454,23 @@ ccd_init() { if [ -z "$CCD_RESULTS_DIR" ]; then CCD_RESULTS_DIR="${g_tmpdir%/}/ccd-$(date '+%Y%m%d-%H%M%S')" + CCD_RESULTS_DIR_AUTO=1 else CCD_RESULTS_DIR="$(realpath -m "$CCD_RESULTS_DIR")" + CCD_RESULTS_DIR_AUTO=0 fi export CCD_RESULTS_DIR mkdir -p "$CCD_RESULTS_DIR" - ccd_log "results dir: $CCD_RESULTS_DIR" + ccd_log "results dir: $CCD_RESULTS_DIR (auto=$CCD_RESULTS_DIR_AUTO)" - CCD_IMAGE="" + # IMAGE:用户 -i 始终优先,否则按 ID 派生 if [ -n "$DOCKER_IMAGE_OVERRIDE" ]; then CCD_IMAGE="$DOCKER_IMAGE_OVERRIDE" else CCD_IMAGE="$(ccd_default_image)" fi export CCD_IMAGE + ccd_validate_image "$CCD_IMAGE" || exit 1 ccd_log "image: $CCD_IMAGE" if [ "$DRY_RUN" = "1" ]; then @@ -892,32 +502,25 @@ ccd_run() { return 0 fi - # 简易并发实现:同时起最多 DOCKER_J 个后台 docker run - pids=() - running_pkgs=() - running_batches=() + local max=$DOCKER_J + [ "$max" -lt 1 ] && max=1 + local pids=() + local cur=0 for batch_file in "${batch_files[@]}"; do - # 控制并发 - while [ "${#pids[@]}" -ge "$DOCKER_J" ]; do - for i in "${!pids[@]}"; do - if ! kill -0 "${pids[$i]}" 2>/dev/null; then - unset 'pids[i]' - fi - done - pids=( "${pids[@]}" ) - if [ "${#pids[@]}" -ge "$DOCKER_J" ]; then - sleep 1 - fi + while [ "$cur" -ge "$max" ]; do + # wait -n 等任意一个后台任务完成(bash 4.3+) + wait -n 2>/dev/null || true + cur=$((cur-1)) done ccd_run_one_batch "$batch_file" & local new_pid=$! pids+=( "$new_pid" ) - running_batches+=( "$(basename "$batch_file" .list)" ) - ccd_log "started batch=$(basename "$batch_file") pid=$new_pid running=${#pids[@]}/$DOCKER_J" + cur=$((cur+1)) + ccd_log "started batch=$(basename "$batch_file") pid=$new_pid running=$cur/$max" done - # 等待所有 + # 收尾等待所有剩余 for pid in "${pids[@]}"; do wait "$pid" 2>/dev/null || true done @@ -939,19 +542,18 @@ do_test() { tc_teardown() { msg "this is tc_teardown" - # 用户未指定时才清理;用户指定的 CCD_RESULTS_DIR 留给他 - if [[ "$CCD_RESULTS_DIR" == "$g_tmpdir"* ]]; then - # 主机默认 tmp 目录;由调用方决定是否清理 - : + if [ "$CCD_RESULTS_DIR_AUTO" = "1" ]; then + # 默认临时目录:由调用方决定是否清理;此处不强制 rm -rf + ccd_log "auto results dir kept at: $CCD_RESULTS_DIR" + else + ccd_log "user results dir kept at: $CCD_RESULTS_DIR" fi - ccd_log "results dir kept at: $CCD_RESULTS_DIR" return 0 } -# ts_setup/ts_teardown 复用 tst_lib 已有定义 +# ts_setup/ts_teardown 复用 tst_lib 既有定义;此处不再覆盖(避免重复 docker 检查) ts_setup() { msg "this is ts_setup" - command -v docker >/dev/null 2>&1 || { err "docker 不可用"; return 1; } return 0 } -- Gitee