[Bug]: Multiple build issues with OpenHarmony 5.1 Release inside a Docker container

### 发生了什么问题？

English version:

Hello, I'm trying to build qemu-based version of OpenHarmony 5.1 Release (with GUI ideally) inside a Docker container, but having a few issues that is causing my build to be unsuccessful. I have tried the official documentation on https://docs.openharmony.cn/pages/v5.1/en/device-dev/subsystems/subsys-build-all.md but many documents are for 3.2 Release NOT 5.1.

I hope for your helpful guidance. Thank you very much for any help!

The issues are mostly around:
- Subsystem & Component names
- ffrt & ylong_runtime related

Machine-translated to Chinese (so forgive me for inaccuracies):
您好，我正在尝试在Docker容器中构建基于Qemu的OpenHarmony5.1版本（理想情况下使用GUI），但有一些问题导致我的构建不成功。 我已经尝试了官方文档https://docs.openharmony.cn/pages/v5.1/en/device-dev/subsystems/subsys-build-all.md 但是很多文档是针对3.2版本而不是5.1版本的。

我希望你有帮助的指导。 非常感谢！

问题主要围绕着:
-"子系统"和"组件"名称
-ffrt&ylong_runtime相关

The kind of issues I'm facing:
- subsystem name config incorrect in '/openharmony/OpenHarmony/device/qemu/x86_64_virt/linux/ohos.build', build file subsystem name is device_x86_64_virt,configured subsystem name is device_qemu-x86_64-linux
- subsystem name config incorrect in '/openharmony/OpenHarmony/third_party/ylong_runtime/ohos.build', build file subsystem name is commonlibrary,configured subsystem name is thirdparty.

- find component device_qemu-x86_64-linux failed, please check it in /openharmony/OpenHarmony/out/preloader/qemu-x86_64-linux-min/parts.json

- find subsystem device_x86_64_virt failed, please check it in /openharmony/OpenHarmony/out/preloader/qemu-x86_64-linux-min/parts.json

- find component product_qemu-x86_64-linux-min failed, please check it in /openharmony/OpenHarmony/out/preloader/qemu-x86_64-linux-min/parts.json

- #36 33.77 [OHOS INFO] [GN] ERROR at //developtools/hdc/BUILD.gn:229:7: OHOS innerapi: (ylong_runtime) not found for component (ylong_runtime).

This is my public repo: https://github.com/SomeAB/openharmonyos_test/

This is my current dockerfile:

```shell
# --- Stage 1: The "Builder" ---
FROM ubuntu:22.04 AS builder

# STEP 1: Set the working directory for all subsequent commands.
# This also creates the /openharmony directory.
WORKDIR /openharmony

# STEP 2: Force the system shell to be bash for better script compatibility.
RUN ln -sf /bin/bash /bin/sh

# STEP 3: Download the full source code archive.
# This large layer will be cached after the first successful download.
COPY code-v5.1.0-Release.tar.gz .

# STEP 4: Extract the source code and remove the archive to save space.
RUN tar --strip-components=1 -xzf code-v5.1.0-Release.tar.gz && rm code-v5.1.0-Release.tar.gz

# STEP 5: Change the working directory post-extraction to /openharmony/OpenHarmony folder
WORKDIR /openharmony/OpenHarmony

# STEP 6: Install the comprehensive list of all build and runtime dependencies.
# This list is designed to be exhaustive to prevent missing package errors.
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    bc \
    bison \
    build-essential \
    cargo \
    ccache \
    clang \
    coreutils \
    cpio \
    curl \
    default-jdk \
    default-jdk-headless \
    dosfstools \
    flex \
    gcc-arm-none-eabi \
    genext2fs \
    git \
    git-lfs \
    gperf \
    jq \
    libelf-dev \
    libffi-dev \
    libglib2.0-dev \
    liblz4-tool \
    libncurses-dev \
    libpixman-1-dev \
    libsdl2-dev \
    libssl-dev \
    libtinfo5 \
    libx11-dev \
    libxml2-dev \
    libxrandr-dev \
    make \
    mtd-utils \
    mtools \
    ninja-build \
    ovmf \
    perl \
    python3 \
    python3-pip \
    qemu-system-x86 \
    ruby \
    rustc \
    rsync \
    scons \
    u-boot-tools \
    unzip \
    wget \
    xsltproc \
    zlib1g-dev \
    && rm -rf /var/lib/apt/lists/*

# STEP 6B: Create a symbolic link so 'python' command points to 'python3'
RUN ln -sf /usr/bin/python3 /usr/bin/python

# STEP 7A: Use 'printf' to reliably create the pip config file and disable the PEP 668 error.
RUN printf "[global]\nbreak-system-packages = true\n" > /etc/pip.conf

# STEP 7B: Add ~/.local/bin to PATH so 'hb' command can be found
ENV PATH="/root/.local/bin:${PATH}"

# STEP 7C: This makes the 'hb' command available and installs its Python dependencies.
RUN python3 -m pip install --user build/hb

# STEP 8: Create compiler wrapper scripts to force-include the missing header.
RUN mkdir -p /usr/local/bin_wrappers && \
    printf '#!/bin/bash\nexec /usr/bin/gcc -include cstdint "$@"' > /usr/local/bin_wrappers/gcc && \
    printf '#!/bin/bash\nexec /usr/bin/g++ -include cstdint "$@"' > /usr/local/bin_wrappers/g++ && \
    printf '#!/bin/bash\nexec /usr/bin/clang -include cstdint "$@"' > /usr/local/bin_wrappers/clang && \
    printf '#!/bin/bash\nexec /usr/bin/clang++ -include cstdint "$@"' > /usr/local/bin_wrappers/clang++ && \
    chmod +x /usr/local/bin_wrappers/*

# STEP 9: Prepend the wrapper directory to the PATH environment variable.
# This ensures our wrappers are called instead of the real compilers.
ENV PATH="/usr/local/bin_wrappers:${PATH}"

# STEP 10: Download the pre-built compiler toolchain required by the build script.
RUN ./build/prebuilts_download.sh

# ===================================================================
# Fix no 1
# ===================================================================

# Diagnostic: Find and Show the contents before the patch.
RUN echo "--- Compile Standard Whitelist BEFORE patch ---" && \
    find build/ -name compile_standard_whitelist.json -print0 | xargs -0 cat || echo "Source whitelist not found."

# Patch 1: Add entries to compile_standard_whitelist.json
RUN echo "=== Applying the patch no 1 ===" && \
    find build/ -name compile_standard_whitelist.json | while read file; do \
        echo "Processing file: $file"; \
        # Check if subsystem_components is empty and add entries
        if jq -e '.subsystem_components | length == 0' "$file" >/dev/null 2>&1; then \
            echo "Adding entries to empty subsystem_components array in $file"; \
            jq '.subsystem_components = ["device_qemu-x86_64-linux", "device_x86_64_virt", "product_qemu-x86_64-linux-min"]' "$file" > tmp.json && mv tmp.json "$file" && echo "Entries added successfully to subsystem_components"; \
        else \
            echo "subsystem_components array is not empty, checking individual entries"; \
            jq '.subsystem_components |= (. + ["device_qemu-x86_64-linux", "device_x86_64_virt", "product_qemu-x86_64-linux-min"] | unique)' "$file" > tmp.json && mv tmp.json "$file" && echo "Missing entries added to subsystem_components"; \
        fi; \
        # Also add the bundle_subsystem_error entry
        if ! jq -e ".bundle_subsystem_error[] | select(. == \"device/qemu/x86_64_virt/linux/ohos.build\")" "$file" >/dev/null 2>&1; then \
            echo "Adding entry to bundle_subsystem_error in $file"; \
            jq ".bundle_subsystem_error += [\"device/qemu/x86_64_virt/linux/ohos.build\"]" "$file" > tmp.json && mv tmp.json "$file" && echo "Entry added successfully to bundle_subsystem_error"; \
        else \
            echo "Entry already exists in bundle_subsystem_error array"; \
        fi; \
    done

# Diagnostic: Find & Show the contents after the patch.
RUN echo "--- Compile Standard Whitelist AFTER patch ---" && \
    find build/ -name compile_standard_whitelist.json -print0 | xargs -0 cat || echo "Source whitelist not found."

# ===================================================================
# Fix no 2
# ===================================================================

# Diagnostic: Find and Show the contents before the patch.
RUN echo "--- Subsystem Components Whitelist BEFORE patch ---" && \
    find build/ -name subsystem_components_whitelist.json -print0 | xargs -0 cat || echo "Subsystem components whitelist not found."

# Patch 2: Add 3 entries to the subsystem_components_whitelist.json
RUN echo "=== Applying the patch no 2 ===" && \
    find build/ -name subsystem_components_whitelist.json | while read file; do \
        echo "Processing file: $file"; \
        device_exists=$(jq -e "has(\"device_qemu-x86_64-linux\")" "$file" 2>/dev/null); \
        product_exists=$(jq -e "has(\"product_qemu-x86_64-linux-min\")" "$file" 2>/dev/null); \
        virt_exists=$(jq -e "has(\"device_x86_64_virt\")" "$file" 2>/dev/null); \
        echo "Device exists: $device_exists, Product exists: $product_exists, Virt exists: $virt_exists"; \
        if [ "$device_exists" = "true" ] || [ "$product_exists" = "true" ] || [ "$virt_exists" = "true" ]; then \
            echo "Info: One or more entries were already present in subsystem_components_whitelist"; \
        fi; \
        echo "Adding entries to $file"; \
        jq ". + {\"device_qemu-x86_64-linux\": \"device_qemu-x86_64-linux\", \"product_qemu-x86_64-linux-min\": \"product_qemu-x86_64-linux-min\", \"device_x86_64_virt\": \"device_x86_64_virt\"}" "$file" > tmp.json && mv tmp.json "$file" && echo "Entries added successfully"; \
    done

# Diagnostic: Find and Show the contents after the patch.
RUN echo "--- Subsystem Components Whitelist AFTER patch ---" && \
    find build/ -name subsystem_components_whitelist.json -print0 | xargs -0 cat || echo "Subsystem components whitelist not found."

# ===================================================================
# Fix no 3
# ===================================================================

# Diagnostic: Find and Show the contents before the patch.
RUN echo "--- config.json BEFORE patch ---" && \
    cat vendor/ohemu/qemu_x86_64_linux_min/config.json || echo "config.json not found."

# Patch 3: Atomically update config.json with robust pre- and post-validation
RUN echo "=== Applying the patch no 3 ===" && \
    config_file="vendor/ohemu/qemu_x86_64_linux_min/config.json" && \
    output_file="config_patched.json" && \
    echo "Processing config file: $config_file" && \
    \
    echo "--- Running Pre-flight Validations ---" && \
    if ! jq -e '.subsystems[] | select(.subsystem == "device_x86_64_virt")' "$config_file" >/dev/null 2>&1; then \
        echo "✗ ERROR: Prerequisite 'device_x86_64_virt' subsystem not found. Aborting patch." >&2; \
        exit 1; \
    fi && \
    if ! jq -e '.subsystems[] | select(.subsystem == "commonlibrary")' "$config_file" >/dev/null 2>&1; then \
        echo "✗ ERROR: Prerequisite 'commonlibrary' subsystem not found. Aborting patch." >&2; \
        exit 1; \
    fi && \
    echo "✓ Pre-flight checks passed." && \
    \
    echo "--- Applying all changes atomically ---" && \
    jq ' \
        (.subsystems[] | select(.subsystem == "device_x86_64_virt").subsystem) = "device_qemu-x86_64-linux" | \
        (.subsystems[] | select(.subsystem == "commonlibrary").components) |= (. + [{"component": "ylong_runtime", "features": []}] | unique_by(.component)) | \
        if any(.subsystems[]; .subsystem == "resourceschedule") then . else .subsystems += [{ \
            "subsystem": "resourceschedule", \
            "components": [ \
                { "component": "ffrt", "features":[] }, \
                { "component": "frame_aware_sched", "features":[] } \
            ] \
        }] end \
    ' "$config_file" > "$output_file" && \
    echo "✓ Changes generated into temporary file." && \
    \
    echo "--- Running Post-flight Validations on generated file ---" && \
    if ! jq -e '.subsystems[] | select(.subsystem == "device_qemu-x86_64-linux")' "$output_file" >/dev/null 2>&1; then \
        echo "✗ Validation failed: 'device_qemu-x86_64-linux' subsystem not found in output." >&2; \
        rm -f "$output_file"; exit 1; \
    fi && \
    if ! jq -e '.subsystems[] | select(.subsystem == "commonlibrary").components[] | select(.component == "ylong_runtime")' "$output_file" >/dev/null 2>&1; then \
        echo "✗ Validation failed: 'ylong_runtime' component not found in output." >&2; \
        rm -f "$output_file"; exit 1; \
    fi && \
    if ! jq -e '.subsystems[] | select(.subsystem == "resourceschedule").components[] | select(.component == "ffrt")' "$output_file" >/dev/null 2>&1; then \
        echo "✗ Validation failed: 'ffrt' component not found in 'resourceschedule' subsystem." >&2; \
        rm -f "$output_file"; exit 1; \
    fi && \
    if ! jq -e '.subsystems[] | select(.subsystem == "resourceschedule").components[] | select(.component == "frame_aware_sched")' "$output_file" >/dev/null 2>&1; then \
        echo "✗ Validation failed: 'frame_aware_sched' component not found in 'resourceschedule' subsystem." >&2; \
        rm -f "$output_file"; exit 1; \
    fi && \
    echo "✓ Post-flight checks passed." && \
    \
    mv "$output_file" "$config_file" && \
    echo "✓✓✓ Patch 3 applied and validated successfully! ✓✓✓"

# Diagnostic: Find and Show the contents after the patch.
RUN echo "--- config.json AFTER patch ---" && \
    cat vendor/ohemu/qemu_x86_64_linux_min/config.json || echo "config.json not found."

# --- START DIAGNOSTIC BLOCK (Essential Pre-Build) ---

# Diagnostic: Confirm the current working directory.
RUN echo "--- Current Working Directory ---" && pwd || true
RUN echo "--- Contents of current directory (first few lines) ---" && ls -la . | head -n 10 || true

# Diagnostic: Show hb set help (useful syntax reminder). Helps to confirm hb is installed.
RUN echo "--- hb set --help ---" && hb set --help || true

# --- END DIAGNOSTIC BLOCK ---

# STEP 11: Set the product configuration for the OpenHarmony build.
# Now that 'hb' is on the PATH, call it directly. Removed the 'touch .hb' from here, as it was causing conflict with folder creation
WORKDIR /openharmony/OpenHarmony
RUN hb set --product-name qemu-x86_64-linux-min@ohemu

# Diagnostic: Check the environment hb is using to find the root path and other variables.
RUN echo "--- hb environment details ---" && \
    hb env || true

# STEP 12: Build using the accompanying 'hb build' command
RUN echo "=== Running hb build to build the OpenHarmony OS ===" && \
    hb build --product-name qemu-x86_64-linux-min@ohemu \
    --target-cpu x86_64 \
    --verbose \
    --log-level debug || \
    (echo "=== BUILD FAILED - Dumping error.log ===" && \
     cat /openharmony/OpenHarmony/out/qemu-x86_64-linux/error.log 2>/dev/null || echo "No error.log found" && \
     echo "=== BUILD FAILED - Dumping any other error logs ===" && \
     find /openharmony/OpenHarmony/out/ -name "*.log" -exec echo "=== {} ===" \; -exec tail -50 {} \; 2>/dev/null || echo "No additional logs found" && \
     false)

# Diagnostic: List all final build artifacts in the main output directory.
RUN echo "--- Listing All Final Build Artifacts in the 'out' directory ---" && \
    ls -lR out/ || true

# --- Stage 2: The "Runtime" ---
FROM ubuntu:22.04

# STEP 13: Copy the compiled OS images and the launch script from the builder stage.
COPY --from=builder /openharmony/out/qemu-x86_64-linux/packages/phone/images /ohos
COPY --from=builder /openharmony/device/qemu/x86_64/launch.sh /

# STEP 14: Install the minimal dependencies needed to RUN the OS.
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    qemu-system-x86 ovmf spice-client-tools && \
    rm -rf /var/lib/apt/lists/*

# STEP 15: Make the launch script executable.
RUN chmod +x /launch.sh

# STEP 16: Document that the container exposes port 5900 for the VNC/SPICE stream.
EXPOSE 5900

# STEP 17: Set the launch script as the default command when the container starts.
ENTRYPOINT ["/launch.sh"]

```

### 期望行为是什么？

Successful Build

### 如何复现该缺陷

Run `docker build filename` command

### 其他补充信息

### 版本或分支信息

- [ ] master
- [x] 5.1-Release
- [ ] 5.0-Release
- [ ] 5.0-Beta1
- [ ] Release 4.1
- [ ] Release 4.0
- [ ] Release 3.2

OpenHarmony/build
关闭

内容风险标识

评论 (2)

OpenHarmony/build关闭 .gitee-modal { width: 500px !important; }

内容风险标识