diff --git a/README.en.md b/README.en.md index 00d38969cdf4911c75c00fb995a251927f7afbd3..2fedd4508e2af0f7f84b36ad57298198b5259d6f 100644 --- a/README.en.md +++ b/README.en.md @@ -1,36 +1,168 @@ # splitter -#### Description -Splitting openEuler packages into multiple slices for building distroless images. +## Introduction +splitter is used to generate the raw materials—slices—for creating openEuler distroless images. The final distroless images are generated and published using [EulerPublisher](https://gitee.com/openeuler/eulerpublisher). -#### Software Architecture -Software architecture description +Typically, creating an application container image involves directly using `RUN yum install` to package the required application and its dependent software packages into the image. This packaging method, which uses RPMs as the smallest unit, leads to redundant files within the image, exposing more attack surfaces and increasing the image size, which affects distribution. -#### Installation +A Distroless image is a streamlined container image designed to minimize the container's size and attack surface. Unlike general container images, distroless images do not include many components of an operating system, containing only the minimal dependencies required to run the application. -1. xxxx -2. xxxx -3. xxxx +When building an openEuler distroless image, splitter is first used to split RPM packages. Each package is divided into multiple slices (each slice contains a set of files with a specific function). The dependencies between packages are also expressed more granularly as dependencies between slices. Then, using slices as the smallest building unit, the final distroless image is generated, which can effectively reduce redundant files and thereby lower security risks. -#### Instructions +![img.png](docs/pictures/package.png) -1. xxxx -2. xxxx -3. xxxx +As shown in the figure above, package B's dependency on package A is equivalent to B_slice1 and B_slice2 depending on A_slice1 and A_slice2. When generating the application image for B, the files contained in A_slice3 no longer need to be packaged. -#### Contribution +## Software Architecture +Software Architecture Description -1. Fork the repository -2. Create Feat_xxx branch -3. Commit your code -4. Create Pull Request +## Installation Guide +splitter is in the development stage and currently supports two methods for installation and operation: -#### Gitee Feature +### Method 1: Deploy on openEuler (One-click installation using [install.sh](./install.sh) is recommended) -1. You can use Readme\_XXX.md to support different languages, such as Readme\_en.md, Readme\_zh.md -2. Gitee blog [blog.gitee.com](https://blog.gitee.com) -3. Explore open source project [https://gitee.com/explore](https://gitee.com/explore) -4. The most valuable open source project [GVP](https://gitee.com/gvp) -5. The manual of Gitee [https://gitee.com/help](https://gitee.com/help) -6. The most popular members [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/) +1. Install system dependencies +``` +dnf install python3-dnf git python3-pip cpio binutils +``` + +2. Clone the source code repository +``` +git clone https://gitee.com/openeuler/splitter.git +cd splitter +pip3 install . +``` + +3. Verification +``` +splitter --help +``` + +### Method 2: Run using the official application container image + +splitter provides an official application container image. You can run splitter commands directly through the [bin/splitter-docker.sh](./bin/splitter-docker.sh) script without installing local dependencies. + +The `bin/splitter-docker.sh` script will automatically pull and use the officially released [openeuler/splitter](https://hub.oepkgs.net/harbor/projects/10/repositories/splitter/artifacts-tab?publicAndNotLogged=yes) image. + +**Specify Splitter Version**: `splitter-docker.sh` uses the latest stable version of splitter by default. You can specify a specific version by setting the environment variable `SPLITTER_VERSION`. For example: + +```bash +SPLITTER_VERSION=1.0.3 ./bin/splitter-docker.sh cut -r 24.03-LTS -a x86_64 -o ./output python3_standard +``` + +## Usage Instructions + +### slice releases + +The generation of openEuler slices depends on the package splitting rules defined in the **Slice Definition File (SDF)**. SDFs are released separately based on the openEuler version to which the slice belongs. All SDFs exist as yaml files in [slice-releases](https://gitee.com/openeuler/slice-releases). The [slice-releases](https://gitee.com/openeuler/slice-releases) repository uses branch names to represent different openEuler versions. + +Taking the SDF for python3.11 (named `python3.11.yaml`) as an example: + +```yaml +package: python3 + +deps: + - python3_copyright + +slices: + core: + deps: + - media-types_data + - python3_bins + - python3_stdlib + + standard: + deps: + - python3_aix-support + - python3_all-os + - python3_concurrency + - python3_core + - python3_crypto + - python3_custom-interpreters + - python3_data-persistence + - python3_data-types + - python3_debug + - python3_development-tools + - python3_distribution + - python3_extras + - python3_file-formats + - python3_filesys + - python3_frameworks + - python3_importing + - python3_internet + - python3_ipc + - python3_language + - python3_markup-tools + - python3_multimedia + - python3_net-data + - python3_numeric + - python3_osx-support + - python3_pydoc + - python3_text + - python3_unix + + utils: + deps: + - python3_debug + - python3_pydoc + - python3_core + contents: + common: + /usr/bin/pydoc3.11: + /usr/bin/pydoc3: + /usr/lib64/libpython3.so*: + /usr/lib64/libpython3.11.so*: + + copyright: + contents: + common: + /usr/share/licenses/python3/LICENSE: +``` + +In the SDF above, i.e., the `python3.yaml` file, `slices` indicates that the `python3` package is split into four slices: `python3_core`, `python3_standard`, `python3_utils`, and `python3_copyright`, along with the file contents these slices contain (for details, please see [slice-releases](https://gitee.com/openeuler/slice-releases)). + +### Generating slices with splitter +splitter uses the `cut` command line to split packages and generate the required slices (you can set the `SPLITTER_SLICE_REPO` environment variable to a custom slice-releases source). + +Using a locally installed splitter: +```bash +# Example +splitter cut -r 24.03-LTS -a x86_64 -o /path/to/output python3_standard python3_utils +``` + +Or, using the official container image to run: +```bash +# Example +./bin/splitter-docker.sh cut -r 24.03-LTS -a x86_64 -o /path/to/output python3_standard python3_utils``` + +In the command above, `-r/--release` specifies the openEuler version to which the required slices belong, `-a/--arch` specifies the OS architecture, `-o/--output` specifies the output path for the generated slices, and `python3_standard python3_utils` are the slices specified by the user to be generated. + +All the finally generated slices are packaged and saved in the `/path/to/output` directory. + +### Automatically generate SDF files + +splitter has added a `gen` command, which can automatically generate an initial SDF file based on the contents of an RPM package, making it convenient for developers to quickly create and maintain SDFs. + +Using a locally installed splitter: +```bash +# Example: Generate an SDF file for the brotli package +splitter gen -r 24.03-LTS -o /path/to/output -p brotli +``` + +Or, using the official container image to run: +```bash +# Example: Generate an SDF file for the brotli package +./bin/splitter-docker.sh gen -r 24.03-LTS -o /path/to/output -p brotli +``` + +This command will generate a basic SDF file `brotli.yaml` for the `brotli` RPM package, which developers can further edit and improve as needed. + +### Building distroless container images +[EulerPublisher](https://gitee.com/openeuler/eulerpublisher) integrates splitter to build and publish the final distroless images. + +## Contributing +We welcome all developers to participate in the construction of the openEuler distroless image ecosystem! + +1. Please file an issue in [slice-releases](https://gitee.com/openeuler/slice-releases) to describe your requirements for distroless images, and discuss and develop related SDFs with the community. +2. For feature requests or bugs related to splitter, please submit an issue or a PR. \ No newline at end of file diff --git a/README.md b/README.md index acc00db1ddfe81d576949d74f82ace3f25d9e3cc..daa4c51d1205282f4472dd02f2ac17fc6147e8e5 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,10 @@ openEuler distroless镜像构建时,首先使用splitter对RPM软件包进行 ## 安装指南 -splitter处于开发阶段,当前仅支持在openEuler上部署(建议使用[install.sh](./install.sh)一键安装): + +splitter处于开发阶段,当前支持两种安装和运行方式: + +### 方法一:在openEuler上部署(推荐使用[install.sh](./install.sh)一键安装) 1. 安装系统依赖 ``` @@ -37,6 +40,18 @@ pip3 install . splitter --help ``` +### 方法二:使用官方应用容器镜像运行 + +splitter 提供了官方应用容器镜像,你可以通过 [bin/splitter-docker.sh](./bin/splitter-docker.sh) 脚本直接运行 splitter 命令,无需安装本地依赖。 + +`bin/splitter-docker.sh` 脚本会自动拉取并使用官方发布的 [openeuler/splitter](https://hub.oepkgs.net/harbor/projects/10/repositories/splitter/artifacts-tab?publicAndNotLogged=yes) 镜像。 + +**指定 Splitter 版本**:`splitter-docker.sh` 默认使用 splitter 最新的稳定版本。你可以通过设置环境变量 `SPLITTER_VERSION` 来指定特定的版本。例如: + +```bash +SPLITTER_VERSION=1.0.3 ./bin/splitter-docker.sh cut -r 24.03-LTS -a x86_64 -o ./output python3_standard +``` + ## 使用说明 ### slice releases @@ -110,14 +125,41 @@ slices: ### splitter生成slice splitter使用cut命令行对软件包切分生成所需的slices(可设置SPLITTER_SLICE_REPO环境变量到自定义slice-releases源) -```angular2html + +使用本地安装的splitter: +```bash # 示例 splitter cut -r 24.03-LTS -a x86_64 -o /path/to/output python3_standard python3_utils ``` + +或者,使用官方容器镜像运行: +```bash +# 示例 +./bin/splitter-docker.sh cut -r 24.03-LTS -a x86_64 -o /path/to/output python3_standard python3_utils +``` + 上述命令中`-r/--release`指定所需slices所属的openEuler版本,`-a/--arch`指定OS架构,`-o/--output`指定生成slices的输出路径,`python3_standard python3_utils`是用户指定要生成的slices。 最终生成的所有slices打包保存在`/path/to/output`目录中。 +### 自动生成SDF文件 + +splitter 新增了 `gen` 命令,可以根据RPM包的内容自动生成初始的SDF文件,方便开发者快速创建和维护SDF。 + +使用本地安装的splitter: +```bash +# 示例:为 brotli 包生成 SDF 文件 +splitter gen -r 24.03-LTS -o /path/to/output -p brotli +``` + +或者,使用官方容器镜像运行: +```bash +# 示例:为 brotli 包生成 SDF 文件 +./bin/splitter-docker.sh gen -r 24.03-LTS -o /path/to/output -p brotli +``` + +此命令将为 `brotli` RPM包生成一个基本的 SDF 文件 `brotli.yaml`,开发者可以根据需要进一步编辑和完善。 + ### 构建distroless容器镜像 [EulerPublisher](https://gitee.com/openeuler/eulerpublisher)集成splitter构建并发布最终distroless镜像 diff --git a/bin/splitter-docker.sh b/bin/splitter-docker.sh index 0bd768034b73c3bb69bce41c2e16465775581055..2d8e2e99b03d0378cdc3ff3de83bdc5d6b63b650 100755 --- a/bin/splitter-docker.sh +++ b/bin/splitter-docker.sh @@ -65,7 +65,7 @@ fi # Docker Image name construction # if SPLITTER_VERSION is unset or null, set it to "1.0.2" -: "${SPLITTER_VERSION:="1.0.2"}" +: "${SPLITTER_VERSION:="1.0.3"}" release_lower=$(echo "$RELEASE" | tr '[:upper:]' '[:lower:]') sanitized_tag=${release_lower//-lts-sp/sp} sanitized_tag=${sanitized_tag//-lts/lts}