diff --git a/archive/overview.md b/archive/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..23bc749dd9e7a8fd33390910f1fe2d4680332ca7 --- /dev/null +++ b/archive/overview.md @@ -0,0 +1,149 @@ +# Memory Management + +## Basic Concepts + +The memory is an important component of a computer, and is used to temporarily store operation data in the CPU and data exchanged with an external memory such as hardware. In particular, a non-uniform memory access architecture (NUMA) is a memory architecture designed for a multiprocessor computer. The memory access time depends on the location of the memory relative to the processor. In NUMA mode, a processor accesses the local memory faster than the non-local memory (the memory is located in another processor or shared between processors). + +## Memory Monitoring + +1. `free`: displays the system memory status. + Example: + + ```shell + # Display the system memory status in MB. + free -m + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# free -m + total used free shared buff/cache available + Mem: 2633 436 324 23 2072 2196 + Swap: 4043 0 4043 + ``` + + The fields in the command output are as follows: + + | Field | Description | + | ---------- | ----------------------------------------------------------------------- | + | total | Total memory size. | + | used | Used memory. | + | free | Free memory. | + | shared | Total memory shared by multiple processes. | + | buff/cache | Total number of buffers and caches. | + | available | Estimated available memory to start a new application without swapping. | + +2. `vmstat`: dynamically monitors the system memory and views the system memory usage. + + Example: + + ```shell + # Monitor the system memory and display active and inactive memory. + vmstat -a + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# vmstat -a + procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- + r b swpd free inact active si so bi bo in cs us sy id wa st + 2 0 520 331980 1584728 470332 0 0 0 2 15 19 0 0 100 0 0 + ``` + + In the command output, the field related to the memory is as follows: + + | Field | Description | + | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | memory | Memory information.
**-swpd**: usage of the virtual memory, in KB.
**-free**: free memory capacity, in KB.
**-inact**: inactive memory capacity, in KB.
**-active**: active memory capacity, in KB. | + +3. `sar`: monitors the memory usage of the system. + + Example: + + ```shell + # Monitor the memory usage in the sampling period in the system. Collect the statistics every two seconds for three times. + sar -r 2 3 + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# sar -r 2 3 + + 04:02:09 PM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kb + dirty + 04:02:11 PM 332180 2249308 189420 7.02 142172 1764312 787948 11.52 470404 1584924 + 36 + 04:02:13 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + 04:02:15 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + Average: 332159 2249287 189441 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + ``` + + The fields in the command output are described as follows: + + | Field | Description | + | --------- | ------------------------------------------------ | + | kbmemfree | Unused memory space. | + | kbmemused | Used memory space. | + | %memused | Percentage of the used space. | + | kbbuffers | Amount of data stored in the buffer. | + | kbcached | Data access volume in all domains of the system. | + +4. `numactl`: displays the NUMA node configuration and status. + + Example: + + ```shell + # Check the current NUMA configuration. + numactl -H + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# numactl -H + available: 1 nodes (0) + node 0 cpus: 0 1 2 3 + node 0 size: 2633 MB + node 0 free: 322 MB + node distances: + node 0 + 0: 10 + ``` + + The server contains one NUMA node. The NUMA node that contains four cores and 6 GB memory. + The command also displays the distance between NUMA nodes. The further the distance, the higher the latency of cross-node memory accesses, which should be avoided as much as possible. + + `numastat`: displays NUMA node status. + + ```shell + # Check the NUMA node status. + numastat + ``` + + ```shell + [root@openEuler ~]# numastat + node0 + numa_hit 5386186 + numa_miss 0 + numa_foreign 0 + interleave_hit 17483 + local_node 5386186 + other_node 0 + ``` + + The fields in the `numastat` command output are described as follows: + + | Field | Description | + | -------------- | ----------------------------------------------------------------------------------------------------------------------------------- | + | numa_hit | Number of times that the CPU core accesses the local memory on a node. | + | numa_miss | Number of times that the core of a node accesses the memory of other nodes. | + | numa_foreign | Number of pages that were allocated to the local node but moved to other nodes. Each numa_foreign corresponds to a numa_miss event. | + | interleave_hit | Number of pages of the interleave policy that are allocated to this node. | + | local_node | Size of memory that was allocated to this node by processes on this node. | + | other_node | Size of memory that was allocated to other nodes by processes on this node. | diff --git a/docs/en/_toc.yaml b/docs/en/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9c5fb46703e152bf1da75868963d78972cf54489 --- /dev/null +++ b/docs/en/_toc.yaml @@ -0,0 +1,10 @@ +label: Document Center +sections: + - href: ./server/_toc.yaml + - href: ./virtualization/_toc.yaml + - href: ./cloud/_toc.yaml + - href: ./edge_computing/_toc.yaml + - href: ./embedded/_toc.yaml + - href: ./devstation/_toc.yaml + - href: ./tools/_toc.yaml + diff --git a/docs/en/cloud/_toc.yaml b/docs/en/cloud/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b05d2437f2e16af7d296b748cc320b67081d6100 --- /dev/null +++ b/docs/en/cloud/_toc.yaml @@ -0,0 +1,59 @@ +label: Cloud +sections: + - label: Container Engines + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/container_engine/isula_container_engine/_toc.yaml + path: ./container_engine/isula_container_engine + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/container_engine/docker_engine/_toc.yaml + path: ./container_engine/docker_engine + - label: Container Forms + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/container_form/secure_container/_toc.yaml + path: ./container_form/secure_container + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/container_form/system_container/_toc.yaml + path: ./container_form/system_container + - label: Container Runtimes + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/container_runtime/kuasar/_toc.yaml + path: ./container_runtime/kuasar + - label: Container Image Building + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/image_builder/isula_build/_toc.yaml + path: ./image_builder/isula_build + - label: Cloud-Native OS + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/kubeos/kubeos/_toc.yaml + path: ./kubeos/kubeos + - label: Cloud Base OS + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/nestos/nestos/_toc.yaml + path: ./nestos/nestos + - label: Hybrid Deployment + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/hybrid_deployment/rubik/_toc.yaml + path: ./hybrid_deployment/rubik + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/hybrid_deployment/oncn_bwm/_toc.yaml + path: ./hybrid_deployment/oncn_bwm + - label: Cluster Deployment + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/cluster_deployment/kubernetes/_toc.yaml + path: ./cluster_deployment/kubernetes + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/cluster_deployment/isulad+k8s/_toc.yaml + path: ./cluster_deployment/isulad+k8s + - label: Service Mesh + sections: + - href: + upstream: https://gitee.com/openeuler/cloudnative-docs/blob/openEuler-24.03-LTS-SP2/docs/en/docs/kmesh/kmesh/_toc.yaml + path: ./kmesh/kmesh diff --git a/docs/en/devstation/_toc.yaml b/docs/en/devstation/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..831bbf7061b9b6774a1e2f072ab28090a44390d7 --- /dev/null +++ b/docs/en/devstation/_toc.yaml @@ -0,0 +1,9 @@ +label: DevStation +sections: + - href: + upstream: https://gitee.com/src-openeuler/oeDeploy/blob/openEuler-24.03-LTS-SP2/docs/en/_toc.yaml + path: ./devstation/oedeploy + - href: + upstream: https://gitee.com/src-openeuler/oeGitExt/blob/openEuler-24.03-LTS-SP2/docs/en/_toc.yaml + path: ./devstation/oeGitExt + diff --git a/docs/en/docs/A-Ops/architecture-awareness-service-manual.md b/docs/en/docs/A-Ops/architecture-awareness-service-manual.md deleted file mode 100644 index 7bd4426c511d9ebba88342b8cfe9f323c742fe93..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/architecture-awareness-service-manual.md +++ /dev/null @@ -1,77 +0,0 @@ -# Architecture Awareness Service Manual - -## Installation - -### Manual Installation - -- Installing using the repo source mounted by Yum. - - Configure the Yum sources **openEuler22.09** and **openEuler22.09:Epol** in the **/etc/yum.repos.d/openEuler.repo** file. - - ```ini - [everything] # openEuler 22.09 officially released repository - name=openEuler22.09 - baseurl=https://repo.openeuler.org/openEuler-22.09/everything/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=https://repo.openeuler.org/openEuler-22.09/everything/$basearch/RPM-GPG-KEY-openEuler - - [Epol] # openEuler 22.09:Epol officially released repository - name=Epol - baseurl=https://repo.openeuler.org/openEuler-22.09/EPOL/main/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=https://repo.openeuler.org/openEuler-22.09/OS/$basearch/RPM-GPG-KEY-openEuler - ``` - - Run the following commands to download and install gala-spider and its dependencies. - - ```shell - # A-Ops architecture awareness service, usually installed on the master node - yum install gala-spider - yum install python3-gala-spider - - # A-Ops architecture awareness probe, usually installed on the master node - yum install gala-gopher - ``` - -- Installing using the RPM packages. Download **gala-spider-vx.x.x-x.oe1.aarch64.rpm**, and then run the following commands to install the modules. (`x.x-x` indicates the version. Replace it with the actual version number.) - - ```shell - rpm -ivh gala-spider-vx.x.x-x.oe1.aarch64.rpm - - rpm -ivh gala-gopher-vx.x.x-x.oe1.aarch64.rpm - ``` - - - -### Installing Using the A-Ops Deployment Service - -#### Editing the Task List - -Modify the deployment task list and enable the steps for gala_spider: - -```yaml ---- -step_list: - ... - gala_gopher: - enable: false - continue: false - gala_spider: - enable: false - continue: false - ... -``` - -#### Editing the Host List - -For details about the host configuration, see section 2.2.3.8 in the [Deployment Management Manual](./deployment-management-manual.md). - -#### Editing the Variable List - -For details about the variable configuration, see section 2.2.3.8 in the [Deployment Management Manual](./deployment-management-manual.md). - -#### Executing the Deployment Task - -See section 3 in the [Deployment Management Manual](./deployment-management-manual.md) to execute the deployment task. \ No newline at end of file diff --git a/docs/en/docs/A-Ops/deploying-aops-agent.md b/docs/en/docs/A-Ops/deploying-aops-agent.md deleted file mode 100644 index 6e8445dbe0a64eb479655266c96c19759458ec61..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/deploying-aops-agent.md +++ /dev/null @@ -1,670 +0,0 @@ - -# Deploying aops-agent -### 1. Environment Requirements - -One host running on openEuler 20.03 or later - -### 2. Configuration Environment Deployment - -#### 2.1 Disabling the Firewall - -```shell -systemctl stop firewalld -systemctl disable firewalld -systemctl status firewalld -``` - -#### 2.2 Deploying aops-agent - -1. Run `yum install aops-agent` to install aops-agent based on the Yum source. - -2. Modify the configuration file. Change the value of the **ip** in the agent section to the IP address of the local host. - -``` -vim /etc/aops/agent.conf -``` - - The following uses 192.168.1.47 as an example. - - ```ini - [agent] - ;IP address and port number bound when the aops-agent is started. - ip=192.168.1.47 - port=12000 - - [gopher] - ;Default path of the gala-gopher configuration file. If you need to change the path, ensure that the file path is correct. - config_path=/opt/gala-gopher/gala-gopher.conf - - ;aops-agent log collection configuration - [log] - ;Level of the logs to be collected, which can be set to DEBUG, INFO, WARNING, ERROR, or CRITICAL - log_level=INFO - ;Location for storing collected logs - log_dir=/var/log/aops - ;Maximum size of a log file - max_bytes=31457280 - ;Number of backup logs - backup_count=40 - ``` - -3. Run `systemctl start aops-agent` to start the service. - -#### 2.3 Registering with aops-manager - -To identify users and prevent APIs from being invoked randomly, aops-agent uses tokens to authenticate users, reducing the pressure on the deployed hosts. - -For security purposes, the active registration mode is used to obtain the token. Before the registration, prepare the information to be registered on aops-agent and run the `register` command to register the information with aops-manager. No database is configured for aops-agent. After the registration is successful, the token is automatically saved to the specified file and the registration result is displayed on the GUI. In addition, save the local host information to the aops-manager database for subsequent management. - -1. Prepare the **register.json** file. - - Prepare the information required for registration on aops-agent and save the information in JSON format. The data structure is as follows: - -```JSON -{ - // Name of the login user - "web_username":"admin", - // User password - "web_password": "changeme", - // Host name - "host_name": "host1", - // Name of the group to which the host belongs - "host_group_name": "group1", - // IP address of the host where aops-manager is running - "manager_ip":"192.168.1.23", - // Whether to register as a management host - "management":false, - // External port for running aops-manager - "manager_port":"11111", - // Port for running aops-agent - "agent_port":"12000" -} -``` - -Note: Ensure that aops-manager is running on the target host, for example, 192.168.1.23, and the registered host group exists. - -2. Run `aops_agent register -f register.json`. -3. The registration result is displayed on the GUI. If the registration is successful, the token character string is saved to a specified file. If the registration fails, locate the fault based on the message and log content (**/var/log/aops/aops.log**). - -The following is an example of the registration result: - -- Registration succeeded. - -```shell -[root@localhost ~]# aops_agent register -f register.json -Agent Register Success -``` - -- Registration failed. The following uses the aops-manager start failure as an example. - -```shell -[root@localhost ~]# aops_agent register -f register.json -Agent Register Fail -[root@localhost ~]# -``` - -- Log content - -```shell -2022-09-05 16:11:52,576 ERROR command_manage/register/331: HTTPConnectionPool(host='192.168.1.23', port=11111): Max retries exceeded with url: /manage/host/add (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) -[root@localhost ~]# -``` - -### 3. Plug-in Support - -#### 3.1 gala-gopher - -##### 3.1.1 Introduction - -gala-gopher is a low-load probe framework based on eBPF. It can be used to monitor the CPU, memory, and network status of hosts and collect data. You can configure the collection status of existing probes based on service requirements. - -##### 3.1.2 Deployment - -1. Run `yum install gala-gopher` to install gala-gopher based on the Yum source. -2. Enable probes based on service requirements. You can view information about probes in **/opt/gala-gopher/gala-gopher.conf**. -3. Run `systemctl start gala-gopher` to start the gala-gopher service. - -##### 3.1.3 Others - -For more information about gala-gopher, see https://gitee.com/openeuler/gala-gopher/blob/master/README.md. - -### 4. API Support - -#### 4.1 List of External APIs - -| No.| API | Type| Description | -| ---- | ------------------------------ | ---- | ----------------------| -| 1 | /v1/agent/plugin/start | POST | Starts a plug-in. | -| 2 | /v1/agent/plugin/stop | POST | Stops a plug-in. | -| 3 | /v1/agent/application/info | GET | Collects running applications in the target application collection.| -| 4 | /v1/agent/host/info | GET | Obtains host information. | -| 5 | /v1/agent/plugin/info | GET | Obtains the plug-in running information in aops-agent. | -| 6 | /v1/agent/file/collect | POST | Collects content of the configuration file. | -| 7 | /v1/agent/collect/items/change | POST | Changes the running status of plug-in collection items. | - -##### 4.1.1 /v1/agent/plugin/start - -+ Description: Starts the plug-in that is installed but not running. Currently, only the gala-gopher plug-in is supported. - -+ HTTP request mode: POST - -+ Data submission mode: query - -+ Request parameter - - | Parameter | Mandatory| Type| Description | - | ----------- | ---- | ---- | ------ | - | plugin_name | True | str | Plug-in name| - -+ Request parameter example - - | Parameter | Value | - | ----------- | ----------- | - | plugin_name | gala-gopher | - -+ Response body parameters - - | Parameter| Type| Description | - | ------ | ---- | ---------------- | - | code | int | Return code | - | msg | str | Information corresponding to the status code| - -+ Response example - - ```json - { - "code": 200, - "msg": "xxxx" - } - ``` - - -##### 4.1.2 /v1/agent/plugin/stop - -+ Description: Stops a running plug-in. Currently, only the gala-gopher plug-in is supported. - -+ HTTP request mode: POST - -+ Data submission mode: query - -+ Request parameter - - | Parameter | Mandatory| Type| Description | - | ----------- | ---- | ---- | ------ | - | plugin_name | True | str | Plug-in name| - -+ Request parameter example - - | Parameter | Value | - | ----------- | ----------- | - | plugin_name | gala-gopher | - -+ Response body parameters - - | Parameter| Type| Description | - | ------ | ---- | ---------------- | - | code | int | Return code | - | msg | str | Information corresponding to the status code| - -+ Response example - - ```json - { - "code": 200, - "msg": "xxxx" - } - ``` - - -##### 4.1.3 /v1/agent/application/info - -+ Description: Collects running applications in the target application collection. Currently, the target application collection contains MySQL, Kubernetes, Hadoop, Nginx, Docker, and gala-gopher. - -+ HTTP request mode: GET - -+ Data submission mode: query - -+ Request parameter - - | Parameter| Mandatory| Type| Description| - | ------ | ---- | ---- | ---- | - | | | | | - -+ Request parameter example - - | Parameter| Value| - | ------ | ------ | - | | | - -+ Response body parameters - - | Parameter| Type| Description | - | ------ | ---- | ---------------- | - | code | int | Return code | - | msg | str | Information corresponding to the status code| - | resp | dict | Response body | - - + resp - - | Parameter | Type | Description | - | ------- | --------- | -------------------------- | - | running | List[str] | List of the running applications| - -+ Response example - - ```json - { - "code": 200, - "msg": "xxxx", - "resp": { - "running": [ - "mysql", - "docker" - ] - } - } - ``` - - -##### 4.1.4 /v1/agent/host/info - -+ Description: Obtains information about the host where aops-agent is installed, including the system version, BIOS version, kernel version, CPU information, and memory information. - -+ HTTP request mode: POST - -+ Data submission mode: application/json - -+ Request parameter - - | Parameter | Mandatory| Type | Description | - | --------- | ---- | --------- | ------------------------------------------------ | - | info_type | True | List[str] | List of the information to be collected. Currently, only the CPU, disk, memory, and OS are supported.| - -+ Request parameter example - - ```json - ["os", "cpu","memory", "disk"] - ``` - -+ Response body parameters - - | Parameter| Type| Description | - | ------ | ---- | ---------------- | - | code | int | Return code | - | msg | str | Information corresponding to the status code| - | resp | dict | Response body | - - + resp - - | Parameter| Type | Description | - | ------ | ---------- | -------- | - | cpu | dict | CPU information | - | memory | dict | Memory information| - | os | dict | OS information | - | disk | List[dict] | Disk information| - - + cpu - - | Parameter | Type| Description | - | ------------ | ---- | --------------- | - | architecture | str | CPU architecture | - | core_count | int | Number of cores | - | l1d_cache | str | L1 data cache size| - | l1i_cache | str | L1 instruction cache size| - | l2_cache | str | L2 cache size | - | l3_cache | str | L3 cache size | - | model_name | str | Model name | - | vendor_id | str | Vendor ID | - - + memory - - | Parameter| Type | Description | - | ------ | ---------- | -------------- | - | size | str | Total memory | - | total | int | Number of DIMMs | - | info | List[dict] | Information about all DIMMs| - - + info - - | Parameter | Type| Description | - | ------------ | ---- | -------- | - | size | str | Memory size| - | type | str | Type | - | speed | str | Speed | - | manufacturer | str | Vendor | - - + os - - | Parameter | Type| Description | - | ------------ | ---- | -------- | - | bios_version | str | BIOS version| - | os_version | str | OS version| - | kernel | str | Kernel version| - -+ Response example - - ```json - { - "code": 200, - "msg": "operate success", - "resp": { - "cpu": { - "architecture": "aarch64", - "core_count": "128", - "l1d_cache": "8 MiB (128 instances)", - "l1i_cache": "8 MiB (128 instances)", - "l2_cache": "64 MiB (128 instances)", - "l3_cache": "128 MiB (4 instances)", - "model_name": "Kunpeng-920", - "vendor_id": "HiSilicon" - }, - "memory": { - "info": [ - { - "manufacturer": "Hynix", - "size": "16 GB", - "speed": "2933 MT/s", - "type": "DDR4" - }, - { - "manufacturer": "Hynix", - "size": "16 GB", - "speed": "2933 MT/s", - "type": "DDR4" - } - ], - "size": "32G", - "total": 2 - }, - "os": { - "bios_version": "1.82", - "kernel": "5.10.0-60.18.0.50", - "os_version": "openEuler 22.03 LTS" - }, - "disk": [ - { - "capacity": "xxGB", - "model": "xxxxxx" - } - ] - } - } - ``` - - -##### 4.1.5 /v1/agent/plugin/info - -+ Description: Obtains the plug-in running status of the host. Currently, only the gala-gopher plug-in is supported. - -+ HTTP request mode: GET - -+ Data submission mode: query - -+ Request parameter - - | Parameter| Mandatory| Type| Description| - | ------ | ---- | ---- | ---- | - | | | | | - -+ Request parameter example - - | Parameter| Value| - | ------ | ------ | - | | | - -+ Response body parameters - - | Parameter| Type | Description | - | ------ | ---------- | ---------------- | - | code | int | Return code | - | msg | str | Information corresponding to the status code| - | resp | List[dict] | Response body | - - + resp - - | Parameter | Type | Description | - | ------------- | ---------- | ------------------ | - | plugin_name | str | Plug-in name | - | collect_items | list | Running status of plug-in collection items| - | is_installed | str | Information corresponding to the status code | - | resource | List[dict] | Plug-in resource usage | - | status | str | Plug-in running status | - - + resource - - | Parameter | Type| Description | - | ------------- | ---- | ---------- | - | name | str | Resource name | - | current_value | str | Resource usage| - | limit_value | str | Resource limit| - -+ Response example - - ``` - { - "code": 200, - "msg": "operate success", - "resp": [ - { - "collect_items": [ - { - "probe_name": "system_tcp", - "probe_status": "off", - "support_auto": false - }, - { - "probe_name": "haproxy", - "probe_status": "auto", - "support_auto": true - }, - { - "probe_name": "nginx", - "probe_status": "auto", - "support_auto": true - }, - ], - "is_installed": true, - "plugin_name": "gala-gopher", - "resource": [ - { - "current_value": "0.0%", - "limit_value": null, - "name": "cpu" - }, - { - "current_value": "13 MB", - "limit_value": null, - "name": "memory" - } - ], - "status": "active" - } - ] - } - ``` - - -##### 4.1.6 /v1/agent/file/collect - -+ Description: Collects information such as the content, permission, and owner of the target configuration file. Currently, only text files smaller than 1 MB, without execute permission, and supporting UTF8 encoding can be read. - -+ HTTP request mode: POST - -+ Data submission mode: application/json - -+ Request parameter - - | Parameter | Mandatory| Type | Description | - | --------------- | ---- | --------- | ------------------------ | - | configfile_path | True | List[str] | List of the full paths of the files to be collected| - -+ Request parameter example - - ```json - [ "/home/test.conf", "/home/test.ini", "/home/test.json"] - ``` - -+ Response body parameters - - | Parameter | Type | Description | - | ------------- | ---------- | ---------------- | - | infos | List[dict] | File collection information | - | success_files | List[str] | List of files successfully collected| - | fail_files | List[str] | List of files that fail to be collected| - - + infos - - | Parameter | Type| Description | - | --------- | ---- | -------- | - | path | str | File path| - | content | str | File content| - | file_attr | dict | File attributes| - - + file_attr - - | Parameter| Type| Description | - | ------ | ---- | ------------ | - | mode | str | Permission of the file type| - | owner | str | File owner| - | group | str | Group to which the file belongs| - -+ Response example - - ```json - { - "infos": [ - { - "content": "this is a test file", - "file_attr": { - "group": "root", - "mode": "0644", - "owner": "root" - }, - "path": "/home/test.txt" - } - ], - "success_files": [ - "/home/test.txt" - ], - "fail_files": [ - "/home/test.txt" - ] - } - ``` - - -##### 4.1.7 /v1/agent/collect/items/change - -+ Description: Changes the collection status of the plug-in collection items. Currently, only the status of the gala-gopher collection items can be changed. For the gala-gopher collection items, see **/opt/gala-gopher/gala-gopher.conf**. - -+ HTTP request mode: POST - -+ Data submission mode: application/json - -+ Request parameter - - | Parameter | Mandatory| Type| Description | - | ----------- | ---- | ---- | -------------------------- | - | plugin_name | True | dict | Expected modification result of the plug-in collection items| - - + plugin_name - - | Parameter | Mandatory| Type | Description | - | ------------ | ---- | ------ | ------------------ | - | collect_item | True | string | Expected modification result of the collection item| - -+ Request parameter example - - ```json - { - "gala-gopher":{ - "redis":"auto", - "system_inode":"on", - "tcp":"on", - "haproxy":"auto" - } - } - ``` - -+ Response body parameters - - | Parameter| Type | Description | - | ------ | ---------- | ---------------- | - | code | int | Return code | - | msg | str | Information corresponding to the status code| - | resp | List[dict] | Response body | - - + resp - - | Parameter | Type| Description | - | ----------- | ---- | ------------------ | - | plugin_name | dict | Modification result of the corresponding collection item| - - + plugin_name - - | Parameter | Type | Description | - | ------- | --------- | ---------------- | - | success | List[str] | Collection items that are successfully modified| - | failure | List[str] | Collection items that fail to be modified| - -+ Response example - - ```json - { - "code": 200, - "msg": "operate success", - "resp": { - "gala-gopher": { - "failure": [ - "redis" - ], - "success": [ - "system_inode", - "tcp", - "haproxy" - ] - } - } - } - ``` - - - - ### FAQs - -1. If an error is reported, view the **/var/log/aops/aops.log** file, rectify the fault based on the error message in the log file, and restart the service. - -2. You are advised to run aops-agent in Python 3.7 or later. Pay attention to the version of the Python dependency library when installing it. - -3. The value of **access_token** can be obtained from the **/etc/aops/agent.conf** file after the registration is complete. - -4. To limit the CPU and memory resources of a plug-in, add **MemoryHigh** and **CPUQuota** to the **Service** section in the service file corresponding to the plug-in. - - For example, set the memory limit of gala-gopher to 40 MB and the CPU limit to 20%. - - ```ini - [Unit] - Description=a-ops gala gopher service - After=network.target - - [Service] - Type=exec - ExecStart=/usr/bin/gala-gopher - Restart=on-failure - RestartSec=1 - RemainAfterExit=yes - ;Limit the maximum memory that can be used by processes in the unit. The limit can be exceeded. However, after the limit is exceeded, the process running speed is limited, and the system reclaims the excess memory as much as possible. - ;The option value can be an absolute memory size in bytes (K, M, G, or T suffix based on 1024) or a relative memory size in percentage. - MemoryHigh=40M - ;Set the CPU time limit for the processes of this unit. The value must be a percentage ending with %, indicating the maximum percentage of the total time that the unit can use a single CPU. - CPUQuota=20% - - [Install] - WantedBy=multi-user.target - ``` - - - - - - diff --git a/docs/en/docs/A-Ops/deploying-aops.md b/docs/en/docs/A-Ops/deploying-aops.md deleted file mode 100644 index f7f2cb8d867ab915bf51bcdf7b002a172936a01e..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/deploying-aops.md +++ /dev/null @@ -1,460 +0,0 @@ -# Deploying A-Ops - -## 1. Environment Requirements - -- Two hosts running on openEuler 22.09 - - These two hosts are used to deploy two modes of the check module: scheduler and executor. Other services, such as MySQL, Elasticsearch, and aops-manager, can be independently deployed on any host. To facilitate operations, deploy these services on host A. - -- More than 8 GB memory - -## 2. Configuring the Deployment Environment - -### Host A: - -Deploy the following A-Ops services on host A: aops-tools, aops-manager, aops-check, aops-web, aops-agent, and gala-gopher. - -Deploy the following third-party services on host A: MySQL, Elasticsearch, ZooKeeper, Kafka, and Prometheus. - -The deployment procedure is as follows: - -#### 2.1 Disabling the Firewall - -Disable the firewall on the local host. - -``` -systemctl stop firewalld -systemctl disable firewalld -systemctl status firewalld -``` - -#### 2.2 Deploying aops-tools - -Install aops-tools. - -``` -yum install aops-tools -``` - -#### 2.3 Deploying Databases MySQL and Elasticsearch - -##### 2.3.1 Deploying MySQL - -Use the **aops-basedatabase** script installed during aops-tools installation to install MySQL. - -``` -cd /opt/aops/aops_tools -./aops-basedatabase mysql -``` - -Modify the MySQL configuration file. - -``` -vim /etc/my.cnf -``` - -Add **bind-address** and set it to the IP address of the local host. - -![1662346986112](./figures/修改mysql配置文件.png) - -Restart the MySQL service. - -``` -systemctl restart mysqld -``` - -Connect to the database and set the permission. - -``` -mysql -show databases; -use mysql; -select user,host from user;// If the value of user is root and the value of host is localhost, the MySQL database can be connected only by the local host but cannot be connected from the external network and by the local software client. -update user set host = '%' where user='root'; -flush privileges;// Refresh the permission. -exit -``` - -##### 2.3.2 Deploying Elasticsearch - -Use the **aops-basedatabase** script installed during aops-tools installation to install Elasticsearch. - -``` -cd /opt/aops/aops_tools -./aops-basedatabase elasticsearch -``` - -Modify the configuration file. - -Modify the Elasticsearch configuration file. - -``` -vim /etc/elasticsearch/elasticsearch.yml -``` - -![1662370718890](./figures/elasticsearch配置2.png) - -![1662370575036](./figures/elasticsearch配置1.png) - -![1662370776219](./figures/elasticsearch3.png) - -Restart the Elasticsearch service. - -``` -systemctl restart elasticsearch -``` - -#### 2.4 Deploying aops-manager - -Install aops-manager. - -``` -yum install aops-manager -``` - -Modify the configuration file. - -``` -vim /etc/aops/manager.ini -``` - -Change the IP address of each service in the configuration file to the actual IP address. Because all services are deployed on host A, you need to set their IP addresses to the actual IP address of host A. - -``` -[manager] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=11111 -host_vault_dir=/opt/aops -host_vars=/opt/aops/host_vars - -[uwsgi] -wsgi-file=manage.py -daemonize=/var/log/aops/uwsgi/manager.log -http-timeout=600 -harakiri=600 - -[elasticsearch] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=9200 -max_es_query_num=10000000 - -[mysql] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=3306 -database_name=aops -engine_format=mysql+pymysql://@%s:%s/%s -pool_size=10000 -pool_recycle=7200 - -[aops_check] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=11112 -``` - -Start the aops-manager service. - -``` -systemctl start aops-manager -``` - -#### 2.5 Deploying aops-web - -Install aops-web. - -``` -yum install aops-web -``` - -Modify the configuration file. Because all services are deployed on host A, set the IP address of each service accessed by aops-web to the actual IP address of host A. - -``` -vim /etc/nginx/aops-nginx.conf -``` - -The following figure shows the configuration of some services. - -![1662378186528](./figures/配置web.png) - -Enable the aops-web service. - -``` -systemctl start aops-web -``` - -#### 2.6 Deploying Kafka - -##### 2.6.1 Deploying ZooKeeper - -Install ZooKeeper. - -``` -yum install zookeeper -``` - -Start the ZooKeeper service. - -``` -systemctl start zookeeper -``` - -##### 2.6.2 Deploying Kafka - -Install Kafka. - -``` -yum install kafka -``` - -Modify the configuration file. - -``` -vim /opt/kafka/config/server.properties -``` - -Change the value of **listeners** to the IP address of the local host. - -![1662381371927](./figures/kafka配置.png) - -Start the Kafka service. - -``` -cd /opt/kafka/bin -nohup ./kafka-server-start.sh ../config/server.properties & -tail -f ./nohup.out # Check all the outputs of nohup. If the IP address of host A and the Kafka startup success INFO are displayed, Kafka is started successfully. -``` - -#### 2.7 Deploying aops-check - -Install aops-check. - -``` -yum install aops-check -``` - -Modify the configuration file. - -``` -vim /etc/aops/check.ini -``` - -Change the IP address of each service in the configuration file to the actual IP address. Because all services are deployed on host A, you need to set their IP addresses to the actual IP address of host A. - -``` -[check] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=11112 -mode=configurable // Configurable mode, which means aops-check is used as the scheduler in common diagnosis mode. -timing_check=on - -[default_mode] -period=30 -step=30 - -[elasticsearch] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=9200 - -[mysql] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=3306 -database_name=aops -engine_format=mysql+pymysql://@%s:%s/%s -pool_size=10000 -pool_recycle=7200 - -[prometheus] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=9090 -query_range_step=15s - -[agent] -default_instance_port=8888 - -[manager] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=11111 - -[consumer] -kafka_server_list=192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. -enable_auto_commit=False -auto_offset_reset=earliest -timeout_ms=5 -max_records=3 -task_name=CHECK_TASK -task_group_id=CHECK_TASK_GROUP_ID -result_name=CHECK_RESULT -[producer] -kafka_server_list = 192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. -api_version = 0.11.5 -acks = 1 -retries = 3 -retry_backoff_ms = 100 -task_name=CHECK_TASK -task_group_id=CHECK_TASK_GROUP_ID -``` - -Start the aops-check service in configurable mode. - -``` -systemctl start aops-check -``` - -#### 2.8 Deploying the Client Services - -aops-agent and gala-gopher must be deployed on the client. For details, see the [Deploying aops-agent](deploying-aops-agent.md). - -Note: Before registering a host, you need to add a host group to ensure that the host group to which the host belongs exists. In this example, only host A is deployed and managed. - -#### 2.9 Deploying Prometheus - -Install Prometheus. - -``` -yum install prometheus2 -``` - -Modify the configuration file. - -``` -vim /etc/prometheus/prometheus.yml -``` - -Add the gala-gopher addresses of all clients to the monitoring host of Prometheus. - -![1662377261742](./figures/prometheus配置.png) - -Start the Prometheus service: - -``` -systemctl start prometheus -``` - -#### 2.10 Deploying gala-ragdoll - -The configuration source tracing function of A-Ops depends on gala-ragdoll. Git is used to monitor configuration file changes. - -Install gala-ragdoll. - -```shell -yum install gala-ragdoll # A-Ops configuration source tracing -``` - -Modify the configuration file. - -```shell -vim /etc/ragdoll/gala-ragdoll.conf -``` - -Change the IP address in **collect_address** of the **collect** section to the IP address of host A, and change the values of **collect_api** and **collect_port** to the actual API and port number. - -``` -[git] -git_dir = "/home/confTraceTest" -user_name = "user_name" -user_email = "user_email" - -[collect] -collect_address = "http://192.168.1.1" // Change the IP address to the actual IP address of host A. -collect_api = "/manage/config/collect" // Change the API to the actual API for collecting configuration files. -collect_port = 11111 // Change the port number to the actual port number of the service. - -[sync] -sync_address = "http://0.0.0.0" -sync_api = "/demo/syncConf" -sync_port = 11114 - - -[ragdoll] -port = 11114 - -``` - -Start the gala-ragdoll service. - -```shell -systemctl start gala-ragdoll -``` - -### Host B: - -Only aops-check needs to be deployed on host B as the executor. - -#### 2.11 Deploying aops-check - -Install aops-check. - -``` -yum install aops-check -``` - -Modify the configuration file. - -``` -vim /etc/aops/check.ini -``` - -Change the IP address of each service in the configuration file to the actual IP address. Change the IP address of the aops-check service deployed on host B to the IP address of host B. Because other services are deployed on host A, change the IP addresses of those services to the IP address of host A. - -``` -[check] -ip=192.168.1.2 // Change the IP address to the actual IP address of host B. -port=11112 -mode=executor // Executor mode, which means aops-check is used as the executor in normal diagnosis mode. -timing_check=on - -[default_mode] -period=30 -step=30 - -[elasticsearch] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=9200 - -[mysql] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=3306 -database_name=aops -engine_format=mysql+pymysql://@%s:%s/%s -pool_size=10000 -pool_recycle=7200 - -[prometheus] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=9090 -query_range_step=15s - -[agent] -default_instance_port=8888 - -[manager] -ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. -port=11111 - -[consumer] -kafka_server_list=192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. -enable_auto_commit=False -auto_offset_reset=earliest -timeout_ms=5 -max_records=3 -task_name=CHECK_TASK -task_group_id=CHECK_TASK_GROUP_ID -result_name=CHECK_RESULT -[producer] -kafka_server_list = 192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. -api_version = 0.11.5 -acks = 1 -retries = 3 -retry_backoff_ms = 100 -task_name=CHECK_TASK -task_group_id=CHECK_TASK_GROUP_ID -``` - -Start the aops-check service in executor mode. - -``` -systemctl start aops-check -``` - - - -The service deployment on the two hosts is complete. diff --git a/docs/en/docs/A-Ops/dnf-command-usage.md b/docs/en/docs/A-Ops/dnf-command-usage.md deleted file mode 100644 index 760809b3b54285b57abfa0eb22f34c8e125472f9..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/dnf-command-usage.md +++ /dev/null @@ -1,542 +0,0 @@ -# DNF Command Usage - -Af ter installing A-Ops Apollo, you can run DNF commands to use Apollo functions related to hot patches, such as hot patch scanning (`dnf hot-updateinfo`), setting and querying (`dnf hotpatch`), and applying (`dnf hotupgrade`). This document describes the usage of the commands. - -## Hot Patch Scanning - -`hot-updateinfo` can scan hot patches and query hot patches for specified CVEs. - -```shell -$ dnf hot-updateinfo list cves [--cve [cve_id]] -General DNF options: - -h, --help, --help-cmd - show command help - --cve CVES, --cves CVES - Include packages needed to fix the given CVE, in updates -``` - -- `--list` - -1. Query the CVEs on the host that can be fixed and their related cold and hot patches. - - ```shell - $ dnf hot-updateinfo list cves - # cve-id level cold-patch hot-patch - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - CVE-2022-3080 Important/Sec. bind-libs-9.16.23-10.oe2203.aarch64 patch-bind-libs-9.16.23-09-name-1-111.aarch64 - CVE-2021-25220 Moderate/Sec. bind-9.16.23-10.oe2203.aarch64 - - CVE-2022-1886 Critical/Sec. vim-common-8.2-39.oe2203.aarch64 patch-vim-common-8.2-38-name-1-233.aarch64 - CVE-2022-1725 Low/Sec. vim-minimal-8.2-58.oe2203.aarch64 patch-vim-minimal-8.2-57-name-2-11.aarch64 - ``` - -2. Query cold and hot patches for specified CVEs. - - ```shell - $ dnf hot-updateinfo list cves --cve CVE-2022-3080 - # cve-id level cold-patch hot-patch - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - CVE-2022-3080 Important/Sec. bind-libs-9.16.23-10.oe2203.aarch64 patch-bind-libs-9.16.23-09-name-1-111.aarch64 - ``` - -3. An empty list will be displayed if the CVE does not exist. - - ```shell - $ dnf hot-updateinfo list cves --cve CVE-2022-3089 - # cve-id level cold-patch hot-patch - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - ``` - -## Hot Patch Statuses - -- A hot patch can be in the following statuses: - - - NOT-APPLIED: The hot patch is not installed. - - - DEACTIVED: The hot patch is installed. - - - ACTIVED: The hot patch is activated. - - - ACCEPT: The hot patch has been accepted and will be applied after a reboot. - - ![Hot patch statuses](./figures/hot_patch_statuses.png) - -## Querying and Changing Hot Patch Statuses - -`hotpatch` can be used to query and convert hot patch statuses. - -```shell -$ dnf hotpatch -General DNF options: - -h, --help, --help-cmd - show command help - --cve CVES, --cves CVES - Include packages needed to fix the given CVE, in updates - -Hotpatch command-specific options: - --list [{cve, cves}] show list of hotpatch - --apply APPLY_NAME apply hotpatch - --remove REMOVE_NAME remove hotpatch - --active ACTIVE_NAME active hotpatch - --deactive DEACTIVE_NAME - deactive hotpatch - --accept ACCEPT_NAME accept hotpatch -``` - -1. `dnf hotpatch --list` lists available hot patches in the system. - - ```shell - $ dnf hotpatch --list - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - base-pkg/hotpatch status - redis-6.2.5-1/HP001 NOT-APPLIED - redis-6.2.5-1/HP001 NOT-APPLIED - redis-6.2.5-1/HP002 ACTIVED - redis-6.2.5-1/HP002 ACTIVED - ``` - -2. `dnf hotpatch --list cves` queries hot patches related to CVEs. - - ```shell - $ dnf hotpatch --list cves - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - CVE-id base-pkg/hotpatch status - CVE-2023-1111 redis-6.2.5-1/HP001 NOT-APPLIED - CVE-2023-1112 redis-6.2.5-1/HP001 NOT-APPLIED - CVE-2023-2221 redis-6.2.5-1/HP002 ACTIVED - CVE-2023-2222 redis-6.2.5-1/HP002 ACTIVED - ``` - -3. `dnf hotpatch --list cves --cve ` queries hot patches for specified CVEs. - - ```shell - $ dnf hotpatch --list cves --cve CVE-2023-1111 - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - CVE-id base-pkg/hotpatch status - CVE-2023-1111 redis-6.2.5-1/HP001 NOT-APPLIED - ``` - -4. An empty list will be displayed if the specified CVE does not exist when running `dnf hotpatch --list cves --cve `. - - ```shell - $ dnf hotpatch --list cves --cve CVE-2023-1 - Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. - ``` - -5. `dnf hotpatch --apply ` applies a hot patch. You can run `syscare list` to query the hot patch status after applying the hot patch. For details about hot patch statuses, see the previous section. - - ```shell - $ dnf hotpatch --apply redis-6.2.5-1/HP2 - Last metadata expiration check: 2:38:51 ago on Thu May 25 13:49:28 2023. - Gonna apply this hot patch: redis-6.2.5-1/HP2 - apply hot patch 'redis-6.2.5-1/HP2' succeed - $ syscare list - Uuid Name Status - 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 ACTIVED - ``` - -6. `dnf hotpatch --deactive ` deactivates a hot patch. You can run `syscare list` to query the hot patch status after deactivating the hot patch. For details about hot patch statuses, see the previous section. - - ```shell - $ dnf hotpatch --deactive redis-6.2.5-1/HP2 - Last metadata expiration check: 2:39:10 ago on Thu May 25 13:49:28 2023. - Gonna deactive this hot patch: redis-6.2.5-1/HP2 - deactive hot patch 'redis-6.2.5-1/HP2' succeed - $ syscare list - Uuid Name Status - 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 DEACTIVED - ``` - -7. `dnf hotpatch --remove ` removes a hot patch. You can run `syscare list` to query the hot patch status after removing the hot patch. For details about hot patch statuses, see the previous section. - - ```shell - $ dnf hotpatch --remove redis-6.2.5-1/HP2 - Last metadata expiration check: 2:53:25 ago on Thu May 25 13:49:28 2023. - Gonna remove this hot patch: redis-6.2.5-1/HP2 - remove hot patch 'redis-6.2.5-1/HP2' succeed - $ syscare list - Uuid Name Status - 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 NOT-APPLIED - ``` - -8. `dnf hotpatch --active ` activating a hot patch.You can run `syscare list` to query the hot patch status after activating the hot patch. For details about hot patch statuses, see the previous section. - - ```shell - $ dnf hotpatch --active redis-6.2.5-1/HP2 - Last metadata expiration check: 2:53:37 ago on Thu May 25 13:49:28 2023. - Gonna active this hot patch: redis-6.2.5-1/HP2 - active hot patch 'redis-6.2.5-1/HP2' failed, remain original status. - $ syscare list - Uuid Name Status - 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 ACTIVED - ``` - -9. `dnf hotpatch --accept ` accepts a hot patch. You can run `syscare list` to query the hot patch status after accepting the hot patch. For details about hot patch statuses, see the previous section. - - ```shell - $ dnf hotpatch --accept redis-6.2.5-1/HP2 - Last metadata expiration check: 2:53:25 ago on Thu May 25 13:49:28 2023. - Gonna accept this hot patch: redis-6.2.5-1/HP2 - remove hot patch 'redis-6.2.5-1/HP2' succeed - $ syscare list - Uuid Name Status - 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 ACCEPTED - ``` - -## Applying Hot Patches - -The `hotupgrade` command is used to apply hot patches to fix specified or all CVEs. - -```shell -$ dnf hotupgrade [--cve [cve_id]] [SPEC ...] - -General DNF options: - -h, --help, --help-cmd - show command help - --cve CVES, --cves CVES - Include packages needed to fix the given CVE, in updates - -command-specific options: - SPEC Hotpatch specification -``` - -- Case 1: If hot patches have been installed, `dnf hotupgrade` applies all available hot patches. - -```shell -$ dnf hotupgrade -Last metadata expiration check: 4:04:34 ago on Fri Jun 02 06:33:41 2023. -Gonna apply these hot patches:['patch-redis-6.2.5-1-HP001-1-1.x86_64', 'patch-redis-6.2.5-1-HP002-1-1.x86_64'] -The target package 'redis-6.2.5-1' has a hotpatch 'HP001' applied -Gonna remove these hot patches: ['redis-6.2.5-1/HP001'] -Remove hot patch redis-6.2.5-1/HP001. -Package patch-redis-6.2.5-1-HP001-1-1.x86_64 is already installed. -Package patch-redis-6.2.5-1-HP002-1-1.x86_64 is already installed. -Dependencies resolved. -Nothing to do. -Complete! -Applying hot patch -Apply hot patch succeed: redis-6.2.5-1/HP001. -Apply hot patch failed: redis-6.2.5-1/HP002. -``` - -- Case 2: If hot patches have not been installed, `dnf hotupgrade` installs and applies all available hot patches. - -```shell -$ dnf hotupgrade -Last metadata expiration check: 4:13:16 ago on Fri Jun 02 06:33:41 2023. -Gonna apply these hot patches:['patch-redis-6.2.5-1-HP002-1-1.x86_64', 'patch-redis-6.2.5-1-HP001-1-1.x86_64'] -Package patch-redis-6.2.5-1-HP002-1-1.x86_64 is already installed. -Dependencies resolved. -xxxx(Install messages) -Is this ok [y/N]: y -Downloading Packages: -xxxx(Install process) -Complete! - -Applying hot patch -Apply hot patch succeed: redis-6.2.5-1/HP001. -``` - -- Case 3: `dnf hotupgrade ` upgrades specified hot patches. - -```shell -$ dnf hotupgrade patch-redis-6.2.5-1-HP001-1-1.x86_64 -Last metadata expiration check: 0:07:49 ago on Thu Jun 08 12:03:46 2023. -Package patch-redis-6.2.5-1-HP001-1-1.x86_64 is already installed. -Dependencies resolved. -Nothing to do. -Complete! -Applying hot patch -Apply hot patch succeed: redis-6.2.5-1/HP001. -``` - -- `--cve` - - - Case 1: `dnf hotupgrade --cve ` is used to install and apply a hot patch for a specified CVE. - - ```shell - $ dnf hotupgrade --cve CVE-2021-11 - Last metadata expiration check: xxx - Dependencies resolved. - xxxx(Install messages) - Is this ok [y/N]: y - Downloading Packages: - xxxx(Install process) - Complete! - Applying hot patch - Apply hot patch succeed: redis-6.2.5-1/HP001 - ``` - - - Case 2: `dnf hotupgrade --cve ` is used for a CVE that does not exist. - - ```shell - $ dnf hotupgrade --cve CVE-2021-11 - Last metadata expiration check: xxx - The cve doesnt exist: CVE-2021-11 - Error: No hot patches marked for install. - ``` - - - Case 3: `dnf hotupgrade --cve ` is used to install and apply a hot patch of a higher version for a CVE that has a hot patch of a lower version. The hot patch of the lower version is uninstalled. - - ```shell - $ dnf hotupgrade --cve CVE-2021-22 - Last metadata expiration check: xxx - The target package 'redis-6.2.5-1' has a hotpatch 'HP001' applied - Gonna remove these hot patches: ['redis-6.2.5-1/HP001'] - Is this ok [y/N]: y - Remove hot patch redis-6.2.5-1/HP001 - xxxx (install messages and process install) - Apply hot patch - apply hot patch succeed: redis-6.2.5-1/HP002 - ``` - - - Case 4: `dnf hotupgrade --cve ` is used to install and apply a hot patch for a CVE that has the latest hot patch. - - ```shell - $ dnf hotupgrade --cve CVE-2021-22 - Package patch -redis-6.2.5-1-HP002-1-1.x86_64 is already installed. - Dependencies resolved. - Nothing to do. - Complete! - Applying hot patch - Apply hot patch succeed: redis-6.2.5-1/HP002 - ``` - -- `SPEC` - - ```shell - dnf hotupgrade bind-libs-hotpatch - ``` - -The output of this sub-command is the same as that of `--cve`. - -## Usage Example - -Assume that the repositories of hot and cold patches on this host have been enabled. - -Scan CVEs that can be fixed on the host. - -```shell -$ dnf hot-updateinfo list cves -Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. -CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - -CVE-2021-1 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 -CVE-2021-11 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 -CVE-2021-2 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 -CVE-2021-22 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 -CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - - -``` - -CVE-2021-1, CVE-2021-11, CVE-2021-2, and CVE-2021-22 can be fixed by hot patches. - -Start the Redis service based on the **redis.conf** configuration file. - -```shell -$ sudo redis-server ./redis.conf & -[1] 285075 -285076:C 25 Mar 2023 12:09:51.503 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo -285076:C 25 Mar 2023 12:09:51.503 # Redis version=255.255.255, bits=64, commit=00000000, modified=0, pid=285076, just started -285076:C 25 Mar 2023 12:09:51.503 # Configuration loaded -285076:M 25 Mar 2023 12:09:51.504 * Increased maximum number of open files to 10032 (it was originally set to 1024). -285076:M 25 Mar 2023 12:09:51.504 * monotonic clock: POSIX clock_gettime - _._ - _.-``__ ''-._ - _.-`` `. `_. ''-._ Redis 255.255.255 (00000000/0) 64 bit - .-`` .-```. ```\/ _.,_ ''-._ - ( ' , .-` | `, ) Running in standalone mode - |`-._`-...-` __...-.``-._|'` _.-'| Port: 6380 - | `-._ `._ / _.-' | PID: 285076 - `-._ `-._ `-./ _.-' _.-' - |`-._`-._ `-.__.-' _.-'_.-'| - | `-._`-._ _.-'_.-' | https://redis.io - `-._ `-._`-.__.-'_.-' _.-' - |`-._`-._ `-.__.-' _.-'_.-'| - | `-._`-._ _.-'_.-' | - `-._ `-._`-.__.-'_.-' _.-' - `-._ `-.__.-' _.-' - `-._ _.-' - `-.__.-' - -285076:M 25 Mar 2023 12:09:51.505 # Server initialized -285076:M 25 Mar 2023 12:09:51.505 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. -285076:M 25 Mar 2023 12:09:51.506 * Ready to accept connections - -``` - -Test the function before applying the hot patch. - -```shell -$ telnet 127.0.0.1 6380 -Trying 127.0.0.1... -Connected to 127.0.0.1. -Escape character is '^]'. - -*100 - --ERR Protocol error: expected '$', got ' ' -Connection closed by foreign host. -``` - -Specify CVE-2021-1 and ensure that the related hot patch is associated and applied. - -```shell -$ dnf hotupgrade --cve CVE-2021-1 -Last metadata expiration check: 0:05:19 ago on Sat Mar 25 11:53:46 2023. -Package patch-redis-6.2.5-1-HP001-1-1.x86_64 is already installed. -Dependencies resolved. -Nothing to do. -Complete! -Applying hot patch -Apply hot patch succeed: redis-6.2.5-1/HP001. -``` - -Run `syscare` to check whether the hot patch has been applied (the status is **ACTIVED**). - -```shell -$ syscare list -Uuid Name Status -cf47649c-b370-4f5a-a914-d2ca4d8f1f3a redis-6.2.5-1/HP001 ACTIVED -``` - -Check whether the CVE has been fixed. Because the **patch-redis-6.2.5-1-HP001-1-1.x86_64** hot patch also fixes CVE-2021-11, CVE-2021-1 and CVE-2021-11 no longer exists. - -```shell -$ dnf hot-updateinfo list cves -Last metadata expiration check: 0:08:48 ago on Sat Mar 25 11:53:46 2023. -CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2023-1076 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2023-26607 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - -CVE-2021-22 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 -CVE-2021-2 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 -CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -``` - -Test the function after applying the hot patch. - -```shell -$ telnet 127.0.0.1 6380 -Trying 127.0.0.1... -Connected to 127.0.0.1. -Escape character is '^]'. - -*100 - --ERR Protocol error: unauthenticated multibulk length -Connection closed by foreign host. -``` - -Run `syscare` and specify the patch name to manually remove the hot patch. - -```shell -$ syscare remove redis-6.2.5-1/HP001 -$ syscare list -Uuid Name Status -cf47649c-b370-4f5a-a914-d2ca4d8f1f3a redis-6.2.5-1/HP001 NOT-APPLIED -``` - -Scan the CVEs to be fixed on the host. CVE-2021-1 and CVE-2021-11 are displayed. - -```shell -$ dnf hot-updateinfo list cves -Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. -CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - -CVE-2021-1 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 -CVE-2021-11 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 -CVE-2021-2 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 -CVE-2021-22 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 -CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -``` - -- Case 1 - -Apply hot patch **patch-redis-6.2.5-1-HP002-1-1.x86_64**. - -```shell -$ dnf hotupgrade patch-redis-6.2.5-1-HP002-1-1.x86_64 -Last metadata expiration check: 0:05:19 ago on Sat Mar 25 11:53:46 2023. -Package patch-redis-6.2.5-1-HP002-1-1.x86_64 is already installed. -Dependencies resolved. -Nothing to do. -Complete! -Applying hot patch -Apply hot patch succeed: redis-6.2.5-1/HP002. -``` - -Scan the CVEs to be fixed on the host. Because the cold patch **redis-6.2.5-3.x86_64** corresponding to **patch-redis-6.2.5-1-HP002-1-1.x86_64** is of a higher version than **redis-6.2.5-2.x86_64**, **redis-6.2.5-2.x86_64** also fixes CVE-2021-1, CVE-2021-11, CVE-2021-2, and CVE-2021-22. - -```shell -$ dnf hot-updateinfo list cves -Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. -CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - -CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -``` - -- Case 2 - -Open the **xxx-updateinfo.xml.gz** file in the **repodata** directory of the hot patch repository. Check the information related to CVE-2021-33 and CVE-2021-3. - -```xml - - openEuler-SA-2022-3 - An update for mariadb is now available for openEuler-22.03-LTS - Important - openEuler - - - - - - patch-redis-6.2.5-2-HP001.(CVE-2022-24048) - - - openEuler - - patch-redis-6.2.5-2-HP001-1-1.aarch64.rpm - - - patch-redis-6.2.5-2-HP001-1-1.x86_64.rpm - - - - -``` - -The format of the **name** field of **package** is **patch-\-\-\-\**. In the example, **patch-redis-6.2.5-2-HP001** requires the source code version of redis-6.2.5-2 to be installed. Check the version of Redis on the host. - -```shell -$ rpm -qa | grep redis -redis-6.2.5-1.x86_64 -``` - -redis-6.2.5-2 is not installed. Therefore, the hot patch will not be displayed. - -```shell -$ dnf hot-updateinfo list cves -Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. -CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - -CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - -CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - -CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - -``` diff --git a/docs/en/docs/A-Ops/figures/0BFA7C40-D404-4772-9C47-76EAD7D24E69.png b/docs/en/docs/A-Ops/figures/0BFA7C40-D404-4772-9C47-76EAD7D24E69.png deleted file mode 100644 index 910f58dbf8fb13d52826b7c74728f4c28599660f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/0BFA7C40-D404-4772-9C47-76EAD7D24E69.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/1631073636579.png b/docs/en/docs/A-Ops/figures/1631073636579.png deleted file mode 100644 index 5aacc487264ac63fbe5322b4f89fca3ebf9c7cd9..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/1631073636579.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/1631073840656.png b/docs/en/docs/A-Ops/figures/1631073840656.png deleted file mode 100644 index 122e391eafe7c0d8d081030a240df90aea260150..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/1631073840656.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/1631101736624.png b/docs/en/docs/A-Ops/figures/1631101736624.png deleted file mode 100644 index 74e2f2ded2ea254c66b221e8ac27a0d8bed9362a..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/1631101736624.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/1631101865366.png b/docs/en/docs/A-Ops/figures/1631101865366.png deleted file mode 100644 index abfbc280a368b93af1e1165385af3a9cac89391d..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/1631101865366.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/1631101982829.png b/docs/en/docs/A-Ops/figures/1631101982829.png deleted file mode 100644 index 0b1c9c7c3676b804dbdf19afbe4f3ec9dbe0627f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/1631101982829.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/1631102019026.png b/docs/en/docs/A-Ops/figures/1631102019026.png deleted file mode 100644 index 54e8e7d1cffbb28711074e511b08c73f66c1fb75..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/1631102019026.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/20210908212726.png b/docs/en/docs/A-Ops/figures/20210908212726.png deleted file mode 100644 index f7d399aecd46605c09fe2d1f50a1a8670cd80432..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/20210908212726.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/D466AC8C-2FAF-4797-9A48-F6C346A1EC77.png b/docs/en/docs/A-Ops/figures/D466AC8C-2FAF-4797-9A48-F6C346A1EC77.png deleted file mode 100644 index 4b937ab846017ead71ca8b5a75b8af1f0f28e1ef..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/D466AC8C-2FAF-4797-9A48-F6C346A1EC77.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/a-ops_architecture.png b/docs/en/docs/A-Ops/figures/a-ops_architecture.png deleted file mode 100644 index 7a831b183e8cba5da16b9be9d965abe9811ada5b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/a-ops_architecture.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/add_fault_tree.png b/docs/en/docs/A-Ops/figures/add_fault_tree.png deleted file mode 100644 index 664efd5150fcb96f009ce0eddc3d9ac91b9e622f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/add_fault_tree.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/add_host_group.png b/docs/en/docs/A-Ops/figures/add_host_group.png deleted file mode 100644 index ed4ab3616d418ecf33a006fee3985b8b6d2d965d..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/add_host_group.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/attach_process.png b/docs/en/docs/A-Ops/figures/attach_process.png deleted file mode 100644 index f76e8f4513cb45fbece12e6237039c41786b0467..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/attach_process.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/check.PNG b/docs/en/docs/A-Ops/figures/check.PNG deleted file mode 100644 index 2dce821dd43eec6f0d13cd6b2dc1e30653f35489..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/check.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/dashboard.PNG b/docs/en/docs/A-Ops/figures/dashboard.PNG deleted file mode 100644 index 2a4a827191367309aad28a8a6c1835df602bdf72..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/dashboard.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/deadlock.png b/docs/en/docs/A-Ops/figures/deadlock.png deleted file mode 100644 index d4f863a1a87d7aad3128481c763ee715aefd0a9f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/deadlock.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/deadlock2.png b/docs/en/docs/A-Ops/figures/deadlock2.png deleted file mode 100644 index 3be42a5a34f90c2f3b351c7077635c580ea847a7..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/deadlock2.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/deadlock3.png b/docs/en/docs/A-Ops/figures/deadlock3.png deleted file mode 100644 index 5ef1a08394daf6433e10f85a5b3c57df25c3e303..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/deadlock3.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/decryption.png b/docs/en/docs/A-Ops/figures/decryption.png deleted file mode 100644 index da07cfdf9296e201a82cceb210e651261fe7ecee..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/decryption.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/delete_host_group.png b/docs/en/docs/A-Ops/figures/delete_host_group.png deleted file mode 100644 index e4d85f6e3f1a269a483943f5115f54daa3de51de..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/delete_host_group.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/delete_hosts.png b/docs/en/docs/A-Ops/figures/delete_hosts.png deleted file mode 100644 index b3da935739369dad1318fe135146755ede13c694..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/delete_hosts.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/deploy.PNG b/docs/en/docs/A-Ops/figures/deploy.PNG deleted file mode 100644 index e30dcb0eb05eb4f41202c736863f3e0ff216398d..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/deploy.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/diag.PNG b/docs/en/docs/A-Ops/figures/diag.PNG deleted file mode 100644 index a67e8515b8313a50b06cb985611ef9c166851811..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/diag.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/diag_error1.png b/docs/en/docs/A-Ops/figures/diag_error1.png deleted file mode 100644 index 9e5b1139febe9f00156b37f3268269ac30a78737..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/diag_error1.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/diag_main_page.png b/docs/en/docs/A-Ops/figures/diag_main_page.png deleted file mode 100644 index b536af938250004bac3053b234bf20bcbf075c9b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/diag_main_page.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/diagnosis.png b/docs/en/docs/A-Ops/figures/diagnosis.png deleted file mode 100644 index 2c85102fe28deaac0a35fde85fd4497994d2c031..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/diagnosis.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/domain.PNG b/docs/en/docs/A-Ops/figures/domain.PNG deleted file mode 100644 index bad499f96df5934565d36edf2308cec5e4147719..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/domain.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/domain_config.PNG b/docs/en/docs/A-Ops/figures/domain_config.PNG deleted file mode 100644 index 8995424b35cda75f08881037446b7816a0ca09dc..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/domain_config.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/execute_diag.png b/docs/en/docs/A-Ops/figures/execute_diag.png deleted file mode 100644 index afb5f7e9fbfb1d1ce46d096a61729766b4940cd3..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/execute_diag.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/flame_muti_ins.png b/docs/en/docs/A-Ops/figures/flame_muti_ins.png deleted file mode 100644 index 5943c7fda223a7fde4d2987ad56af4ffa776bd81..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/flame_muti_ins.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/host.PNG b/docs/en/docs/A-Ops/figures/host.PNG deleted file mode 100644 index 3c00681a567cf8f1e1baddfb6fdb7b6cf7df43de..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/host.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/hosts.png b/docs/en/docs/A-Ops/figures/hosts.png deleted file mode 100644 index f4c7b9103baab7748c83392f6120c8f00880860f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/hosts.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/hosts_in_group.png b/docs/en/docs/A-Ops/figures/hosts_in_group.png deleted file mode 100644 index 9f188d207162fa1418a61a10f83ef9c51a512e65..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/hosts_in_group.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/lockcompete1.png b/docs/en/docs/A-Ops/figures/lockcompete1.png deleted file mode 100644 index 5848b114e02d09f23303da8cff7aef56216f655f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/lockcompete1.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/lockcompete2.png b/docs/en/docs/A-Ops/figures/lockcompete2.png deleted file mode 100644 index ed02a882a145dafeafb76469f328085edecc6775..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/lockcompete2.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/lockcompete3.png b/docs/en/docs/A-Ops/figures/lockcompete3.png deleted file mode 100644 index 3992edc5b7ea61d8a2aa08ce47f0876b7d2e8cf3..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/lockcompete3.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/lockcompete4.png b/docs/en/docs/A-Ops/figures/lockcompete4.png deleted file mode 100644 index 049ac49bcc1fb71ea9fe6866bd27e84d0acf42b1..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/lockcompete4.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/lockcompete5.png b/docs/en/docs/A-Ops/figures/lockcompete5.png deleted file mode 100644 index 8b5cf5aaef43f125abdf3adb8a7f798dd2c86b54..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/lockcompete5.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/lockcompete6.png b/docs/en/docs/A-Ops/figures/lockcompete6.png deleted file mode 100644 index c3b1f5f097b9e9bcabf75229eabc6ce8fe126a71..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/lockcompete6.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/spider.PNG b/docs/en/docs/A-Ops/figures/spider.PNG deleted file mode 100644 index 53bad6dd38e36db9cadfdbeda21cbc3ef59eddf7..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/spider.PNG and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/spider_detail.jpg b/docs/en/docs/A-Ops/figures/spider_detail.jpg deleted file mode 100644 index b69636fe2161380be56f37caf7fd904d2e63e302..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/spider_detail.jpg and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/tprofiling-dashboard-detail.png b/docs/en/docs/A-Ops/figures/tprofiling-dashboard-detail.png deleted file mode 100644 index 2093808bc4e1654956f6143393757c1244f08f98..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/tprofiling-dashboard-detail.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/tprofiling-dashboard.png b/docs/en/docs/A-Ops/figures/tprofiling-dashboard.png deleted file mode 100644 index 15f4917f5a0bfcf5dee1f8fe68e65635ffebd85e..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/tprofiling-dashboard.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/tprofiling-run-arch.png b/docs/en/docs/A-Ops/figures/tprofiling-run-arch.png deleted file mode 100644 index e18e28672beb6306050c42cab1b46f588de83eaa..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/tprofiling-run-arch.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/view_fault_tree.png b/docs/en/docs/A-Ops/figures/view_fault_tree.png deleted file mode 100644 index a566417b18e8bcf19153730904893fc8d827d885..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/view_fault_tree.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/view_report.png b/docs/en/docs/A-Ops/figures/view_report.png deleted file mode 100644 index 2029141179302ecef45d34cb0c9dc916b9142e7b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/view_report.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/figures/view_report_list.png b/docs/en/docs/A-Ops/figures/view_report_list.png deleted file mode 100644 index 58307ec6ef4c73b6b0f039b1052e5870629ac2e8..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/figures/view_report_list.png and /dev/null differ diff --git a/docs/en/docs/A-Ops/image/image-20230607161545732.png.jpg b/docs/en/docs/A-Ops/image/image-20230607161545732.png.jpg deleted file mode 100644 index ad4d525d163c2e001f7980cde8712519a13125d6..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Ops/image/image-20230607161545732.png.jpg and /dev/null differ diff --git a/docs/en/docs/A-Ops/overview.md b/docs/en/docs/A-Ops/overview.md deleted file mode 100644 index 1dff96bb38202330db97f431a91f0c27919071fd..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/overview.md +++ /dev/null @@ -1,3 +0,0 @@ -# A-Ops User Guide - -This document describes the A-Ops intelligent O&M framework and how to install and use services such as intelligent location and configuration source tracing, helping you quickly understand and use A-Ops. By using A-Ops, you can reduce the O&M cost of the system cluster, quickly locate system faults, and centrally manage configuration items. diff --git a/docs/en/docs/A-Ops/using-gala-anteater.md b/docs/en/docs/A-Ops/using-gala-anteater.md deleted file mode 100644 index 92907378c87b4d1b338584bc598db0763aadb5bf..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/using-gala-anteater.md +++ /dev/null @@ -1,157 +0,0 @@ -# Using gala-anteater - -gala-anteater is an AI-based operating system exception detection platform. It provides functions such as time series data preprocessing, exception detection, and exception reporting. Based on offline pre-training, online model incremental learning and model update, it can be well adapted to multi-dimensional and multi-modal data fault diagnosis. - -This chapter describes how to deploy and use the gala-anteater service. - -#### Installation - -Mount the repo sources. - -```basic -[oe-2209] # openEuler 22.09 officially released repository -name=oe2209 -baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 -enabled=1 -gpgcheck=0 -priority=1 - -[oe-2209:Epol] # openEuler 22.09: Epol officially released repository -name=oe2209_epol -baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ -enabled=1 -gpgcheck=0 -priority=1 -``` - -Install gala-anteater. - -```bash -# yum install gala-anteater -``` - - - -#### Configuration - -> Note: gala-anteater does not contain the config file that needs to be configured. Its parameters are passed through the startup parameters using the command line. - -##### Startup Parameters - -| Parameter| Parameter Full Name| Type| Mandatory (Yes/No)| Default Value| Name| Description| -|---|---|---|---|---|---|---| -| -ks | --kafka_server | string | True | | KAFKA_SERVER | IP address of the Kafka server, for example, **localhost / xxx.xxx.xxx.xxx**.| -| -kp | --kafka_port | string | True | | KAFKA_PORT | Port number of the Kafka server, for example, **9092**.| -| -ps | --prometheus_server | string | True | | PROMETHEUS_SERVER | IP address of the Prometheus server, for example, **localhost / xxx.xxx.xxx.xxx**.| -| -pp | --prometheus_port | string | True | | PROMETHEUS_PORT | Port number of the Prometheus server, for example, **9090**.| -| -m | --model | string | False | vae | MODEL | Exception detection model. Currently, two exception detection models are supported: **random_forest** and **vae**.
**random_forest**: random forest model, which does not support online learning
**vae**: Variational Atuoencoder (VAE), which is an unsupervised model and supports model update based on historical data during the first startup.| -| -d | --duration | int | False | 1 | DURATION | Frequency of executing the exception detection model. The unit is minute, which means that the detection is performed every *x* minutes.| -| -r | --retrain | bool | False | False | RETRAIN | Whether to use historical data to update and iterate the model during startup. Currently, only the VAE model is supported.| -| -l | --look_back | int | False | 4 | LOOK_BACK | Whether to update the model based on the historical data of the last *x* days.| -| -t | --threshold | float | False | 0.8 | THRESHOLD | Threshold of the exception detection model, ranging from 0 to 1. A larger value can reduce the false positive rate of the model. It is recommended that the value be greater than or equal to 0.5.| -| -sli | --sli_time | int | False | 400 | SLI_TIME | Application performance metric. The unit is ms. A larger value can reduce the false positive rate of the model. It is recommended that the value be greater than or equal to 200.
For scenarios with a high false positive rate, it is recommended that the value be greater than 1000.| - - - -#### Start - -Start gala-anteater. - -> Note: gala-anteater can be started and run in command line mode, but cannot be started and run in systemd mode. - -- Running in online training mode (recommended) -```bash -gala-anteater -ks {ip} -kp {port} -ps {ip} -pp {port} -m vae -r True -l 7 -t 0.6 -sli 400 -``` - -- Running in common mode -```bash -gala-anteater -ks {ip} -kp {port} -ps {ip} -pp {port} -m vae -t 0.6 -sli 400 -``` - -Query the gala-anteater service status. - -If the following information is displayed, the service is started successfully. The startup log is saved to the **logs/anteater.log** file in the current running directory. - -```log -2022-09-01 17:52:54,435 - root - INFO - Run gala_anteater main function... -2022-09-01 17:52:54,436 - root - INFO - Start to try updating global configurations by querying data from Kafka! -2022-09-01 17:52:54,994 - root - INFO - Loads metric and operators from file: xxx\metrics.csv -2022-09-01 17:52:54,997 - root - INFO - Loads metric and operators from file: xxx\metrics.csv -2022-09-01 17:52:54,998 - root - INFO - Start to re-train the model based on last day metrics dataset! -2022-09-01 17:52:54,998 - root - INFO - Get training data during 2022-08-31 17:52:00+08:00 to 2022-09-01 17:52:00+08:00! -2022-09-01 17:53:06,994 - root - INFO - Spends: 11.995422840118408 seconds to get unique machine_ids! -2022-09-01 17:53:06,995 - root - INFO - The number of unique machine ids is: 1! -2022-09-01 17:53:06,996 - root - INFO - Fetch metric values from machine: xxxx. -2022-09-01 17:53:38,385 - root - INFO - Spends: 31.3896164894104 seconds to get get all metric values! -2022-09-01 17:53:38,392 - root - INFO - The shape of training data: (17281, 136) -2022-09-01 17:53:38,444 - root - INFO - Start to execute vae model training... -2022-09-01 17:53:38,456 - root - INFO - Using cpu device -2022-09-01 17:53:38,658 - root - INFO - Epoch(s): 0 train Loss: 136.68 validate Loss: 117.00 -2022-09-01 17:53:38,852 - root - INFO - Epoch(s): 1 train Loss: 113.73 validate Loss: 110.05 -2022-09-01 17:53:39,044 - root - INFO - Epoch(s): 2 train Loss: 110.60 validate Loss: 108.76 -2022-09-01 17:53:39,235 - root - INFO - Epoch(s): 3 train Loss: 109.39 validate Loss: 106.93 -2022-09-01 17:53:39,419 - root - INFO - Epoch(s): 4 train Loss: 106.48 validate Loss: 103.37 -... -2022-09-01 17:53:57,744 - root - INFO - Epoch(s): 98 train Loss: 97.63 validate Loss: 96.76 -2022-09-01 17:53:57,945 - root - INFO - Epoch(s): 99 train Loss: 97.75 validate Loss: 96.58 -2022-09-01 17:53:57,969 - root - INFO - Schedule recurrent job with time interval 1 minute(s). -2022-09-01 17:53:57,973 - apscheduler.scheduler - INFO - Adding job tentatively -- it will be properly scheduled when the scheduler starts -2022-09-01 17:53:57,974 - apscheduler.scheduler - INFO - Added job "partial" to job store "default" -2022-09-01 17:53:57,974 - apscheduler.scheduler - INFO - Scheduler started -2022-09-01 17:53:57,975 - apscheduler.scheduler - DEBUG - Looking for jobs to run -2022-09-01 17:53:57,975 - apscheduler.scheduler - DEBUG - Next wakeup is due at 2022-09-01 17:54:57.973533+08:00 (in 59.998006 seconds) -``` - - - -#### Output Data - -If gala-anteater detects an exception, it sends the result to Kafka. The output data format is as follows: - -```json -{ - "Timestamp":1659075600000, - "Attributes":{ - "entity_id":"xxxxxx_sli_1513_18", - "event_id":"1659075600000_1fd37742xxxx_sli_1513_18", - "event_type":"app" - }, - "Resource":{ - "anomaly_score":1.0, - "anomaly_count":13, - "total_count":13, - "duration":60, - "anomaly_ratio":1.0, - "metric_label":{ - "machine_id":"1fd37742xxxx", - "tgid":"1513", - "conn_fd":"18" - }, - "recommend_metrics":{ - "gala_gopher_tcp_link_notack_bytes":{ - "label":{ - "__name__":"gala_gopher_tcp_link_notack_bytes", - "client_ip":"x.x.x.165", - "client_port":"51352", - "hostname":"localhost.localdomain", - "instance":"x.x.x.172:8888", - "job":"prometheus-x.x.x.172", - "machine_id":"xxxxxx", - "protocol":"2", - "role":"0", - "server_ip":"x.x.x.172", - "server_port":"8888", - "tgid":"3381701" - }, - "score":0.24421279500639545 - }, - ... - }, - "metrics":"gala_gopher_ksliprobe_recent_rtt_nsec" - }, - "SeverityText":"WARN", - "SeverityNumber":14, - "Body":"TimeStamp, WARN, APP may be impacting sli performance issues." -} -``` diff --git a/docs/en/docs/A-Ops/using-gala-gopher.md b/docs/en/docs/A-Ops/using-gala-gopher.md deleted file mode 100644 index e3bfae8300c64ada481ea7ef755c785aaacfebe6..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/using-gala-gopher.md +++ /dev/null @@ -1,1087 +0,0 @@ -# Using gala-gopher - -As a data collection module, gala-gopher provides OS-level monitoring capabilities, supports dynamic probe installation and uninstallation, and integrates third-party probes in a non-intrusive manner to quickly expand the monitoring scope. - -This chapter describes how to deploy and use the gala-gopher service. - -## Installation - -Mount the repo sources. - -```basic -[oe-{version}] # openEuler official released repository -name=oe{version} -baseurl=https://repo.openeuler.org/openEuler-{version}/everything/$basearch/ -enabled=1 -gpgcheck=0 -priority=1 - -[oe-{version}:Epol] # openEuler:Epol official released repository -name=oe{version}_epol -baseurl=https://repo.openeuler.org/openEuler-{version}/EPOL/main/$basearch/ -enabled=1 -gpgcheck=0 -priority=1 -``` - -Install gala-gopher. - -RPM installation applies to single-node observation scenarios without containers. openEuler 22.03 LTS SP1 or later is supported. - -```bash -# yum install gala-gopher -``` - -## Configuration - -### Configuration Description - -The configuration file of gala-gopher is **/etc/gala-gopher/gala-gopher.conf**. The configuration items in the file are described as follows (the parts that do not need to be manually configured are not described): - -The following configurations can be modified as required: - -- **global**: gala-gopher global configuration information. - - **log_file_name**: gala-gopher log file name. - - **log_level**: gala-gopher log level (not supported currently) - - **pin_path**: path for storing the map shared by the eBPF probe. You are advised to retain the default value. -- **metric**: metric output mode. - - **out_channel**: metric output channel. The value can be **web_server**, **logs**, or **kafka**. If this parameter is left empty, the output channel is disabled. - - **kafka_topic**: topic configuration information if the output channel is Kafka. -- **event**: output mode of abnormal events. - - **out_channel**: event output channel. The value can be **logs** or **kafka**. If this parameter is left empty, the output channel is disabled. - - **kafka_topic**: topic configuration information if the output channel is Kafka. - - **timeout**: interval for reporting the same abnormal event - - **desc_language**: Language of the abnormal event description. Currently, the value can be **zh_CN** or **en_US**. -- **meta**: metadata output mode. - - **out_channel**: metadata output channel. The value can be **logs** or **kafka**. If this parameter is left empty, the output channel is disabled. - - **kafka_topic**: topic configuration information if the output channel is Kafka. -- **ingress**: configuration related to probe data reporting - - **interval**: not in use -- **egress**: configuration related to database reporting - - **interval**: not in use - - **time_range**: not in use -- **imdb**: cache specification configuration. - - **max_tables_num**: maximum number of cache tables. In the **/opt/gala-gopher/meta** directory, each meta corresponds to a table. - - **max_records_num**: maximum number of records in each cache table. Generally, each probe generates at least one observation record in an observation period. - - **max_metrics_num**: maximum number of metrics contained in each observation record. - - **record_timeout**: aging time of the cache table. If a record in the cache table is not updated within the aging time, the record is deleted. The unit is second. -- **web_server**: configuration of the web_server output channel. - - **port**: listening port. - - **ssl_auth**: whether to enable HTTPS encryption and authentication for the web server. The value can be **on** or **off**. You are advised to enable it in the production environment. - - **private_key**: absolute path of the server private key file used for web server HTTPS encryption. This parameter is mandatory when **ssl_auth** is **on**. - - **cert_file**: absolute path of the server certificate used for web server HTTPS encryption. This parameter is mandatory when **ssl_auth** is **on**. - - **ca_file**: absolute path of the CA certificate used by the web server to authenticate the client. This parameter is mandatory when **ssl_auth** is **on**. -- rest_api_server - - **port**: RestFul API listening port. - - **ssl_auth**: whether to enable HTTPS encryption and authentication for the RESTful API. The value can be **on** or **off**. You are advised to enable it in the production environment. - - **private_key**: absolute path of the server private key file used for RESTful API HTTPS encryption. This parameter is mandatory when **ssl_auth** is **on**. - - **cert_file**: absolute path of the server certificate used for RESTful API HTTPS encryption. This parameter is mandatory when **ssl_auth** is **on**. - - **ca_file**: absolute path of the CA certificate used by the RESTful API to authenticate the client. This parameter is mandatory when **ssl_auth** is **on**. -- **kafka**: configuration of the Kafka output channel. - - **kafka_broker**: IP address and port number of the Kafka server. - - **batch_num_messages**: number of messages sent in each batch. - - **compression_codec**: message compression type. - - **queue_buffering_max_messages**: maximum number of messages allowed in the producer buffer. - - **queue_buffering_max_kbytes**: maximum number of bytes allowed in the producer buffer. - - **queue_buffering_max_ms**: maximum time for a producer to wait for more messages to join a batch before sending the batch. -- **logs**: configuration of the logs output channel. - - **metric_dir**: path for storing metric data logs. - - **event_dir**: path for storing abnormal event data logs. - - **meta_dir**: metadata log path. - - **debug_dir**: path of gala-gopher run logs. - -### Configuration File Example - -- Select the data output channels. - - ```yaml - metric = - { - out_channel = "web_server"; - kafka_topic = "gala_gopher"; - }; - - event = - { - out_channel = "kafka"; - kafka_topic = "gala_gopher_event"; - }; - - meta = - { - out_channel = "kafka"; - kafka_topic = "gala_gopher_metadata"; - }; - ``` - -- Configure Kafka and Web Server. - - ```yaml - web_server = - { - port = 8888; - ssl_auth = "off"; - private_key = ""; - cert_file = ""; - ca_file = ""; - }; - - kafka = - { - kafka_broker = ":9092"; - }; - ``` - -## Start - -After the configuration is complete, start gala-gopher. - -```bash -# systemctl start gala-gopher.service -``` - -Query the status of the gala-gopher service. - -```bash -# systemctl status gala-gopher.service -``` - -If the following information is displayed, the service is started successfully: Check whether the enabled probe is started. If the probe thread does not exist, check the configuration file and gala-gopher run log file. - -![gala-gopher successful start](./figures/gala-gopher_start_success.png) - -> Note: The root permission is required for deploying and running gala-gopher. - -### Dynamic Configuration RESTful APIs - -The port number of the web server is configurable (9999 by default). The URL format is **http://\[IP address of the gala-gopher node]:\[Port number]/\[Function]**. For example, the URL of the flame graph is ****. (The following uses the flame graph as an example.) - -#### Configuring the Probe Monitoring Scope - -The probe is disabled by default. You can dynamically enable the probe and set the monitoring scope through the API. Take the flame graph as an example. Use the REST API to enable the oncpu, offcpu, and mem flame graph capabilities. The monitoring scope can be set by process ID, process name, container ID, or Pod. - -The following is an example of enabling oncpu and offcpu information collection for a flame graph: - -```bash -curl -X PUT http://localhost:9999/flamegraph --data-urlencode json=' -{ - "cmd": { - "bin": "/opt/gala-gopher/extend_probes/stackprobe", - "check_cmd": "", - "probe": [ - "oncpu", - "offcpu" - ] - }, - "snoopers": { - "proc_id": [ - 101, - 102 - ], - "proc_name": [ - { - "comm": "app1", - "cmdline": "", - "debugging_dir": "" - }, - { - "comm": "app2", - "cmdline": "", - "debugging_dir": "" - } - ], - "pod_id": [ - "pod1", - "pod2" - ], - "container_id": [ - "container1", - "container2" - ] - } -}' -``` - -The collection features are described as follows: - -| Feature | Description | Collection Subitems | Supported Monitored Object Range | Startup File | Startup Condition | -| ------------- | -------------------------------------------- | ------------------------------------------------------------ | ---------------------------------------- | ---------------------------------- |---------------------------| -| flamegraph | On-line performance flame graph observation capability | oncpu, offcpu, mem | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/stackprobe | NA | -| l7 | Application layer (layer 7) protocol observation capability | l7_bytes_metrics,l7_rpc_metrics,l7_rpc_trace | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/l7probe | NA | -| tcp | TCP exception and status observation capability | tcp_abnormal, tcp_rtt, tcp_windows, tcp_rate, tcp_srtt, tcp_sockbuf, tcp_stats,tcp_delay | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/tcpprobe | NA | -| socket | Socket (TCP/UDP) exception observation capability | tcp_socket, udp_socket | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/endpoint | NA | -| io | Block layer I/O observation capability | io_trace, io_err, io_count, page_cache | NA | /opt/gala-gopher/extend_probes/ioprobe | NA | -| proc | Process system call, I/O, DNS, VFS, and ioctl observation capabilities| proc_syscall, proc_fs, proc_io, proc_dns,proc_pagecache,proc_net,proc_offcpu,proc_ioctl | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/taskprobe | NA | -| jvm | JVM layer GC, thread, memory, and cache observation capabilities | NA | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/jvmprobe | NA | -| ksli | Redis performance SLI (access latency) observation capability | NA | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/ksliprobe | NA | -| postgre_sli | PostgreSQL database performance SLI (access latency) observation capability | NA | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/pgsliprobe | NA | -| opengauss_sli | openGauss access throughput observation capability | NA | \[ip, port, dbname, user,password] | /opt/gala-gopher/extend_probes/pg_stat_probe.py | NA | -| dnsmasq | DNS session observation capability | NA | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/rabbitmq_probe.sh | NA | -| lvs | LVS session observation capability | NA | NA | /opt/gala-gopher/extend_probes/trace_lvs | lsmod\|grep ip_vs\| wc -l | -| nginx | Nginx layer 4/layer 7 session observation capability | NA | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/nginx_probe | NA | -| haproxy | HAProxy layer 4/layer 7 session observation capability | NA | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/trace_haproxy | NA | -| kafka | Kafka producer/consumer topic observation capability | NA | NA | /opt/gala-gopher/extend_probes/kafkaprobe | NA | -| baseinfo | Basic system information | cpu, mem, nic, disk, net, fs, proc, host, con | proc_id, proc_name, pod_id, container_id | system_infos | NA | -| virt | Virtualization management information | NA | NA | virtualized_infos | NA | -| tprofiling | Thread-level performance profiling observation capability | oncpu, syscall_file, syscall_net, syscall_lock, syscall_sched | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/tprofiling | NA | -| container | Container information | NA | proc_id, proc_name, container_id | /opt/gala-gopher/extend_probes/cadvisor_probe.py | NA | -| sermant | Java application layer 7 protocol observation capability. Currently, the dubbo protocol is supported.| l7_bytes_metrics, l7_rpc_metrics | proc_id, proc_name, pod_id, container_id | /opt/gala-gopher/extend_probes/sermant_probe.py | NA | - -#### Configuring the Probe Monitoring Parameters - -During probe running, you need to set some parameters, such as the sampling period and reporting period of the flame graph. - -```bash -curl -X PUT http://localhost:9999/flamegraph --data-urlencode json=' -{ - "params": { - "report_period": 180, - "sample_period": 180, - "metrics_type": [ - "raw", - "telemetry" - ] - } -}' -``` - -The detailed running parameters are as follows: - -| Parameter | Description | Default Value & Range | Unit | Supported Features | Supported or Not| -| :-----------------: | :--------------------------------: | :----------------------------------------------------------: |:-------:| :-----------------------------------------: | :--------: | -| sample_period | Sampling period | 5000, \[100~10000] | ms | io, tcp | Y | -| report_period | Report period | 60, \[5~600] | s | ALL | Y | -| latency_thr | Latency report threshold | 0, \[10~100000] | ms | tcp, io, proc, ksli | Y | -| offline_thr | Process offline report threshold | 0, \[10~100000] | ms | proc | Y | -| drops_thr | Packet loss report threshold | 0, \[10~100000] | package | tcp, nic | Y | -| res_lower_thr | Lower resource threshold | 0, \[0~100] | percent | ALL | Y | -| res_upper_thr | Upper resource threshold | 0, \[0~100] | percent | ALL | Y | -| report_event | Exception event report | 0, \[0, 1] | - | ALL | Y | -| metrics_type | Telemetry metric report | "raw", \["raw", "telemetry"] | - | ALL | N | -| env | Environment type | "node", \["node", "container", "kubenet"] | - | ALL | N | -| l7_protocol | Layer 7 protocol scope | "",\["http", "pgsql", "redis","mysql", "kafka", "mongo", "dns"] | - | l7 | Y | -| support_ssl | SSL observation support | 0, \[0, 1] | - | l7 | Y | -| multi_instance | Independent flame graph for each process | 0, \[0, 1] | - | flamegraph | Y | -| native_stack | Local language stack display (for Java processes)| 0, \[0, 1] | - | flamegraph | Y | -| cluster_ip_backend | Cluster IP backend conversion | 0, \[0, 1] | - | tcp, l7 | Y | -| pyroscope_server | IP address of the flame graph UI server | "localhost:4040" | - | flamegraph | Y | -| svg_period | Flame graph SVG file generation period | 180, \[30, 600] | s | flamegraph | Y | -| perf_sample_period | Stack information collection period for **oncpu** flame graphs | 10, \[10, 1000] | ms | flamegraph | Y | -| svg_dir | Directory for storing flame graph SVG files | "/var/log/gala-gopher/stacktrace" | - | flamegraph | Y | -| flame_dir | Directory for storing original stack information of flame graphs | "/var/log/gala-gopher/flamegraph" | - | flamegraph | Y | -| dev_name | Observed NIC/drive device name | "" | - | io, kafka, ksli, postgre_sli, baseinfo, tcp| Y | -| continuous_sampling | Continuous sampling | 0, \[0, 1] | - | ksli | Y | -| elf_path | Observed executable file path | "" | - | baseinfo, nginx, haproxy, dnsmasq | Y | -| kafka_port | Observed Kafka port | 9092, \[1, 65535] | - | kafka | Y | -| cadvisor_port | Started cAdvisor port | 8080, \[1, 65535] | - | cadvisor | Y | - -#### Starting and Stopping a Probe - -```bash -curl -X PUT http://localhost:9999/flamegraph --data-urlencode json=' -{ - "state": "running" // optional: running,stopped -}' -``` - -#### Restrictions - -1. The interface is stateless. The settings uploaded each time are the final running results of the probe, including the status, parameters, and monitoring scope. -2. The monitored objects can be combined randomly, and the monitoring scope is the combination. -3. The startup file must be authentic and valid. -4. You can enable some or all subitems of a collection feature as required. However, you can only disable all subitems of a collection feature at once. -5. The monitored object of OpenGauss is the database instance (**ip**/**port**/**dbname**/**user**/**password**). -6. The interface can receive a maximum of 2048 bytes of data each time. - -#### Obtaining the Probe Configurations and Running Status - -```bash -curl -X GET http://localhost:9999/flamegraph -{ - "cmd": { - "bin": "/opt/gala-gopher/extend_probes/stackprobe", - "check_cmd": "" - "probe": [ - "oncpu", - "offcpu" - ] - }, - "snoopers": { - "proc_id": [ - 101, - 102 - ], - "proc_name": [ - { - "comm": "app1", - "cmdline": "", - "debugging_dir": "" - }, - { - "comm": "app2", - "cmdline": "", - "debugging_dir": "" - } - ], - "pod_id": [ - "pod1", - "pod2" - ], - "container_id": [ - "container1", - "container2" - ] - }, - "params": { - "report_period": 180, - "sample_period": 180, - "metrics_type": [ - "raw", - "telemetry" - ] - }, - "state": "running" -} -``` - -## Introduction to stackprobe - -stackprobe generates performance flame graphs for cloud native environments. - -### Features - -- C/C++, Go, Rust, and Java applications can be observed. - -- The call stack supports container and process granularities. For processes in a container, the pod name and container name of the workload are prefixed with **\[Pod]** and **\[Con]** at the bottom of the call stack, respectively. A process name is prefixed with **\[__]**. Threads and functions (methods) do not have prefixes. - -- The flame graph in SVG format can be generated locally or the call stack data can be uploaded to the middleware. - -- Flame graphs can be generated or uploaded by multiple instances based on the process granularity. - -- For the flame graph of a Java process, both local and Java methods can be displayed. - -- Supports multiple types of flame graphs, such as **oncpu**, **offcpu**, and **mem**. - -- The sampling period can be customized. - -### Usage Instruction - -Startup command example (basic): Use the default parameters to start the performance flame graph. - -```shell -curl -X PUT http://localhost:9999/flamegraph -d json='{ "cmd": {"probe": ["oncpu"] }, "snoopers": {"proc_name": [{ "comm": "cadvisor"}] }, "state": "running"}' -``` - -Startup command example (advanced): Use custom parameters to start the performance flame graph. For details about the configurable parameters, see [Configuring the Probe Monitoring Parameters](#configuring-the-probe-monitoring-parameters). - -```shell -curl -X PUT http://localhost:9999/flamegraph -d json='{ "cmd": { "check_cmd": "", "probe": ["oncpu", "offcpu", "mem"] }, "snoopers": { "proc_name": [{ "comm": "cadvisor", "cmdline": "", "debugging_dir": "" }, { "comm": "java", "cmdline": "", "debugging_dir": "" }] }, "params": { "perf_sample_period": 100, "svg_period": 300, "svg_dir": "/var/log/gala-gopher/stacktrace", "flame_dir": "/var/log/gala-gopher/flamegraph", "pyroscope_server": "localhost:4040", "multi_instance": 1, "native_stack": 0 }, "state": "running"}' -``` - -The main configuration items are described as follows: - -- Set the type of the flame graph to be enabled. - - The **probe** parameter can be set to **oncpu**, **offcpu**, or **mem**, indicating the CPU occupation time, blocking time, and memory size of a process, respectively. - - Example: - - ```json - "probe": ["oncpu", "offcpu", "mem"] - ``` - -- Set the period for generating a local flame graph SVG file. - - The **svg_period** parameter can be set to an integer ranging from \[30, 600]. The unit is second. The default value is **180**. - - Example: - - ```json - "svg_period": 300 - ``` - -- Enable or disable stack information uploading to Pyroscope - - Set the **pyroscope_server** parameter. The parameter value must contain **addr** and **port**. If the parameter is empty or the format is incorrect, the probe does not attempt to upload stack information. - - The upload period is 30s. - - Example: - - ```json - "pyroscope_server": "localhost:4040" - ``` - -- Set the call stack sampling period. - - Set the **perf_sample_period** parameter. The unit is millisecond. The default value is **10**. You can set this parameter to an integer ranging from \[10, 1000]. This parameter is valid only for flame graphs of the **oncpu** type. - - Example: - - ```json - "perf_sample_period": 100 - ``` - -- Enable or disable flame graph generation for multiple instances - - Set the **multi_instance** parameter. The value can be **0** or **1**. The default value is **0**. **0** indicates that the flame graphs of all processes are combined. **1** indicates that the flame graph of each process is generated separately. - - Example: - - ```json - "multi_instance": 1 - ``` - -- Enable or disable custom local call stack collection. - - Set the **native_stack** parameter. The value can be **0** or **1**. The default value is **0**. This parameter is valid only for Java processes. **0** indicates that the JVM call stack is not collected. **1** indicates that the JVM call stack is collected. - - Example: - - ```json - "native_stack": 1 - ``` - - Output: (left: **"native_stack": 1**; right: **"native_stack": 0**) - - ![image-20230804172905729](./figures/flame_muti_ins.png) - -### Implementation - -#### 1. User-Mode Program Logic - -The program periodically (every 30s) converts the stack information reported by the kernel mode from an address to a symbol based on the symbol table. Then, the flamegraph plugin or pyroscope is used to convert the symbolic call stack into a flame graph. - -Methods for obtaining the symbol tables for the code segment type are different. - -- Obtaining the kernel symbol table: Read **/proc/kallsyms**. - -- Obtaining the local language symbol table: Query the virtual memory mapping file (**/proc/***{pid}***/maps**) of the process to obtain the address mapping of each code segment in the process memory, and then use the libelf library to load the symbol table of the module corresponding to each code segment. - -- Obtaining the Java language symbol table: - - Because Java methods do not statically map to the virtual address space of the process, other ways are used to obtain the symbolic Java call stack. - -##### Method 1: perf - -Load the JVM agent dynamic library to the Java process to trace JVM method compilation and loading events, obtain and record the mapping between memory addresses and Java symbols, and generate the symbol table of the Java process in real time. This method requires the Java process to enable the `-XX:+PreserveFramePointer` startup parameter. The advantage of this method is that the call stack of the JVM can be displayed in the flame graph, and the Java flame graph generated in this method can be displayed together with the flame graphs of other processes. - -##### Method 2: JFR - -Track various events and metrics of Java applications by dynamically turning on the JVM built-in analyzer JFR. To enable JFR, load the Java agent to the Java process. The Java agent invokes the JFR API. The advantage of this method is that the collection of Java method call stacks is more accurate and detailed. - -The preceding two performance analysis methods for Java processes can be loaded in real time (without restarting the Java process) and have the advantage of low overhead. If the stackprobe startup parameters are **"multi_instance": 1** and **"native_stack": 0**, stackprobe uses method 2 to generate Java process flame graphs. Otherwise, method 1 is used. - -#### 2. Kernel-Mode Program Logic - -The kernel-mode logic is implemented based on eBPF. Different flame graph types correspond to different eBPF programs. The eBPF programs traverse the call stacks of the current user mode and kernel mode periodically or by triggering events, and report the call stacks to the user mode. - -##### 2.1 oncpu Flame Graph - -Mount the sampling eBPF program to the **PERF_COUNT_SW_CPU_CLOCK** tracepoint for perf software events to periodically sample the call stack. - -##### 2.2 offcpu Flame Graph - -Mount the sampling eBPF program to the tracepoint (**sched_switch**) for process scheduling. The sampling eBPF program records the time when the process is scheduled and the process ID. When the process is scheduled back, the call stack is sampled. - -#### 2.3 mem Flame Graph - -Mount the sampling eBPF program to the tracepoint (**page_fault_user**) for page faults. When the event is triggered, the call stack is sampled. - -#### 3. Java Language Support - -- Main stackprobe process: - - 1. The IPC message is received to obtain the Java process to be observed. - 2. Use the Java agent loading module to load the JVM agent program **jvm_agent.so** to the Java process to be observed (corresponding to [Method 1](#method-1-perf)) or **JstackProbeAgent.jar** (corresponding to [Method 2](#method-2-jfr)). - 3. The main process of method 1 loads the **java-symbols.bin** file of the corresponding Java process for query during address conversion. The main process of method 2 loads the **stacks-**_{flame_type}_**.txt** file of the corresponding Java process, which can be directly used to generate a flame graph. - -- Java agent loading module - - 1. If a new Java process is found, copy the JVM agent program to the process space **/proc/**_\_**/root/tmp** (because the agent program needs to be visible to the JVM in the container during attach). - - 2. Set the owner of the preceding directory and JVM agent program to be the same as that of the Java process to be observed. - - 3. Start the jvm_attach subprocess and transfers parameters related to the Java process to be observed. - -- JVM agent program - - - **jvm_agent.so**: Registers the JVMTI callback function. - - When the JVM loads a Java method or dynamically compiles a local method, the JVM invokes the callback function. The callback function writes the Java class name, method name, and corresponding memory address to the observed Java process space (**/proc/**_\_**/root/tmp/java-data-**_\_**/java-symbols.bin**). - - - **JstackProbeAgent.jar**: Calls the JFR API. - - Enable the JFR function for 30 seconds, convert the JFR statistics result to the stack format available for the flame graph, and export the result to the observed Java process space (**/proc/**_\_**/root/tmp/java-data-**_\_**/stacks-**_\_**.txt**). For details, see [Introduction to JstackProbe](https://gitee.com/openeuler/gala-gopher/blob/dev/src/probes/extends/java.probe/jstack.probe/readme.md). - -- jvm_attach: Loads the JVM agent program to the JVM of the observed process in real time. - For details, see **sun.tools.attach.LinuxVirtualMachine** and **jattach** in the JDK source code. - - 1. Set its own namespace. (When the JVM loads the agent, the namespace of the loading process must be the same as that of the observed process.) - - 2. Check whether the JVM attach listener is started (whether the UNIX socket file **/proc/**_\_**/root/tmp/.java\_pid**_\_ exists). - - 3. If the process is not started, create **/proc/**_\_**/cwd/.attach_pid**_\_ and send the SIGQUIT signal to the JVM. - - 4. Connect to the UNIX socket. - - 5. If the read response is **0**, the attach operation is successful. - - The following figure shows the attach agent process. - - ![attach process](./figures/attach_process.png) - -### Notes - -- To obtain the best observation effect on Java applications, set the stackprobe startup option to **"multi_instance": 1, "native_stack": 0** to enable JFR observation (JDK8u262+). Otherwise, stackprobe generates a Java flame graph using perf. In perf mode, enable the JVM option `XX:+PreserveFramePointer` (JDK8 or later). - -### Constraints - -- Hotspot JVM-based Java application observation is supported. - -## Introduction to tprofiling - -tprofiling is an eBPF-based thread-level application performance diagnosis tool provided by gala-gopher. It uses the eBPF technology to observe key system performance events of threads and associate rich event content. In this way, the running status and key actions of threads are recorded in real time, helping users quickly identify application performance problems. - -### Features - -From the perspective of the OS, a running application program consists of multiple processes, and each process consists of multiple running threads. tprofiling observes and records key actions (events) performed by these threads during thread running, and displays the actions in a time line on the front-end page. In this way, you can intuitively analyze the actions in a certain period of time, whether an action is performed on the CPU or blocked on a file or network operation. When a performance issue occurs in an application, you can analyze the execution sequence of key performance events of the corresponding thread to quickly demarcate and locate the issue. - -Based on the implemented event observation scope, tProfiling can locate application performance issues in the following scenarios: - -- File I/O time consumption and blocking -- Network I/O time consumption and blocking -- Lock contention -- Deadlock - -As more types of events are supplemented and improved, tprofiling will be able to cover more types of application performance issue scenarios. - -### Event Observation Scope - -Currently, tprofiling supports two types of system performance events: system call events and **oncpu** events. - -**System call events** - -Application performance issues are usually caused by system resource bottlenecks, such as high CPU usage and I/O resource waiting. Applications often access these system resources through system calls. Therefore, key system call events can be observed to identify time-consuming or blocked resource access operations. - -For details about the observed system call events, see [Supported System Call Events](#supported-system-call-events). The events include file operations (**file**), network operations (**net**), lock operations (**lock**), and scheduling operations (**sched**). Some of the observed system call events are as follows: - -- File operations (**file**) - - **read**/**write**: Reading or writing drive files or network resources, which may be time-consuming and blocked. - - **sync**/**fsync**: Synchronizing files to drives. The thread is blocked until the synchronization is complete. -- Network operations (**net**) - - **send**/**recv**: Reading or writing network resources, which may be time-consuming and blocked. -- Lock operations (**lock**) - - **futex**: System call related to user-mode lock implementation. If **futex** is triggered, lock contention occurs and the thread may be blocked. -- Scheduling operations (**sched**): System call events that may change the thread status, such as CPU release, sleep, and waiting for other threads. - - **nanosleep**: The thread enters the sleep state. - - **epoll_wait**: The thread is waiting for the arrival of an I/O event. The thread is blocked before the event arrives. - -**oncpu events** - -The running state of a thread can be **oncpu** or **offcpu** according to whether the thread is running on the CPU. By observing the **oncpu** event of a thread, you can identify whether the thread is performing time-consuming CPU operations. - -### Event Information - -Thread profiling events include the following information: - -- Event sources: thread ID, thread name, process ID, process name, container ID, container name, host ID, and host name of the event. - - - **thread.pid**: ID of the thread to which the event belongs. - - **thread.comm**: name of the thread to which the event belongs. - - **thread.tgid**: ID of the process to which the event belongs. - - **proc.name**: name of the process to which the event belongs. - - **container.id**: ID of the container to which the event belongs. - - **container.name**: name of the container to which the event belongs. - - **host.id**: ID of the host to which the event belongs. - - **host.name**: name of the host to which the event belongs. - -- Event attributes: common event attributes and extended event attributes. - - - Common event attributes: - - - **event.name**: event name. - - **event.type**: event type. Currently, **oncpu**, **file**, **net**, **lock**, and **sched** are supported. - - **start_time**: start time of the event or the first event in an aggregated event. For details about aggregated events, see [Aggregated Events](#aggregated-events). - - **end_time**: event end time or the end time of the last event in an aggregated event. - - **duration**: event execution time. The value is (**end_time** - **start_time**). - - **count**: number of events that are aggregated. - - - Extended event attributes: - - - **func.stack**: information about the function call stack of an event. - - **file.path**: file path of a file event. - - **sock.conn**: TCP connection information about a network event. - - **futex.op**: operation type of a **futex** system call event. The value can be **wait** or **wake**. - - For details about the extended event attributes supported by different event types, see [Supported System Call Events](#supported-system-call-events). - -### Event Output - -As an extended eBPF probe program provided by gala-gopher, tprofiling sends generated system events to gala-gopher for processing. gala-gopher then outputs the events in the open source openTelemetry event format, and sends a message to the Kafka message queue in JSON format. The frontend can connect to Kafka to consume tprofiling events. - -The following is the example output of a thread profiling event: - -```json -{ - "Timestamp": 1661088145000, - "SeverityText": "INFO", - "SeverityNumber": 9, - "Body": "", - "Resource": { - "host.id": "", - "host.name": "", - "thread.pid": 10, - "thread.tgid": 10, - "thread.comm": "java", - "proc.name": "xxx.jar", - "container.id": "", - "container.name": "", - }, - "Attributes": { - values: [ - { - // common info - "event.name": "read", - "event.type": "file", - "start_time": 1661088145000, - "end_time": 1661088146000, - "duration": 0.1, - "count": 1, - // extend info - "func.stack": "read;", - "file.path": "/test.txt" - }, - { - "event.name": "oncpu", - "event.type": "oncpu", - "start_time": 1661088146000, - "end_time": 1661088147000, - "duration": 0.1, - "count": 1, - } - ] - } -} -``` - -Field description: - -- **Timestamp**: event point reported by the event. -- **Resource**: event source information. -- **Attributes**: event attribute information, which contains a **values** list. Each item in the list indicates a tprofiling event from the same source and contains the attribute information of the event. - -### Getting Started - -#### Installation and Deployment - -tprofiling is an extended eBPF probe program provided by gala-gopher. Therefore, you need to install and deploy gala-gopher before enabling tprofiling. - -In addition, to use the tprofiling capability on the frontend user interface (UI), [gala-ops](https://gitee.com/openeuler/gala-docs) sets up a UI demonstrating the tprofiling function based on the open source solution of Kafka + Logstash + Elasticsearch + Grafana. You can use the deployment tool provided by gala-ops for quick deployment. - -#### Operating Architecture - -![](./figures/tprofiling-run-arch.png) - -Frontend software description: - -- Kafka: open source message queue middleware used to receive and store tprofiling events collected by gala-gopher. -- Logstash: real-time open source log collection engine used to consume tprofiling events from Kafka, filter and convert the events, and send them to Elasticsearch. -- Elasticsearch: open distributed search and analysis engine used to store processed tprofiling events for Grafana query and visualization. -- Grafana: open source visualization tool used to query and visualize collected tprofiling events. You can use the tprofiling function on the UI provided by Grafana to analyze application performance issues. - -#### Deploying the tprofiling Probe - -You need to install gala-gopher first. For details about how to install and deploy gala-gopher, see the _Gala-gopher Documentation_. tprofiling events are sent to Kafka. Therefore, you need to configure the Kafka service address during deployment. - -After installing and running gala-gopher, use the HTTP-based dynamic configuration interface provided by gala-gopher to start the tprofiling probe. - -```sh -curl -X PUT http://:9999/tprofiling -d json='{"cmd": {"probe": ["oncpu", "syscall_file", "syscall_net", "syscall_sched", "syscall_lock"]}, "snoopers": {"proc_name": [{"comm": "java"}]}, "state": "running"}' -``` - -Parameter description: - -- **: IP address of the node where gala-gopher is deployed. -- **probe**: **probe** of **cmd** specifies the range of system events observed by the tprofiling probe. **oncpu**, **syscall_file**, **syscall_net**, **syscall_sched** and **syscall_lock** correspond to the **oncpu** event and system call events **file**, **net**, **sched**, and **lock**. You can choose tprofiling event types to observe. -- **proc_name**: **proc_name** of **snoopers** filters the names of the processes to be observed. You can also use the **proc_id** configuration item to filter the IDs of the processes to be observed. For details, see [Dynamic Configuration RESTful APIs](#dynamic-configuration-restful-apis). - -To disable the tprofiling probe, run the following command: - -```sh -curl -X PUT http://:9999/tprofiling -d json='{"state": "stopped"}' -``` - -#### Deploying the Frontend Software - -The software required for using the tprofiling function includes Kafka, Logstash, Elasticsearch, and Grafana. They are installed on the management node. You can use the deployment tool provided by gala-ops to quickly install and deploy them. For details, see the documents of gala. - -On the management node, run the following command to obtain the deployment script and install middleware Kafka, Logstash, and Elasticsearch: - -```sh -sh deploy.sh middleware -K -E -A -p -``` - -Run the following command to install Grafana: - -```sh -sh deploy.sh grafana -P -E -``` - -#### Usage - -After the deployment is complete, visit **http://**_\_**:3000** and log in to Grafana to use A-Ops. The default username and password are **admin**. - -After logging in to Grafana, find the dashboard named **ThreadProfiling**. - -![image-20230628155002410](./figures/tprofiling-dashboard.png) - -Click to access the frontend page of the tprofiling function. - -![image-20230628155249009](./figures/tprofiling-dashboard-detail.png) - -### Usage Examples - -#### Case 1: Locating a Deadlock Issue - -![image-20230628095802499](./figures/deadlock.png) - -The preceding figure shows the thread profiling result of a deadlock demo process. According to the statistics of the process event execution time in the pie chart, the proportion of lock events (in gray) is high during this period. The lower part displays the thread profiling result of the entire process. The vertical axis displays the execution sequence of profiling events of different threads in the process. The **java** thread is the main thread and is always blocked. After executing some **oncpu** events and **file** events, the service threads **LockThd1** and **LockThd2** intermittently execute **lock** events for a long time. The **lock** event details (as shown in the following figure) shows that the **futex** system call event is triggered and the execution time is 60 seconds. - -![image-20230628101056732](./figures/deadlock2.png) - -Based on the preceding observation, it can be found that the service threads **LockThd1** and **LockThd2** may be abnormal. The thread view shows the thread profiling results of the two service threads **LockThd1** and **LockThd2**. - -![image-20230628102138540](./figures/deadlock3.png) - -The preceding figure shows the profiling result of each thread. The vertical axis displays the execution sequence of different event types in the thread. As shown in the figure, threads **LockThd1** and **LockThd2** periodically execute **oncpu** events, including **file** events and **lock** events. However, at a certain moment (near 10:17:00), they simultaneously execute a **futex** event of the **lock** type for a long time, and no **oncpu** event occurs during this period, indicating that they enter the blocking state. **futex** is a system call related to user-mode lock implementation. If **futex** is triggered, lock contention occurs and threads may enter the blocking state. - -Based on the preceding analysis, deadlocks may occur on threads **LockThd1** and **LockThd2**. - -#### Case 2: Lock Contention Issue Locating - -![image-20230628111119499](./figures/lockcompete1.png) - -The preceding figure shows the running result of thread profiling of a lock contention demo process. As shown in the figure, the process executes three types of events: **lock**, **net**, and **oncpu**. The process contains three running service threads. From 11:05:45 to 11:06:45, the event execution time of the three service threads becomes long, which may indicate performance issues. Similarly, the thread view shows the thread profiling result of each thread. In addition, the time range can be narrowed down to a point near the time when an exception may occur. - -![image-20230628112709827](./figures/lockcompete2.png) - -By viewing the event execution sequence of each thread, we may understand the functions that each thread is performing during this period. - -- Thread **CompeteThd1**: A short **oncpu** event is triggered at a interval to execute a computing task. However, a long oncpu event is triggered around 11:05:45, indicating that a time-consuming computing task is being executed. - - ![image-20230628113336435](./figures/lockcompete3.png) - -- Thread **CompeteThd2**: A short **net** event is triggered at a interval. The event details show that the thread is sending network messages using the **write** system call and the corresponding TCP connection information is displayed. At 11:05:45, the **futex** event is executed for a long time and enters the blocking state. In this case, the execution interval of the write network event is prolonged. - - ![image-20230628113759887](./figures/lockcompete4.png) - - ![image-20230628114340386](./figures/lockcompete5.png) - -- Thread **tcp-server**: The TCP server continuously reads requests sent by the client using the **read** system call. The **read** event execution time is prolonged around 11:05:45, indicating that the server is waiting for network requests. - - ![image-20230628114659071](./figures/lockcompete6.png) - -Based on the preceding analysis, it can be found that each time thread **CompeteThd1** performs a time-consuming **oncpu** operation, thread **CompeteThd2** uses the **futex** system call and enters the blocking state. Once thread **CompeteThd1** completes the **oncpu** operation, thread **CompeteThd2** obtains the CPU and performs the network **write** operation. Therefore, there is a high probability that lock contention occurs between threads **CompeteThd1** and **CompeteThd2**. TCP network communication exists between the **tcp-server** thread and the **CompeteThd2** thread. The **CompeteThd2** thread cannot send network requests because it waits for lock resources. As a result, the **tcp-server** thread waits for **read** network requests most of the time. - -### topics - -#### Supported System Call Events - -The basic principles for selecting the system call events to be observed are as follows: - -1. Select events (such as file operations, network operations, and lock operations) that may be time-consuming and blocked. Such events usually involve access to system resources. -2. Select events that affect the status of threads. - -| Event/System Call| Description | Default Event Type| Extended Event Content | -| ----------------- | ----------------------------------------------------- | -------------- | -------------------------------- | -| read | Reading or writing drive files or network resources, which may be time-consuming and blocked. | file | file.path, sock.conn, func.stack | -| write | Reading or writing drive files or network resources, which may be time-consuming and blocked. | file | file.path, sock.conn, func.stack | -| readv | Reading or writing drive files or network resources, which may be time-consuming and blocked. | file | file.path, sock.conn, func.stack | -| writev | Reading or writing drive files or network resources, which may be time-consuming and blocked. | file | file.path, sock.conn, func.stack | -| preadv | Reading or writing drive files or network resources, which may be time-consuming and blocked. | file | file.path, sock.conn, func.stack | -| pwritev | Reading or writing drive files or network resources, which may be time-consuming and blocked. | file | file.path, sock.conn, func.stack | -| sync | Synchronizing files to drives. The thread is blocked until the synchronization is complete. | file | func.stack | -| fsync | Synchronizing files to drives. The thread is blocked until the synchronization is complete. | file | file.path, sock.conn, func.stack | -| fdatasync | Synchronizing files to drives. The thread is blocked until the synchronization is complete. | file | file.path, sock.conn, func.stack | -| sched_yield | Proactively releasing the CPU for rescheduling. | sched | func.stack | -| nanosleep | Entering the sleep state. | sched | func.stack | -| clock_nanosleep | Entering the sleep state. | sched | func.stack | -| wait4 | Blocked. | sched | func.stack | -| waitpid | Blocked. | sched | func.stack | -| select | Blocked and waiting when no event arrives. | sched | func.stack | -| pselect6 | When no event arrives, the thread is blocked and waits. | sched | func.stack | -| poll | When no event arrives, the thread is blocked and waits. | sched | func.stack | -| ppoll | When no event arrives, the thread is blocked and waits. | sched | func.stack | -| epoll_wait | When no event arrives, the thread is blocked and waits. | sched | func.stack | -| sendto | Reading or writing network resources, which may be time-consuming and blocked. | net | sock.conn, func.stack | -| recvfrom | Reading or writing network resources, which may be time-consuming and blocked. | net | sock.conn, func.stack | -| sendmsg | Reading or writing network resources, which may be time-consuming and blocked. | net | sock.conn, func.stack | -| recvmsg | Reading or writing network resources, which may be time-consuming and blocked. | net | sock.conn, func.stack | -| sendmmsg | Reading or writing network resources, which may be time-consuming and blocked. | net | sock.conn, func.stack | -| recvmmsg | Reading or writing network resources, which may be time-consuming and blocked. | net | sock.conn, func.stack | -| futex | Usually indicates lock wait and thread blocking state.| lock | futex.op, func.stack | - -#### Aggregated Events - -Currently, tprofiling supports two types of system performance events: system call events and **oncpu** events. The **oncpu** event and some system call events (such as **read**/**write**) may be frequently triggered in a specific application scenario. As a result, a large number of system events are generated, which greatly affects the performance of the observed application program and the performance of the tprofiling probe. - -To optimize performance, tprofiling aggregates multiple system events that belong to the same thread and have the same event name within a period of time (1s) into one event for reporting. Therefore, a tprofiling event actually refers to an aggregated event that contains one or more identical system events. Compared with a real system event, the meanings of some attributes of an aggregated event are changed as follows: - -- **start_time**: start time of the event or the first event in an aggregated event. -- **end_time**: event end time, which is (**start_time** + **duration**) in an aggregated event. -- **duration**: event execution time or the accumulated execution time of all system events in an aggregation event. -- **count**: number of system events in an aggregated event. When the value is **1**, an aggregated event is equivalent to a system event. -- Extended event attribute: extended attribute of the first system event in an aggregation event. - -## Introduction to L7Probe - -L7Probe observes layer 7 traffic, covering common HTTP1.X, PG, MySQL, Redis, Kafka, HTTP2.0, MongoDB, RocketMQ, and encrypted traffic. - -It covers node, container, and pod (K8s) scenarios. - -### Code Framework Design - -```text -L7Probe - | --- included //Common header files - - | --- connect.h // L7 connect object definition - - | --- pod.h // Pod/Container object definition - - | --- conn_tracker.h // L7 protocol tracker object definition - - | --- protocol // L7 protocol parsing - - | --- http // HTTP1.X L7 message structure definition and parsing - - | --- mysql // mysql L7 message structure definition and parsing - - | --- pgsql // pgsql L7 message structure definition and parsing - - | --- bpf // Kernel BPF code - - | --- L7.h // BPF program for parsing L7 protocol types - - | --- kern_sock.bpf.c // Kernel socket layer observation - - | --- libssl.bpf.c // openSSL layer observation - - | --- gossl.bpf.c // GO SSL layer observation - - | --- cgroup.bpf.c // Pod life cycle observation - - | --- pod_mng.c // Pod/Container instance management (pod/container lifecycle awareness) - - | --- conn_mng.c // L7 Connect instance management (processing BPF observation events, such as Open/Close events and Stats statistics) - - | --- conn_tracker.c // L7 traffic tracker (tracking BPF observation data, such as data generated by system events such as send/write and read/recv) - - | --- bpf_mng.c // BPF program life cycle management (on-demand and real-time open, load, attach, and unload BPF programs, including uprobe BPF programs) - - | --- session_conn.c // Manages jsse sessions (records the mapping between jsse sessions and sock connections and reports jsse connection information). - - | --- L7Probe.c // Main program of the probe -``` - -### Probe Output - -| metrics_name | table_name | metrics_type | unit | metrics description | -| --------------- | ---------- | ------------ | ---- | ------------------------------------------------------------ | -| tgid | NA | key | NA | Process ID of l7 session. | -| client_ip | NA | key | NA | Client IP address of l7 session. | -| server_ip | NA | key | NA | Server IP address of l7 session.
Remarks: In the K8s scenario, the cluster IP address can be converted into the backend IP address.| -| server_port | NA | key | NA | Server Port of l7 session.
Remarks: In the K8s scenario, the cluster port can be converted to the backend port.| -| l4_role | NA | key | NA | Role of l4 protocol(TCP Client/Server or UDP) | -| l7_role | NA | key | NA | Role of l7 protocol(Client or Server) | -| protocol | NA | key | NA | Name of l7 protocol(http/http2/mysql...) | -| ssl | NA | label | NA | Indicates whether an SSL-encrypted l7 session is used. | -| bytes_sent | l7_link | gauge | NA | Number of bytes sent by a l7 session. | -| bytes_recv | l7_link | gauge | NA | Number of bytes recv by a l7 session. | -| segs_sent | l7_link | gauge | NA | Number of segs sent by a l7 session. | -| segs_recv | l7_link | gauge | NA | Number of segs recv by a l7 session. | -| throughput_req | l7_rpc | gauge | qps | Request throughput of l7 session. | -| throughput_resp | l7_rpc | gauge | qps | Response throughput of l7 session. | -| req_count | l7_rpc | gauge | NA | Request num of l7 session. | -| resp_count | l7_rpc | gauge | NA | Response num of l7 session. | -| latency_avg | l7_rpc | gauge | ns | L7 session averaged latency. | -| latency | l7_rpc | histogram | ns | L7 session histogram latency. | -| latency_sum | l7_rpc | gauge | ns | L7 session sum latency. | -| err_ratio | l7_rpc | gauge | % | L7 session error rate. | -| err_count | l7_rpc | gauge | NA | L7 session error count. | - -### Dynamic Control - -#### Controlling the Scope of Observed Pods - -1. REST->gala-gopher. -2. gala-gopher->L7Probe. -3. L7Probe obtains related containers based on pods. -4. L7Probe obtains the cgroup ID (**cpuacct_cgrp_id**) based on the container and writes the cgroup ID to the object module (the **cgrp_add** API). -5. In the socket system event context, obtain the cgroup (**cpuacct_cgrp_id**) to which the process belongs. For details, see the Linux code (**task_cgroup**). -6. The object module (the **is_cgrp_exist** API) can be filtered during observation. - -#### Observation Capability Control - -1. REST->gala-gopher. -2. gala-gopher->L7Probe. -3. L7Probe dynamically enables or disables BPF observation capabilities (including the throughput, latency, trace, and protocol type) based on the input parameters. - -### Observation Points - -#### Kernel Socket System Calls - -TCP-related system calls - -```c -int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen); - -int accept(int sockfd, struct sockaddr *addr, socklen_t*addrlen); - -int accept4(int sockfd, struct sockaddr *addr, socklen_t*addrlen, int flags); - -ssize_t write(int fd, const void *buf, size_t count); - -ssize_t send(int sockfd, const void *buf, size_t len, int flags); - -ssize_t read(int fd, void *buf, size_t count); - -ssize_t recv(int sockfd, void *buf, size_t len, int flags); - -ssize_t writev(int fd, const struct iovec *iov, int iovcnt); - -ssize_t readv(int fd, const struct iovec *iov, int iovcnt); -``` - -TCP&UDP-related system calls - -```c -ssize_t sendto(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr*dest_addr, socklen_t addrlen); - -ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags, struct sockaddr*src_addr, socklen_t *addrlen); - -ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags); - -ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags); - -int close(int fd); -``` - -Notes: - -1. The **read**/**write** and **readv**/**writev** operations may be confused with common file I/O operations. You can observe the **security_socket_sendmsg** function of the kernel to determine whether the FD is a socket operation. -2. **sendto**/**recvfrom** and **sendmsg**/**recvmsg** are used by TCP and UDP. For details, see the following manuals. -3. **sendmmsg**/**recvmmsg** and **sendfile** are not supported currently. - -[**sendto** manual](https://man7.org/linux/man-pages/man2/send.2.html): If **sendto()** is used on a connection-mode (**SOCK_STREAM**, **SOCK_SEQPACKET**) socket, the arguments _dest\_addr_ and _addrlen_ are ignored (and the error **EISCONN** may be returned when they are not NULL and 0), and the error **ENOTCONN** is returned when the socket was not actually connected.otherwise, the address of the target is given by dest_addr with addrlen specifying its size. - -If the value of **dest_addr** is NULL, TCP is used. Otherwise, UDP is used. - -[**recvfrom** manual](https://linux.die.net/man/2/recvfrom): The **recvfrom()** and **recvmsg()** calls are used to receive messages from a socket, and may be used to receive data on a socket whether or not it is connection-oriented. - -If the value of **src_addr** is NULL, TCP is used. Otherwise, UDP is used. - -[**sendmsg** manual](https://man7.org/linux/man-pages/man3/sendmsg.3p.html): The **sendmsg()** function shall send a message through a connection-mode or connectionless-mode socket.If the socket is a connectionless-mode socket, the message shall be sent to the address specified by msghdr if no pre-specified peer address has been set. If a peer address has been pre-specified, either themessage shall be sent to the address specified in msghdr (overriding the pre-specified peer address), or the function shall return -1 and set errno to \[EISCONN]. If the socket is connection-mode, the destination address in msghdr shall be ignored. - -If the value of **msghdr->msg_name** is NULL, TCP is used. Otherwise, UDP is used. - -[recvmsg manual](https://man7.org/linux/man-pages/man3/recvmsg.3p.html): The recvmsg() function shall receive a message from a connection-mode or connectionless-mode socket. It is normally used with connectionless-mode sockets because it permits the application to retrieve the source address of received data. - -If the value of **msghdr->msg_name** is NULL, TCP is used. Otherwise, UDP is used. - -#### libSSL API - -SSL_write - -SSL_read - -#### Go SSL API - -#### JSSE API - -sun/security/ssl/SSLSocketImpl$AppInputStream - -sun/security/ssl/SSLSocketImpl$AppOutputStream - -### JSSE Observation Scheme - -#### Loading JSSEProbe - -In the **main** function, **l7_load_jsse_agent** is used to load JSSEProbe. - -Processes in allowlist (**g_proc_obj_map_fd**) are observed in polling mode. If the processes are Java processes, **jvm_attach** is used to load **JSSEProbeAgent.jar** to the processes. After the loading is successful, the Java process exports the observation information to the jsse-metrics output file (**/tmp/java-data-**_\_**/jsse-metrics.txt**) at the specified observation point. For details, see [JSSE API](#jsse-api). - -#### Processing JSSEProbe Messages - -The **l7_jsse_msg_handler** thread processes JSSEProbe messages. - -Processes in allowlist (**g_proc_obj_map_fd**) are observed in polling mode. If the processes have the corresponding jsse-metrics output file, the file is read by line, and JSSE read and write information is parsed, converted, and reported. - -##### 1. Parsing JSSE Read and Write Information - -The output format of **jsse-metrics.txt** is as follows, from which the PID, session ID, time, read/write operations, IP address, port number, and payload information of a JSSE request is obtained: - -```text -|jsse_msg|662220|Session(1688648699909|TLS_AES_256_GCM_SHA384)|1688648699989|Write|127.0.0.1|58302|This is test message| -``` - -The parsed original information is stored in **session_data_args_s**. - -##### 2. Converting JSSE Read and Write Information - -Information in **session_data_args_s** is converted to **sock_conn** and **conn_data**. - -During conversion, the following hash maps are queried: - -**session_head**: records the mapping between session IDs and sock connection IDs of JSSE connections. If the process IDs and the quadruplet information are the same, the session corresponds to the sock connection. - -**file_conn_head**: records the last session ID of the Java process in case L7probe does not read the JSSEProbe output from the beginning of the request and cannot find session ID. - -##### 3. Reporting JSSE Read and Write Information - -**sock_conn** and **conn_data** are reported to the map. - -## Usage - -### Deployment of External Dependencies - -![gala-gopher architecture](./figures/gala-gopher_architecture.png) - -As shown in the preceding figure, the green parts are external dependent components of gala-gopher. gala-gopher outputs metric data to Prometheus, metadata and abnormal events to Kafka. gala-anteater and gala-spider in gray rectangles obtain data from Prometheus and Kafka. - -> Note: Obtain the installation packages of Kafka and Prometheus from the official websites. - -### Output Data - -- **Metric** - - Prometheus Server has a built-in Express Browser UI. You can use PromQL statements to query metric data. For details, see [Using the expression browser](https://prometheus.io/docs/prometheus/latest/getting_started/#using-the-expression-browser) in the official document. The following is an example. - - If the specified metric is **gala_gopher_tcp_link_rcv_rtt**, the metric data displayed on the UI is as follows: - - ```basic - gala_gopher_tcp_link_rcv_rtt{client_ip="x.x.x.165",client_port="1234",hostname="openEuler",instance="x.x.x.172:8888",job="prometheus",machine_id="1fd3774xx",protocol="2",role="0",server_ip="x.x.x.172",server_port="3742",tgid="1516"} 1 - ``` - -- **Metadata** - - You can directly consume data from the Kafka topic **gala_gopher_metadata**. The following is an example. - - ```bash - # Input request - ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_metadata - # Output data - {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]} - ``` - -- **Abnormal events** - - You can directly consume data from the Kafka topic **gala_gopher_event**. The following is an example. - - ```bash - # Input request - ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_event - # Output data - {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]} - ``` diff --git a/docs/en/docs/A-Ops/using-gala-ragdoll.md b/docs/en/docs/A-Ops/using-gala-ragdoll.md deleted file mode 100644 index 3b0c460b9d765c0e9d2775dfc6e283e7c63fa9e7..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/using-gala-ragdoll.md +++ /dev/null @@ -1,167 +0,0 @@ -gala-ragdoll Usage Guide -============================ - -## Installation - -#### Manual Installation - -- Installing using the repo source mounted by Yum. - - Configure the Yum sources **openEuler22.09** and **openEuler22.09:Epol** in the **/etc/yum.repos.d/openEuler.repo** file. - - ```ini - [everything] # openEuler 22.09 officially released repository - name=openEuler22.09 - baseurl=https://repo.openeuler.org/openEuler-22.09/everything/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=https://repo.openeuler.org/openEuler-22.09/everything/$basearch/RPM-GPG-KEY-openEuler - - [Epol] # openEuler 22.09:Epol officially released repository - name=Epol - baseurl=https://repo.openeuler.org/openEuler-22.09/EPOL/main/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=https://repo.openeuler.org/openEuler-22.09/OS/$basearch/RPM-GPG-KEY-openEuler - - ``` - - Run the following commands to download and install gala-ragdoll and its dependencies. - - ```shell - yum install gala-ragdoll # A-Ops configuration source tracing service - yum install python3-gala-ragdoll - - yum install gala-spider # A-Ops architecture awareness service - yum install python3-gala-spider - ``` - -- Installing using the RPM packages. Download **gala-ragdoll-vx.x.x-x.oe1.aarch64.rpm**, and then run the following commands to install the modules. (`x.x-x` indicates the version. Replace it with the actual version number.) - - ```shell - rpm -ivh gala-ragdoll-vx.x.x-x.oe1.aarch64.rpm - ``` - - - -#### Installing Using the A-Ops Deployment Service - -##### Editing the Task List - -Modify the deployment task list and enable the steps for gala_ragdoll: - -```yaml ---- -step_list: - ... - gala_ragdoll: - enable: false - continue: false - ... -``` - -##### Editing the Host List - -For details about the host configuration, see section 2.2.3.10 in the [Deployment Management Manual](./deployment-management-manual.md). - -##### Editing the Variable List - -For details about the variable configuration, see section 2.2.3.10 in the [Deployment Management Manual](./deployment-management-manual.md). - -##### Executing the Deployment Task - -See section 3 in the [Deployment Management Manual](./deployment-management-manual.md) to execute the deployment task. - - - -### Configuration File Description - -```/etc/yum.repos.d/openEuler.repo``` is the configuration file used to specify the Yum source address. The content of the configuration file is as follows: - -``` -[OS] -name=OS -baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ -enabled=1 -gpgcheck=1 -gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler -``` - -### YANG Model Description - -`/etc/yum.repos.d/openEuler.repo` is expressed using the YANG language. For details, see `gala-ragdoll/yang_modules/openEuler-logos-openEuler.repo.yang`. -The following extended fields are added: - -| Extended Field Name | Extended Field Format| Example| -| ------------ | ---------------------- | ----------------------------------------- | -| path | OS_TYPE:CONFIGURATION_FILE_PATH | openEuler:/etc/yum.repos.d/openEuler.repo | -| type | Configuration file type | ini, key-value, json, text, and more | -| spacer | Spacer between a configuration item and its value | " ", "=", ":", and more | - -Attachment: Learning the YANG language: https://datatracker.ietf.org/doc/html/rfc7950/. - -### Creating Domains using Configuration Source Tracing - -#### Viewing the configuration file. - -gala-ragdoll contains the configuration file of the configuration source tracing. - -``` -[root@openeuler-development-1-1drnd ~]# cat /etc/ragdoll/gala-ragdoll.conf -[git] // Defines the current Git information, including the directory and user information of the Git repository. -git_dir = "/home/confTraceTestConf" -user_name = "user" -user_email = "email" - -[collect] // The collect interface provided by A-Ops. -collect_address = "http://192.168.0.0:11111" -collect_api = "/manage/config/collect" - -[ragdoll] -port = 11114 - -``` - -#### Creating the Configuration Domain - - -![](./figures/create_service_domain.png) - - - -#### Adding Managed Nodes to the Configuration Domain - -![](./figures/add_node.png) - - - -#### Adding Configurations to the Configuration Domain - - -![](./figures/add_config.png) - -#### Querying the Expected Configuration - - -![](./figures/view_expected_config.png) - -#### Deleting Configurations - -![](./figures/delete_config.png) - -#### Querying the Actual Configuration - -![](./figures/query_actual_config.png) - - - -#### Verifying the Configuration - - -![](./figures/query_status.png) - - - -#### Configuration Synchronization - -![](./figures/sync_conf.png) diff --git a/docs/en/docs/A-Ops/using-gala-spider.md b/docs/en/docs/A-Ops/using-gala-spider.md deleted file mode 100644 index 2949e6f18d9de8263579daeaae23a0cceacb67f6..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Ops/using-gala-spider.md +++ /dev/null @@ -1,527 +0,0 @@ -# Using gala-spider - -This chapter describes how to deploy and use gala-spider and gala-inference. - -## gala-spider - -gala-spider provides the OS-level topology drawing function. It periodically obtains the data of all observed objects collected by gala-gopher (an OS-level data collection software) at a certain time point and calculates the topology relationship between them. The generated topology is saved to the graph database ArangoDB. - -### Installation - -Mount the Yum sources. - -```basic -[oe-2209] # openEuler 22.09 officially released repository -name=oe2209 -baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 -enabled=1 -gpgcheck=0 -priority=1 - -[oe-2209:Epol] # openEuler 22.09: Epol officially released repository -name=oe2209_epol -baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ -enabled=1 -gpgcheck=0 -priority=1 -``` - -Install gala-spider. - -```sh -# yum install gala-spider -``` - -### Configuration - -#### Configuration File Description - -The configuration file of gala-spider is **/etc/gala-spider/gala-spider.yaml**. The configuration items in this file are described as follows: - -- `global`: global configuration information. - - `data_source`: database for collecting observation metrics. Currently, only `prometheus` is supported. - - `data_agent`: agent for collecting observation metrics. Currently, only `gala_gopher` is supported. -- `spider`: spider configuration information. - - `log_conf`: log configuration information. - - `log_path`: log file path. - - `log_level`: level of the logs to be printed. The value can be `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`. - - `max_size`: log file size, in MB. - - `backup_count`: number of backup log files. -- `storage`: configuration information about the topology storage service. - - `period`: storage period, in seconds, indicating the interval for storing the topology. - - `database`: graph database for storage. Currently, only `arangodb` is supported. - - `db_conf`: configuration information of the graph database. - - `url`: IP address of the graph database server. - - `db_name`: name of the database where the topology is stored. -- `kafka`: Kafka configuration information. - - `server`: Kafka server address. - - `metadata_topic`: topic name of the observed metadata messages. - - `metadata_group_id`: consumer group ID of the observed metadata messages. -- `prometheus`: Prometheus database configuration information. - - `base_url`: IP address of the Prometheus server. - - `instant_api`: API for collecting data at a single time point. - - `range_api`: API for collecting data in a time range. - - `step`: collection time step, which is configured for `range_api`. - -#### Configuration File Example - -```yaml -global: - data_source: "prometheus" - data_agent: "gala_gopher" - -prometheus: - base_url: "http://localhost:9090/" - instant_api: "/api/v1/query" - range_api: "/api/v1/query_range" - step: 1 - -spider: - log_conf: - log_path: "/var/log/gala-spider/spider.log" - # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL - log_level: INFO - # unit: MB - max_size: 10 - backup_count: 10 - -storage: - # unit: second - period: 60 - database: arangodb - db_conf: - url: "http://localhost:8529" - db_name: "spider" - -kafka: - server: "localhost:9092" - metadata_topic: "gala_gopher_metadata" - metadata_group_id: "metadata-spider" -``` - -### Start - -- Run the following command to start gala-spider. - - ```sh - # spider-storage - ``` - -- Use the systemd service to start gala-spider. - - ```sh - # systemctl start gala-spider - ``` - -### How to Use - -#### Deployment of External Dependent Software - -The running of gala-spider depends on multiple external software for interaction. Therefore, before starting gala-spider, you need to deploy the software on which gala-spider depends. The following figure shows the software dependency of gala-spider. - -![gala-spider-arch](./figures/gala-spider-arch.png) - -The dotted box on the right indicates the two functional components of gala-spider. The green parts indicate the external components that gala-spider directly depends on, and the gray rectangles indicate the external components that gala-spider indirectly depends on. - -- **spider-storage**: core component of gala-spider, which provides the topology storage function. - 1. Obtains the metadata of the observation object from Kafka. - 2. Obtains information about all observation object instances from Prometheus. - 3. Saves the generated topology to the graph database ArangoDB. -- **gala-inference**: core component of gala-spider, which provides the root cause locating function. It subscribes to abnormal KPI events from Kafka to trigger the root cause locating process of abnormal KPIs, constructs a fault propagation graph based on the topology obtained from the ArangoDB, and outputs the root cause locating result to Kafka. -- **prometheus**: time series database. The observation metric data collected by the gala-gopher component is reported to Prometheus for further processing. -- **kafka**: messaging middleware, which is used to store the observation object metadata reported by gala-gopher, exception events reported by the exception detection component gala-anteater, and root cause locating results reported by the cause-inference component. -- **arangodb**: graph database, which is used to store the topology generated by spider-storage. -- **gala-gopher**: data collection component. It must be deployed in advance. -- **arangodb-ui**: UI provided by ArangoDB, which can be used to query topologies. - -The two functional components in gala-spider are released as independent software packages. - -​ **spider-storage**: corresponds to the gala-spider software package in this section. - -​ **gala-inference**: corresponds to the gala-inference software package. - -For details about how to deploy the gala-gopher software, see [Using gala-gopher](using-gala-gopher.md). This section only describes how to deploy ArangoDB. - -The current ArangoDB version is 3.8.7, which has the following requirements on the operating environment: - -- Only the x86 system is supported. -- GCC 10 or later - -For details about ArangoDB deployment, see [Deployment](https://www.arangodb.com/docs/3.9/deployment.html) in the ArangoDB official document. - -The RPM-based ArangoDB deployment process is as follows: - -1. Configure the Yum sources. - - ```basic - [oe-2209] # openEuler 22.09 officially released repository - name=oe2209 - baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 - enabled=1 - gpgcheck=0 - priority=1 - - [oe-2209:Epol] # openEuler 22.09: Epol officially released repository - name=oe2209_epol - baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ - enabled=1 - gpgcheck=0 - priority=1 - ``` - -2. Install arangodb3. - - ```sh - # yum install arangodb3 - ``` - -3. Modify the configurations. - - The configuration file of the arangodb3 server is **/etc/arangodb3/arangod.conf**. You need to modify the following configurations: - - - `endpoint`: IP address of the arangodb3 server. - - `authentication`: whether identity authentication is required for accessing the arangodb3 server. Currently, gala-spider does not support identity authentication. Therefore, set `authentication` to `false`. - - The following is an example. - - ```yaml - [server] - endpoint = tcp://0.0.0.0:8529 - authentication = false - ``` - -4. Start arangodb3. - - ```sh - # systemctl start arangodb3 - ``` - -#### Modifying gala-spider Configuration Items - -After the dependent software is started, you need to modify some configuration items in the gala-spider configuration file. The following is an example. - -Configure the Kafka server address. - -```yaml -kafka: - server: "localhost:9092" -``` - -Configure the Prometheus server address. - -```yaml -prometheus: - base_url: "http://localhost:9090/" -``` - -Configure the IP address of the ArangoDB server. - -```yaml -storage: - db_conf: - url: "http://localhost:8529" -``` - -#### Starting the Service - -Run `systemctl start gala-spider` to start the service. Run `systemctl status gala-spider` to check the startup status. If the following information is displayed, the startup is successful: - -```sh -[root@openEuler ~]# systemctl status gala-spider -● gala-spider.service - a-ops gala spider service - Loaded: loaded (/usr/lib/systemd/system/gala-spider.service; enabled; vendor preset: disabled) - Active: active (running) since Tue 2022-08-30 17:28:38 CST; 1 day 22h ago - Main PID: 2263793 (spider-storage) - Tasks: 3 (limit: 98900) - Memory: 44.2M - CGroup: /system.slice/gala-spider.service - └─2263793 /usr/bin/python3 /usr/bin/spider-storage -``` - -#### Output Example - -You can query the topology generated by gala-spider on the UI provided by ArangoDB. The procedure is as follows: - -1. Enter the IP address of the ArangoDB server in the address box of the browser, for example, ****. The ArangoDB UI is displayed. - -2. Click **DB** in the upper right corner of the page to switch to the spider database. - -3. On the **COLLECTIONS** page, you can view the collections of observation object instances and topology relationships stored in different time segments, as shown in the following figure. - - ![spider_topology](./figures/spider_topology.png) - -4. You can query the stored topology using the AQL statements provided by ArangoDB. For details, see the [AQL Documentation](https://www.arangodb.com/docs/3.8/aql/). - -## gala-inference - -gala-inference provides the capability of locating root causes of abnormal KPIs. It uses the exception detection result and topology as the input and outputs the root cause locating result to Kafka. The gala-inference component is archived in the gala-spider project. - -### Installation - -Mount the Yum sources. - -```basic -[oe-2209] # openEuler 22.09 officially released repository -name=oe2209 -baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 -enabled=1 -gpgcheck=0 -priority=1 - -[oe-2209:Epol] # openEuler 22.09: Epol officially released repository -name=oe2209_epol -baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ -enabled=1 -gpgcheck=0 -priority=1 -``` - -Install gala-inference. - -```sh -# yum install gala-inference -``` - -### Configuration - -#### Configuration File Description - -The configuration items in the gala-inference configuration file **/etc/gala-inference/gala-inference.yaml** are described as follows: - -- `inference`: configuration information about the root cause locating algorithm. - - `tolerated_bias`: tolerable time offset for querying the topology at the exception time point, in seconds. - - `topo_depth`: maximum depth for topology query. - - `root_topk`: yop *K* root cause metrics generated in the root cause locating result. - - `infer_policy`: root cause derivation policy, which can be `dfs` or `rw`. - - `sample_duration`: sampling period of historical metric data, in seconds. - - `evt_valid_duration`: valid period of abnormal system metric events during root cause locating, in seconds. - - `evt_aging_duration`: aging period of abnormal metric events during root cause locating, in seconds. -- `kafka`: Kafka configuration information. - - `server`: IP address of the Kafka server. - - `metadata_topic`: configuration information about the observed metadata messages. - - `topic_id`: topic name of the observed metadata messages. - - `group_id`: consumer group ID of the observed metadata messages. - - `abnormal_kpi_topic`: configuration information about abnormal KPI event messages. - - `topic_id`: topic name of the abnormal KPI event messages. - - `group_id`: consumer group ID of the abnormal KPI event messages. - - `abnormal_metric_topic`: configuration information about abnormal metric event messages. - - `topic_id`: topic name of the abnormal metric event messages. - - `group_id`: consumer group ID of the abnormal system metric event messages. - - `consumer_to`: timeout interval for consuming abnormal system metric event messages, in seconds. - - `inference_topic`: configuration information about the output event messages of the root cause locating result. - - `topic_id`: topic name of the output event messages of the root cause locating result. -- `arangodb`: configuration information about the ArangoDB graph database, which is used to query sub-topologies required for root cause locating. - - `url`: IP address of the graph database server. - - `db_name`: name of the database where the topology is stored. -- `log_conf`: log configuration information. - - `log_path`: log file path. - - `log_level`: level of the logs to be printed. The value can be `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`. - - `max_size`: log file size, in MB. - - `backup_count`: number of backup log files. -- `prometheus`: Prometheus database configuration information, which is used to obtain historical time series data of metrics. - - `base_url`: IP address of the Prometheus server. - - `range_api`: API for collecting data in a time range. - - `step`: collection time step, which is configured for `range_api`. - -#### Configuration File Example - -```yaml -inference: - # Tolerable time offset for querying the topology at the exception time point, in seconds. - tolerated_bias: 120 - topo_depth: 10 - root_topk: 3 - infer_policy: "dfs" - # Unit: second - sample_duration: 600 - # Valid period of abnormal metric events during root cause locating, in seconds. - evt_valid_duration: 120 - # Aging period of abnormal metric events, in seconds. - evt_aging_duration: 600 - -kafka: - server: "localhost:9092" - metadata_topic: - topic_id: "gala_gopher_metadata" - group_id: "metadata-inference" - abnormal_kpi_topic: - topic_id: "gala_anteater_hybrid_model" - group_id: "abn-kpi-inference" - abnormal_metric_topic: - topic_id: "gala_anteater_metric" - group_id: "abn-metric-inference" - consumer_to: 1 - inference_topic: - topic_id: "gala_cause_inference" - -arangodb: - url: "http://localhost:8529" - db_name: "spider" - -log: - log_path: "/var/log/gala-inference/inference.log" - # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL - log_level: INFO - # unit: MB - max_size: 10 - backup_count: 10 - -prometheus: - base_url: "http://localhost:9090/" - range_api: "/api/v1/query_range" - step: 5 -``` - -### Start - -- Run the following command to start gala-inference. - - ```sh - # gala-inference - ``` - -- Use the systemd service to start gala-inference. - - ```sh - # systemctl start gala-inference - ``` - -### How to Use - -#### Dependent Software Deployment - -The running dependency of gala-inference is the same as that of gala-spider. For details, see [Deployment of External Dependent Software](#deployment-of-external-dependent-software). In addition, gala-inference indirectly depends on the running of [gala-spider](#gala-spider) and [gala-anteater](using-gala-anteater.md). Deploy gala-spider and gala-anteater in advance. - -#### Modify configuration items - -Modify some configuration items in the gala-inference configuration file. The following is an example. - -Configure the Kafka server address. - -```yaml -kafka: - server: "localhost:9092" -``` - -Configure the Prometheus server address. - -```yaml -prometheus: - base_url: "http://localhost:9090/" -``` - -Configure the IP address of the ArangoDB server. - -```yaml -arangodb: - url: "http://localhost:8529" -``` - -#### Starting the Service - -Run `systemctl start gala-inference` to start the service. Run `systemctl status gala-inference` to check the startup status. If the following information is displayed, the startup is successful: - -```sh -[root@openEuler ~]# systemctl status gala-inference -● gala-inference.service - a-ops gala inference service - Loaded: loaded (/usr/lib/systemd/system/gala-inference.service; enabled; vendor preset: disabled) - Active: active (running) since Tue 2022-08-30 17:55:33 CST; 1 day 22h ago - Main PID: 2445875 (gala-inference) - Tasks: 10 (limit: 98900) - Memory: 48.7M - CGroup: /system.slice/gala-inference.service - └─2445875 /usr/bin/python3 /usr/bin/gala-inference -``` - -#### Output Example - -When the exception detection module gala-anteater detects a KPI exception, it exports the corresponding abnormal KPI event to Kafka. The gala-inference keeps monitoring the message of the abnormal KPI event. If gala-inference receives the message of the abnormal KPI event, root cause locating is triggered. The root cause locating result is exported to Kafka. You can view the root cause locating result on the Kafka server. The basic procedure is as follows: - -1. If Kafka is installed using the source code, go to the Kafka installation directory. - - ```sh - cd /root/kafka_2.13-2.8.0 - ``` - -2. Run the command for consuming the topic to obtain the output of root cause locating. - - ```sh - ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic gala_cause_inference - ``` - - Output example: - - ```json - { - "Timestamp": 1661853360000, - "event_id": "1661853360000_1fd37742xxxx_sli_12154_19", - "Atrributes": { - "event_id": "1661853360000_1fd37742xxxx_sli_12154_19" - }, - "Resource": { - "abnormal_kpi": { - "metric_id": "gala_gopher_sli_rtt_nsec", - "entity_id": "1fd37742xxxx_sli_12154_19", - "timestamp": 1661853360000, - "metric_labels": { - "machine_id": "1fd37742xxxx", - "tgid": "12154", - "conn_fd": "19" - } - }, - "cause_metrics": [ - { - "metric_id": "gala_gopher_proc_write_bytes", - "entity_id": "1fd37742xxxx_proc_12154", - "metric_labels": { - "__name__": "gala_gopher_proc_write_bytes", - "cmdline": "/opt/redis/redis-server x.x.x.172:3742", - "comm": "redis-server", - "container_id": "5a10635e2c43", - "hostname": "openEuler", - "instance": "x.x.x.172:8888", - "job": "prometheus", - "machine_id": "1fd37742xxxx", - "pgid": "12154", - "ppid": "12126", - "tgid": "12154" - }, - "timestamp": 1661853360000, - "path": [ - { - "metric_id": "gala_gopher_proc_write_bytes", - "entity_id": "1fd37742xxxx_proc_12154", - "metric_labels": { - "__name__": "gala_gopher_proc_write_bytes", - "cmdline": "/opt/redis/redis-server x.x.x.172:3742", - "comm": "redis-server", - "container_id": "5a10635e2c43", - "hostname": "openEuler", - "instance": "x.x.x.172:8888", - "job": "prometheus", - "machine_id": "1fd37742xxxx", - "pgid": "12154", - "ppid": "12126", - "tgid": "12154" - }, - "timestamp": 1661853360000 - }, - { - "metric_id": "gala_gopher_sli_rtt_nsec", - "entity_id": "1fd37742xxxx_sli_12154_19", - "metric_labels": { - "machine_id": "1fd37742xxxx", - "tgid": "12154", - "conn_fd": "19" - }, - "timestamp": 1661853360000 - } - ] - } - ] - }, - "SeverityText": "WARN", - "SeverityNumber": 13, - "Body": "A cause inferring event for an abnormal event" - } - ``` diff --git a/docs/en/docs/A-Tune/A-Tune.md b/docs/en/docs/A-Tune/A-Tune.md deleted file mode 100644 index cb94a36db10e5d10f1ed758055c3a7ad99011d38..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/A-Tune.md +++ /dev/null @@ -1,5 +0,0 @@ -# A-Tune User Guide - -This document describes how to install and use A-Tune, which is a performance self-optimization software for openEuler. - -This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use A-Tune. You need to have basic knowledge of the Linux OS. \ No newline at end of file diff --git a/docs/en/docs/A-Tune/appendixes.md b/docs/en/docs/A-Tune/appendixes.md deleted file mode 100644 index 2d776555c04a00f5a7c56e5d8b503925019af32a..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/appendixes.md +++ /dev/null @@ -1,25 +0,0 @@ -# Appendixes - -- [Appendixes](#appendixes) - - [Acronyms and Abbreviations](#acronyms-and-abbreviations) - - -## Acronyms and Abbreviations - -**Table 1** Terminology - - - - - - - - - -

Term

-

Description

-

profile

-

Set of optimization items and optimal parameter configuration.

-
- - diff --git a/docs/en/docs/A-Tune/faqs.md b/docs/en/docs/A-Tune/faqs.md deleted file mode 100644 index 632b9d4d5a40dc02ab8b4069216851fea241e65a..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/faqs.md +++ /dev/null @@ -1,52 +0,0 @@ -# FAQs - -## Q1: An error occurs when the **train** command is used to train a model, and the message "training data failed" is displayed - -Cause: Only one type of data is collected by using the **collection**command. - -Solution: Collect data of at least two data types for training. - -## Q2: atune-adm cannot connect to the atuned service - -Possible cause: - -1. Check whether the atuned service is started and check the atuned listening address. - - ```shell - systemctl status atuned - netstat -nap | grep atuned - ``` - -2. The firewall blocks the atuned listening port. -3. The HTTP proxy is configured in the system. As a result, the connection fails. - -Solution: - -1. If the atuned service is not started, run the following command to start the service: - - ```shell - systemctl start atuned - ``` - -2. Run the following command on the atuned and atune-adm servers to allow the listening port to receive network packets. In the command, **60001** is the listening port number of the atuned server. - - ```shell - iptables -I INPUT -p tcp --dport 60001 -j ACCEPT - iptables -I INPUT -p tcp --sport 60001 -j ACCEPT - ``` - -3. Run the following command to delete the HTTP proxy or disable the HTTP proxy for the listening IP address without affecting services: - - ```shell - no_proxy=$no_proxy, Listening_IP_address - ``` - -## Q3: The atuned service cannot be started, and the message "Job for atuned.service failed because a timeout was exceeded." is displayed - -Cause: The hosts file does not contain the localhost information. - -Solution: Add localhost to the line starting with **127.0.0.1** in the **/etc/hosts** file. - -```text -127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 -``` diff --git a/docs/en/docs/A-Tune/figures/en-us_image_0213178479.png b/docs/en/docs/A-Tune/figures/en-us_image_0213178479.png deleted file mode 100644 index 62ef0decdf6f1e591059904001d712a54f727e68..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/en-us_image_0213178479.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/figures/en-us_image_0213178480.png b/docs/en/docs/A-Tune/figures/en-us_image_0213178480.png deleted file mode 100644 index ad5ed3f7beeb01e6a48707c4806606b41d687e22..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/en-us_image_0213178480.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/figures/en-us_image_0227497000.png b/docs/en/docs/A-Tune/figures/en-us_image_0227497000.png deleted file mode 100644 index 3df66e5f25177cba7fe65cfb859fab860bfb7b46..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/en-us_image_0227497000.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/figures/en-us_image_0227497343.png b/docs/en/docs/A-Tune/figures/en-us_image_0227497343.png deleted file mode 100644 index a8654b170295b4b0be3c37187e4b227ca635fbc0..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/en-us_image_0227497343.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/figures/en-us_image_0231122163.png b/docs/en/docs/A-Tune/figures/en-us_image_0231122163.png deleted file mode 100644 index c61c39c5f5119d84c6799b1e17285a7fe313639f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/en-us_image_0231122163.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/figures/picture1.png b/docs/en/docs/A-Tune/figures/picture1.png deleted file mode 100644 index 624d148b98bc9890befbecc53f29d6a4890d06af..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/picture1.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/figures/picture4.png b/docs/en/docs/A-Tune/figures/picture4.png deleted file mode 100644 index c576fd0369008e847e6943d6f99351caccf9f3e5..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/A-Tune/figures/picture4.png and /dev/null differ diff --git a/docs/en/docs/A-Tune/getting-to-know-a-tune.md b/docs/en/docs/A-Tune/getting-to-know-a-tune.md deleted file mode 100644 index 2092e0152e2c31ea4bf1aa95277302bcc981b6a9..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/getting-to-know-a-tune.md +++ /dev/null @@ -1,195 +0,0 @@ -# Getting to Know A-Tune - -- [Getting to Know A-Tune](#getting-to-know-a-tune) - - [Introduction](#introduction) - - [Architecture](#architecture) - - [Supported Features and Service Models](#supported-features-and-service-models) - - - -## Introduction - -An operating system \(OS\) is basic software that connects applications and hardware. It is critical for users to adjust OS and application configurations and make full use of software and hardware capabilities to achieve optimal service performance. However, numerous workload types and varied applications run on the OS, and the requirements on resources are different. Currently, the application environment composed of hardware and software involves more than 7000 configuration objects. As the service complexity and optimization objects increase, the time cost for optimization increases exponentially. As a result, optimization efficiency decreases sharply. Optimization becomes complex and brings great challenges to users. - -Second, as infrastructure software, the OS provides a large number of software and hardware management capabilities. The capability required varies in different scenarios. Therefore, capabilities need to be enabled or disabled depending on scenarios, and a combination of capabilities will maximize the optimal performance of applications. - -In addition, the actual business embraces hundreds and thousands of scenarios, and each scenario involves a wide variety of hardware configurations for computing, network, and storage. The lab cannot list all applications, business scenarios, and hardware combinations. - -To address the preceding challenges, openEuler launches A-Tune. - -A-Tune is an AI-based engine that optimizes system performance. It uses AI technologies to precisely profile business scenarios, discover and infer business characteristics, so as to make intelligent decisions, match with the optimal system parameter configuration combination, and give recommendations, ensuring the optimal business running status. - -![](figures/en-us_image_0227497000.png) - -## Architecture - -The following figure shows the A-Tune core technical architecture, which consists of intelligent decision-making, system profile, and interaction system. - -- Intelligent decision-making layer: consists of the awareness and decision-making subsystems, which implements intelligent awareness of applications and system optimization decision-making, respectively. -- System profile layer: consists of the feature engineering and two-layer classification model. The feature engineering is used to automatically select service features, and the two-layer classification model is used to learn and classify service models. -- Interaction system layer: monitors and configures various system resources and executes optimization policies. - -![](figures/en-us_image_0227497343.png) - -## Supported Features and Service Models - -### Supported Features - -[Table 1](#table1919220557576) describes the main features supported by A-Tune, feature maturity, and usage suggestions. - -**Table 1** Feature maturity - - - - - - - - - - - - - - - - - - - -

Feature

-

Maturity

-

Usage Suggestion

-

Auto optimization of 15 applications in 11 workload types

-

Tested

-

Pilot

-

User-defined profile and service models

-

Tested

-

Pilot

-

Automatic parameter optimization

-

Tested

-

Pilot

-
- - -### Supported Service Models - -Based on the workload characteristics of applications, A-Tune classifies services into 11 types. For details about the bottleneck of each type and the applications supported by A-Tune, see [Table 2](#table2819164611311). - -**Table 2** Supported workload types and applications - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Service category

-

Type

-

Bottleneck

-

Supported Application

-

default

-

Default type

-

Low resource usage in terms of cpu, memory, network, and I/O

-

N/A

-

webserver

-

Web application

-

Bottlenecks of cpu and network

-

Nginx, Apache Traffic Server

-

database

-

Database

-		Bottlenecks of cpu, memory, and I/O

-

Mongodb, Mysql, Postgresql, Mariadb

-

big_data

-

Big data

-

Bottlenecks of cpu and memory

-

Hadoop-hdfs, Hadoop-spark

-

middleware

-

Middleware framework

-

Bottlenecks of cpu and network

-

Dubbo

-

in-memory_database

-

Memory database

-

Bottlenecks of memory and I/O

-

Redis

-

basic-test-suite

-

Basic test suite

-

Bottlenecks of cpu and memory

-

SPECCPU2006, SPECjbb2015

-

hpc

-

Human genome

-

Bottlenecks of cpu, memory, and I/O

-

Gatk4

-

storage

-

Storage

-

Bottlenecks of network, and I/O

-

Ceph

-

virtualization

-

Virtualization

-

Bottlenecks of cpu, memory, and I/O

-

Consumer-cloud, Mariadb

-

docker

-

Docker

-

Bottlenecks of cpu, memory, and I/O

-

Mariadb

-
- - - diff --git a/docs/en/docs/A-Tune/installation-and-deployment.md b/docs/en/docs/A-Tune/installation-and-deployment.md deleted file mode 100644 index ae6bca81115139c4be67069003add94f6a030dfa..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/installation-and-deployment.md +++ /dev/null @@ -1,524 +0,0 @@ -# Installation and Deployment - -This chapter describes how to install and deploy A-Tune. - -- [Installation and Deployment](#installation-and-deployment) - - [Software and Hardware Requirements](#software-and-hardware-requirements) - - [Hardware Requirement](#hardware-requirement) - - [Software Requirement](#software-requirement) - - [Environment Preparation](#environment-preparation) - - [A-Tune Installation](#a-tune-installation) - - [Installation Modes](#installation-modes) - - [Installation Procedure](#installation-procedure) - - [A-Tune Deployment](#a-tune-deployment) - - [Overview](#overview) - - [Example](#example) - - [Example](#example-1) - - [Starting A-Tune](#starting-a-tune) - - [Starting A-Tune engine](#starting-a-tune-engine) - -## Software and Hardware Requirements - -### Hardware Requirement - -- Huawei Kunpeng 920 processor - -### Software Requirement - -- OS: openEuler 22.03 - -## Environment Preparation - -For details about installing an openEuler OS, see the [_openEuler Installation Guide_](../Installation/Installation.md). - -## A-Tune Installation - -This section describes the installation modes and methods of the A-Tune. - -### Installation Modes - -A-Tune can be installed in single-node, distributed, and cluster modes. - -- Single-node mode - - The client and server are installed on the same system. - -- Distributed mode - - The client and server are installed on different systems. - -- Cluster mode - A cluster consists of a client and more than one servers. - -The installation modes are as follows: - -![](./figures/en-us_image_0231122163.png) - -### Installation Procedure - -To install the A-Tune, perform the following steps: - -1. Mount an openEuler ISO image. - - ```shell - mount openEuler-22.03-LTS-everything-x86_64-dvd.iso /mnt - ``` - - > Use the **everything** ISO image. - -2. Configure the local Yum source. - - ```shell - vim /etc/yum.repos.d/local.repo - ``` - - The configured contents are as follows: - - ```conf - [local] - name=local - baseurl=file:///mnt - gpgcheck=1 - enabled=1 - ``` - -3. Import the GPG public key of the RPM digital signature to the system. - - ```shell - rpm --import /mnt/RPM-GPG-KEY-openEuler - ``` - -4. Install an A-Tune server. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >In this step, both the server and client software packages are installed. For the single-node deployment, skip **Step 5**. - - ```shell - yum install atune -y - yum install atune-engine -y - ``` - -5. For a distributed mode, install an A-Tune client on associated server. - - ```shell - yum install atune-client -y - ``` - -6. Check whether the installation is successful. - - ```shell - $ rpm -qa | grep atune - atune-client-xxx - atune-db-xxx - atune-xxx - atune-engine-xxx - ``` - - If the preceding information is displayed, the installation is successful. - -## A-Tune Deployment - -This section describes how to deploy A-Tune. - -### Overview - -The configuration items in the A-Tune configuration file **/etc/atuned/atuned.cnf** are described as follows: - -- A-Tune service startup configuration (modify the parameter values as required). - - - **protocol**: Protocol used by the gRPC service. The value can be **unix** or **tcp**. **unix** indicates the local socket communication mode, and **tcp** indicates the socket listening port mode. The default value is **unix**. - - **address**: Listening IP address of the gRPC service. The default value is **unix socket**. If the gRPC service is deployed in distributed mode, change the value to the listening IP address. - - **port**: Listening port of the gRPC server. The value ranges from 0 to 65535. If **protocol** is set to **unix**, you do not need to set this parameter. - - **connect**: IP address list of the nodes where the A-Tune is located when the A-Tune is deployed in a cluster. IP addresses are separated by commas (,). - - **rest_host**: Listening address of the REST service. The default value is localhost. - - **rest_port**: Listening port of the REST service. The value ranges from 0 to 65535. The default value is 8383. - - **engine_host**: IP address for connecting to the A-Tune engine service of the system. - - **engine_port**: Port for connecting to the A-Tune engine service of the system. - - **sample_num**: Number of samples collected when the system executes the analysis process. The default value is 20. - - **interval**: Interval for collecting samples when the system executes the analysis process. The default value is 5s. - - **grpc_tls**: Indicates whether to enable SSL/TLS certificate verification for the gRPC service. By default, this function is disabled. After grpc_tls is enabled, you need to set the following environment variables before running the **atune-adm** command to communicate with the server: - - export ATUNE_TLS=yes - - export ATUNED_CACERT= - - export ATUNED_CLIENTCERT= - - export ATUNED_CLIENTKEY= - - export ATUNED_SERVERCN=server - - **tlsservercafile**: Path of the gPRC server's CA certificate. - - **tlsservercertfile**: Path of the gPRC server certificate. - - **tlsserverkeyfile**: Path of the gPRC server key. - - **rest_tls**: Indicates whether to enable SSL/TLS certificate verification for the REST service. This function is enabled by default. - - **tlsrestcacertfile**: Path of the server's CA certificate of the REST service. - - **tlsrestservercertfile**: Path of the server certificate of the REST service. - - **tlsrestserverkeyfile**: Indicates the key path of the REST service. - - **engine_tls**: Indicates whether to enable SSL/TLS certificate verification for the A-Tune engine service. This function is enabled by default.. - - **tlsenginecacertfile**: Path of the client CA certificate of the A-Tune engine service. - - **tlsengineclientcertfile**: Client certificate path of the A-Tune engine service. - - **tlsengineclientkeyfile**: Client key path of the A-Tune engine service. - -- System information - - System is the parameter information required for system optimization. You must modify the parameter information according to the actual situation. - - - **disk**: Disk information to be collected during the analysis process or specified disk during disk optimization. - - **network**: NIC information to be collected during the analysis process or specified NIC during NIC optimization. - - **user**: User name used for ulimit optimization. Currently, only the user **root** is supported. - -- Log information - - Change the log level as required. The default log level is info. Log information is recorded in the **/var/log/messages** file. - -- Monitor information - - Hardware information that is collected by default when the system is started. - -- Tuning information - - Tuning is the parameter information required for offline tuning. - - - **noise**: Evaluation value of Gaussian noise. - - **sel_feature**: Indicates whether to enable the function of generating the importance ranking of offline tuning parameters. By default, this function is disabled. - -#### Example - -```conf -#################################### server ############################### - # atuned config - [server] - # the protocol grpc server running on - # ranges: unix or tcp - protocol = unix - - # the address that the grpc server to bind to - # default is unix socket /var/run/atuned/atuned.sock - # ranges: /var/run/atuned/atuned.sock or ip address - address = /var/run/atuned/atuned.sock - - # the atune nodes in cluster mode, separated by commas - # it is valid when protocol is tcp - # connect = ip01,ip02,ip03 - - # the atuned grpc listening port - # the port can be set between 0 to 65535 which not be used - # port = 60001 - - # the rest service listening port, default is 8383 - # the port can be set between 0 to 65535 which not be used - rest_host = localhost - rest_port = 8383 - - # the tuning optimizer host and port, start by engine.service - # if engine_host is same as rest_host, two ports cannot be same - # the port can be set between 0 to 65535 which not be used - engine_host = localhost - engine_port = 3838 - - # when run analysis command, the numbers of collected data. - # default is 20 - sample_num = 20 - - # interval for collecting data, default is 5s - interval = 5 - - # enable gRPC authentication SSL/TLS - # default is false - # grpc_tls = false - # tlsservercafile = /etc/atuned/grpc_certs/ca.crt - # tlsservercertfile = /etc/atuned/grpc_certs/server.crt - # tlsserverkeyfile = /etc/atuned/grpc_certs/server.key - - # enable rest server authentication SSL/TLS - # default is true - rest_tls = true - tlsrestcacertfile = /etc/atuned/rest_certs/ca.crt - tlsrestservercertfile = /etc/atuned/rest_certs/server.crt - tlsrestserverkeyfile = /etc/atuned/rest_certs/server.key - - # enable engine server authentication SSL/TLS - # default is true - engine_tls = true - tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt - tlsengineclientcertfile = /etc/atuned/engine_certs/client.crt - tlsengineclientkeyfile = /etc/atuned/engine_certs/client.key - - - #################################### log ############################### - [log] - # either "debug", "info", "warn", "error", "critical", default is "info" - level = info - - #################################### monitor ############################### - [monitor] - # with the module and format of the MPI, the format is {module}_{purpose} - # the module is Either "mem", "net", "cpu", "storage" - # the purpose is "topo" - module = mem_topo, cpu_topo - - #################################### system ############################### - # you can add arbitrary key-value here, just like key = value - # you can use the key in the profile - [system] - # the disk to be analysis - disk = sda - - # the network to be analysis - network = enp189s0f0 - - user = root - - #################################### tuning ############################### - # tuning configs - [tuning] - noise = 0.000000001 - sel_feature = false -``` - -The configuration items in the configuration file **/etc/atuned/engine.cnf** of the A-Tune engine are described as follows: - -- Startup configuration of the A-Tune engine service (modify the parameter values as required). - - - **engine_host**: Listening address of the A-Tune engine service. The default value is localhost. - - **engine_port**: Listening port of the A-Tune engine service. The value ranges from 0 to 65535. The default value is 3838. - - **engine_tls**: Indicates whether to enable SSL/TLS certificate verification for the A-Tune engine service. This function is enabled by default. - - **tlsenginecacertfile**: Path of the server CA certificate of the A-Tune engine service. - - **tlsengineservercertfile**: Path of the server certificate of the A-Tune engine service. - - **tlsengineserverkeyfile**: Server key path of the A-Tune engine service. - -- Log information - - Change the log level as required. The default log level is info. Log information is recorded in the **/var/log/messages** file. - -#### Example - -```conf -#################################### engine ############################### - [server] - # the tuning optimizer host and port, start by engine.service - # if engine_host is same as rest_host, two ports cannot be same - # the port can be set between 0 to 65535 which not be used - engine_host = localhost - engine_port = 3838 - - # enable engine server authentication SSL/TLS - # default is true - engine_tls = true - tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt - tlsengineservercertfile = /etc/atuned/engine_certs/server.crt - tlsengineserverkeyfile = /etc/atuned/engine_certs/server.key - - #################################### log ############################### - [log] - # either "debug", "info", "warn", "error", "critical", default is "info" - level = info -``` - -## Starting A-Tune - -After A-Tune is installed, you need to configure the A-Tune service before starting it. - -- Configure the A-Tune service. - Modify the network adapter and drive information in the **atuned.cnf** configuration file. - > Note: - > - > If atuned is installed through `make install`, the network adapter and drive information in the configuration file is automatically updated to the default devices on the machine. To collect data from other devices, perform the following steps to configure atuned. - - Run the following command to search for the network adapter that needs to be specified for optimization or data collection, and change the value of **network** in the **/etc/atuned/atuned.cnf** file to the specified network adapter: - - ```shell - ip addr - ``` - - Run the following command to search for the drive that need to be specified for optimization or data collection, and change the value of **disk** in the **/etc/atuned/atuned.cnf** file to the specified drive: - - ```shell - fdisk -l | grep dev - ``` - -- About the certificate: - The A-Tune engine and client use the gRPC communication protocol. Therefore, you need to configure a certificate to ensure system security. For information security purposes, A-Tune does not provide a certificate generation method. You need to configure a system certificate by yourself. - If security is not considered, set **rest_tls** and **engine_tls** in the **/etc/atuned/atuned.cnf** file to **false**, set **engine_tls** in the **/etc/atuned/engine.cnf** file to **false**. - A-Tune is not liable for any consequences incurred if no security certificate is configured. - -- Start the atuned service. - - ```shell - systemctl start atuned - ``` - -- Query the atuned service status. - - ```shell - systemctl status atuned - ``` - - If the following command output is displayed, the service is started successfully: - - ![](./figures/en-us_image_0214540398.png) - -## Starting A-Tune Engine - -To use AI functions, you need to start the A-Tune engine service. - -- Start the atune-engine service. - - ```shell - systemctl start atune-engine - ``` - -- Query the atune-engine service status. - - ```shell - systemctl status atune-engine - ``` - - If the following command output is displayed, the service is started successfully: - - ![](./figures/en-us_image_0245342444.png) - -## Distributed Deployment - -### Purpose of Distributed Deployment - -A-Tune supports distributed deployment to implement distributed architecture and on-demand deployment. The components of A-Tune can be deployed separately. Lightweight component deployment has little impact on services and avoids installing too many dependencies to reduce the system load. - -This document describes only a common deployment mode: deploying the client and server on the same node and deploying the engine module on another node. For details about other deployment modes, contact A-Tune developers. - -**Deployment relationship** -![](figures/picture1.png) - -### Configuration File - -In distributed deployment mode, you need to configure the write the IP address and port number of the engine in the configuration file so that other components can access the engine component through the IP address. - -1. Modify the **/etc/atuned/atuned.cnf** file on the server node. - - - Change the values of **engine_host** and **engine_port** in line 34 to the IP address and port number of the engine node. For the deployment in the preceding figure, the values are **engine_host = 192.168.0.1 engine_port = 3838**. - - Change the values of **rest_tls** and **engine_tls** in lines 49 and 55 to **false**. Otherwise, you need to apply for and configure certificates. You do not need to configure SSL certificates in the test environment. However, you need to configure SSL certificates in the production environment to prevent security risks. - -2. Modify the**/etc/atuned/engine.cnf** file on the engine node. - - - Change the values of **engine_host** and **engine_port** in lines 17 and 18 to the IP address and port number of the engine node. For the deployment in the preceding figure, the value are **engine_host = 192.168.0.1 engine_port = 3838**. - - Change the value of **engine_tls** in line 22 to **false**. - -3. After modifying the configuration file, restart the service for the modification to take effect. - - - Run the `systemctl restart atuned command` on the server node. - - Run the `systemctl restart atune-engine` command on the engine node. - -4. (Optional) Run the `tuning` command in the **A-Tune/examples/tuning/compress** folder. - - - For details, see **A-Tune/examples/tuning/compress/README**. - - Run the `atune-adm tuning --project compress --detail compress_client.yaml` command. - - This step is to check whether the distributed deployment is successful. - -### Precautions - -1. This document does not describe how to configure the authentication certificates. You can set **rest_tls** or **engine_tls** in the **atuned.cnf** and **engine.cnf** files to **false** if necessary. -2. After modifying the configuration file, restart the service. Otherwise, the modification does not take effect. -3. Do not enable the proxy when using A-Tune. -4. The **disk** and **network** items of the **\[system]** section in the **atuned.cnf** file need to be modified. For details about how to modify the items, see the [A-Tune User Guide](https://gitee.com/gaoruoshu/A-Tune/blob/master/Documentation/UserGuide/A-Tune%E7%94%A8%E6%88%B7%E6%8C%87%E5%8D%97.md). - -### Example - -#### atuned.cnf - -```conf -# ...... - -# the tuning optimizer host and port, start by engine.service -# if engine_host is same as rest_host, two ports cannot be same -# the port can be set between 0 to 65535 which not be used -engine_host = 192.168.0.1 -engine_port = 3838 - -# ...... -``` - -#### engine.cnf - -```bash -[server] -# the tuning optimizer host and port, start by engine.service -# if engine_host is same as rest_host, two ports cannot be same -# the port can be set between 0 to 65535 which not be used -engine_host = 192.168.0.1 -engine_port = 3838 -``` - -## Cluster Deployment - -### Purpose of Cluster Deployment - -To support fast tuning in multi-node scenarios, A-Tune supports dynamic tuning of parameter settings on multiple nodes at the same time. In this way, you do not need to tune each node separately, improving tuning efficiency. -Cluster deployment mode consists of one master node and several agent nodes. The client and server are deployed on the master node to receive commands and interact with the engine. Other nodes receive instructions from the master node and configure the parameters of the current node. - -**Deployment relationship** - ![](figures/picture4.png) - -In the preceding figure, the client and server are deployed on the node whose IP address is 192.168.0.0. Project files are stored on this node. Other nodes do not contain project files. -The master node communicates with the agent nodes through TCP. Therefore, you need to modify the configuration file. - -### Modifications to atuned.cnf - -1. Set the value of **protocol** to **tcp**. -2. Set the value of **address** to the IP address of the current node. -3. Set the value of **connect** to the IP addresses of all nodes. The first IP address is the IP address of the master node, and the subsequent IP addresses are the IP addresses of agent nodes. Use commas (,) to separate the IP addresses. -4. During debugging, you can set **rest_tls** and **engine_tls** to **false**. -5. Perform the same modification on the **atuned.cnf** files of all the master and agent nodes. - -### Precautions - -1. The values of **engine_host** and **engine_port** must be consistent in the **engine.cnf** file and the **atuned.cnf** file on the server. -2. This document does not describe how to configure the authentication certificates. You can set **rest_tls** or **engine_tls** in the **atuned.cnf** and **engine.cnf** files to **false** if necessary. -3. After modifying the configuration file, restart the service. Otherwise, the modification does not take effect. -4. Do not enable the proxy when using A-Tune. - -### Example - -#### atuned.cnf - -```conf -# ...... - -[server] -# the protocol grpc server running on -# ranges: unix or tcp -protocol = tcp - -# the address that the grpc server to bind to -# default is unix socket /var/run/atuned/atuned.sock -# ranges: /var/run/atuned/atuned.sock or ip address -address = 192.168.0.0 - -# the atune nodes in cluster mode, separated by commas -# it is valid when protocol is tcp -connect = 192.168.0.0,192.168.0.1,192.168.0.2,192.168.0.3 - -# the atuned grpc listening port -# the port can be set between 0 to 65535 which not be used -port = 60001 - -# the rest service listening port, default is 8383 -# the port can be set between 0 to 65535 which not be used -rest_host = localhost -rest_port = 8383 - -# the tuning optimizer host and port, start by engine.service -# if engine_host is same as rest_host, two ports cannot be same -# the port can be set between 0 to 65535 which not be used -engine_host = 192.168.1.1 -engine_port = 3838 - -# ...... -``` - -#### engine.cnf - -```bash -[server] -# the tuning optimizer host and port, start by engine.service -# if engine_host is same as rest_host, two ports cannot be same -# the port can be set between 0 to 65535 which not be used -engine_host = 192.168.1.1 -engine_port = 3838 -``` - -Note: For details about the **engine.cnf** file, see the configuration file for distributed deployment. diff --git a/docs/en/docs/A-Tune/native-turbo.md b/docs/en/docs/A-Tune/native-turbo.md deleted file mode 100644 index 4b1050a5044e73309fd471a96c84814818bbed9b..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/native-turbo.md +++ /dev/null @@ -1,54 +0,0 @@ -# Native-Turbo - -## Overview - -The code segment and data segment of a large program can reach hundreds of MB, and the TLB miss rate of key service processes is high. The size of the kernel page table affects the performance. - -To facilitate the use of huge pages, the Native-Turbo feature enables the system to automatically use huge pages when loading programs. Huge pages can be used for code segments and data segments. - -## How to Use - -1. Enable the feature. - - This feature has two levels of switches. `sysctl fs.exec-use-hugetlb` determines whether to enable this feature in the system. (This command can only be run by the **root** user. The value `0` indicates that this feature is disabled, and the value `1` indicates that this feature is enabled. Other values are invalid.) - - If not enabled, this feature will not be used even if users set environment variables because the kernel will ignore related processes. - - After this feature is enabled, common users can use the environment variable `HUGEPAGE_PROBE` to determine whether to use huge pages for running programs. If the value is `1`, huge pages are used. If the value is not set, huge pages are not used. - - ```shell - sysctl fs.exec-use-hugetlb=1 # The main program uses huge pages. - export HUGEPAGE_PROBE=1 # The dynamic library uses huge pages. - ``` - - You can also configure the environment variable `LD_HUGEPAGE_LIB=1` to force all segments to use huge pages. - -2. Mark the segments that need to use huge pages. By default, all segments are marked. `-x` only marks code segments. `-d` clears existing marks. - - ```shell - hugepageedit [-x] [-d] app - ``` - - This tool is provided by the glibc-devel package. - -3. Run the application. - - ./app - -## Restrictions - -1. The program and dynamic library must be compiled in 2 MB alignment mode by adding the following GCC compilation parameters: - - ```shell - -zcommon-page-size=0x200000 -zmax-page-size=0x200000 - ``` - -2. Sufficient huge pages must be reserved before use. Otherwise, the program wil fail to be executed. - - If the cgroup is used, pay attention to the `hugetlb` limit. If the limit is less than the number of required huge pages, the system may break down during running. - -3. The size of the process page table is changed to 2 MB. Therefore, the parameters invoked by the system such as `mprotect` must be aligned by 2 MB. Otherwise, the execution will fail. - -4. The LibcarePlus hot patch mechanism is not supported. - -5. Huge pages cannot be shared among multiple processes because they will consume multiple times of memory. diff --git a/docs/en/docs/A-Tune/usage-instructions.md b/docs/en/docs/A-Tune/usage-instructions.md deleted file mode 100644 index 49524f10328b470080145f008d8d3e520f5f0e5d..0000000000000000000000000000000000000000 --- a/docs/en/docs/A-Tune/usage-instructions.md +++ /dev/null @@ -1,1146 +0,0 @@ -# Usage Instructions - -You can use functions provided by A-Tune through the CLI client atune-adm. This chapter describes the functions and usage of the A-Tune client. - -- [Usage Instructions](#usage-instructions) - - [Overview](#overview) - - [Querying Workload Types](#querying-workload-types) - - [list](#list) - - [Workload Type Analysis and Auto Optimization](#workload-type-analysis-and-auto-optimization) - - [analysis](#analysis) - - [User-defined Model](#user-defined-model) - - [define](#define) - - [collection](#collection) - - [train](#train) - - [undefine](#undefine) - - [Querying Profiles](#querying-profiles) - - [info](#info) - - [Updating a Profile](#updating-a-profile) - - [update](#update) - - [Activating a Profile](#activating-a-profile) - - [profile](#profile) - - [Rolling Back Profiles](#rolling-back-profiles) - - [rollback](#rollback) - - [Updating Database](#updating-database) - - [upgrade](#upgrade) - - [Querying System Information](#querying-system-information) - - [check](#check) - - [Automatic Parameter Optimization](#automatic-parameter-optimization) - - [Tuning](#tuning) - - - -## Overview - -- You can run the **atune-adm help/--help/-h** command to query commands supported by atune-adm. -- The **define**, **update**, **undefine**, **collection**, **train**, and **upgrade **commands do not support remote execution. -- In the command format, brackets \(\[\]\) indicate that the parameter is optional, and angle brackets \(<\>\) indicate that the parameter is mandatory. The actual parameters prevail. - - -## Querying Workload Types - - - -### list - -#### Function - -Query the supported profiles, and the values of Active. - -#### Format - -**atune-adm list** - -#### Example - -``` -# atune-adm list - -Support profiles: -+------------------------------------------------+-----------+ -| ProfileName | Active | -+================================================+===========+ -| arm-native-android-container-robox | false | -+------------------------------------------------+-----------+ -| basic-test-suite-euleros-baseline-fio | false | -+------------------------------------------------+-----------+ -| basic-test-suite-euleros-baseline-lmbench | false | -+------------------------------------------------+-----------+ -| basic-test-suite-euleros-baseline-netperf | false | -+------------------------------------------------+-----------+ -| basic-test-suite-euleros-baseline-stream | false | -+------------------------------------------------+-----------+ -| basic-test-suite-euleros-baseline-unixbench | false | -+------------------------------------------------+-----------+ -| basic-test-suite-speccpu-speccpu2006 | false | -+------------------------------------------------+-----------+ -| basic-test-suite-specjbb-specjbb2015 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-hdfs-dfsio-hdd | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-hdfs-dfsio-ssd | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-bayesian | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-kmeans | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql1 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql10 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql2 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql3 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql4 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql5 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql6 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql7 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql8 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-sql9 | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-tersort | false | -+------------------------------------------------+-----------+ -| big-data-hadoop-spark-wordcount | false | -+------------------------------------------------+-----------+ -| cloud-compute-kvm-host | false | -+------------------------------------------------+-----------+ -| database-mariadb-2p-tpcc-c3 | false | -+------------------------------------------------+-----------+ -| database-mariadb-4p-tpcc-c3 | false | -+------------------------------------------------+-----------+ -| database-mongodb-2p-sysbench | false | -+------------------------------------------------+-----------+ -| database-mysql-2p-sysbench-hdd | false | -+------------------------------------------------+-----------+ -| database-mysql-2p-sysbench-ssd | false | -+------------------------------------------------+-----------+ -| database-postgresql-2p-sysbench-hdd | false | -+------------------------------------------------+-----------+ -| database-postgresql-2p-sysbench-ssd | false | -+------------------------------------------------+-----------+ -| default-default | false | -+------------------------------------------------+-----------+ -| docker-mariadb-2p-tpcc-c3 | false | -+------------------------------------------------+-----------+ -| docker-mariadb-4p-tpcc-c3 | false | -+------------------------------------------------+-----------+ -| hpc-gatk4-human-genome | false | -+------------------------------------------------+-----------+ -| in-memory-database-redis-redis-benchmark | false | -+------------------------------------------------+-----------+ -| middleware-dubbo-dubbo-benchmark | false | -+------------------------------------------------+-----------+ -| storage-ceph-vdbench-hdd | false | -+------------------------------------------------+-----------+ -| storage-ceph-vdbench-ssd | false | -+------------------------------------------------+-----------+ -| virtualization-consumer-cloud-olc | false | -+------------------------------------------------+-----------+ -| virtualization-mariadb-2p-tpcc-c3 | false | -+------------------------------------------------+-----------+ -| virtualization-mariadb-4p-tpcc-c3 | false | -+------------------------------------------------+-----------+ -| web-apache-traffic-server-spirent-pingpo | false | -+------------------------------------------------+-----------+ -| web-nginx-http-long-connection | true | -+------------------------------------------------+-----------+ -| web-nginx-https-short-connection | false | -+------------------------------------------------+-----------+ - -``` - ->![](public_sys-resources/icon-note.gif) **NOTE:** ->If the value of Active is **true**, the profile is activated. In the example, the profile of web-nginx-http-long-connection is activated. - -## Workload Type Analysis and Auto Optimization - - -### analysis - -#### Function - -Collect real-time statistics from the system to identify and automatically optimize workload types. - -#### Format - -**atune-adm analysis** \[OPTIONS\] - -#### Parameter Description - -- OPTIONS - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

--model, -m

-

New model generated after user self-training

-

--characterization, -c

-

Use the default model for application identification and do not perform automatic optimization

-

--times value, -t value

-

Time duration for data collection

-

--script value, -s value

-

File to be executed

-
- - -#### Example - -- Use the default model for application identification. - - ``` - # atune-adm analysis --characterization - ``` - -- Use the default model to identify applications and perform automatic tuning. - - ``` - # atune-adm analysis - ``` - -- Use the user-defined training model for recognition. - - ``` - # atune-adm analysis --model /usr/libexec/atuned/analysis/models/new-model.m - ``` - - -## User-defined Model - -A-Tune allows users to define and learn new models. To define a new model, perform the following steps: - -1. Run the **define** command to define a new profile. -2. Run the **collection** command to collect the system data corresponding to the application. -3. Run the **train** command to train the model. - - -### define - -#### Function - -Add a user-defined application scenarios and the corresponding profile tuning items. - -#### Format - -**atune-adm define** - -#### Example - -Add a profile whose service_type is **test_service**, application_name is **test_app**, scenario_name is **test_scenario**, and tuning item configuration file is **example.conf**. - -``` -# atune-adm define test_service test_app test_scenario ./example.conf -``` - -The **example.conf** file can be written as follows (the following optimization items are optional and are for reference only). You can also run the **atune-adm info** command to view how the existing profile is written. - -``` - [main] - # list its parent profile - [kernel_config] - # to change the kernel config - [bios] - # to change the bios config - [bootloader.grub2] - # to change the grub2 config - [sysfs] - # to change the /sys/* config - [systemctl] - # to change the system service status - [sysctl] - # to change the /proc/sys/* config - [script] - # the script extension of cpi - [ulimit] - # to change the resources limit of user - [schedule_policy] - # to change the schedule policy - [check] - # check the environment - [tip] - # the recommended optimization, which should be performed manunaly -``` - -### collection - -#### Function - -Collect the global resource usage and OS status information during service running, and save the collected information to a CSV output file as the input dataset for model training. - ->![](public_sys-resources/icon-note.gif) **NOTE:** ->- This command depends on the sampling tools such as perf, mpstat, vmstat, iostat, and sar. ->- Currently, only the Kunpeng 920 CPU is supported. You can run the **dmidecode -t processor** command to check the CPU model. - -#### Format - -**atune-adm collection** - -#### Parameter Description - -- OPTIONS - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

--filename, -f

-

Name of the generated CSV file used for training: name-timestamp.csv

-

--output_path, -o

-

Path for storing the generated CSV file. The absolute path is required.

-

--disk, -b

-

Disk used during service running, for example, /dev/sda.

-

--network, -n

-

Network port used during service running, for example, eth0.

-

--app_type, -t

-

Mark the application type of the service as a label for training.

-

--duration, -d

-

Data collection time during service running, in seconds. The default collection time is 1200 seconds.

-

--interval, -i

-

Interval for collecting data, in seconds. The default interval is 5 seconds.

-
- - -#### Example - -``` -# atune-adm collection --filename name --interval 5 --duration 1200 --output_path /home/data --disk sda --network eth0 --app_type test_service-test_app-test_scenario -``` - -> Note: -> -> In the example, data is collected every 5 seconds for a duration of 1200 seconds. The collected data is stored as the *name* file in the **/home/data** directory. The application type of the service is defined by the `atune-adm define` command, which is **test_service-test_app-test_scenario** in this example. -> The data collection interval and duration can be specified using the preceding command options. - - -### train - -#### Function - -Use the collected data to train the model. Collect data of at least two application types during training. Otherwise, an error is reported. - -#### Format - -**atune-adm train** - -#### Parameter Description - -- OPTIONS - - | Parameter | Description | - | ----------------- | ------------------------------------------------------ | - | --data_path, -d | Path for storing CSV files required for model training | - | --output_file, -o | Model generated through training | - - -#### Example - -Use the CSV file in the **data** directory as the training input. The generated model **new-model.m** is stored in the **model** directory. - -``` -# atune-adm train --data_path /home/data --output_file /usr/libexec/atuned/analysis/models/new-model.m -``` - -### undefine - -#### Function - -Delete a user-defined profile. - -#### Format - -**atune-adm undefine** - -#### Example - -Delete the user-defined profile. - -``` -# atune-adm undefine test_service-test_app-test_scenario -``` - -## Querying Profiles - - -### info - -#### Function - -View the profile content. - -#### Format - -**atune-adm info** - -#### Example - -View the profile content of web-nginx-http-long-connection. - -``` -# atune-adm info web-nginx-http-long-connection - -*** web-nginx-http-long-connection: - -# -# nginx http long connection A-Tune configuration -# -[main] -include = default-default - -[kernel_config] -#TODO CONFIG - -[bios] -#TODO CONFIG - -[bootloader.grub2] -iommu.passthrough = 1 - -[sysfs] -#TODO CONFIG - -[systemctl] -sysmonitor = stop -irqbalance = stop - -[sysctl] -fs.file-max = 6553600 -fs.suid_dumpable = 1 -fs.aio-max-nr = 1048576 -kernel.shmmax = 68719476736 -kernel.shmall = 4294967296 -kernel.shmmni = 4096 -kernel.sem = 250 32000 100 128 -net.ipv4.tcp_tw_reuse = 1 -net.ipv4.tcp_syncookies = 1 -net.ipv4.ip_local_port_range = 1024 65500 -net.ipv4.tcp_max_tw_buckets = 5000 -net.core.somaxconn = 65535 -net.core.netdev_max_backlog = 262144 -net.ipv4.tcp_max_orphans = 262144 -net.ipv4.tcp_max_syn_backlog = 262144 -net.ipv4.tcp_timestamps = 0 -net.ipv4.tcp_synack_retries = 1 -net.ipv4.tcp_syn_retries = 1 -net.ipv4.tcp_fin_timeout = 1 -net.ipv4.tcp_keepalive_time = 60 -net.ipv4.tcp_mem = 362619 483495 725238 -net.ipv4.tcp_rmem = 4096 87380 6291456 -net.ipv4.tcp_wmem = 4096 16384 4194304 -net.core.wmem_default = 8388608 -net.core.rmem_default = 8388608 -net.core.rmem_max = 16777216 -net.core.wmem_max = 16777216 - -[script] -prefetch = off -ethtool = -X {network} hfunc toeplitz - -[ulimit] -{user}.hard.nofile = 102400 -{user}.soft.nofile = 102400 - -[schedule_policy] -#TODO CONFIG - -[check] -#TODO CONFIG - -[tip] -SELinux provides extra control and security features to linux kernel. Disabling SELinux will improve the performance but may cause security risks. = kernel -disable the nginx log = application -``` - -## Updating a Profile - -You can update the existing profile as required. - - -### update - -#### Function - -Update the original tuning items in the existing profile to the content in the **new.conf** file. - -#### Format - -**atune-adm update** - -#### Example - -Change the tuning item of the profile named **test_service-test_app-test_scenario** to **new.conf**. - -``` -# atune-adm update test_service-test_app-test_scenario ./new.conf -``` - -## Activating a Profile - -### profile - -#### Function - -Manually activate the profile to make it in the active state. - -#### Format - -**atune-adm profile** - -#### Parameter Description - -For details about the profile name, see the query result of the list command. - -#### Example - -Activate the profile corresponding to the web-nginx-http-long-connection. - -``` -# atune-adm profile web-nginx-http-long-connection -``` - -## Rolling Back Profiles - -### rollback - -#### Functions - -Roll back the current configuration to the initial configuration of the system. - -#### Format - -**atune-adm rollback** - -#### Example - -``` -# atune-adm rollback -``` - -## Updating Database - -### upgrade - -#### Function - -Update the system database. - -#### Format - -**atune-adm upgrade** - -#### Parameter Description - -- DB\_FILE - - New database file path. - - -#### Example - -The database is updated to **new\_sqlite.db**. - -``` -# atune-adm upgrade ./new_sqlite.db -``` - -## Querying System Information - - -### check - -#### Function - -Check the CPU, BIOS, OS, and NIC information. - -#### Format - -**atune-adm check** - -#### Example - -``` -# atune-adm check - cpu information: - cpu:0 version: Kunpeng 920-6426 speed: 2600000000 HZ cores: 64 - cpu:1 version: Kunpeng 920-6426 speed: 2600000000 HZ cores: 64 - system information: - DMIBIOSVersion: 0.59 - OSRelease: 4.19.36-vhulk1906.3.0.h356.eulerosv2r8.aarch64 - network information: - name: eth0 product: HNS GE/10GE/25GE RDMA Network Controller - name: eth1 product: HNS GE/10GE/25GE Network Controller - name: eth2 product: HNS GE/10GE/25GE RDMA Network Controller - name: eth3 product: HNS GE/10GE/25GE Network Controller - name: eth4 product: HNS GE/10GE/25GE RDMA Network Controller - name: eth5 product: HNS GE/10GE/25GE Network Controller - name: eth6 product: HNS GE/10GE/25GE RDMA Network Controller - name: eth7 product: HNS GE/10GE/25GE Network Controller - name: docker0 product: -``` - -## Automatic Parameter Optimization - -A-Tune provides the automatic search capability with the optimal configuration, saving the trouble of manually configuring parameters and performance evaluation. This greatly improves the search efficiency of optimal configurations. - - -### Tuning - -#### Function - -Use the specified project file to search the dynamic space for parameters and find the optimal solution under the current environment configuration. - -#### Format - -**atune-adm tuning** \[OPTIONS\] - ->![](public_sys-resources/icon-note.gif) **NOTE:** ->Before running the command, ensure that the following conditions are met: ->1. The YAML configuration file on the server has been edited and stored in the **/etc/atuned/tuning/** directory of the atuned service. ->2. The YAML configuration file of the client has been edited and stored on the atuned client. - -#### Parameter Description - -- OPTIONS - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

--restore, -r

-

Restores the initial configuration before tuning.

-

--project, -p

-

Specifies the project name in the YAML file to be restored.

-

--restart, -c

-

Perform tuning based on historical tuning results.

-

--detail, -d

-

Print detailed information about the tuning process.

-
- - - >![](public_sys-resources/icon-note.gif) **NOTE:** - >If this parameter is used, the -p parameter must be followed by a specific project name and the YAML file of the project must be specified. - - -- **PROJECT\_YAML**: YAML configuration file of the client. - -#### Configuration Description - -**Table 1** YAML file on the server - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Name

-

Description

-

Type

-

Value Range

-

project

-

Project name.

-

Character string

-

-

-

startworkload

-

Script for starting the service to be optimized.

-

Character string

-

-

-

stopworkload

-

Script for stopping the service to be optimized.

-

Character string

-

-

-

maxiterations

-

Maximum number of optimization iterations, which is used to limit the number of iterations on the client. Generally, the more optimization iterations, the better the optimization effect, but the longer the time required. Set this parameter based on the site requirements.

-

Integer

-

>10

-

object

-

Parameters to be optimized and related information.

-

For details about the object configuration items, see Table 2.

-

-

-

-

-
- -**Table 2** Description of object configuration items - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Name

-

Description

-

Type

-

Value Range

-

name

-

Parameter to be optimized.

-

Character string

-

-

-

desc

-

Description of parameters to be optimized.

-

Character string

-

-

-

get

-

Script for querying parameter values.

-

-

-

-

-

set

-

Script for setting parameter values.

-

-

-

-

-

needrestart

-

Specifies whether to restart the service for the parameter to take effect.

-

Enumeration

-

true or false

-

type

-

Parameter type. Currently, the discrete and continuous types are supported.

-

Enumeration

-

discrete or continuous

-

dtype

-

This parameter is available only when type is set to discrete. Currently, int, float and string are supported.

-

Enumeration

-

int, float, string

-

scope

-

Parameter setting range. This parameter is valid only when type is set to discrete and dtype is set to int or float, or type is set to continuous.

-

Integer/Float

-

The value is user-defined and must be within the valid range of this parameter.

-

step

-

Parameter value step, which is used when dtype is set to int or float.

-

Integer/Float

-

This value is user-defined.

-

items

-

Enumerated value of which the parameter value is not within the scope. This is used when dtype is set to int or float.

-

Integer/Float

-

The value is user-defined and must be within the valid range of this parameter.

-

options

-

Enumerated value range of the parameter value, which is used when dtype is set to string.

-

Character string

-

The value is user-defined and must be within the valid range of this parameter.

-
- -**Table 3** Description of configuration items of a YAML file on the client - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Name

-

Description

-

Type

-

Value Range

-

project

-

Project name, which must be the same as that in the configuration file on the server.

-

Character string

-

-

-

engine

-

Tuning algorithm.

-

Character string

-

"random", "forest", "gbrt", "bayes", "extraTrees"

-

iterations

-

Number of optimization iterations.

-

Integer

-

≥ 10

-

random_starts

-

Number of random iterations.

-

Integer

-

< iterations

-

feature_filter_engine

-

Parameter search algorithm, which is used to select important parameters. This parameter is optional.

-

Character string

-

"lhs"

-

feature_filter_cycle

-

Parameter search cycles, which is used to select important parameters. This parameter is used together with feature_filter_engine.

-

Integer

-

-

-

feature_filter_iters

-

Number of iterations for each cycle of parameter search, which is used to select important parameters. This parameter is used together with feature_filter_engine.

-

Integer

-

-

-

split_count

-

Number of evenly selected parameters in the value range of tuning parameters, which is used to select important parameters. This parameter is used together with feature_filter_engine.

-

Integer

-

-

-

benchmark

-

Performance test script.

-

-

-

-

-

evaluations

-

Performance test evaluation index.

-

For details about the evaluations configuration items, see Table 4.

-

-

-

-

-
- - -**Table 4** Description of evaluations configuration item - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Name

-

Description

-

Type

-

Value Range

-

name

-

Evaluation index name.

-

Character string

-

-

-

get

-

Script for obtaining performance evaluation results.

-

-

-

-

-

type

-

Specifies a positive or negative type of the evaluation result. The value positive indicates that the performance value is minimized, and the value negative indicates that the performance value is maximized.

-

Enumeration

-

positive or negative

-

weight

-

Weight of the index. The value ranges from 0 to 100.

-

Integer

-

0-100

-

threshold

-

Minimum performance requirement of the index.

-

Integer

-

User-defined

-
- -#### Example - -The following is an example of the YAML file configuration on a server: - -``` -project: "compress" -maxiterations: 500 -startworkload: "" -stopworkload: "" -object : - - - name : "compressLevel" - info : - desc : "The compresslevel parameter is an integer from 1 to 9 controlling the level of compression" - get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressLevel=' | awk -F '=' '{print $2}'" - set : "sed -i 's/compressLevel=\\s*[0-9]*/compressLevel=$value/g' /root/A-Tune/examples/tuning/compress/compress.py" - needrestart : "false" - type : "continuous" - scope : - - 1 - - 9 - dtype : "int" - - - name : "compressMethod" - info : - desc : "The compressMethod parameter is a string controlling the compression method" - get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressMethod=' | awk -F '=' '{print $2}' | sed 's/\"//g'" - set : "sed -i 's/compressMethod=\\s*[0-9,a-z,\"]*/compressMethod=\"$value\"/g' /root/A-Tune/examples/tuning/compress/compress.py" - needrestart : "false" - type : "discrete" - options : - - "bz2" - - "zlib" - - "gzip" - dtype : "string" -``` - -   - -The following is an example of the YAML file configuration on a client: - -``` -project: "compress" -engine : "gbrt" -iterations : 20 -random_starts : 10 - -benchmark : "python3 /root/A-Tune/examples/tuning/compress/compress.py" -evaluations : - - - name: "time" - info: - get: "echo '$out' | grep 'time' | awk '{print $3}'" - type: "positive" - weight: 20 - - - name: "compress_ratio" - info: - get: "echo '$out' | grep 'compress_ratio' | awk '{print $3}'" - type: "negative" - weight: 80 -``` - -   - -#### Example - -- Download test data. - ``` - wget http://cs.fit.edu/~mmahoney/compression/enwik8.zip - ``` -- Prepare the tuning environment. - - Example of **prepare.sh**: - ``` - #!/usr/bin/bash - if [ "$#" -ne 1 ]; then - echo "USAGE: $0 the path of enwik8.zip" - exit 1 - fi - - path=$( - cd "$(dirname "$0")" - pwd - ) - - echo "unzip enwik8.zip" - unzip "$path"/enwik8.zip - - echo "set FILE_PATH to the path of enwik8 in compress.py" - sed -i "s#compress/enwik8#$path/enwik8#g" "$path"/compress.py - - echo "update the client and server yaml files" - sed -i "s#python3 .*compress.py#python3 $path/compress.py#g" "$path"/compress_client.yaml - sed -i "s# compress/compress.py# $path/compress.py#g" "$path"/compress_server.yaml - - echo "copy the server yaml file to /etc/atuned/tuning/" - cp "$path"/compress_server.yaml /etc/atuned/tuning/ - ``` - Run the script. - ``` - sh prepare.sh enwik8.zip - ``` -- Run the `tuning` command to tune the parameters. - - ``` - atune-adm tuning --project compress --detail compress_client.yaml - ``` - -- Restore the configuration before running `tuning`. **compress** indicates the project name in the YAML file. - - ``` - atune-adm tuning --restore --project compress - ``` \ No newline at end of file diff --git a/docs/en/docs/Administration/administration.md b/docs/en/docs/Administration/administration.md deleted file mode 100644 index 0c32e256b1d6e2e2f7b268b47f7466b660f7744b..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/administration.md +++ /dev/null @@ -1,4 +0,0 @@ -# Administrator Guide - -This document provides common administrator operations of the openEuler system to help administrators better use the system. -This document is intended for all administrators who use openEuler. diff --git a/docs/en/docs/Administration/basic-configuration.md b/docs/en/docs/Administration/basic-configuration.md deleted file mode 100644 index 670406e23a8acf2d4fc17c3ec6138b6e79bdbce8..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/basic-configuration.md +++ /dev/null @@ -1,457 +0,0 @@ -# Basic Configuration - - - -- [Basic Configuration](#basic-configuration) - - [Setting the System Locale](#setting-the-system-locale) - - [Displaying the Current Locale Status](#displaying-the-current-locale-status) - - [Listing Available Locales](#listing-available-locales) - - [Setting the Locale](#setting-the-locale) - - [Setting the Keyboard Layout](#setting-the-keyboard-layout) - - [Displaying the Current Settings](#displaying-the-current-settings) - - [Listing Available Keyboard Layouts](#listing-available-keyboard-layouts) - - [Setting the Keyboard Layout](#setting-the-keyboard-layout-1) - - [Setting the Date and Time](#setting-the-date-and-time) - - [Using the timedatectl Command](#using-the-timedatectl-command) - - [Displaying the Current Date and Time](#displaying-the-current-date-and-time) - - [Synchronizing the System Clock with a Remote Server](#synchronizing-the-system-clock-with-a-remote-server) - - [Changing the Current Date](#changing-the-current-date) - - [Changing the Current Time](#changing-the-current-time) - - [Changing the Time Zone](#changing-the-time-zone) - - [Using the date Command](#using-the-date-command) - - [Displaying the Current Date and Time](#displaying-the-current-date-and-time-1) - - [Changing the Current Time](#changing-the-current-time-1) - - [Changing the Current Date](#changing-the-current-date-1) - - [Using the hwclock Command](#using-the-hwclock-command) - - [Real-Time Clock and System Clock](#real-time-clock-and-system-clock) - - [Displaying the Current Date and Time](#displaying-the-current-date-and-time-2) - - [Setting the Date and Time](#setting-the-date-and-time-1) - - [Setting kdump](#setting-kdump) - - [Setting the Memory Reserved for kdump](#setting-the-memory-reserved-for-kdump) - - [Parameter Formats of the Memory Reserved for kdump](#parameter-formats-of-the-memory-reserved-for-kdump) - - [Recommended Reserved Memory](#recommended-reserved-memory) - - [Disabling Network Drivers](#disabling-network-drivers) - - -## Setting the System Locale - -System locale settings are stored in the /etc/locale.conf file and can be modified by the localectl command. These settings are read at system boot by the systemd daemon. - -### Displaying the Current Locale Status - -To display the current locale status, run the following command: - -```shell -localectl status -``` - -Example command output: - -```shell -$ localectl status - System Locale: LANG=zh_CN.UTF-8 - VC Keymap: cn - X11 Layout: cn -``` - -### Listing Available Locales - -To display available locales, run the following command: - -```shell -localectl list-locales -``` - -You can check that by listing all Chinese locales with the following command: - -```shell -$ localectl list-locales | grep zh -zh_CN.UTF-8 -``` - -### Setting the Locale - -To set the language environment, run the following command as the user **root**. In the command, _locale_ indicates the language type to be set. Run the **localectl list-locales** command to obtain the value range. Change the value as required. - -```shell -localectl set-locale LANG=locale -``` - -For example, if you want to use Simplified Chinese as the locale, run the following command as the user **root**: - -```shell -localectl set-locale LANG=zh_CN.UTF-8 -``` - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> After the modification, log in again or run the command `source /etc/locale.conf` as the user **root** to update the configuration file for the modification to take effect: - -## Setting the Keyboard Layout - -Keyboard layout settings are stored in the /etc/locale.conf file and can be modified by the localectl command. These settings are read at early boot by the systemd daemon. - -### Displaying the Current Settings - -To display the current keyboard layout settings, run the following command: - -```shell -localectl status -``` - -Example command output: - -```shell -$ localectl status - System Locale: LANG=zh_CN.UTF-8 - VC Keymap: cn - X11 Layout: cn -``` - -### Listing Available Keyboard Layouts - -To list all available keyboard layouts that can be configured on openEuler, run the following command: - -```shell -localectl list-keymaps -``` - -For example, the command output of the Chinese keyboard layout is as follows: - -```shell -$ localectl list-keymaps | grep cn -cn -``` - -### Setting the Keyboard Layout - -To set the keyboard layout, run the following command as the user **root**. In the command, _map_ indicates the keyboard layout to be set. Run the **localectl list-keymaps** command to obtain the value range. Change it as required. - -```shell -localectl set-keymap map -``` - -The keyboard layout will be equally applied to graphical user interfaces. - -Then you can verify if your setting was successful by checking the status: - -```shell -$ localectl status - System Locale: LANG=zh_CN.UTF-8 - VC Keymap: cn - X11 Layout: us -``` - -## Setting the Date and Time - -This topic describes how to set the system date, time, and time zone by using timedatectl, date, and hwclock commands. - -### Using the timedatectl Command - -#### Displaying the Current Date and Time - -To display the current date and time, run the following command: - -```shell -timedatectl -``` - -Example command output: - -```shell -$ timedatectl - Local time: Mon 2019-09-30 04:05:00 EDT - Universal time: Mon 2019-09-30 08:05:00 UTC - RTC time: Mon 2019-09-30 08:05:00 - Time zone: China Standard Time (CST), UTC +8 -System clock synchronized: no - NTP service: inactive - RTC in local TZ: no -``` - -#### Synchronizing the System Clock with a Remote Server - -Your system clock can be automatically synchronized with a remote server using the Network Time Protocol (NTP). Run the following command as the user **root** to enable or disable NTP. The value of _boolean_ is **yes** or **no**, indicating that the NTP is enabled or disabled for automatic system clock synchronization. Change the value as required. - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> If the remote NTP server is enabled to automatically synchronize the system clock, you cannot manually change the date and time. If you need to manually change the date or time, ensure that automatic NTP system clock synchronization is disabled. You can run the **timedatectl set-ntp no** command to disable the NTP service. - -```shell -timedatectl set-ntp boolean -``` - -For example, to enable automatic remote time synchronization, run the following command: - -```shell -timedatectl set-ntp yes -``` - -#### Changing the Current Date - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> Before changing the date, ensure that automatic NTP system clock synchronization has been disabled. - -Run the following command as the user **root** to change the current date. In the command, _YYYY_ indicates the year, _MM_ indicates the month, and _DD_ indicates the day. Change them as required. - -```shell -timedatectl set-time YYYY-MM-DD -``` - -For example, to change the current date to August 14, 2019, run the following command as the user **root**: - -```shell -timedatectl set-time '2019-08-14' -``` - -#### Changing the Current Time - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> Before changing the time, ensure that automatic NTP system clock synchronization has been disabled. - -To change the current time, run the following command as the user **root**. In the command, _HH_ indicates the hour, _MM_ indicates the minute, and _SS_ indicates the second. Change them as required. - -```shell -timedatectl set-time HH:MM:SS -``` - -For example, to change the current time to 15:57:24, run the following command: - -```shell -timedatectl set-time 15:57:24 -``` - -#### Changing the Time Zone - -To list all available time zones, run the following command: - -```shell -timedatectl list-timezones -``` - -To change the current time zone, run the following command as the user **root**. In the command, _time\_zone_ indicates the time zone to be set. Change it as required. - -```shell -timedatectl set-timezone time_zone -``` - -Imagine you want to identify which time zone is closest to your present location while you are in Asia. You can check that by listing all available time zones in Asia with the following command: - -```shell -$ timedatectl list-timezones | grep Asia -Asia/Aden -Asia/Almaty -Asia/Amman -Asia/Anadyr -Asia/Aqtau -Asia/Aqtobe -Asia/Ashgabat -Asia/Baghdad -Asia/Bahrain -...... - -Asia/Seoul -Asia/Shanghai -Asia/Singapore -Asia/Srednekolymsk -Asia/Taipei -Asia/Tashkent -Asia/Tbilisi -Asia/Tehran -Asia/Thimphu -Asia/Tokyo -``` - -To change the time zone to Asia/Shanghai, run the following command: - -```shell -timedatectl set-timezone Asia/Shanghai -``` - -### Using the date Command - -#### Displaying the Current Date and Time - -To display the current date and time, run the following command: - -```shell -date -``` - -By default, the **date** command displays the local time. To display the time in Coordinated Universal Time (UTC), run the command with the --utc or -u command line option: - -```shell -date --utc -``` - -You can also customize the format of the displayed information by providing the + "format" option on the command line: - -```shell -date +"format" -``` - -**Table 1** Formatting options - -| Format Option | Description | -| :---- | :---- | -| %H | The hour in the HH format (for example, 17) | -| %M | The minute in the MM format (for example, 37) | -| %S | The second in the SS format (for example, 25) | -| %d | The day of the month in the DD format (for example, 15) | -| %m | The month in the MM format (for example, 07) | -| %Y | The year in the YYYY format (for example, 2019) | -| %Z | The time zone abbreviation (for example, CEST) | -| %F | The full date in the YYYY-MM-DD format (for example, 2019-7-15). This option is equal to %Y-%m-%d | -| %T | The full time in the HH:MM:SS format (for example, 18:30:25). This option is equal to %H:%M:%S | - -Example commands and outputs: - -- To display the current date and time: - - ```shell - $ date - Sat Aug 17 17:26:34 CST 2019 - ``` - -- To display the current date and time in UTC: - - ```shell - $ date --utc - Sat Aug 17 09:26:18 UTC 2019 - ``` - -- To customize the output of the date command: - - ```shell - $ date +"%Y-%m-%d %H:%M" - 2019-08-17 17:24 - ``` - -#### Changing the Current Time - -To change the current time, run the date command with the --set or -s option as the root user: Run the following command as the user **root**. In the command, _HH_ indicates the hour, _MM_ indicates the minute, and _SS_ indicates the second. Change them as required. - -```shell -date --set HH:MM:SS -``` - -By default, the date command sets the local time. To set the system clock in UTC instead, run the command with the --utc or -u command line option: - -```shell -date --set HH:MM:SS --utc -``` - -For example, to change the current time to 23:26:00, run the following command as the user **root**: - -```shell -date --set 23:26:00 -``` - -#### Changing the Current Date - -To change the current date, run the command with the --set or -s command line option. Run the following command as the user **root**. In the command, _YYYY_ indicates the year, _MM_ indicates the month, and _DD_ indicates the day. Change them as required. - -```shell -date --set YYYY-MM-DD -``` - -For example, to change the current date to November 2, 2019, run the following command as the user **root**: - -```shell -date --set 2019-11-02 -``` - -### Using the hwclock Command - -You can run the hwclock command to set the real time clock (RTC). - -#### Real-Time Clock and System Clock - -Linux divides clocks into the following types: - -- System clock: clock of the current Linux kernel. -- Hardware clock RTC: hardware clock of the mainboard powered by the battery. This clock can be set in the **Standard BIOS Feature** option of the BIOS. - -When Linux starts, it reads the RTC and sets the system clock time based on the RTC time. - -#### Displaying the Current Date and Time - -To display the current RTC date and time, run the following command as the user **root**: - -```shell -hwclock -``` - -Example command output: - -```shell -$ hwclock -2019-08-26 10:18:42.528948+08:00 -``` - -#### Setting the Date and Time - -Run the following command as the user **root** to change the date and time of the current hardware. In the command, _dd_ indicates the day, _mm_ indicates the month, _yyyy_ indicates the year, _HH_ indicates the hour, and _MM_ indicates the minute. Change them as required. - -```shell -hwclock --set --date "dd mm yyyy HH:MM" -``` - -For example, to change the current time to 21:17 on October 21, 2019, run the following command: - -```shell -hwclock --set --date "21 Oct 2019 21:17" --utc -``` - -## Setting kdump - -This section describes how to set the memory reserved for kdump and modify parameters in the kdump configuration file. - -### Setting the Memory Reserved for kdump - -#### Parameter Formats of the Memory Reserved for kdump - -The memory reserved for kdump must be added to the bootargs in the **/boot/efi/EFI/openEuler/grub.cfg** configuration file. The memory reserved for kdump has been added to the released openEuler version by default and can be adjusted as required. After adding or modifying the bootargs, restart the system for the setting to take effect. The parameter formats of the memory reserved for kdump are as follows: - -| Bootarg| Description| Default Value| Remarks| -|----------|----------|----------|----------| -| crashkernel=x| If the physical memory size is less than 4 GB, x of the memory is reserved for kdump.| The default value is 512 MB for x86.| This configuration method is used only when the available memory size is less than 4 GB. In this case, ensure that the available contiguous memory is sufficient for reservation.| -| crashkernel=x@y| x of the memory is reserved at the start address of y for kdump.| Unused| Ensure that x of the memory at the start address of y is not reserved for other modules.| -| crashkernel=x,high| If the physical memory size is less than 4 GB, 256 MB memory is reserved. If the physical memory size is greater than 4 GB, x of the memory is reserved for kdump. | The default value is 1024M,high for ARM64.| Ensure that the available physical contiguous memory size is greater than or equal to 256 MB when the memory size is less than 4 GB, and is greater than or equal to x when the memory size is greater than 4 GB. The actual reserved memory size is 256 MB + x. | -| crashkernel=x,low crashkernel=y,high| x of the memory is reserved for kdump when the physical memory size is less than 4 GB, and y of the memory is reserved for kdump when the physical memory size is greater than 4 GB. | Unused| Ensure that the available physical contiguous memory size is greater than or equal to x when the physical memory size is less than 4 GB, and is greater than or equal to y when the physical memory size is greater than 4 GB.| - -### Recommended Reserved Memory - -| Recommended Solution| Reserved Parameter| Description| -|----------|----------|----------| -| General solution| crashkernel=2048M,high| If the memory size is less than 4 GB, 256 MB is reserved for kdump. If the memory size is greater than 4 GB, 2048 MB is reserved for kdump. 256 + 2048 MB in total.| -| Economical solution| crashkernel=1024M,high| If the memory size is less than 4 GB, 256 MB is reserved for kdump. If the memory size is greater than 4 GB, 1024 MB is reserved for kdump. 256 + 1024 MB in total. It is recommended that kdump files not be dumped using the network in scenarios where the system memory size is less than 512 GB. In VM scenarios, you can reduce the reserved memory. You are advised to set crashkernel to 512M or crashkernel to 256M,high.| - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> If kdump files are not dumped using the network, you need to set the kdump file system not to pack network drivers. Loading the network driver requires a large amount of memory. As a result, the memory reserved for kdump may be insufficient and kdump may fail. Therefore, you are advised to disable network drivers. - -### Disabling Network Drivers - -In the kdump configuration file **/etc/kdump.conf**, the dracut parameters can be used to set the tailored driver module. You can configure the network driver to the tailored driver list to prevent the kdump file system from loading the driver. After the configuration file is modified, restart the kdump service for the modification to take effect. Set the dracut parameters as follows: - -`dracut_args --omit-drivers "mdio-gpi usb_8dev et1011c rt2x00usb bcm-phy-lib mac80211_hwsim rtl8723be rndis_host hns3_cae amd vrf rtl8192cu mt76x02-lib int51x1 ppp_deflate team_mode_loadbalance smsc911x aweth bonding mwifiex_usb hnae dnet rt2x00pci vaser_pci hdlc_ppp marvell rtl8xxxu mlxsw_i2c ath9k_htc rtl8150 smc91x cortina at803x rockchip cxgb4 spi_ks8995 mt76x2u smsc9420 mdio-cavium bnxt_en ch9200 dummy macsec ice mt7601u rtl8188ee ixgbevf net1080 liquidio_vf be2net mlxsw_switchx2 gl620a xilinx_gmii2rgmii ppp_generic rtl8192de sja1000_platform ath10k_core cc770_platform realte igb c_can_platform c_can ethoc dm9601 smsc95xx lg-vl600 ifb enic ath9 mdio-octeon ppp_mppe ath10k_pci cc770 team_mode_activebackup marvell10g hinic rt2x00lib mlx4_en iavf broadcom igc c_can_pci alx rtl8192se rtl8723ae microchip lan78xx atl1c rtl8192c-common almia ax88179_178a qed netxen_nic brcmsmac rt2800usb e1000 qla3xxx mdio-bitbang qsemi mdio-mscc-miim plx_pci ipvlan r8152 cx82310_eth slhc mt76x02-usb ems_pci xen-netfront usbnet pppoe mlxsw_minimal mlxsw_spectrum cdc_ncm rt2800lib rtl_usb hnae3 ath9k_common ath9k_hw catc mt76 hns_enet_drv ppp_async huawei_cdc_ncm i40e rtl8192ce dl2 qmi_wwan mii peak_usb plusb can-dev slcan amd-xgbe team_mode_roundrobin ste10Xp thunder_xcv pptp thunder_bgx ixgbe davicom icplus tap tun smsc75xx smsc dlci hns_dsaf mlxsw_core rt2800mmi softing uPD60620 vaser_usb dp83867 brcmfmac mwifiex_pcie mlx4_core micrel team macvlan bnx2 virtio_net rtl_pci zaurus hns_mdi libcxgb hv_netvsc nicvf mt76x0u teranetics mlxfw cdc_eem qcom-emac pppox mt76-usb sierra_net i40evf bcm87xx mwifiex pegasus rt2x00mmi sja1000 ena hclgevf cnic cxgb4vf ppp_synctty iwlmvm team_mode_broadcast vxlan vsockmon hdlc_cisc rtl8723-common bsd_comp fakelb dp83822 dp83tc811 cicada fm10 8139t sfc hs geneve hclge xgene-enet-v2 cdc_mbim hdlc asix netdevsim rt2800pci team_mode_random lxt ems_usb mlxsw_pci sr9700 mdio-thunder mlxsw_switchib macvtap atlantic cdc_ether mcs7830 nicpf mdi peak_pci atl1e cdc_subset ipvtap btcoexist mt76x0-common veth slip iwldvm bcm7xxx vitesse netconsole epic100 myri10ge r8169 qede microchip_t1 liquidi bnx2x brcmutil mwifiex_sdi mlx5_core rtlwifi vmxnet3 nlmon hns3 hdlc_raw esd_usb2 atl2 mt76x2-common iwlwifi mdio-bcm-unimac national ath rtwpci rtw88 nfp rtl8821ae fjes thunderbolt-net 8139cp atl1 mscc vcan dp83848 dp83640 hdlc_fr e1000e ipheth net_failover aquantia rtl8192ee igbvf rocker intel-xway tg3" --omit "ramdisk network ifcfg qemu-net" --install "chmod" --nofscks` - -## Setting the Disk Scheduling Algorithm - -This section describes how to set the disk scheduling algorithm. - -### Temporarily Modifying the Scheduling Policy - -For example, if all I/O scheduling algorithms are changed to **mq-deadline**, the modification becomes invalid after the system is restarted. - -```shell -echo mq-deadline > /sys/block/sd*/queue/scheduler -``` - -### Permanently Setting the Scheduling Policy - -You can add **elevator=mq-deadline** to the kernel line in the kernel boot configuration file **grub.cfg**. The setting takes effect after the system is restarted. - -```shell -linux /vmlinuz-4.19.90-2003.4.0.0036.oe1.x86_64 root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap quiet crashkernel=512M elevator=mq-deadline -``` diff --git a/docs/en/docs/Administration/configuring-services.md b/docs/en/docs/Administration/configuring-services.md deleted file mode 100644 index 35ffda1bb6f4f8a2eae7527a03101872449a82fc..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/configuring-services.md +++ /dev/null @@ -1,4 +0,0 @@ -# Configuring Services - - - diff --git a/docs/en/docs/Administration/configuring-the-ftp-server.md b/docs/en/docs/Administration/configuring-the-ftp-server.md deleted file mode 100644 index ebc2ec1def16cc75c2c567ffdff23e474fc3539c..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/configuring-the-ftp-server.md +++ /dev/null @@ -1,503 +0,0 @@ -# Configuring the FTP Server - -## General Introduction - -### FTP Overview - -File Transfer Protocol \(FTP\) is one of the earliest transmission protocols on the Internet. It is used to transfer files between the server and client. FTP allows users to access files on a remote system using a set of standard commands without logging in to the remote system. In addition, the FTP server provides the following functions: - -- Subscriber classification - - By default, the FTP server classifies users into real users, guest users, and anonymous users based on the login status. The three types of users have different access permissions. Real users have complete access permissions, while anonymous users have only the permission to downloading resources. - -- Command records and log file records - - FTP can use the syslogd to record data, including historical commands and user transmission data \(such as the transmission time and file size\). Users can obtain log information from the /var/log/ directory. - -- Restricting the access scope of users - - FTP can limit the work scope of a user to the home directory of the user. After a user logs in to the system through FTP, the root directory displayed by the system is the home directory of the user. This environment is called change root \(chroot for short\). In this way, users can access only the main directory, but not important directories such as /etc, /home, and /usr/local. This protects the system and keeps the system secure. - -### Port Used by the FTP Server - -The FTP service requires multiple network ports. The server uses the following ports: - -- Command channel. The default port number is 21. -- Data channel. The default port number is 20. - -Port 21 is used to receive connection requests from the FTP client, and port 20 is used by the FTP server to proactively connect to the FTP client. - -### Introduction to vsftpd - -FTP has a long history and uses the unencrypted transmission mode, and is therefore considered insecure. This section describes the Very Secure FTP Daemon \(vsftpd\), to use FTP in a more secure way. - -The vsftpd is introduced to build a security-centric FTP server. The vsftpd is designed with the following features: - -- The startup user of the vsftpd service is a common user who has low system permission. In addition, the vsftpd service uses chroot to change the root directory, preventing the risk of misusing system tools. -- Any vsftpd command that requires high execution permission is controlled by a special upper-layer program. The upper-layer program has low permission and does not affect the system. -- vsftpd integrates most of the extra commands \(such as dir, ls, and cd\) used by FTP. Generally, the system does not need to provide extra commands, which are secure for the system. - -## Using vsftpd - -### Installing vsftpd - -To use the vsftpd service, you need to install the vsftpd software. If the yum source has been configured, run the following command as the root user to install the vsftpd service: - -```shell -dnf install vsftpd -``` - -### Service Management - -To start, stop, or restart the vsftpd service, run the corresponding command as the root user. - -- Starting vsftpd services - - ```shell - systemctl start vsftpd - ``` - - You can run the netstat command to check whether communication port 21 is enabled. If the following information is displayed, the vsftpd service has been enabled. - - ```shell - $ netstat -tulnp | grep 21 - tcp6 0 0 :::21 :::* LISTEN 19716/vsftpd - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >If the **netstat** command does not exist, run the **dnf install net-tools** command to install the **net-tools** software and then run the **netstat** command. - -- Stopping the vsftpd services - - ```shell - systemctl stop vsftpd - ``` - -- Restarting the vsftpd service - - ```shell - systemctl restart vsftpd - ``` - -## Configuring vsftpd - -### vsftpd Configuration Files - -You can modify the vsftpd configuration file to control user permissions. [Table 1](#table1541615718372) describes the vsftpd configuration files. You can modify the configuration files as required. You can run the man command to view more parameter meanings. - -**Table 1** vsftpd configuration files - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Configuration File

-

Description

-

/etc/vsftpd/vsftpd.conf

-

Main configuration file of the vsftpd process. The configuration format is Parameter=Parameter value. The parameter and parameter value cannot be empty.

-

You can run the following command to view details about the vsftpd.conf file:

-

man 5 vsftpd.conf

-

/etc/pam.d/vsftpd

-

Pluggable authentication modules (PAMs) are used for identity authentication and restrict some user operations.

-

/etc/vsftpd/ftpusers

-

List of users who are not allowed to use the vsftpd. By default, the system account is also in this file. Therefore, the system account cannot use vsftpd by default.

-

/etc/vsftpd/user_list

-

List of users who are allowed or not allowed to log in to the vsftpd server. Whether the file takes effect depends on the following parameters in the main configuration file vsftpd.conf:

-

userlist_enable: indicates whether to enable the userlist mechanism. The value YES indicates that the userlist mechanism is enabled. In this case, the userlist_deny configuration is valid. The value NO indicates that the userlist mechanism is disabled.

-

userlist_deny: indicates whether to forbid users in the user list to log in. YES indicates that users in the user list are forbidden to log in. NO indicates that users in the command are allowed to log in.

-

For example, if userlist_enable is set to YES and userlist_deny is set to YES, all users in the user list cannot log in.

-

/etc/vsftpd/chroot_list

-

Whether to restrict the user list in the home directory. By default, this file does not exist. You need to create it manually. It is the value of chroot_list_file in the vsftpd.conf file.

-

The function of this parameter is determined by the following parameters in the vsftpd.conf file:

-
  • chroot_local_user: indicates whether to restrict all users to the home directory. The value YES indicates that all users are restricted to the home directory, and the value NO indicates that all users are not restricted to the home directory.
  • chroot_list_enable: indicates whether to enable the list of restricted users. The value YES indicates that the list is enabled, and the value NO indicates that the list is disabled.
-

For example, if chroot_local_user is set to YES, chroot_list_enable is set to YES, and chroot_list_file is set to /etc/vsftpd/chroot_list, all users are restricted to their home directories, and users in chroot_list are not restricted.

-

/usr/sbin/vsftpd

-

Unique execution file of vsftpd.

-

/var/ftp/

-

Default root directory for anonymous users to log in. The root directory is related to the home directory of the ftp user.

-
- -### Default Configuration Description - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The configuration content in this document is for reference only. You can modify the content based on the site requirements \(for example, security hardening requirements\). - -In the openEuler system, vsftpd does not open to anonymous users by default. Run the vim command to view the main configuration file. The content is as follows: - -```shell -$ vim /etc/vsftpd/vsftpd.conf -anonymous_enable=NO -local_enable=YES -write_enable=YES -local_umask=022 -dirmessage_enable=YES -xferlog_enable=YES -connect_from_port_20=YES -xferlog_std_format=YES -listen=NO -listen_ipv6=YES -pam_service_name=vsftpd -userlist_enable=YES -``` - -[Table 2](#table18185162512499) describes the parameters. - -**Table 2** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

anonymous_enable

-

Indicates whether to allow anonymous users to log in. YES indicates that anonymous users are allowed to log in; NO indicates that anonymous users are not allowed to log in.

-

local_enable

-

Whether to allow local users to log in. YES indicates that local users are allowed to log in. NO indicates that local users are not allowed to log in.

-

write_enable

-

Whether to allow the login user to have the write permission. YES indicates that the upload and write function is enabled, and NO indicates that the function is disabled.

-

local_umask

-

Indicates the umask value when a local user adds a profile.

-

dirmessage_enable

-

Indicates whether to display the contents that users need to pay attention to when a user accesses a directory. The options are YES (yes) and NO (no).

-

xferlog_enable

-

Indicates whether to record file upload and download operations. The options are YES (record operations) and NO (not record operations).

-

connect_from_port_20

-

Indicates whether port 20 is used for data transmission in port mode. YES indicates that port 20 is used, and NO indicates that port 20 is not used.

-

xferlog_std_format

-

Indicates whether the transfer log file is written in the standard xferlog format. The options are YES (yes) and NO (no).

-

listen

-

Indicates whether the vsftpd service is started in standalone mode. The options are YES (yes) and NO (no).

-

pam_service_name

-

Support for PAM management. The value is a service name, for example, vsftpd.

-

userlist_enable

-

Indicates whether to support account login control in the /etc/vsftpd/user_list file. The options are YES (yes) and NO (no).

-

tcp_wrappers

-

Indicates whether to support the firewall mechanism of the TCP Wrappers. The options are YES (yes) and NO (no).

-

listen_ipv6

-

Indicates whether to listen to IPv6 FTP requests. The options are YES (yes) and NO (no). listen and listen_ipv6 cannot be enabled at the same time.

-
- -### Setting the Local Time - -#### Overview - -In the openEuler system, vsftpd uses the Greenwich Mean Time \(GMT\) time by default, which may be different from the local time. For example, the GMT time is 8 hours later than the Beijing time. You need to change the GMT time to the local time. Otherwise, the server time and client time are inconsistent, which may cause errors during file upload and download. - -#### Setting Method - -To set the vsftpd time to the local time, perform the following steps as the **root** user: - -1. Open the vsftpd.conf file and change the value of use\_localtime to **YES**. Run the following command: - - ```shell - vim /etc/vsftpd/vsftpd.conf - ``` - - Modify the file contents as follows: - - ```shell - use_localtime=YES - ``` - -2. Restart the vsftpd service. - - ```shell - systemctl restart vsftpd - ``` - -3. Set the vsftpd service to start automatically upon power-on. - - ```shell - systemctl enable vsftpd - ``` - -### Configuring Welcome Information - -You are advised to configure a welcome information file for the vsftpd service. To configure the **welcome.txt** file of the vsftp service, perform the following steps as the **root** user: - -1. Open the vsftpd.conf configuration file, add the welcome information to the file, save the file, and exit. - - ```shell - vim /etc/vsftpd/vsftpd.conf - ``` - - The following configuration lines need to be added: - - ```text - banner_file=/etc/vsftpd/welcome.txt - ``` - -2. Create welcome information. Specifically, open the welcome.txt file, write the welcome information, save the file, and exit. - - ```shell - vim /etc/vsftpd/welcome.txt - ``` - - The following is an example: - - ```text - Welcome to this FTP server! - ``` - -### Configuring the Login Permission of a System Account - -Generally, users need to restrict the login permission of some accounts. You can set the restriction as required. - -By default, vsftpd manages and restricts user identities based on user lists stored in two files. FTP requests from a user in any of the files will be denied. - -- **/etc/vsftpd/user_list** can be used as an allowlist, blocklist, or invalid list, which is determined by the **userlist_enable** and **userlist_deny** parameters. -- **/etc/vsftpd/ftpusers** can be used as a blocklist only, regardless of the parameters. - -## Verifying Whether the FTP Service Is Successfully Set Up - -You can use the FTP client provided by openEuler for verification. The command and output are as follows. Enter the user name \(an existing user in the system\) and password as prompted. If the message "Login successful" is displayed, the FTP server is successfully set up. - -```shell -$ ftp localhost -Trying 127.0.0.1... -Connected to localhost (127.0.0.1). -220-Welcome to this FTP server! -220 -Name (localhost:root): USERNAME -331 Please specify the password. -Password: -230 Login successful. -Remote system type is UNIX. -Using binary mode to transfer files. -ftp> bye -221 Goodbye. -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If the **ftp** command does not exist, run the **dnf install ftp** command as the **root** user to install the **ftp** software and then run the **ftp** command. - -## Configuring a Firewall - -To open the FTP service to the Internet, you need to configure the firewall and SElinux as the **root** user. - -```shell -$ firewall-cmd --add-service=ftp --permanent -success -$ firewall-cmd --reload -success -$ setsebool -P ftpd_full_access on -``` - -## File Transmission - -### Overview - -This section describes how to transfer files after the vsftpd service is started. - -### Connecting to the Server - -#### Command Format - -**ftp** \[_hostname_ | _ip-address_\] - -**hostname** indicates the name of the server, and **ip-address** indicates the IP address of the server. - -#### Requirements - -Run the following command on the command-line interface \(CLI\) of the openEuler OS: - -```shell -ftp ip-address -``` - -Enter the user name and password as prompted. If the following information is displayed after the authentication is successful, the FTP connection is successful. In this case, you have accessed the directory of the connected server. - -```shell -ftp> -``` - -At this prompt, you can enter different commands to perform related operations. - -- Display the current path of the server. - - ```shell - ftp>pwd - ``` - -- Display the local path. You can upload the files in this path to the corresponding location on the FTP server. - - ```shell - ftp>lcd - ``` - -- Exit the current window and return to the local Linux terminal. - - ```shell - ftp>! - ``` - -### Downloading a File - -Generally, the get or mget command is used to download files. - -#### How to Use get - -- Function description: Transfers files from a remote host to a local host. -- Command format: **get** \[_remote-file_\] \[_local-file_\] - - _remote-file_ indicates a remote file, and _local-file_ indicates a local file. - -- For example, run the following command to obtain the /home/openEuler/openEuler.htm file on the remote server to the local directory /home/myopenEuler/ and change the file name to myopenEuler.htm - - ```shell - ftp> get /home/openEuler/openEuler.htm /home/myopenEuler/myopenEuler.htm - ``` - -#### How to Use mget - -- Function description: Receives a batch of files from the remote host to the local host. -- Command format: **mget** \[_remote-file_\] - - _remote-file_ indicates a remote file. - -- For example, to obtain all files in the /home/openEuler/ directory on the server, run the following command: - - ```shell - ftp> cd /home/openEuler/ - ftp> mget *.* - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >- In this case, a message is displayed each time a file is downloaded. To block the prompt information, run the **prompt off** command before running the **mget \*.\*** command. - >- The files are downloaded to the current directory on the Linux host. For example, if you run the ftp command in /home/myopenEuler/, all files are downloaded to /home/myopenEuler/. - -### Uploading a File - -Generally, the put or mput command is used to upload files. - -#### How to Use put - -- Function: Transfers a local file to a remote host. -- Command format: **put** \[_local-file_\] \[_remote-file_\] - - _remote-file_ indicates a remote file, and _local-file_ indicates a local file. - -- For example, run the following command to transfer the local myopenEuler.htm file to the remote host /home/openEuler/ and change the file name to openEuler.htm: - - ```shell - ftp> put myopenEuler.htm /home/openEuler/openEuler.htm - ``` - -#### How to Use mput - -- Function: Transfers a batch of files from the local host to a remote host. -- Command format: **mput** \[_local-file_\] - - _local-file_ indicates a local file. - -- For example, run the following command to upload all HTM files in the local directory to the /home/openEuler/ directory on the server: - - ```shell - ftp> cd /home/openEuler/ - ftp> mput *.htm - ``` - -### Deleting a File - -Generally, the **delete** or **mdelete** command is used to delete a file. - -#### How to Use delete - -- Function description: Deletes one or more files from the remote server. -- Command format: **delete** \[_remote-file_\] - - _remote-file_ indicates a remote file. - -- For example, to delete the /home/openEuler/openEuler.htm from the remote server, run the following command: - - ```shell - ftp> cd /home/openEuler/ - ftp> delete openEuler.htm - ``` - -#### How to Use mdelete - -- Function description: Deletes files from a remote server. This function is used to delete files in batches. -- Command format: **mdelete** \[_remote-file_\] - - _remote-file_ indicates a remote file. - -- For example, to delete all files whose names start with **a** from the /home/openEuler/ directory on the remote server, run the following command: - - ```shell - ftp> cd /home/openEuler/ - ftp> mdelete a* - ``` - -### Disconnecting from the Server - -Run the bye command to disconnect from the server. - -```shell -ftp> bye -``` diff --git a/docs/en/docs/Administration/configuring-the-network.md b/docs/en/docs/Administration/configuring-the-network.md deleted file mode 100644 index 5fe150f1b54293fede7db9a575787bd71e379326..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/configuring-the-network.md +++ /dev/null @@ -1,1351 +0,0 @@ -# Configuring the Network - - -- [Configuring the Network](#configuring-the-network) - - [Configuring an IP Address](#configuring-an-ip-address) - - [Using the nmcli Command](#using-the-nmcli-command) - - [Using the ip Command](#using-the-ip-command) - - [Configuring the Network Through the ifcfg File](#configuring-the-network-through-the-ifcfg-file) - - [Configuring a Host Name](#configuring-a-host-name) - - [Introduction](#introduction) - - [Configuring a Host Name by Running the hostnamectl Command](#configuring-a-host-name-by-running-the-hostnamectl-command) - - [Configuring a Host Name by Running the nmcli Command](#configuring-a-host-name-by-running-the-nmcli-command) - - [Configuring Network Bonding](#configuring-network-bonding) - - [Running the nmcli Command](#running-the-nmcli-command) - - [Configuring Network Bonding by Using a Command Line](#configuring-network-bonding-by-using-a-command-line) - - [IPv6 Differences \(vs IPv4\)](#ipv6-differences-vs-ipv4) - - [Restrictions](#restrictions) - - [Configuration Description](#configuration-description) - - [FAQs](#faqs) - - - -## Configuring an IP Address - -### Using the nmcli Command - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The network configuration configured by running the **nmcli** command takes effect immediately and will not be lost after the system restarts. - -#### Introduction to nmcli - -**nmcli** \(NetworkManager Command Line Interface\) is the command-line utility to configure networking through NetworkManager. The basic format of using **nmcli** is as follows: - -```shell -nmcli [OPTIONS] OBJECT { COMMAND | help } -``` - -In the preceding command, **OBJECT** can be one of the following options: **general**, **networking**, **radio**, **connection**, and **device**. **OPTIONS** can be optional options, such as **-t**, **\-\-terse** \(for script processing\),**-p**, **\-\-pretty** \(for human-readable output\), **-h**, and **\-\-help**. For more information, run the **nmcli help** command. - -```shell -nmcli help -``` - -Common commands are listed as follows: - -- To display the general status of NetworkManager, run the following command: - - ```shell - nmcli general status - ``` - -- To display all connections, run the following command: - - ```shell - nmcli connection show - ``` - -- To display the current active connections only, add the **-a** or **\-\-active** option as follows: - - ```shell - nmcli connection show --active - ``` - -- To display the device identified by NetworkManager and its connection status, run the following command: - - ```shell - nmcli device status - ``` - -- To start or stop network interfaces, for example, run the nmcli commands as the **root** user: - - ```shell - nmcli connection up id enp3s0 - nmcli device disconnect enp3s0 - ``` - -#### Device Management - -##### Connecting to a Device - -Run the following command to connect NetworkManager to the corresponding network device. Try to find the proper connection configuration and activate it. - - ```shell - nmcli device connect "$IFNAME" - ``` - -> If the corresponding connection configuration does not exist, NetworkManager creates and activates a configuration file with default settings. - -##### Disconnecting to a Device - -Run the following command to disconnect NetworkManager with the network device and prevent the device from being automatically activated. - - ```shell - nmcli device disconnect "$IFNAME" - ``` - -#### Setting Network Connections - -Run the following command to display all the available network connections: - -```shell -$ nmcli con show - - -NAME UUID TYPE DEVICE -enp4s0 5afce939-400e-42fd-91ee-55ff5b65deab ethernet enp4s0 -enp3s0 c88d7b69-f529-35ca-81ab-aa729ac542fd ethernet enp3s0 -virbr0 ba552da6-f014-49e3-91fa-ec9c388864fa bridge virbr0 -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->In the command output, **NAME** indicates the connection ID \(name\). - -After a network connection is added, the corresponding configuration file is generated and associated with the corresponding device. To check for available devices, run the following command: - -```shell -$ nmcli dev status - -DEVICE TYPE STATE CONNECTION -enp3s0 ethernet connected enp3s0 -enp4s0 ethernet connected enp4s0 -virbr0 bridge connected virbr0 -lo loopback unmanaged -- -virbr0-nic tun unmanaged -- -``` - -##### Configuring Dynamic IP Connections - -###### Configuring IP Addresses - -When DHCP is used to allocate a network, run the following command to add a network configuration file: - -```shell -nmcli connection add type ethernet con-name connection-name ifname interface-name -``` - -For example, to create a dynamic connection configuration file named **net-test**, run the following command as the **root** user: - -```shell -$ nmcli connection add type ethernet con-name net-test ifname enp3s0 -Connection 'net-test' (a771baa0-5064-4296-ac40-5dc8973967ab) successfully added. -``` - -The NetworkManager sets **connection.autoconnect** to **yes** and saves the setting to the **/etc/sysconfig/network-scripts/ifcfg-net-test** file. In the **/etc/sysconfig/network-scripts/ifcfg-net-test** file, **ONBOOT** is set to **yes**. - -###### Activating a Connection and Checking Device Connection Status - -Run the following command as the **root** user to activate a network connection: - -```shell -$ nmcli con up net-test -Connection successfully activated (D-Bus active path:/org/freedesktop/NetworkManager/ActiveConnection/5) -``` - -Run the following command to check the connection status of devices: - -```shell -$ nmcli device status - -DEVICE TYPE STATE CONNECTION -enp4s0 ethernet connected enp4s0 -enp3s0 ethernet connected net-test -virbr0 bridge connected virbr0 -lo loopback unmanaged -- -virbr0-nic tun unmanaged -- -``` - -##### Configuring Static IP Connections - -###### Configuring IP Addresses - -To add a static IPv4 network connection, run the following command: - -```shell -nmcli connection add type ethernet con-name connection-name ifname interface-name ip4 address gw4 address -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->To add an IPv6 address and related gateway information, use the **ip6** and **gw6** options. - -For example, to create a static connection configuration file named **net-static**, run the following command as the **root** user: - -```shell -nmcli con add type ethernet con-name net-static ifname enp3s0 ip4 192.168.0.10/24 gw4 192.168.0.254 -``` - -You can also specify the IPv6 address and gateway for the device. The following is an example: - -```shell -$ nmcli con add type ethernet con-name test-lab ifname enp3s0 ip4 192.168.0.10/24 gw4 192.168.0.254 ip6 abbe::**** gw6 2001:***::* -Connection 'net-static' (63aa2036-8665-f54d-9a92-c3035bad03f7) successfully added. -``` - -The NetworkManager sets the internal parameter **ipv4.method** to **manual**, **connection.autoconnect** to **yes**, and writes the setting to the **/etc/sysconfig/network-scripts/ifcfg-my-office** file. In the file, **BOOTPROTO** is set to **none**, and **ONBOOT** is set to **yes**. - -Run the following command as the **root** user to set IPv4 addresses of two DNS servers: - -```shell -nmcli con mod net-static ipv4.dns "*.*.*.* *.*.*.*" -``` - -Run the following command as the **root** user to set IPv6 addresses of two DNS servers: - -```shell -nmcli con mod net-static ipv6.dns "2001:4860:4860::**** 2001:4860:4860::****" -``` - -###### Activating a Connection and Checking Device Connection Status - -Run the following command as the **root** user to activate a network connection: - -```shell -$ nmcli con up net-static ifname enp3s0 -Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6) -``` - -Run the following command to check the connection status of devices: - -```shell -$ nmcli device status - -DEVICE TYPE STATE CONNECTION -enp4s0 ethernet connected enp4s0 -enp3s0 ethernet connected net-static -virbr0 bridge connected virbr0 -lo loopback unmanaged -- -virbr0-nic tun unmanaged -- -``` - -Run the following command to view the connection details \(with the **-p** and **\-\-pretty** options to add the title and segment to the output\): - -```shell -$ nmcli -p con show net-static -=============================================================================== -Connection profile details (net-static ) -=============================================================================== -connection.id: net-static -connection.uuid: b9f18801-6084-4aee-af28-c8f0598ff5e1 -connection.stable-id: -- -connection.type: 802-3-ethernet -connection.interface-name: enp3s0 -connection.autoconnect: yes -connection.autoconnect-priority: 0 -connection.autoconnect-retries: -1 (default) -connection.multi-connect: 0 (default) -connection.auth-retries: -1 -connection.timestamp: 1578988781 -connection.read-only: no -connection.permissions: -- -connection.zone: -- -connection.master: -- -connection.slave-type: -- -connection.autoconnect-slaves: -1 (default) -connection.secondaries: -- -connection.gateway-ping-timeout: 0 -connection.metered: unknown -connection.lldp: default -connection.mdns: -1 (default) -connection.llmnr: -1 (default) -``` - -##### Adding a Wi-Fi Connection - -You can add the Wi-Fi connection using either of the following methods: - -**Method 1: Connect to the Wi-Fi network using a network port.** - -Connect to the Wi-Fi network specified by the SSID or BSSID. Run the following command to find a matching connection or create a connection, and then activate the connection on the device. - -```shell -nmcli device wifi connect "$SSID" password "$PASSWORD" ifname "$IFNAME" -nmcli --ask device wifi connect "$SSID" -``` - -**Method 2: Connect to the Wi-Fi network using the configuration file.** - -1，Run the following command to check for available Wi-Fi access points: - -```shell -nmcli dev wifi list -``` - -2，Run the following command to generate a static IP address configuration that allows Wi-Fi connections automatically allocated by the DNS: - -```shell -nmcli con add con-name Wifi ifname wlan0 type wifi ssid MyWifi ip4 192.168.100.101/24 gw4 192.168.100.1 -``` - -3，Run the following command to set a WPA2 password, for example, **answer**: - -```shell -nmcli con modify Wifi wifi-sec.key-mgmt wpa-psk -nmcli con modify Wifi wifi-sec.psk answer -``` - -4，Run the following command to change the Wi-Fi status: - -```shell -nmcli radio wifi [ on | off ] -``` - -##### Modifying Attributes - -Run the following command to check a specific attribute, for example, mtu: - -```shell -$ nmcli connection show id 'Wifi ' | grep mtu -802-11-wireless.mtu: auto -``` - -Run the following command to modify the attribute: - -```shell -nmcli connection modify id 'Wifi ' 802-11-wireless.mtu 1350 -``` - -Run the following command to confirm the modification: - -```shell -$ nmcli connection show id 'Wifi ' | grep mtu -802-11-wireless.mtu: 1350 -``` - -#### Configuring a Static Route - -- Run the nmcli command to configure a static route for a network connection: - - ```shell - nmcli connection modify enp3s0 +ipv4.routes "192.168.122.0/24 10.10.10.1" - ``` - -- Run the following command to configure the static route using the editor: - - ```shell - $ nmcli con edit type ethernet con-name enp3s0 - ===| nmcli interactive connection editor |=== - Adding a new '802-3-ethernet' connection - Type 'help' or '?' for available commands. - Type 'describe [.]' for detailed property description. - You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, ipv4, ipv6, dcb - nmcli> set ipv4.routes 192.168.122.0/24 10.10.10.1 - nmcli> - nmcli> save persistent - Saving the connection with 'autoconnect=yes'. That might result in an immediate activation of the connection. - Do you still want to save? [yes] yes - Connection 'enp3s0' (1464ddb4-102a-4e79-874a-0a42e15cc3c0) successfully saved. - nmcli> quit - ``` - -### Using the ip Command - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The network configuration configured using the **ip** command takes effect immediately, but the configuration will be lost after the system restarts. - -#### Configuring IP Addresses - -Run the **ip** command to configure an IP address for the interface. The command format is as follows, where _interface-name_ indicates the NIC name. - -```shell -ip addr [ add | del ] address dev interface-name -``` - -##### Configuring a Static IP Address - -Run the following command as the **root** user to configure an IP address: - -```shell -ip address add 192.168.0.10/24 dev enp3s0 -``` - -Run the following command as the **root** user to view the configuration result: - -```shell -$ ip addr show dev enp3s0 -2: enp3s0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 - link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff - inet 192.168.202.248/16 brd 192.168.255.255 scope global dynamic noprefixroute enp3s0 - valid_lft 9547sec preferred_lft 9547sec - inet 192.168.0.10/24 scope global enp3s0 - valid_lft forever preferred_lft forever - inet6 fe80::32e8:cc22:9db2:f4d4/64 scope link noprefixroute - valid_lft forever preferred_lft forever -``` - -##### Configuring Multiple IP Addresses - -The **ip** command can be used to assign multiple IP addresses to an interface. You can run the **ip** command multiple times as the **root** user to assign IP addresses to an interface. The following is an example: - -```shell -$ ip address add 192.168.2.223/24 dev enp4s0 -$ ip address add 192.168.4.223/24 dev enp4s0 -$ ip addr - -3: enp4s0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 - link/ether 52:54:00:aa:da:e2 brd ff:ff:ff:ff:ff:ff - inet 192.168.203.12/16 brd 192.168.255.255 scope global dynamic noprefixroute enp4s0 - valid_lft 8389sec preferred_lft 8389sec - inet 192.168.2.223/24 scope global enp4s0 - valid_lft forever preferred_lft forever - inet 192.168.4.223/24 scope global enp4s0 - valid_lft forever preferred_lft forever - inet6 fe80::1eef:5e24:4b67:f07f/64 scope link noprefixroute - valid_lft forever preferred_lft forever -``` - -#### Configuring a Static Route - -To add a static route to the routing table, run the **ip route add** command. To delete a route, run the **ip route del** command. The following shows the common format of the **ip route** command: - -```shell -ip route [ add | del | change | append | replace ] destination-address -``` - -To display the current IP routing table, run the **ip route** command as the **root** user. The following is an example: - -```shell -$ ip route - -default via 192.168.0.1 dev enp3s0 proto dhcp metric 100 -default via 192.168.0.1 dev enp4s0 proto dhcp metric 101 -192.168.0.0/16 dev enp3s0 proto kernel scope link src 192.168.202.248 metric 100 -192.168.0.0/16 dev enp4s0 proto kernel scope link src 192.168.203.12 metric 101 -192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown -``` - -To add a static route to the host address, run the following command as the **root** user: - -```shell -ip route add 192.168.2.1 via 10.0.0.1 [dev interface-name] -``` - -In the preceding command, **192.168.2.1** is the IP address in the dot-decimal notation, **10.0.0.1** is the next hop, and _interface-name_ is the exit interface for entering the next hop. - -To add a static route to the network, that is, an IP address that represents an IP address range, run the following command as the **root** user: - -```shell -ip route add 192.168.2.0/24 via 10.0.0.1 [dev interface-name] -``` - -In the preceding command, **192.168.2.1** is the IP address of the target network, _10.0.0.1_ is the network prefix, and _interface-name_ is the NIC name. - -### Configuring the Network Through the ifcfg File - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The network configured in the **ifcfg** file does not take effect immediately. After modifying the file (for example, **ifcfg-enp3s0**), you need to run the **nmcli con reload;nmcli con up enp3s0** command as the **root** user to reload the configuration file and activate the connection for the modification to take effect. - -#### Configuring a Static Network - -The following uses the **enp4s0** network interface as an example to describe how to configure a static network by modifying the **ifcfg** file as the **root** user. The **ifcfg-enp4s0** file is generated in the **/etc/sysconfig/network-scripts/** directory. Modify the following parameters in the file: - -```text -TYPE=Ethernet -PROXY_METHOD=none -BROWSER_ONLY=no -BOOTPROTO=none -IPADDR=192.168.0.10 -PREFIX=24 -DEFROUTE=yes -IPV4_FAILURE_FATAL=no -IPV6INIT=yes -IPV6_AUTOCONF=yes -IPV6_DEFROUTE=yes -IPV6_FAILURE_FATAL=no -IPV6_ADDR_GEN_MODE=stable-privacy -NAME=enp4s0static -UUID=08c3a30e-c5e2-4d7b-831f-26c3cdc29293 -DEVICE=enp4s0 -ONBOOT=yes -``` - -#### Configuring a Dynamic Network - -The following uses the **em1** network interface as an example to describe how to configure a dynamic network by modifying the **ifcfg** file. The **ifcfg-em1** file is generated in the **/etc/sysconfig/network-scripts/** directory. Modify the following parameters in the file: - -```text -DEVICE=em1 -BOOTPROTO=dhcp -ONBOOT=yes -``` - -To configure an interface to send different host names to the DHCP server, add the following content to the **ifcfg** file: - -```text -DHCP_HOSTNAME=hostname -``` - -To configure an interface to ignore the routes sent by the DHCP server to prevent network services from updating the /etc/resolv.conf file using the DNS server received from the DHCP server, add the following content to the **ifcfg** file: - -```text -PEERDNS=no -``` - -To configure an interface to use a specific DNS server, set the **PEERDNS** parameter to **no** and add the following content to the **ifcfg** file: - -```text -DNS1=ip-address -DNS2=ip-address -``` - -**ip-address** is the IP address of the DNS server. This allows the network service to update the **/etc/resolv.conf** file using the specified DNS server. - -#### Default Gateway Configuration - -When determining the default gateway, parse the **/etc/sysconfig/network** file and then the **ifcfg** file, and uses the value of **GATEWAY** that is read last as the default route in the routing table. - -In a dynamic network environment, when the NetworkManager is used to manage hosts, you are advised to set the default gateway to DHCP assignment. - -## Configuring a Host Name - -### Introduction - -There are three types of host names: **static**, **transient**, and **pretty**. - -- **static**: Static host name, which can be set by users and saved in the **/etc/hostname** file. -- **transient**: Dynamic host name, which is maintained by the kernel. The initial value is a static host name. The default value is **localhost**. The value can be changed when the DHCP or mDNS server is running. -- **pretty**: Flexible host name, which can be set in any form \(including special characters/blanks\). Static and transient host names are subject to the general domain name restrictions. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Static and transient host names can contain only letters \(a–z and A–Z\), digits \(0–9\), hyphens \(-\), and periods \(.\). The host names cannot start or end with a period \(.\) or contain two consecutive periods \(.\). The host name can contain a maximum of 64 characters. - -### Configuring a Host Name by Running the hostnamectl Command - -#### Viewing All Host Names - -Run the following command to view the current host name: - -```shell -hostnamectl status -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If no option is specified in the command, the **status** option is used by default. - -#### Setting All Host Names - -Run the following command as the **root** user to set all host names: - -```shell -hostnamectl set-hostname name -``` - -#### Setting a Specific Host Name - -Run the following command as the **root** user to set a specific host name: - -```shell -hostnamectl set-hostname name [option...] -``` - -The option may be one or more of **\-\-pretty**, **\-\-static**, and **\-\-transient**. - -If **\-\-static** or **\-\-transient** is used together with **\-\-pretty**, the host names of the **static** or **transient** type will be simplified to the host names of the **pretty** type with spaces replaced with hyphens \(-\) and special characters deleted. - -When setting a host name of the **pretty** type, use double quotation marks if the host name contains spaces or single quotation marks. An example is as follows: - -```shell -hostnamectl set-hostname "Stephen's notebook" --pretty -``` - -#### Clearing a Specific Host Name - -To clear a specific host name and restore it to the default format, run the following command as the **root** user: - -```shell -hostnamectl set-hostname "" [option...] -``` - -In the preceding command, **""** is a blank character string, and the _option_ may be one or more of **\-\-pretty**, **\-\-static**, and **\-\-transient**. - -#### Remotely Changing a Host Name - -To change the host name in a remote system, run the **hostnamectl** command as the **root** user with the **-H** or **\-\-host** option. - -```shell -hostnamectl set-hostname -H [username]@hostname new_hostname -``` - -In the preceding command, _hostname_ indicates the name of the remote host to be configured, _username_ indicates the user-defined name, and _new\_hostname_ indicates the new host name. **hostnamectl** is used to connect to the remote system through SSH. - -### Configuring a Host Name by Running the nmcli Command - -To query a static host name, run the following command: - -```shell -nmcli general hostname -``` - -To name a static host as **host-server**, run the following command as **root** user: - -```shell -nmcli general hostname host-server -``` - -To enable the system to detect the change of the static host name, run the following command as the **root** user to restart the hostnamed service: - -```shell -systemctl restart systemd-hostnamed -``` - -## Configuring Network Bonding - -### Running the nmcli Command - -- To create a bond named **mybond0**, run the following command: - - ```shell - nmcli con add type bond con-name mybond0 ifname mybond0 mode active-backup - ``` - -- To add a slave interface, run the following command: - - ```shell - nmcli con add type bond-slave ifname enp3s0 master mybond0 - ``` - - To add another slave interface, repeat the preceding command with the new interface name: - - ```shell - $ nmcli con add type bond-slave ifname enp4s0 master mybond0 - Connection 'bond-slave-enp4s0' (05e56afc-b953-41a9-b3f9-0791eb49f7d3) successfully added. - ``` - -- To enable a bond, run the following command to enable the slave interface first: - - ```shell - $ nmcli con up bond-slave-enp3s0 - Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/14) - ``` - - ```shell - $ nmcli con up bond-slave-enp4s0 - Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/15) - ``` - - Then, run the following command to enable the bond: - - ```shell - $ nmcli con up mybond0 - Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/16) - ``` - -### Configuring Network Bonding by Using a Command Line - -#### Checking Whether the Bonding Kernel Module Is Installed - -By default, the bonding kernel module is loaded. To load this module, run the following command as the **root** user: - -```shell -modprobe --first-time bonding -``` - -Run the following command as the **root** user to display the information about the module: - -```shell -modinfo bonding -``` - -For more commands, run the modprobe \-\-help command as the **root** user. - -#### Creating a Channel Bonding Interface - -To create a channel bonding interface, you can create a file named **ifcfg-bondN** in the **/etc/sysconfig/network-scripts/** directory as the **root** user \(replacing N with the actual interface number, for example, 0\). - -Write the corresponding content to the configuration file according to the type of the interface to be bonded, for example, network interface. An example of the interface configuration file is as follows: - -```text -DEVICE=bond0 -NAME=bond0 -TYPE=Bond -BONDING_MASTER=yes -IPADDR=192.168.1.1 -PREFIX=24 -ONBOOT=yes -BOOTPROTO=none -BONDING_OPTS="bonding parameters separated by spaces" -``` - -#### Creating a Slave Interface - -After creating a channel bonding interface, you must add the **MASTER** and **SLAVE** instructions to the configuration file of the slave interface. - -For example, to bind the two network interfaces enp3s0 and enp4s0 in channel mode, the configuration files are as follows: - -```text -TYPE=Ethernet -NAME=bond-slave-enp3s0 -UUID=3b7601d1-b373-4fdf-a996-9d267d1cac40 -DEVICE=enp3s0 -ONBOOT=yes -MASTER=bond0 -SLAVE=yes -``` - -```text -TYPE=Ethernet -NAME=bond-slave-enp4s0 -UUID=00f0482c-824f-478f-9479-abf947f01c4a -DEVICE=enp4s0 -ONBOOT=yes -MASTER=bond0 -SLAVE=yes -``` - -#### Activating Channel Bonding - -To activate channel bonding, you need to enable all the slave interfaces. Run the following command as the **root** user: - -```shell -$ ifup enp3s0 -Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7) -``` - -```shell -$ ifup enp4s0 -Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8) -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If an interface is in **up** state, run the **ifdown** _enp3s0_ command to change the state to **down**. In the command, _enp3s0_ indicates the actual NIC name. - -After that, enable all the slave interfaces to enable the bonding \(do not set them to **Down**\). - -To enable the NetworkManager to detect the modifications made by the system, run the following command as the **root** user after each modification: - -```shell -nmcli con load /etc/sysconfig/network-scripts/ifcfg-device -``` - -Run the following command as the **root** user to check the status of the bonded interface: - -```shell -$ ip link show - -1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 -2: enp3s0: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 - link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff -3: enp4s0: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 - link/ether 52:54:00:aa:da:e2 brd ff:ff:ff:ff:ff:ff -4: virbr0: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 - link/ether 86:a1:10:fb:ef:07 brd ff:ff:ff:ff:ff:ff -5: virbr0-nic: mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000 - link/ether 52:54:00:29:35:4c brd ff:ff:ff:ff:ff:ff -``` - -#### Creating Multiple Bondings - -The system creates a channel bonding interface for each bonding, including the **BONDING\_OPTS** instruction. This configuration method allows multiple bonded devices to use different configurations. Perform the following operations to create multiple channel bonding interfaces: - -- Create multiple **ifcfg-bondN** files that contain the **BONDING\_OPTS** instruction so that network scripts can create bonding interfaces as required. -- Create or edit the existing interface configuration file to be bonded, and add the **SLAVE** instruction. -- Use the MASTER instruction to assign the interface to be bonded, that is, the slave interface, to the channel bonding interface. - -The following is an example of the configuration file of a channel bonding interface: - -```text -DEVICE=bondN -NAME=bondN -TYPE=Bond -BONDING_MASTER=yes -IPADDR=192.168.1.1 -PREFIX=24 -ONBOOT=yes -BOOTPROTO=none -BONDING_OPTS="bonding parameters separated by spaces" -``` - -In this example, replace N with the number of the bonded interface. For example, to create two interfaces, you need to create two configuration files **ifcfg-bond0** and **ifcfg-bond1** with correct IP addresses. - -## IPv6 Differences \(vs IPv4\) - -### Restrictions - -- chrony supports global addresses but not link-local addresses. -- Firefox supports the access to the global address through HTTP or HTTPS, but does not support the access to the link-local address. - -### Configuration Description - -#### Setting the MTU of an Interface Device - -##### Overview - -In an IPv6 scenario, the minimum MTU value of the entire routing path is used as the PMTU value of the current link. The source end determines whether to fragment packets based on the PMTU value. Other devices on the entire path do not need to fragment packets. This reduces the load of intermediate routing devices. The minimum value of IPv6 PMTU is 1280. - -##### Setting the MTU of the Interface Device - -If the MTU of an interface configured with an IPv6 address is set to a value smaller than **1280** \(the minimum value of the IPv6 PMTU\), the IPv6 address of the interface will be deleted and cannot be added again. Therefore, in IPv6 scenarios, the MTU of the interface device must be greater than or equal to 1280. Run the following commands as the **root** user to view the details: - -```shell -$ ip addr show enp3s0 -3: enp3s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 - link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx - inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 - valid_lft 38663sec preferred_lft 38663sec - inet6 2001:222::2/64 scope global - valid_lft forever preferred_lft forever -``` - -```shell -$ ip link set dev enp3s0 mtu 1200 -$ ip addr show enp3s0 -3: enp3s0: mtu 1200 qdisc pfifo_fast state UP group default qlen 1000 - link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx - inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 - valid_lft 38642sec preferred_lft 38642sec -``` - -```shell -$ ip addr add 2001:222::2/64 dev enp3s0 -RTNETLINK answers: No buffer space available -``` - -```shell -$ ip link set dev enp3s0 mtu 1500 -$ ip addr show enp3s0 -3: enp3s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 - link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx - inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 - valid_lft 38538sec preferred_lft 38538sec -``` - -```shell -$ ip addr add 2001:222::2/64 dev enp3s0 -$ ip addr show enp3s0 -3: enp3s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 - link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx - inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 - valid_lft 38531sec preferred_lft 38531sec - inet6 2001:222::2/64 scope global - valid_lft forever preferred_lft forever -``` - -#### Stateful IPv6 Address Autoconfiguration - -##### Overview - -Both IPv6 and IPv4 addresses can be obtained through DHCP as the **root** user. There are configuration methods for IPv6 address: stateless autoconfiguration and stateful autoconfiguration. - -- Stateless autoconfiguration - - The DHCP server is not required for management. The device obtains the network prefix according to the router advertisement \(RA\), or the prefix of a link-local address is fixed to fe80::. The interface ID is automatically obtained based on the value of IPV6\_ADDR\_GEN\_MODE in the ifcfg file. - - 1. If the value of IPv6\_ADDR\_GEN\_MODE is stable-privacy, the device determines a random interface ID based on the device and network environment. - 2. If the value of IPv6\_ADDR\_GEN\_MODE is EUI64, the device determines the interface ID based on the device MAC address. - -- Stateful autoconfiguration: The DHCP server manages and leases IPv6 addresses from the DHCPv6 server base on the DHCPv6 protocol. - - In stateful autoconfiguration, the DHCPv6 server can classify clients based on the vendor class configured on the clients and assign IPv6 addresses in different address segments to different types of clients. In IPv4 scenarios, the client can use the -V option of the dhclient command to set the vendor-class-identifier field. The DHCP server classifies clients based on the vendor-class-identifier field in the configuration file. In IPv6 scenarios, if the same method is used to classify clients, the classification does not take effect. - - ```shell - dhclient -6 -V - ``` - - This is because DHCPv6 differs greatly from DHCP. The vendor-class-option in DHCPv6 replaces the vendor-class-identifier in DHCP. However, the -V option of dhclient cannot be set to vendor-class-option. - -##### Setting the vendor class for dhclient in Stateful IPv6 Address Autoconfiguration - -- On the client, add the setting of vendor class by using the configuration file. - - Client configuration file \(/etc/dhcp/dhclient6.conf\): The file location can be customized. You need to specify the configuration file using the dhclient -cf option. - - ```text - option dhcp6.vendor-class code 16 = {integer 32, integer 16, string}; - interface "enp3s0" { - send dhcp6.vendor-class ; - } - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- \: a 32-digit integer, indicating the enterprise ID. The enterprise is registered through the IANA. - >- \: a 16-digit integer, indicating the length of the vendor class string. - >- \: character string of the vendor class to be set, for example, HWHW. - - On the client: - - ```shell - dhclient -6 -cf /etc/dhcp/dhclient6.conf - ``` - -- The DHCPv6 server configuration file \(/etc/dhcp/dhcpd6.conf\) needs to be specified by the dhcpd -cf option. - - ```text - option dhcp6.vendor-class code 16 = {integer 32, integer 16, string}; - subnet6 fc00:4:12:ffff::/64 { - class "hw" { - match if substring ( option dhcp6.vendor-class, 6, 10 ) = "HWHW"; - } - pool6 { - allow members of "hw"; - range6 fc00:4:12:ffff::ff10 fc00:4:12:ffff::ff20; - } - pool6 { - allow unknown clients; - range6 fc00:4:12:ffff::100 fc00:4:12:ffff::120; - } - } - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >In substring \(option dhcp6.vendor-class, 6, 10\), the start position of the substring is 6, because the substring contains four bytes of and two bytes of . The end position of the substring is 6+. In this example, the vendor class string is HWHW, and the length of the string is 4. Therefore, the end position of the substring is 6 + 4 = 10. You can specify and as required. - - On the server: - - ```shell - dhcpd -6 -cf /etc/dhcp/dhcpd6.conf - ``` - -#### Kernel Supporting Socket-Related System Calls - -##### Overview - -The length of an IPv6 address is extended to 128 bits, indicating that there are sufficient IPv6 addresses for allocation. Compared with the IPv4 header, the IPv6 header is simplified, and the IPv6 automatic configuration function is enhanced. IPv6 addresses are classified into unicast addresses, multicast addresses, and anycast addresses. Common unicast addresses include link-local addresses, unique local addresses, and global addresses. As there are sufficient global IPv6 addresses, unique local addresses are not used. \(formerly known as site-local addresses, which were discarded in 2004.\) Currently, the mainstream unicast addresses are link-local address and global address. The current kernel supports socket system invoking. The link-local address and global address using unicast addresses are different. - -##### Differences Between the link-local Address and global Address During Socket Invoking - -RFC 2553: Basic Socket Interface Extensions for IPv6 defines the sockaddr\_in6 data structure as follows: - -```c -struct sockaddr_in6 { - uint8_t sin6_len; /* length of this struct */ - sa_family_t sin6_family; /* AF_INET6 */ - in_port_t sin6_port; /* transport layer port # */ - uint32_t sin6_flowinfo; /* IPv6 flow information */ - struct in6_addr sin6_addr; /* IPv6 address */ - uint32_t sin6_scope_id; /* set of interfaces for a scope */ -}; -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->sin6\_scope\_id: a 32-bit integer. For the link-local address, it identifies the index of the specified interface. For the link-range sin6\_addr, it identifies the index of the specified interface. For the site-range sin6\_addr, it is used as the site identifier \(the site-local address has been discarded\). - -When the link-local address is used for socket communication, the interface index corresponding to the address needs to be specified when the destination address is constructed. Generally, you can use the if\_nametoindex function to convert an interface name into an interface index number. Details are as follows: - -```c -int port = 1234; -int sk_fd; -int iff_index = 0; -char iff_name[100] = "enp3s0"; -char * ll_addr[100] = "fe80::123:456:789"; -struct sockaddr_in6 server_addr; - -memset(&server_addr,0,sizeof(structsockaddr_in6)); -iff_index=if_nametoindex(iff_name); - -server_addr.sin6_family=AF_INET6; -server_addr.sin6_port=htons(port); -server_addr.sin6_scope_id=iff_index; -inet_pton(AF_INET6, ll_addr, &(server_addr.sin6_addr)); - -sk_fd=socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP); -connect(sk_fd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr_in6)); -``` - -#### Persistency Configuration of the IPv4 dhclient Daemon Process - -##### Overview - -When the NetworkManager service is used to manage network services, if the ifcfg- configuration file of an interface is configured to obtain an IP address in DHCP mode, the NetworkManager service starts the dhclient daemon process to obtain an IP address from the DHCP server. - -The dhclient provides the -1 option to determine whether the dhclient process persistently attempts to request an IP address or exits after the request times out before receiving a response from the DHCP server. For the IPv4 dhclient daemon process, you can set PERSISTENT\_DHCLIENT in the ifcfg- configuration file to determine whether to set the persistence of the IPv4 dhclient process. - -##### Restrictions - -1. If the ongoing dhclient process is killed, the network service cannot automatically start it. Therefore, you need to ensure the reliability. -2. If PERSISTENT\_DHCLIENT is configured, ensure that the corresponding DHCP server exists. If no DHCP server is available when the network service is started and the dhclient process continuously attempts to send request packets but does not receive any response, the network service is suspended until the network service times out. The network service starts the IPv4 dhclient processes of multiple NICs in serial mode. If persistency is configured for a NIC but the DHCP server is not ready, the network service will be suspended when obtaining an IPv4 address for the NIC. As a result, the NIC cannot obtain an IPv4 or IPv6 address. - -The preceding restrictions apply to special scenarios. You need to ensure reliability. - -##### Configuration Differences Between IPv4 DHCP and IPv6 DHCPv6 - -You can configure the ifcfg- parameter on an interface to enable IPv4 and IPv6 to dynamically obtain IP addresses using DHCP or DHCPv6. The configuration is as follows: - -```text -BOOTPROTO=none|bootp|dhcp -DHCPV6C=yes|no -PERSISTENT_DHCLIENT=yes|no|1|0 -``` - -- BOOTPROTO: **none** indicates that an IPv4 address is statically configured. bootp|dhcp enables DHCP dhclient to dynamically obtain an IPv4 address. -- DHCPV6C: **no** indicates that an IPv6 address is statically configured, and **yes** indicates that the DHCPv6 dhclient is enabled to dynamically obtain the IPv6 address. -- PERSISTENT\_DHCLIENT: **no|0** indicates that the IPv4 dhclient process is configured as nonpersistent. If the dhclient sends a request packet to the DHCP server but does not receive any response, the dhclient exits after a period of time and the exit value is 2. **yes|1** indicates that the IPv4 dhclient process is configured to be persistent. The dhclient process repeatedly sends request packets to the DHCP server. **If PERSISTENT\_DHCLIENT is not configured, dhclient of IPv4 is set to yes|1 by default.** - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The PERSISTENT\_DHCLIENT configuration takes effect only for IPv4 and does not take effect for IPv6-related dhclient -6 processes. By default, the persistence configuration is not performed for IPv6. - -#### Differences Between IPv4 and IPv6 Configuration Using the iproute Command - -##### Overview - -IPv4 and IPv6 are two different protocol standards. Therefore, the iproute commands are different in usage. This section describes the differences between IPv4 and IPv6 commands in the iproute package. - -To run the iproute commands, you must have the root permission. - -##### Lifecycle of an IPv6 Address - - - - - - - - - - - - - - - - - - - -

IPv6 status

-

Description

-

tentative

-

Temporary state: The newly added address is still in the DAD process.

-

preferred

-

Preferred state: The DAD process is complete, but no NA packet is received, indicating that the address does not conflict.

-

deprecated

-

Deprecated state: An address has a validity period (valid_lft or preferred_lft). After preferred_lft expires, the address changes to the deprecated state.

-

The address in this state cannot be used to create a new connection, but the original connection can still be used.

-

invalid

-

Invalid state: If the lease renewal fails after the preferred_lft time expires, the address status is set to invalid after the valid_lft time expires, indicating that the address cannot be used again.

-
- -Remarks: - -- preferred\_lft: preferred lifetime. The preferred\_lft address has not expired and can be used for normal communication. If there are multiple preferred addresses, the address is selected based on the kernel mechanism. -- valid\_lft: valid lifetime. The address cannot be used for creating new connections within the period of \[preferred\_lft, valid\_lft\]. The existing connections are still valid. - -##### Command ip link - -The commands are as follows: - -```shell -ip link set IFNAME mtu MTU -``` - -The minimum PMTU of IPv6 is 1280. If the MTU is set to a value smaller than 1280, IPv6 addresses will be lost. Other devices cannot ping the IPv6 address. - -##### Command ip addr - -1. The commands are as follows: - - ```shell - ip [-6] addr add IFADDR dev IFNAME - ``` - - You can choose to add the -6 option or not to add the IPv6 address. The ip addr command determines whether the address is an IPv4 address or an IPv6 address based on the address type. - - If the -6 option is specified but IFADDR is an IPv4 address, an error message is returned. - -2. The commands are as follows: - - ```shell - ip [-6] addr add IFADDR dev IFNAME [home|nodad] - ``` - - \[home|nodad\] is valid only for IPv6 addresses. - - - home: specifies the home address defined in RFC 6275. \(This address is obtained by the mobile node from the home link, and is a permanent address of the mobile node. If the mobile node remains in the same home link, communication between various entities is performed normally.\) - - nodad: indicates that DAD is not performed when this IPv6 address is added. \(RFC 4862\) If multiple interfaces on a device are configured with the same IPv6 address through nodad, the IPv6 address is used in the interface sequence. An IPv6 address with both nodad and non-nodad cannot be added the same interface because the two IP addresses are the same. Otherwise, the message "RTNETLINK answers: File exists" is displayed. - -3. The commands are as follows: - - ```shell - ip [-6] addr del IFADDR dev IFNAME - ``` - - You can choose to add the -6 option or not to delete an IPv6 address. The ip addr del command determines whether an IPv4 address or an IPv6 address is used based on the address type. - -4. The commands are as follows: - - ```shell - ip [-6] addr show dev IFNAME [tentative|-tentative|deprecated|-deprecated|dadfailed|-dadfailed|temporary] - ``` - - - If the -6 option is not specified, both IPv4 and IPv6 addresses are displayed. If the -6 option is specified, only IPv6 addresses are displayed. - - \[tentative|-tentative|deprecated|-deprecated|dadfailed|-dadfailed|temporary\]. These options are only for IPv6. You can filter and view addresses based on the IPv6 address status. - 1. tentative: \(only for IPv6\) lists only the addresses that have not passed duplicate address detection \(DAD\). - 2. -tentative: \(only for IPv6\) lists only the addresses that are not in the DAD process. - 3. deprecated: \(only for IPv6\) lists only the deprecated addresses. - 4. -deprecated: \(only for IPv6\) lists only the addresses that are not deprecated. - 5. dadfailed: \(only for IPv6\) lists only the addresses that fail the DAD. - 6. -dadfailed: \(only for IPv6\) lists only the addresses that do not encounter DAD failures. - 7. temporary: \(only for IPv6\) lists only the temporary addresses. - -##### Command ip route - -1. The commands are as follows: - - ```shell - ip [-6] route add ROUTE [mtu lock MTU] - ``` - - - -6 option: You can add the -6 option or not when adding an IPv6 route. The ip route command determines whether an IPv4 or IPv6 address is used based on the address type. - - - mtu lock MTU: specifies the MTU of the locked route. If the MTU is not locked, the MTU value may be changed by the kernel during the PMTUD process. If the MTU is locked, PMTUD is not attempted. All IPv4 packets are not set with the DF bit and IPv6 packets are segmented based on the MTU. - -2. The commands are as follows: - - ```shell - ip [-6] route del ROUTE - ``` - - You can choose whether to add the -6 option when deleting an IPv6 route. The ip route command determines whether an IPv4 address or an IPv6 address is used based on the address type. - -##### Command ip rule - -1. The commands are as follows: - - ```shell - ip [-6] rule list - ``` - - -6 option: If the -6 option is set, IPv6 policy-based routes are printed. If the -6 option is not set, IPv4 policy-based routes are printed. Therefore, you need to configure the -6 option according to the specific protocol type. - -2. The commands are as follows: - - ```shell - ip [-6] rule [add|del] [from|to] ADDR table TABLE pref PREF - ``` - - -6 option: IPv6-related policy routing entries need to be configured with the -6 option. Otherwise, the error message "Error: Invalid source address." is displayed. Accordingly, the -6 option cannot be set for IPv4-related policy routing entries. Otherwise, the error message "Error: Invalid source address." is displayed. - -#### Configuration Differences of the NetworkManager Service - -##### Overview - -The NetworkManager service uses the ifup/ifdown logical interface definition to perform advanced network settings. Most of the parameters are set in the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg- configuration files. The former is a global setting, and the latter is a setting of a specified NIC. When the two settings conflict, the latter takes effect. - -##### Configuration Differences - -The configuration differences in /etc/sysconfig/network are as follows: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IPv4

-

IPv6

-

Description

-

N/A

-

IPV6FORWARDING=yes|no

-

IPv6 forwarding. By default, IPv6 packets are not forwarded.

-

N/A

-

IPV6_AUTOCONF=yes|no

-

If IPv6 forwarding is enabled, the value is no. Otherwise, the value is yes.

-

N/A

-

IPV6_ROUTER=yes|no

-

If IPv6 forwarding is enabled, the value is yes. Otherwise, the value is no.

-

N/A

-

IPV6_AUTOTUNNEL=yes|no

-

Indicates the automatic tunnel mode. The default value is no.

-

GATEWAY

-

IPV6_DEFAULTGW=<IPv6 address[%interface]> (optional)

-

Indicates the default gateway in IPv6.

-

N/A

-

IPV6_DEFAULTDEV=<interface> (optional)

-

Specifies the default forwarding NIC.

-

N/A

-

IPV6_RADVD_PIDFILE=<pid-file> (optional)

-

The default path of ipv6_radvd_pid is /var/run/radvd/radvd.pid.

-

N/A

-

IPV6_RADVD_TRIGGER_ACTION=startstop|reload|restart|SIGHUP (optional)

-

Default radvd trigger action.

-
- -The differences in /etc/sysconfig/network-scripts/ifcfg- are as follows: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IPv4

-

IPv6

-

Description

-

IPADDR

-

IPV6ADDR=<IPv6 address>[/<prefix length>]

-

indicates the IP address.

-

PREFIX

-

N/A

-

The network prefix, network alias, and PPP are invalid. The priority is higher than that of NETMASK.

-

NETMASK

-

N/A

-

Indicates the subnet mask. It is used only for the alias and PPP.

-

GATEWAY

-

IPV6_DEFAULTGW=<IPv6 address[%interface]> (optional)

-

Default gateway

-

MTU

-

IPV6_MTU=<MTU of link> (optional)

-

Default MTU

-

IPV4_FAILURE_FATAL=yes|no

-

IPV6_FAILURE_FATAL

-

The default value is no. If this parameter is set to yes, ifup-eth exits when dhclient fails.

-

N/A

-

IPV6_PRIVACY=rfc3041

-

Disabled by default.

-

N/A

-

IPV6INIT=yes|no

-

IPv6 is enabled by default.

-

N/A

-

IPV6FORWARDING=yes|no

-

This function is disabled by default and has been discarded.

-
- -### FAQs - -#### The iscsi-initiator-utils Does Not Support the fe80 IPv6 Address - -##### Symptom - -When a client uses an IPv6 address to log in to the iSCSI server, run the iscsiadm -m node -p ipv6address -l command. If the global address is used, replace ipv6address in the command example with the global address. However, the link-local address \(IPv6 address starting with fe80\) cannot be used because the current mechanism of iscsi-initiator-utils does not support the link-local address to log in to the iSCSI server. - -##### Possible Cause - -If you log in to the system using the iscsiadm -m node -p fe80::xxxx -l format, a login timeout error is returned. This is because you must specify an interface when using the link-local address. Otherwise, the iscsi\_io\_tcp\_connect function fails to invoke the connect function, and the standard error code 22 is generated. - -If you use the iscsiadm -m node -p fe80::xxxx%enp3s0 -l format for login, the iscsi\_addr\_match function will compare the address fe80::xxxx%enp3s0 with the address fe80::xxxx in the node information returned by the server. The comparison result does not match, causing the login failure. - -Therefore, **the current mechanism of iscsi-initiator-utils does not support login to the iSCSI server using a link-local address.** - -#### The IPv6 Address Is Lost After the NIC Is Down - -##### Symptom - -Run the ip link down+up NIC or ifconfig down+up NIC command to disable the NIC and then enable it to go online. Check the IP address configured on the NIC. It is found that the IPv4 address is not lost but the configured IPv6 address is lost. - -##### Possible Cause - -According to the processing logic in the kernel, if the NIC is set to the down state, all IPv4 and IPv6 addresses will be cleared. After the NIC is set to the up state, the IPv4 address is automatically restored, and the automatically configured IPv6 link-local address on the NIC is also restored. However, other IPv6 addresses are lost by default. To retain these IPv6 addresses, run the **sysctl -w net.ipv6.conf.\< _NIC name_ \>.keep\_addr\_on\_down=1** command. - -#### Taking a Long Time to Add or Delete an IPv6 Address for a Bond Interface with Multiple IPv6 Addresses - -##### Symptom - -When users run the following command to add or delete \(including flush\) an IPv6 address, the waiting time increases linearly along with the number of IPv6 addresses configured on a bond interface. **X** is the least significant 16 bits that dynamically change. For example, it takes about five minutes to add 3000 IPv6 address to or delete them from a bond interface that already has four physical NICs using a single thread, while for a common physical NIC, it takes less than 10 seconds. - -```shell -ip a add/del 192:168::18:X/64 dev DEVICE -``` - -##### Possible Cause - -When an IPv6 address is added to a bond interface, the IPv6 multicast address is generated and synchronized to all physical NICs. The time required increases with the number of IPv6 addresses. As a result, it takes a too long time. - -##### Solution - -The IPv6 multicast address is generated by combining the least significant 24 bits of the IPv6 address and 33-33-ff. If there are too many multicast addresses, it takes a long time to add or delete the address. If there are a few multicast addresses, the time required is not affected. - -It is recommended that you set the least significant 24 bits of the IPv6 address to be the same as the most significant 24 bits of the IPv6 address. In this way, a single NIC can communicate with external devices using only one IP address in a network segment. - -#### Rsyslog Log Transmission Is Delayed in the Scenario Where Both IPv4 and IPv6 Are Used - -##### Symptom - -When both IPv4 and IPv6 addresses are configured in the configuration file of the rsyslog client and the port configurations are the same, there is a possibility that log output is delayed when the server collects logs. - -##### Possible Cause - -The delay is caused by the buffer queue mechanism of rsyslog. By default, rsyslog writes data to a file only when the number of buffer queues reaches a specified value. - -##### Solution - -You can disable the buffer queue mechanism by configuring the Direct mode as the **root** user. Add the following information at the beginning of the new remote transmission configuration file in the /etc/rsyslog.d directory on the rsyslog remote transmission server: - -```text -$ActionQueueType Direct -$MainMsgQueueType Direct -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- In direct mode, the queue size is reduced by 1. Therefore, one log is reserved in the queue for the next log output. ->- The direct mode degrades the rsyslog performance of the server. diff --git a/docs/en/docs/Administration/configuring-the-repo-server.md b/docs/en/docs/Administration/configuring-the-repo-server.md deleted file mode 100644 index 18855a77ae4131b94e1e06a734dbf37b3626d100..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/configuring-the-repo-server.md +++ /dev/null @@ -1,407 +0,0 @@ -# Configuring the Repo Server - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> openEuler provides multiple repo sources for online usage. For details about the repo sources, see [OS Installation](../Releasenotes/installing-the-os.md). If you cannot obtain the openEuler repo source online, you can use the ISO release package provided by openEuler to create a local openEuler repo source. This section uses the **openEuler-21.09-aarch64-dvd.iso** file as an example. Modify the ISO file as required. - - - -- [Configuring the Repo Server](#configuring-the-repo-server) - - [Overview](#overview) - - [Creating or Updating a Local Repo Source](#creating-or-updating-a-local-repo-source) - - [Obtaining the ISO File](#obtaining-the-iso-file) - - [Mounting an ISO File to Create a Repo Source](#mounting-an-iso-file-to-create-a-repo-source) - - [Creating a Local Repo Source](#creating-a-local-repo-source) - - [Updating the Repo Source](#updating-the-repo-source) - - [Deploying the Remote Repo Source](#deploying-the-remote-repo-source) - - [Installing and Configuring Nginx](#installing-and-configuring-nginx) - - [Starting Nginx](#starting-nginx) - - [Deploying the Repo Source](#deploying-the-repo-source) - - [Using the Repo Source](#using-the-repo-source) - - [Configuring Repo as the Yum Source](#configuring-repo-as-the-yum-source) - - [Repo Priority](#repo-priority) - - [Related Commands of dnf](#related-commands-of-dnf) - - -## Overview - -Create the **openEuler-21.09-aarch64-dvd.iso** file provided by openEuler as the repo source. The following uses Nginx as an example to describe how to deploy the repo source and provide the HTTP service. - -## Creating or Updating a Local Repo Source - -Mount the openEuler ISO file **openEuler-21.09-aarch64-dvd.iso** to create and update a repo source. - -### Obtaining the ISO File - -Obtain the openEuler ISO file from the following website: - -[https://repo.openeuler.org/openEuler-21.09/ISO/](https://repo.openeuler.org/openEuler-21.09/ISO/) - -### Mounting an ISO File to Create a Repo Source - -Run the **mount** command as the **root** user to mount the ISO file. - -The following is an example: - -```shell -mount /home/openEuler/openEuler-21.09-aarch64-dvd.iso /mnt/ -``` - -The mounted mnt directory is as follows: - -```text -. -│── boot.catalog -│── docs -│── EFI -│── images -│── Packages -│── repodata -│── TRANS.TBL -└── RPM-GPG-KEY-openEuler -``` - -In the preceding directory, **Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repo source metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler. - -### Creating a Local Repo Source - -You can copy related files in the ISO file to a local directory to create a local repo source. The following is an example: - -```shell -mount /home/openEuler/openEuler-21.09-aarch64-dvd.iso /mnt/ -mkdir -p /home/openEuler/srv/repo/ -cp -r /mnt/Packages /home/openEuler/srv/repo/ -cp -r /mnt/repodata /home/openEuler/srv/repo/ -cp -r /mnt/RPM-GPG-KEY-openEuler /home/openEuler/srv/repo/ -``` - -The local Repo directory is as follows: - -```text -. -│── Packages -│── repodata -└── RPM-GPG-KEY-openEuler -``` - -**Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repo source metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler. - -### Updating the Repo Source - -You can update the repo source in either of the following ways: - -- Use the latest ISO file to update the existing repo source. The method is the same as that for creating a repo source. That is, mount the ISO file or copy the ISO file to the local directory. - -- Add a RPM package to the **Packages** directory of the repo source and run the **createrepo** command to update the repo source. - - ```shell - createrepo --update --workers=10 ~/srv/repo - ``` - -In this command, **--update** indicates the update, and **--workers** indicates the number of threads, which can be customized. - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -If the command output contains "createrepo: command not found", run the **dnf install createrepo** command as the **root** user to install the **createrepo** softeware. - -## Deploying the Remote Repo Source - -Install openEuler OS and deploy the repo source using Nginx on openEuler OS. - -### Installing and Configuring Nginx - -1. Download the Nginx tool and install it as the **root** user. - -2. After Nginx is installed, configure /etc/nginx/nginx.conf as the **root** user. - - > ![](./public_sys-resources/icon-note.gif) **NOTE:** -The configuration content in this document is for reference only. You can configure the content based on the site requirements (for example, security hardening requirements). - - ```text - user nginx; - worker_processes auto; # You are advised to set this parameter to **core-1** . - error_log /var/log/nginx/error.log warn; # Log storage location - pid /var/run/nginx.pid; - - events { - worker_connections 1024; - } - - http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /var/log/nginx/access.log main; - sendfile on; - keepalive_timeout 65; - - server { - listen 80; - server_name localhost; # Server name (URL) - client_max_body_size 4G; - root /usr/share/nginx/repo; # Default service directory - - location / { - autoindex on; # Enable the access to lower-layer files in the directory. - autoindex_exact_size on; - autoindex_localtime on; - } - - } - - } - ``` - -### Starting Nginx - -1. Run the following `systemctl` commands as the **root** user to start the Nginx service. - - ```shell - systemctl enable nginx - systemctl start nginx - ``` - -2. You can run the following command to check whether Nginx is started successfully: - - ```shell - systemctl status nginx - ``` - - - [Figure 1](#en-us_topic_0151920971_fd25e3f1d664b4087ae26631719990a71) indicates that the Nginx service is started successfully. - - **Figure 1** The Nginx service is successfully started. -![](./figures/the-nginx-service-is-successfully-started.png "the-nginx-service-is-successfully-started") - - - If the Nginx service fails to be started, view the error information. - - ```shell - systemctl status nginx.service --full - ``` - - **Figure 2** The Nginx service startup fails - ![](./figures/nginx-startup-failure.png "nginx-startup-failure") - - As shown in [Figure 2](#en-us_topic_0151920971_f1f9f3d086e454b9cba29a7cae96a4c54), the Nginx service fails to be created because the /var/spool/nginx/tmp/client\_body directory fails to be created. You need to manually create the directory as the **root** user. Solve similar problems as follows: - - ```shell - mkdir -p /var/spool/nginx/tmp/client_body - mkdir -p /var/spool/nginx/tmp/proxy - mkdir -p /var/spool/nginx/tmp/fastcgi - mkdir -p /usr/share/nginx/uwsgi_temp - mkdir -p /usr/share/nginx/scgi_temp - ``` - -### Deploying the Repo Source - -1. Run the following command as the **root** user to create the /usr/share/nginx/repo directory specified in the Nginx configuration file /etc/nginx/nginx.conf: - - ```shell - mkdir -p /usr/share/nginx/repo - ``` - -2. Run the following command as the **root** user to modify the /usr/share/nginx/repo directory permission: - - ```shell - chmod -R 755 /usr/share/nginx/repo - ``` - -3. Configure firewall rules as the **root** user to enable the port (port 80) configured for Nginx. - - ```shell - firewall-cmd --add-port=80/tcp --permanent - firewall-cmd --reload - ``` - - Check whether port 80 is enabled as the **root** user. If the output is **yes**, port 80 is enabled. - - ```shell - firewall-cmd --query-port=80/tcp - ``` - - You can also enable port 80 using iptables as the **root** user. - - ```shell - iptables -I INPUT -p tcp --dport 80 -j ACCEPT - ``` - -4. After the Nginx service is configured, you can use the IP address to access the web page, as shown in [Figure 3](#en-us_topic_0151921017_fig1880404110396). - - **Figure 3** Nginx deployment succeeded -![](./figures/nginx-deployment-succeeded.png "nginx-deployment-succeeded") - -5. Use either of the following methods to add the repo source to the /usr/share/nginx/repo directory: - - - Copy related files in the image to the /usr/share/nginx/repo directory as the **root** user. - - ```shell - mount /home/openEuler/openEuler-21.09-aarch64-dvd.iso /mnt/ - cp -r /mnt/Packages /usr/share/nginx/repo/ - cp -r /mnt/repodata /usr/share/nginx/repo/ - cp -r /mnt/RPM-GPG-KEY-openEuler /usr/share/nginx/repo/ - chmod -R 755 /usr/share/nginx/repo - ``` - - The **openEuler-21.09-aarch64-dvd.iso** file is stored in the **/home/openEuler** directory. - - - Create a soft link for the repo source in the /usr/share/nginx/repo directory as the **root** user. - - ```shell - ln -s /mnt /usr/share/nginx/repo/os - ``` - - **/mnt** is the created repo source, and **/usr/share/nginx/repo/os** points to **/mnt** . - -## Using the repo Source - -The repo source can be configured as a yum source, which is a shell front-end software package manager. Based on the Redhat package manager (RPM), YUM can automatically download the RPM package from the specified server, install the package, and process dependent relationship. It supports one-off installation for all dependent software packages. - -### Configuring Repo as the Yum Source - -You can configure the built repo as the yum source and create the \*\*\*.repo configuration file (the extension .repo is mandatory) in the /etc/yum.repos.d/ directory as the **root** user. You can configure the yum source on the local host or HTTP server. - -- Configuring the local yum source. - - Create the **openEuler.repo** file in the **/etc/yum.repos.d** directory and use the local repository as the yum source. The content of the **openEuler.repo** file is as follows: - - ```text - [base] - name=base - baseurl=file:///home/openEuler/srv/repo - enabled=1 - gpgcheck=1 - gpgkey=file:///home/openEuler/srv/repo/RPM-GPG-KEY-openEuler - ``` - - > ![](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - **repoid** indicates the ID of the software repository. Repoids in all .repo configuration files must be unique. In the example, **repoid** is set to **base**. - > - **name** indicates the string that the software repository describes. - > - **baseurl** indicates the address of the software repository. - > - **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled. - > - **gpgcheck** indicates whether to enable the GNU privacy guard (GPG) to check the validity and security of sources of RPM packages. **1** indicates GPG check is enabled. **0** indicates the GPG check is disabled. - > - **gpgkey** indicates the public key used to verify the signature. - -- Configuring the yum source for the HTTP server - - Create the **openEuler.repo** file in the **/etc/yum.repos.d** directory. - - - If the repo source of the HTTP server deployed by the user is used as the yum source, the content of **openEuler.repo** is as follows: - - ```text - [base] - name=base - baseurl=http://192.168.139.209/ - enabled=1 - gpgcheck=1 - gpgkey=http://192.168.139.209/RPM-GPG-KEY-openEuler - ``` - - > ![](./public_sys-resources/icon-note.gif) **NOTE:** - > 192.168.139.209 is an example. Replace it with the actual IP address. - - - If the openEuler repo source provided by openEuler is used as the yum source, the content of **openEuler.repo** is as follows (the AArch64-based OS repo source is used as an example): - - ```text - [base] - name=base - baseurl=http://repo.openeuler.org/openEuler-21.09/OS/aarch64/ - enabled=1 - gpgcheck=1 - gpgkey=http://repo.openeuler.org/openEuler-21.09/OS/aarch64/RPM-GPG-KEY-openEuler - ``` - -### repo Priority - -If there are multiple repo sources, you can set the repo priority in the .repo file. If the priority is not set, the default priority is **99** . If the same RPM package exists in the sources with the same priority, the latest version is installed. **1** indicates the highest priority and **99** indicates the lowest priority. The following shows how to set the priority of **openEuler.repo** to **2**. - -```text -[base] -name=base -baseurl=http://192.168.139.209/ -enabled=1 -priority=2 -gpgcheck=1 -gpgkey=http://192.168.139.209/RPM-GPG-KEY-openEuler -``` - -### Related Commands of dnf - -The **dnf** command can automatically parse the dependency between packages during installation and upgrade. The common usage method is as follows: - -```shell -dnf -``` - -Common commands are as follows: - -- Installation - - Run the following command as the **root** user. - - ```shell - dnf install - ``` - -- Upgrade - - Run the following command as the **root** user. - - ```shell - dnf update - ``` - -- Rollback - - Run the following command as the **root** user. - - ```shell - dnf downgrade - ``` - -- Update check - - ```shell - dnf check-update - ``` - -- Uninstallation - - Run the following command as the **root** user. - - ```shell - dnf remove - ``` - -- Query - - ```shell - dnf search - ``` - -- Local installation - - Run the following command as the **root** user. - - ```shell - dnf localinstall - ``` - -- Historical records check - - ```shell - dnf history - ``` - -- Cache records clearing - - ```shell - dnf clean all - ``` - -- Cache update - - ```shell - dnf makecache - ``` diff --git a/docs/en/docs/Administration/configuring-the-web-server.md b/docs/en/docs/Administration/configuring-the-web-server.md deleted file mode 100644 index 412f2575e868169626b87f3cfe72cda61e556d25..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/configuring-the-web-server.md +++ /dev/null @@ -1,518 +0,0 @@ -# Configuring the Web Server - - -- [Configuring the Web Server](#configuring-the-web-server) - - [Apache Server](#apache-server) - - [Overview](#overview) - - [Managing httpd](#managing-httpd) - - [Configuration File Description](#configuration-file-description) - - [Management Module and SSL](#management-module-and-ssl) - - [Verifying Whether the Web Service Is Successfully Set Up](#verifying-whether-the-web-service-is-successfully-set-up) - - [Nginx Server](#nginx-server) - - [Overview](#overview-1) - - [Installing Nginx](#installing-nginx) - - [Managing Nginx](#managing-nginx) - - [Configuration File Description](#configuration-file-description-1) - - [Management Modules](#management-modules) - - [Verifying Whether the Web Service Is Successfully Set Up](#verifying-whether-the-web-service-is-successfully-set-up-1) - - - -## Apache Server - -### Overview - -World Wide Web \(Web\) is one of the most commonly used Internet protocols. At present, the web server in the Unix-Like system is mainly implemented through the Apache server software. To operate dynamic websites, LAMP \(Linux + Apache + MySQL + PHP\) is developed. Web services can be combined with multimedia such as text, graphics, images, and audio, and support information transmission through hyperlinks. - -The web server version in the openEuler system is Apache HTTP server 2.4, that is, httpd, which is an open-source web server developed by the Apache Software Foundation. - -### Managing httpd - -#### Overview - -You can use the systemctl tool to manage the httpd service, including starting, stopping, and restarting the service, and viewing the service status. This section describes how to manage the Apache HTTP service. - -#### Prerequisites - -- To use the Apache HTTP service, ensure that the rpm package of the httpd service has been installed in your system. Run the following command as the **root** user to install the rpm package: - - ```shell - dnf install httpd - ``` - - For more information about service management, see [Service Management](./service-management.md). - -- To start, stop, and restart the httpd service, you must have the root permission. - -#### Starting a Service - -- Run the following command to start and run the httpd service: - - ```shell - systemctl start httpd - ``` - -- If you want the httpd service to automatically start when the system starts, the command and output are as follows: - - ```shell - $ systemctl enable httpd - Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If the running Apache HTTP server functions as a secure server, a password is required after the system is started. The password is an encrypted private SSL key. - -#### Stopping the Service - -- Run the following command to stop the httpd service: - - ```shell - systemctl stop httpd - ``` - -- If you want to prevent the service from automatically starting during system startup, the command and output are as follows: - - ```shell - $ systemctl disable httpd - Removed /etc/systemd/system/multi-user.target.wants/httpd.service. - ``` - -#### Restarting a Service - -You can restart the service in any of the following ways: - -- Restart the service by running the restart command: - - ```shell - systemctl restart httpd - ``` - - This command stops the ongoing httpd service and restarts it immediately. This command is generally used after a service is installed or when a dynamically loaded module \(such as PHP\) is removed. - -- Reload the configuration. - - ```shell - systemctl reload httpd - ``` - - This command causes the running httpd service to reload its configuration file. Any requests that are currently being processed will be interrupted, causing the client browser to display an error message or re-render some pages. - -- Re-load the configuration without affecting the activation request. - - ```shell - apachectl graceful - ``` - - This command causes the running httpd service to reload its configuration file. Any requests that are currently being processed will continue to use the old configuration file. - -#### Verifying the Service Status - -Check whether the httpd service is running. - -```shell -systemctl is-active httpd -``` - -If active is displayed in the command output, the service is running. - -### Configuration File Description - -After the httpd service is started, it reads the configuration file shown in [Table 1](#table24341012096) by default. - -**Table 1** Configuration file description - - - - - - - - - - - - - -

File

-

Description

-

/etc/httpd/conf/httpd.conf

-

Main configuration files.

-

/etc/httpd/conf.d

-

Secondary directory of configuration files, which are also contained in the main configuration file.

-

The secondary directory of a configuration file is contained in the main configuration file.

-
- -Although the default configuration can be used in most cases, you need to be familiar with some important configuration items. After the configuration file is modified, run the following command as the **root** user to check the syntax errors that may occur in the configuration file: - -```shell -apachectl configtest -``` - -If the following information is displayed, the syntax of the configuration file is correct: - -```text -Syntax OK -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- Before modifying the configuration file, back up the original file so that the configuration file can be quickly restored if a fault occurs. ->- The modified configuration file takes effect only after the web service is restarted. - -### Management Module and SSL - -#### Overview - -The httpd service is a modular application that is distributed with many Dynamic Shared Objects \(DSOs\). DSOs can be dynamically loaded or unloaded when running if necessary. These modules are located in the /usr/lib64/httpd/modules/ directory of the server operating system. This section describes how to load and write a module. - -#### Loading a Module - -To load a special DSO module, you can use the load module indication in the configuration file. The modules provided by the independent software package have their own configuration files in the /etc/httpd/conf.modules.d directory. - -For example, to load the asis DSO module, perform the following steps: - -1. In the **/etc/httpd/conf.modules.d/00-optional.conf** file, uncomment the following configuration line as the **root** user: - - ```conf - LoadModule asis_module modules/mod_asis.so - ``` - -2. After the loading is complete, restart the httpd service as the **root** user to reload the configuration file. - - ```shell - systemctl restart httpd - ``` - -3. After the loading is complete, run the httpd -M command as the **root** user to check whether the asis DSO module is loaded. - - ```shell - httpd -M | grep asis - ``` - - If the following information is displayed, the asis DSO module is successfully loaded: - - ```text - asis_module (shared) - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->**Common httpd commands** -> ->- httpd -v: views the httpd version number. ->- httpd -l: views the static modules compiled into the httpd program. ->- httpd -M: views the static modules and loaded dynamic modules that have been compiled into the httpd program. - -#### Introduction to SSL - -Secure Sockets Layer \(SSL\) is an encryption protocol that allows secure communication between the server and client. The Transport Layer Security \(TLS\) protocol ensures security and data integrity for network communication. openEuler supports Mozilla Network Security Services \(NSS\) as the security protocol TLS. To load the SSL, perform the following steps: - -1. Install the **mod\_ssl** RPM package as the **root** user. - - ```shell - dnf install mod_ssl - ``` - -2. After the loading is complete, restart the httpd service as the **root** user to reload the configuration file. - - ```shell - systemctl restart httpd - ``` - -3. After the loading is complete, run the **httpd -M** command as the **root** user to check whether the SSL is loaded. - - ```shell - httpd -M | grep ssl - ``` - - If the following information is displayed, the SSL has been loaded successfully. - - ```text - ssl_module (shared) - ``` - -### Verifying Whether the Web Service Is Successfully Set Up - -After the web server is set up, perform the following operations to check whether the web server is set up successfully: - -1. Run the following command as the **root** user to check the IP address of the server: - - ```shell - ip a - ``` - -2. Configure the firewall as the **root** user. - - ```shell - $ firewall-cmd --add-service=http --permanent - success - # firewall-cmd --reload - success - ``` - -3. Verify whether the web server is successfully set up. You can select the Linux or Windows operating system for verification. - - Using the Linux OS - - Run the following command to check whether the web page can be accessed. If the service is successfully set up, the web page can be accessed. - - ```shell - curl http://192.168.1.60 - ``` - - Run the following command to check whether the command output is 0. If the command output is 0, the httpd server is successfully set up. - - ```shell - echo $? - ``` - - - Using the Windows OS - - Open the browser and enter the following address in the address box. If the web page can be accessed, the httpd server is successfully set up. - - - - If the port number is changed, enter the address in the following format: - - - -## Nginx Server - -### Overview - -Nginx is a lightweight web server which also acts as a reverse proxy server and email \(IMAP/POP3\) proxy server. It features low memory usage and strong concurrency capability. Nginx supports FastCGI, SSL, virtual hosts, URL rewrite, Gzip, and extension of many third-party modules. - -### Installing Nginx - -1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). -2. Clear the cache. - - ```shell - dnf clean all - ``` - -3. Create a cache. - - ```shell - dnf makecache - ``` - -4. Install the Nginx server as the **root** user. - - ```shell - dnf install nginx - ``` - -5. Check the installed RPM package. - - ```shell - dnf list all | grep nginx - ``` - -### Managing Nginx - -#### Overview - -You can use the systemctl tool to manage the Nginx service, including starting, stopping, and restarting the service, and viewing the service status. This section describes how to manage the Nginx service. - -#### Prerequisites - -- Ensure that the Nginx service has been installed. If not, install it by referring to [Installing Nginx](#installing-nginx). - - For more information about service management, see [Service Management](./service-management.md). - -- To start, stop, and restart the Nginx service, you must have the **root** permission. - -#### Starting a Service - -- Run the following command to start and run the Nginx service: - - ```shell - systemctl start nginx - ``` - -- If you want the Nginx service to automatically start when the system starts, the command and output are as follows: - - ```shell - $ systemctl enable nginx - Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service. - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If the running Nginx server functions as a secure server, a password is required after the system is started. The password is an encrypted private SSL key. - -#### Stopping the Service - -- Run the following command to stop the Nginx service: - - ```shell - systemctl stop nginx - ``` - -- If you want to prevent the service from automatically starting during system startup, the command and output are as follows: - - ```shell - $ systemctl disable nginx - Removed /etc/systemd/system/multi-user.target.wants/nginx.service. - ``` - -#### Restarting a Service - -You can restart the service in any of the following ways: - -- Restart the service. - - ```shell - systemctl restart nginx - ``` - - This command stops the ongoing Nginx service and restarts it immediately. This command is generally used after a service is installed or when a dynamically loaded module \(such as PHP\) is removed. - -- Reload the configuration. - - ```shell - systemctl reload nginx - ``` - - This command causes the running Nginx service to reload its configuration file. Any requests that are currently being processed will be interrupted, causing the client browser to display an error message or re-render some pages. - -- Smoothly restart Nginx. - - ```shell - kill -HUP PID - ``` - - This command causes the running Nginx service to reload its configuration file. Any requests that are currently being processed will continue to use the old configuration file. - -#### Verifying the Service Status - -Check whether the Nginx service is running. - -```shell -systemctl is-active nginx -``` - -If **active** is displayed in the command output, the service is running. - -### Configuration File Description - -After the Nginx service is started, it reads the configuration file shown in [Table 2](#table24341012096) by default. - -**Table 2** Configuration file description - - - - - - - - - - - - - -

File

-

Description

-

/etc/nginx/nginx.conf

-

Main configuration files.

-

/etc/nginx/conf.d

-

Secondary directory of configuration files, which are also contained in the main configuration file.

-

The secondary directory of a configuration file is contained in the main configuration file.

-
- -Although the default configuration can be used in most cases, you need to be familiar with some important configuration items. After the configuration file is modified, run the following command as the **root** user to check the syntax errors that may occur in the configuration file: - -```shell -nginx -t -``` - -If the command output contains **syntax is ok**, the syntax of the configuration file is correct. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- Before modifying the configuration file, back up the original file so that the configuration file can be quickly restored if a fault occurs. ->- The modified configuration file takes effect only after the web service is restarted. - -### Management Modules - -#### Overview - -The Nginx service is a modular application that is distributed with many Dynamic Shared Objects \(DSOs\). DSOs can be dynamically loaded or unloaded when running if necessary. These modules are located in the **/usr/lib64/nginx/modules/** directory of the server operating system. This section describes how to load and write a module. - -#### Loading a Module - -To load a special DSO module, you can use the load module indication in the configuration file. Generally, the modules provided by independent software packages have their own configuration files in the **/usr/share/nginx/modules** directory. - -The DSO is automatically loaded when the **dnf install nginx** command is used to install the Nginx in the openEuler operating system. - -### Verifying Whether the Web Service Is Successfully Set Up - -After the web server is set up, perform the following operations to check whether the web server is set up successfully: - -1. Run the following command as the **root** user to check the IP address of the server: - - ```shell - ip a - ``` - - If the following information is displayed, the IP address of the server is **192.168.1.60**. - - ```text - enp3s0: flags=4163 mtu 1500 - inet 192.168.1.60 netmask 255.255.255.0 broadcast 192.168.1.255 - inet6 fe80::5054:ff:fe95:499f prefixlen 64 scopeid 0x20 - ether 52:54:00:95:49:9f txqueuelen 1000 (Ethernet) - RX packets 150713207 bytes 49333673733 (45.9 GiB) - RX errors 0 dropped 43 overruns 0 frame 0 - TX packets 2246438 bytes 203186675 (193.7 MiB) - TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 - - enp4s0: flags=4163 mtu 1500 - ether 52:54:00:7d:80:9e txqueuelen 1000 (Ethernet) - RX packets 149937274 bytes 44652889185 (41.5 GiB) - RX errors 0 dropped 1102561 overruns 0 frame 0 - TX packets 0 bytes 0 (0.0 B) - TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 - - lo: flags=73 mtu 65536 - inet 127.0.0.1 netmask 255.0.0.0 - inet6 ::1 prefixlen 128 scopeid 0x10 - loop txqueuelen 1000 (Local Loopback) - RX packets 37096 bytes 3447369 (3.2 MiB) - RX errors 0 dropped 0 overruns 0 frame 0 - TX packets 37096 bytes 3447369 (3.2 MiB) - TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 - ``` - -2. Configure the firewall as the **root** user. - - ```shell - $ firewall-cmd --add-service=http --permanent - success - # firewall-cmd --reload - success - ``` - -3. Verify whether the web server is successfully set up. You can select the Linux or Windows operating system for verification. - - Using the Linux OS - - Run the following command to check whether the web page can be accessed. If the service is successfully set up, the web page can be accessed. - - ```shell - curl http://192.168.1.60 - ``` - - Run the following command to check whether the command output is **0**. If the command output is **0**, the Nginx server is successfully set up. - - ```shell - echo $? - ``` - - - Using the Windows OS - - Open the browser and enter the following address in the address box. If the web page can be accessed, the Nginx server is successfully set up. - - - - If the port number is changed, enter the address in the following format: - - diff --git a/docs/en/docs/Administration/faqs.md b/docs/en/docs/Administration/faqs.md deleted file mode 100644 index d39cc95ce3648e3f69ef2a1f04f78d604f1ab37e..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/faqs.md +++ /dev/null @@ -1,413 +0,0 @@ -# FAQs - - -- [FAQs](#faqs) - - [Why Is the Memory Usage of the libvirtd Service Queried by Running the systemctl and top Commands Different?](#why-is-the-memory-usage-of-the-libvirtd-service-queried-by-running-the-systemctl-and-top-commands-different) - - [An Error Occurs When stripsize Is Set to 4 During RAID 0 Volume Configuration](#an-error-occurs-when-stripsize-is-set-to-4-during-raid-0-volume-configuration) - - [Failed to Compile MariaDB Using rpmbuild](#failed-to-compile-mariadb-using-rpmbuild) - - [Failed to Start the SNTP Service Using the Default Configuration](#failed-to-start-the-sntp-service-using-the-default-configuration) - - [Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package](#installation-failure-caused-by-software-package-conflict-file-conflict-or-missing-software-package) - - [Failed to Downgrade libiscsi](#failed-to-downgrade-libiscsi) - - [Failed to Downgrade xfsprogs](#failed-to-downgrade-xfsprogs) - - [Failed to Downgrade elfutils](#failed-to-downgrade-elfutils) - - [CPython/Lib Detects CVE-2019-9674: Zip Bomb](#cpythonlib-detects-cve-2019-9674-zip-bomb) - - [ReDoS Attack Occurs Due to Improper Use of glibc Regular Expressions](#redos-attack-occurs-due-to-improper-use-of-glibc-regular-expressions) - - [An Error Is Reported When gdbm-devel Is Installed or Uninstalled During the Installation and Uninstallation of httpd-devel and apr-util-devel](#an-error-is-reported-when-gdbm-devel-is-installed-or-uninstalled-during-the-installation-and-uninstallation-of-httpd-devel-and-apr-util-devel) - - [An rpmdb Error Is Reported When Running the yum or dnf Command After the System Is Rebooted](#an-rpmdb-error-is-reported-when-running-the-yum-or-dnf-command-after-the-system-is-rebooted) - - [Failed to Run `rpmrebuild -d /home/test filesystem` to Rebuild the filesystem Package](#failed-to-run-rpmrebuild--d-hometest-filesystem-to-rebuild-the-filesystem-package) - - [An Error Is Reported When modprobe or `insmod` Is Executed With the `-f` Option](#an-error-is-reported-when-modprobe-or-insmod-is-executed-with-the--f-option) - - -## Why Is the Memory Usage of the libvirtd Service Queried by Running the systemctl and top Commands Different? - -### Symptom - -The output of the **systemctl** and **systemd-cgtop** commands shows that the libvirtd service occupies more than 1.5 GB memory, but the output of the **top** command shows that the libvirtd service occupies about 70 MB memory. - -### Possible Cause - -The memory displayed in the services \(including systemctl and systemd-cgtop\) managed by systemd can be obtained from **memory.usage\_in\_bytes** in Cgroup. Running the **top** command is to query the memory information in the **/proc** directory. The query results are different because the statistical method varies. - -Generally, the memory used by service processes has the following types: - -- anon\_rss: anonymous pages in user mode address spaces, for example, memory allocated by calling the malloc function or the mmap function with configured **MAP\_ANONYMOUS**. When the system memory is insufficient, this type of memory can be swapped by the kernel. -- file\_rss: mapped pages in user mode address spaces, including map file \(such as mmap of a specified file\) and map tmpfs \(such as IPC shared memory\). When the system memory is insufficient, the kernel can reclaim these pages. Data may need to be synchronized between the kernel and map file before reclamation. -- file\_cache: file cache \(page in page cache of disk file\), which is generated when a file is read or written. When the system memory is insufficient, the kernel can reclaim these pages. Data may need to be synchronized between the kernel and map file before reclamation. -- buffer pages: belongs to page cache, for example, cache generated when block device files are read. - -anon\_rss and file\_rss belong to the resident set size \(RSS\) of processes, and file\_cache and buffer pages belong to page cache. In brief: - -RSS in the output of the **top** command = anon\_rss + file\_rss; Shared memory \(SHR\) = file\_rss - -**memory.usage\_in\_bytes** in Cgroup = cache + RSS + swap - -In conclusion, the definition of memory usage obtained by running the **systemd** command is different from that obtained by running the **top** command. Therefore, the query results are different. - -## An Error Occurs When stripsize Is Set to 4 During RAID 0 Volume Configuration - -### Symptom - -An error occurs when the **stripsize** parameter is set to **4** during RAID 0 volume configuration. - -### Possible Cause - -The 64 KB page table can be enabled only in the scenario where **stripsize** is set to **64**. - -### Solution - -You do not need to modify the configuration file. When running the **lvcreate** command on openEuler, set **stripesize** to **64** because the minimum supported stripe size is 64 KB. - -## Failed to Compile MariaDB Using rpmbuild - -### Symptom - -When you log in to the system as user **root** and run the **rpmbuild** command to compile the MariaDB source code, the compilation fails and the following information is displayed: - -```text -+ echo 'mysql can'\''t run test as root' -mysql can't run test as root -+ exit 1 -``` - -### Possible Cause - -The MariaDB does not allow user **root** to execute test cases. However, test cases are automatically executed during compilation. As a result, the compilation process is blocked. - -### Solution - -Use a text editor, such as vi, to modify the value of the **runtest** variable in the **mariadb.spec** file. - -Before the modification: - -```text -%global runtest 1 -``` - -After the modification: - -```text -%global runtest 0 -``` - -The modification disables the function of executing test cases during compilation, which does not affect the compilation and the RPM package content after compilation. - -## Failed to Start the SNTP Service Using the Default Configuration - -### Symptom - -The SNTP service fails to be started with the default configuration. - -### Possible Cause - -The domain name of the NTP server is not added to the default configuration. - -### Solution - -Modify the **/etc/sysconfig/sntp** file and add the domain name of the NTP server in China: **0.generic.pool.ntp.org**. - -## Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package - -### Symptom - -Software package conflict, file conflict, or missing software packages may occur during software package installation. As a result, the upgrade is interrupted and the installation fails. The error information about software package conflict, file conflict, and missing software packages is as follows: - -The following is an example of software package conflict error information \(the conflict between **libev-libevent-devel-4.24-11.oe1.aarch64** and **libevent-devel-2.1.11-2.oe1.aarch64** is used as an example\): - -```text -package libev-libevent-devel-4.24-11.oe1.aarch64 conflicts with libevent-devel provided by libevent-devel-2.1.11-2.oe1.aarch64 - - cannot install the best candidate for the job - - conflicting requests -``` - -The following is an example of file conflict error information \(the **/usr/bin/containerd** file conflict is used as an example\): - -```text -Error: Transaction test error: - file /usr/bin/containerd from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64 - file /usr/bin/containerd-shim from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64 -``` - -The following is an example of the error message indicating that the **blivet-data** software package is missing: - -```text -Error: - Problem: cannot install both blivet-data-1:3.1.1-6.oe1.noarch and blivet-data-1:3.1.1-5.noarch - - package python2-blivet-1:3.1.1-5.noarch requires blivet-data = 1:3.1.1-5, but none of the providers can be installed - - cannot install the best update candidate for package blivet-data-1:3.1.1-5.noarch - - problem with installed package python2-blivet-1:3.1.1-5.noarch(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) -``` - -### Possible Cause - -- In the software packages provided by openEuler, some software packages have different names but the same functions. As a result, the software packages cannot be installed at the same time. -- In the software packages provided by openEuler, some software packages have different names but the same functions. As a result, the files after installation are the same, causing file conflict. -- Some software packages are depended on by other software packages before the upgrade. After the software packages are upgraded, the software packages that depend on them may fail to be installed due to lack of software packages. - -### Solution - -If a software package conflict occurs, perform the following steps \(the software package conflict in "Symptom" is used as an example\): - -1. According to the error message displayed during the installation, the software package that conflicts with the to-be-installed software package **libev-libevent-devel-4.24-11.oe1.aarch64** is **libevent-devel-2.1.11-2.oe1.aarch64**. -2. Run the **dnf remove** command to uninstall the software package that conflicts with the software package to be installed. - - ```shell - dnf remove libevent-devel-2.1.11-2.oe1.aarch64 - ``` - -3. Perform the installation again. - -If a file conflict occurs, perform the following steps \(the file conflict in "Symptom" is used as an example\): - -1. According to the error message displayed during the installation, the names of the software packages that cause the file conflict are **containerd-1.2.0-101.oe1.aarch64** and **docker-engine-18.09.0-100.aarch64**. -2. Record the names of the software packages that do not need to be installed. The following uses **docker-engine-18.09.0-100.aarch64** as an example. -3. Run the **dnf remove** command to uninstall the software package that does not need to be installed. - - ```shell - dnf remove docker-engine-18.09.0-100.aarch64 - ``` - -4. Perform the installation again. - -If a software package is missing, perform the following steps \(the missed software package in "Symptom" is used as an example\): - -1. Determine the name of the software package to be upgraded \(**blivet-data-1:3.1.1-5.noarch**\) and the name of the dependent software package \(**python2-blivet-1:3.1.1-5.noarch**\) based on the error information displayed during the upgrade. -2. Run the **dnf remove** command to uninstall the software package that depends on the upgrade package or add the **\-\-allowerasing** parameter when upgrading the software package. - - Run the **dnf remove** command to uninstall the software package that depends on the **blivet-data-1:3.1.1-5.noarch** software package. - - ```shell - dnf remove python2-blivet-1:3.1.1-5.noarch - ``` - - - Add the **\-\-allowerasing** parameter when upgrading the software package. - - ```shell - yum update blivet-data-1:3.1.1-5.noarch -y --allowerasing - ``` - -3. Perform the upgrade again. - -### Installing Conflicting Instances - -- A file conflict occurs. - - The **python3-edk2-devel.noarch** file conflicts with the **build.noarch** file due to duplicate file names. - - ```shell - $ yum install python3-edk2-devel.noarch build.noarch - ... - Error: Transaction test error: - file /usr/bin/build conflicts between attempted installs of python3-edk2-devel-202002-3.oe1.noarch and build-20191114-324.4.oe1.noarch - ``` - -## Failed to Downgrade libiscsi - -### Symptom - -libiscsi-1.19.4 or later fails to be downgraded to libiscsi-1.19.3 or earlier. - -```text -Error: -Problem: problem with installed package libiscsi-utils-1.19.0-4.oe1.x86_64 -- package libiscsi-utils-1.19.0-4.oe1.x86_64 requires libiscsi(x86-64) = 1.19.0-4.oe1, but none of the providers can be installed -- cannot install both libiscsi-1.19.0-3.oe1.x86_64 and libiscsi-1.19.0-4.oe1.x86_64 -- cannot install both libiscsi-1.19.0-4.oe1.x86_64 and libiscsi-1.19.0-3.oe1.x86_64 -- conflicting requests -(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) -``` - -### Possible Cause - -In libiscsi-1.19.3 or earlier, binary files named **iscsi-xxx** are packed into the main package **libiscsi**. However, these binary files introduce improper dependency CUnit. To solve this problem, in libiscsi-1.19.4, these binary files are separated into the **libiscsi-utils** subpackage. The main package is weakly dependent on the subpackage. You can integrate or uninstall the subpackage during image building based on product requirements. If the subpackage is not integrated or is uninstalled, the functions of the **libiscsi** main package are not affected. -When libiscsi-1.19.4 or later is downgraded to libiscsi-1.19.3 or earlier and the **libiscsi-utils** subpackage is installed in the system, because libiscsi-1.19.3 or earlier does not contain **libiscsi-utils**, **libiscsi-utils** will fail to be downgraded. Due to the fact that **libiscsi-utils** depends on the **libiscsi** main package before the downgrade, a dependency problem occurs and the libiscsi downgrade fails. - -### Solution - -Run the following command to uninstall the **libiscsi-utils** subpackage and then perform the downgrade: - -```shell -yum remove libiscsi-utils -``` - -## Failed to Downgrade xfsprogs - -### Symptom - -xfsprogs-5.6.0-2 or later fails to be downgraded to xfsprogs-5.6.0-1 or earlier. - -```text -Error: -Problem: problem with installed package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64 -- package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64 requires xfsprogs = 5.6.0-2.oe1, but none of the providers can be installed -- cannot install both xfsprogs-5.6.0-1.oe1.x86_64 and xfsprogs-5.6.0-2.oe1.x86_64 -- cannot install both xfsprogs-5.6.0-2.oe1.x86_64 and xfsprogs-5.6.0-1.oe1.x86_64 -- conflicting requests -``` - -### Possible Cause - -In xfsprogs-5.6.0-2, to reduce improper dependencies of the **xfsprogs** main package and separate experimental commands from the main package, the `xfs_scrub*` commands are separated into the **xfsprogs-xfs_scrub** subpackage. The **xfsprogs** main package is weakly dependent on the **xfsprogs-xfs_scrub** sub-package. You can integrate or uninstall the subpackage during image creation based on product requirements. If the subpackage is not integrated or is uninstalled, the functions of the **xfsprogs** main package are not affected. - -When xfsprogs-5.6.0-2 or later is downgraded to xfsprogs-5.6.0-1 or earlier and the **xfsprogs-xfs_scrub** subpackage is installed in the system, because xfsprogs-5.6.0-1 or earlier does not contain **xfsprogs-xfs_scrub**, **xfsprogs-xfs_scrub** will fail to be downgraded. Due to the fact that **xfsprogs-xfs_scrub** depends on the **xfsprogs** main package before the downgrade, a dependency problem occurs and the xfsprogs downgrade fails. - -### Solution - -Run the following command to uninstall the **xfsprogs-xfs_scrub** subpackage and then perform the downgrade: - -```shell -yum remove xfsprogs-xfs_scrub -``` - -## Failed to Downgrade elfutils - -### Symptom - -The dependency is missing. As a result, elfutils failed to be downgraded. - -![](figures/1665628542704.png) - -### Possible Cause - -22.03-LTS, 22.03-LTS-Next: elfutils-0.185-12 - -master: elfutils-0.187-7 - -20.03-LTS-SP1: elfutils-0.180-9 - -In the preceding versions, the **eu-objdump**, **eu-readelf**, and **eu-nm** commands provided by the elfutils main package are split into the elfutils-extra subpackage. When elfutils-extra has been installed in the system and elfutils is downgraded, because an earlier version (such as the version of an earlier branch) cannot provide the corresponding elfutils-extra package, the elfutils-extra subpackage is not downgraded. However, the elfutils-extra subpackage depends on the elfutils package before the downgrade. As a result, the dependency problem cannot be resolved, and the elfutils subpackage fails to be downgraded. - -### Solution - -Run the following command to uninstall the elfutils-extra subpackage and then perform the downgrade: - -```shell -yum remove -y elfutils-extra -``` - -## CPython/Lib Detects CVE-2019-9674: Zip Bomb - -### Symptom - -**Lib/zipfile.py** in Python 3.7.2 or earlier allows remote attackers to create DoS requests using zip bombs, resulting in high resource consumption. - -### Possible Cause - -Remote attackers use zip bombs to cause denial of service, affecting target system services or even crashing the system. A zip bomb is a zip file with a high compression ratio. It may be several MB or dozens of MB in size. However, after decompression, a large amount of data is generated, consuming a large amount of resources. - -### Solution - -Add the alarm information to **zipfile** at . - -## ReDoS Attack Occurs Due to Improper Use of glibc Regular Expressions - -### Symptom - -The regcomp/regexec interface of glibc is used for programming, or the glibc regular expressions, such as grep/sed, are used in shell commands. Improper regular expressions or inputs cause ReDoS attacks (CVE-2019-9192/CVE-2018-28796). -The typical regular expression pattern is the combination of the"reverse reference (\1)" with the "asterisk (*)" (zero match or multiple matches), "plus sign (+)" (one match or multiple matches), or "{m,n}" (minimum match: m; maximum match: n); or the combination of ultra-long character strings with regular expressions. The following is an example: - -```shell -$ echo D | grep -E "$(printf '(\0|)(\\1\\1)*')"Segmentation fault (core dumped) -$ grep -E "$(printf '(|)(\\1\\1)*')" -Segmentation fault (core dumped) -$ echo A | sed '/\(\)\(\1\1\)*/p' -Segmentation fault (core dumped) -$ time python -c 'print "a"*40000' | grep -E "a{1,32767}" -Segmentation fault (core dumped) -$ time python -c 'print "a"*40900' | grep -E "(a)\\1" -Segmentation fault (core dumped) -``` - -### Possible Cause - -A core dump occurs on the process that uses the regular expression. The glibc regular expression is implemented using the NFA/DFA hybrid algorithm. The internal principle is to use a greedy algorithm for recursive query to match as many character strings as possible. The greedy algorithm causes the ReDoS attack when processing the recursive regular expression. - -### Solution - -1. Strict permission control is required to reduce the attack surface. - -2. Ensure that the regular expression is correct. Do not enter an invalid regular expression or a combination of ultra-long character strings with regular expressions (references or asterisks) that may trigger infinite recursion. - - ```text - # ()(\1\1)* - # "a"*400000 - ``` - -3. After a user program detects a process exception, the user program can restart the process to restore services, improving program reliability. - -## An Error Is Reported When gdbm-devel Is Installed or Uninstalled During the Installation and Uninstallation of httpd-devel and apr-util-devel - -### Symptom - -1. An error is reported when gdbm-devel-1.18.1-1 is installed or uninstalled. -2. After the error is rectified, gdbm and gdbm-devel are upgraded to the 1.18.1-2 version. However, the default version of gdbm-devel is still 1.18.1-1 when httpd-devel and apr-util-devel (dependent on gdbm-devel) are installed. As a result, the error persists. - -### Possible Cause - -1. The gdbm-devel-1.18.1-1 package does not contain the help package that provides `info`. As a result, the help package cannot be introduced when gdbm-devel is installed independently, and the following alarm information is displayed: - - ```text - install-info: No such file or directory for /usr/share/info/gdbm.info.gz - ``` - -2. By default, the gdbm-1.18.1-1 main package is installed in the system, but the gdbm-devel package is not installed. The software packages depending on gdbm-devel still match the version of the gdbm main package and install gdbm-devel-1.18.1-1. As a result, the error persists. - -### Solution - -1. Install gdbm-1.18.1-2 to upgrade gdbm. The error is rectified. -2. Upgrade gdbm, and then install gdbm-devel to make it depend on the gdbm of the later version. The error is rectified. - -## An rpmdb Error Is Reported When Running the yum or dnf Command After the System Is Rebooted - -### Symptom - -1. After the system is rebooted, an error is reported when running an RPM-related command `yum` or `dnf` as follows: - error: db5 error(-30973) from dbenv->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery - error: cannot open Packages index using db5 - (-30973) - error: cannot open Packages database in /var/lib/rpm - Error: Error: rpmdb open failed - -### Possible Cause - -1. During an installation or upgrade, read and write operations are performed on the **/var/lib/rpm/__db.00\*** file. If an unexpected interruption occurs, such as forced power-off, drive space full, or `kill -9`, the **_db** file will be damaged. An error will be reported when a `dnf` or `yum` command is executed. - -### Solution - -Step 1 Run the `kill -9` command to terminate all running RPM-related commands. -Step 2 Run `rm -rf /var/lib/rpm/__db.00*` to delete all db.00 files. -Step 3 Run the `rpmdb --rebuilddb` command to rebuild the RPM database. - -## Failed to Run `rpmrebuild -d /home/test filesystem` to Rebuild the filesystem Package - -### Symptom - -Failed to run the `rpmrebuild --comment-missing=y --keep-perm -b -d /home/test filesystem-3.16-3.oe1.aarch64` command to rebuild the **filesystem** package. The following information is displayed: - -```text -/usr/lib/rpmrebuild/rpmrebuild.sh:Error:(RpmBuild) Package 'filesystem-3.16-3.oe1.aarch64' build failed. -/usr/lib/rpmrebuild/rpmrebuild.sh:Error: RpmBuild -``` - -### Possible Cause - -The software package creates the directory in the **%pretrans -p** phase, and modify the directory in the **%ghost** phase. If you create a file or directory in the directory and use `rpmrebuild` to build the package, the created file or directory will be included in the package. - -The root cause of the symptom is that **filesystem** creates the **/proc** directory in the **%pretrans** phase and modifies the directory in the **%ghost** phase, but some small processes are dynamically created during system running. As a result, `rpmrebuild` cannot include the processes in the package because they are not files or directories and fails to rebuild the package. - -### Solution - -Do not use `rpmrebuild` to rebuild the **filesystem** package. - -## An Error Is Reported When modprobe or `insmod` Is Executed With the `-f` Option - -### Symptom - -An error is reported when `modprobe -f ` or `insmod -f .ko.xz` is executed. For example, when `insmod -f xfs.ko.xz` is executed, error message **modprobe: ERROR: could not insert 'xfs': Key was rejected by service** is displayed. - -Up till now (2022.09.20, kmod v30), this issue has not been fixed by the kmod community.Linux v5.17 [b1ae6dc](https://github.com/torvalds/linux/commit/b1ae6dc41eaaa98bb75671e0f3665bfda248c3e7) introduced support for compressed kernel modules, which is not supported by kmod. - -### Possible Cause - -`modprobe` and `insmod` use the `finit_module()` system call to load uncompressed ko files. For compressed ko files, kmod uses the `init_module()` system call to decompress them. `init_module()` does not take the ignore check flag. As a result, `mod_verify_sig()` is always executed by the kernel. the `-f` option of `modprobe` and `insmod` changes verification information about the ko file, resulting in verification failure of `mod_verify_sig()`. - -### Solution - -Do not use the `-f` option when running `insmod` or `modprobe` on compressed ko files. diff --git a/docs/en/docs/Administration/figures/ima_performance.png b/docs/en/docs/Administration/figures/ima_performance.png deleted file mode 100644 index f5d641e8682ad2b9c0fbfad191add1819f5b2eef..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Administration/figures/ima_performance.png and /dev/null differ diff --git a/docs/en/docs/Administration/figures/ima_verification.png b/docs/en/docs/Administration/figures/ima_verification.png deleted file mode 100644 index fc879949db5387c61ccf6176f948b9a00f4fb053..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Administration/figures/ima_verification.png and /dev/null differ diff --git a/docs/en/docs/Administration/figures/mariadb-logical-architecture.png b/docs/en/docs/Administration/figures/mariadb-logical-architecture.png deleted file mode 100644 index c4e65e786d918c84bbb14c101b69bc4ad36ccb4b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Administration/figures/mariadb-logical-architecture.png and /dev/null differ diff --git a/docs/en/docs/Administration/figures/postgresql-architecture.png b/docs/en/docs/Administration/figures/postgresql-architecture.png deleted file mode 100644 index cc73eb31f746409efc1e997072bf3d18b013012e..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Administration/figures/postgresql-architecture.png and /dev/null differ diff --git a/docs/en/docs/Administration/managing-hard-disks-through-lvm.md b/docs/en/docs/Administration/managing-hard-disks-through-lvm.md deleted file mode 100644 index a51290929f1a51d04b4e819b863f0f099efe8020..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/managing-hard-disks-through-lvm.md +++ /dev/null @@ -1,574 +0,0 @@ -# Managing Drives Through LVM - - -- [Managing Drives Through LVM](#managing-drives-through-lvm) - - [LVM Overview](#lvm-overview) - - [Basic Terms](#basic-terms) - - [Installing the LVM](#installing-the-lvm) - - [Managing PVs](#managing-pvs) - - [Creating a PV](#creating-a-pv) - - [Viewing a PV](#viewing-a-pv) - - [Modifying PV Attributes](#modifying-pv-attributes) - - [Deleting a PV](#deleting-a-pv) - - [Managing VGs](#managing-vgs) - - [Creating a VG](#creating-a-vg) - - [Viewing a VG](#viewing-a-vg) - - [Modifying VG Attributes](#modifying-vg-attributes) - - [Extending a VG](#extending-a-vg) - - [Shrinking a VG](#shrinking-a-vg) - - [Deleting a VG](#deleting-a-vg) - - [Managing LVs](#managing-lvs) - - [Creating an LV](#creating-an-lv) - - [Viewing an LV](#viewing-an-lv) - - [Adjusting the LV Size](#adjusting-the-lv-size) - - [Extending an LV](#extending-an-lv) - - [Shrinking an LV](#shrinking-an-lv) - - [Deleting an LV](#deleting-an-lv) - - [Creating and Mounting a File System](#creating-and-mounting-a-file-system) - - [Creating a File System](#creating-a-file-system) - - [Manually Mounting a File System](#manually-mounting-a-file-system) - - [Automatically Mounting a File System](#automatically-mounting-a-file-system) - - - -## LVM Overview - -Logical Volume Manager \(LVM\) is a mechanism used for managing drive partitions in Linux. By adding a logical layer between drives and file systems, LVM shields the drive partition layout for file systems, thereby improving flexibility in managing drive partitions. - -The procedure of managing a drive through LVM is as follows: - -1. Create physical volumes for a drive. -2. Combine several physical volumes into a volume group. -3. Create logical volumes in the volume group. -4. Create file systems on logical volumes. - -When drives are managed using LVM, file systems are distributed on multiple drives and can be easily resized as needed. Therefore, file system space will no longer be limited by drive capacities. - -### Basic Terms - -- Physical media: refers to physical storage devices in the system, such as drives \(**/dev/hda** and **/dev/sda**\). It is the storage unit at the lowest layer of the storage system. - -- Physical volume \(PV\): refers to a drive partition or device \(such as a RAID\) that has the same logical functions as a drive partition. PVs are basic logical storage blocks of LVM. A PV contains a special label that is stored in the second 512-byte sector by default. It can also be stored in one of the first four sectors. A label contains the universal unique identifier \(UUID\) of the PV, size of the block device, and the storage location of LVM metadata in the device. - -- Volume group \(VG\): consists of PVs and shields the details of underlying PVs. You can create one or more logical volumes within a VG without considering detailed PV information. - -- Logical volume \(LV\): A VG cannot be used directly. It can be used only after being partitioned into LVs. LVs can be formatted into different file systems and can be directly used after being mounted. - -- Physical extent \(PE\): A PE is a small storage unit in a PV. The PE size is the same as the size of the logical extent in the VG. - -- Logical extent \(LE\): An LE is a small storage unit in an LV. In one VG, the LEs of all the LVs have the same size. - -## Installing the LVM - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The LVM has been installed on the openEuler OS by default. You can run the **rpm -qa | grep lvm2** command to check whether it is installed. If the command output contains "lvm2", the LVM has been installed. In this case, skip this section. If no information is output, the LVM is not installed. Install it by referring to this section. - -1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). -2. Clear the cache. - - ```bash - dnf clean all - ``` - -3. Create a cache. - - ```bash - dnf makecache - ``` - -4. Install the LVM as the **root** user. - - ```bash - dnf install lvm2 - ``` - -5. Check the installed RPM package. - - ```bash - rpm -qa | grep lvm2 - ``` - -## Managing PVs - -### Creating a PV - -Run the **pvcreate** command as the **root** user to create a PV. - -```bash -pvcreate [option] devname ... -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-f**: forcibly creates a PV without user confirmation. - - **-u**: specifies the UUID of the device. - - **-y**: answers yes to all questions. - -- _devname_: specifies the name of the device corresponding to the PV to be created. If multiple PVs need to be created in batches, set this option to multiple device names and separate the names with spaces. - -Example 1: Create PVs based on **/dev/sdb** and **/dev/sdc**. - -```bash -pvcreate /dev/sdb /dev/sdc -``` - -Example 2: Create PVs based on **/dev/sdb1** and **/dev/sdb2**. - -```bash -pvcreate /dev/sdb1 /dev/sdb2 -``` - -### Viewing a PV - -Run the **pvdisplay** command as the **root** user to view PV information, including PV name, VG to which the PV belongs, PV size, PE size, total number of PEs, number of available PEs, number of allocated PEs, and UUID. - -```bash -pvdisplay [option] devname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-s**: outputs information in short format. - - **-m**: displays the mapping from PEs to LEs. - -- _devname_: indicates the device corresponding to the PV to be viewed. If no PVs are specified, information about all PVs is displayed. - -Example: Run the following command to display the basic information about the PV **/dev/sdb**: - -```bash -pvdisplay /dev/sdb -``` - -### Modifying PV Attributes - -Run the **pvchange** command as the **root** user to modify the attributes of a PV. - -```bash -pvchange [option] pvname ... -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-u**: generates a new UUID. - - **-x**: indicates whether PE allocation is allowed. - -- _pvname_: specifies the name of the device corresponding to the PV to be modified. If multiple PVs need to be modified in batches, set this option to multiple device names and separate the names with spaces. - -Example: Run the following command to prohibit PEs on the PV **/dev/sdb** from being allocated. Running `pvdisplay` for a PV that is not added to a VG will return the **Allocatable** attribute with the value **NO**. You need to add the PV to a VG before you can change the attribute. - -```bash -pvchange -x n /dev/sdb -``` - -### Deleting a PV - -Run the **pvremove** command as the **root** user to delete a PV. - -```bash -pvremove [option] pvname ... -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-f**: forcibly deletes a PV without user confirmation. - - **-y**: answers yes to all questions. - -- _pvname_: specifies the name of the device corresponding to the PV to be deleted. If multiple PVs need to be deleted in batches, set this option to multiple device names and separate the names with spaces. - -Example: Run the following command to delete the PV **/dev/sdb**. If the PV has been added to a VG, you need to delete the VG or remove the PV from the VG in advance. - -```bash -pvremove /dev/sdb -``` - -## Managing VGs - -### Creating a VG - -Run the **vgcreate** command as the **root** user to create a VG. - -```bash -vgcreate [option] vgname pvname ... -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-l**: specifies the maximum number of LVs that can be created on the VG. - - **-p**: specifies the maximum number of PVs that can be added to the VG. - - **-s**: specifies the PE size of a PV in the VG. - -- _vgname_: name of the VG to be created. -- _pvname_: name of the PV to be added to the VG. - -Example: Run the following command to create VG **vg1** and add the PVs **/dev/sdb** and **/dev/sdc** to the VG. - -```bash -vgcreate vg1 /dev/sdb /dev/sdc -``` - -### Viewing a VG - -Run the **vgdisplay** command as the **root** user to view VG information. - -```bash -vgdisplay [option] [vgname] -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-s**: outputs information in short format. - - **-A**: displays only attributes of active VGs. - -- _vgname_: name of the VG to be viewed. If no VGs are specified, information about all VGs is displayed. - -Example: Run the following command to display the basic information about VG **vg1**: - -```bash -vgdisplay vg1 -``` - -### Modifying VG Attributes - -Run the **vgchange** command as the **root** user to modify the attributes of a VG. - -```bash -vgchange [option] vgname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-a**: sets the active status of the VG. - -- _vgname_: name of the VG whose attributes are to be modified. - -Example: Run the following command to change the status of **vg1** to active. - -```bash -vgchange -ay vg1 -``` - -### Extending a VG - -Run the **vgextend** command as the **root** user to dynamically extend a VG. In this way, the VG size is extended by adding PVs to the VG. - -```bash -vgextend [option] vgname pvname ... -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **dev**: debugging mode. - - **-t**: test only. - -- _vgname_: name of the VG whose size is to be extended. -- _pvname_: name of the PV to be added to the VG. - -Example: Run the following command to add PV **/dev/sdb** to VG **vg1**: - -```bash -vgextend vg1 /dev/sdb -``` - -### Shrinking a VG - -Run the **vgreduce** command as the **root** user to delete PVs from a VG to reduce the VG size. A VG must contain at least one PV. - -```bash -vgreduce [option] vgname pvname ... -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-a**: If no PVs are specified in the command, all empty PVs are deleted. - - **\-\-removemissing**: deletes lost PVs in the VG to restore the VG to the normal state. - -- _vgname_: name of the VG to be shrunk. -- _pvname_: name of the PV to be deleted from the VG. - -Example: Run the following command to remove PV **/dev/sdb2** from VG **vg1**: - -```bash -vgreduce vg1 /dev/sdb2 -``` - -### Deleting a VG - -Run the **vgremove** command as the **root** user to delete a VG. - -```bash -vgremove [option] vgname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-f**: forcibly deletes a VG without user confirmation. - -- _vgname_: name of the VG to be deleted. - -Example: Run the following command to delete VG **vg1**. - -```bash -vgremove vg1 -``` - -## Managing LVs - -### Creating an LV - -Run the **lvcreate** command as the **root** user to create an LV. - -```bash -lvcreate [option] vgname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-L**: specifies the size of the LV in kKmMgGtT. - - **-l**: specifies the size of the LV \(number of LEs\). - - **-n**: specifies the name of the LV to be created. - - **-s**: creates a snapshot. - -- _vgname_: name of the VG to be created. - -Example 1: Run the following command to create a 10 GB LV in VG **vg1**. The default LV name is **lvo10**. - -```bash -lvcreate -L 10G vg1 -``` - -Example 2: Run the following command to create a 200 MB LV in VG **vg1** and name the LV **lv1**. - -```bash -lvcreate -L 200M -n lv1 vg1 -``` - -### Viewing an LV - -Run the **lvdisplay** command as the **root** user to view the LV information, including the size of the LV, its read and write status, and snapshot information. - -```bash -lvdisplay [option] [lvname] -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-v**: displays the mapping from LEs to PEs. - -- _lvname_: device file corresponding to the LV whose attributes are to be displayed. If this option is not set, attributes of all LVs are displayed. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Device files corresponding to LVs are stored in the VG directory. For example, if LV **lv1** is created in VG **vg1**, the device file corresponding to **lv1** is **/dev/vg1/lv1**. - -Example: Run the following command to display the basic information about LV **lv1**: - -```bash -lvdisplay /dev/vg1/lv1 -``` - -### Adjusting the LV Size - -Run the **lvresize** command as the **root** user to increase or reduce the size of an LVM LV. This may cause data loss. Therefore, exercise caution when running this command. - -```bash -lvresize [option] vgname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-L**: specifies the size of the LV in kKmMgGtT. - - **-l**: specifies the size of the LV \(number of LEs\). - - **-f**: forcibly adjusts the size of the LV without user confirmation. - -- _lvname_: name of the LV to be adjusted. - -Example 1: Run the following command to increase the size of LV **/dev/vg1/lv1** by 200 MB. - -```bash -lvresize -L +200 /dev/vg1/lv1 -``` - -Example 2: Run the following command to reduce the size of LV **/dev/vg1/lv1** by 200 MB. - -```bash -lvresize -L -200 /dev/vg1/lv1 -``` - -### Extending an LV - -Run the **lvextend** command as the **root** user to dynamically extend the size of an LV online without interrupting the access of applications to the LV. - -```bash -lvextend [option] lvname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-L**: specifies the size of the LV in kKmMgGtT. - - **-l**: specifies the size of the LV \(number of LEs\). - - **-f**: forcibly adjusts the size of the LV without user confirmation. - -- _lvname_: device file of the LV whose size is to be extended. - -Example: Run the following command to increase the size of LV **/dev/vg1/lv1** by 100 MB. - -```bash -lvextend -L +100M /dev/vg1/lv1 -``` - -### Shrinking an LV - -Run the **lvreduce** command as the **root** user to reduce the size of an LV. This may delete existing data on the LV. Therefore, confirm whether the data can be deleted before running the command. - -```bash -lvreduce [option] lvname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-L**: specifies the size of the LV in kKmMgGtT. - - **-l**: specifies the size of the LV \(number of LEs\). - - **-f**: forcibly adjusts the size of the LV without user confirmation. - -- _lvname_: device file of the LV whose size is to be extended. - -Example: Run the following command to reduce the space of LV **/dev/vg1/lvl** by 100 MB: - -```bash -lvreduce -L -100M /dev/vg1/lv1 -``` - -### Deleting an LV - -Run the **lvremove** command as the **root** user to delete an LV. If the LV has been mounted by running the **mount** command, you need to run the **umount** command to unmount the LV before running the **lvremove** command. - -```bash -lvremove [option] lvname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-f**: forcibly deletes an LV without user confirmation. - -- _lvname_: device name of the LV to be deleted. - -Example: Run the following command to delete LV **/dev/vg1/lv1**. - -```bash -lvremove /dev/vg1/lv1 -``` - -## Creating and Mounting a File System - -After creating an LV, you need to create a file system on the LV and mount the file system to the corresponding directory. - -### Creating a File System - -Run the **mkfs** command as the **root** user to create a file system. - -```bash -mkfs [option] lvname -``` - -In the preceding information: - -- _option_: command parameter options. Common parameter options are as follows: - - **-t**: specifies the type of the Linux file system to be created, such as **ext2**, **ext3**, and **ext4**. The default type is **ext2**. - -- _lvname_: name of the LV device file corresponding to the file system to be created. - -Example: Run the following command to create the **ext4** file system on LV **/dev/vg1/lv1**: - -```bash -mkfs -t ext4 /dev/vg1/lv1 -``` - -### Manually Mounting a File System - -The file system that is manually mounted is not valid permanently. It does not exist after the OS is restarted. - -Run the **mount** command as the **root** user to mount a file system. - -```bash -mount lvname mntpath -``` - -In the preceding information: - -- _lvname_: name of the LV device file corresponding to the file system to be mounted. -- _mntpath_: mount path. - -Example: Run the following command to mount LV **/dev/vg1/lv1** to the directory **/mnt/data**. - -```bash -mount /dev/vg1/lv1 /mnt/data -``` - -### Automatically Mounting a File System - -A file system that is automatically mounted does not exist after the OS is restarted. You need to manually mount the file system again. If you perform the following steps as the **root** user after manually mounting the file system, the file system can be automatically mounted after the OS is restarted. - -1. Run the **blkid** command to query the UUID of an LV. The following uses LV **/dev/vg1/lv1** as an example: - - ```bash - blkid /dev/vg1/lv1 - ``` - - Check the command output. It contains the following information in which _uuidnumber_ is a string of digits, indicating the UUID, and _fstype_ indicates the file system type. - - /dev/vg1/lv1: UUID=" _uuidnumber_ " TYPE=" _fstype_ " - -2. Run the **vi /etc/fstab** command to edit the **fstab** file and add the following content to the end of the file: - - ```vim - UUID=uuidnumber mntpath fstype defaults 0 0 - ``` - - In the preceding information: - - - Column 1: indicates the UUID. Enter _uuidnumber_ obtained in [1](#li65701520154311). - - Column 2: indicates the mount directory of the file system. Replace _mntpath_ with the actual value. - - Column 3: indicates the file system format. Enter _fstype_ obtained in [1](#li65701520154311). - - Column 4: indicates the mount option. In this example, **defaults** is used. - - Column 5: indicates the backup option. Enter either **1** \(the system automatically backs up the file system\) or **0** \(the system does not back up the file system\). In this example, **0** is used. - - Column 6: indicates the scanning option. Enter either **1** \(the system automatically scans the file system during startup\) or **0** \(the system does not scan the file system\). In this example, **0** is used. - -3. Verify the automatic mounting function. - 1. Run the **umount** command to unmount the file system. The following uses LV **/dev/vg1/lv1** as an example: - - ```bash - umount /dev/vg1/lv1 - ``` - - 2. Run the following command to reload all content in the **/etc/fstab** file: - - ```bash - mount -a - ``` - - 3. Run the following command to query the file system mounting information \(**/mnt/data** is used as an example\): - - ```bash - mount | grep /mnt/data - ``` - - Check the command output. If the command output contains the following information, the automatic mounting function takes effect: - - ```text - /dev/vg1/lv1 on /mnt/data - ``` diff --git a/docs/en/docs/Administration/memory-management.md b/docs/en/docs/Administration/memory-management.md deleted file mode 100644 index 7b1e4947f988016c63acf2247594934267d83f57..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/memory-management.md +++ /dev/null @@ -1,772 +0,0 @@ -# etmem for Tiered Memory Expansion - -## Introduction - -The development of CPU computing power - particularly lower costs of ARM cores - makes memory cost and capacity become the core frustration that restricts business costs and performance. Therefore, the most pressing issue is how to save memory cost and how to expand memory capacity. - -etmem is a tiered memory expansion technology that uses DRAM+memory compression/high-performance storage media to form tiered memory storage. Memory data is tiered, and cold data is migrated from memory media to high-performance storage media to release memory space and reduce memory costs. - -The tools provided by the etmem software package include the etmem client and the etmemd server. etmemd runs continuously after being launched and implements functions such as recognition and elimination of cold and hot memory of target processes. etmem runs once when called and controls etmemd to respond with different operations based on different command parameters. - -## Compilation - -1. Download the etmem source code. - - ```shell - git clone https://gitee.com/openeuler/etmem.git - ``` - -2. Install the compilation and running dependency. The compilation and running of etmem depend on the libboundscheck component. - - Install the dependency: - - ```bash - yum install libboundscheck - ``` - - Use the `rpm` command to check if the package is installed: - - ```bash - rpm -qi libboundscheck - ``` - -3. Build source code. - - ```bash - cd etmem - mkdir build - cd build - cmake .. - make - ``` - -## Precautions - -### Dependencies - -As a memory expansion tool, etmem needs to rely on kernel features. To identify memory access conditions and support the active writing of memory into the swap partition to achieve the requirement of vertical memory expansion, etmem needs to insert the **etmem_scan** and **etmem_swap** modules at runtime: - -```bash -modprobe etmem_scan -modprobe etmem_swap -``` - -### Restrictions - -The etmem process requires root privileges. The root user has the highest system privileges. When using the root user to perform operations, strictly follow the operation instructions to avoid system management and security risks. - -### Constraints - -- The client and server of etmem must be deployed on the same server. Cross-server communication is not supported. -- etmem can scan target processes whose process name is less than or equal to 15 characters. Supported characters in process names are letters, numbers, periods (.), slashes (/), hyphens (-), and underscores (\_). -- When AEP media is used for memory expansion, it relies on the system being able to correctly recognize the AEP device and initialize the device as a NUMA node. Additionally, the **vm_flags** field in the configuration file can only be configured as **ht**. -- The private commands of the engine are only valid for the corresponding engine and tasks under the engine, such as `showhostpages` and `showtaskpages` supported by cslide. -- In a third-party policy implementations, **fd** in the `eng_mgt_func` interface cannot be written with the **0xff** and **0xfe** characters. -- Multiple different third-party policy dynamic libraries, distinguished by **eng_name** in the configuration file, can be added within a project. -- Concurrent scanning of the same process is prohibited. -- Using the **/proc/xxx/idle_pages** and **/proc/xxx/swap_pages** files is prohibited when **etmem_scan** and **etmem_swap** modules are not loaded. -- The etmem configuration file requires the owner to be the root user, with permissions of 600 or 400. The size of the configuration file cannot exceed 10 MB. -- When etmem injects a third-party policy, the **so** of the third-party policy requires the owner to be the root user, with permissions of 500 or 700. - -## Instructions - -### etmem Configuration Files - -Before running the etmem process, the administrator needs to decide the memory of which processes needs to be expanded, configure the process information in the etmem configuration files, and configure information such as the memory scanning cycle, scanning times, and memory hot and cold thresholds. - -The configuration file examples are included in the source package and stored in the **/etc/etmem** directory. There are three example files: - -```text -/etc/etmem/cslide_conf.yaml -/etc/etmem/slide_conf.yaml -/etc/etmem/thirdparty_conf.yaml -``` - -Contents of the files are as follows: - -```sh -#slide engine example -#slide_conf.yaml -[project] -name=test -loop=1 -interval=1 -sleep=1 -sysmem_threshold=50 -swapcache_high_vmark=10 -swapcache_low_vmark=6 - -[engine] -name=slide -project=test - -[task] -project=test -engine=slide -name=background_slide -type=name -value=mysql -T=1 -max_threads=1 -swap_threshold=10g -swap_flag=yes - -#cslide engine example -#cslide_conf.yaml -[engine] -name=cslide -project=test -node_pair=2,0;3,1 -hot_threshold=1 -node_mig_quota=1024 -node_hot_reserve=1024 - -[task] -project=test -engine=cslide -name=background_cslide -type=pid -name=23456 -vm_flags=ht -anon_only=no -ign_host=no - -#Third-party engine example -#thirdparty_conf.yaml -[engine] -name=thirdparty -project=test -eng_name=my_engine -libname=/usr/lib/etmem_fetch/my_engine.so -ops_name=my_engine_ops -engine_private_key=engine_private_value - -[task] -project=test -engine=my_engine -name=background_third -type=pid -value=12345 -task_private_key=task_private_value -``` - -Fields in the configuration files are described as follows: - -| Item | Description | Mandatory | Contains Parameters | Parameter Range | Example | -|-----------|---------------------|------|-------|------------|-----------------------------------------------------------------| -| \[project\] | Beginning identifier of the project public configuration section | No | No | N/A | Beginning identifier of the project parameters, indicating that the parameters below are within the range of the project section until another \[xxx\] or the end of the file -| name | Name of the project | Yes | Yes | String of up to 64 characters | Specifies that the project, engine and task need to be mounted to the specified project during configuration. | -| loop | Number of loops for memory scan | Yes | Yes | 1~120 | loop=3 // Memory is scanned 3 times. | -| interval | Time interval for each memory scan | Yes | Yes | 1~1200 | interval=5 // The interval is 5s. | -| sleep | Time interval for each memory scan+operation | Yes | Yes | 1~1200 | sleep=10 //The interval is 10s | -| sysmem_threshold| Memory swapping threshold. This is a slide engine configuration item. | No | Yes | 0~100 | sysmem_threshold=50 // When available memory is less than 50%, etmem swaps out memory.| -| swapcache_high_wmark| High watermark of swapcache. This is a slide engine configuration item. | No | Yes | 1~100 | swapcache_high_wmark=5 // swapcache can be up to 5% of the system memory. If this ratio is reached, etmem triggers swapcache recycling.
Note: swapcache_high_wmark must be greater than swapcache_low_wmark.| -| swapcache_low_wmark| Low watermark of swapcache. This is a slide engine configuration item. | No | Yes | \[1~swapcache_high_wmark\) | swapcache_low_wmark=3 //When swapcache recycling is triggered, the system recycles the swapcache memory occupancy to less than 3%.| -| \[engine\] | Beginning identifier of the engine public configuration section | No | No | N/A | Beginning identifier of the engine parameters, indicating that the parameters below are within the range of the engine section until another \[xxx\] or the end of the file | -| project | project to which the engine belongs | Yes | Yes | String of up to 64 characters | If a project named test exists, the item can be **project=test**. | -| engine | engine to which the engine belongs | Yes | Yes | slide/cslide/thridparty | Specifies the policy to use (**slide**, **cslide**, or **thirdparty**) | -| node_pair | Node pair of AEP and DRAM. This is a cslide engine configuration item. | Yes when **engine** is **cslide** | Yes | Pair the node numbers of AEP and DRAM. Separate AEP and DRAM using a comma, and separate each pair using semicolons. | node_pair=2,0;3,1 | -| hot_threshold | Threshold of hot memory watermark. This is a cslide engine configuration item. | Yes when **engine** is **cslide** | Yes | An integer greater than or equal to 0 and less than or equal to INT_MAX | hot_threshold=3 // Memory with less than 3 accesses will be recognized as cold memory. | -|node_mig_quota|Maximum one-way flow when DRAM and AEP migrate mutually. This is a cslide engine configuration item.|Yes when **engine** is **cslide**|Yes|An integer greater than or equal to 0 and less than or equal to INT_MAX|node_mig_quota=1024 // The unit is MB. A maximum of 1024 MB can be migrated from AEP to DRAM or from DRAM to AEP each time.| -|node_hot_reserve| Size of the reserved space for hot memory in DRAM. This is a cslide engine configuration item.|Yes when **engine** is **cslide**|Yes|An integer greater than or equal to 0 and less than or equal to INT_MAX|node_hot_reserve=1024 //The unit is MB. When the hot memory of all VMs is greater than this configuration value, the hot memory will also be migrated to AEP.| -|eng_name|Name of the engine for mounting by task. This is a thirdparty engine configuration item.|Yes when **engine** is **thirdparty**|Yes|String of up to 64 characters|eng_name=my_engine // When mounting a task to the third-party policy engine, specify **engine=my_engine** in the task.| -|libname|Absolute path to the dynamic library of the third-party policy. This is a thirdparty engine configuration item.|Yes when **engine** is **thirdparty**|Yes|String of up to 256 characters|libname=/user/lib/etmem_fetch/code_test/my_engine.so| -|ops_name|Name of the operator in the dynamic library of the third-party policy. This is a third-party engine configuration item.|Yes when **engine** is **thirdparty**|Yes|String of up to 256 characters|ops_name=my_engine_ops // Name of the struct for the third-party policy implementation interface| -|engine_private_key|Reserved item for third-party policies to parse private parameters by themselves. This is a thirdparty engine configuration item.|No|No|Restrict according to the third-party policy's private parameters.|Configure the private engine parameters according to the third-party policy.| -| \[task\] | Beginning identifier of the task public configuration section | No | No | N/A | Beginning identifier of the task parameters, indicating that the parameters below are within the range of the project section until another \[xxx\] or the end of the file | -| project | project to which the task belongs | Yes | Yes | String of up to 64 characters | If a project named test exists, the item can be **project=test**. | -| engine | engine to which the task belongs | Yes | Yes | String of up to 64 characters | Name of the engine to which the task belongs | -| name | Name of the task | Yes | Yes | String of up to 64 characters | name=background1 // The name of the task is background1. | -| type | How the target process is identified | Yes | Yes | pid/name | **pid** specifies to identify by PID. **name** specifies to identify by name. | -| value | Value to be identified for the target process | Yes | Yes | Actual PID/name | Used with **type** to specify the PID or name of the target process. Ensure the configuration is correct and unique. | -| T | Threshold of hot memory watermark. This is a slide engine configuration item. | Yes when **engine** is **slide** | Yes | 0~loop * 3 | T=3 // Memory with less than 3 accesses will be recognized as cold memory. | -| max_threads | Maximum number of threads in the etmem internal thread pool, with each thread handling a process/subprocess memory scan+operation task. This is a slide engine configuration item. | No | Yes | 1~2 * number of cores + 1, the default value is 1. | Controls the number of internal processing threads for the etmemd server without external representation. When the target process has multiple child processes, the larger the item value, the more concurrent executions, but the more resources consumed. | -| vm_flags | Flag of the VMA to be scanned. This is a cslide engine configuration item. | No | Yes | String of up to 256 characters, with different flags separated by spaces. | vm_flags=ht // Scans memory of the VMA whose flag is ht. | -| anon_only | Scans anonymous pages only. This is a cslide engine configuration item. | No | Yes | yes/no | anon_only=no | -| ign_host | Ignores page table scan information on the host. This is a cslide engine configuration item. | No | Yes | yes/no | ign_host=no | -| task_private_key | Reserved for a task of a third-party policy to parse private parameters. This is a third-party engine configuration item. | No | No |Restrict according to the third-party policy's private parameters.|Configure the private task parameters according to the third-party policy.| -| swap_threshold | Process memory swapping threshold. This is a slide engine configuration item. | No | Yes | Absolute value of memory available to the process | swap_threshold=10g // Memory swapping will not be triggered when the process memory is less than 10 GB.
Currently, the unit can only be **g** or **G**. This item is used with **sysmem_threshold**. When system memory is lower than **sysmem_threshold**, memory of processes in the allowlist is checked. | -| swap_flag| Enables process memory swapping. This is a slide engine configuration item. | No | Yes | yes/no | swap_flag=yes | - -### Starting etmemd - -Modify related configuration files before using etmem services. After being started, etmemd stays in the system to operate the memory of the target processes.To start etmemd, you can either run the `etmemd` command or configure a service file for `systemctl` to start etmemd. The latter requires the `mode-systemctl` option. - -#### How to Use - -Run the following command to start etmemd: - -```bash -etmemd -l 0 -s etmemd_socket -``` - -or - -```bash -etmemd --log-level 0 --socket etmemd_socket -``` - -The `0` parameter of option `-l` and the `etmemd_socket` parameter of option `-s` are user-defined parameters and are described as follows. - -#### Command Parameters - -| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | -| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ | -| -l or \-\-log-level | etmemd log level | No | Yes | 0~3 | 0: debug level
1: info level
2: warning level
3: error level
Logs whose levels are higher than the specified value are printed to **/var/log/message**. | -| -s or \-\-socket | Socket listened by etmemd to interact with the client | Yes | Yes | String of up to 107 characters | Socket listened by etmemd | -| -m or \-\-mode-systemctl| Starts the etmemd service through systemctl | No| No| N/A| The `-m` option needs to be specified in the service file.| -| -h or \-\-help | Prints help information | No | No | N/A | This option prints help information and exit. | - -### Adding and Deleting Projects, Engines, and Tasks Using the etmem Client - -#### Scenario - -1. The administrator adds a project, engine, or task to etmem (a project can contain multiple etmem engines, an engine can contain multiple tasks). - -2. The administrator deletes an existing etmem project, engine, or task (all tasks in a project is stopped before the project is deleted). - -#### Usage - -When etmemd is running normally, run `etmem` with the `obj` option to perform addition and deletion. etmem automatically identifies projects, engines, or tasks according to the content of the configuration file. - -- Add an object. - - ```bash - etmem obj add -f /etc/etmem/slide_conf.yaml -s etmemd_socket - ``` - - or - - ```bash - etmem obj add --file /etc/etmem/slide_conf.yaml --socket etmemd_socket - ``` - -- Delete an object. - - ```bash - etmem obj del -f /etc/etmem/slide_conf.yaml -s etmemd_socket - ``` - - or - - ```bash - etmem obj del --file /etc/etmem/slide_conf.yaml --socket etmemd_socket - -#### Command Parameters - -| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | -| ------------ | ------------------------------------------------------------ | -------- | ---------- | -------------------------------------------------------- | -| -f or \-\-file | Specifies the configuration file of the object. | Yes | Yes | Specify the path. | -| -s or \-\-socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes | Yes | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. | - -### Querying, Starting, and Stopping Projects Using the etmem Client - -#### Scenario - -A project is added by using `etmem obj add` and is not deleted by using `etmem obj del`. In this case, the project can be started and stopped. - -1. The administrator starts an added project. - -2. The administrator stops a started project. - -A started project will be stopped if the administrator run `obj del` to delete the project. - -#### Usage - -Added projects can be started and stopped by using `etmem project` commands. - -- Query a project. - - ```bash - etmem project show -n test -s etmemd_socket - ``` - - or - - ```bash - etmem project show --name test --socket etmemd_socket - ``` - -- Start a project. - - ```bash - etmem project start -n test -s etmemd_socket - ``` - - or - - ```bash - etmem project start --name test --socket etmemd_socket - ``` - -- Stop a project. - - ```bash - etmem project stop -n test -s etmemd_socket - ``` - - or - - ```bash - etmem project stop --name test --socket etmemd_socket - ``` - -- Print help information. - - ```bash - etmem project help - ``` - -#### Command Parameters - -| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | -| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ | -| -n or \-\-name | Name of the project | Yes | Yes | The project name corresponds to the configuration file. | -| -s or \-\-socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes | Yes | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. | - -### Specifying System Memory Swapping Threshold and Process Memory Swapping Using the etmem Client - -Only slide policies support private features. - -- Process or system memory swapping threshold - -It is necessary to consider the timing of etmem memory swapping for optimal performance. Memory swapping is not performed when the system has enough available memory or a process occupies a low amount of memory. Memory swapping threshold can be specified for the system and processes. - -- Process memory swapping - -The memory of I/O latency-sensitive service processes should not be swapped in the storage scenario. In this case, you can disable memory swapping for certain services. - -Process and system memory swapping thresholds and process memory swapping are controlled by the **sysmem_threshold**, **swap_threshold**, and **swap_flag** parameters in the configuration file. For details, see [etmem Configuration Files](#etmem-configuration-files). - -```sh -#slide_conf.yaml -[project] -name=test -loop=1 -interval=1 -sleep=1 -sysmem_threshold=50 - -[engine] -name=slide -project=test - -[task] -project=test -engine=slide -name=background_slide -type=name -value=mysql -T=1 -max_threads=1 -swap_threshold=10g -swap_flag=yes -``` - -#### System Memory Swapping Threshold - -The **sysmem_threshold** parameter is used to set system memory swapping threshold. The value range for **sysmem_threshold** is 0 to 100. If **sysmem_threshold** is set in the configuration file, etmem will swap memory when system memory is lower than **sysmem_threshold**. - -For example: - -1. Compose the configuration according to the example. Set **sysmem_threshold** to **20**. -2. Start the server, add a project to the server, and start the project. - - ```bash - etmemd -l 0 -s monitor_app & - etmem obj add -f etmem_config -s monitor_app - etmem project start -n test -s monitor_app - etmem project show -s monitor_app - ``` - -3. Observe the memory swapping results. etmem swaps memory only when the system available memory is less than 20%. - -#### Process Memory Swapping Threshold - -The **swap_threshold** parameter is used to set process memory swapping threshold. **swap_threshold** is the absolute memory usage of a process in the format of \**g/G**. If **swap_threshold** is set in the configuration file, etmem will not swap memory of the process when the process memory usage is lower then **swap_threshold**. - -For example: - -1. Compose the configuration according to the example. Set **swap_threshold** to **5g**. -2. Start the server, add a project to the server, and start the project. - - ```bash - etmemd -l 0 -s monitor_app & - etmem obj add -f etmem_config -s monitor_app - etmem project start -n test -s monitor_app - etmem project show -s monitor_app - ``` - -3. Observe the memory swapping results. etmem swaps memory only when the process memory usage reaches 5 GB. - -#### Process Memory Swapping - -The **swap_flag** parameter is used to enable the process memory swapping feature. The value of **swap_flag** can be **yes** or **no**. If **swap_flag** is **no** or not configured, etmem swaps memory normally. If **swap_flag** is **yes**, etmem swaps memory of the specified processes only. - -For example: - -1. Compose the configuration according to the example. Set **swap_flag** to **yes**. -2. Flag the memory to be swapped for the service process. - - ```bash - madvise(addr_start, addr_len, MADV_SWAPFLAG) - ``` - -3. Start the server, add a project to the server, and start the project. - - ```bash - etmemd -l 0 -s monitor_app & - etmem obj add -f etmem_config -s monitor_app - etmem project start -n test -s monitor_app - etmem project show -s monitor_app - ``` - -4. Observe the memory swapping results. Only the flagged memory is swapped. Other memory is retained in the DRAM. - -In the process memory page swapping scenario, `ioctl` is added to the original scan interface file **idle_pages** to ensure that VMAs that are not flagged do not participate in memory scanning and swapping. - -Scan management interface: - -- Function prototype - - ```c - ioctl(fd, cmd, void *arg); - ``` - -- Input parameters - 1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle_pages using the open system call - 2. cmd: controls the scan actions. The following values are supported: - VMA_SCAN_ADD_FLAGS: adds VMA memory swapping flags to scan only flagged VMAs - VMA_SCAN_REMOVE_FLAGS: removes added VMA memory swapping flags - 3. args: integer pointer parameter used to pass a specific mask. The following value is supported: - VMA_SCAN_FLAG: Before the etmem_scan.ko module starts scanning, the walk_page_test interface is called to determine whether the VMA address meets the scanning requirements. If this flag is set, only the VMA addresses that contain specific swap flags are scanned. - -- Return values - 1. 0 if the command succeeds - 2. Other values if the command fails - -- Precautions - Unsupported flags are ignored and do not return errors. - -### Specifying swapcache Memory Recycling Instructions Using the etmem Client - -The user-mode etmem initiates a memory elimination and recycling operation and interacts with the kernel-mode memory recycling module through the **write procfs** interface. The memory recycling module parses the virtual address sent from the user space, obtains the page corresponding to the address, and calls the native kernel interface to swap and recycle the memory corresponding to the page. During memory swapping, swapcache will use some system memory. To further save memory, the swapcache memory recycling feature is added. - -Add **swapcache_high_wmark** and **swapcache_low_wmark** parameters to use the swapcache memory recycling feature. - -- **swapcache_high_wmark**: High system memory water of swapcache -- **swapcache_low_wmark**: Low system memory water of swapcache - -After etmem swaps memory, it checks the swapcache memory occupancy. When the occupancy exceeds the high watermark, an `ioctl` instruction will be issued through **swap_pages** to trigger the swapcache memory recycling and stop when swapcache memory occupancy reaches the low watermark. - -An example configuration file is as follows. For details, see [etmem Configuration Files](#etmem-configuration-files). - -```sh -#slide_conf.yaml -[project] -name=test -loop=1 -interval=1 -sleep=1 -swapcache_high_vmark=5 -swapcache_low_vmark=3 - -[engine] -name=slide -project=test - -[task] -project=test -engine=slide -name=background_slide -type=name -value=mysql -T=1 -max_threads=1 -``` - -During memory swapping, swapcache memory needs to be recycled to further save memory. An `ioctl` interface is added to the original memory swap interface to configure swapcache watermarks and swapcache memory recycling. - -- Function prototype - - ```c - ioctl(fd, cmd, void *arg); - ``` - -- Input parameters - 1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle_pages using the open system call - 2. cmd: controls the scan actions. The following values are supported: - RECLAIM_SWAPCACHE_ON: enables swapcache memory swapping - RECLAIM_SWAPCACHE_OFF: disables swapcache memory swapping - SET_SWAPCACHE_WMARK: configures swapcache memory watermarks - 3. args: integer pointer parameter used to pass a specific mask. The following value is supported: - Parameters that pass the values of swapcache watermarks - -- Return values - 1. 0 if the command succeeds - 2. Other values if the command fails - -- Precautions - Unsupported flags are ignored and do not return errors. - -### Executing Private Commands and Functions Using the etmem Client - -Only the cslide policy support private commands. - -- `showtaskpages` -- `showhostpages` - -For engines and tasks of engines that use the cslide policy, you can run the commands above to query the page access of tasks and the usage of system huge pages on the host of VMs. - -For example: - -```bash -etmem engine showtaskpages <-t task_name> -n proj_name -e cslide -s etmemd_socket - -etmem engine showhostpages -n proj_name -e cslide -s etmemd_socket -``` - -**Note**: `showtaskpages` and `showhostpages` are supported by the cslide policy only. - -#### Command Parameters - -| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | -| ------------ | ------------------------------------------------------------ | -------- | ---------- | -------------------------------------------------------- | -|-n or \-\-proj_name|Name of the project| Yes|Yes|Name of an existing project to run| -| -s or \-\-socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes | Yes | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. | -|-e or \-\-engine|Name of the engine to run|Yes|Yes|Name of an existing engine to run| -|-t or \-\-task_name|Name of the task to run|No|Yes|Name of an existing task to run| - -### Enabling and Disabling Kernel Swap - -When etmem swaps memory to the drive to expand memory, you can choose to enable the kernel swap feature. You can disable the native kernel swap mechanism to void swapping memory undesirably, resulting in problems with user-mode processes. - -A sys interface is provided to implement such control. A **kobj** named **kernel_swap_enable** is created in **/sys/kernel/mm/swap** to enable and disable kerne swap. The default value of **kernel_swap_enable** is **true**. - -For example: - -```sh -# Enable kernel swap -echo true > /sys/kernel/mm/swap/kernel_swap_enable -or -echo 1 > /sys/kernel/mm/swap/kernel_swap_enable - -# Disable kernel swap -echo false > /sys/kernel/mm/swap/kernel_swap_enable -or -echo 0 > /sys/kernel/mm/swap/kernel_swap_enable - -``` - -### Starting etmem Upon System Startup - -#### Scenario - -You can configure the systemd configuration file to run etmemd as a forking service of systemd. - -#### Usage - -Compose a service configuration file to start etmemd with the `-m` option. For example: - -```bash -etmemd -l 0 -s etmemd_socket -m -``` - -#### Command Parameters - -| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | -| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ | -| -l or \-\-log-level | etmemd log level | No | Yes | 0~3 | 0: debug level
1: info level
2: warning level
3: error level
Logs whose levels are higher than the specified value are printed to **/var/log/message**. | -| -s or \-\-socket | Socket listened by etmemd to interact with the client | Yes | Yes | String of up to 107 characters | Socket listened by etmemd | -| -m or \-\-mode-systemctl| Starts the etmemd service through systemctl | No| No| N/A| The `-m` option needs to be specified in the service file.| -| -h or \-\-help | Prints help information | No | No | N/A | This option prints help information and exit. | - -### Supporting Third-party Memory Expansion Policies With etmem - -#### Scenario - -etmem provides third-party memory expansion policy registration and module scanning dynamic library and can eliminate memory according to third-party policies. - -You can use the module scanning dynamic library to implement the interface of the struct required for connecting to etmem. - -#### Usage - -To use a third-party memory expansion elimination policy, perform the following steps: - -1. Invoke the scanning interface of the module as required. - -2. Implement the interfaces using the function template provided by the etmem header file and encapsulate them into a struct. - -3. Build a dynamic library of the third-party memory expansion elimination policy. - -4. Specify the **thirdparty** engine in the configuration file. - -5. Enter the names of the library and the interface struct to the corresponding **task** fields in the configuration file. - -Other steps are similar to those of using other engines. - -Interface struct template: - -```c -struct engine_ops { - -/* Parsing private parameters of the engine. Implement the interface if required, otherwise, set it to NULL. */ - -int (*fill_eng_params)(GKeyFile *config, struct engine *eng); - -/* Clearing private parameters of the engine. Implement the interface if required, otherwise, set it to NULL. */ - -void (*clear_eng_params)(struct engine *eng); - -/* Parsing private parameters of the task. Implement the interface if required, otherwise, set it to NULL. */ - -int (*fill_task_params)(GKeyFile *config, struct task *task); - -/* Clearing private parameters of the task. Implement the interface if required, otherwise, set it to NULL. */ - -void (*clear_task_params)(struct task *tk); - -/* Task starting interface */ - -int (*start_task)(struct engine *eng, struct task *tk); - -/* Task stopping interface */ - -void (*stop_task)(struct engine *eng, struct task *tk); - -/* Allocate PID-related private parameters */ - -int (*alloc_pid_params)(struct engine *eng, struct task_pid **tk_pid); - -/* Destroy PID-related private parameters */ - -void (*free_pid_params)(struct engine *eng, struct task_pid **tk_pid); - -/* Support for private commands required by the third-party policy. If this interface is not required, set it to NULL */ - -int (*eng_mgt_func)(struct engine *eng, struct task *tk, char *cmd, int fd); - -}; -``` - -External interfaces of the scanning module: - -| Interface |Description| -| ------------ | --------------------- | -| etmemd_scan_init | Initializes the scanning module| -| etmemd_scan_exit | Exits the scanning module| -| etmemd_get_vmas | Gets the VMAs to be scanned| -| etmemd_free_vmas | Releases VMAs scanned by `etmemd_get_vmas`| -| etmemd_get_page_refs | Scans pages in VMAs| -| etmemd_free_page_refs | Release the page access information list obtained by `etmemd_get_page_refs` | - -In the VM scanning scenario, `ioctl` is added to the original scan interface file **idle_pages** to distinguish the EPT scanning granularity and specify whether to ignore page access flags on the hosts. - -In the process memory page swapping scenario, `ioctl` is added to the original scan interface file **idle_pages** to ensure that VMAs that are not flagged do not participate in memory scanning and swapping. - -Scan management interface: - -- Function prototype - - ```c - ioctl(fd, cmd, void *arg); - ``` - -- Input parameters - 1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle_pages using the open system call - 2. cmd: controls the scan actions. The following values are supported: - IDLE_SCAN_ADD_FLAG: adds a scanning flag - IDLE_SCAM_REMOVE_FLAGS: removes a scanning flag - VMA_SCAN_ADD_FLAGS: adds VMA memory swapping flags to scan only flagged VMAs - VMA_SCAN_REMOVE_FLAGS: removes added VMA memory swapping flags - 3. args: integer pointer parameter used to pass a specific mask. The following value is supported: - SCAN_AS_HUGE: scans the pages according to the 2 MB granularity to see if the pages have been accessed when scanning the EPT page table. If this parameter is not set, the granularity will be the granularity of the EPT page table itself. - SCAN_IGN_HUGE: ignores page access flags on the hosts when scanning VMs. - VMA_SCAN_FLAG: Before the etmem_scan.ko module starts scanning, the walk_page_test interface is called to determine whether the VMA address meets the scanning requirements. If this flag is set, only the VMA addresses that contain specific swap flags are scanned. - -- Return values - 1. 0 if the command succeeds - 2. Other values if the command fails - -- Precautions - Unsupported flags are ignored and do not return errors. - -An example configuration file is as follows. For details, see [etmem Configuration Files](#etmem-configuration-files). - -```text -#thirdparty -[engine] - -name=thirdparty - -project=test - -eng_name=my_engine - -libname=/user/lib/etmem_fetch/code_test/my_engine.so - -ops_name=my_engine_ops - -engine_private_key=engine_private_value - -[task] - -project=test - -engine=my_engine - -name=background1 - -type=pid - -value=1798245 - -task_private_key=task_private_value -``` - - **Note**: - -You need to use the module scanning dynamic library to implement the interface of the struct required for connecting to etmem. - -**fd** in the `eng_mgt_func` interface cannot be written with the **0xff** and **0xfe** characters. - -Multiple different third-party policy dynamic libraries, distinguished by **eng_name** in the configuration file, can be added within a project. - -### Help Information of the etmem Client and Server - -Run the following command to print help information of the etmem server: - -```bash -etmemd -h -``` - -or: - -```bash -etmemd --help -``` - -Run the following command to print help information of the etmem client: - -```bash -etmem help -``` - -Run the following command to print help information of project, engine, and task operations: - -```bash -etmem obj help -``` - -Run the following command to print help information of projects: - -```bash -etmem project help -``` - -## How to Contribute - -1. Fork this repository. -2. Create a branch. -3. Commit your code. -4. Create a pull request (PR). diff --git a/docs/en/docs/Administration/overview.md b/docs/en/docs/Administration/overview.md deleted file mode 100644 index b802495f1d8859189e2038d5ee423f9f0f602ca5..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/overview.md +++ /dev/null @@ -1,149 +0,0 @@ -# Memory Management - -## Basic Concepts - -The memory is an important component of a computer, and is used to temporarily store operation data in the CPU and data exchanged with an external memory such as hardware. In particular, a non-uniform memory access architecture (NUMA) is a memory architecture designed for a multiprocessor computer. The memory access time depends on the location of the memory relative to the processor. In NUMA mode, a processor accesses the local memory faster than the non-local memory (the memory is located in another processor or shared between processors). - -## Viewing Memory - -1. **free**: displays the system memory status. - Example: - - ```bash - # Display the system memory status in MB. - free -m - ``` - - The output is as follows: - - ```text - [root@openEuler ~]# free -m - total used free shared buff/cache available - Mem: 2633 436 324 23 2072 2196 - Swap: 4043 0 4043 - ``` - - The fields in the command output are described as follows: - - |Field|Description| - |--|--| - |total|Total memory size.| - |used|Used memory.| - |free|Free memory.| - |shared|Total memory shared by multiple processes.| - |buff/cache|Total number of buffers and caches.| - |available|Estimated available memory to start a new application without swapping.| - -2. **vmstat**: dynamically monitors the system memory and views the system memory usage. - - Example: - - ```bash - # Monitor the system memory and display active and inactive memory. - vmstat -a - ``` - - The output is as follows: - - ```text - [root@openEuler ~]# vmstat -a - procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- - r b swpd free inact active si so bi bo in cs us sy id wa st - 2 0 520 331980 1584728 470332 0 0 0 2 15 19 0 0 100 0 0 - ``` - - In the command output, the field related to the memory is described as follows: - - |Field|Description| - |--|--| - |memory|Memory information.
**-swpd**: usage of the virtual memory, in KB.
**-free**: free memory capacity, in KB.
**-inact**: inactive memory capacity, in KB.
**-active**: active memory capacity, in KB.| - -3. **sar**: monitors the memory usage of the system. - - Example: - - ```bash - # Monitor the memory usage in the sampling period in the system. Collect the statistics every two seconds for three times. - sar -r 2 3 - ``` - - The output is as follows: - - ```text - [root@openEuler ~]# sar -r 2 3 - - 04:02:09 PM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kb - dirty - 04:02:11 PM 332180 2249308 189420 7.02 142172 1764312 787948 11.52 470404 1584924 - 36 - 04:02:13 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 - 36 - 04:02:15 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 - 36 - Average: 332159 2249287 189441 7.03 142172 1764312 787948 11.52 470404 1584924 - 36 - ``` - - The fields in the command output are described as follows: - - |Field|Description| - |--|--| - |kbmemfree|Unused memory space.| - |kbmemused|Used memory space.| - |%memused|Percentage of the used space.| - |kbbuffers|Amount of data stored in the buffer.| - |kbcached|Data access volume in all domains of the system.| - -4. **numactl**: displays the NUMA node configuration and status. - - Example: - - ```bash - # Check the current NUMA configuration. - numactl -H - ``` - - The output is as follows: - - ```text - [root@openEuler ~]# numactl -H - available: 1 nodes (0) - node 0 cpus: 0 1 2 3 - node 0 size: 2633 MB - node 0 free: 322 MB - node distances: - node 0 - 0: 10 - ``` - - The server contains one NUMA node. The NUMA node that contains four cores and 6 GB memory. - The command also displays the distance between NUMA nodes. The further the distance, the higher the latency of cross-node memory accesses, which should be avoided as much as possible. - - **numastat**: displays NUMA node status. - - ```bash - # Check the NUMA node status. - numastat - ``` - - ```text - [root@openEuler ~]# numastat - node0 - numa_hit 5386186 - numa_miss 0 - numa_foreign 0 - interleave_hit 17483 - local_node 5386186 - other_node 0 - ``` - - The the fields in the command output and their meanings are as follows: - - |Field|Description| - |--|--| - |numa_hit|Number of times that the CPU core accesses the local memory on a node.| - |numa_miss|Number of times that the core of a node accesses the memory of other nodes.| - |numa_foreign|Number of pages that were allocated to the local node but moved to other nodes. Each numa_foreign corresponds to a numa_miss event.| - |interleave_hit|Number of pages of the interleave policy that are allocated to this node.| - |local_node|Size of memory that was allocated to this node by processes on this node.| - |other_node|Size of memory that was allocated to other nodes by processes on this node.| diff --git a/docs/en/docs/Administration/process-management.md b/docs/en/docs/Administration/process-management.md deleted file mode 100644 index 74abc6ff58335e3de37f4e3b0ba3da5ea1a09362..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/process-management.md +++ /dev/null @@ -1,349 +0,0 @@ -# Process Management - -The operating system (OS) manages multiple user requests and tasks. In most cases, the OS comes with only one CPU and one main memory, but multiple tier-2 disks and input/output \(I/O\) devices. Therefore, users have to share resources, but it appears to users that they are exclusively occupying resources. The OS places user tasks, OS tasks, mailing, print tasks, and other pending tasks in a queue and schedules the tasks according to predefined rules. This topic describes how the OS manages processes. - - -- [Process Management](#process-management) - - [Viewing Processes](#viewing-processes) - - [who Command](#who-command) - - [ps Command](#ps-command) - - [top Command](#top-command) - - [kill Command](#kill-command) - - [Scheduling a Process](#scheduling-a-process) - - [Using the at Command to Run Processes at the Scheduled Time](#using-the-at-command-to-run-processes-at-the-scheduled-time) - - [Using the cron Service to Run Commands Periodically](#using-the-cron-service-to-run-commands-periodically) - - [Suspending/Resuming a Process](#suspendingresuming-a-process) - - - -## Viewing Processes - -Linux is a multi-task system and needs to get process information during process management. To manage processes, you need to know the number of processes and their statuses. Multiple commands are available to view processes. - -### who Command - -The `who` command is used to display system user information. For example, before running the `talk` command to establish instant communication with another user, you need to run the `who` command to determine whether the target user is online. In another example, the system administrator can run the `who` command to learn what each login user is doing at the current time. The `who` command is widely seen in system administration since it is easy to use and can return a comprehensive set of accurate user information. - -The following is an example output of the `who` command, where system users and their status are displayed: The use of the `who` command is as follows: - -```shell -$ who -admin tty1 Jul 28 15:55 -admin pts/0 Aug 5 15:46 (192.168.0.110) -admin pts/2 Jul 29 19:52 (192.168.0.110) -root pts/3 Jul 30 12:07 (192.168.0.110) -root pts/4 Jul 31 10:29 (192.168.0.144) -root pts/5 Jul 31 14:52 (192.168.0.11) -root pts/6 Aug 6 10:12 (192.168.0.234) -root pts/8 Aug 6 11:34 (192.168.0.234) -``` - -### ps Command - -The **ps** command is the most basic and powerful command to view process information. The ps command is used to display process information, including which processes are running, terminated, resource-hungry, or stay as zombies. - -The `ps` command is the most basic and powerful command to view process information, including which processes are running, terminated, resource-hungry, or stay as zombies. - -A common scenario is to monitor background processes, which do not interact with your screen, keyboard, and other I/O devices. [Table 1](#en-us_topic_0151921029_t34619d964a3d41ad8694189ec383359c) lists the common `ps` command options. - -**Table 1** Common ps command options - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Option

-

Description

-

-e

-

Displays all processes.

-

-f

-

Full output format.

-

-h

-

Hides column headings in the process information.

-

-l

-

Long output format.

-

-w

-

Wide output format.

-

-a

-

Lists all processes on a terminal, including those of other users.

-

-r

-

Lists only running processes.

-

-x

-

Lists all processes without control terminals.

-
- -For example, to list all processes on a terminal, run the following command: - -```shell -$ ps -a - PID TTY TIME CMD -12175 pts/6 00:00:00 bash -24526 pts/0 00:00:00 vsftpd -29478 pts/5 00:00:00 ps -32461 pts/0 1-01:58:33 sh -``` - -### top Command - -Both the `top` and `ps` commands can display a list of currently running processes, but the `top` command allows you to update the displayed list of processes by pressing a button repeatedly. If the `top` command is executed in foreground, it exclusively occupies foreground until it is terminated. The `top` command provides real-time visibility into system processor status. You can sort the list of CPU tasks by CPU usage, memory usage, or task execution time. Extensive display customization, such as choosing the columns or sorting method, can be achieved using interactive commands or the customization file. - -[Figure 1](#en-us_topic_0151921029_f289234fcdbac453796200d80e9889cd1) provides an example output of the `top` command. - -**Figure 1** Example command output -![](./figures/example-command-output.png "example-command-output") - -### kill Command - -The `kill` command is used to terminate a process regardless of whether the process is running in foreground or background. It differs from the combo key **Ctrl+C**, which can terminate only foreground processes. The reason for terminating a background process can be heavy use of CPU resources or deadlock. - -The `kill` command sends a signal to terminate running processes. By default, the `TERM` signal is used, terminating all processes incapable of capturing it. To terminate a process capable of capturing the `TERM` signal, use the `KILL` signal \(signal ID: 9\) instead. - -Two types of syntax of the `kill` command: - -```shell -kill [-s signal | -p] [-a] PID… -kill -l [signal] -``` - -The process ID can be retrieved by running the `ps` command. The `-s` option indicates the signal sent to the specified program. The signal details can be viewed by running the `kill -l` command. The `-p` option indicates the specified process ID. - -For example, to terminate the process whose ID is 1409, run the following command as the **root** user: - -```shell -kill -9 1409 -``` - -Example output of the `kill` command with the `-l` option - -```shell -$ kill -l - 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP - 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 -11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM -16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP -21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ -26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR -31) SIGSYS 34) SIGRTMIN 35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3 -38) SIGRTMIN+4 39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8 -43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13 -48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12 -53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7 -58) SIGRTMAX-6 59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2 -63) SIGRTMAX-1 64) SIGRTMAX -``` - -## Scheduling a Process - -The time-consuming and resource-demanding part of maintenance work is often performed at late night. You can schedule relevant processes to get started at the scheduled time instead of staying up all night. The following describes the process scheduling commands. - -### Using the at Command to Run Processes at the Scheduled Time - -#### Function - -The `at` command is used to run a batch of processes \(a series of commands\) at the scheduled time or time and date. - -Syntax of the `at` command: - -```shell -at [-V] [-q queue] [-f filename] [-mldbv] time -at -c job [job...] -``` - -#### Time Format - -The scheduled time can be in any of the following formats: - -- _hh:mm_ today: If _hh:mm_ is earlier than the current time, the selected commands will be run at _hh:mm_ the next day. -- midnight, noon, teatime \(typically at 16:00\), or the like -- 12-hour format followed by am or pm -- Time + date \(_month day_, _mm/dd/yy_, or _dd.mm.yy_\). The scheduled date must follow the scheduled time. - -The scheduled time can also be relative time, which is suitable for scheduling commands that are going to be executed soon. For example, now+_N_ minutes, hours, days, or weeks. _N_ indicates the specified time, which may be a few days or hours. Further, the scheduled time can be words like today, tomorrow, or the like. Here are some examples of the scheduled time. - -Assume that the current time is 12:30 June 7 2019 and you want to run a command at 4:30 pm. The time scheduled by the `at` command can be any of the following: - -```shell - at 4:30pm - at 16:30 - at 16:30 today - at now+4 hours - at now+ 240 minutes - at 16:30 7.6.19 - at 16:30 6/7/19 - at 16:30 Jun 7 -``` - -Although you can select any of the preceding examples according to your preference, absolute time in 24-hour format, such as `at 16:30 6/7/19`, is recommended. - -#### Privileges - -Only commands from standard input or from the file specified by the **-f** option can be scheduled by the `at` command. If the `su` command is executed to switch the OS from user A to user B and then the `at` command is executed at the shell prompt of user B, the `at` command execution result is sent to user B, whereas emails \(if any\) are sent to user A. - -For example, to run the `slocate -u` command at 10 am on June 8, 2019, run the following commands as the **root** user: - -```shell -$ at 10:00 6/8/19 -at> slocate -u -at> -[1]+ Stopped at 10:00 6/8/19 -``` - -When the **at\>** prompt appears, type `slocate -u` and press **Enter**. Repeat the step to add other commands that need to be run at 10 am on 8 June 2019. Then, press **Ctrl+D** to exit the `at` command. - -The administrator is authorized to run the `at` command unconditionally. For other users, their privileges to run the `at` command is defined in the **/etc/at.allow** and **/etc/at.deny** files. - -### Using the cron Service to Run Commands Periodically - -The `at` command can run commands at the scheduled time, but only once. It means that after the commands to be run is specified, the system completes the task at the specified time. If you need to run the commands repeatedly, the **cron** service is a good choice. - -#### Cron Service - -The **cron** service searches the **/var/spool/cron** directory for the **crontab** files named by the user name in the **/etc/passwd** file and loads the search results into memory to execute the commands in the **crontab** files. Each user has a **crontab** file with the same name as the user name. For example, the **crontab** file of the **userexample** user is **/var/spool/cron/userexample**. - -The **cron** service also reads the cron configuration file **/etc/crontab** every minute, which can be edited in various formats. If no **crontab** files are found, the **cron** service enters sleep mode and releases system resources. One minute later, the **cron** service is waken up to repeat the search work and command execution. Therefore, the background process occupies few resources and is wakened up every minute to check whether there are commands to be executed. - -Command execution results are then mailed to users specified by the environment variable `MAILTO` in the **/etc/crontab** file. The **cron** service, once started, does not require manual intervention except when you need to replace the scheduled commands with new ones. - -#### crontab Command - -The `crontab` command is used to install, edit, remove, list, and perform other operations on **crontab** files. Each user has its own **crontab** files and can add commands to be executed to the files. - -Here are common `crontab` command options: - -- crontab -u //Set the **cron** service of a user. This option is required only when the `crontab` command is run by the **root** user. -- crontab -l //List details about the **cron** service of a user. -- crontab -r //Remove the **cron** service of a user. -- crontab -e //Edit the **cron** service of a user. - -For example, to list the **cron** service settings of the **root** user, run the following command: - -```shell -crontab -u root -l -``` - -#### crontab Files - -Enter the commands to be executed and their scheduled time in **crontab** files. Each line in the files contains six fields. The first five fields are the time when the specified command is executed, and the last field is the command to be executed. Fields are separated by spaces or tabs. The format is as follows: - -```shell -minute hour day-of-month month-of-year day-of-week commands -``` - -[Table 2](#en-us_topic_0151921016_t7d97d1204fe249d7ae0a87b4cf9a9353) describes the fields in each line. - -**Table 2** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

minute

-

The minute of the hour at which commands will be executed. Value range: 0–59.

-

hour

-

The hour of the day at which scheduled commands will be executed. Value range: 0–23.

-

day-of-month

-

The day of the month on which scheduled commands will be executed. Value range: 1–31.

-

month-of-year

-

The month of the year in which scheduled commands will be executed. Value range: 1–12.

-

day-of-week

-

The day of the week on which scheduled commands will be executed. Value range: 0–6.

-

commands

-

Scheduled commands.

-
- -The fields cannot be left unspecified. In addition to numerical values, the following special characters are allowed: asterisk \(\*\), indicating a wildcard value; forward slash \(/\), followed by a numeral value _N_ to indicate that commands will be executed at a regular interval of _N_; hyphen \(-\), used with a range; and comma \(,\), used to separate discrete values. A complete path to the commands must be provided. - -For example, to allow the OS to add **sleepy** to the **/tmp/test.txt** file every two hours from 18 pm to 22 pm, add the following line to a **crontab** file: - -```shell -* 18-22/2 * * * echo "sleepy" >> /tmp/test.txt -``` - -Each time the **cron** service settings of a user are edited, the **cron** service generates a **crontab** file with the same name as the user in the **/var/spool/cron directory**. The **crontab** file can be edited only using the `crontab -e` command. Alternatively, the user can create a file and run the `crontab _filename_` command to import its **cron** settings to the new file. - -For example, to create a **crontab** file for the **userexample** user, perform the following steps: - -1. Create a file using any text editor. Add the commands that need to be executed periodically and the command execution interval to the new file. In this example, the new file is **\~/userexample.cron**. -2. Run the following command as the **root** user to install the new file as the **crontab** file of the **userexample** user: - - ```shell - crontab -u userexample ~/userexample.cron - ``` - -After the new file is installed, you will find a file named **userexample** in the **/var/spool/cron** directory. This file is the required **crontab** file. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Do not restart the **cron** service after a **crontab** file is modified, because the **cron** service, once started, reads the **crontab** file every minute to check whether there are commands that need to be executed periodically. - -#### /etc/crontab File - -The **cron** service reads all files in the **/var/spool/cron** directory and the **/etc/crontab** file every minute. Therefore, you can use the **cron** service by configuring the **/etc/crontab** file. A **crontab** file contains user-specific commands, whereas the **/etc/crontab** file contains system-wide commands. The following is an example of the **/etc/crontab** file. - -```text -SHELL=/bin/sh -PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin -MAILTO=root //If an error occurs or data is output, the data is sent to the account by email. -HOME=/ -# run-parts -01 * * * * root run-parts /etc/cron.hourly //Run scripts in the /etc/cron.hourly directory once an hour. -02 4 * * * root run-parts /etc/cron.daily //Run scripts in the /etc/cron.daily directory once a day. -22 4 * * 0 root run-parts /etc/cron.weekly //Run scripts in the /etc/cron.weekly directory once a week. -42 4 1 * * root run-parts /etc/cron.monthly //Run scripts in the /etc/cron.monthly directory once a month. -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If the **run-parts** parameter is deleted, a script name instead of a directory name is used. - -## Suspending/Resuming a Process - -A process can be suspended or resumed by job control, and the process will continue to work from the suspended point after being resumed. To suspend a foreground process, press **Ctrl+Z**. After you press **Ctrl+Z**, the `cat` command is suspended together with the foreground process you want to suspend. You can use the `jobs` command instead to display a list of shell jobs, including their names, IDs, and status. - -To resume a process in foreground or background, run the `fg` or `bg` command, respectively. The process then starts from where it was suspended previously. diff --git a/docs/en/docs/Administration/service-management.md b/docs/en/docs/Administration/service-management.md deleted file mode 100644 index ac9581a070b1a1b4ae6681c7e37ece609ce353e4..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/service-management.md +++ /dev/null @@ -1,845 +0,0 @@ -# Service Management - -This topic describes how to manage your operating system and services using the systemd. - - -- [Service Management](#service-management) - - [Introduction to systemd](#introduction-to-systemd) - - [Systemd Units](#systemd-units) - - [Features](#features) - - [Fast Activation](#fast-activation) - - [On-Demand Activation](#on-demand-activation) - - [Service Lifecycle Management by Cgroups](#service-lifecycle-management-by-cgroups) - - [Mount and Automount Point Management](#mount-and-automount-point-management) - - [Transactional Dependency Management](#transactional-dependency-management) - - [Compatibility with SysVinit Scripts](#compatibility-with-sysvinit-scripts) - - [System State Snapshots and System Restoration](#system-state-snapshots-and-system-restoration) - - [Managing System Services](#managing-system-services) - - [Comparison Between SysVinit and systemd Commands](#comparison-between-sysvinit-and-systemd-commands) - - [Listing Services](#listing-services) - - [Displaying Service Status](#displaying-service-status) - - [Starting a Service](#starting-a-service) - - [Stopping a Service](#stopping-a-service) - - [Restarting a Service](#restarting-a-service) - - [Enabling a Service](#enabling-a-service) - - [Disabling a Service](#disabling-a-service) - - [Changing a Runlevel](#changing-a-runlevel) - - [Targets and Runlevels](#targets-and-runlevels) - - [Viewing the Default Startup Target](#viewing-the-default-startup-target) - - [Viewing All Startup Targets](#viewing-all-startup-targets) - - [Changing the Default Target](#changing-the-default-target) - - [Changing the Current Target](#changing-the-current-target) - - [Changing to Rescue Mode](#changing-to-rescue-mode) - - [Changing to Emergency Mode](#changing-to-emergency-mode) - - [Shutting Down, Suspending, and Hibernating the Operating System](#shutting-down-suspending-and-hibernating-the-operating-system) - - [systemctl Command](#systemctl-command) - - [Shutting Down the Operating System](#shutting-down-the-operating-system) - - [Restarting the Operating System](#restarting-the-operating-system) - - [Suspending the Operating System](#suspending-the-operating-system) - - [Hibernating the Operating System](#hibernating-the-operating-system) - - - -## Introduction to systemd - -The systemd is a system and service manager for Linux operating systems. It is designed to be backward compatible with SysV and LSB init scripts, and provides a number of features such as Socket & D-Bus based activation of services, on-demand activation of daemons, system state snapshots, and mount & automount point management. With systemd, the service control logic and parallelization are refined. - -### Systemd Units - -In systemd, the targets of most actions are units, which are resources systemd know how to manage. Units are categorized by the type of resources they represent and defined in unit configuration files. For example, the avahi.service unit represents the Avahi daemon and is defined in the **avahi.service** file. [Table 1](#en-us_topic_0151921012_t2dcb6d973cc249ed9ccd56729751ca6b) lists available types of systemd units. - -**Table 1** Available types of systemd units - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Unit Type

-

File Extension

-

Description

-

Service unit

-

.service

-

A system service.

-

Target unit

-

.target

-

A group of systemd units.

-

Automount unit

-

.automount

-

A file system automount point.

-

Device unit

-

.device

-

A device file recognized by the kernel.

-

Mount unit

-

.mount

-

A file system mount point.

-

Path unit

-

.path

-

A file or directory in a file system.

-

Scope unit

-

.scope

-

An externally created process.

-

Slice unit

-

.slice

-

A group of hierarchically organized units that manage system processes.

-

Socket unit

-

.socket

-

An inter-process communication socket.

-

Swap unit

-

.swap

-

A swap device or a swap file.

-

Timer unit

-

.timer

-

A systemd timer.

-
- -All available types of systemd units are located in one of the following directories listed in [Table 2](#en-us_topic_0151921012_t2523a0a9a0c54f9b849e52d1efa0160c). - -**Table 2** Locations of available systemd units - - - - - - - - - - - - - - - - -

Directory

-

Description

-

/usr/lib/systemd/system/

-

Systemd units distributed with installed RPM packages.

-

/run/systemd/system/

-

Systemd units created at runtime.

-

/etc/systemd/system/

-

Systemd units created and managed by the system administrator.

-
- -## Features - -### Fast Activation - -The systemd provides more aggressive parallelization than UpStart. The use of Socket- and D-Bus based activation reduces the time required to boot the operating system. - -To accelerate system boot, systemd seeks to: - -- Activate only the necessary processes -- Activate as many processes as possible in parallel - -### On-Demand Activation - -During SysVinit initialization, it activates all the possible background service processes that might be used. Users can log in only after all these service processes are activated. The drawbacks in SysVinit are obvious: slow system boot and a waste of system resources. - -Some services may rarely or even never be used during system runtime. For example, CUPS, printing services are rarely used on most servers. SSHD is rarely accessed on many servers. It is unnecessary to spend time on starting these services and system resources. - -systemd can only be activated when a service is requested. If the service request is over, systemd stops. - -### Service Lifecycle Management by Cgroups - -An important role of an init system is to track and manage the lifecycle of services. It can start and stop a service. However, it is more difficult than you could ever imagine to encode an init system into stopping services. - -Service processes often run in background as daemons and sometimes fork twice. In UpStart, the expect stanza in the configuration file must be correctly configured. Otherwise, UpStart is unable to learn a daemon's PID by counting the number of forks. - -Things are made simpler with Cgroups, which have long been used to manage system resource quotas. The ease of use comes largely from its file-system-like user interface. When a parent service creates a child service, the latter inherits all attributes of the Cgroup to which the parent service belongs. This means that all relevant services are put into the same Cgroup. The systemd can find the PIDs of all relevant services simply by traversing their control group and then stop them one by one. - -### Mount and Automount Point Management - -In traditional Linux systems, users can use the **/etc/fstab** file to maintain fixed file system mount points. These mount points are automatically mounted during system startup. Once the startup is complete, these mount points are available. These mount points are file systems critical to system running, such as the **HOME** directory. Like SysVinit, systemd manages these mount points so that they can be automatically mounted at system startup. systemd is also compatible with the **/etc/fstab** file. You can continue to use this file to manage mount points. - -There are times when you need to mount or unmount on demand. For example, a temporary mounting point is required for you to access the DVD content, and the mounting point is canceled \(using the **umount** command\) if you no longer need to access the content, thereby saving resources. This is traditionally achieved using the autofs service. - -The systemd allows automatic mount without a need to install autofs. - -### Transactional Dependency Management - -System boot involves a host of separate jobs, some of which may be dependent on each other. For example, a network file system \(NFS\) can be mounted only after network connectivity is activated. The systemd can run a large number of dependent jobs in parallel, but not all of them. Looking back to the NFS example, it is impossible to mount NFS and activate network at the same time. Before running a job, systemd calculates its dependencies, creates a temporary transaction, and verifies that this transaction is consistent \(all relevant services can be activated without any dependency on each other\). - -### Compatibility with SysVinit Scripts - -Like UpStart, systemd introduces new configuration methods and has new requirements for application development. If you want to replace the currently running initialization system with systemd, systemd must be compatible with the existing program. It is difficult to modify all the service code in any Linux distribution in a short time for the purpose of using systemd. - -The systemd provides features compatible with SysVinit and LSB initscripts. You do not need to modify the existing services and processes in the system. This reduces the cost of migrating the system to systemd, making it possible for users to replace the existing initialization system with systemd. - -### System State Snapshots and System Restoration - -The systemd can be started on demand. Therefore, the running status of the system changes dynamically, and you cannot know the specific services that are running in the system. systemd snapshots enable the current system running status to be saved and restored. - -For example, if services A and B are running in the system, you can run the **systemd** command to create a snapshot for the current system running status. Then stop process A or make any other change to the system, for example, starting process C. After these changes, run the snapshot restoration command of systemd to restore the system to the point at which the snapshot was taken. That is, only services A and B are running. A possible application scenario is debugging. For example, when an exception occurs on the server, a user saves the current status as a snapshot for debugging, and then perform any operation, for example, stopping the service. After the debugging is complete, restore the snapshot. - -## Managing System Services - -The systemd provides the systemctl command to start, stop, restart, view, enable, and disable system services. - -### Comparison Between SysVinit and systemd Commands - -The **systemctl** command from the **systemd** command has the functions similar to the **SysVinit** command. Note that the **service** and **chkconfig** commands are supported in this version. For details, see [Table 3](#en-us_topic_0151920917_ta7039963b0c74b909b72c22cbc9f2e28). You are advised to manage system services by running the **systemctl** command. - -**Table 3** Comparison between SysVinit and systemd commands - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

SysVinit Command

-

systemd Command

-

Description

-

service network start

-

systemctl start network.service

-

Starts a service.

-

service network stop

-

systemctl stop network.service

-

Stops a service.

-

service network restart

-

systemctl restart network.service

-

Restarts a service.

-

service network reload

-

systemctl reload network.service

-

Reloads a configuration file without interrupting an operation.

-

service network condrestart

-

systemctl condrestart network.service

-

Restarts a service only if it is running.

-

service network status

-

systemctl status network.service

-

Checks the service running status.

-

chkconfig network on

-

systemctl enable network.service

-

Enables a service when the service activation time arrives or a trigger condition for enabling the service is met.

-

chkconfig network off

-

systemctl disable network.service

-

Disables a service when the service activation time arrives or a trigger condition for disabling the service is met.

-

chkconfig network

-

systemctl is-enabled network.service

-

Checks whether a service is enabled.

-

chkconfig --list

-

systemctl list-unit-files --type=service

-

Lists all services in each runlevel and checks whether they are enabled.

-

chkconfig network --list

-

ls /etc/systemd/system/*.wants/network.service

-

Lists the runlevels in which a service is enabled and those in which the service is disabled.

-

chkconfig network --add

-

systemctl daemon-reload

-

Used when you need to create a service file or change settings.

-
- -### Listing Services - -To list all currently loaded services, run the following command: - -```shell -systemctl list-units --type service -``` - -To list all services regardless of whether they are loaded, run the following command \(with the `--all` option\): - -```shell -systemctl list-units --type service --all -``` - -Example list of all currently loaded services: - -```shell -$ systemctl list-units --type service -UNIT LOAD ACTIVE SUB DESCRIPTION -atd.service loaded active running Deferred execution scheduler -auditd.service loaded active running Security Auditing Service -avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack -chronyd.service loaded active running NTP client/server -crond.service loaded active running Command Scheduler -dbus.service loaded active running D-Bus System Message Bus -dracut-shutdown.service loaded active exited Restore /run/initramfs on shutdown -firewalld.service loaded active running firewalld - dynamic firewall daemon -getty@tty1.service loaded active running Getty on tty1 -gssproxy.service loaded active running GSSAPI Proxy Daemon -...... -``` - -### Displaying Service Status - -To display the status of a service, run the following command: - -```shell -systemctl status name.service -``` - -[Table 4](#en-us_topic_0151920917_t36cd267d69244ed39ae06bb117ed8e62) describes the parameters in the command output. - -**Table 4** Output parameters - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Loaded

-

Information on whether the service has been loaded, the absolute path to the service file, and a note of whether the service is enabled.

-

Active

-

Information on whether the service is running and a time stamp.

-

Main PID

-

PID of the service.

-

CGroup

-

Additional information about related control groups.

-
- -To verify whether a particular service is running, run the following command: - -```shell -systemctl is-active name.service -``` - -The output of the **is-active** command is as follows: - -**Table 5** Output of the is-active command - - - - - - - - - - - - - - - - - - - -

Status

-

Description

-

active(running)

-

One or more services are running in the system.

-

active(exited)

-

A service that ends properly after being executed only once. Currently, no program is running in the system. For example, the quotaon function is performed only when the program is started or mounted.

-

active(waiting)

-

The program needs to wait for other events to continue running. For example, the print queue service is being started, but it needs to be queued (print jobs) so that it can continue to wake up the printer service to perform the next print function.

-

inactive

-

The service is not running.

-
- -Similarly, to determine whether a particular service is enabled, run the following command: - -```shell -systemctl is-enabled name.service -``` - -The output of the **is-enabled** command is as follows: - -**Table 6** Output of the is-enabled command - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Status

-

Description

-

enabled

-

Has been permanently enabled through Alias= Alias, .wants/, or .requires/ soft link in the /etc/systemd/system/ directory.

-

enabled-runtime

-

Has been temporarily enabled through Alias= Alias, .wants/, or .requires/ soft link in the /run/systemd/system/ directory.

-

linked

-

Although the unit file is not in the standard unit directory, one or more soft links pointing to the unit file exist in the /etc/systemd/system/ permanent directory.

-

linked-runtime

-

Although the unit file is not in the standard unit directory, one or more soft links pointing to the unit file exist in the /run/systemd/system/ temporary directory.

-

masked

-

Has been masked permanently by the /etc/systemd/system/ directory (soft link to /dev/null). Therefore, the start operation fails.

-

masked-runtime

-

Has been masked temporarily by the /run/systemd/systemd/ directory (soft link to /dev/null). Therefore, the start operation fails.

-

static

-

Not enabled. There is no option available for the enable command in the [Install] section of the unit file.

-

indirect

-

Not enabled. But the list of values for the Also= option in the [Install] section of the unit file is not empty (that is, some units in the list may have been enabled), or the unit file has an alias soft link which is not in the Also= list. For a template unit, it indicates that an instance different from DefaultInstance= is enabled.

-

disabled

-

Not enabled. But the [Install] section of the unit file contains options available for the enable command.

-

generated

-

The unit file is dynamically generated by the unit generator. The generated unit file may not be directly enabled, but is implicitly enabled by the unit generator.

-

transient

-

The unit file is dynamically and temporarily generated by the runtime API. The temporary unit may not be enabled.

-

bad

-

The unit file is incorrect or other errors occur. is-enabled does not return this status, but displays an error message. The list-unit-files command may display this unit.

-
- -For example, to display the status of gdm.service, run the **systemctl status gdm.service** command. - -```shell -# systemctl status gdm.service -gdm.service - GNOME Display Manager - Loaded: loaded (/usr/lib/systemd/system/gdm.service; enabled) - Active: active (running) since Thu 2013-10-17 17:31:23 CEST; 5min ago - Main PID: 1029 (gdm) - CGroup: /system.slice/gdm.service - ├─1029 /usr/sbin/gdm - ├─1037 /usr/libexec/gdm-simple-slave --display-id /org/gno... - └─1047 /usr/bin/Xorg :0 -background none -verbose -auth /r...Oct 17 17:31:23 localhost systemd[1]: Started GNOME Display Manager. -``` - -### Starting a Service - -To start a service, run the following command as the user **root**: - -```shell -systemctl start name.service -``` - -For example, to start the httpd service, run the following command: - -```shell -# systemctl start httpd.service -``` - -### Stopping a Service - -To stop a service, run the following command as the user **root**: - -```shell -systemctl stop name.service -``` - -For example, to stop the Bluetooth service, run the following command: - -```shell -# systemctl stop bluetooth.service -``` - -### Restarting a Service - -To restart a service, run the following command as the user **root**: - -```shell -systemctl restart name.service -``` - -This command stops the selected service in the current session and immediately starts it again. If the selected service is not running, this command starts it too. - -For example, to restart the Bluetooth service, run the following command: - -```shell -# systemctl restart bluetooth.service -``` - -### Enabling a Service - -To configure a service to start automatically at system boot time, run the following command as the user **root**: - -```shell -systemctl enable name.service -``` - -For example, to configure the httpd service to start automatically at system boot time, run the following command: - -```shell -# systemctl enable httpd.service -ln -s '/usr/lib/systemd/system/httpd.service' '/etc/systemd/system/multi-user.target.wants/httpd.service' -``` - -### Disabling a Service - -To prevent a service from starting automatically at system boot time, run the following command as the user **root**: - -```shell -systemctl disable name.service -``` - -For example, to prevent the Bluetooth service from starting automatically at system boot time, run the following command: - -```shell -# systemctl disable bluetooth.service -Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service. -Removed /etc/systemd/system/dbus-org.bluez.service. -``` - -## Changing a Runlevel - -### Targets and Runlevels - -In systemd, the concept of runlevels has been replaced with systemd targets to improve flexibility. For example, you can inherit an existing target and turn it into your own target by adding other services. [Table 7](#en-us_topic_0151920939_t9af92c282ad240ea9a79fb08d26e8181) provides a complete list of runlevels and their corresponding systemd targets. - -**Table 7** Mapping between runlevels and targets - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Runlevel

-

systemd Target

-

Description

-

0

-

runlevel0.target, poweroff.target

-

The operating system is powered off.

-

1, s, single

-

runlevel1.target, rescue.target

-

The operating system is in single user mode.

-

2, 4

-

runlevel2.target, runlevel4.target, multi-user.target

-

The operating system is in user-defined or domain-specific runlevel (by default, it is equivalent to runlevel 3).

-

3

-

runlevel3.target, multi-user.target

-

The operating system is in non-graphical multi-user mode, and can be accessed from multiple consoles or networks.

-

5

-

runlevel5.target, graphical.target

-

The operating system is in graphical multi-user mode. All the services running at level 3 can be accessed through graphical login.

-

6

-

runlevel6.target, reboot.target

-

The operating system is rebooted.

-

emergency

-

emergency.target

-

Emergency shell.

-
- -### Viewing the Default Startup Target - -Run the following command to view the default startup target of the system: - -```shell -systemctl get-default -``` - -### Viewing All Startup Targets - -Run the following command to view all startup targets of the system: - -```shell -systemctl list-units --type=target -``` - -### Changing the Default Target - -To change the default target, run the following command as the user **root**: - -```shell -systemctl set-default name.target -``` - -### Changing the Current Target - -To change the current target, run the following command as the user **root**: - -```shell -systemctl isolate name.target -``` - -### Changing to Rescue Mode - -To change the operating system to rescue mode, run the following command as the user **root**: - -```shell -systemctl rescue -``` - -This command is similar to the **systemctl isolate rescue.target** command. After the command is executed, the following information is displayed on the serial port: - -```console -You are in rescue mode. After logging in, type "journalctl -xb" to viewsystem logs, "systemctl reboot" to reboot, "systemctl default" or "exit"to boot into default mode. -Give root password for maintenance -(or press Control-D to continue): -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->You need to restart the system to enter the normal mode from the rescue mode. - -### Changing to Emergency Mode - -To change the operating system to emergency mode, run the following command as the user **root**: - -```shell -systemctl emergency -``` - -This command is similar to the **systemctl isolate emergency.target** command. After the command is executed, the following information is displayed on the serial port: - -```console -You are in emergency mode. After logging in, type "journalctl -xb" to viewsystem logs, "systemctl reboot" to reboot, "systemctl default" or "exit"to boot into default mode. -Give root password for maintenance -(or press Control-D to continue): -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->You need to restart the system to enter the normal mode from the emergency mode. - -## Shutting Down, Suspending, and Hibernating the Operating System - -### systemctl Command - -The systemd uses the systemctl command instead of old Linux system management commands to shut down, restart, suspend, and hibernate the operating system. Although previous Linux system management commands are still available in systemd for compatibility reasons, you are advised to use **systemctl** when possible. The mapping relationship is shown in [Table 8](#en-us_topic_0151920964_t3daaaba6a03b4c36be9668efcdb61f3b). - -**Table 8** Mapping between old Linux system management commands and systemctl - - - - - - - - - - - - - - - - - - - - -

Linux Management Command

-

systemctl Command

-

Description

-

halt

-

systemctl halt

-

Shuts down the operating system.

-

poweroff

-

systemctl poweroff

-

Powers off the operating system.

-

reboot

-

systemctl reboot

-

Reboots the operating system.

-
- -### Shutting Down the Operating System - -To shut down the system and power off the operating system, run the following command as the user **root**: - -```shell -systemctl poweroff -``` - -To shut down the operating system without powering it off, run the following command as the user **root**: - -```shell -systemctl halt -``` - -By default, running either of these commands causes systemd to send an informative message to all login users. To prevent systemd from sending this message, run this command with the **--no\-wall** option. The command is as follows: - -```shell -systemctl --no-wall poweroff -``` - -### Restarting the Operating System - -To restart the operating system, run the following command as the user **root**: - -```shell -systemctl reboot -``` - -By default, running either of these commands causes systemd to send an informative message to all login users. To prevent systemd from sending this message, run this command with the **--no\-wall** option. The command is as follows: - -```shell -systemctl --no-wall reboot -``` - -### Suspending the Operating System - -To suspend the operating system, run the following command as the user **root**: - -```shell -systemctl suspend -``` - -### Hibernating the Operating System - -To hibernate the operating system, run the following command as the user **root**: - -```shell -systemctl hibernate -``` - -To suspend and hibernate the operating system, run the following command as the user **root**: - -```shell -systemctl hybrid-sleep -``` diff --git a/docs/en/docs/Administration/setting-up-the-database-server.md b/docs/en/docs/Administration/setting-up-the-database-server.md deleted file mode 100644 index b84c1dc97a4923bc3d3b86bcb3a512304f5edada..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/setting-up-the-database-server.md +++ /dev/null @@ -1,2173 +0,0 @@ -# Setting Up the Database Server - -- [Setting Up the Database Server](#setting-up-the-database-server) - - [PostgreSQL Server](#postgresql-server) - - [Software Description](#software-description) - - [Configuring the Environment](#configuring-the-environment) - - [Installing, Running, and Uninstalling PostgreSQL](#installing-running-and-uninstalling-postgresql) - - [Managing Database Roles](#managing-database-roles) - - [Managing Databases](#managing-databases) - - [MariaDB Server](#mariadb-server) - - [Software Description](#software-description-1) - - [Configuring the Environment](#configuring-the-environment-1) - - [Installing, Running, and Uninstalling MariaDB Server](#installing-running-and-uninstalling-mariadb-server) - - [Managing Database Users](#managing-database-users) - - [Managing Databases](#managing-databases-1) - - [MySQL Server](#mysql-server) - - [Software Description](#software-description-2) - - [Configuring the Environment](#configuring-the-environment-2) - - [Installing, Running, and Uninstalling MySQL](#installing-running-and-uninstalling-mysql) - - [Managing Database Users](#managing-database-users-1) - - [Managing Databases](#managing-databases-2) - - - -## PostgreSQL Server - -### Software Description - -[Figure 1](#fig26022387391) shows the PostgreSQL architecture and [Table 1](#table62020913417) describes the main processes. - -**Figure 1** PostgreSQL architecture -![](./figures/postgresql-architecture.png "postgresql-architecture") - -**Table 1** Main processes in PostgreSQL - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Process Type

-

Process Name

-

Description

-

Main process

-

Postmaster

-

Postmaster process controls all database instances in general and is responsible for starting and stopping database instances.

-

Resident process

-

Postgres (resident process)

-

This process manages backend resident processes and is also called postmaster. By default, this process listens Unix domain sockets and the 5432 port of TCP/IP and waits for the front end to process the connections. You can change the listening port number in the postgresql.conf file of PostgreSQL.

-

Subprocess

-

Postgres (subprocess)

-

The subprocess determines whether to allow the connection according to the security policy defined by the pg_hba.conf file. According to the security policy, the subprocess rejects certain IP addresses and networks, allows only certain users to connect to the databases, or allows only certain databases to be connected.

-

Postgres receives the query from the front end, searches the database, and returns the results. Sometimes, it also updates the database. The updated data is recorded in transaction logs (WAL logs for PostgreSQL). This method is used when the system is powered off, the server breaks down, or the server is restarted. In addition, the logs can also be used for data recovery in other scenarios. In PostgreSQL 9.0 or later, WAL logs can be transferred to other PostgreSQL systems to replicate database in real-time.

-

Auxiliary processes

-

SysLogger (system log)

-

The main process starts the Syslogger auxiliary process only when logging_collection in the Postgres.conf file is set to on.

-

BgWriter (background write)

-

This process writes dirty pages from the shared memory to the drive. The purpose is to improve the performance of inserting, updating, and deleting data.

-

WALWriter (write-ahead log)

-

This process writes modification operations into drives before data is modified so that the data does not need to be persisted into files in subsequent real-time data updates.

-

PgArch (archive)

-

write-ahead logs (WALs) are recycled. The PgArch process backs up WALs before archiving them. After the entire database is backed up, the Point in Time Recovery (PITR) technology can be used to archive WALs generated after the backup. The database can be restored to any point after the full backup by using the full backup data and the subsequently archived WALs.

-

AutoVacuum (automatic cleanup)

-

In the PostgreSQL database, after a DELETE operation is performed on a table, old data is not immediately deleted. When new data is added, the system creates a data row instead of overwriting the old data. The old data is only marked as deleted and will be cleared only when no other concurrent transactions are reading the data. In this case, the data is cleared by the AutoVacuum process.

-

PgStat (statistics collection)

-

This process collects data statistics. It is used to estimate the cost during query optimization, including the number of insertions update, and deletion operations performed on a table or index, the number of drive block read and write operations, and the number of row read operations. pg_statistic stores the information collected by the PgStat.

-

CheckPoint (checkpoint)

-

A checkpoint is a transaction sequence point set by the system. It is used to ensure that log information before a checkpoint written into the drives.

-
- -### Configuring the Environment - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The following environment configuration is for reference only. Configure the environment based on the site requirements. - -#### Disabling the Firewall and Automatic Startup - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements. - -1. Stop the firewall service as the **root** user. - - ```shell - systemctl stop firewalld - ``` - -2. Disable the firewall service as the **root** user. - - ```shell - systemctl disable firewalld - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The automatic startup is automatically disabled as the firewall is disabled. - -#### Disabling SELinux - -1. Modify the configuration file as the **root** user. - - ```shell - sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config - ``` - -#### Creating a User Group and a User - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database. - -1. Create a PostgreSQL user or user group as the **root** user. - - ```shell - groupadd postgres - ``` - - ```shell - useradd -g postgres postgres - ``` - -2. Set the postgres user password as the **root** user. \(Enter the password twice for confirmation.\) - - ```shell - passwd postgres - ``` - -#### Creating Data Drives - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- When testing the ultimate performance, you are advised to attach NVMe SSDs with better I/O performance to create PostgreSQL test instances to avoid the impact of disk I/O on the performance test result. This section uses NVMe SSDs as an example. For details, see Step 1 to Step 4. ->- In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section. -> `mkdir /data` - -1. Create a file system \(xfs is used as an example as the **root** user. Create the file system based on the site requirements.\). If a file system has been created for a disk, an error will be reported when you run this command. You can use the **-f** parameter to forcibly create a file system. - - ```shell - mkfs.xfs /dev/nvme0n1 - ``` - -2. Create a data directory. - - ```shell - mkdir /data - ``` - -3. Mount disks. - - ```shell - mount -o noatime,nobarrier /dev/nvme0n1 /data - ``` - -#### Data Directory Authorization - -1. Modify the directory permission as the **root** user. - - ```shell - chown -R postgres:postgres /data/ - ``` - -### Installing, Running, and Uninstalling PostgreSQL - -#### Installing PostgreSQL - -1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). -2. Clear the cache. - - ```shell - dnf clean all - ``` - -3. Create a cache. - - ```shell - dnf makecache - ``` - -4. Install the PostgreSQL server as the **root** user. - - ```shell - dnf install postgresql-server - ``` - -5. Check the installed RPM package. - - ```shell - rpm -qa | grep postgresql - ``` - -#### Running PostgreSQL - -##### Initializing the Database - ->![](./public_sys-resources/icon-notice.gif) **NOTICE:** ->Perform this step as the postgres user. - -1. Switch to the created PostgreSQL user. - - ```shell - su - postgres - ``` - -2. Initialize the database. In the command, **/usr/bin** is the directory where the **initdb** command is located. - - ```shell - /usr/bin/initdb -D /data/ - ``` - -##### Starting the Database - -1. Enable the PostgreSQL database. - - ```shell - /usr/bin/pg_ctl -D /data/ -l /data/logfile start - ``` - -2. Check whether the PostgreSQL database process is started properly. - - ```shell - ps -ef | grep postgres - ``` - - If the following information is displayed, the PostgreSQL processes have been started. - - ![](./figures/postgres.png) - -##### Logging In to the Database - -1. Log in to the database. - - ```shell - /usr/bin/psql -U postgres - ``` - - ![](./figures/login.png) - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >You do not need to enter a password when logging in to the database for the first time. - -##### Configuring the Database Accounts and Passwords - -1. After login, set the postgres user password. - - ```shell - postgres=#alter user postgres with password '123456'; - ``` - - ![](./figures/en-us_image_0230050789.png) - -##### Exiting the Database - -1. Run **\\q** to exit from the database. - - ```shell - postgres=# \q - ``` - -##### Stopping the Database - -1. Stop the PostgreSQL database. - - ```shell - /usr/bin/pg_ctl -D /data/ -l /data/logfile stop - ``` - -#### Uninstalling PostgreSQL - -1. Stop the database as the postgres user. - - ```shell - /usr/bin/pg_ctl -D /data/ -l /data/logfile stop - ``` - -2. Run the **dnf remove postgresql-server** command as the user **root** to uninstall the PostgreSQL database. - - ```shell - dnf remove postgresql-server - ``` - -### Managing Database Roles - -#### Creating a Role - -You can use the **CREATE ROLE** statement or **createuser** command to create a role. The **createuser** command encapsulates the **CREATE ROLE** statement and needs to be executed on the shell GUI instead of the database GUI. - -```pgsql -CREATE ROLE rolename [ [ WITH ] option [ ... ] ]; -``` - -```shell -createuser rolename -``` - -In the preceding information: - -- **rolename**: indicates a role name. -- Parameters of the _option_ are as follows: - - **SUPERUSER | NOSUPERUSER**: determines whether a new role is a superuser. If this parameter is not specified, the default value **NOSUPERUSER** is used, indicating that the role is not a superuser. - - **CREATEDB | NOCREATEDB**: specifies whether a role can create a database. If this parameter is not specified, the default value **NOCREATEDB** is used, indicating that the role cannot create a database. - - **CREATEROLE | NOCREATEROLE**: determines whether a role can create roles. If this parameter is not specified, the default value **NOCREATEROLE** is used, indicating that the role cannot create roles. - - **INHERIT | NOINHERIT**: determines whether a role inherits the other roles' permissions in the group to which the role belongs. A role with the INHERIT attribute can automatically use any permissions that have been assigned to its direct or indirect group. If this parameter is not specified, the default value **INHERIT** is used. - - **LOGIN | NOLOGIN**: determines whether a role can log in. A role with the LOGIN attribute can be considered as a user. A role without this attribute can be used to manage database permissions but is not a user. If this attribute is not specified, the default value **NOLOGIN** is used. However, if **CREATE USER** instead of **CREATE ROLE** is used to create a role, the LOGIN attribute is used by default. - - **\[ENCRYPTED | UNENCRYPTED\] PASSWORD'password'**: password of a role. The password is valid only for roles with the LOGIN attribute. **ENCRYPTED | UNENCRYPTED**: determines whether to encrypt the password. If this parameter is not specified, the value **ENCRYPTED** is used, that is, the password is encrypted. - - **VALID UNTIL'timestamp'**: specifies the timestamp when the password of a role expires. If this parameter is not specified, the password is permanently valid. - - **IN ROLE rolename1**: lists one or more existing roles. The new role _rolename_ will be added to and become a member of **rolename1**. - - **ROLE rolename2**: lists one or more existing roles. These roles will be automatically added as members of the new role _rolename_. That is, the new role is a user group. - -To run this command, you must have the CREATEROLE permission or is the database superuser. - -##### Example - - Create a role **roleexample1** who can log in. - -```shell -postgres=# CREATE ROLE roleexample1 LOGIN; -``` - - Create a role **roleexample2** with the password **123456**. - -```shell -postgres=# CREATE ROLE roleexample2 WITH LOGIN PASSWORD '123456'; -``` - - Create a role named **roleexample3**. - -```console -[postgres@localhost ~]$ createuser roleexample3 -``` - -#### Viewing Roles - -You can run the **SELECT** statement or the PostgreSQL meta-command **\\du** to view the role. - -```pgsql -SELECT rolename FROM pg_roles; -``` - -```pgsql -\du -``` - -In the preceding command, _rolename_ indicates the role name. - -##### Example - - View the **roleexample1** role. - -```shell -postgres=# SELECT roleexample1 from pg_roles; -``` - - View the existing roles. - -```shell -postgres=# \du -``` - -#### Modifying a Role - -##### Modifying a Username - -Use the **ALTER ROLE** statement to modify an existing role name. - -```pgsql -ALTER ROLE oldrolername RENAME TO newrolename; -``` - -In the preceding information: - -- _oldrolername_: original role name. -- _newrolename_: new role name. - -##### Example of Modifying a User - - Change the role name **roleexample1** to **roleexapme2**. - -```shell -postgres=# ALTER ROLE roleexample1 RENAME TO roleexample2; -``` - -##### Modifying a User Password - -Use the **ALTER ROLE** statement to modify the login password of a role. - -```pgsql -ALTER ROLE rolename PASSWORD 'password' -``` - -In the preceding information: - -- _rolename_: indicates a role name. -- _password_: password. - -##### Example of Modifying the Password of a Role - - Modify the password of **roleexample1** to **456789**. - -```shell -postgres=# ALTER ROLE roleexample1 WITH PASSWORD '456789'; -``` - -#### Deleting a Role - -You can use the **DROP ROLE** statement or **dropuser** command to delete a role. The **dropuser** command encapsulates the **DROP ROLE** statement and needs to be executed on the shell GUI instead of the database GUI. - -```pgsql -DROP ROLE rolename; -``` - -```shell -dropuser rolename -``` - -In the preceding command, _rolename_ indicates the role name. - -##### Example - - Delete the **userexample1** role. - -```shell -postgres=# DROP ROLE userexample1; -``` - - Delete the **userexample2** role. - -```console -[postgres@localhost ~]$ dropuser userexample2 -``` - -#### Role Permissions - -You can use the **GRANT** statement to grant permissions to a role. - -Grant the table operation permission to a role. - -```pgsql -GRANT { { SELECT | INSERT | UPDATE | DELETE | REFERENCES | TRIGGER } [,...] | ALL [ PRIVILEGES ] } ON [ TABLE ] tablename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Grant the sequence operation permission to a role. - -```pgsql -GRANT { { USAGE | SELECT | UPDATE } [,...] | ALL [ PRIVILEGES ] } ON SEQUENCE sequencename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Grant the database operation permission to a role. - -```pgsql -GRANT { { CREATE | CONNECT | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] } ON DATABASE databasename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Grant the function operation permission to a role. - -```pgsql -GRANT { EXECUTE | ALL [ PRIVILEGES ] } ON FUNCTION funcname ( [ [ argmode ] [ argname ] argtype [, ...] ] ) [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Grant the operation permission of the procedural language to a role. - -```pgsql -GRANT { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE langname [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Grant the schema operation permission to a role. - -```pgsql -GRANT { { CREATE | USAGE } [,...] | ALL [ PRIVILEGES ] } ON SCHEMA schemaname [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Grant the tablespace operation permission to a role. - -```pgsql -GRANT { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE tablespacename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] -``` - -Assign the member relationship of rolename1 to rolename2. - -```pgsql -GRANT rolename1 [, ...] TO rolename2 [, ...] [ WITH ADMIN OPTION ] -``` - -In the preceding information: - -- **SELECT**, **INSERT**, **UPDATE**, **DELETE**, **REFERENCES**, **TRIGGER**, **USAGE**, **CREATE**, **CONNECT**, **TEMPORARY**, **TEMP**, **EXECUTE**, and **ALL \[**_PRIVILEGES_**\]** indicate user operation permissions. **ALL \[**_PRIVILEGES_**\]** indicates all permissions, the _PRIVILEGES_ keyword is optional in PostgreSQL, but it is required in strict SQL statements. -- **ON** clause: specifies the object on which the permission is granted. -- **tablename**: table name. -- **TO** clause: specifies the role to which the permission is granted. -- **rolename**, **rolename1**, and **rolename2**: role names. -- **groupname**: name of a role group. -- **PUBLIC**: indicates that the permission is granted to all roles, including users who may be created later. -- **WITH GRANT OPTION**: indicates that the recipient of a permission can grant the permission to others. This option cannot be assigned to PUBLIC. -- **sequencename**: sequence name. -- **databasename**: database name. -- **funcname \(\[\[argmode\] \[argname\] argtype \[, ...\]\]\)**: function name and its parameters. -- **langname**: procedural language name. -- **schemaname**: schema name. -- **tablespacename**: tablespace name. -- **WITH ADMIN OPTION**: A member can assign the member relationship of a role to other roles and cancel the member relationship of other roles. - -##### Example - - Grant the CREATE permission on database1 to userexample. - -```shell -postgres=# GRANT CREATE ON DATABASE database1 TO userexample; -``` - - Grant all permissions on table1 to all users. - -```shell -postgres=# GRANT ALL PRIVILEGES ON TABLE table1 TO PUBLIC; -``` - -#### Deleting User Permissions - -You can use the **REVOKE** statement to revoke the permissions previously granted to one or more roles. - -Revoke the table operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { { SELECT | INSERT | UPDATE | DELETE | REFERENCES | TRIGGER } [,...] | ALL [ PRIVILEGES ] } ON [ TABLE ] tablename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] -``` - -Revoke the sequence operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { { USAGE | SELECT | UPDATE } [,...] | ALL [ PRIVILEGES ] } ON SEQUENCE sequencename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] -``` - -Revoke the database operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { { CREATE | CONNECT | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] } ON DATABASE databasename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] -``` - -Revoke the function operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { EXECUTE | ALL [ PRIVILEGES ] } ON FUNCTION funcname ( [ [ argmode ] [ argname ] argtype [, ...] ] ) [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] -``` - -Revoke the procedural language operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE langname [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] -``` - -Revoke the schema operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { { CREATE | USAGE } [,...] | ALL [ PRIVILEGES ] } ON SCHEMA schemaname [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] -``` - -Revoke the tablespace operation permission from a role. - -```pgsql -REVOKE [ GRANT OPTION FOR ] { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE tablespacename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] -``` - -Revoke the member relationship of rolename1 from rolename2. - -```pgsql -REVOKE [ ADMIN OPTION FOR ] rolename1 [, ...] FROM rolename2 [, ...] [ CASCADE | RESTRICT ] -``` - -In the preceding information: - -- **GRANT OPTION FOR**: The permission cannot be granted to others, but permission itself is not revoked. -- **SELECT**, **INSERT**, **UPDATE**, **DELETE**, **REFERENCES**, **TRIGGER**, **USAGE**, **CREATE**, **CONNECT**, **TEMPORARY**, **TEMP**, **EXECUTE**, and **ALL \[**_PRIVILEGES_**\]** indicate user operation permissions. **ALL \[**_PRIVILEGES_**\]** indicates all permissions, the _PRIVILEGES_ keyword is optional in PostgreSQL, but it is required in strict SQL statements. -- **ON** clause: specifies the object on which the permission is revoked. -- _tablename_: table name. -- **FROM** clause: specifies the role whose permission is revoked. -- _rolename_, _rolename1_, and _rolename2_: role names. -- _groupname_: name of a role group. -- **PUBLIC**: revokes the implicitly defined groups that have all roles. However, this does not mean that all roles lose the permissions. The permissions directly obtained and the permissions obtained through a group are still valid. -- _sequencename_: sequence name. -- **CASCADE**: revokes all dependent permissions. -- **RESTRICT**: does not revoke all dependent permissions. -- _databasename_: database name. -- **funcname \(**_\[\[argmode\] \[argname\] argtype \[, ...\]\]_**\)**: function name and its parameters. -- _langname_: procedural language name. -- _schemaname_: schema name. -- _tablespacename_: tablespace name. -- **ADMIN OPTION FOR**: The transferred authorization is not automatically revoked. - -##### Example - - Grant the CREATE permission on database1 to userexample. - -```shell -postgres=# GRANT CREATE ON DATABASE database1 TO userexample; -``` - - Grant all permissions on table1 to all users. - -```shell -postgres=# GRANT ALL PRIVILEGES ON TABLE table1 TO PUBLIC; -``` - -### Managing Databases - -#### Creating a Database - -You can use the **CREATE DATABASE** statement or the **createdb** command to create a database. The **createdb** command encapsulates the **CREATE DATABASE** statement and needs to be executed on the shell GUI instead of the database GUI. - -```pgsql -CREATE DATABASE databasename; -``` - -```shell -createdb databasename -``` - -In the preceding command, **databasename** indicates the database name. - -To use this command, you must have the CREATEDB permission. - -##### Example - - Create a database named **database1**. - -```shell -postgres=# CREATE DATABASE database1; -``` - -#### Selecting a Database - -Use the **\\c** statement to select a database. - -```pgsql -\c databasename; -``` - -In the preceding command, **databasename** indicates the database name. - -##### Example - - Select the **databaseexample** database. - -```shell -postgres=# \c databaseexample; -``` - -#### Viewing a Database - -Use the **\\l** statement to view the database. - -```pgsql -\l; -``` - -##### Example - - View all databases. - -```shell -postgres=# \l; -``` - -#### Deleting a Database - -You can run the **DROP DATABASE** statement or **dropdb** command to delete a database. The **dropdb** command encapsulates the **DROP DATABASE** statement and needs to be executed on the shell GUI instead of the database GUI. - ->![](./public_sys-resources/icon-caution.gif) **CAUTION:** ->Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted. - -```pgsql -DROP DATABASE databasename; -``` - -```shell -dropdb databasename -``` - -In the preceding command, **databasename** indicates the database name. - -The **DROP DATABASE** statement deletes the system directory items of the database and the file directories that contain data. - -**DROP DATABASE** can be executed only by the super administrator or database owner. - -##### Example - - Delete the **databaseexample** database. - -```shell -postgres=# DROP DATABASE databaseexample; -``` - -#### Backing Up a Database - -Run the **pg\_dump** command to back up the database and dump the database to a script file or another archive file. - -```shell -pg_dump [option]... [databasename] > outfile -``` - -In the preceding information: - -- _databasename_: database name. If this parameter is not specified, the environment variable **PGDATABASE** is used. If that environment variable is not specified, use the username that initiates the connection. -- _outfile_: database backup file. -- _option_: parameter option of the **pg\_dump** command. Multiple parameters can be separated by spaces. The common parameters of the **pg\_dump** command are as follows: - - **-f, \-\-file**= _filename_: specified output file. If this parameter is ignored, the standard output is used. - - **-d, \-\-dbname**= _databasename_: database to be dumped. - - **-h, \-\-host**= _hostname_: specifies the hostname. - - **-p, \-\-port**= _portnumber_: port number. - - **-U, \-\-username**= _username_: username of the connection. - - **-W, \-\-password**: forces PostgreSQL to prompt for a password before connecting to a database. - -##### Example - - Back up the database1 database of user **postgres** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. - -```shell -[postgres@localhost ~]$ pg_dump -h 192.168.202.144 -p 3306 -U postgres -W database1 > db1.sql -``` - -#### Restoring a Database - -Run the **psql** command to restore the database. - -```shell -psql [option]... [databasename [username]] < infile -``` - -In the preceding information: - -- _databasename_: database name. If this parameter is not specified, the environment variable **PGDATABASE** is used. If that environment variable is not specified, use the username that initiates the connection. -- _username_: name of a user. -- _infile_: **outfile** parameter in the **pg\_dump** command. -- _option_: parameter option of the **psql** command. Multiple parameters can be separated by spaces. The common parameters of the **psql** command are as follows: - - **-f, \-\-file**= _filename_: specified output file. If this parameter is ignored, the standard output is used. - - **-d, \-\-dbname**= _databasename_: database to be dumped. - - **-h, \-\-host**= _hostname_: specifies the hostname. - - **-p, \-\-port**= _portnumber_: port number. - - **-U, \-\-username**= _username_: username of the connection. - - **-W, \-\-password**: forces PostgreSQL to prompt for a password before connecting to a database. - -The **psql** command cannot be used to automatically create the **databasename** database. Therefore, you need to create the **databasename** database before running the **psql** command to restore the database. - -##### Example - - Import the **db1.sql** script file to the newdb database of the postgres user on the host **192.168.202.144** through port **3306**. - -```shell -[postgres@localhost ~]$ createdb newdb -[postgres@localhost ~]$ psql -h 192.168.202.144 -p 3306 -U postgres -W -d newdb < db1.sql -``` - -## MariaDB Server - -### Software Description - -The MariaDB database management system is a branch of MySQL and is maintained by the open-source community. The MariaDB database management system uses the General Public License \(GPL\). MariaDB is designed to be fully compatible with MySQL, including APIs and command lines, so that it can easily replace MySQL. MariaDB also provides many new features. - -[Figure 2](#fig13492418164520) shows the MariaDB architecture. - -**Figure 2** MariaDB logical architecture -![](./figures/mariadb-logical-architecture.png "mariadb-logical-architecture") - -When MariaDB receives a SQL statement, the execution process is as follows: - -1. When a client connects to MariaDB, the hostname, username, and password of the client are authenticated. The authentication function can be implemented as a plug-in. -2. If the login is successful, the client sends SQL commands to the server. The parser parses the SQL statements. -3. The server checks whether the client has the permission to obtain the required resources. -4. If the query has been stored in the query cache, the result is returned immediately. -5. The optimizer will find the fastest execution policy or plan. That is, the optimizer can determine which tables will be read, which indexes will be accessed, and which temporary tables will be used. A good policy can reduce a large number of disk access and sorting operations. -6. Storage engines read and write data and index files. Caches are used to accelerate these operations. Other features such as transactions and foreign keys are processed at the storage engine layer. - -Storage engines manage and control data at the physical layer. They manage data files, data, indexes, and caches, making data management and reading more efficient. Each table has a .frm file that contains table definitions. - -Each storage engine manages and stores data in different ways, and supports different features and performance. For example: - -- MyISAM: suitable for environments with more reads and fewer writes. It does not support transactions and supports full-text indexes. -- noDB: supports transactions, row locks, and foreign keys. -- MEMORY: stores data in the memory. -- CSV: stores data in CSV format. - -### Configuring the Environment - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The following environment configuration is for reference only. Configure the environment based on the site requirements. - -#### Disabling the Firewall and Automatic Startup - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements. - -1. Stop the firewall service as the **root** user. - - ```shell - systemctl stop firewalld - ``` - -2. Disable the firewall service as the **root** user. - - ```shell - systemctl disable firewalld - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The automatic startup is automatically disabled as the firewall is disabled. - -#### Disabling SELinux - -1. Modify the configuration file as the **root** user. - - ```shell - sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux - ``` - -#### Creating a User Group and a User - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database. - -1. Create a MySQL user or user group as the **root** user. - - ```shell - groupadd mysql - ``` - - ```shell - useradd -g mysql mysql - ``` - -2. Set the user password as the **root** user. - - ```shell - passwd mysql - ``` - - Enter the password twice for confirmation. - -#### Creating Data Drives - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1 or Method 2. ->- In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section. -> `mkdir /data` - -##### Method 1: Using fdisk for Drive Management as the **root** user - -1. Create a partition, for example, **/dev/sdb**. - - ```shell - fdisk /dev/sdb - ``` - -2. Enter **n** and press **Enter**. -3. Enter **p** and press **Enter**. -4. Enter **1** and press **Enter**. -5. Retain the default settings and press **Enter**. -6. Retain the default settings and press **Enter**. -7. Enter **w** and press **Enter**. -8. Create a file system, for example, **xfs**. - - ```shell - mkfs.xfs /dev/sdb1 - ``` - -9. Mount the partition to **/data** for the OS. - - ```shell - mkdir /data - ``` - - ```shell - mount /dev/sdb1 /data - ``` - -10. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. - - In the last line, **/dev/nvme0n1p1** is only an example. - - ![](./figures/creat_datadisk1.png) - -##### Method 2: Using LVM for Drive Management as the **root** user -> ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Install the LVM2 package in the image as follows: -> ->1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). If the repository has been configured, skip this step. ->2. Install LVM2. -> **yum install lvm2** - -1. Create a physical volume, for example, **sdb**. - - ```shell - pvcreate /dev/sdb - ``` - -2. Create a physical volume group, for example, **datavg**. - - ```shell - vgcreate datavg /dev/sdb - ``` - -3. Create a logical volume, for example, **datalv** of 600 GB. - - ```shell - lvcreate -L 600G -n datalv datavg - ``` - -4. Create a file system. - - ```shell - mkfs.xfs /dev/datavg/datalv - ``` - -5. Create a data directory and mount it. - - ```shell - mkdir /data - ``` - - ```shell - mount /dev/datavg/datalv /data - ``` - -6. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. - - In the last line, **/dev/datavg/datalv** is only an example. - - ![](./figures/d1376b2a-d036-41c4-b852-e8368f363b5e.png) - -#### Creating a Database Directory and Granting Permissions - -1. In the created data directory **/data**, create directories for processes and grant permissions to the MySQL group or user created as the **root** user. - - ```shell - mkdir -p /data/mariadb - cd /data/mariadb - mkdir data tmp run log - chown -R mysql:mysql /data - ``` - -### Installing, Running, and Uninstalling MariaDB Server - -#### Installing MariaDB - -1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). -2. Clear the cache. - - ```shell - dnf clean all - ``` - -3. Create a cache. - - ```shell - dnf makecache - ``` - -4. Install the MariaDB server. - - ```shell - dnf install mariadb-server - ``` - -5. Check the installed RPM package. - - ```shell - rpm -qa | grep mariadb - ``` - -#### Running MariaDB Server - -1. Start the MariaDB server as the **root** user. - - ```shell - systemctl start mariadb - ``` - -2. Initialize the database as the **root** user. - - ```shell - /usr/bin/mysql_secure_installation - ``` - - During the command execution, you need to enter the password of the database user **root**. If no password is set, press **Enter**. Then, set the password as prompted. - -3. Log in to the database. - - ```shell - mysql -u root -p - ``` - - After the command is executed, the system prompts you to enter the password. The password is the one set in [2](#li197143190587). - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Run the **\\q** or **exit** command to exit the database. - -#### Uninstalling MariaDB - -1. Stop the database process as the **root** user. - - ```shell - $ ps -ef | grep mysql - # kill -9 PID - ``` - -2. Run the **dnf remove mariadb-server** command as the **root** user to uninstall MariaDB. - - ```shell - dnf remove mariadb-server - ``` - -### Managing Database Users - -#### Creating Users - -Run the **CREATE USER** statement to create one or more users and set corresponding passwords. - -```pgsql -CREATE USER 'username'@'hostname' IDENTIFIED BY 'password'; -``` - -In the preceding information: - -- _username_: name of a user. -- _host_: hostname, that is, the name of the host where the user connects to the database. As a local user, you can set the parameter to **localhost**. If the host name is not specified during user creation, the host name is **%** by default, indicating a group of hosts. -- _password_: password for logging in to the server. The password can be null. If the password is null, the user can log in to the server without entering the password. This method, however, is not recommended because it provides low security. - -To use the **CREATE USER** statement, you must have the INSERT permission on the database or the global CREATE USER permission. - -After a user account is created using the **CREATE USER** statement, a record is added to the user table in the database. If the account to be created exists, an error will occur during statement execution. - -A new user has few permissions and can perform only operations that do not require permissions. For example, a user can run the **SHOW** statement to query the list of all storage engines and character sets. - -##### Example - - Create a local user whose password is 123456 and username is userexample1. - -```pgsql -> CREATE USER 'userexample1'@'localhost' IDENTIFIED BY '123456'; -``` - - Create a user whose password is 123456, username is userexample2, and hostname is 192.168.1.100. - -```pgsql -> CREATE USER 'userexample2'@'192.168.1.100' IDENTIFIED BY '123456'; -``` - -#### Viewing Users - -Run the **SHOW GRANTS** or **SELECT** statement to view one or more users. - -View a specific user: - -```pgsql -SHOW GRANTS [FOR 'username'@'hostname']; -``` - -```pgsql -SELECT USER,HOST,PASSWORD FROM mysql.user WHERE USER='username'; -``` - -View all users: - -```pgsql -SELECT USER,HOST,PASSWORD FROM mysql.user; -``` - -In the preceding information: - -- _username_: name of a user. -- _hostname_: host name. - -##### Example - - View the user userexample1. - -```pgsql -> SHOW GRANTS FOR 'userexample1'@'localhost'; -``` - - View all users in the MySQL database. - -```pgsql -> SELECT USER,HOST,PASSWORD FROM mysql.user; -``` - -#### Modifying Users - -##### Modifying a Username - -Run the **RENAME USER** statement to change one or more existing usernames. - -```pgsql -RENAME USER 'oldusername'@'hostname' TO 'newusername'@'hostname'; -``` - -In the preceding information: - -- _oldusername_: original username. -- _newusername_: new username. -- _hostname_: host name. - -The **RENAME USER** statement is used to rename an existing account. If the original account does not exist in the system or the new account exists, an error will occur when the statement is executed. - -To use the **RENAME USER** statement, you must have the UPDATE permission on the database or the global CREATE USER permission. - -##### Example of Modifying a User - - Change the username **userexample1** to **userexample2** and change the hostname to **localhost**. - -```pgsql -> RENAME USER 'userexample1'@'localhost' TO 'userexample2'@'localhost'; -``` - -##### Modifying a User Password - -Use the **SET PASSWORD** statement to modify the login password of a user. - -```pgsql -SET PASSWORD FOR 'username'@'hostname' = PASSWORD('newpassword'); -``` - -In the preceding information: - -- **FOR 'username'@'hostname'**: specifies the username and hostname whose password is to be changed. This parameter is optional. -- **PASSWORD\('newpassword'\)**: indicates that the **PASSWORD\(\)** function is used to set a new password. That is, the new password must be transferred to the **PASSWORD\(\)** function for encryption. - ->![](./public_sys-resources/icon-caution.gif) **CAUTION:** ->The **PASSWORD\(\)** function is a unidirectional encryption function. Once encrypted, the original plaintext cannot be decrypted. - -If the **FOR** clause is not added to the **SET PASSWORD** statement, the password of the current user is changed. - -The **FOR** clause must be given in the format of **'**_username_**'@'**_hostname_**'**, where _username_ indicates the username of the account and _hostname_ indicates the hostname of the account. - -The account whose password is to be changed must exist in the system. Otherwise, an error occurs when the statement is executed. - -##### Example of Changing a User Password - - Change the password of user **userexample** whose hostname is **localhost** to **0123456**. - -```pgsql -> SET PASSWORD FOR 'userexample'@'localhost' = PASSWORD('0123456') ; -``` - -#### Deleting Users - -Use the **DROP USER** statement to delete one or more user accounts and related permissions. - -```pgsql -DROP USER 'username1'@'hostname1' [,'username2'@'hostname2']...; -``` - ->![](./public_sys-resources/icon-caution.gif) **CAUTION:** ->The deletion of users does not affect the tables, indexes, or other database objects that they have created, because the database does not record the accounts that have created these objects. - -The **DROP USER** statement can be used to delete one or more database accounts and their original permissions. - -To use the **DROP USER** statement, you must have the DELETE permission on the database or the global CREATE USER permission. - -In the **DROP USER** statement, if the hostname of an account is not specified, the hostname is **%** by default. - -##### Example - - Delete the local user **userexample**. - -```pgsql -> DROP USER 'userexample'@'localhost'; -``` - -#### Granting Permissions to a User - -Run the **GRANT** statement to grant permissions to a new user. - -```pgsql -GRANT privileges ON databasename.tablename TO 'username'@'hostname'; -``` - -In the preceding information: - -- **ON** clause: specifies the object and its level on which the permission is granted. -- **privileges**: indicates the operation permissions of a user, such as **SELECT**, INSERT, and **UPDATE**. To grant all permissions to a user, use **ALL**. -- _databasename_: database name. -- _tablename_: table name. -- **TO** clause: sets the user password and specifies the user to whom the permission is granted. -- _username_: name of a user. -- _hostname_: host name. - -To grant the user the permission to operate all databases and tables, use asterisks \(\*\), for example, **\*.\***. - -If you specify a password for an existing user in the **TO** clause, the new password will overwrite the original password. - -If the permission is granted to a non-existent user, a **CREATE USER** statement is automatically executed to create the user, but the password must be specified for the user. - -##### Example - - Grant the SELECT and INSERT permissions to local user userexample. - -```pgsql -> GRANT SELECT,INSERT ON *.* TO 'userexample'@'localhost'; -``` - -#### Deleting User Permissions - -Run the **REVOKE** statement to delete the permissions of a user, but the user will not be deleted. - -```pgsql -REVOKE privilege ON databasename.tablename FROM 'username'@'hostname'; -``` - -The parameters in the **REVOKE** statement are the same as those in the **GRANT** statement. - -To use the **REVOKE** statement, you must have the global CREATE USER or UPDATE permission for the database. - -##### Example - - Delete the INSERT permission of local user userexample. - -```pgsql -> REVOKE INSERT ON *.* FROM 'userexample'@'localhost'; -``` - -### Managing Databases - -#### Creating a Database - -Run the **CREATE DATABASE** statement to create a database. - -```pgsql -CREATE DATABASE databasename; -``` - -In the preceding command, _databasename_ can be replaced with the database name, which is case insensitive. - -##### Example - - Create a database named **databaseexample**. - -```pgsql -> CREATE DATABASE databaseexample; -``` - -#### Viewing a Database - -Run the **SHOW DATABASES** statement to view a database. - -```pgsql -SHOW DATABASES; -``` - -##### Example - - View all databases. - -```pgsql -> SHOW DATABASES; -``` - -#### Selecting a Database - -Generally, you need to select a target database before creating or querying a table. Use the **USE** statement to select a database. - -```pgsql -USE databasename; -``` - -In the preceding command, **databasename** indicates the database name. - -##### Example - - Select the **databaseexample** database. - -```pgsql -> USE databaseexample; -``` - -#### Deleting a Database - -You can run the **DROP DATABASE** statement to delete a database. - ->![](./public_sys-resources/icon-caution.gif) **CAUTION:** ->Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted. - -```pgsql -DROP DATABASE databasename; -``` - -In the preceding command, **databasename** indicates the database name. - -The **DROP DATABASE** command is used to delete an existing database. After this command is executed, all tables in the database are deleted, but the user permissions of the database are not automatically deleted. - -To use **DROP DATABASE**, you need the **DROP** permission on the database. - -**DROP SCHEMA** is a synonym of **DROP DATABASE**. - -##### Example - - Delete the **databaseexample** database. - -```pgsql -> DROP DATABASE databaseexample; -``` - -#### Backing Up a Database - -Run the **mysqldump** command as the **root** user to back up the database. - -Back up one or more tables. - -```shell -mysqldump [options] databasename [tablename ...] > outfile -``` - -Back up one or more databases: - -```shell -mysqldump [options] -databases databasename ... > outfile -``` - -Back up all databases: - -```shell -mysqldump [options] -all-databases > outputfile -``` - -In the preceding information: - -- _databasename_: database name. -- _tablename_: name of a data table. -- _outfile_: database backup file. -- _options_: parameter option of the **mysqldump** command. Multiple parameters can be separated by spaces. The common parameters of the **mysqldump** command are as follows: - - **-u, \-\-user**= _username_: specifies the username. - - **-p, \-\-password**\[= _password_\]: specifies the password. - - **-P, \-\-port**= _portnumber_: specifies the port number. - - **-h, \-\-host**= _hostname_: specifies the hostname. - - **-r, \-\-result-file**= _filename_: saves the export result to a specified file, which is equivalent to **\>**. - - **-t**: backs up data only. - - **-d**: backs up the table structure only. - -##### Example - - Back up all the databases of the user **root** on the host **192.168.202.144** through port **3306** to the **alldb.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --all-databases > alldb.sql -``` - - Back up the db1 database of the user **root** on the host **192.168.202.144** through port **3306** to the **db1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --databases db1 > db1.sql -``` - - Back up the tb1 table of the db1 database of the user **root** on the host **192.168.202.144** through port **3306** to the **db1tb1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 db1 tb1 > db1tb1.sql -``` - - Back up only the table structure of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -d db1 > db1.sql -``` - - Back up only the data of the db1 database of the user **root** on the host **192.168.202.144** through port **3306** to the **db1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 > db1.sql -``` - -#### Restoring a Database - -Run the **mysql** command as the **root** user to restore the database. - -Restore one or more tables: - -```shell -mysql -h hostname -P portnumber -u username -ppassword databasename < infile -``` - -In the preceding information: - -- _hostname_: host name. -- _portnumber_: port number. -- _username_: name of a user. -- _password_: password. -- _databasename_: database name. -- _infile_: **outfile** parameter in the **mysqldump** command. - -##### Example - - Restore a database. - -```shell -mysql -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 < db1.sql -``` - -## MySQL Server - -### Software Description - -MySQL is a relational database management system \(RDBMS\) developed by the Swedish company MySQL AB, which was bought by Sun Microsystems \(now Oracle\). It is one of the most popular Relational Database Management Systems \(RDBMSs\) in the industry, especially for web applications. - -A relational database stores data in different tables instead of in a large data warehouse to improve efficiency and flexibility. - -The Structured Query Language \(SQL\) used by MySQL is the most common standard language for accessing databases. MySQL uses dual-licensing distribution and is available in two editions: Community Edition and Commercial Edition. MySQL is optimal for small or medium-sized websites because of its small size, fast speed, low cost, and especially the open source code. - -### Configuring the Environment - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The following environment configuration is for reference only. Configure the environment based on the site requirements. - -#### Disabling the Firewall and Automatic Startup - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements. - -1. Stop the firewall service as the **root** user. - - ```shell - systemctl stop firewalld - ``` - -2. Disable the firewall service as the **root** user. - - ```shell - systemctl disable firewalld - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The automatic startup is automatically disabled as the firewall is disabled. - -#### Disabling SELinux - -1. Modify the configuration file as the **root** user. - - ```shell - sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux - ``` - -#### Creating a User Group and a User - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database. - -1. Create a MySQL user or user group as the **root** user. - - ```shell - groupadd mysql - ``` - - ```shell - useradd -g mysql mysql - ``` - -2. Set the user password as the **root** user. - - ```shell - passwd mysql - ``` - - Enter the password twice for confirmation. - -#### Creating Data Drives - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1 or Method 2. ->- In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section. -> `mkdir /data` - -##### Method 1: Using fdisk for Drive Management as the **root** user - -1. Create a partition, for example, **/dev/sdb**. - - ```shell - fdisk /dev/sdb - ``` - -2. Enter **n** and press **Enter**. -3. Enter **p** and press **Enter**. -4. Enter **1** and press **Enter**. -5. Retain the default settings and press **Enter**. -6. Retain the default settings and press **Enter**. -7. Enter **w** and press **Enter**. -8. Create a file system, for example, **xfs**. - - ```shell - mkfs.xfs /dev/sdb1 - ``` - -9. Mount the partition to **/data** for the OS. - - ```shell - mkdir /data - ``` - - ```shell - mount /dev/sdb1 /data - ``` - -10. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. - - In the last line, **/dev/nvme0n1p1** is only an example. - - ![](./figures/creat_datadisk.png) - -##### Method 2: Using LVM for Drive Management as the **root** user -> ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Install the LVM2 package in the image as follows: -> ->1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). If the repository has been configured, skip this step. ->2. Install LVM2. -> **yum install lvm2** - -1. Create a PV, for example, **sdb**. - - ```shell - pvcreate /dev/sdb - ``` - -2. Create a physical VG, for example, **datavg**. - - ```shell - vgcreate datavg /dev/sdb - ``` - -3. Create an LV, for example, **datalv** of 600 GB. - - ```shell - lvcreate -L 600G -n datalv datavg - ``` - -4. Create a file system. - - ```shell - mkfs.xfs /dev/datavg/datalv - ``` - -5. Create a data directory and mount it. - - ```shell - mkdir /data - ``` - - ```shell - mount /dev/datavg/datalv /data - ``` - -6. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. - - In the last line, **/dev/datavg/datalv** is only an example. - - ![](./figures/d1376b2a-d036-41c4-b852-e8368f363b5e-1.png) - -#### Creating a Database Directory and Granting Permissions - -1. In the created data directory **/data**, create directories for processes and grant permissions to the MySQL group or user created as the **root** user. - - ```shell - mkdir -p /data/mysql - cd /data/mysql - mkdir data tmp run log - chown -R mysql:mysql /data - ``` - -### Installing, Running, and Uninstalling MySQL - -#### Installing MySQL - -1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). -2. Clear the cache. - - ```shell - dnf clean all - ``` - -3. Create a cache. - - ```shell - dnf makecache - ``` - -4. Install the MySQL server as the **root** user. - - ```shell - dnf install mysql-server - ``` - -5. Check the installed RPM package. - - ```shell - rpm -qa | grep mysql-server - ``` - -#### Running MySQL - -1. Modify the configuration file. - 1. Create the **my.cnf** file as the **root** user and change the file paths \(including the software installation path **basedir** and data path **datadir**\) based on the actual situation. - - ```shell - vi /etc/my.cnf - ``` - - Edit the **my.cnf** file as follows: - - ```shell - [mysqld_safe] - log-error=/data/mysql/log/mysql.log - pid-file=/data/mysql/run/mysqld.pid - [mysqldump] - quick - [mysql] - no-auto-rehash - [client] - default-character-set=utf8 - [mysqld] - basedir=/usr/local/mysql - socket=/data/mysql/run/mysql.sock - tmpdir=/data/mysql/tmp - datadir=/data/mysql/data - default_authentication_plugin=mysql_native_password - port=3306 - user=mysql - ``` - - 2. Ensure that the **my.cnf** file is correctly modified. - - ```shell - cat /etc/my.cnf - ``` - - ![](./figures/en-us_image_0231563132.png) - - >![](./public_sys-resources/icon-caution.gif) **CAUTION:** - >In the configuration file, **basedir** specifies the software installation path. Change it based on actual situation. - - 3. Change the group and user of the **/etc/my.cnf** file to **mysql:mysql** as the **root** user. - - ```shell - chown mysql:mysql /etc/my.cnf - ``` - -2. Configure environment variables. - 1. Add the path of the MySQL binary files to the **PATH** parameter as the **root** user. - - ```shell - echo export PATH=$PATH:/usr/local/mysql/bin >> /etc/profile - ``` - - >![](./public_sys-resources/icon-caution.gif) **CAUTION:** - >In the command, **/usr/local/mysql/bin** is the absolute path of the **bin** files in the MySQL software installation directory. Change it based on actual situation. - - 2. Run the following command as the **root** user to make the environment variables take effect: - - ```shell - source /etc/profile - ``` - -3. Initialize the database as the **root** user. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The second line from the bottom contains the initial password, which will be used when you log in to the database. - - ```shell - $ mysqld --defaults-file=/etc/my.cnf --initialize - 2020-03-18T03:27:13.702385Z 0 [System] [MY-013169] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.17) initializing of server in progress as process 34014 - 2020-03-18T03:27:24.112453Z 5 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: iNat=)#V2tZu - 2020-03-18T03:27:28.576003Z 0 [System] [MY-013170] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.17) initializing of server has completed - ``` - - If the command output contains "initializing of server has completed", the database has been initialized. In the command output, "iNat=\)\# V2tZu" in "A temporary password is generated for root@localhost: iNat=\)\# V2tZu" is the initial password. - -4. Start the database. - - >![](./public_sys-resources/icon-caution.gif) **CAUTION:** - >Start MySQL as user **mysql** if it is the first time to start the database service. If you start MySQL as user **root**, a message will be displayed indicating that the **mysql.log** file is missing. If you start MySQL as user **mysql**, the **mysql.log** file will be generated in the **/data/mysql/log** directory. No error will be displayed if you start the database as user **root** again. - - 1. Modify the file permission as the **root** user. - - ```shell - chmod 777 /usr/local/mysql/support-files/mysql.server - chown mysql:mysql /var/log/mysql/* - ``` - - 2. Start MySQL as the **root** user. - - ```shell - cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysql - chkconfig mysql on - ``` - - Start MySQL as user **mysql**. - - ```shell - su - mysql - service mysql start - ``` - -5. Log in to the database. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- Enter the initial password generated during database initialization \([3](#li15634560582)\). - >- If MySQL is installed by using an RPM package obtained from the official website, the **mysqld** file is located in the **/usr/sbin** directory. Ensure that the directory specified in the command is correct. - - ```shell - /usr/local/mysql/bin/mysql -uroot -p -S /data/mysql/run/mysql.sock - ``` - - ![](./figures/en-us_image_0231563134.png) - -6. Configure the database accounts and passwords. - 1. After logging in to the database, change the password of user **root** for logging in to the database. - - ```shell - mysql>alter user 'root'@'localhost' identified by "123456"; - ``` - - 2. Create a user **root** for all the other hosts in the domain. - - ```shell - mysql>create user 'root'@'%' identified by '123456'; - ``` - - 3. Grant permissions to the user **root**. - - ```shell - mysql>grant all privileges on *.* to 'root'@'%'; - mysql>flush privileges; - ``` - - ![](./figures/en-us_image_0231563135.png) - -7. Exit the database. - - Run the **\\q** or **exit** command to exit the database. - - ```shell - mysql>exit - ``` - - ![](./figures/en-us_image_0231563136.png) - -#### Uninstalling MySQL - -1. Stop the database process as the **root** user. - - ```shell - $ ps -ef | grep mysql - # kill -9 PID - ``` - -2. Run the **dnf remove mysql** command as the **root** user to uninstall MySQL. - - ```shell - dnf remove mysql - ``` - -### Managing Database Users - -#### Creating Users - -Run the **CREATE USER** statement to create one or more users and set corresponding passwords. - -```pgsql -CREATE USER 'username'@'hostname' IDENTIFIED BY 'password'; -``` - -In the preceding information: - -- _username_: name of a user. -- _host_: hostname, that is, the name of the host where the user connects to the database. As a local user, you can set the parameter to **localhost**. If the host name is not specified during user creation, the host name is **%** by default, indicating a group of hosts. -- _password_: password for logging in to the server. The password can be null. If the password is null, the user can log in to the server without entering the password. This method, however, is not recommended because it provides low security. - -To use the **CREATE USER** statement, you must have the **INSERT** permission on the database or the global **CREATE USER** permission. - -After a user account is created using the **CREATE USER** statement, a record is added to the user table in the database. If the account to be created exists, an error will occur during statement execution. - -A new user has few permissions and can perform only operations that do not require permissions. For example, a user can run the **SHOW** statement to query the list of all storage engines and character sets. - -##### Example - - Create a local user whose password is **123456** and username is **userexample1**. - -```pgsql -> CREATE USER 'userexample1'@'localhost' IDENTIFIED BY '123456'; -``` - - Create a user whose password is **123456**, username is **userexample2**, and hostname is **192.168.1.100**. - -```pgsql -> CREATE USER 'userexample2'@'192.168.1.100' IDENTIFIED BY '123456'; -``` - -#### Viewing Users - -Run the **SHOW GRANTS** or **SELECT** statement to view one or more users. - -View a specific user: - -```pgsql -SHOW GRANTS [FOR 'username'@'hostname']; -``` - -```pgsql -SELECT USER,HOST,PASSWORD FROM mysql.user WHERE USER='username'; -``` - -View all users: - -```pgsql -SELECT USER,HOST FROM mysql.user; -``` - -In the preceding information: - -- _username_: name of a user. -- _hostname_: host name. - -##### Example - - View the user **userexample1**. - -```pgsql -> SHOW GRANTS FOR 'userexample1'@'localhost'; -``` - - View all users in the MySQL database. - -```pgsql -> SELECT USER,HOST FROM mysql.user; -``` - -#### Modifying Users - -##### Modifying a Username - -Run the **RENAME USER** statement to change one or more existing usernames. - -```pgsql -RENAME USER 'oldusername'@'hostname' TO 'newusername'@'hostname'; -``` - -In the preceding information: - -- _oldusername_: original username. -- _newusername_: new username. -- _hostname_: host name. - -The **RENAME USER** statement is used to rename an existing account. If the original account does not exist in the system or the new account exists, an error will occur when the statement is executed. - -To use the **RENAME USER** statement, you must have the **UPDATE** permission on the database or the global **CREATE USER** permission. - -##### Example of Modifying a User - - Change the username **userexample1** to **userexample2** and change the hostname to **localhost**. - -```pgsql -> RENAME USER 'userexample1'@'localhost' TO 'userexample2'@'localhost'; -``` - -##### Modifying a User Password - -Use the **SET PASSWORD** statement to modify the login password of a user. - -```pgsql -SET PASSWORD FOR 'username'@'hostname' = 'newpassword'; -``` - -In the preceding information: - -- **FOR'**_username_**'@'**_hostname_**'**: specifies the username and hostname whose password is to be changed. This parameter is optional. -- _newpassword_: new password. - -If the **FOR** clause is not added to the **SET PASSWORD** statement, the password of the current user is changed. - -The **FOR** clause must be given in the format of **'**_username_**'@'**_hostname_**'**, where _username_ indicates the username of the account and _hostname_ indicates the hostname of the account. - -The account whose password is to be changed must exist in the system. Otherwise, an error occurs when the statement is executed. - -##### Example of Changing a User Password - - Change the password of user **userexample** whose hostname is **localhost** to **0123456**. - -```pgsql -> SET PASSWORD FOR 'userexample'@'localhost' = '0123456'; -``` - -#### Deleting Users - -Use the **DROP USER** statement to delete one or more user accounts and related permissions. - -```pgsql -DROP USER 'username1'@'hostname1' [,'username2'@'hostname2']...; -``` - ->![](./public_sys-resources/icon-caution.gif) **CAUTION:** ->The deletion of users does not affect the tables, indexes, or other database objects that they have created, because the database does not record the accounts that have created these objects. - -The **DROP USER** statement can be used to delete one or more database accounts and their original permissions. - -To use the **DROP USER** statement, you must have the **DELETE** permission on the database or the global **CREATE USER** permission. - -In the **DROP USER** statement, if the hostname of an account is not specified, the hostname is **%** by default. - -##### Example - - Delete the local user **userexample**. - -```pgsql -> DROP USER 'userexample'@'localhost'; -``` - -#### Granting Permissions to a User - -Run the **GRANT** statement to grant permissions to a new user. - -```pgsql -GRANT privileges ON databasename.tablename TO 'username'@'hostname'; -``` - -In the preceding information: - -- **ON** clause: specifies the object and level on which the permission is granted. -- _privileges_: indicates the operation permissions of a user, such as **SELECT**, INSERT, and **UPDATE**. To grant all permissions to a user, use **ALL**. -- _databasename_: database name. -- _tablename_: table name. -- **TO** clause: sets the user password and specifies the user to whom the permission is granted. -- _username_: name of a user. -- _hostname_: host name. - -To grant the user the permission to operate all databases and tables, use asterisks \(\*\), for example, **\*.\***. - -If you specify a password for an existing user in the **TO** clause, the new password will overwrite the original password. - -If the permission is granted to a non-existent user, a **CREATE USER** statement is automatically executed to create the user, but the password must be specified for the user. - -##### Example - - Grant the **SELECT** and **INSERT** permissions to local user **userexample**. - -```pgsql -> GRANT SELECT,INSERT ON *.* TO 'userexample'@'localhost'; -``` - -#### Deleting User Permissions - -Run the **REVOKE** statement to delete the permissions of a user, but the user will not be deleted. - -```pgsql -REVOKE privilege ON databasename.tablename FROM 'username'@'hostname'; -``` - -The parameters in the **REVOKE** statement are the same as those in the **GRANT** statement. - -To use the **REVOKE** statement, you must have the global **CREATE USER** or **UPDATE** permission for the database. - -##### Example - - Delete the **INSERT** permission of local user **userexample**. - -```pgsql -> REVOKE INSERT ON *.* FROM 'userexample'@'localhost'; -``` - -### Managing Databases - -#### Creating a Database - -Run the **CREATE DATABASE** statement to create a database. - -```pgsql -CREATE DATABASE databasename; -``` - -In the preceding command, _databasename_ can be replaced with the database name, which is case insensitive. - -##### Example - - Create a database named **databaseexample**. - -```pgsql -> CREATE DATABASE databaseexample; -``` - -#### Viewing a Database - -Run the **SHOW DATABASES** statement to view a database. - -```pgsql -SHOW DATABASES; -``` - -##### Example - - View all databases. - -```pgsql -> SHOW DATABASES; -``` - -#### Selecting a Database - -Generally, you need to select a target database before creating or querying a table. Use the **USE** statement to select a database. - -```pgsql -USE databasename; -``` - -In the preceding command, _databasename_ indicates the database name. - -##### Example - - Select the **databaseexample** database. - -```pgsql -> USE databaseexample; -``` - -#### Deleting a Database - -Run the **DROP DATABASE** statement to delete a database. - ->![](./public_sys-resources/icon-caution.gif) **CAUTION:** ->Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted. - -```pgsql -DROP DATABASE databasename; -``` - -In the preceding command, _databasename_ indicates the database name. - -The **DROP DATABASE** command is used to delete an existing database. After this command is executed, all tables in the database are deleted, but the user permissions of the database are not automatically deleted. - -To use **DROP DATABASE**, you need the **DROP** permission on the database. - -**DROP SCHEMA** is a synonym of **DROP DATABASE**. - -##### Example - - Delete the **databaseexample** database. - -```pgsql -> DROP DATABASE databaseexample; -``` - -#### Backing Up a Database - -Run the **mysqldump** command as the **root** user to back up the database. - -Back up one or more tables: - -```shell -mysqldump [options] databasename [tablename ...] > outfile -``` - -Back up one or more databases: - -```shell -mysqldump [options] -databases databasename ... > outfile -``` - -Back up all databases: - -```shell -mysqldump [options] -all-databases > outputfile -``` - -In the preceding information: - -- _databasename_: database name. -- _tablename_: name of a data table. -- _outfile_: database backup file. -- _options_: parameter option of the **mysqldump** command. Multiple parameters can be separated by spaces. The common parameters of the **mysqldump** command are as follows: - - **-u, \-\-user**= _username_: specifies the username. - - **-p, \-\-password**\[= _password_\]: specifies the password. - - **-P, \-\-port**= _portnumber_: specifies the port number. - - **-h, \-\-host**= _hostname_: specifies the hostname. - - **-r, \-\-result-file**= _filename_: saves the export result to a specified file, which is equivalent to **\>**. - - **-t**: backs up data only. - - **-d**: backs up the table structure only. - -##### Example - - Back up all the databases of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **alldb.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --all-databases > alldb.sql -``` - - Back up the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --databases db1 > db1.sql -``` - - Back up the tb1 table of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1tb1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 db1 tb1 > db1tb1.sql -``` - - Back up only the table structure of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -d db1 > db1.sql -``` - - Back up only the table structure of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. - -```shell -mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 > db1.sql -``` - -#### Restoring a Database - -Run the **mysql** command as the **root** user to restore the database. - -Restore one or more tables: - -```shell -mysql -h hostname -P portnumber -u username -ppassword databasename < infile -``` - -In the preceding information: - -- _hostname_: host name. -- _portnumber_: port number. -- _username_: name of a user. -- _password_: password. -- _databasename_: database name. -- _infile_: **outfile** parameter in the **mysqldump** command. - -##### Example - - Restore a database. - -```shell -mysql -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 < db1.sql -``` diff --git a/docs/en/docs/Administration/trusted-computing.md b/docs/en/docs/Administration/trusted-computing.md deleted file mode 100644 index 5ba4816f745e1d327673467a9882645ae6b1d100..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/trusted-computing.md +++ /dev/null @@ -1,1844 +0,0 @@ -# Trusted Computing - -## Trusted Computing Basics - -### What Is Trusted Computing - -The definition of being trusted varies with international organizations. - -1. Trusted Computing Group (TCG): - - An entity that is trusted always achieves the desired goal in an expected way. - -2. International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) (1999): - - The components, operations, or processes involved in computing are predictable under any conditions and are resistant to viruses and a certain degree of physical interference. - -3. IEEE Computer Society Technical Committee on Dependable Computing: - - Being trusted means that the services provided by the computer system can be proved to be reliable, and mainly refers to the reliability and availability of the system. - -In short, being trusted means that the system operates according to a pre-determined design and policy. - -A trusted computing system consists of a root of trust, a trusted hardware platform, operating system (OS), and application. The basic idea of the system is to create a trusted computing base (TCB) first, and then establish a trust chain that covers the hardware platform, OS, and application. In the trust chain, authentication is performed from the root to the next level, extending trust level by level and building a secure and trusted computing environment. - -![](./figures/trusted_chain.png) - -Unlike the traditional security mechanism that eliminates viruses without solving the root of the problem, trusted computing adopts the whitelist mechanism to allow only authorized kernels, kernel modules, and applications to run on the system. The system will reject the execution of a program that is unknown or has been changed. - -## Kernel Integrity Measurement Architecture (IMA) - -### Overview - -#### IMA - -The integrity measurement architecture (IMA) is a subsystem in the kernel. The IMA can measure files accessed through **execve()**, **mmap()**, and **open()** systems based on user-defined policies. The measurement result can be used for **local or remote attestation**, or can be compared with an existing reference value to **control the access to files**. - -According to the Wiki definition, the function of the kernel integrity subsystem includes three parts: - -- Measure: Detects accidental or malicious modifications to files, either remotely or locally. -- Appraise: Measures a file and compares it with a reference value stored in the extended attribute to control the integrity of the local file. -- Audit: Writes the measurement result into system logs for auditing. - -Figuratively, IMA measurement is an observer that only records modification without interfering in it, and IMA appraisal is more like a strict security guard that rejects any unauthorized access to programs. - -#### EVM - -The extended verification module (EVM) is used to calculate a hash value based on the security extended attributes of a file in the system, including **security.ima** and **security.selinux**. Then this value is signed by the key stored in the TPM or other trusted environments. The signature value is stored in **security.evm** and cannot be tampered with. If the value is tampered with, the signature verification fails when the file is accessed again. - -In summary, the EVM is used to provide offline protection for security extended attributes by calculating the digest of the attributes and signing and storing them in **security.evm**. - -#### IMA Digest Lists - -IMA Digest Lists are an enhancement of the original kernel integrity protection mechanism provided by openEuler. It replaces the original IMA mechanism to protect file integrity. - -Digest lists are binary data files in a special format. Each digest list corresponds to an RPM package and records the hash values of protected files (executable files and dynamic library files) in the RPM package. - -After the startup parameters are correctly configured, the kernel maintains a hash table (invisible to the user space) and provides interfaces (**digest\_list\_data** and **digest\_list\_data\_del**) that update the hash table using **securityfs**. The digest lists are signed by the private key when they are built. When uploaded to the kernel through the interface, the digest lists need to be verified by the public key in the kernel. - -![](./figures/ima_digest_list_update.png) - -When IMA appraisal is enabled, each time an executable file or dynamic library file is accessed, the hook in the kernel is invoked to calculate the hash values of the file content and extended attributes and search in the kernel hash table. If the calculated hash values match the one in the table, the file is allowed to be executed. Otherwise, the access is denied. - -![1599719649188](./figures/ima_verification.png) - -The IMA Digest Lists extension provided by the openEuler kernel provides higher security, performance, and usability than the native IMA mechanism of the kernel community, facilitating the implementation of the integrity protection mechanism in the production environment. - -- **A complete trust chain for high security** - - The native IMA mechanism requires that the file extended attribute be generated and marked in advance on the live network. When the file is accessed, the file extended attribute is used as a reference value, resulting in an incomplete trust chain. - - The IMA Digest Lists extension saves the reference digest value of the file in the kernel space. During the construction, the reference digest value of the file is carried in the released RPM package in the form of a digest list. When the RPM package is installed, the digest list is imported and the signature is verified, ensuring that the reference value comes from the software publisher and implementing a complete trust chain. - -- **Superior performance** - - The trusted platform module (TPM) chip is a low-speed chip, making the PCR extension operation a performance bottleneck in the IMA measurement scenario. To shatter this bottleneck, the Digest Lists extension reduces unnecessary PCR extension operations while ensuring security, providing 65% higher performance than the native IMA mechanism. - - In the IMA appraisal scenario, the Digest Lists extension performs signature verification in the startup phase to prevent signature verification from being performed each time the file is accessed. This helps deliver a 20% higher file access performance in the operation phase than that in the native IMA appraisal scenario. - -- **Fast deployment and smooth upgrade** - - When the native IMA mechanism is deployed for the first time or the software package is updated, you need to switch to the fix mode, manually mark the extended attributes of the file, and then restart the system to enter the enforcing mode. In this way, the installed program can be accessed normally. - - The Digest Lists extension can be used immediately after the installation is completed. In addition, the RPM package can be directly installed or upgraded in the enforcing mode without restarting the system or manually marking the extended attributes of the file. This minimizes user perception during the operation, allowing for quick deployment and smooth upgrade on the live network. - -Note: The IMA Digest Lists extension advances the signature verification of the native IMA to the startup phase. This causes the assumption that the memory in the kernel space cannot be tampered with. As a result, the IMA depends on other security mechanisms (secure startup of kernel module and dynamic memory measurement) to protect the integrity of the kernel memory. - -However, either the native IMA mechanism of the community or the IMA Digest Lists extension is only a link in the trust chain of trusted computing, and cannot ensure the system security alone. Security construction is always a systematic project that builds in-depth defense. - -### Constraints - -1. The current IMA appraisal mode can only protect immutable files in the system, including executable files and dynamic library files. -2. The IMA provides integrity measurement at the application layer. The security of the IMA depends on the reliability of the previous links. -3. Currently, the IMA does not support the import of the third-party application digest lists. -4. The startup log may contain `Unable to open file: /etc/keys/x509_ima.der`. This error is reported from the open source community and does not affect the use of the IMA digest lists feature. -5. In the ARM version, audit errors may occur when the log mode is enabled for the IMA. This occurs because the modprobe loads the kernel module before the digest lists are imported, but does not affect the normal functions. - -### Application Scenario - -#### IMA Measurement - -The purpose of IMA measurement is to detect unexpected or malicious modifications to system files. The measurement result can be used for local or remote attestation. - -If a TPM chip exists in the system, the measurement result is extended to a specified PCR register of the TPM chip. Due to the unidirectional PCR extension and the hardware security of the TPM chip, a user cannot modify the extended measurement result, thereby ensuring authenticity of the measurement result. - -The file scope and triggering conditions of IMA measurement can be configured by the user using the IMA policy. - -By default, IMA is disabled. However, the system searches for the **ima-policy** policy file in the `/etc/ima/` path. If the file is found, the system measures the files in the system based on the policy during startup. If you do not want to manually compile the policy file, you can configure the `ima_policy=tcb` in the startup parameters using the default policy. For details about more policy parameters, see the section *IMA Startup Parameters* in *Appendix*. - -You can check the currently loaded IMA policy in the `/sys/kernel/security/ima/policy` file. The IMA measurement log is located in the `/sys/kernel/security/ima/ascii_runtime_measurements` file, as shown in the following figure: - -```shell -$ head /sys/kernel/security/ima/ascii_runtime_measurements -10 ddee6004dc3bd4ee300406cd93181c5a2187b59b ima-ng sha1:9797edf8d0eed36b1cf92547816051c8af4e45ee boot_aggregate -10 180ecafba6fadbece09b057bcd0d55d39f1a8a52 ima-ng sha1:db82919bf7d1849ae9aba01e28e9be012823cf3a /init -10 ac792e08a7cf8de7656003125c7276968d84ea65 ima-ng sha1:f778e2082b08d21bbc59898f4775a75e8f2af4db /bin/bash -10 0a0d9258c151356204aea2498bbca4be34d6bb05 ima-ng sha1:b0ab2e7ebd22c4d17d975de0d881f52dc14359a7 /lib64/ld-2.27.so -10 0d6b1d90350778d58f1302d00e59493e11bc0011 ima-ng sha1:ce8204c948b9fe3ae67b94625ad620420c1dc838 /etc/ld.so.cache -10 d69ac2c1d60d28b2da07c7f0cbd49e31e9cca277 ima-ng sha1:8526466068709356630490ff5196c95a186092b8 /lib64/libreadline.so.7.0 -10 ef3212c12d1fbb94de9534b0bbd9f0c8ea50a77b ima-ng sha1:f80ba92b8a6e390a80a7a3deef8eae921fc8ca4e /lib64/libc-2.27.so -10 f805861177a99c61eabebe21003b3c831ccf288b ima-ng sha1:261a3cd5863de3f2421662ba5b455df09d941168 /lib64/libncurses.so.6.1 -10 52f680881893b28e6f0ce2b132d723a885333500 ima-ng sha1:b953a3fa385e64dfe9927de94c33318d3de56260 /lib64/libnss_files-2.27.so -10 4da8ce3c51a7814d4e38be55a2a990a5ceec8b27 ima-ng sha1:99a9c095c7928ecca8c3a4bc44b06246fc5f49de /etc/passwd -``` - -From left to right, the content of each record indicates: - -1. PCR: PCR register for extending measurement results (The default value is 10. This register is valid only when the TPM chip is installed in the system.) -2. Template hash value: hash value that is finally used for extension, combining the file content hash and the length and value of the file path -3. Template: template of the extended measurement value, for example, **ima-ng** -4. File content hash value: hash value of the measured file content -5. File path: path of the measured file - -#### IMA Appraisal - -The purpose of IMA appraisal is to control access to local files by comparing the reference value with the standard reference value. - -IMA uses the security extension attributes **security.ima** and **security.evm** to store the reference values of file integrity measurement. - -- **security.ima**: stores the hash value of the file content -- **security.evm**: stores the hash value signature of a file extended attribute - -When a protected file is accessed, the hook in the kernel is triggered to verify the integrity of the extended attributes and content of the file. - -1. Use the public key in the kernel keyring to verify the signature value in the extended attribute of the **security.evm** file, and compare this signature value with the hash value of the extended attribute of the current file. If they match, the extended attribute of the file is complete (including **security.ima**). -2. When the extended attribute of the file is complete, the system compares the extended attribute of the file **security.ima** with the digest value of the current file content. If they match, the system allows for the access to the file. - -Likewise, the file scope and trigger conditions for IMA appraisal can be configured by users using IMA policies. - -#### IMA Digest Lists - -Currently, the IMA Digest Lists extension supports the following three combinations of startup parameters: - -- IMA measurement mode: - - ```shell - ima_policy=exec_tcb ima_digest_list_pcr=11 - ``` - -- IMA appraisal log mode + IMA measurement mode: - - ```shell - ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest - ``` - -- IMA appraisal enforcing mode + IMA measurement mode: - - ```shell - ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest - ``` - -### Procedure - -#### Initial Deployment in the Native IMA Scenario - -When the system is started for the first time, you need to configure the following startup parameters: - -```shell -ima_appraise=fix ima_policy=appraise_tcb -``` - -In the `fix` mode, the system can be started when no reference value is available. `appraise_tcb` corresponds to an IMA policy. For details, see *IMA Startup Parameters* in the *Appendix*. - -Next, you need to access all the files that need to be verified to add IMA extended attributes to them: - -```shell -time find / -fstype ext4 -type f -uid 0 -exec dd if='{}' of=/dev/null count=0 status=none \; -``` - -This process takes some time. After the command is executed, you can see the marked reference value in the extended attributes of the protected file. - -```shell -$ getfattr -m - -d /sbin/init -# file: sbin/init -security.ima=0sAXr7Qmun5mkGDS286oZxCpdGEuKT -security.selinux="system_u:object_r:init_exec_t" -``` - -Configure the following startup parameters and restart the system: - -```shell -ima_appraise=enforce ima_policy=appraise_tcb -``` - -#### Initial Deployment in the Digest Lists Scenario - -1. Set kernel parameters to enter the log mode. - - Add the following parameters to edit the `/boot/efi/EFI/openEuler/grub.cfg` file: - - ```shell - ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest - ``` - - Run the `reboot` command to restart the system and enter the log mode. In this mode, integrity check has been enabled, but the system can be started even if the check fails. - -2. Install the dependency package. - - Run the **yum** command to install **digest-list-tools** and **ima-evm-utils**. Ensure that the versions are not earlier than the following: - - ```shell - $ yum install digest-list-tools ima-evm-utils - $ rpm -qa | grep digest-list-tools - digest-list-tools-0.3.93-1.oe1.x86_64 - $ rpm -qa | grep ima-evm-utils - ima-evm-utils-1.2.1-9.oe1.x86_64 - ``` - -3. If the **plymouth** package is installed, you need to add `-a` to the end of the **cp** command in line 147 in the `/usr/libexec/plymouth/plymouth-populate-initrd` script file: - - ```shell - ... - ddebug "Installing $_src" - cp -a --sparse=always -pfL "$PLYMOUTH_SYSROOT$_src" "${initdir}/$target" - } - ``` - -4. Run `dracut` to generate **initrd** again: - - ```shell - dracut -f -e xattr - ``` - - Edit the `/boot/efi/EFI/openEuler/grub.cfg` file by changing **ima\_appraise=log** to **ima\_appraise=enforce-evm**. - - ```shell - ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest - ``` - - Run the **reboot** command to complete the initial deployment. - -#### Building Digest Lists on OBS - -Open Build Service (OBS) is a compilation system that was first used for building software packages in openSUSE and supports distributed compilation of multiple architectures. - -Before building a digest list, ensure that your project contains the following RPM packages from openEuler: - -- digest-list-tools -- pesign-obs-integration -- selinux-policy -- rpm -- openEuler-rpm-config - -Add **Project Config** in the deliverable project: - -```shell -Preinstall: pesign-obs-integration digest-list-tools selinux-policy-targeted -Macros: -%__brp_digest_list /usr/lib/rpm/openEuler/brp-digest-list %{buildroot} -:Macros -``` - -- The following content is added to **Preinstall**: **digest-list-tools** for generating the digest list; **pesign-obs-integration** for generating the digest list signature; **selinux-policy-targeted**, ensuring that the SELinux label in the environment is correct when the digest list is generated. -- Define the macro **%\_\_brp\_digest\_list** in Macros. The RPM runs this macro to generate a digest list for the compiled binary file in the build phase. This macro can be used as a switch to control whether the digest list is generated in the project. - -After the configuration is completed, OBS automatically performs full build. In normal cases, the following two files are added to the software package: - -- **/etc/ima/digest\_lists/0-metadata\_list-compact-\[package name]-\[version number]** -- **/etc/ima/digest\_lists.tlv/0-metadata\_list-compact\_tlv-\[package name]-\[version number]** - -#### Building Digest Lists on Koji - -Koji is a compilation system of the Fedora community. The openEuler community will support Koji in the future. - -### FAQs - -1. Why does the system fail to be started, or commands fail to be executed, or services are abnormal after the system is started in enforcing mode? - - In enforcing mode, IMA controls file access. If the content or extended attributes of a file to be accessed are incomplete, the access will be denied. If key commands that affect system startup cannot be executed, the system cannot be started. - - Check whether the following problems exist: - - - **Check whether the digest list is added to initrd.** - - Check whether the **dracut** command is executed to add the digest list to the kernel during the initial deployment. If the digest list is not added to **initrd**, the digest list cannot be imported during startup. As a result, the startup fails. - - - **Check whether the official RPM package is used.** - - If a non-official openEuler RPM package is used, the RPM package may not carry the digest list, or the private key for signing the digest list does not match the public key for signature verification in the kernel. As a result, the digest list is not imported to the kernel. - - If the cause is not clear, enter the log mode and find the cause from the error log: - - ```shell - dmesg | grep appraise - ``` - -2. Why access control is not performed on system files in enforcing mode? - - When the system does not perform access control on the file as expected, check whether the IMA policy in the startup parameters is correctly configured: - - ```shell - $ cat /proc/cmdline - ...ima_policy=exec_tcb|appraise_exec_tcb|appraise_exec_immutable... - ``` - - Run the following command to check whether the IMA policy in the current kernel has taken effect: - - ```shell - cat /sys/kernel/security/ima/policy - ``` - - If the policy file is empty, it indicates that the policy fails to be set. In this case, the system does not perform access control. - -3. After the initial deployment is completed, do I need to manually run the **dracut** command to generate **initrd** after installing, upgrading, or uninstalling the software package? - - No. The **digest\_list.so** plug-in provided by the RPM package can automatically update the digest list at the RPM package granularity, allowing users to be unaware of the digest list. - -### Appendixes - -#### Description of the IMA securityfs Interface - -The native IMA provides the following **securityfs** interfaces: - -> Note: The following interface paths are in the `/sys/kernel/security/` directory. - -| Path | Permission | Description | -| ------------------------------ | ---------- | ------------------------------------------------------------ | -| ima/policy | 600 | IMA policy interface | -| ima/ascii_runtime_measurement | 440 | IMA measurement result in ASCII code format | -| ima/binary_runtime_measurement | 440 | IMA measurement result in binary format | -| ima/runtime_measurement_count | 440 | Measurement result statistics | -| ima/violations | 440 | Number of IMA measurement result conflicts | -| evm | 660 | EVM mode, that is, the mode for verifying the integrity of extended attributes of files | - -The values of `/sys/kernel/security/evm` are as follows: - -- 0: EVM uninitialized. -- 1: Uses HMAC (symmetric encryption) to verify the integrity of extended attributes. -- 2: Uses the public key signature (asymmetric encryption) to verify the integrity of extended attributes. -- 6: Disables the integrity check of extended attributes (This mode is used for openEuler). - -The additional **securityfs** interfaces provided by the IMA Digest Lists extension are as follows: - -| Path | Permission | Description | -| ------------------------ | ---------- | ---------------------------------------------------------- | -| ima/digests_count | 440 | Total number of digests (IMA+EVM) in the system hash table | -| ima/digest_list_data | 200 | New interfaces in the digest list | -| ima/digest_list_data_del | 200 | Interfaces deleted from the digest list | - -#### IMA Policy Syntax - -Each IMA policy statement must start with an **action** represented by the keyword action and be followed by a **filtering condition**: - -- **action**: indicates the action of a policy. Only one **action** can be selected for a policy. - - > Note: You can **ignore the word action** and directly write **dont\_measure** instead of **action=dont\_measure**. - -- **func**: indicates the type of the file to be measured or authenticated. It is often used together with **mask**. Only one **func** can be selected for a policy. - - - **FILE\_CHECK** can be used only with **MAY\_EXEC**, **MAY\_WRITE**, and **MAY\_READ**. - - **MODULE\_CHECK**, **MMAP\_CHECK**, and **BPRM\_CHECK** can be used only with **MAY\_EXEC**. - - A combination without the preceding matching relationships does not take effect. - -- **mask**: indicates the operation upon which files will be measured or appraised. Only one **mask** can be selected for a policy. - -- **fsmagic**: indicates the hexadecimal magic number of the file system type, which is defined in the `/usr/include/linux/magic.h` file. - - > Note: By default, all file systems are measured unless you use the **dont\_measure/dont\_appraise** to mark a file system not to be measured. - -- **fsuid**: indicates the UUID of a system device. The value is a hexadecimal string of 16 characters. - -- **objtype**: indicates the file type. Only one file type can be selected for a policy. - - > Note: **objtype** has a finer granularity than **func**. For example, **obj\_type=nova\_log\_t** indicates the nova log file. - -- **uid**: indicates the user (represented by the user ID) who performs operations on the file. Only one **uid** can be selected for a policy. - -- **fowner**: indicates the owner (represented by the user ID) of the file. Only one **fowner** can be selected for a policy. - -The values and description of the keywords are as follows: - -| Keyword | Value | Description | -| ------------- | ------------------ | ------------------------------------------------------------ | -| action | measure | Enables IMA measurement | -| | dont_measure | Disables IMA measurement | -| | appraise | Enables IMA appraisal | -| | dont_appraise | Disables IMA appraisal | -| | audit | Enables audit | -| func | FILE_CHECK | File to be opened | -| | MODULE_CHECK | Kernel module file to be loaded | -| | MMAP_CHECK | Dynamic library file to be mapped to the memory space of the process | -| | BRPM_CHECK | File to be executed (excluding script files opened by programs such as `/bin/hash`) | -| | POLICY_CHECK | File to be loaded as a supplement to the IMA policy | -| | FIRMWARE_CHECK | Firmware to be loaded into memory | -| | DIGEST_LIST_CHECK | Digest list file to be loaded into the kernel | -| | KEXEC_KERNEL_CHECK | kexec kernel to be switched to | -| mask | MAY_EXEC | Executes a file | -| | MAY_WRITE | Writes data to a file This operation is not recommended because it is restricted by open source mechanisms such as echo and vim (the essence of modification is to create a temporary file and then rename it). The IMA measurement of **MAY\_WRITE** is not triggered each time the file is modified. | -| | MAY_READ | Reads a file | -| | MAY_APPEND | Extends file attributes | -| fsmagic | fsmagic=xxx | Hexadecimal magic number of the file system type | -| fsuuid | fsuuid=xxx | UUID of a system device. The value is a hexadecimal string of 16 characters. | -| fowner | fowner=xxx | User ID of the file owner | -| uid | uid=xxx | ID of the user who operates the file | -| obj_type | obj_type=xxx_t | File type (based on the SELinux tag) | -| pcr | pcr= | Selects the PCR used to extend the measurement values in the TPM. The default value is 10. | -| appraise_type | imasig | Signature-based IMA appraisal | -| | meta_immutable | Evaluates the extended attributes of the file based on signatures (supporting the digest list). | - -> Note: **PATH\_CHECK** is equivalent to **FILE\_CHECK**, and **FILE\_MMAP** is equivalent to **MMAP\_CHECK**. They are not mentioned in this table. - -#### IMA Native Startup Parameters - -The following table lists the kernel startup parameters of the native IMA. - -| Parameter | Value | Description | -| ---------------- | ------------ | ------------------------------------------------------------ | -| ima_appraise | off | Disables the IMA appraisal mode. The integrity check is not performed when the file is accessed and no new reference value is generated for the file. | -| | enforce | Enables the IMA appraisal enforcing mode to perform the integrity check when the file is accessed. That is, the file digest value is calculated and compared with the reference value. If the comparison fails, the file access is rejected. In this case, the IMA generates a new reference value for the new file. | -| | fix | Enables the IMA repair mode. In this mode, the reference value of a protected file can be updated. | -| | log | Enables the IMA appraisal log mode to perform the integrity check when the file is accessed. However, commands can be executed even if the check fails, and only logs are recorded. | -| ima_policy | tcb | Measures all file execution, dynamic library mapping, kernel module import, and device driver loading. The file read behavior of the root user is also measured. | -| | appraise_tcb | Evaluates all files whose owner is the root user. | -| | secure_boot | Evaluates the kernel module import, hardware driver loading, kexec kernel switchover, and IMA policies. The prerequisite is that these files have IMA signatures. | -| ima_tcb | None | Equivalent to **ima\_policy=tcb**. | -| ima_appraise_tcb | None | Equivalent to **ima\_policy=appraise\_tcb**. | -| ima_hash | sha1/md5/... | IMA digest algorithm. The default value is sha1. | -| ima_template | ima | IMA measurement extension template | -| | ima-ng | IMA measurement extension template | -| | ima-sig | IMA measurement extension template | -| integrity_audit | 0 | Basic integrity audit information (default) | -| | 1 | Additional integrity audit information | - -> Note: The **ima\_policy** parameter can specify multiple values at the same time, for example, **ima\_policy=tcb\|appraise\_tcb**. After the system is started, the IMA policy of the system is the sum of the policies for the two parameters. - -The IMA policy for the `ima_policy=tcb` startup parameter is as follows: - -```console -# PROC_SUPER_MAGIC = 0x9fa0 -dont_measure fsmagic=0x9fa0 -# SYSFS_MAGIC = 0x62656572 -dont_measure fsmagic=0x62656572 -# DEBUGFS_MAGIC = 0x64626720 -dont_measure fsmagic=0x64626720 -# TMPFS_MAGIC = 0x01021994 -dont_measure fsmagic=0x1021994 -# DEVPTS_SUPER_MAGIC=0x1cd1 -dont_measure fsmagic=0x1cd1 -# BINFMTFS_MAGIC=0x42494e4d -dont_measure fsmagic=0x42494e4d -# SECURITYFS_MAGIC=0x73636673 -dont_measure fsmagic=0x73636673 -# SELINUX_MAGIC=0xf97cff8c -dont_measure fsmagic=0xf97cff8c -# SMACK_MAGIC=0x43415d53 -dont_measure fsmagic=0x43415d53 -# CGROUP_SUPER_MAGIC=0x27e0eb -dont_measure fsmagic=0x27e0eb -# CGROUP2_SUPER_MAGIC=0x63677270 -dont_measure fsmagic=0x63677270 -# NSFS_MAGIC=0x6e736673 -dont_measure fsmagic=0x6e736673 -measure func=MMAP_CHECK mask=MAY_EXEC -measure func=BPRM_CHECK mask=MAY_EXEC -measure func=FILE_CHECK mask=MAY_READ uid=0 -measure func=MODULE_CHECK -measure func=FIRMWARE_CHECK -``` - -The IMA policy for the `ima_policy=tcb_appraise` startup parameter is as follows: - -```console -# PROC_SUPER_MAGIC = 0x9fa0 -dont_appraise fsmagic=0x9fa0 -# SYSFS_MAGIC = 0x62656572 -dont_appraise fsmagic=0x62656572 -# DEBUGFS_MAGIC = 0x64626720 -dont_appraise fsmagic=0x64626720 -# TMPFS_MAGIC = 0x01021994 -dont_appraise fsmagic=0x1021994 -# RAMFS_MAGIC -dont_appraise fsmagic=0x858458f6 -# DEVPTS_SUPER_MAGIC=0x1cd1 -dont_appraise fsmagic=0x1cd1 -# BINFMTFS_MAGIC=0x42494e4d -dont_appraise fsmagic=0x42494e4d -# SECURITYFS_MAGIC=0x73636673 -dont_appraise fsmagic=0x73636673 -# SELINUX_MAGIC=0xf97cff8c -dont_appraise fsmagic=0xf97cff8c -# SMACK_MAGIC=0x43415d53 -dont_appraise fsmagic=0x43415d53 -# NSFS_MAGIC=0x6e736673 -dont_appraise fsmagic=0x6e736673 -# CGROUP_SUPER_MAGIC=0x27e0eb -dont_appraise fsmagic=0x27e0eb -# CGROUP2_SUPER_MAGIC=0x63677270 -dont_appraise fsmagic=0x63677270 -appraise fowner=0 -``` - -The IMA policy for the `ima_policy=secure_boot` startup parameter is as follows: - -```console -appraise func=MODULE_CHECK appraise_type=imasig -appraise func=FIRMWARE_CHECK appraise_type=imasig -appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig -appraise func=POLICY_CHECK appraise_type=imasig -``` - -#### IMA Digest List Startup Parameters - -The kernel startup parameters added to the IMA digest list feature are as follows: - -| Parameter | Value | Description | -| ------------------------ | ----------------------- | ------------------------------------------------------------ | -| integrity | 0 | Disables the IMA feature (by default) | -| | 1 | Enables the IMA feature | -| ima_appraise | off | Disables the IMA appraisal mode | -| | enforce-evm | Enables the IMA appraisal forced mode to perform the integrity check when the file is accessed and control the access. | -| ima_appraise_digest_list | digest | When the EVM is disabled, the abstract list is used for IMA appraise. The abstract list protects both the content and extended attributes of the file. | -| | digest-nometadata | If the EVM digest value does not exist, the integrity check is performed only based on the IMA digest value (the file extended attribute is not protected). | -| evm | fix | Allows for any modification to the extended attribute (even if the modification causes the failure to verify the integrity of the extended attribute). | -| | ignore | Allowed to modify the extended attribute only when it does not exist or is incorrect. | -| ima_policy | exec_tcb | IMA measurement policy. For details, see the following policy description. | -| | appraise_exec_tcb | IMA appraisal policy. For details, see the following policy description. | -| | appraise_exec_immutable | IMA appraisal policy. For details, see the following policy description. | -| ima_digest_list_pcr | 11 | Uses PCR 11 instead of PCR 10, and uses only the digest list for measurement. | -| | +11 | The PCR 10 measurement is reserved. When the TPM chip is available, the measurement result is written to the TPM chip. | -| initramtmpfs | None | Adds the support for **tmpfs**. | - -The IMA policy for the `ima_policy=exec_tcb` startup parameter is as follows: - -```console -dont_measure fsmagic=0x9fa0 -dont_measure fsmagic=0x62656572 -dont_measure fsmagic=0x64626720 -dont_measure fsmagic=0x1cd1 -dont_measure fsmagic=0x42494e4d -dont_measure fsmagic=0x73636673 -dont_measure fsmagic=0xf97cff8c -dont_measure fsmagic=0x43415d53 -dont_measure fsmagic=0x27e0eb -dont_measure fsmagic=0x63677270 -dont_measure fsmagic=0x6e736673 -measure func=MMAP_CHECK mask=MAY_EXEC -measure func=BPRM_CHECK mask=MAY_EXEC -measure func=MODULE_CHECK -measure func=FIRMWARE_CHECK -measure func=POLICY_CHECK -measure func=DIGEST_LIST_CHECK -measure parser -``` - -The IMA policy for the `ima_policy=appraise_exec_tcb` startup parameter is as follows: - -```console -appraise func=MODULE_CHECK appraise_type=imasig -appraise func=FIRMWARE_CHECK appraise_type=imasig -appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig -appraise func=POLICY_CHECK appraise_type=imasig -appraise func=DIGEST_LIST_CHECK appraise_type=imasig -dont_appraise fsmagic=0x9fa0 -dont_appraise fsmagic=0x62656572 -dont_appraise fsmagic=0x64626720 -dont_appraise fsmagic=0x858458f6 -dont_appraise fsmagic=0x1cd1 -dont_appraise fsmagic=0x42494e4d -dont_appraise fsmagic=0x73636673 -dont_appraise fsmagic=0xf97cff8c -dont_appraise fsmagic=0x43415d53 -dont_appraise fsmagic=0x6e736673 -dont_appraise fsmagic=0x27e0eb -dont_appraise fsmagic=0x63677270 -``` - -The IMA policy for the `ima_policy=appraise_exec_immutable` startup parameter is as follows: - -```console -appraise func=BPRM_CHECK appraise_type=imasig appraise_type=meta_immutable -appraise func=MMAP_CHECK -appraise parser appraise_type=imasig -``` - -#### IMA Kernel Compilation Options - -The native IMA provides the following compilation options: - -| Compilation Option | Description | -| -------------------------------- | ------------------------------------------------------- | -| CONFIG_INTEGRITY | IMA/EVM compilation switch | -| CONFIG_INTEGRITY_SIGNATURE | Enables IMA signature verification | -| CONFIG_INTEGRITY_ASYMMETRIC_KEYS | Enables IMA asymmetric signature verification | -| CONFIG_INTEGRITY_TRUSTED_KEYRING | Enables IMA/EVM key ring | -| CONFIG_INTEGRITY_AUDIT | Compiles the IMA audit module | -| CONFIG_IMA | IMA compilation switch | -| CONFIG_IMA_WRITE_POLICY | Allows updating the IMA policy in the running phase | -| CONFIG_IMA_MEASURE_PCR_IDX | Allows specifying the PCR number of the IMA measurement | -| CONFIG_IMA_LSM_RULES | Allows configuring LSM rules | -| CONFIG_IMA_APPRAISE | IMA appraisal compilation switch | -| IMA_APPRAISE_BOOTPARAM | Enables IMA appraisal startup parameters | -| CONFIG_EVM | EVM compilation switch | - -The additional compilation options provided by the IMA Digest Lists extension are as follows: - -| Compilation Option | Description | -| ------------------ | ----------------------------------- | -| CONFIG_DIGEST_LIST | Enables the IMA Digest List feature | - -#### IMA Performance Reference Data - -The following figure compares the performance when IMA is disabled, native IMA is enabled, and IMA digest list is enabled. - -![img](./figures/ima_performance.png) - -#### Impact of IMA on the kdump Service - -When the IMA enforce mode is enabled and kexec system call verification is configured in the policy, kdump may fail to be started. - -```console -appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig -``` - -Cause of the kdump startup failure: After IMA is enabled, file integrity needs to be verified. Therefore, the **kexec_file_load** system call is restricted when kdump loads kernel image files. You can modify **KDUMP_FILE_LOAD** in the **/etc/sysconfig/kdump** configuration file to enable the **kexec_file_load** system call. - -```console -KDUMP_FILE_LOAD="on" -``` - -At the same time, the **kexec_file_load** system call itself also verifies the signature of the file. Therefore, the loaded kernel image file must contain the correct secure boot signature, and the current kernel must contain the corresponding verification certificate. - -#### IMA Root Certificate Configuration - -Currently, openEuler uses the RPM key to sign the IMA digest list. To ensure that the IMA function is available out of the box, openEuler imports the RPM root certificate (PGP certificate) to the kernel by default during kernel compilation. Currently, there are two PGP certificates, namely, the OBS certificate used in the earlier version and the openEuler certificate used in the switchover of openEuler 22.03 LTS SP1. - -```shell -$ cat /proc/keys | grep PGP -1909b4ad I------ 1 perm 1f030000 0 0 asymmetri private OBS b25e7f66: PGP.rsa b25e7f66 [] -2f10cd36 I------ 1 perm 1f030000 0 0 asymmetri openeuler fb37bc6f: PGP.rsa fb37bc6f [] -``` - -The current kernel does not support the import of the PGP sub-public key, and the switched openEuler certificate uses the sub-key signature. Therefore, the openEuler kernel preprocesses the certificate before compilation, extracts the sub-public key, and imports it to the kernel. For details, see the [process_pgp_certs.sh](https://gitee.com/src-openeuler/kernel/blob/openEuler-22.03-LTS-SP1/process_pgp_certs.sh) script file in the code repository of the kernel software package. - -If the user does not use the IMA digest list function or uses other keys to implement signature/verification, you can remove the related code and configure the kernel root certificate by yourself. - -## Dynamic Integrity Measurement (DIM) - -This section describes the DIM feature and its usage. - -### Context - -With the development of the IT industry, information systems are facing an increasing number of security risks. Information systems run a large amount of software, some of which is inevitably vulnerable. Once exploited by attackers, these vulnerabilities could severely damage system services, resulting in data leakage and service unavailability. - -Most software attacks are accompanied by integrity damage, such as malicious process execution, configuration file tampering, and backdoor implantation. Therefore, protection technologies are proposed in the industry. Key data is measured and verified during system startup to ensure that the system can run properly. However, popular integrity protection technologies (such as secure boot and file integrity measurement) cannot protect memory data during process running. If an attacker somehow modifies the instructions of a process, the process may be hijacked or implanted with a backdoor, which is highly destructive and covert. To defend against such attacks, the DIM technology is proposed to measure and protect key data in the memory of a running process. - -### Terminology - -Static baseline: baseline measurement data generated by parsing the binary file of the measurement target - -Dynamic baseline: result of the first measurement on the measurement target - -Measurement policy: configuration information for measuring the target - -Measurement log: list of measurement information, including the measurement targets and measurement results - -### Description - -The DIM feature measures key data (such as code sections and data sections) in the memory during program running and compares the measurement result with the baseline value to determine whether the memory data has been tampered with. In this way, DIM can detect attacks and take countermeasures. - -#### Function Scope - -- Currently, DIM supports only AArch64 and x86 architectures. -- Currently, DIM supports measurement of the following key memory data: - - Code section of a user-mode process (the section whose attribute is **PT_LOAD** and permission is **RX** in the ELF file, and the virtual memory area whose permission is **RX** after the corresponding process is loaded) - - Kernel module code section, whose start address and length are **core_layout.base** and **core_layout.text_size** respectively in the **struct module** structure corresponding to the kernel module. - - Kernel code section, corresponding to **\_stext** to **\_etext** (Addresses that may change due to the kernel static key mechanism are skipped.) -- DIM can work with the following hardware: - - The measurement result can be extended to the Platform Configuration Register (PCR) of Trusted Platform Module (TPM) 2.0 to connect to the remote attestation service. - -#### Technical Limitations - -- For user-mode processes, only mapped code sections of files can be measured. Anonymous code sections cannot be measured. -- Kernel hot patches cannot be measured. -- Measurement can only be triggered proactively. If a file is attacked and recovered during two measurement processes, the attack cannot be identified. -- Proactive modification of code sections, such as relocation of code sections, self-modification, and hot patches, will be identified as attacks. -- For kernel and kernel module measurement, the measurement result when dynamic baseline establishment is triggered is used as the measurement baseline value. The static baseline value only serves as a fixed identifier. -- The measurement target must have been loaded to the memory when dynamic baseline establishment is triggered (for example, process running or kernel module loading). Otherwise, the object cannot be measured. -- If the TPM PCR needs to be used to verify measurement logs, the DIM module cannot be removed. Otherwise, the measurement logs will be cleared and cannot match the PCR. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->After DIM is enabled, system performance is affected in the following aspects: -> -> - Loading the DIM feature and managing baseline data and measurement logs consume system memory. The impact depends on the protection policy configuration. -> - Hashing is performed during DIM running, which consumes CPU resources. The impact depends on the size of the data to be measured. -> - Resources will be locked and semaphores need to be obtained during DIM running, which may cause other concurrent processes to wait. - -#### Specification Constraints - -| Item | Value | -| ------------------------------------------------------------ | ---- | -| Maximum size of a policy file, static baseline file, signature file, or certificate file | 10MB | -| Maximum number of tampering measurement logs recorded for a measurement target during multiple measurement periods after a dynamic baseline is established.| 10| -| Maximum number of measurement policies that can be stored in **/etc/dim/policy**|10,000| - -#### Architecture Description - -DIM includes the dim_tools and dim software packages, which contain the following components: - -| Software Package | Component | Description | -| --------- | ---------------- | ------------------------------------------------------------ | -| dim_tools | dim_gen_baseline| User-mode component for generating the static baseline, which is required for generating the dynamic measurement. The baseline data is imported when DIM runs.| -| dim | dim_core | Kernel module for executing the core dynamic measurement logic, including policy parsing, static baseline parsing, dynamic baseline establishment, measurement execution, measurement logging, and TPM extension operations, to measure key memory data.| -| dim | dim_monitor | Kernel module for measuring and protecting dim_core code sections and key data to prevent DIM function failures caused by attacks on dim_core. | - -The following figure shows the overall architecture: - -![](./figures/dim_architecture.jpg) - -#### Key Procedures - -Both the dim_core and dim_monitor modules provide the memory data measurement function, including the following core processes: - -- Dynamic baseline process: The dim_core module reads and parses the policy and static baseline file, measures the code section of the target process, stores the measurement result as a dynamic baseline in the memory, compares the dynamic baseline data with the static baseline data, and records the comparison result in measurement logs. The dim_monitor module measures the code sections and key data of the dim_core module, uses the data as the dynamic baseline, and records measurement logs. -- Dynamic measurement process: The dim_core and dim_monitor modules measure the target and compare the measurement result with the dynamic baseline. If the measurement result is inconsistent with the dynamic baseline, the dim_core and dim_monitor modules record the result in measurement logs. - -#### Interfaces - -##### Interface Files - -| Path | Description | -| ------------------------------- | ------------------------------------------------------------ | -| /etc/dim/policy | Measurement policy file | -| /etc/dim/policy.sig | Measurement policy signature file, which is used to store the signature information of the policy file and is used when the signature verification function is enabled| -| /etc/dim/digest_list/*.hash | Static baseline file, which is used to store measurement baseline values | -| /etc/dim/digest_list/*.hash.sig | Static baseline signature file, which is used to store the signature information of the static baseline file and is used when the signature verification function is enabled| -| /etc/keys/x509_dim.der | Certificate file, which is used to verify the signature information of the policy file and static baseline file and is used when the signature verification function is enabled| -| /sys/kernel/security/dim | DIM file system directory, which is generated after the DIM kernel module is loaded and provides kernel interfaces for operating the DIM function| - -##### File Formats - -1. Measurement policy file format - - The lines, each representing a measurement policy, are in plaintext and are separated by Unix newline (LF) characters. The following configuration formats are supported: - - 1. User-mode process code section measurement - - ```text - measure obj=BPRM_TEXT path= - ``` - - 2. Kernel module code section measurement - - ```text - measure obj=MODULE_TEXT name= - ``` - - 3. Kernel measurement - - ```text - measure obj=KERNEL_TEXT - ``` - - **Example:** - - ```shell - # cat /etc/dim/policy - measure obj=BPRM_TEXT path=/usr/bin/bash - measure obj=BPRM_TEXT path=/usr/lib64/libc.so.6 - measure obj=MODULE_TEXT name=ext4 - measure obj=KERNEL_TEXT - ``` - -2. Static baseline file format - - The lines, each representing a static baseline, are in plaintext and are separated by Unix newline (LF) characters. The following configuration formats are supported: - - 1. User-mode process baseline - - ```text - dim USER sha256:6ae347be2d1ba03bf71d33c888a5c1b95262597fbc8d00ae484040408a605d2b - ``` - - 2. Kernel module baseline - - ```text - dim KERNEL sha256:a18bb578ff0b6043ec5c2b9b4f1c5fa6a70d05f8310a663ba40bb6e898007ac5 / - ``` - - 3. Kernel baseline - - ```text - dim KERNEL sha256:2ce2bc5d65e112ba691c6ab46d622fac1b7dbe45b77106631120dcc5441a3b9a - ``` - - **Example:** - - ```text - dim USER sha256:6ae347be2d1ba03bf71d33c888a5c1b95262597fbc8d00ae484040408a605d2b /usr/bin/bash - dim USER sha256:bc937f83dee4018f56cc823f5dafd0dfedc7b9872aa4568dc6fbe404594dc4d0 /usr/lib64/libc.so.6 - dim KERNEL sha256:a18bb578ff0b6043ec5c2b9b4f1c5fa6a70d05f8310a663ba40bb6e898007ac5 6.4.0-1.0.1.4.oe2309.x86_64/dim_monitor - dim KERNEL sha256:2ce2bc5d65e112ba691c6ab46d622fac1b7dbe45b77106631120dcc5441a3b9a 6.4.0-1.0.1.4.oe2309.x86_64 - ``` - -3. Measurement log format - - The lines, each representing a measurement log, are in plaintext and are separated by Unix newline (LF) characters. The format is as follows: - - ```text - : - ``` - - **Example:** - - 1. The code section of the bash process is measured. The measurement result is consistent with the static baseline. - - ```text - 12 0f384a6d24e121daf06532f808df624d5ffc061e20166976e89a7bb24158eb87 sha256:db032449f9e20ba37e0ec4a506d664f24f496bce95f2ed972419397951a3792e /usr/bin.bash [static baseline] - ``` - - 2. The code section of the bash process is measured. The measurement result is inconsistent with the static baseline. - - ```text - 12 0f384a6d24e121daf06532f808df624d5ffc061e20166976e89a7bb24158eb87 sha256:db032449f9e20ba37e0ec4a506d664f24f496bce95f2ed972419397951a3792e /usr/bin.bash [tampered] - ``` - - 3. The code section of the ext4 kernel module is measured. No static baseline is found. - - ```text - 12 0f384a6d24e121daf06532f808df624d5ffc061e20166976e89a7bb24158eb87 sha256:db032449f9e20ba37e0ec4a506d664f24f496bce95f2ed972419397951a3792e ext4 [no static baseline] - ``` - - 4. dim_monitor measures dim_core and records the measurement result of the baseline. - - ```text - 12 660d594ba050c3ec9a7cdc8cf226c5213c1e6eec50ba3ff51ff76e4273b3335a sha256:bdab94a05cc9f3ad36d29ebbd14aba8f6fd87c22ae580670d18154b684de366c dim_core.text [dynamic baseline] - 12 28a3cefc364c46caffca71e7c88d42cf3735516dec32796e4883edcf1241a7ea sha256:0dfd9656d6ecdadc8ec054a66e9ff0c746d946d67d932cd1cdb69780ccad6fb2 dim_core.data [dynamic baseline] - ``` - -4. Certificate and signature file formats - -The files are in the common format. For details, see [Enabling Signature Verification](#enabling-signature-verification). - -##### Kernel Module Parameters - -1. dim_core parameters - - | Parameter | Description | Value Range | Default Value| - | -------------------- | ------------------------------------------------------------ | ------------------------ | ------ | - | measure_log_capacity | Maximum number of measurement logs recorded by dim_core. When this value is reached, dim_core stops recording measurement logs.| 100-UINT_MAX (64-bit OS)| 100000 | - | measure_schedule | Scheduling time after a process or module is measured, in milliseconds. The value 0 indicates that measurement is not scheduled.| 0-1000 | 0 | - | measure_interval | Automatic measurement interval, in minutes. The value 0 indicates that automatic measurement is not enabled. | 0-525600 | 0 | - | measure_hash | Measurement hash algorithm | sha256, sm3 | sha256 | - | measure_pcr | The TPM PCR to which the measurement result is extended. The value 0 indicates that the measurement result is not extended. The value must be an actual TPM PCR number.| 0-128 | 0 | - | signature | Whether to enable the policy file and signature verification. The value 0 indicates that they are disabled, and the value 1 indicates that they are enabled. | 0, 1 | 0 | - - **Example:** - - ```shell - insmod /path/to/dim_core.ko measure_log_capacity=10000 measure_schedule=10 measure_pcr=12 - modprobe dim_core measure_log_capacity=10000 measure_schedule=10 measure_pcr=12 - ``` - -2. dim_monitor parameters - - | Parameter | Description | Value Range | Default Value| - | -------------------- | ------------------------------------------------------------ | ------------------------ | ------ | - | measure_log_capacity | Maximum number of measurement logs recorded by dim_monitor. When this value is reached, dim_monitor stops recording measurement logs.| 100-UINT_MAX (64-bit OS)| 100000 | - | measure_hash | Measurement hash algorithm | sha256, sm3 | sha256 | - | measure_pcr | The TPM PCR to which the measurement result is extended. The value 0 indicates that the measurement result is not extended. | 0-128 | 0 | - - **Example:** - - ```shell - insmod /path/to/dim_monitor.ko measure_log_capacity=10000 measure_hash=sm3 - modprobe dim_monitor measure_log_capacity=10000 measure_hash=sm3 - ``` - -##### Kernel Interfaces - -1. dim_core interface - - | Interface | Attribute| Function | Example | - | -------------------------- | ---- | ------------------------------------------------------------ | --------------------------------------------------------- | - | measure | Write-only| Write **1** to trigger dynamic measurement. If the operation is successful, **0** is returned. If the operation fails, an error code is returned. | echo 1 > /sys/kernel/security/dim/measure | - | baseline_init | Write-only| Write **1** to trigger dynamic baseline establishment. If the operation is successful, **0** is returned. If the operation fails, an error code is returned. | echo 1 > /sys/kernel/security/dim/baseline_init | - | ascii_runtime_measurements | Read-only| Read the interface to query measurement logs. | cat /sys/kernel/security/dim/ascii_runtime_measurements | - | runtime_status | Read-only| Read the interface to query the status information. If the operation fails, an error code is returned. | cat /sys/kernel/security/dim/runtime_status | - | interval | Read/Write| Write a numeric string to set the automatic measurement interval (the value range is the same as that of **measure_interval**). Read the interface to query the current automatic measurement interval. If the query fails, an error code is returned.| echo 10 > /sys/kernel/security/dim/interval
cat /sys/kernel/security/dim/interval | - - **dim_core Statuses** - - The possible statuses of dim_core are as follows: - - - DIM_NO_BASELINE: dim_core is loaded but no operation is performed. - - DIM_BASELINE_RUNNING: The dynamic baseline is being established. - - DIM_MEASURE_RUNNING: Dynamic measurement is being performed. - - DIM_PROTECTED: The dynamic baseline has been established and is protected. - - DIM_ERROR: An error occurs during dynamic baseline establishment or dynamic measurement. You need to rectify the error and trigger dynamic baseline establishment or dynamic measurement again. - -2. dim_monitor interfaces - - | Interface | Attribute| Description | Example | - | ---------------------------------- | ---- | ---------------------------------------------- | ------------------------------------------------------------ | - | monitor_run | Write-only| Write **1** to trigger measurement. If the operation is successful, **0** is returned. If the operation fails, an error code is returned.| echo 1 > /sys/kernel/security/dim/monitor_run | - | monitor_baseline | Write-only| Write **1** to trigger baseline establishment. If the operation is successful, **0** is returned. If the operation fails, an error code is returned.| echo 1 > /sys/kernel/security/dim/monitor_baseline | - | monitor_ascii_runtime_measurements | Read-only| Read the interface to query measurement logs. | cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements | - | monitor_status | Read-only| Read the interface to query the status information. If the operation fails, an error code is returned. | cat /sys/kernel/security/dim/monitor_status | - - **dim_monitor Statuses** - - - ready: dim_monitor is loaded but no operation is performed. - - running: The dynamic baseline is being established or dynamic measurement is being performed. - - error: An error occurs during dynamic baseline establishment or dynamic measurement. You need to rectify the error and trigger dynamic baseline establishment or dynamic measurement again. - - protected: The dynamic baseline has been established and is protected. - -##### User-Mode Tool Interface - -See for the details of the `dim_gen_baseline` CLI interface. - -### Usage - -#### Installing and Uninstalling DIM - -**Prerequisites**: - -- OS: openEuler 23.09 or later -- Kernel: openEuler kernel 5.10 or 6.4 - -Install dim_tools and dim software packages. The following uses openEuler 23.09 as an example: - -```shell -# yum install -y dim_tools dim -``` - -After the software packages are installed, the DIM kernel components are not loaded by default. You can run the following commands to load or unload them: - -```shell -# modprobe dim_core or insmod /path/to/dim_core.ko -# modprobe dim_monitor or insmod /path/to/dim_monitor.ko -# rmmod dim_monitor -# rmmod dim_core -``` - -After the components are loaded successfully, you can run the following commands to query the components: - -```shell -# lsmod | grep dim_core -dim_core 77824 1 dim_monitor -# lsmod | grep dim_monitor -dim_monitor 36864 0 -``` - -Unload the KO files before uninstalling the RPM package. - -```shell -# rmmod dim_monitor -# rmmod dim_core -# rpm -e dim -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> dim_monitor must be loaded after dim_core and removed before dim_core. -> You can also install DIM from source. For details, see . - -#### Measuring Code Sections of User-Mode Processes - -**Prerequisites**: - -- The dim_core module is loaded successfully. - -- A user-mode resident measurement target program has been prepared. This section uses **/opt/dim/demo/dim_test_demo** as an example. - - ```shell - # /opt/dim/demo/dim_test_demo & - ``` - -**Step 1**: Generate a static baseline for the binary file corresponding to the measurement target process. - -```shell -# mkdir -p /etc/dim/digest_list -# dim_gen_baseline /opt/dim/demo/dim_test_demo -o /etc/dim/digest_list/test.hash -``` - -**Step 2**: Configure a measurement policy. - -```shell -# echo "measure obj=BPRM_TEXT path=/opt/dim/demo/dim_test_demo" > /etc/dim/policy -``` - -**Step 3**: Trigger dynamic baseline establishment. - -```shell -# echo 1 > /sys/kernel/security/dim/baseline_init -``` - -**Step 4**: Query measurement logs. - -```shell -# cat /sys/kernel/security/dim/ascii_runtime_measurements -0 e9a79e25f091e03a8b3972b1a0e4ae2ccaed1f5652857fe3b4dc947801a6913e sha256:02e28dff9997e1d81fb806ee5b784fd853eac8812059c4dba7c119c5e5076989 /opt/dim/demo/dim_test_demo [static baseline] -``` - -If the preceding measurement log is displayed, the target process is measured successfully and the measurement result is consistent with the static baseline. - -**Step 5**: Trigger dynamic measurement. - -```shell -# echo 1 > /sys/kernel/security/dim/measure -``` - -After the measurement is complete, you can perform **Step 4** to query the measurement logs. If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. If an attacker attempts to tamper with the target program (for example, by modifying and recompiling the code) and restart the target program, for example: - -```shell -# pkill dim_test_demo -# /opt/dim/demo/dim_test_demo & -``` - -Trigger the measurement again and query the measurement logs. You can see a measurement log marked with "tampered." - -```shell -# echo 1 > /sys/kernel/security/dim/measure -# cat /sys/kernel/security/dim/ascii_runtime_measurements -0 e9a79e25f091e03a8b3972b1a0e4ae2ccaed1f5652857fe3b4dc947801a6913e sha256:02e28dff9997e1d81fb806ee5b784fd853eac8812059c4dba7c119c5e5076989 /opt/dim/demo/dim_test_demo [static baseline] -0 08a2f6f2922ad3d1cf376ae05cf0cc507c2f5a1c605adf445506bc84826531d6 sha256:855ec9a890ff22034f7e13b78c2089e28e8d217491665b39203b50ab47b111c8 /opt/dim/demo/dim_test_demo [tampered] -``` - -#### Measuring Code Sections of Kernel Modules - -**Prerequisites**: - -- The dim_core module is loaded successfully. - -- A measurement target kernel module has been prepared. This section uses dim_test_module as an example, whose path is **/opt/dim/demo/dim_test_module.ko**. - - ```shell - # insmod /opt/dim/demo/dim_test_module.ko - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->The kernel version of the environment where the module is compiled must be the same as the current kernel version. Run the following command to confirm: -> ->```shell -># modinfo dim_monitor.ko | grep vermagic | grep "$(uname -r)" ->vermagic: 6.4.0-1.0.1.4.oe2309.x86_64 SMP preempt mod_unload modversions ->``` - -The first field of the vermagic kernel module information must be the same as the current kernel version. - -**Step 1**: Generate a static baseline for the measurement target kernel module. - -```shell -# mkdir -p /etc/dim/digest_list -# dim_gen_baseline /opt/dim/demo/dim_test_module.ko -o /etc/dim/digest_list/test.hash -``` - -**Step 2**: Configure a measurement policy. - -```shell -# echo "measure obj=MODULE_TEXT name=dim_test_module" > /etc/dim/policy -``` - -**Step 3**: Trigger dynamic baseline establishment. - -```shell -# echo 1 > /sys/kernel/security/dim/baseline_init -``` - -**Step 4**: Query measurement logs. - -```shell -# cat /sys/kernel/security/dim/ascii_runtime_measurements -0 9603a9d5f87851c8eb7d2619f7abbe28cb8a91f9c83f5ea59f036794e23d1558 sha256:9da4bccc7ae1b709deab8f583b244822d52f3552c93f70534932ae21fac931c6 dim_test_module [static baseline] -``` - -The preceding measurement log indicates that dim_test_module is successfully measured and the current measurement result is used as the baseline value for subsequent measurement. The hash value in the measurement log is not the actual measurement value. - -**Step 5**: Trigger dynamic measurement. - -```shell -echo 1 > /sys/kernel/security/dim/measure -``` - -After the measurement is complete, you can perform **Step 4** to query the measurement logs. If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. If an attacker attempts to tamper with the target kernel module (for example, by modifying and recompiling the code) and reload the module, for example: - -```shell -rmmod dim_test_module -insmod /opt/dim/demo/dim_test_module.ko -``` - -Trigger the measurement again and query the measurement logs. You can see a measurement log marked with "tampered." - -```shell -# cat /sys/kernel/security/dim/ascii_runtime_measurements -0 9603a9d5f87851c8eb7d2619f7abbe28cb8a91f9c83f5ea59f036794e23d1558 sha256:9da4bccc7ae1b709deab8f583b244822d52f3552c93f70534932ae21fac931c6 dim_test_module [static baseline] -0 6205915fe63a7042788c919d4f0ff04cc5170647d7053a1fe67f6c0943cd1f40 sha256:4cb77370787323140cb572a789703be1a4168359716a01bf745aa05de68a14e3 dim_test_module [tampered] -``` - -#### Measuring Code Sections of the Kernel - -**Prerequisites**: - -- The dim_core module is loaded successfully. - -**Step 1**: generate a static baseline for the kernel. - -```shell -# mkdir -p /etc/dim/digest_list -# dim_gen_baseline -k "$(uname -r)" -o /etc/dim/digest_list/test.hash /boot/vmlinuz-6.4.0-1.0.1.4.oe2309.x86_64 -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->The file name **/boot/vmlinuz-6.4.0-1.0.1.4.oe2309.x86_64** is used as an example. - -**Step 2**: Configure a DIM policy. - -```shell -# echo "measure obj=KERNEL_TEXT" > /etc/dim/policy -``` - -**Step 3**: Trigger dynamic baseline establishment. - -```shell -# echo 1 > /sys/kernel/security/dim/baseline_init -``` - -**Step 4**: Query measurement logs. - -```shell -# cat /sys/kernel/security/dim/ascii_runtime_measurements -0 ef82c39d767dece1f5c52b31d1e8c7d55541bae68a97542dda61b0c0c01af4d2 sha256:5f1586e95b102cd9b9f7df3585fe13a1306cbd464f2ebe47a51ad34128f5d0af 6.4.0-1.0.1.4.oe2309.x86_64 [static baseline] -``` - -The preceding measurement log indicates that the kernel is successfully measured and the current measurement result is used as the baseline value for subsequent measurement. The hash value in the measurement log is not the actual measurement value. - -**Step 5**: Trigger dynamic measurement. - -```shell -# echo 1 > /sys/kernel/security/dim/measure -``` - -After the measurement is complete, you can perform **Step 4** to query the measurement logs. If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. - -#### Measuring the dim_core Module - -**Prerequisites**: - -- The dim_core and dim_monitor modules are loaded successfully. -- The measurement policy has been configured. - -**Step 1**: Trigger dynamic baseline establishment for dim_core. - -```shell -# echo 1 > /sys/kernel/security/dim/baseline_init -``` - -**Step2**: Trigger dynamic baseline establishment for dim_monitor. - -```shell -# echo 1 > /sys/kernel/security/dim/monitor_baseline -``` - -**Step 3**: Query dim_monitor measurement logs. - -```shell -# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements -0 c1b0d9909ddb00633fc6bbe7e457b46b57e165166b8422e81014bdd3e6862899 sha256:35494ed41109ebc9bf9bf7b1c190b7e890e2f7ce62ca1920397cd2c02a057796 dim_core.text [dynamic baseline] -0 9be7121cd2c215d454db2a8aead36b03d2ed94fad0fbaacfbca83d57a410674f sha256:f35d20aae19ada5e633d2fde6e93133c3b6ae9f494ef354ebe5b162398e4d7fa dim_core.data [dynamic baseline] -``` - -The preceding measurement log indicates that dim_core is successfully measured and the current measurement result is used as the baseline value for subsequent measurement. ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->If you skip dynamic baseline establishment and directly perform measurement, "tampered" is displayed in the log. - -**Step 4**: Trigger dynamic measurement of dim_monitor. - -```shell -# echo 1 > /sys/kernel/security/dim/monitor_run -``` - -If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. If dynamic baseline establishment for dim_core is triggered again after the policy is modified, the measurement target changes, and the baseline data managed by dim_core also changes. As a result, the dim_monitor measurement result changes. - -```shell -# echo "measure obj=BPRM_TEXT path=/usr/bin/bash" > /etc/dim/policy -# echo 1 > /sys/kernel/security/dim/baseline_init -``` - -Trigger the measurement of dim_monitor again and query the measurement logs. You can see a measurement log marked with "tampered." - -```shell -# echo 1 > /sys/kernel/security/dim/monitor_run -# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements -0 c1b0d9909ddb00633fc6bbe7e457b46b57e165166b8422e81014bdd3e6862899 sha256:35494ed41109ebc9bf9bf7b1c190b7e890e2f7ce62ca1920397cd2c02a057796 dim_core.text [dynamic baseline] -0 9be7121cd2c215d454db2a8aead36b03d2ed94fad0fbaacfbca83d57a410674f sha256:f35d20aae19ada5e633d2fde6e93133c3b6ae9f494ef354ebe5b162398e4d7fa dim_core.data [dynamic baseline] -0 6a60d78230954aba2e6ea6a6b20a7b803d7adb405acbb49b297c003366cfec0d sha256:449ba11b0bfc6146d4479edea2b691aa37c0c025a733e167fd97e77bbb4b9dab dim_core.data [tampered] -``` - -#### Extending to the TPM PCR - -**Prerequisites**: - -- The TPM 2.0 has been installed in the system. After the following command is executed, the command output is not empty: - - ```shell - # ls /dev/tpm* - /dev/tpm0 /dev/tpmrm0 - ``` - -- The tpm2-tools software package has been installed in the system. After the following command is executed, the command output is not empty: - - ```shell - # rpm -qa tpm2-tools - ``` - -- The measurement policy and static baseline have been configured. - -**Step 1**: Load the dim_core and dim_monitor modules and set the numbers of the PCRs to which the measurement results are extended. In this example, PCR 12 is specified for the dim_core measurement result, and PCR 13 is specified for the dim_monitor measurement result. - -```shell -# modprobe dim_core measure_pcr=12 -# modprobe dim_monitor measure_pcr=13 -``` - -**Step 2**: Trigger baseline establishment for dim_core and dim_monitor. - -```shell -# echo 1 > /sys/kernel/security/dim/baseline_init -# echo 1 > /sys/kernel/security/dim/monitor_baseline -``` - -**Step 3**: Check the measurement logs. Each log is marked with the corresponding TPM PCR number. - -```shell -# cat /sys/kernel/security/dim/ascii_runtime_measurements -12 2649c414d1f9fcac1c8d0df8ae7b1c18b5ea10a162b957839bdb8f8415ec6146 sha256:83110ce600e744982d3676202576d8b94cea016a088f99617767ddbd66da1164 /usr/lib/systemd/systemd [static baseline] -# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements -13 72ee3061d5a80eb8547cd80c73a80c3a8dc3b3e9f7e5baa10f709350b3058063 sha256:5562ed25fcdf557efe8077e231399bcfbcf0160d726201ac8edf7a2ca7c55ad0 dim_core.text [dynamic baseline] -13 8ba44d557a9855c03bc243a8ba2d553347a52c1a322ea9cf8d3d1e0c8f0e2656 sha256:5279eadc235d80bf66ba652b5d0a2c7afd253ebaf1d03e6e24b87b7f7e94fa02 dim_core.data [dynamic baseline] -``` - -**Step 4**: Check the TPM PCRs. Extended values have been written to the corresponding PCRs. - -```shell -# tpm2_pcrread sha256 | grep "12:" - 12: 0xF358AC6F815BB29D53356DA2B4578B4EE26EB9274E553689094208E444D5D9A2 -# tpm2_pcrread sha256 | grep "13:" - 13: 0xBFB9FF69493DEF9C50E52E38B332BDA8DE9C53E90FB96D14CD299E756205F8EA -``` - -#### Enabling Signature Verification - -**Prerequisites**: - -- A public key certificate and a signature private key have been prepared. The signature algorithm is RSA, the hash algorithm is sha256, and the certificate format is DER. You can also run the following commands to generate the files: - - ```shell - # openssl genrsa -out dim.key 4096 - # openssl req -new -sha256 -key dim.key -out dim.csr -subj "/C=AA/ST=BB/O=CC/OU=DD/CN=DIM Test" - # openssl x509 -req -days 3650 -signkey dim.key -in dim.csr -out dim.crt - # openssl x509 -in dim.crt -out dim.der -outform DER - ``` - -- The measurement policy has been configured. - -**Step 1**: Save the DER certificate as **/etc/keys/x509_dim.der**. - -```shell -# mkdir -p /etc/keys -# cp dim.der /etc/keys/x509_dim.der -``` - -**Step 2**: Sign the policy file and static baseline file. The signature file name must be the original file name suffixed with **.sig**. - -```shell -# openssl dgst -sha256 -out /etc/dim/policy.sig -sign dim.key /etc/dim/policy -# openssl dgst -sha256 -out /etc/dim/digest_list/test.hash.sig -sign dim.key /etc/dim/digest_list/test.hash -``` - -**Step 3**: Load the dim_core module and enable the signature verification function. - -```shell -modprobe dim_core signature=1 -``` - -Now, the policy file and static baseline file can be loaded only after they pass the signature verification. -The baseline establishment will fail if it is triggered after the policy file is modified: - -```shell -# echo "" >> /etc/dim/policy -# echo 1 > /sys/kernel/security/dim/baseline_init --bash: echo: write error: Key was rejected by service -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->If the signature verification of a static baseline file fails, dim_core skips the parsing of the file without causing a baseline establishment failure. - -#### Configuring Measurement Algorithms - -**Prerequisites**: - -- The measurement policy has been configured. - -**Step 1**: Load the dim_core and dim_monitor modules and configure the measurement algorithm. The following uses the SM3 algorithm as an example. - -```shell -# modprobe dim_core measure_hash=sm3 -# modprobe dim_monitor measure_hash=sm3 -``` - -**Step 2**: Configure a policy that establishes a static baseline of the SM3 algorithm for the measurement target program. - -```shell -# echo "measure obj=BPRM_TEXT path=/opt/dim/demo/dim_test_demo" > /etc/dim/policy -# dim_gen_baseline -a sm3 /opt/dim/demo/dim_test_demo -o /etc/dim/digest_list/test.hash -``` - -**Step 3**: Trigger baseline establishment. - -```shell -# echo 1 > /sys/kernel/security/dim/baseline_init -# echo 1 > /sys/kernel/security/dim/monitor_baseline -``` - -**Step 4**: Check the measurement logs. Each log is marked with the corresponding hash algorithm. - -```shell -# cat /sys/kernel/security/dim/ascii_runtime_measurements -0 585a64feea8dd1ec415d4e67c33633b97abb9f88e6732c8a039064351d24eed6 sm3:ca84504c02bef360ec77f3280552c006ce387ebb09b49b316d1a0b7f43039142 /opt/dim/demo/dim_test_demo [static baseline] -# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements -0 e6a40553499d4cbf0501f32cabcad8d872416ca12855a389215b2509af76e60b sm3:47a1dae98182e9d7fa489671f20c3542e0e154d3ce941440cdd4a1e4eee8f39f dim_core.text [dynamic baseline] -0 2c862bb477b342e9ac7d4dd03b6e6705c19e0835efc15da38aafba110b41b3d1 sm3:a4d31d5f4d5f08458717b520941c2aefa0b72dc8640a33ee30c26a9dab74eae9 dim_core.data [dynamic baseline] -``` - -#### Configuring Automatic Measurement - -**Prerequisites**: - -- The measurement policy has been configured. - -**Method 1**: Load the dim_core module and set the automatic measurement interval to 1 minute. - -```shell -modprobe dim_core measure_interval=1 -``` - -After the module is loaded, dynamic baseline establishment is automatically triggered. Then, dynamic measurement is triggered every minute. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->In this case, do not configure the measurement policy for dim_core to measure its own code sections. Otherwise, false alarms may occur. ->In addition, you need to configure **/etc/dim/policy** in advance. Otherwise, the module fails to be loaded when **measure_interval** is set to **1**. - -**Method 2**: After the dim_core module is loaded, you can configure the measurement interval through the kernel module interface. In the following example, the measurement interval is set to 1 minute. - -```shell -modprobe dim_core -echo 1 > /sys/kernel/security/dim/interval -``` - -In this case, the measurement is not triggered immediately. One minute later, dynamic baseline establishment, or dynamic measurement, is triggered. Subsequently, dynamic measurement is triggered every minute. - -#### Configuring the Measurement Scheduling Time - -**Prerequisites**: - -- The measurement policy has been configured. - -Load the dim_core module and set the measurement scheduling time to 10 ms. - -```shell -modprobe dim_core measure_schedule=10 -``` - -When the dynamic baseline establishment or dynamic measurement is triggered, dim_core schedules the CPU to release 10 ms each time a process is measured. - -## Remote Attestation (Kunpeng Security Library) - -### Introduction - -This project develops basic security software components running on Kunpeng processors. In the early stage, the project focuses on trusted computing fields such as remote attestation to empower security developers in the community. - -### Software Architecture - -On the platform without TEE enabled, this project can provide the platform remote attestation feature, and its software architecture is shown in the following figure: - -![img](./figures/RA-arch-1.png) - -On the platform that has enabled TEE, this project can provide TEE remote attestation feature, and its software architecture is shown in the following figure: - -![img](./figures/RA-arch-2.png) - -### Installation and Configuration - -1. Run the following command to use the RPM package of the Yum installation program: - - ```shell - yum install kunpengsecl-ras kunpengsecl-rac kunpengsecl-rahub kunpengsecl-qcaserver kunpengsecl-attester kunpengsecl-tas kunpengsecl-devel - ``` - -2. Prepare the database environment. Go to the `/usr/share/attestation/ras` directory and run the p`prepare-database-env.sh` script to automatically configure the database environment. - -3. The configuration files required for program running are stored in three paths: current path `./config.yaml`, home path `${HOME}/.config/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml`, and system path `/etc/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml`. - -4. (Optional) To create a home directory configuration file, run the `prepare-ras(rac)(hub)(qca)(attester)(tas)conf-env.sh` script in `/usr/share/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)` after installing the RPM package. - -### Options - -#### RAS Boot Options - -Run the `ras` command to start the RAS program. Note that you need to provide the ECDSA public key in the current directory and name it `ecdsakey.pub`. Options are as follows: - -```console - -H --https HTTP/HTTPS mode switch. The default value is https(true), false=http. - -h --hport RESTful API port listened by RAS in HTTPS mode. - -p, --port string Client API port listened by RAS. - -r, --rest string RESTful API port listened by RAS in HTTP mode. - -T, --token Generates a verification code for test and exits. - -v, --verbose Prints more detailed RAS runtime log information. - -V, --version Prints the RAS version and exits. -``` - -#### RAC Boot Options - -Run the `sudo raagent` command to start the RAC program. Note that the sudo permission is required to enable the physical TPM module. Options are as follows: - -```console - -s, --server string Specifies the RAS service port to be connected. - -t, --test Starts in test mode. - -v, --verbose Prints more detailed RAC runtime log information. - -V, --version Prints the RAC version and exits. - -i, --imalog Specifies the path of the IMA file. - -b, --bioslog Specifies the path of the BIOS file. - -T, --tatest Starts in TA test mode. -``` - -**Note:** ->1.To use TEE remote attestation feature, you must start RAC not in TA test mode. And place the uuid, whether to use TCB, mem_hash and img_hash of the TA to be attestated sequentially in the **talist** file under the RAC execution path. At the same time, pre install the **libqca.so** and **libteec.so** library provided by the TEE team. The format of the **talist** file is as follows: -> ->```text ->e08f7eca-e875-440e-9ab0-5f381136c600 false ccd5160c6461e19214c0d8787281a1e3c4048850352abe45ce86e12dd3df9fde 46d5019b0a7ffbb87ad71ea629ebd6f568140c95d7b452011acfa2f9daf61c7a ->``` -> ->2.To not use TEE remote attestation feature, you must copy the **libqca.so** and **libteec.so** library in `${DESTDIR}/usr/share/attestation/qcaserver` path to `/usr/lib` or `/usr/lib64` path, and start RAC in TA test mode. - -#### QCA Boot Options - -Run the `${DESTDIR}/usr/bin/qcaserver` command to start the QCA program. Note that to start QTA normally, the full path of qcaserver must be used, and the CA path parameter in QTA needs to be kept the same as the path. Options are as follows: - -```console - -C, --scenario int Sets the application scenario of the program, The default value is sce_no_as(0), 1=sce_as_no_daa, 2=sce_as_with_daa. - -S, --server string Specifies the open server address/port. -``` - -#### ATTESTER Boot Options - -Run the `attester` command to start the ATTESTER program. Options are as follows: - -```console - -B, --basevalue string Sets the base value file read path - -M, --mspolicy int Sets the measurement strategy, which defaults to -1 and needs to be specified manually. 1=compare only img-hash values, 2=compare only hash values, and 3=compare both img-hash and hash values at the same time. - -S, --server string Specifies the address of the server to connect to. - -U, --uuid int Specifies the trusted apps to verify. - -V, --version Prints the program version and exit. - -T, --test Reads fixed nonce values to match currently hard-coded trusted reports. -``` - -#### TAS Boot Options - -Run the `tas` command to start the TAS program. Options are as follows: - -```console - -T, --token Generates a verification code for test and exits. -``` - -**Note:** ->1.To enable the TAS, you must configure the private key for TAS. Run the following command to modify the configuration file in the home directory: -> ->```shell ->$ cd ${HOME}/.config/attestation/tas ->$ vim config.yaml -> # The values of the following DAA_GRP_KEY_SK_X and DAA_GRP_KEY_SK_Y are for testing purposes only. -> # Be sure to update their contents to ensure safety before normal use. ->tasconfig: -> port: 127.0.0.1:40008 -> rest: 127.0.0.1:40009 -> akskeycertfile: ./ascert.crt -> aksprivkeyfile: ./aspriv.key -> huaweiitcafile: ./Huawei IT Product CA.pem -> DAA_GRP_KEY_SK_X: 65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c -> DAA_GRP_KEY_SK_Y: 126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56 ->``` -> ->Then enter `tas` to start TAS program. -> ->2.In an environment with TAS, in order to improve the efficiency of QCA's certificate configuration process, not every boot needs to access the TAS to generate the certificate, but through the localized storage of the certificate. That is, read the certification path configured in `config.yaml` on QCA side, check if a TAS-issued certificate has been saved locally through the `func hasAKCert(s int) bool` function. If the certificate is successfully read, there is no need to access TAS. If the certificate cannot be read, you need to access TAS and save the certificate returned by TAS locally. - -### API Definition - -#### RAS APIs - -To facilitate the administrator to manage the target server, RAS and the user TA in the TEE deployed on the target server, the following APIs are designed for calling: - -| API | Method | -| --------------------------------- | --------------------------- | -| / | GET | -| /{id} | GET, POST, DELETE | -| /{from}/{to} | GET | -| /{id}/reports | GET | -| /{id}/reports/{reportid} | GET, DELETE | -| /{id}/basevalues | GET | -| /{id}/newbasevalue | POST | -| /{id}/basevalues/{basevalueid} | GET, POST, DELETE | -| /{id}/ta/{tauuid}/status | GET | -| /{id}/ta/{tauuid}/tabasevalues | GET | -| /{id}/ta/{tauuid}/tabasevalues/{tabasevalueid} | GET, POST, DELETE | -| /{id}/ta/{tauuid}/newtabasevalue | POST | -| /{id}/ta/{tauuid}/tareports | GET | -| /{id}/ta/{tauuid}/tareports/{tareportid} | GET, POST, DELETE | -| /{id}/basevalues/{basevalueid} | GET, DELETE | -| /version | GET | -| /config | GET, POST | -| /{id}/container/status | GET | -| /{id}/device/status | GET | - -The usage of the preceding APIs is described as follows: - -To query information about all servers, use `/`. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/ -``` - -*** -To query detailed information about a target server, use the GET method of `/{id}`. **{id}** is the unique ID allocated by RAS to the target server. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/1 -``` - -*** -To modify information about the target server, use the POST method of `/{id}`. `$AUTHTOKEN` is the identity verification code automatically generated by running the `ras -T` command. - -```go -type clientInfo struct { - Registered *bool `json:"registered"` // Registration status of the target server - IsAutoUpdate *bool `json:"isautoupdate"`// Target server base value update policy -} -``` - -```shell -curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1 -d '{"registered":false, "isautoupdate":false}' -``` - -*** -To delete a target server, use the DELETE method of `/{id}`. - -**Note:** ->This method does not delete all information about the target server. Instead, it sets the registration status of the target server to `false`. - -```shell -curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1 -``` - -*** -To query information about all servers in a specified range, use the GET method of `/{from}/{to}`. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/9 -``` - -*** -To query all trust reports of the target server, use the GET method of `/{id}/reports`. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports -``` - -*** -To query details about a specified trust report of the target server, use the GET method of `/{id}/reports/{reportid}`. **{reportid}** indicates the unique ID assigned by RAS to the trust report of the target server. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports/1 -``` - -*** -To delete a specified trust report of the target server, use the DELETE method of `/{id}/reports/{reportid}`. - -**Note:** ->This method will delete all information about the specified trusted report, and the report cannot be queried through the API. - -```shell -curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/reports/1 -``` - -*** -To query all base values of the target server, use the GET method of `/{id}/reports/{reportid}`. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues -``` - -*** -To add a base value to the target server, use the POST method of `/{id}/newbasevalue`. - -```go -type baseValueJson struct { - BaseType string `json:"basetype"` // Base value type - Uuid string `json:"uuid"` // ID of a container or device - Name string `json:"name"` // Base value name - Enabled bool `json:"enabled"` // Whether the base value is available - Pcr string `json:"pcr"` // PCR value - Bios string `json:"bios"` // BIOS value - Ima string `json:"ima"` // IMA value - IsNewGroup bool `json:"isnewgroup"` // Whether this is a group of new reference values -} -``` - -```shell -curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/newbasevalue -d '{"name":"test", "basetype":"host", "enabled":true, "pcr":"testpcr", "bios":"testbios", "ima":"testima", "isnewgroup":true}' -``` - -*** -To query details about a specified base value of a target server, use the get method of `/{id}/basevalues/{basevalueid}`. **{basevalueid}** indicates the unique ID allocated by RAS to the specified base value of the target server. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1 -``` - -*** -To change the availability status of a specified base value of the target server, use the POST method of `/{id}/basevalues/{basevalueid}`. - -```shell -curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/basevalues/1 -d '{"enabled":true}' -``` - -*** -To delete a specified base value of the target server, use the DELETE method of `/{id}/basevalues/{basevalueid}`. - -**Note:** ->This method will delete all the information about the specified base value, and the base value cannot be queried through the API. - -```shell -curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1 -``` - -To query the trusted status of a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/status"` interface. Where {id} is the unique identification number assigned by RAS to the target server, and {tauuid} is the identification number of the specific user TA. - -```shell -curl -X GET -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/ta/test/status -``` - -*** -To query all the baseline value information of a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/tabasevalues"` interface. - -```shell -curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues -``` - -*** -To query the details of a specified base value for a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. where {tabasevalueid} is the unique identification number assigned by RAS to the specified base value of a specific user TA on the target server. - -```shell -curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues/1 -``` - -*** -To modify the available status of a specified base value for a specific user TA on the target server, use the `POST` method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. - -```shell -curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/ta/test/tabasevalues/1 --data '{"enabled":true}' -``` - -*** -To delete the specified base value of a specific user TA on the target server, use the `DELETE` method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. - -**Note:** ->This method will delete all information about the specified base value, and the base value cannot be queried through the API. - -```shell -curl -X DELETE -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/tabasevalues/1 -``` - -*** -To add a baseline value to a specific user TA on the target server, use the `POST` method of the `"/{id}/ta/{tauuid}/newtabasevalue"` interface. - -```go -type tabaseValueJson struct { - Uuid string `json:"uuid"` // the identification number of the user TA - Name string `json:"name"` // base value name - Enabled bool `json:"enabled"` // whether a baseline value is available - Valueinfo string `json:"valueinfo"` // mirror hash value and memory hash value -} -``` - -```shell -curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/newtabasevalue -d '{"uuid":"test", "name":"testname", "enabled":true, "valueinfo":"test info"}' -``` - -*** -To query the target server for all trusted reports for a specific user TA, use the `GET` method of the `"/{id}/ta/{tauuid}/tareports"` interface. - -```shell -curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports -``` - -*** -To query the details of a specified trusted report for a specific user TA on the target server, use the `GET` method of the `"/{id}/ta/{tauuid}/tareports/{tareportid}"` interface. Where {tareportid} is the unique identification number assigned by RAS to the specified trusted report of a specific user TA on the target server. - -```shell -curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2 -``` - -*** -To delete the specified trusted report of a specific user TA on the target server, use the `DELETE` method of the `"/{id}/ta/{tauuid}/tareports/{tareportid}"` interface. - -**Note:** ->This method will delete all information of the specified trusted report, and the report cannot be queried through the API. - -```shell -curl -X DELETE -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2 -``` - -*** -To obtain the version information of the program, use the GET method of `/version`. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/version -``` - -*** -To query the configuration information about the target server, RAS, or database, use the GET method of `/config`. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40002/config -``` - -*** -To modify the configuration information about the target server, RAS, or database, use the POST method of /config. - -```go -type cfgRecord struct { - // Target server configuration - HBDuration string `json:"hbduration" form:"hbduration"` - TrustDuration string `json:"trustduration" form:"trustduration"` - DigestAlgorithm string `json:"digestalgorithm" form:"digestalgorithm"` - // RAS configuration - MgrStrategy string `json:"mgrstrategy" form:"mgrstrategy"` - ExtractRules string `json:"extractrules" form:"extractrules"` - IsAllupdate *bool `json:"isallupdate" form:"isallupdate"` - LogTestMode *bool `json:"logtestmode" form:"logtestmode"` -} -``` - -```shell -curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/config -d '{"hbduration":"5s","trustduration":"20s","DigestAlgorithm":"sha256"}' -``` - -#### TAS APIs - -To facilitate the administrator's management of TAS for remote control, the following API is designed for calling: - -| API | Method | -| --------------------| ------------------| -| /config | GET, POST | - -To query the configuration information, use the GET method of the `/config` interface. - -```shell -curl -X GET -H "Content-Type: application/json" http://localhost:40009/config -``` - -*** -To modify the configuration information, use the POST method of the `/config` interface. - -```shell -curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40009/config -d '{"basevalue":"testvalue"}' -``` - -**Note:** ->Currently, only the base value in the configuration information of TAS is supported for querying and modifying. - -### FAQs - -1. Why cannot RAS be started after it is installed? - - > In the current RAS design logic, after the program is started, it needs to search for the `ecdsakey.pub` file in the current directory and read the file as the identity verification code for accessing the program. If the file does not exist in the current directory, an error is reported during RAS boot. - >> Solution 1: Run the `ras -T` command to generate a test token. The `ecdsakey.pub` file is generated. - >> Solution 2: After deploying the oauth2 authentication service, save the verification public key of the JWT token generator as `ecdsakey.pub`. - -2. Why cannot RAS be accessed through REST APIs after it is started? - - > RAS is started in HTTPS mode by default. Therefore, you need to provide a valid certificate for RAS to access it. However, RAS started in HTTP mode does not require a certificate. - -## Trusted Platform Control Module - -### Background - -Trusted computing has undergone continuous development and improvement in the past 40 years and has become an important branch of information security. Trusted computing technologies have developed rapidly in recent years and have solved the challenges in Trusted Computing 2.0—integration of trusted systems and existing systems, trusted management, and simplification of trusted application development. These technical breakthroughs form Trusted Computing 3.0, that is, trusted computing based on an active immune system. Compared with the passive plug-in architecture of the previous generation, Trusted Computing 3.0 proposes a new trusted system framework based on self-controlled cryptography algorithms, control chips, trusted software, trusted connections, policy management, and secure and trusted protection applications, implementing trust across the networks. - -The trusted platform control module (TPCM) is a base and core module that can be integrated into a trusted computing platform to establish and ensure a trust source. As one of the innovations in Trusted Computing 3.0 and the core of active immunity, TPCM implements active control over the entire platform. - -The TPCM-based Trusted Computing 3.0 architecture consists of the protection module and the computing module. On the one hand, based on the Trusted Cryptography Module (TPM), the TPCM main control firmware measures the reliability of the protection and computing modules, as well as their firmware. On the other hand, the Trusted Software Base (TSB) measures the reliability of system software and application software. In addition, the TPCM management platform verifies the reliability measurement and synchronizes and manages the trust policies. - -### Feature Description - -The overall system design consists of the protection module, computing module, and trusted management center software, as shown in the following figure. - -![](./figures/TPCM.png) - -- Trusted management center: This centralized management platform, provided by a third-party vendor, formulates, delivers, maintains, and stores protection policies and reference values for trusted computing nodes. -- Protection module: This module operates independently of the computing module and provides trusted computing protection functions that feature active measurement and active control to implement security protection during computing. The protection module consists of the TPCM main control firmware, TCB, and TCM. As a key module for implementing trust protection in a trusted computing node, the TPCM can be implemented in multiple forms, such as cards, chips, and IP cores. It contains a CPU and memory, firmware, and software such as an OS and trusted function components. The TPCM operates alongside the computing module and works according to the built-in protection policy to monitor the trust of protected resources, such as hardware, firmware, and software of the computing module. The TPCM is the Root of Trust in a trusted computing node. - -- Computing module: This module includes hardware, an OS, and application layer software. The running of the OS can be divided into the boot phase and the running phase. In the boot phase, GRUB2 and shim of openEuler support the reliability measurement capability, which protects boot files such as shim, GRUB2, kernel, and initramfs. In the running phase, openEuler supports the deployment of the trusted verification agent (provided by third-party vendor HTTC). The agent sends data to the TPCM for trusted measurement and protection in the running phase. - -The TPCM interacts with other components as follows: - -1. The TPCM hardware, firmware, and software provide an operating environment for the TSB. The trusted function components of the TPCM provide support for the TSB to implement measurement, control, support, and decision-making based on the policy library interpretation requirements. -2. The TPCM accesses the TCM for trusted cryptography functions to complete computing tasks such as trusted verification, measurement, and confidential storage, and provides services for TCM access. -3. The TPCM connects to the trusted management center through the management interface to implement protection policy management and trusted report processing. -4. The TPCM uses the built-in controller and I/O port to interact with the controller of the computing module through the bus to actively monitor the computing module. -5. The built-in protection agent in the OS of the computing module obtains the code and data related to the preset protection object and provides them to the TPCM. The TPCM forwards the monitoring information to the TSB, and the TSB analyzes and processes the information according to the policy library. - -### Constraints - -Supported server: TaiShan 200 server (model 2280) -Supported BMC card: BC83SMMC - -### Application Scenarios - -The TPCM enables a complete trust chain to ensure that the OS boots into a trusted computing environment. diff --git a/docs/en/docs/Administration/user-and-user-group-management.md b/docs/en/docs/Administration/user-and-user-group-management.md deleted file mode 100644 index 32c30d32458c0966aa9a837cf533d7175afa0690..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/user-and-user-group-management.md +++ /dev/null @@ -1,341 +0,0 @@ -# User and User Group Management - -In Linux, each common user has an account, including the user name, password, and home directory. There are also special users created for specific purposes, and the most important special user is the admin account whose default user name is root. In addition, Linux provides user groups so that each user belongs to at least one group, facilitating permission management. - -The control of users and user groups is a core element of openEuler security management. This topic introduces the user and group management commands and explains how to assign privileges to common users in graphical user interface and on command lines. - - -- [User and User Group Management](#user-and-user-group-management) - - [Managing Users](#managing-users) - - [Adding a User](#adding-a-user) - - [Modifying a User Account](#modifying-a-user-account) - - [Deleting a User](#deleting-a-user) - - [Granting Rights to a Common User](#granting-rights-to-a-common-user) - - [Managing User Groups](#managing-user-groups) - - [Adding a User Group](#adding-a-user-group) - - [Modifying a User Group](#modifying-a-user-group) - - [Deleting a User Group](#deleting-a-user-group) - - [Adding a User to a Group or Removing a User from a Group](#adding-a-user-to-a-group-or-removing-a-user-from-a-group) - - [Changing the Current Group of a User to a Specified Group](#changing-the-current-group-of-a-user-to-a-specified-group) - - - -## Managing Users - -### Adding a User - -#### useradd Command - -Run the **useradd** command as the user **root** to add user information to the system. In the command, _options_ indicates related parameters and _username_ indicates the user name. - -```shell -useradd [options] username -``` - -#### User Information Files - -The following files contain user account information: - -- /etc/passwd: user account information -- /etc/shadow file: user account encryption information -- /etc/group file: group information -- /etc/default/useradd: default configurations -- /etc/login.defs: system wide settings -- /etc/skel: default directory that holds initial configuration files - -#### Example - -For example, to create a user named userexample, run the following command as the user **root**: - -```shell -useradd userexample -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If no prompt is displayed, the user is successfully created. After the user is created, run the **passwd** command to assign a password to the user. A new account without a password will be banned. - -To view information about the new user, run the **id** command: - -```shell -$ id userexample -uid=502(userexample) gid=502(userexample) groups=502(userexample) -``` - -To change the password of the userexample, run the following command: - -```shell -passwd userexample -``` - -It is recommended that the new user password meet the complexity requirements. The password complexity requirements are as follows: - -1. A password must contain at least eight characters. -2. A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters. -3. A password must be different from the account name. -4. A password cannot contain words in the dictionary. - - Querying a dictionary - In the installed openEuler environment, you can run the following command to export the dictionary library file **dictionary.txt**, and then check whether the password is in the dictionary. - - ```shell - cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt - ``` - - - Modifying a dictionary - 1. Modify the exported dictionary library file, and then run the following command to update the dictionary library: - - ```shell - create-cracklib-dict dictionary.txt - ``` - - 2. Run the following command to add another dictionary file **custom.txt** to the original dictionary library. - - ```shell - create-cracklib-dict dictionary.txt custom.txt - ``` - -Then, enter the password and confirm it as prompted: - -```shell -$ passwd userexample -Changing password for user userexample. -New password: -Retype new password: -passwd: all authentication tokens updated successfully. -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If the command output contains **BAD PASSWORD: The password fails the dictionary check - it is too simplistic/sytematic**, the password is too simple and needs to be reset. - -### Modifying a User Account - -#### Changing a Password - -Common users can change their passwords using the **passwd** command. Only the admin is allowed to use the **passwd username** command to change passwords for other users. - -#### Changing User's Login Shell - -Common users can use the **chsh** command to change their login shell. Only the admin is allowed to run the **chsh username** command to change login shell for other users. - -Users can also run the **usermod** command as the user **root** to modify the shell information. In the command, _new_shell_path_ indicates the target shell path, and _username_ indicates the user name to be modified. Change them based on the site requirements. - -```shell -usermod -s new_shell_path username -``` - -For example, to change the shell of userexample to csh, run the following command: - -```shell -usermod -s /bin/csh userexample -``` - -#### Changing the Home Directory - -- To change the home directory, run the following command as the user **root**. In the command, _new\_home\_directory_ indicates the created target home directory, and _username_ indicates the user name to be changed. Change them based on the site requirements. - - ```shell - usermod -d new_home_directory username - ``` - -- To move the content in the current home directory to a new one, run the usermod command with the -m option: - - ```shell - usermod -d new_home_directory -m username - ``` - -#### Changing a UID - -To change the user ID, run the following command as the user **root**. In the command, _UID_ indicates the target user ID and _username_ indicates the user name. Change them based on the site requirements. - -```shell -usermod -u UID username -``` - -The **usermod** command automatically changes the UID of the owner of files and directories under the user's home directory. However, for files outside the user's home directory, their owners can only be changed using the **chown** command. - -#### Changing Account Expiry Date - -If the shadow password is used, run the following command as the user **root** to change the validity period of an account. In the command, _MM_, _DD_, and _YY_ indicate the month, day, and year, respectively, and _username_ indicates the user name. Change them based on the site requirements. - -```shell -usermod -e MM/DD/YY username -``` - -### Deleting a User - -Run the **userdel** command as the user **root** to delete an existing user. - -For example, run the following command to delete user Test: - -```shell -userdel Test -``` - -If you also need to delete the user's home directory and all contents in the directory, run the **userdel** command with the -r option to delete them recursively. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->You are not advised to directly delete a user who has logged in to the system. To forcibly delete a user, run the **userdel -f** _Test_ command. - -### Granting Rights to a Common User - -The **sudo** command allows common users to execute commands that can be executed only by administrator accounts. - -The **sudo** command allows the user specified in the **/etc/sudoers** file to execute the administrator account commands. For example, an authorized common user can run: - -```shell -sudo /usr/sbin/useradd newuserl -``` - -The **sudo** command can specify a common user that has been added to the **/etc/sudoers** file to process tasks as required. - -The information configured in the **/etc/sudoers** file is as follows: - -- Blank lines or comment lines starting with **\#**: Have no specific functions. -- Optional host alias lines: Create the name of a host list. The lines must start with **Host\_Alias**. The host names in the list must be separated by commas \(,\). For example: - - ```shell - Host_Alias linux=ted1,ted2 - ``` - - **ted1** and **ted2** are two host names, which can be called **linux**. - -- Optional user alias lines: Create the name of a user list. The lines must start with **User\_Alias**. The user names in the list must be separated by commas \(,\). The user alias lines have the same format as the host alias lines. -- Optional command alias lines: Create the name of a command list. The lines must start with **Cmnd\_Alias**. The commands in the list must be separated by commas \(,\). -- Optional running mode alias lines: Create the name of a user list. The difference is that such alias can enable a user in the list to run the **sudo** command. -- Necessary declaration lines for user access: - - The declaration syntax for user access is as follows: - - ```shell - user host = [ run as user ] command list - ``` - - Set the user to a real user name or a defined user alias, and set the host to a real host name or a defined host alias. By default, all the commands executed by sudo are executed as user **root**. If you want to use another account, you can specify it. **command list** is either a command list separated by commas \(,\) or a defined command alias. For example: - - ```shell - ted1 ted2=/sbin/shutdown - ``` - - In this example, ted1 can run the shutdown command on ted2. - - ```shell - newuser1 ted1=(root) /usr/sbin/useradd,/usr/sbin/userdel - ``` - - This indicates that newuser1 on the ted1 host can run the **useradd** and **userdel** commands as the user **root**. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- You can define multiple aliases in a line and separate them with colons \(:\). - >- You can add an exclamation mark \(!\) before a command or a command alias to make the command or the command alias invalid. - >- There are two keywords: ALL and NOPASSWD. ALL indicates all files, hosts, or commands, and NOPASSWD indicates that no password is required. - >- By modifying user access, you can change the access permission of a common user to be the same as that of the user **root**. Then, you can grant rights to the common user. - -The following is an example of the **sudoers** file: - -```text -#sudoers files -#User alias specification -User_Alias ADMIN=ted1:POWERUSER=globus,ted2 -#user privilege specification -ADMIN ALL=ALL -POWERUSER ALL=ALL,!/bin/su -``` - -In the preceding information: - -- User\_Alias ADMIN=ted1:POWERUSER=globus,ted2 - - Two aliases ADMIN and POWERUSER are defined. - -- ADMIN ALL=ALL - - ADMIN can run all commands as the user **root** on all hosts. - -- POWERUSER ALL=ALL,!/bin/su - - POWERUSER can run all commands except the **su** command as the user **root** on all hosts. - -## Managing User Groups - -### Adding a User Group - -#### groupadd Command - -Run the **groupadd** command as the **root** user to add user group information to the system. In the command, _options_ indicates related parameters and _groupname_ indicates the group name. - -```shell -groupadd [options] groupname -``` - -#### User Group Information Files - -The following files contain user group information: - -- /etc/gshadow file: user group encryption information -- /etc/group file: group information -- /etc/login.defs: system wide settings - -#### Example - -For example, to create a user group named groupexample, run the following command as the **root** user: - -```shell -groupadd groupexample -``` - -### Modifying a User Group - -#### Changing a GID - -To change the user group ID, run the following command as the **root** user. In the command, _GID_ indicates the target user group ID and _groupname_ indicates the user group name. Change them based on the site requirements. - -```shell -groupmod -g GID groupname -``` - -#### Changing a User Group Name - -To change the user group name, run the following command as the **root** user. In the command, _newgroupname_ indicates the user group new name and _oldgroupname_ indicates the user group name. Change them based on the site requirements. - -```shell -groupmod -n newgroupname oldgroupname -``` - -### Deleting a User Group - -Run the **groupdel** command as the **root** user to delete an existing user group. - -For example, run the following command to delete user group Test: - -```shell -groupdel Test -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The user's primary group cannot be directly deleted. To forcibly delete a user's primary group, run the **groupdel -f** _Test_ command. - -### Adding a User to a Group or Removing a User from a Group - -Run the **gpasswd** command as the **root** user to add a user to a group or remove a user from a group. - -For example, run the following command to add the user userexample to the user group Test: - -```shell -gpasswd -a userexample Test -``` - -For example, run the following command to remove the user userexample from the user group Test: - -```shell -gpasswd -d userexample Test -``` - -### Changing the Current Group of a User to a Specified Group - -If a user belongs to multiple user groups, run the **newgrp** command to switch the user to another user group after logging in to the system. Then, the user has the permission of other user groups. - -For example, run the following command to change the current group of the user userexample to the user group Test: - -```shell -newgrp Test -``` diff --git a/docs/en/docs/Administration/using-dnf-to-manage-software-packages.md b/docs/en/docs/Administration/using-dnf-to-manage-software-packages.md deleted file mode 100644 index 227a21428b84eaa44ef34573d17d6847f517c527..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/using-dnf-to-manage-software-packages.md +++ /dev/null @@ -1,568 +0,0 @@ -# Using DNF to Manage Software Packages - -DNF is a Linux software package management tool used to manage RPM software packages. The DNF can query software package information, obtain software packages from a specified software library, automatically process dependencies to install or uninstall software packages, and update the system to the latest available version. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- DNF is fully compatible with YUM and provides YUM-compatible command lines and APIs for extensions and plug-ins. ->- You must have the administrator rights to use the DNF. All commands in this chapter must be executed by the administrator. - -## Configuring the DNF - -### The DNF Configuration File - -The main configuration file of the DNF is **/etc/dnf/dnf.conf** which consists of two parts: - -- The **main** part in the file stores the global settings of the DNF. - -- The **repository** part in the file stores the settings of the software source. You can add zero or more **repository** sections to the file. - -In addition, the **/etc/yum.repos.d** directory stores zero or more repo source files, which define different repositories. - -You can configure a software source by either directly configuring the **/etc/dnf/dnf.conf** file or configuring the .repo file in the **/etc/yum.repos.d** directory. - -#### Configuring the main Part - -The **/etc/dnf/dnf.conf** file contains the **main** part. The following is an example of the configuration file: - -```text -[main] -gpgcheck=1 -installonly_limit=3 -clean_requirements_on_remove=True -best=True -``` - -Common options are as follows: - -**Table 1** main parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

cachedir

-

Cache directory for storing RPM packages and database files.

-

keepcache

-

The options are 1 and 0, indicating whether to cache the RPM packages and header files that have been successfully installed. The default value is 0, indicating that the RPM packages and header files are not cached.

-

debuglevel

-

Sets debugging information generated by the DNF. The value ranges from 0 to 10. A larger value indicates more detailed debugging information. The default value is 2. The value 0 indicates that the debug information is not displayed.

-

clean_requirements_on_remove

-

Deletes the dependency items that are no longer used during DNF removal. If the software package is installed through the DNF instead of the explicit user request, the software package can be deleted only through clean_requirements_on_remove, that is, the software package is introduced as a dependency item. The default value is True.

-

best

-

The system always attempts to install the latest version of the upgrade package. If the latest version cannot be installed, the system displays the cause and stops the installation. The default value is True.

-

obsoletes

-

The options are 1 and 0, indicating whether to allow the update of outdated RPM packages. The default value is 1, indicating that the update is allowed.

-

gpgcheck

-

The options are 1 and 0, indicating whether to perform GPG verification. The default value is 1, indicating that verification is required.

-

plugins

-

The options are 1 and 0, indicating that the DNF plug-in is enabled or disabled. The default value is 1, indicating that the DNF plug-in is enabled.

-

installonly_limit

-

Sets the number of packages that can be installed at the same time by running the installonlypkgs command. The default value is 3. You are advised not to decrease the value.

-
- -#### Configuring the repository Part - -The repository part allows you to customize openEuler software source repositories. The name of each repository must be unique. Otherwise, conflicts may occur. You can configure a software source by either directly configuring the **/etc/dnf/dnf.conf** file or configuring the .repo file in the **/etc/yum.repos.d** directory. - -- Configuring the **/etc/dnf/dnf.conf** file - - The following is a minimum configuration example of the \[repository\] section: - - ```text - [repository] - name=repository_name - baseurl=repository_url - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >openEuler provides an online image source at [https://repo.openeuler.org/](https://repo.openeuler.org/). For example, if the openEuler 21.03 version is aarch64, the **baseurl** can be set to [https://repo.openeuler.org/openEuler-21.03/OS/aarch64/](https://repo.openeuler.org/openEuler-21.03/OS/aarch64/). - - Common options are as follows: - - **Table 2** repository parameter description - - - - - - - - - - - - -

Parameter

-

Description

-

name=repository_name

-

Name string of a software repository.

-

baseurl=repository_url

-

Address of the software repository.

-
  • Network location using the HTTP protocol, for example, http://path/to/repo
  • Network location using the FTP protocol, for example, ftp://path/to/repo
  • Local path: for example, file:///path/to/local/repo
-
- -- Configuring the .repo file in the **/etc/yum.repos.d** directory - openEuler provides multiple repo sources for users online. For details about the repo sources, see [OS Installation](../Releasenotes/installing-the-os.md). - - For example, run the following command as the **root** user to add the openeuler repo source to the openEuler.repo file. - - ```shell - vi /etc/yum.repos.d/openEuler.repo - ``` - - ```text - [OS] - name=openEuler-$releasever - OS - baseurl=https://repo.openeuler.org/openEuler-21.03/OS/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=https://repo.openeuler.org/openEuler-21.03/OS/$basearch/RPM-GPG-KEY-openEuler - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled. - > - **gpgkey** is the public key used to verify the signature. - -#### Displays the Current Configuration - -- To display the current configuration information, run the following command: - - ```shell - dnf config-manager --dump - ``` - -- To display the configuration of a software source, query the repo id: - - ```shell - dnf repolist - ``` - - Run the following command to display the software source configuration of the corresponding ID. In the command, _repository_ indicates the repository ID. - - ```shell - dnf config-manager --dump repository - ``` - -- You can also use a global regular expression to display all matching configurations. - - ```shell - dnf config-manager --dump glob_expression - ``` - -### Creating a Local Software Repository - -To create a local repository of software sources, perform the following steps. - -1. Install the createrepo software package. Run the following command as the root user: - - ```shell - dnf install createrepo - ``` - -2. Copy the required software packages to a directory, for example, **/mnt/local\_repo/**. -3. Run the following command to create a software source: - - ```shell - createrepo /mnt/local_repo - ``` - -### Adding, Enabling, and Disabling Software Sources - -This section describes how to add, enable, and disable the software source repository by running the **dnf config-manager** command. - -#### Adding Software Source - -To define a new software repository, you can add the repository part to the **/etc/dnf/dnf.conf** file or add the .repo file to the **/etc/yum.repos.d/** directory. You are advised to add the .repo file. Each software source has its own .repo file. The following describes how to add the .repo file. - -To add such a source to your system, run the following command as the user **root**. After the command is executed, the corresponding .repo file is generated in the **/etc/yum.repos.d/** directory. In the command, _repository\_url_ indicates the repo source address. For details, see [Table 2](#en-us_topic_0151921080_t7c83ace02ab94e9986c0684f417e3436). - -```shell -dnf config-manager --add-repo repository_url -``` - -#### Enabling a Software Repository - -To enable the software source, run the following command as the user **root**. In the command, _repository_ indicates the repository ID in the new .repo file. You can run the **dnf repolist** command to query the repository ID. - -```shell -dnf config-manager --set-enable repository -``` - -You can also use a global regular expression to enable all matching software sources. In the command, _glob\_expression_ indicates the regular expression used to match multiple repository IDs. - -```shell -dnf config-manager --set-enable glob_expression -``` - -#### Disabling a Software Repository - -To disable a software source, run the following command as the user **root**: - -```shell -dnf config-manager --set-disable repository -``` - -You can also use a global regular expression to disable all matching software sources. - -```shell -dnf config-manager --set-disable glob_expression -``` - -## Managing Software Package - -The DNF enables you to query, install, and delete software packages. - -### Searching for Software Packages - -You can search for the required RPM package by its name, abbreviation, or description. The command is as follows: - -```shell -dnf search term -``` - -The following is an example: - -```shell -$ dnf search httpd -========================================== N/S matched: httpd ========================================== -httpd.aarch64 : Apache HTTP Server -httpd-devel.aarch64 : Development interfaces for the Apache HTTP server -httpd-manual.noarch : Documentation for the Apache HTTP server -httpd-tools.aarch64 : Tools for use with the Apache HTTP Server -libmicrohttpd.aarch64 : Lightweight library for embedding a webserver in applications -mod_auth_mellon.aarch64 : A SAML 2.0 authentication module for the Apache Httpd Server -mod_dav_svn.aarch64 : Apache httpd module for Subversion server -``` - -### Listing Software Packages - -To list all installed and available RPM packages in the system, run the following command: - -```shell -dnf list all -``` - -To list a specific RPM package in the system, run the following command: - -```shell -dnf list glob_expression... -``` - -The following is an example: - -```shell -$ dnf list httpd -Available Packages -httpd.aarch64 2.4.34-8.h5.oe1 Local -``` - -### Displaying RPM Package Information - -To view information about one or more RPM packages, run the following command: - -```shell -dnf info package_name... -``` - -The following is a command example: - -```shell -$ dnf info httpd -Available Packages -Name : httpd -Version : 2.4.34 -Release : 8.h5.oe1 -Arch : aarch64 -Size : 1.2 M -Repo : Local -Summary : Apache HTTP Server -URL : http://httpd.apache.org/ -License : ASL 2.0 -Description : The Apache HTTP Server is a powerful, efficient, and extensible - : web server. -``` - -### Installing an RPM Package - -To install a software package and all its dependencies that have not been installed, run the following command as the user **root**: - -```shell -dnf install package_name -``` - -You can also add software package names to install multiple software packages at the same time. Add the **strict=False** parameter to the **/etc/dnf/dnf.conf** configuration file and run the **dnf** command with `--setopt=strict=0`. Run the following command as the user **root**: - -```shell -dnf install package_name package_name... --setopt=strict=0 -``` - -The following is an example: - -```shell -dnf install httpd -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If the RPM package fails to be installed, see [Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package](./faqs.md#installation-failure-caused-by-software-package-conflict-file-conflict-or-missing-software-package). - -### Downloading Software Packages - -To download the software package using the DNF, run the following command as the user **root**: - -```shell -dnf download package_name -``` - -If you need to download the dependency packages that are not installed, add **\-\-resolve**. The command is as follows: - -```shell -dnf download --resolve package_name -``` - -The following is an example: - -```shell -dnf download --resolve httpd -``` - -### Deleting a Software Package - -To uninstall the software package and related dependent software packages, run the following command as the user **root**: - -```shell -dnf remove package_name... -``` - -The following is an example: - -```shell -dnf remove totem -``` - -## Managing Software Package Groups - -A software package set is a group of software packages that serve a common purpose, for example, a system tool set. You can use the DNF to install or delete software package groups, improving operation efficiency. - -### Listing Software Package Groups - -The summary parameter can be used to list the number of all installed software package groups, available groups, and available environment groups in the system. The command is as follows: - -```shell -dnf groups summary -``` - -The following is an example: - -```shell -$ dnf groups summary -Last metadata expiration check: 0:11:56 ago on Sat 17 Aug 2019 07:45:14 PM CST. -Available Groups: 8 -``` - -To list all software package groups and their group IDs, run the following command: - -```shell -dnf group list -``` - -The following is an example: - -```shell -$ dnf group list -Last metadata expiration check: 0:10:32 ago on Sat 17 Aug 2019 07:45:14 PM CST. -Available Environment Groups: - Minimal Install - Custom Operating System - Server -Available Groups: - Development Tools - Graphical Administration Tools - Headless Management - Legacy UNIX Compatibility - Network Servers - Scientific Support - Security Tools - System Tools -``` - -### Displaying the Software Package Group Information - -To list the mandatory and optional packages contained in a software package group, run the following command: - -```shell -dnf group info glob_expression... -``` - -The following is an example of displaying the Development Tools information: - -```shell -$ dnf group info "Development Tools" -Last metadata expiration check: 0:14:54 ago on Wed 05 Jun 2019 08:38:02 PM CST. - -Group: Development Tools - Description: A basic development environment. - Mandatory Packages: - binutils - glibc-devel - make - pkgconf - pkgconf-m4 - pkgconf-pkg-config - rpm-sign - Optional Packages: - expect -``` - -### Installation Software Package Group - -Each software package group has its own name and corresponding group ID. You can use the software package group name or its ID to install the software package. - -To install a software package group, run the following command as the user **root**: - -```shell -dnf group install group_name -``` - -```shell -dnf group install groupid -``` - -For example, to install the software package group of Development Tools, run the following command: - -```shell -dnf group install "Development Tools" -``` - -```shell -dnf group install development -``` - -### Deleting a Software Package Group - -To uninstall a software package group, you can use the group name or ID to run the following command as the user **root**: - -```shell -dnf group remove group_name -``` - -```shell -dnf group remove groupid -``` - -For example, to delete the software package group of Development Tools, run the following command: - -```shell -dnf group remove "Development Tools" -``` - -```shell -dnf group remove development -``` - -## Check and Update - -You can use the DNF to check whether any software package in your system needs to be updated. You can use the DNF to list the software packages to be updated. You can choose to update all packages at a time or update only specified packages. - -### Checking for Update - -To list all currently available updates, run the following command: - -```shell -dnf check-update -``` - -The following is an example: - -```shell -$ dnf check-update -Last metadata expiration check: 0:02:10 ago on Sun 01 Sep 2019 11:28:07 PM CST. - -anaconda-core.aarch64 19.31.123-1.14 updates -anaconda-gui.aarch64 19.31.123-1.14 updates -anaconda-tui.aarch64 19.31.123-1.14 updates -anaconda-user-help.aarch64 19.31.123-1.14 updates -anaconda-widgets.aarch64 19.31.123-1.14 updates -bind-libs.aarch64 32:9.9.4-29.3 updates -bind-libs-lite.aarch64 32:9.9.4-29.3 updates -bind-license.noarch 32:9.9.4-29.3 updates -bind-utils.aarch64 32:9.9.4-29.3 updates -... -``` - -### Upgrade - -To upgrade a single software package, run the following command as the user **root**: - -```shell -dnf update package_name -``` - -For example, to upgrade the RPM package, run the following command: - -```shell -$ dnf update anaconda-gui.aarch64 -Last metadata expiration check: 0:02:10 ago on Sun 01 Sep 2019 11:30:27 PM CST. -Dependencies Resolved -================================================================================ - Package Arch Version Repository Size -================================================================================ -Updating: - anaconda-gui aarch64 19.31.123-1.14 updates 461 k - anaconda-core aarch64 19.31.123-1.14 updates 1.4 M - anaconda-tui aarch64 19.31.123-1.14 updates 274 k - anaconda-user-help aarch64 19.31.123-1.14 updates 315 k - anaconda-widgets aarch64 19.31.123-1.14 updates 748 k - -Transaction Summary -================================================================================ -Upgrade 5 Package - -Total download size: 3.1 M -Is this ok [y/N]: -``` - -Similarly, to upgrade a software package group, run the following command as the user **root**: - -```shell -dnf group update group_name -``` - -### Updating All Packages and Their Dependencies - -To update all packages and their dependencies, run the following command as the user **root**: - -```shell -dnf update -``` diff --git a/docs/en/docs/Administration/using-the-kae.md b/docs/en/docs/Administration/using-the-kae.md deleted file mode 100644 index 2a2e1aee1a140554a1b2708ed5cddc99fb400152..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/using-the-kae.md +++ /dev/null @@ -1,742 +0,0 @@ -# Using the Kunpeng Accelerator Engine (KAE) - - -- [Using the Kunpeng Accelerator Engine (KAE)](#using-the-kunpeng-accelerator-engine-kae) - - [Overview](#overview) - - [Application Scenarios](#application-scenarios) - - [Installing, Running, and Uninstalling the KAE](#installing-running-and-uninstalling-the-kae) - - [Installing the Accelerator Software Packages](#installing-the-accelerator-software-packages) - - [Upgrading the Accelerator Software Packages](#upgrading-the-accelerator-software-packages) - - [Uninstalling the Accelerator Software Packages](#uninstalling-the-accelerator-software-packages) - - [Querying Logs](#querying-logs) - - [Acceleration Engine Application](#acceleration-engine-application) - - [Example Code for the KAE](#example-code-for-the-kae) - - [Usage of the KAE in the OpenSSL Configuration File openssl.cnf](#usage-of-the-kae-in-the-openssl-configuration-file-opensslcnf) - - [Troubleshooting](#troubleshooting) - - [Failed to Initialize the Accelerator Engine](#failed-to-initialize-the-accelerator-engine) - - [Failed to Identify Accelerator Devices After the Acceleration Engine Is Installed](#failed-to-identify-accelerator-devices-after-the-acceleration-engine-is-installed) - - [Failed to Upgrade the Accelerator Drivers](#failed-to-upgrade-the-accelerator-drivers) - - -## Overview - -Kunpeng Accelerator Engine \(KAE\) is a software acceleration library of openEuler, which provides hardware acceleration engine function on the Kunpeng 920 processor. It supports symmetric encryption, asymmetric encryption, and digital signature. It is ideal for accelerating SSL/TLS applications, reducing processor consumption and improving processor efficiency. In addition, users can quickly migrate existing services through the standard OpenSSL interface. - -The KAE supports the following algorithms: - -- Digest algorithm SM3, which supports asynchronous mode. -- Symmetric encryption algorithm SM4, which supports asynchronous, CTR, XTS, and CBC modes. -- Symmetric encryption algorithm AES, which supports asynchronous, ECB, CTR, XTS, and CBC modes. -- Asymmetric algorithm RSA, which supports asynchronous mode and key sizes 1024, 2048, 3072, and 4096. -- Key negotiation algorithm DH, which supports asynchronous mode and key sizes 768, 1024, 1536, 2048, 3072, and 4096. - -## Application Scenarios - -The KAE applies to the following scenarios, as shown in [Table 1](#table11915824163418). - -**Table 1** Application scenarios - - - - - - - - - - - - - - - - - - - -

Scenario

-

Data

-

Big data

-

Stream data

-

Data encryption

-

Block data

-

Intelligent security protection

-

Video stream data

-

Web service

-

Handshake connections

-
- -## Installing, Running, and Uninstalling the KAE - -### Installing the Accelerator Software Packages - -#### Preparing for Installation - -##### Environment Requirements - -- The accelerator engine is enabled on TaiShan 200 servers. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- You need to import the accelerator license. For details, see section "License Management" in the [TaiShan Rack Server iBMC \(V500 or Later\) User Guide](https://support.huawei.com/enterprise/en/doc/EDOC1100121685/426cffd9?idPath=7919749|9856522|21782478|8060757). ->- If the accelerator is used in the physical machine scenario, the SMMU must be disabled. For details, see the [TaiShan 200 Server BIOS Parameter Reference](https://support.huawei.com/enterprise/en/doc/EDOC1100088647). - -- CPU: Kunpeng 920 -- OS: openEuler-21.09-aarch64-dvd.iso - -##### KAE Software Description - -**Table 2** RPM software packages of the KAE - - - - - - - - - - - - - - - - -

Software Package

-

Description

-

kae_driver-version number-1.OS type.aarch64.rpm

-

Accelerator driver, including the uacce.ko, hisi_qm.ko, hisi_sec2.ko, and hisi_hpre.ko kernel modules.

-

Algorithms supported: SM3, SM4, AES, RSA, and DH.

-

libwd-version number-1.OS type.aarch64.rpm

-

Coverage: libwd.so dynamic link library.

-

It provides interfaces for the KAE.

-

libkae-version number-1.OS type.aarch64.rpm

-

Dependency: libwd RPM package.

-

Coverage: libkae.so dynamic library.

-

Algorithms supported: SM3, SM4, AES, RSA, and DH.

-
- -#### Installing the Accelerator Software Package - -##### Prerequisites - -- The remote SSH login tool has been installed on the local PC. -- The openEuler OS has been installed. -- The RPM tool is running properly. -- OpenSSL 1.1.1a or a later version has been installed. - - You can run the following commands to query the version number of OpenSSL: - - - openssl version - -##### Procedure - -1. Log in to the openEuler OS CLI as user **root**. -2. Create a directory for storing accelerator engine software packages. -3. Use SSH to copy all accelerator engine software packages to the created directory. -4. In the directory, run the **rpm -ivh** command to install the accelerator engine software packages. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Install the **libwd** package first because the **libkae** package installation depends on the **libwd** package. - - ```shell - rpm -ivh uacce*.rpm hisi*.rpm libwd-*.rpm libkae*.rpm - ``` - - ```text - Verifying... ################################# [100%] - Preparing... ################################# [100%] - checking installed modules - uacce modules start to install - Updating / installing... - 1:uacce-1.2.10-4.oe1 ################################# [ 14%] - uacce modules installed - 2:libwd-1.2.10-3.oe1 ################################# [ 29%] - 3:libkae-1.2.10-3.oe1 ################################# [ 43%] - checking installed modules - hisi_hpre modules start to install - 4:hisi_hpre-1.2.10-4.oe1 ################################# [ 57%] - hisi_hpre modules installed - checking installed modules - hisi_rde modules start to install - 5:hisi_rde-1.2.10-4.oe1 ################################# [ 71%] - hisi_rde modules installed - checking installed modules - hisi_sec2 modules start to install - 6:hisi_sec2-1.2.10-4.oe1 ################################# [ 86%] - hisi_sec2 modules installed - checking installed modules - hisi_zip modules start to install - 7:hisi_zip-1.2.10-4.oe1 ################################# [100%] - hisi_zip modules installed - ``` - -5. Run the **rpm -qa** command to check whether the accelerator software packages have been installed successfully. Run the **rpm -ql** command to check whether files in the software packages are correct. The following is an example: - - ```shell - rpm -qa|grep -E "hisi|uacce|libwd|libkae" - ``` - - ```text - hisi_rde-1.2.10-4.oe1.aarch64 - hisi_sec2-1.2.10-4.oe1.aarch64 - libkae-1.2.10-3.oe1.aarch64 - hisi_hpre-1.2.10-4.oe1.aarch64 - uacce-1.2.10-4.oe1.aarch64 - libwd-1.2.10-3.oe1.aarch64 - hisi_zip-1.2.10-4.oe1.aarch64 - ``` - - ```shell - rpm -ql uacce hisi* libwd* libkae - ``` - - ```text - /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_qm.ko - /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/uacce.ko - /etc/modprobe.d/hisi_hpre.conf - /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_hpre.ko - /etc/modprobe.d/hisi_rde.conf - /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_rde.ko - /etc/modprobe.d/hisi_sec2.conf - /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_sec2.ko - /etc/modprobe.d/hisi_zip.conf - /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_zip.ko - /usr/include/warpdrive/config.h - /usr/include/warpdrive/include/uacce.h - /usr/include/warpdrive/smm.h - /usr/include/warpdrive/wd.h - /usr/include/warpdrive/wd_bmm.h - /usr/include/warpdrive/wd_cipher.h - /usr/include/warpdrive/wd_comp.h - /usr/include/warpdrive/wd_dh.h - /usr/include/warpdrive/wd_digest.h - /usr/include/warpdrive/wd_rsa.h - /usr/lib64/libwd.so.1.2.10 - /usr/local/lib/engines-1.1/libkae.so.1.2.10 - ``` - -6. Restart the system or run commands to manually load the accelerator engine drivers to the kernel in sequence, and check whether the drivers are successfully loaded. - - ```shell - modprobe uacce - lsmod | grep uacce - modprobe hisi_qm - lsmod | grep hisi_qm - modprobe hisi_sec2 # Loads the hisi_sec2 driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_sec2.conf. - modprobe hisi_hpre # Loads the hisi_hpre driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_hpre.conf. - ``` - -##### Environment Variables Setup - -Run the following command to export the environment variables \(If you have specified the installation directory, set **/usr/local** to the actual one\): - -```shell -export OPENSSL_ENGINES=/usr/local/lib/engines-1.1 -``` - -##### Post-Installation Check - -Run the **rpm -qa** command to check whether the accelerator engine software packages are successfully installed. - -If the command output contains _software package name_**-**_version number_**-**, the software packages are successfully installed. The following is an example: - -```shell -rpm -qa|grep -E "hisi|uacce|libwd|libkae" -``` - -```text -hisi_rde-1.2.10-4.oe1.aarch64 -hisi_sec2-1.2.10-4.oe1.aarch64 -libkae-1.2.10-3.oe1.aarch64 -hisi_hpre-1.2.10-4.oe1.aarch64 -uacce-1.2.10-4.oe1.aarch64 -libwd-1.2.10-3.oe1.aarch64 -hisi_zip-1.2.10-4.oe1.aarch64 -``` - -#### Required Operations After Installation - -##### Testing the OpenSSL Accelerator Engine - -You can run the following commands to test some accelerator functions. - -- Use the OpenSSL software algorithm to test the RSA performance. - - ```shell - $ ./openssl speed -elapsed rsa2048 - ... - sign verify sign/s verify/s - rsa 2048 bits 0.001384s 0.000035s 724.1 28365.8. - ``` - -- Use the KAE to test the RSA performance. - - ```shell - $ ./openssl speed -elapsed -engine kae rsa2048 - .... - sign verify sign/s verify/s - rsa 2048 bits 0.000355s 0.000022s 2819.0 45478.4 - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->After the KAE is used, the signature performance is improved from 724.1 sign/s to 2819 sign/s. - -- Use the OpenSSL software algorithm to test the asynchronous RSA performance. - - ```shell - $ ./openssl speed -elapsed -async_jobs 36 rsa2048 - .... - sign verify sign/s verify/s - rsa 2048 bits 0.001318s 0.000032s 735.7 28555 - ``` - -- Use the KAE to test the asynchronous RSA performance. - - ```shell - $ ./openssl speed -engine kae -elapsed -async_jobs 36 rsa2048 - .... - sign verify sign/s verify/s - rsa 2048 bits 0.000018s 0.000009s 54384.1 105317.0 - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->After the KAE is used, the asynchronous RSA signature performance is improved from 735.7 sign/s to 54384.1 sign/s. - -- Use the OpenSSL software algorithm to test the performance of the SM4 CBC mode. - - ```shell - $ ./openssl speed -elapsed -evp sm4-cbc - You have chosen to measure elapsed time instead of user CPU time. - .... - Doing sm4-cbc for 3s on 10240 size blocks: 2196 sm4-cbc's in 3.00s .... - type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes - sm4-cbc 82312.53k 85196.80k 85284.18k 85000.85k 85284.18k 85261.26k - ``` - -- Use the KAE to test the SM4 CBC mode performance. - - ```shell - $ ./openssl speed -elapsed -engine kae -evp sm4-cbc - engine "kae" set. - You have chosen to measure elapsed time instead of user CPU time. - ... - Doing sm4-cbc for 3s on 1048576 size blocks: 11409 sm4-cbc's in 3.00s - ... - type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes - sm4-cbc 383317.33k 389427.20k 395313.15k 392954.73k 394264.58k 394264.58k - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->After the KAE is used, the SM4 CBC mode performance is improved from 82312.53 kbit/s to 383317.33 kbit/s when the input data block size is 8 MB. - -- Use the OpenSSL software algorithm to test the SM3 mode performance. - - ```shell - $ ./openssl speed -elapsed -evp sm3 - You have chosen to measure elapsed time instead of user CPU time. - Doing sm3 for 3s on 102400 size blocks: 1536 sm3's in 3.00s - .... - type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes - sm3 50568.53k 52428.80k 52428.80k 52428.80k 52428.80k 52428.80k - ``` - -- Use the KAE to test the SM3 mode performance. - - ```shell - $ ./openssl speed -elapsed -engine kae -evp sm3 - engine "kae" set. - You have chosen to measure elapsed time instead of user CPU time. - Doing sm3 for 3s on 102400 size blocks: 19540 sm3's in 3.00s - .... - type 51200 bytes 102400 bytes 1048576 bytes 2097152 bytes 4194304 bytes 8388608 bytes - sm3 648243.20k 666965.33k 677030.57k 678778.20k 676681.05k 668292.44k - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->After the KAE is used, the SM3 algorithm performance is improved from 52428.80 kbit/s to 668292.44 kbit/s when the input data block size is 8 MB. - -- Use the OpenSSL software algorithm to test the asynchronous performance of the AES algorithm in CBC mode. - - ```shell - $ ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4 - You have chosen to measure elapsed time instead of user CPU time. - Doing aes-128-cbc for 3s on 51200 size blocks: 65773 aes-128-cbc's in 3.00s - Doing aes-128-cbc for 3s on 102400 size blocks: 32910 aes-128-cbc's in 3.00s - .... - type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes - aes-128-cbc 1122525.87k 1123328.00k 1120578.22k 1121277.27k 1119879.17k 1115684.86k - ``` - -- Use the KEA engine to test the asynchronous performance of the AES algorithm in CBC mode. - - ```shell - $ ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4 -engine kae - engine "kae" set. - You have chosen to measure elapsed time instead of user CPU time. - Doing aes-128-cbc for 3s on 51200 size blocks: 219553 aes-128-cbc's in 3.00s - Doing aes-128-cbc for 3s on 102400 size blocks: 117093 aes-128-cbc's in 3.00s - .... - type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes - aes-128-cbc 3747037.87k 3996774.40k 1189085.18k 1196774.74k 1196979.11k 1199570.94k - ``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- The AES algorithm supports only asynchronous mode when the data length is 256 KB or less. ->- After the KAE is used, the AES algorithm performance is improved from 1123328.00 kbit/s to 3996774.40 kbit/s when the input data block size is 100 KB. - -### Upgrading the Accelerator Software Packages - -#### Scenario - -You can run the **rpm -Uvh** command to upgrade the accelerator software. - -#### Procedure - -1. Download the latest accelerator engine software packages from the openEuler community. -2. Use SSH to log in to the Linux CLI as user **root**. -3. Save the downloaded software packages to a directory. -4. In the directory, run the **rpm -Uvh** command to upgrade the accelerator driver package and engine library package. The following is an example: - - The command and output are as follows: - - ![](./figures/en-us_image_0231143189.png) - - ![](./figures/en-us_image_0231143191.png) - -5. Run the **rpm -qa** command to check whether the upgrade is successful. Ensure that the queried version is the latest version. - - ![](./figures/en-us_image_0231143193.png) - - ![](./figures/en-us_image_0231143195.png) - -6. Restart the system or run the following commands to manually uninstall the drivers of the earlier version, load the drivers of the latest version, and check whether the new drivers are successfully loaded. - - ```shell - # Uninstall the existing drivers. - $ lsmod | grep uacce - uacce 262144 3 hisi_hpre,hisi_sec2,hisi_qm - $ - $ rmmod hisi_hpre - $ rmmod hisi_sec2 - $ rmmod hisi_qm - $ rmmod uacce - $ lsmod | grep uacce - $ - # Load the new drivers. - $ modprobe uacce - $ modprobe hisi_qm - $ modprobe hisi_sec2 # Loads the hisi_sec2 driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_sec2.conf. - $ modprobe hisi_hpre # Loads the hisi_hpre driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_hpre.conf. - $ lsmod | grep uacce - uacce 36864 3 hisi_sec2,hisi_qm,hisi_hpre - ``` - -### Uninstalling the Accelerator Software Packages - -#### Scenario - -You do not need the accelerator engine software or you want to install a new one. - -#### Procedure - -1. Use SSH to log in to the Linux CLI as user **root**. -2. Restart the system or run commands to manually uninstall the accelerator drivers loaded to the kernel, and check whether the drivers are successfully uninstalled. - - ```shell - # lsmod | grep uacce - uacce 36864 3 hisi_sec2,hisi_qm,hisi_hpre - # rmmod hisi_hpre - # rmmod hisi_sec2 - # rmmod hisi_qm - # rmmod uacce - # lsmod | grep uacce - # - ``` - -3. Run the **rpm -e** command to uninstall the accelerator engine software packages. The following is an example: - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Due to the dependency relationships, the **libkae** package must be uninstalled before the **libwd** package. - - ![](./figures/en-us_image_0231143196.png) - - ![](./figures/en-us_image_0231143197.png) - -4. Run the **rpm -qa |grep** command to check whether the uninstallation is successful. - - ![](./figures/en-us_image_0231143198.png) - -## Querying Logs - -[Table 3](#table52821836) lists log information related to the accelerator engine. - -**Table 3** Log information - - - - - - - - - - - - - - - - -

Directory

-

File

-

Description

-

/var/log/

-

kae.log

-

By default, the log level of the OpenSSL engine log is error. To set the log level, perform the following procedure:

-
  1. Run export KAE_CONF_ENV=/var/log/.
  2. Create the kae.cnf file in /var/log/.
  3. In the kae.cnf file, configure the content as follows:

    [LogSection]

    -

    debug_level=error #Value: none, error, info, warning or debug

    -
-
NOTE:

In normal cases, you are advised not to enable the info or debug log level. Otherwise, the accelerator performance will deteriorate.

-
-

/var/log/

-

messages/syslog

-
  • Kernel logs are stored in the /var/log/messages directory.
-
NOTE:

Alternatively, you can run the dmesg > /var/log/dmesg.log command to collect driver and kernel logs.

-
-
- -## Acceleration Engine Application - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If you have not purchased the engine license, you are advised not to use the KAE to invoke the corresponding algorithms. Otherwise, the performance of the OpenSSL encryption algorithm may be affected. - -### Example Code for the KAE - -```c -#include - -#include - -/* OpenSSL headers */ - -#include - -#include - -#include - -#include - -int main(int argc, char **argv) - -{ - - /* Initializing OpenSSL */ - - SSL_load_error_strings(); - - ERR_load_BIO_strings(); - - OpenSSL_add_all_algorithms(); - - /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/ - - ENGINE *e = ENGINE_by_id("kae"); - - /* Enable the accelerator asynchronization function. This parameter is optional. The value 0 indicates disabled, and the value 1 indicates enabled. The asynchronous function is enabled by default. */ - - ENGINE_ctrl_cmd_string(e, "KAE_CMD_ENABLE_ASYNC", "1", 0) - - ENGINE_init(e); - - RSA*rsa=RSA_new_method(e);#Specify the engine for RSA encryption and decryption. - - /*The user code*/ - - ...... - -; - - ENGINE_free(e); - -; - -} -``` - -### Usage of the KAE in the OpenSSL Configuration File openssl.cnf - -Create the **openssl.cnf** file and add the following configuration information to the file: - -```text -openssl_conf=openssl_def -[openssl_def] -engines=engine_section -[engine_section] -kae=kae_section -[kae_section] -engine_id=kae -dynamic_path=/usr/local/lib/engines-1.1/kae.so -KAE_CMD_ENABLE_ASYNC=1 #The value 0 indicates that the asynchronous function is disabled. The value 1 indicates that the asynchronous function is enabled. The asynchronous function is enabled by default. -default_algorithms=ALL -init=1 -``` - -Export the environment variable **OPENSSL\_CONF**. - -```shell -export OPENSSL_CONF=/home/app/openssl.cnf #Path for storing the openssl.cnf file -``` - -The following is an example of the OpenSSL configuration file: - -```c -#include - -#include - -/* OpenSSL headers */ - -#include - -#include - -#include - -#include - -int main(int argc, char **argv) - -{ - - /* Initializing OpenSSL */ - - SSL_load_error_strings(); - - ERR_load_BIO_strings(); - -#Load openssl configure - -OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL); OpenSSL_add_all_algorithms(); - - /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/ - - ENGINE *e = ENGINE_by_id("kae"); - - /*The user code*/ - - ...... - -; - - ENGINE_free(e); - -; -} -``` - -## Troubleshooting - -### Failed to Initialize the Accelerator Engine - -#### Symptom - -The accelerator engine is not completely loaded. - -#### Solution - -1. Check whether the accelerator drivers are loaded successfully. Specifically, run the **lsmod** command to check whether uacce.ko, qm.ko, sgl.ko, hisi\_sec2.ko, hisi\_hpre.ko, hisi\_zip.ko, and hisi\_rde.ko exist. - - ```shell - $ lsmod | grep uacce - uacce 262144 2 hisi_hpre,hisi_qm,hisi_sec2,hisi_zip,hisi_rde - ``` - -2. Check whether the accelerator engine library exists in **/usr/lib64** \(directory for RPM installation\) or **/usr/local/lib** \(directory for source code installation\) and the OpenSSL installation directory, and check whether the correct soft link is established. - - ```shell - $ ll /usr/local/lib/engines-1.1/ |grep kae - # Check whether the KAE has been correctly installed and whether a soft link has been established. If yes, the displayed information is as follows: - lrwxrwxrwx. 1 root root 22 Nov 12 02:33 kae.so -> kae.so.1.0.1 - lrwxrwxrwx. 1 root root 22 Nov 12 02:33 kae.so.0 -> kae.so.1.0.1 - -rwxr-xr-x. 1 root root 112632 May 25 2019 kae.so.1.0.1 - $ - $ ll /usr/lib64/ | grep libwd - # Check whether libwd has been correctly installed and whether a soft link has been established. If yes, the displayed information is as follows: - lrwxrwxrwx. 1 root root 14 Nov 12 02:33 libwd.so -> libwd.so.1.0.1 - lrwxrwxrwx. 1 root root 14 Nov 12 02:33 libwd.so.0 -> libwd.so.1.0.1 - -rwxr-xr-x. 1 root root 137120 May 25 2019 libwd.so.1.0.1 - $ - ``` - -3. Check whether the path of the OpenSSL engine library can be exported by running the **export** command. - - ```shell - $ echo $OPENSSL_ENGINES - $ export OPENSSL_ENGINES=/usr/local/lib/engines-1.1 - $ echo $OPENSSL_ENGINES - /usr/local/lib/engines-1.1 - ``` - -### Failed to Identify Accelerator Devices After the Acceleration Engine Is Installed - -#### Symptom - -After the acceleration engine is installed, the accelerator devices cannot be identified. - -#### Solution - -1. Check whether the device exists in the virtual file system. Normally, the following accelerator devices are displayed: - - ```shell - $ ls -al /sys/class/uacce/ - total 0 - lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-2 -> ../../devices/pci0000:78/0000:78:00.0/0000:79:00.0/uacce/hisi_hpre-2 - lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-3 -> ../../devices/pci0000:b8/0000:b8:00.0/0000:b9:00.0/uacce/hisi_hpre-3 - lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-4 -> ../../devices/pci0000:78/0000:78:01.0/uacce/hisi_rde-4 - lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-5 -> ../../devices/pci0000:b8/0000:b8:01.0/uacce/hisi_rde-5 - lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-0 -> ../../devices/pci0000:74/0000:74:01.0/0000:76:00.0/uacce/hisi_sec-0 - lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-1 -> ../../devices/pci0000:b4/0000:b4:01.0/0000:b6:00.0/uacce/hisi_sec-1 - lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-6 -> ../../devices/pci0000:74/0000:74:00.0/0000:75:00.0/uacce/hisi_zip-6 - lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-7 -> ../../devices/pci0000:b4/0000:b4:00.0/0000:b5:00.0/uacce/hisi_zip-7 - ``` - -2. If you want to use the HPRE device but the device is not found in [1](#li1760055514614), check whether the accelerator software is correctly installed by referring to [Failed to Upgrade the Accelerator Drivers](#failed-to-upgrade-the-accelerator-drivers). -3. If the accelerator software is correctly installed, run the **lspci** command to check whether the physical device exists. - - ```shell - $ lspci | grep HPRE - 79:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21) - b9:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21) - $ lspci | grep SEC - 76:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21) - b6:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21) - $ lspci | grep RDE - 78:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21) - b8:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21) - $ lspci | grep ZIP - 75:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21) - b5:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21) - $ - ``` - -4. If no physical device is found in [3](#li1560012551369), perform the following operations: - - Check whether the accelerator license has been imported. If no, import the accelerator license. For details, see "License Management" in the [TaiShan Rack Server iBMC \(V500 or Later\) User Guide](https://support.huawei.com/enterprise/en/doc/EDOC1100121685/426cffd9?idPath=7919749|9856522|21782478|8060757). After the accelerator license is imported, power off and restart the iBMC to enable the license. - - Check whether the iBMC and BIOS versions support the accelerator feature. - -### Failed to Upgrade the Accelerator Drivers - -#### Symptom - -After the accelerator drivers are upgraded, the driver version is not changed after the system is restarted. - -#### Possible Cause - -Before the accelerator drivers are upgraded, the system upgrades other driver packages. These driver packages may update the boot file system initramfs, and update the accelerator drivers to initramfs before upgrade. For example, if the NIC driver is updated or initramfs is manually updated, the system loads the accelerator drivers from initramfs first during restart. - -#### Solution - -After the accelerator drivers are upgraded, run the **dracut \-\-force** command to update initramfs again. diff --git a/docs/en/docs/Administration/viewing-system-information.md b/docs/en/docs/Administration/viewing-system-information.md deleted file mode 100644 index 5ac2d113813c5dd4c05daaf1ea86141e89e9fecb..0000000000000000000000000000000000000000 --- a/docs/en/docs/Administration/viewing-system-information.md +++ /dev/null @@ -1,47 +0,0 @@ -# Viewing System Information - -- View the system information. - - ``` - $ cat /etc/os-release - ``` - - For example, the command and output are as follows: - - ``` - $ cat /etc/os-release - NAME="openEuler" - VERSION="21.09" - ID="openEuler" - VERSION_ID="21.09" - PRETTY_NAME="openEuler 21.09" - ANSI_COLOR="0;31" - ``` - - -- View system resource information. - - Run the following command to view the CPU information: - - ``` - # lscpu - ``` - - Run the following command to view the memory information: - - ``` - $ free - ``` - - Run the following command to view the disk information: - - ``` - $ fdisk -l - ``` - - -- View the real-time system resource information. - - ``` - $ top - ``` \ No newline at end of file diff --git a/docs/en/docs/ApplicationDev/FAQ.md b/docs/en/docs/ApplicationDev/FAQ.md deleted file mode 100644 index f10dab063696adb8e4e563e75cae8d906003bc6d..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/FAQ.md +++ /dev/null @@ -1,24 +0,0 @@ -# FAQ - -- [FAQ](#faq) - - [Self-compilation of Some Applications Depending on the **java-devel** Package Fails](#self-compilation-of-some-applications-depending-on-the-java-devel-package-fails) - - -## Self-compilation of Some Applications Depending on the **java-devel** Package Fails - -### Symptom - -The self-compilation of some applications that depend on java-devel fails when the rpmbuild command is executed. - -### Cause Analysis - -To provide OpenJDK features that are updated and compatible with Java applications, the openEuler provides OpenJDK of multiple versions, such as OpenJDK 1.8.0 and OpenJDK 11. The compilation of some applications depends on the **java-devel** package. When the **java-devel** package is installed, the system installs java-11-openjdk of a later version by default. As a result, the compilation of these applications fails. - -### Solution - -You need to run the following command to install java-1.8.0-openjdk and then run the **rpmbuild** command to perform self-compilation: - -```shell -# yum install java-1.8.0-openjdk - -``` \ No newline at end of file diff --git a/docs/en/docs/ApplicationDev/application-development.md b/docs/en/docs/ApplicationDev/application-development.md deleted file mode 100644 index a4865d3a139e27ba40712d3e8e547ca84e3839b4..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/application-development.md +++ /dev/null @@ -1,95 +0,0 @@ -# Application Development Guide - -This document describes the common tools used for application development and guides users to develop applications based on openEuler. - -## Overview - -This document describes the following four parts to guide users to use openEuler and develop code based on openEuler. - -- Install and use the GCC compiler in the openEuler operating system \(OS\), and complete the development, compilation, and execution of simple code. -- In the openEuler OS, use the JDK built-in tool to compile and execute code. -- Install IntelliJ IDEA in the openEuler OS for Java development. -- Create an RPM package locally or using the Open Build Service \(OBS\). - -## Intended Audience - -This document is intended for all users who use the openEuler OS for code development. You are expected to have the following experience or capabilities: - -- Have basic knowledge of the Linux OS. -- Know how to use Linux command lines. - -## Symbol Conventions - -The symbols that may be found in this document are defined as follows. - - - - - - - - - - - - - - -

Symbol

-

Description

-

![](./figures/en-us_image_0229243712.png)

-

Indicates a potentially hazardous situation which, if not avoided, could result in equipment damage, data loss, performance deterioration, or unanticipated results.

-

NOTICE is used to address practices not related to personal injury.

-

![](./figures/en-us_image_0229243671.png)

-

Supplements the important information in the main text.

-

NOTE is used to address information not related to personal injury, equipment damage, and environment deterioration.

-
- -## Command Conventions - -**Table 1** Command conventions - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Format

-

Description

-

Boldface

-

Command keywords, which remain unchanged in the commands, are in boldface.

-

Italic

-

Command parameters, which are replaced with actual values in the commands, are in italic.

-

[ ]

-

Items in square brackets are optional.

-

{ x | y | ... }

-

Optional items are grouped in braces and separated by vertical bars. One item is selected.

-

[ x | y | ... ]

-

Optional items are grouped in brackets and separated by vertical bars. One item is selected or no item is selected.

-

{ x | y | ... }\*

-

Optional items are grouped in brackets and separated by vertical bars. A minimum of one or a maximum of all can be selected.

-

[ x | y | ... ]\*

-

Optional items are grouped in brackets and separated by vertical bars. One or more items are selected or no item is selected.

-
diff --git a/docs/en/docs/ApplicationDev/building-an-rpm-package.md b/docs/en/docs/ApplicationDev/building-an-rpm-package.md deleted file mode 100644 index adff77c5db14f3d8e145525e3c278c41fd3e5ff4..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/building-an-rpm-package.md +++ /dev/null @@ -1,548 +0,0 @@ -# Building an RPM Package - -This section describes how to build an RPM software package on a local PC or using OBS. For details, see the openEuler Packaging Guide. - -## Packaging Description - -### Principles - -During RPM packaging, the source code needs to be compiled. The compiled configuration files and binary command files need to be placed in proper positions. The RPM packages need to be tested as required. A workspace is required for these operations. The **rpmbuild** command uses a set of standard workspaces. - -```shell -rpmdev-setuptree -``` - -The **rpmdev-setuptree** command is used to install rpmdevtools. After the command is executed, the **rpmbuild** folder is generated in the **/root** directory \(or the **/home/**_username_ directory for non-root users\). The directory structure is as follows: - -```shell -$ tree rpmbuild -rpmbuild -├── BUILD -├── RPMS -├── SOURCES -├── SPECS -└── SRPMS -``` - -The content is described as follows: - -| Directory | Macro | Name | Function | -|--------------------|--------------|-----------------|-------------------------------------------| -|~/rpmbuild/BUILD |%_builddir |Build directory |The source code package is decompressed and compiled in a subdirectory of this directory. | -|~/rpmbuild/RPMS |%_rpmdir |Standard RPM package directory |The binary RPM package is generated and stored in this directory. | -|~/rpmbuild/SOURCES |%_sourcedir |Source code directory |The source code package (for example, .tar package) and all patches are stored in this directory. | -|~/rpmbuild/SPECS |%_specdir |Spec file directory |The RPM package configuration file (.spec) is stored in this directory. | -|~/rpmbuild/SPECS |%_srcrpmdir |Source RPM package directory |The source RPM (SRPM) package is stored in this directory. | -|~/rpmbuild/BUILDROOT|%_buildrootdir|Installation directory |Files installed during the %install phase is stored in this directory. | - -The **\~/rpmbuild/SPECS** directory contains the configuration file of the RPM package, which is the drawing of the RPM package. This file tells the **rpmbuild** command how to build the RPM package. The **Macro Code** column contains the corresponding directories in the .spec file, which is similar to the macro or global variable in the programming language. - -### Packaging Process - -The packaging process is as follows: - -1. Place the source code in **%\_sourcedir**. -2. Compile the source code in **%\_builddir**. Generally, the source code is compressed and needs to be decompressed first. -3. Install the RPM package. The installation is similar to pre-assembling the software package. Copy the contents \(such as binary files, configuration files, and man files\) that should be contained in the software package to **%\_buildrootdir** and assemble the contents based on the actual directory structure after installation. For example, if binary commands are stored in **/usr/bin**, copy the directory structure to **%\_buildrootdir**. -4. Perform necessary configurations, such as preparations before installation and cleanup after installation. These are configured in the SPEC file to tell the **rpmbuild** command how to build. -5. Check whether the software is running properly. -6. The generated RPM package is stored in **%\_rpmdir**, and the source code package is stored in **%\_srcrpmdir**. - -In the SPEC file, each phase is described as follows: - -| Phase | Directory to Be Read | Directory to Which Data Is Written | Action | -|-------------------|--------------|-----------------|-------------------------------------------| -| %prep | %_sourcedir | %_builddir | Read the source code and patch in the **%_sourcedir** directory. Then, decompress the source code to the **%_builddir** subdirectory and apply all patches. | -| %build | %_builddir | %_builddir |Compile files in the **%_builddir** build directory. Run a command similar to `./configure && make`.| -| %install | %_builddir | %_buildrootdir |Read files in the **%_builddir** build directory and install them to the **%_buildrootdir** directory. These files are generated after the RPM is installed.| -| %check | %_builddir | %_builddir | Check whether the software is running properly. Run a command similar to `make test`.| -| bin | %_buildrootdir | %_rpmdir| Read files in the **%_buildrootdir** final installation directory to create RPM packages in the **%_rpmdir** directory. In this directory, RPM packages of different architectures are stored in different subdirectories. The **noarch** directory stores RPM packages applicable to all architectures. These RPM files are the RPM packages that are finally installed by users. | -| src | %_sourcedir | %_srcrpmdir | Create the source code RPM package (SRPM for short, with the file name extension .src.rpm) and save it to the **%_srcrpmdir** directory. The SRPM package is usually used to review and upgrade software packages. | - -### Packaging Options - -Run the **rpmbuild** command to build the software package. The **rpmbuild** command can be used to build software packages by building .spec, .tar, and source files. - -The format of the **rpmbuild** command is rpmbuild \[_option_...\] - -[Table 1](#table1342946175212) describes the common rpmbuild packaging options. - -**Table 1** rpmbuild Packaging Options -| Option | Description | -|----------|--------------| -|-bp _specfile_ |Starts build from the **%prep** phase of the _specfile_ (decompress the source code package and install the patch). | -|-bc _specfile_ |Starts build from the **%build** phase of the _specfile_. | -|-bi _specfile_ |Starts build from the **%install** phase of the _specfile_. | -|-bl _specfile_ |Starts check from the **%files** phase of the _specfile_. | -|-ba _specfile_ |Uses the _specfile_ to build the source code package and binary package. | -|-bb _specfile_ |Uses the _specfile_ to build the binary package. | -|-bs _specfile_ |Uses the _specfile_ to build the source code package. | -|-rp _sourcefile_ |Starts build from the **%prep** phase of the _sourcefile_ (decompress the source code package and install the patch). | -|-rc _sourcefile_ |Starts build from the **%build** phase of the _sourcefile_. | -|-ri _sourcefile_ |Starts build from the **%install** phase of the _sourcefile_. | -|-rl _sourcefile_ |Starts build from the **%files** phase of the _sourcefile_. | -|-ra _sourcefile_ |Uses the _sourcefile_ to build the source code package and binary package. | -|-rb _sourcefile_ |Uses the _sourcefile_ to build the binary package. | -|-rs _sourcefile_ |Uses the _sourcefile_ to build the source code package. | -|-tp _tarfile_ |Starts build from the **%prep** phase of the _tarfile_ (decompress the source code package and install the patch). | -|-tc _tarfile_ |Starts build from the **%build** phase of the _tarfile_. | -|-ti _tarfile_ |Starts build from the **%install** phase of the _tarfile_. | -|-ta _tarfile_ |Uses the _tarfile_ to build the source code package and binary package. | -|-tb _tarfile_ |Uses the _tarfile_ to build the binary package. | -|-ts _tarfile_ |Uses the _tarfile_ to build the source code package. | -|--buildroot=_DIRECTORY_ |During the build, uses _DIRECTORY_ to overwrite the default **/root** directory. | -|--clean |Deletes the files in the **BUILD** directory. | -|--nobuild |No actual build steps are performed. It can be used to test the .spec file. | -|--noclean |Skips the %clean phase of the .spec file (even if it does exist). | -|--nocheck |Skips the %check phase of the .spec file (even if it does exist). | -|--dbpath _DIRECTORY_ |Uses the database in _DIRECTORY_ instead of the default directory **/var/lib/rpm**. | -|--root _DIRECTORY_ |Sets _DIRECTORY_ to the highest level. The default value is **/**, indicating the highest level. | -|--recompile _sourcefile_ |Installs the specified source code package _sourcefile_, that is, start preparation, compilation, and installation of the source code package. | -|--rebuild _sourcefile_ |Builds a new binary package based on `--recompile`. When the build is complete, the build directory, source code, and .spec file are deleted. The deletion effect is the same as that of --clean. | -|-?，--help |Displays detailed help information. | -|--version |Displays detailed version information. | - -## Building an RPM Package Locally - -This section uses an example to describe how to build an RPM software package locally. - -### Setting Up the Development Environment - -#### Prerequisites - -You have obtained the **root** permission, and have configured a repo source for openEuler. - -#### Procedure - -You can use the DNF tool to install rpmdevtools, including the **rpmbuild** command and related dependencies \(such as make and gdb\). Run the following command: - -```shell -dnf install rpmdevtools* -``` - -### Creating a Hello World RPM Package - -The following uses the packaging process of the GNU Hello World project as an example. The package contains the most common peripheral components related to the typical Free and Open Source Software \(FOSS\) project, including the configuration, compilation, and installation environments, documents, and internationalization \(i18n\) information. - -#### Obtaining the Source Code - -Run the following command to download the source code of the official example: - -```shell -rpmdev-setuptree -cd ~/rpmbuild/SOURCES -wget http://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz -``` - -#### Editing the SPEC File - -Run the following command to create the .spec file in the **~/rpmbuild/SPECS** directory: - -```shell -cd ~/rpmbuild/SPECS -vi hello.spec -``` - -Write the corresponding content to the file and save the file. The following is an example of the file content. Modify the corresponding fields based on the actual requirements. - -```text -Name: hello -Version: 2.10 -Release: 1%{?dist} -Summary: The "Hello World" program from GNU -Summary(zh_CN): GNU Hello World program -License: GPLv3+ -URL: http://ftp.gnu.org/gnu/hello -Source0: http://ftp.gnu.org/gnu/hello/%{name}-%{version}.tar.gz - -BuildRequires: gettext -Requires(post): info -Requires(preun): info - -%description -The "Hello World" program, done with all bells and whistles of a proper FOSS -project, including configuration, build, internationalization, help files, etc. - -%description -l zh_CN -The Hello World program contains all parts required by the FOSS project, including configuration, build, i18n, and help files. - -%prep -%setup -q - -%build -%configure -make %{?_smp_mflags} - -%install -make install DESTDIR=%{buildroot} -%find_lang %{name} -rm -f %{buildroot}/%{_infodir}/dir - -%post -/sbin/install-info %{_infodir}/%{name}.info %{_infodir}/dir || : - -%preun -if [ $1 = 0 ] ; then -/sbin/install-info --delete %{_infodir}/%{name}.info %{_infodir}/dir || : -fi - -%files -f %{name}.lang -%doc AUTHORS ChangeLog NEWS README THANKS TODO -%license COPYING -%{_mandir}/man1/hello.1.* -%{_infodir}/hello.info.* -%{_bindir}/hello - -%changelog -* Thu Dec 26 2019 Your Name - 2.10-1 -- Update to 2.10 -* Sat Dec 3 2016 Your Name - 2.9-1 -- Update to 2.9 -``` - -- The **Name** tag indicates the software name, the **Version** tag indicates the version number, and the **Release** tag indicates the release number. -- The **Summary** tag is a brief description. The first letter of the tag must be capitalized to prevent the rpmlint tool \(packaging check tool\) from generating alarms. -- The **License** tag describes the protocol version of the software package. The packager is responsible for checking the license status of the software, which can be implemented by checking the source code or license file or communicating with the author. -- The **Group** tag is used to classify software packages by **/usr/share/doc/rpm/GROUPS**. Currently, this tag has been discarded. However, the VIM template still has this tag. You can delete it. However, adding this tag does not affect the system. The **%changelog** tag should contain the log of changes made for each release, especially the description of the upstream security/vulnerability patches. The **%changelog** tag should contain the version string to avoid the rpmlint tool from generating alarms. -- If multiple lines are involved, such as %changelog or %description, start from the next line of the instruction and end with a blank line. -- Some unnecessary lines \(such as BuildRequires and Requires\) can be commented out with a number sign \(\#\) at the beginning of the lines. -- The default values of **%prep**, **%build**, **%install**, and **%files** are retained. - -#### Building an RPM Package - -Run the following command in the directory where the .spec file is located to build the source code, binary files, and software packages that contain debugging information: - -```shell -rpmbuild -ba hello.spec -``` - -Run the following command to view the execution result: - -```shell -$ tree ~/rpmbuild/*RPMS - -/home/testUser/rpmbuild/RPMS -└── aarch64 - ├── hello-2.10-1.aarch64.rpm - ├── hello-debuginfo-2.10-1.aarch64.rpm - └── hello-debugsource-2.10-1.aarch64.rpm -/home/testUser/rpmbuild/SRPMS -└── hello-2.10-1.src.rpm -``` - -## Building an RPM Package Using the OBS - -This section describes how to build RPM software packages using the OBS on the web page or with OSC. There are two methods: - -- Modifying an existing software package: Modify the source code of an existing software package and build the modified source code into an RPM software package. -- Adding a software package: A new software source file is developed from scratch, and the newly developed source file is used to build an RPM software package. - -### OBS Overview - -OBS is a general compilation framework based on the openSUSE distribution. It is used to build source code packages into RPM software packages or Linux images. OBS uses the automatic distributed compilation mode and supports the compilation of images and installation packages of multiple Linux OS distributions \(such as openEuler, SUSE, and Debian\) on multiple architecture platforms \(such as x86 and ARM64\). - -OBS consists of the backend and frontend. The backend implements all core functions. The frontend provides web applications and APIs for interaction with the backend. In addition, OBS provides an API command line client OSC, which is developed in an independent repository. - -OBS uses the project organization software package. Basic permission control, related repository, and build targets \(OS and architecture\) can be defined in the project. A project can contain multiple subprojects. Each subproject can be configured independently to complete a task. - -### Building an RPM Software Package Online - -This section describes how to build an RPM software package online on OBS. - -#### Building an Existing Software Package - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- If you use OBS for the first time, register an individual account on the OBS web page. ->- With this method, you must copy the modified code and commit it to the code directory before performing the following operations. The code directory is specified in the **\_service** file. - -To modify the source code of the existing software and build the modified source file into an RPM software package on the OBS web client, perform the following steps: - -1. Log in to OBS at [https://build.openeuler.openatom.cn/](https://build.openeuler.openatom.cn/). -2. Click **All Projects**. The **All Projects** page is displayed. -3. Click the project to be modified. The project details page is displayed. For example, click **openEuler:Mainline**. -4. On the project details page, search for the software package to be modified and click the software package name. The software package details page is displayed. -5. Click **Branch package**. In the displayed dialog box, click **Accept**, as shown in [Figure 1](#fig77646143214). - - **Figure 1** **Branch Confirmation** page - ![](./figures/branch-confirmation-page.png "branch-confirmation-page") - -6. Click the **\_service** file to go to the editing page, modify the file content, and click **Save**. An example of the **\_service** file content is as follows. _userCodeURL_ and _userCommitID_ indicate the user code path and commission version number or branch, respectively. - - ```xml - - - git - userCodeURL - userCommitID - - - bz2 - *.tar - - - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Click **Save** to save the **\_service** file. OBS downloads the source code from the specified URL to the software directory of the corresponding OBS project based on the **\_service** file description and replaces the original file. For example, the **kernel** directory of the **openEuler:Mainline** project in the preceding example. - -7. After the files are copied and replaced, OBS automatically starts to build the RPM software package. Wait until the build is complete and view the build status in the status bar on the right. - - **succeeded**: The build is successful. You can click **succeeded** to view the build logs, as shown in [Figure 2](#fig10319114217337). - - **Figure 2** **Succeeded** page - ![](./figures/succeeded-page.png "succeeded-page") - - - **failed**: The build failed. Click **failed** to view error logs, locate the fault, and rebuild again. - - **unresolvable**: The build is not performed. The possible cause is that the dependency is missing. - - **disabled**: The build is manually closed or is queuing for build. - - **excluded**: The build is prohibited. The possible cause is that the .spec file is missing or the compilation of the target architecture is prohibited in the .spec file. - -#### Adding a Software Package - -To add a new software package on the OBS web page, perform the following steps: - -1. Log in to the OBS console. -2. Select a project based on the dependency of the new software package. That is, click **All Projects** and select the corresponding project, for example, **openEuler:Mainline**. -3. Click a software package in the project. The software package details page is displayed. -4. Click **Branch package**. On the confirmation page that is displayed, click **Accept**. -5. Click **Delete package** to delete the software package in the new subproject, as shown in [Figure 3](#fig18306181103615). - - **Figure 3** Deleting a software package from a subproject - ![](./figures/deleting-a-software-package-from-a-subproject.png "deleting-a-software-package-from-a-subproject") - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The purpose of creating a project by using existing software is to inherit the dependency such as the environment. Therefore, you need to delete these files. - -6. Click **Create Package**. On the page that is displayed, enter the software package name, title, and description, and click **Create** to create a software package, as shown in [Figure 4](#fig6762111693811) and [Figure 5](#fig18351153518389). - - **Figure 4** **Create Package** page - ![](./figures/create-package-page.png "create-package-page") - - **Figure 5** Creating a software package - ![](./figures/creating-a-software-package.png "creating-a-software-package") - -7. Click **Add file** to upload the .spec file and the file to be compiled \(specified in the .spec file\), as shown in [Figure 6](#fig1475845284011). - - **Figure 6** **Add file** page - ![](./figures/add-file-page.png "add-file-page") - -8. After the file is uploaded, OBS automatically starts to build the RPM software package. Wait until the build is complete and view the build status in the status bar on the right. - - **succeeded**: The build is successful. You can click **succeeded** to view the build logs. - - **failed**: The build failed. Click **failed** to view error logs, locate the fault, and rebuild again. - - **unresolvable**: The build is not performed. The possible cause is that the dependency is missing. - - **disabled**: The build is manually closed or is queuing for build. - - **excluded**: The build is prohibited. The possible cause is that the .spec file is missing or the compilation of the target architecture is prohibited in the .spec file. - -#### Obtaining the Software Package - -After the RPM software package is built, perform the following operations to obtain the RPM software package on the web page: - -1. Log in to the OBS console. -2. Click **All Projects** and find the project corresponding to the required software package, for example, **openEuler:Mainline**. -3. Click the name of the required software package in the project. The software package details page is displayed, for example, the **kernel** page in the preceding example. - -4. Click the **Repositories** tab. On the software repository management page that is displayed, click **Enable** in **Publish Flag** to enable the RPM software package download function \(the status changes from ![](./figures/en-us_image_0229243704.png) to ![](./figures/en-us_image_0229243702.png)\), as shown in [Figure 7](#fig17480830144217). - - **Figure 7** **Repositories** page - ![](./figures/repositories-page.png "repositories-page") - -5. Click the project name in the **Repository** column. On the RPM software package download page that is displayed, click **Download** on the right of the RPM software package to download the RPM software package, as shown in [Figure 8](#fig12152145615438). - - **Figure 8** RPM software package download page - ![](./figures/rpm-software-package-download-page.png "rpm-software-package-download-page") - -### Building a Software Package Using OSC - -This section describes how to use the OBS command line tool OSC to create a project and build an RPM software package. - -#### Installing and Configuring the OSC - -##### Prerequisites - -You have obtained the **root** permission, and have configured a repo source for openEuler. - -##### Procedure - -1. Install the OSC command line tool and its dependency as the **root** user. - - ```shell - dnf install osc build - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The compilation of RPM software packages depends on build. - -2. Configure the OSC. - 1. Run the following command to open the **\~/.oscrc** file: - - ```shell - vi ~/.oscrc - ``` - - 2. Add the **user** and **pass** fields to **\~/.oscrc**. The values of _userName_ and _passWord_ are the account and password registered on the OBS website \([https://build.openeuler.openatom.cn/](https://build.openeuler.openatom.cn/)\). - - ```text - [general] - apiurl = https://build.openeuler.openatom.cn/ - [https://build.openeuler.openatom.cn/] - user=userName - pass=passWord - ``` - -#### Building an Existing Software Package - -**Creating a Project** - -1. You can copy an existing project to create a subproject of your own. For example, to copy the **zlib** software package in the **openEuler:Mainline** project to the new branch, run the following command: - - ```shell - osc branch openEuler:Mainline zlib - ``` - - If the following information is displayed, a new branch project **home:testUser:branches:openEuler:Mainline** is created for user **testUser**. - - ```console - A working copy of the branched package can be checked out with: - osc co home:testUser:branches:openEuler:Mainline/zlib - ``` - -2. Download the configuration file \(for example, **\_service**\) of the software package to be modified to the local directory. In the preceding command, _testUser_ indicates the account name configured in the **\~/.oscrc** configuration file. Change it based on the actual requirements. - - ```shell - osc co home:testUser:branches:openEuler:Mainline/zlib - ``` - - Information similar to the following is displayed: - - ```console - A home:testUser:branches:openEuler:Mainline - A home:testUser:branches:openEuler:Mainline/zlib - A home:testUser:branches:openEuler:Mainline/zlib/_service - ``` - -3. Go to the local subproject directory and synchronize the remote code of the software package to the local host. - - ```shell - cd home:testUser:branches:openEuler:Mainline/zlib - osc up -S - ``` - - Information similar to the following is displayed: - - ```console - A _service:tar_scm_kernel_repo:0001-Neon-Optimized-hash-chain-rebase.patch - A _service:tar_scm_kernel_repo:0002-Porting-optimized-longest_match.patch - A _service:tar_scm_kernel_repo:0003-arm64-specific-build-patch.patch - A _service:tar_scm_kernel_repo:zlib-1.2.11-optimized-s390.patch - A _service:tar_scm_kernel_repo:zlib-1.2.11.tar.xz - A _service:tar_scm_kernel_repo:zlib-1.2.5-minizip-fixuncrypt.patch - A _service:tar_scm_kernel_repo:zlib.spec - ``` - -**Building an RPM Package** - -1. Rename the source file and add the renamed source file to the temporary storage of OBS. - - ```shell - rm -f _service;for file in `ls | grep -v .osc`;do new_file=${file##*:};mv $file $new_file;done - osc addremove * - ``` - -2. Modify the source code and .spec file, and run the following command to update the file. - - ```shell - osc up - ``` - -3. Synchronize all modifications of the corresponding software package to the OBS server. The following is an example of command. The information after the **-m** parameter indicates the submmission record. - - ```shell - osc ci -m "commit log" - ``` - -4. Run the following command to obtain the repository name and architecture of the current project: - - ```shell - osc repos home:testUser:branches:openEuler:Mainline - ``` - -5. After the modification is committed, OBS automatically compiles the software package. You can run the following command to view the compilation logs of the corresponding repository. In the command, _standard\_aarch64_ and _aarch64_ indicate the repository name and architecture obtained in the command output. - - ```shell - osc buildlog standard_aarch64 aarch64 - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >You can also open the created project on the web client to view the build logs. - -#### Adding a Software Package - -To use the OSC tool of OBS to add a new software package, perform the following steps: - -**Creating a Project** - -1. Create a project based on the dependency of the new software package and a proper project. For example, to create a project based on **zlib** of the **openEuler:Mainline** project, run the following command \(**zlib** is any software package in the project\): - - ```shell - osc branch openEuler:Mainline zlib - ``` - -2. Delete unnecessary software packages added during project creation. For example, to delete the **zlib** software package, run the following command: - - ```shell - cd home:testUser:branches:openEuler:Mainline - osc rm zlib - osc commit -m "commit log" - ``` - -3. Create a software package in your own project. For example, to add the **my-first-obs-package** software package, run the following command: - - ```shell - mkdir my-first-obs-package - cd my-first-obs-package - ``` - -**Building an RPM Package** - -1. Add the prepared source file and .spec file to the software package directory. -2. Modify the source code and .spec file, and upload all files of the corresponding software package to the OBS server. The following is a command example. The information after the **-m** parameter is the commission record. - - ```shell - cd home:testUser:branches:openEuler:Mainline - osc add my-first-obs-package - osc ci -m "commit log" - ``` - -3. Run the following command to obtain the repository name and architecture of the current project: - - ```shell - osc repos home:testUser:branches:openEuler:Mainline - ``` - -4. After the modification is committed, OBS automatically compiles the software package. You can run the following command to view the compilation logs of the corresponding repository. In the command, _standard\_aarch64_ and _aarch64_ indicate the repository name and architecture obtained in the command output. - - ```shell - cd home:testUser:branches:openEuler:Mainline/my-first-obs-package - osc buildlog standard_aarch64 aarch64 - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >You can also open the created project on the web client to view the build logs. - -#### Obtaining the Software Package - -After the RPM software package is built, run the following command to obtain the RPM software package using the OSC: - -```shell -osc getbinaries home:testUser:branches:openEuler:Mainline my-first-obs-package standard_aarch64 aarch64 -``` - -The parameters in the command are described as follows. You can modify the parameters according to the actual situation. - -- _home:testUser:branches:openEuler:Mainline_: name of the project to which the software package belongs. -- _my-first-obs-package_: name of the software package. -- _standard\_aarch64_: repository name. -- _aarch64_: repository architecture name. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->You can also obtain the software package built using OSC from the web page. For details, see [Obtaining the Software Package](#obtaining-the-software-package). diff --git a/docs/en/docs/ApplicationDev/preparations-for-development-environment.md b/docs/en/docs/ApplicationDev/preparations-for-development-environment.md deleted file mode 100644 index 1bc30c69702d52652cac66eecb259ee4490c26de..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/preparations-for-development-environment.md +++ /dev/null @@ -1,391 +0,0 @@ -# Preparations for Development Environment - -## Environment Requirements - -- If physical machines (PMs) are used, the minimum hardware requirements of the development environment are described in [Table 1](#table154419352610). - - **Table 1** Minimum hardware specifications - - - - - - - - - - - - - - - - - - - - - - - - - -

Component

-

Minimum Hardware Specification

-

Description

-

Architecture

-
  • AArch64
  • x86_64
-
  • 64-bit Arm architecture
  • 64-bit Intel x86 architecture
-

CPU

-
  • Huawei Kunpeng 920 series
  • Intel ® Xeon® processor
-

-

-

Memory

-

≥ 4 GB (8 GB or higher recommended for better user experience)

-

-

-

Hard disk

-

≥ 120 GB (for better user experience)

-

IDE, SATA, SAS interfaces are supported.

-
- -- If virtual machines (VMs) are used, the minimum virtualization space required for the development environment is described in [Table 2](#table780410493819). - - **Table 2** Minimum virtualization space - - - - - - - - - - - - - - - - - - - - - - - - - -

Component

-

Minimum Virtualization Space

-

Description

-

Architecture

-
  • AArch64
  • x86_64
-

-

-

CPU

-

Two CPUs

-

-

-

Memory

-

≥ 4 GB (8 GB or higher recommended for better user experience)

-

-

-

Hard disk

-

≥ 32 GB (120 GB or higher recommended for better user experience)

-

-

-
- -### OS Requirements - -The openEuler OS is required. - -For details about how to install the openEuler OS, see the _Installation Guide_. On the **SOFTWARE SELECTION** page, select **Development Tools** in the **Add-Ons for Selected Environment** area. - -## Configuring the openEuler Yum Source - -Configure an online Yum source using the online openEuler repo source. Alternatively, configure a local Yum source by mounting an ISO file and creating a local openEuler repo source. - -### Configuring an Online Yum Source by Obtaining the Online openEuler Repo Source - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> openEuler provides multiple repo sources for users online. For details about the repo sources, see [Installing the OS](../Releasenotes/installing-the-os.md). This section uses the OS repo source file of the AArch64 architecture as an example. - -1. Go to the yum source directory and check the .repo configuration file. - - ```shell - $ cd /etc/yum.repos.d - $ ls - openEuler-xxx.repo - ``` - -2. Edit the **openEuler-xxx.repo** file as the **root** user. Configure the online openEuler repo source as the yum source. - - ```shell - vi openEuler-xxx.repo - ``` - - Edit the **openEuler-xxx.repo** file as follows: - - ```text - [osrepo] - name=osrepo - baseurl=http://repo.openeuler.org/openEuler-{version}/OS/{arch}/ - enabled=1 - gpgcheck=1 - gpgkey=http://repo.openeuler.org/openEuler-{version}/OS/{arch}/RPM-GPG-KEY-openEuler - ``` - - > ![](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - **repoid** indicates the ID of the software repository. Repoids in all .repo configuration files must be unique. In the example, **repoid** is set to **base**. - > - **name** indicates the string that the software repository describes. - > - **baseurl** indicates the address of the software repository. - > - **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled. - > - **gpgcheck** indicates whether to enable the GNU privacy guard (GPG) to check the validity and security of sources of RPM packages. **1** indicates GPG check is enabled. **0** indicates the GPG check is disabled. If this option is not specified, the GPG check is enabled by default. - > - **gpgkey** indicates the public key used to verify the signature. - -### Configuring a Local Yum Source by Mounting an ISO File - -> ![](./public_sys-resources/icon-note.gif) ********NOTE:******** -> openEuler provides multiple ISO release packages. For details about each ISO release package, see [OS Installation](../Releasenotes/installing-the-os.md). This section does not specify the version and architecture of related files. Choose them based on the actual requirements. - -1. Download the ISO release package. - - Download an ISO image using a cross-platform file transfer tool. - 1. Visit the [openEuler community](https://www.openeuler.org/en/). - 2. Choose **Downloads** > **Community Editions**. - 3. Locate the target version. Then, click **Download**. The download list is displayed. - 4. The download list includes the following architectures: - - **x86\_64**: ISO of the x86\_64 architecture. - - **AArch64**: ISO of the AArch64 architecture. - - **ARM32**: ISO for embedded devices. - - 5. Click **AArch64** and **Server**. - 6. Choose the required ISO type and click **Download** to download the openEuler release package to the local host. - 7. Click **SHA256** to copy the checksum. Save the checksum as a local verification file. - 8. Log in to the openEuler OS and create a directory for storing the release package and verification file, for example, **~/iso**. - - ```shell - mkdir ~/iso - ``` - - 9. Use a cross-platform file transfer tool (such as WinSCP) to upload the local openEuler release package and verification file to the openEuler OS. - - - Run the **wget** command to download the ISO image. - 1. Visit the [openEuler community](https://www.openeuler.org/en/). - 2. Choose **Downloads** > **Community Editions**. - 3. Locate the target version. Then, click **Download**. The download list is displayed. - 4. The download list includes the following architectures: - - **x86\_64**: ISO of the x86\_64 architecture. - - **AArch64**: ISO of the AArch64 architecture. - - **ARM32**: ISO for embedded devices. - - 5. Click **AArch64**. - 6. Click **Server**. - 7. Choose the required ISO type, right-click **Download**, and copy the link address. - 8. Right-click **SHA256** and copy the link address. - 9. Log in to the openEuler OS, create a directory for storing the release package and verification file, for example, **~/iso**. Then switch to the directory. - - ```shell - mkdir ~/iso - cd ~/iso - ``` - - 10. Run the **wget** command to remotely download the release package and verification file. In the command, replace **ipaddriso** with the address copied in steps 7. - - ```shell - wget ipaddriso - ``` - -2. Release Package Integrity Check - - 1. Calculate the SHA256 verification value of the openEuler release package. - - ```shell - sha256sum openEuler-xxx-dvd.iso - ``` - - After the command is run, the verification value is displayed. - - 2. Check whether the calculated value is the same as that of the saved SHA256 value. - - If the verification values are consistent, the .iso file is not damaged. If they are inconsistent, the file is damaged and you need to obtain the file again. - -3. Mount the ISO file and configure it as a repo source. - - ```shell - mount /home/iso/openEuler-xxx-dvd.iso /mnt/ - ``` - - ```text - . - │── boot.catalog - │── docs - │── EFI - │── images - │── Packages - │── repodata - │── TRANS.TBL - └── RPM-GPG-KEY-openEuler - ``` - - In the preceding directory, **Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repo source metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler. - -4. Go to the yum source directory and check the .repo configuration file in the directory. - - ```shell - $ cd /etc/yum.repos.d - $ ls - openEuler-xxx.repo - ``` - -5. Edit the **openEuler-xxx.repo** file as the **root** user. Configure the local openEuler repo source created in step [3](#li6236932222) as the yum source. - - ```shell - vi openEuler-xxx.repo - ``` - - Edit the **openEuler-xxx.repo** file as follows: - - ```text - [localosrepo] - name=localosrepo - baseurl=file:///mnt - enabled=1 - gpgcheck=1 - gpgkey=file:///mnt/RPM-GPG-KEY-openEuler - ``` - -## Installing the Software Packages - -The software required varies in different development environments. However, the installation methods are the same. This section describes how to install common software packages (such as JDK and rpm-build). Some development software, such as GCC and GNU make, is provided by the openEuler OS by default. - -### Installing the JDK Software Package - -1. Run the `dnf list installed | grep jdk` command to check whether the JDK software is installed. - - Check the command output. If the command output contains "jdk", the JDK has been installed. If no such information is displayed, the software is not installed. - -2. Run `dnf clean all` to clear the cache. - -3. Run `dnf makecache` to create the cache. - -4. Run `dnf search jdk | grep jdk` to query the JDK software package that can be installed. - - View the command output and install the **java-{version}-openjdk-devel.aarch64** software package. - -5. Install the JDK software package as the **root** user. Run `dnf install java-{version}-openjdk-devel.aarch64`. - -6. Query information about the JDK software by running `java -version`. - - Check the command output. If the command output contains "openjdk version", the JDK has been correctly installed. - -### Installing the rpm-build Software Package - -1. Run the `dnf list installed | grep rpm-build` command to check whether the rpm-build software is installed. - - Check the command output. If the command output contains "rpm-build", the software has been installed. If no such information is displayed, the software is not installed. - -2. Run `dnf clean all` to clear the cache. - -3. Run `dnf makecache` to create the cache. - -4. Install the rpm-build package as the **root** user. Run `dnf install rpm-build`. - -5. Query the rpm-build software version. Run `rpmbuild --version`. - -## Using the IDE for Java Development - -For small-sized Java applications, you can directly use JDK to compile them to run Java applications. However, for medium- and large-sized Java applications, this method does not meet the development requirements. You can perform the following steps to install and use the IDE to facilitate Java development on the openEuler OS. - -### Overview - -IntelliJ IDEA is a popular Java IDE. You can download the community edition of IntelliJ IDEA for free. Currently, openEuler supports Java development using IntelliJ IDEA, improving the work efficiency of developers. - -### Logging In to the Server Using MobaXterm - -MobaXterm is an excellent SSH client. It has an X Server and can easily solve remote GUI display problems. - -You need to download, install, and start MobaXterm in advance, and then log in to your server in SSH mode to perform the following operations. - -### Setting the JDK Environment - -Before setting JAVA\_HOME, you need to find the installation path of the JDK. You are supported to have installed the JDK. If you have not installed the JDK, install it by referring to the preceding section "Installing the JDK Software Package". - -Run the following commands to view the Java path: - -```shell -$ which java -/usr/bin/java -``` - -Run the following commands to check the directory to which the soft link points: - -```shell -$ ls -la /usr/bin/java -lrwxrwxrwx. 1 root root 22 Mar 6 20:28 /usr/bin/java -> /etc/alternatives/java -$ ls -la /etc/alternatives/java -lrwxrwxrwx. 1 root root 83 Mar 6 20:28 /etc/alternatives/java -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-1.h2.aarch64/jre/bin/java -``` - -**/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-1.h2.aarch64** indicates the actual JDK path. Run the following commands to set **JAVA\_HOME** and **PATH**: - -```shell -export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-1.h2.aarch64 -export PATH=$JAVA_HOME/bin:$PATH -``` - -### Downloading and Installing the GTK Library - -Run the following command: - -```shell -dnf list installed | grep gtk -``` - -If **gtk2** or **gtk3** is displayed, the GTK library has been installed. In this case, skip this step. Otherwise, run the following command as the **root** user to automatically download and install the GTK library: - -```shell -dnf -y install gtk2 libXtst libXrender xauth -``` - -### Setting X11 Forwarding - -Switch to the SSHD configuration directory. - -```shell -cd ~/.ssh -``` - -If the directory does not exist, run the following command to create it and then switch to it: - -```shell -mkdir ~/.ssh -``` - -Edit the configuration file in the **.ssh** directory and save the file. - -1. Run the **vim** command to open the configuration file. - - ```shell - vim config - ``` - -2. Add the following content to the end of the file and save the file: - - ```text - Host * - ForwardAgent yes - ForwardX11 yes - ``` - -### Downloading and Running IntelliJ IDEA - -After the preceding environment configuration is complete, you can download and run IntelliJ IDEA. The latest version of IntelliJ IDEA is incompatible with openEuler in some functions. You are advised to click [here](https://www.jetbrains.com/idea/download/other.html) to download the Linux package of the 2018 version. Move the downloaded package to the directory where you want to install the software and decompress the package. - -```shell -tar xf ideaIC-2018.3.tar.gz -``` - -After the decompression is complete, switch to the IntelliJ IDEA directory and run IntelliJ IDEA. - -```shell -cd ./idea-IC-183.4284.148 -bin/idea.sh & -``` diff --git a/docs/en/docs/ApplicationDev/using-clang-for-compilation.md b/docs/en/docs/ApplicationDev/using-clang-for-compilation.md deleted file mode 100644 index e1c79c9f6bd3616d2f351192d63b457279d742fc..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/using-clang-for-compilation.md +++ /dev/null @@ -1,100 +0,0 @@ -# Using LLVM/Clang for Compilation - -This chapter describes the basic knowledge of LLVM/Clang compilation and provides examples for demonstration. For more information about how to use Clang, run the **clang --help** command. - -- [Using LLVM/Clang for Compilation](#using-llvmclang-for-compilation) - - [Overview](#overview) - - [LLVM/Clang Installation](#llvmclang-installation) - - [Coexistence of Multiple Versions](#coexistence-of-multiple-versions) - - [Example](#example) - -## Overview - -LLVM is a compiler that covers multiple programming languages and target processors. It uses Clang as the compiler and driver of C and C++. Clang can not only compile C and C++ programs into the LLVM intermediate representations (IRs), but also invoke all LLVM optimization passes for code generation until the final binary file is generated. - -## LLVM/Clang Installation - -Install the Clang and LLVM software packages using the Yum source in the openEuler OS. llvm12 is installed by default. - -```shell -yum install llvm -yum install clang -yum install lld // The openEuler Yum source does not contain LLD12. -``` - -Check whether the installation is successful. - -```shell -clang -v -``` - -If the command output contains the Clang version information, the installation is successful. - -## Coexistence of Multiple Versions - -Configure the openEuler LLVM/Clang multi-version support as follows: - -```text -Yum package name: -llvm{-*} -clang{-*} -lld{-*} -bolt{-*} -Example: -clang15 -llvm15-devel - -Installation path: -/usr/lib64/llvm -Example: -/usr/lib64/llvm15 - -Executable files with the - suffix are installed in the /usr/bin directory. -Example: -/usr/bin/clang-15 -/usr/bin/lld-15 -``` - -Currently, the following LLVM/Clang versions are supported: - -```shell -llvm //By default, llvm12 is installed. -llvm-15 -``` - -Install other versions using Yum. - -```shell -yum install llvm15 -yum install clang15 -yum install lld15 -``` - -Check whether the installation is successful. - -```shell -clang-15 -v -``` - -## Example - -Compile and run C and C++ programs. - -```shell -clang [command line flags] hello.c -o hello.o -./hello.o -``` - -```shell -clang++ [command line flags] hello.cpp -o hello.o -./hello.o -``` - -Specify the LLD of LLVM as the linker. If it is not specified, the default LLD is used. - -```shell -clang [command line flags] -fuse-ld=lld hello.c -o hello.o -./hello.o -``` - -For more information, see the [LLVM User Guides](https://llvm.org/docs/UserGuides.html). diff --git a/docs/en/docs/ApplicationDev/using-gcc-for-compilation.md b/docs/en/docs/ApplicationDev/using-gcc-for-compilation.md deleted file mode 100644 index 84997d895bba8099d2bc4a18020fbc4f2b37ed07..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/using-gcc-for-compilation.md +++ /dev/null @@ -1,612 +0,0 @@ -# Using GCC for Compilation - -This chapter describes the basic knowledge of GCC compilation and provides examples for demonstration. For more information about GCC, run the **man gcc** command. - - -- [Using GCC for Compilation](#using-gcc-for-compilation) - - [Overview](#overview) - - [Basics](#basics) - - [File Type](#file-type) - - [Compilation Process](#compilation-process) - - [Compilation Options](#compilation-options) - - [Multi-file Compilation](#multi-file-compilation) - - [Libraries](#libraries) - - [Dynamic Link Library](#dynamic-link-library) - - [Static Link Library](#static-link-library) - - [Examples](#examples) - - [Example for Using GCC to Compile C Programs](#example-for-using-gcc-to-compile-c-programs) - - [Example for Creating and Using a DLL Using GCC](#example-for-creating-and-using-a-dll-using-gcc) - - [Example for Creating and Using an SLL Using GCC](#example-for-creating-and-using-an-sll-using-gcc) - - - -## Overview - -The GNU Compiler Collection \(GCC\) is a powerful and high-performance multi-platform compiler developed by GNU. The GCC compiler can compile and link source programs, assemblers, and target programs of C and C++ into executable files. By default, the GCC software package is installed in the openEuler OS. - -## Basics - -### File Type - -For any given input file, the file type determines which compilation to perform. [Table 1](#table634145764320) describes the common GCC file types. - -**Table 1** Common GCC file types - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Extension (Suffix)

-

Description

-

.c

-

C source code file.

-

.C, .cc, or .cxx

-

C++ source code file.

-

.m

-

Objective-C source code file.

-

.s

-

Assembly language source code file.

-

.i

-

Preprocessed C source code file.

-

.ii

-

Preprocessed C++ source code file.

-

.S

-

Pre-processed assembly language source code file.

-

.h

-

Header file contained in the program.

-

.o

-

Target file after compilation.

-

.so

-

Dynamic link library, which is a special target file.

-

.a

-

Static link library.

-

.out

-

Executable files, which do not have a fixed suffix. The system distinguishes executable files from inexecutable files based on file attributes. If the name of an executable file is not given, GCC generates a file named a.out.

-
- -### Compilation Process - -Using GCC to generate executable files from source code files requires preprocessing, compilation, assembly, and linking. - -1. Preprocessing: Preprocess the source program \(such as a **.c** file\) to generate an **.i** file. -2. Compilation: Compile the preprocessed **.i** file into an assembly language to generate an **.s** file. -3. Assemble: Assemble the assembly language file to generate the target file **.o**. -4. Linking: Link the **.o** files of each module to generate an executable program file. - -The **.i**, **.s**, and **.o** files are intermediate or temporary files. If the GCC is used to compile programs in C language at a time, these files will be deleted. - -### Compilation Options - -GCC compilation command format: **gcc** \[_options_\] \[_filenames_\] - -In the preceding information: - -_options_ : compilation options. - -_filenames_ : file name. - -GCC is a powerful compiler. It has many _options_, but most of them are not commonly used. [Table 2](#table1342946175212) describes the common _options_. - -**Table 2** Common GCC compilation options - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

options Value

-

Description

-

Example

-

-c

-

Compiles and assembles specified source files to generate target files without linking them. It is usually used to compile subprogram files.

-

# Use the -c option to compile the source files test1.c and test2.c.

-

gcc -c test1.c test2.c

-

-S

-

Compiles the specified source file to generate an assembly language file with the .s suffix but without assembling it.

-

# Use the compiler to preprocess circle.c, translate it into assembly language, and store the result in circle.s.

-

gcc -S circle.c

-

-E

-

Preprocesses specified source files without compiling them.

-

By default, the output of the preprocessor is imported to a standard output stream, such as a display. You can use the -o option to import it to an output file.

-

# Export the preprocessing result to the circle.i file.

-

gcc -E circle.c -o circle.i

-

-o file

-

Generates a specified output file when an executable file is generated. The name must be different from that of the source file. If this option is not given, GCC generates the preset executable file a.out.

-

# Use the source file as the input file and the executable file as the output file. That is, compile the entire program.

-

gcc main.c func.c -o app.out

-

-g

-

Contains standard debugging information in executable programs.

-

-

-

-L library_path

-

Adds the library_path to the library file search path list.

-

-

-

-I library

-

Searches for the specified function library during linking.

-

When GCC is used to compile and link programs, GCC links libc.a or libc.so by default. However, other libraries (such as non-standard libraries and third-party libraries) need to be manually added.

-

# Use the -l option to link the math library.

-

gcc main.c -o main.out -lm

-
NOTE:

The file name of the math library is libm.a. The prefix lib and suffix .a are standard, and m is the basic name. GCC automatically adds these prefixes and suffixes to the basic name following the -l option. In this example, the basic name is m.

-
-

-I head_path

-

Adds the head_path to the search path list of the header file.

-

-

-

-static

-

Performs static compilation and links static libraries. Do not link dynamic libraries.

-

-

-

-shared

-

Default option, which can be omitted.

-
  • A dynamic library file can be generated.
  • During dynamic compilation, the dynamic library is preferentially linked. The static library with the same name is linked only when there is no dynamic library.
-

-

-

-fPIC (or -fpic)

-

Generates location-independent target code that uses a relative address. Generally, the -static option is used to generate a dynamic library file from the PIC target file.

-

-

-
- -### Multi-file Compilation - -There are two methods provided for compiling multiple source files. - -- Multiple source files are compiled at the same time. All files need to be recompiled during compilation. - - Example: Compile **test1.c** and **test2.c** and link them to the executable file **test**. - - ```shell - gcc test1.c test2.c -o test - ``` - -- Compile each source file, and then link the target files generated after compilation. During compilation, only modified files need to be recompiled. - - For example, compile **test1.c** and **test2.c**, and link the target files **test1.o** and **test2.o** to the executable file **test**. - - ```shell - gcc -c test1.c - gcc -c test2.c - gcc test1.o test2.o -o test - ``` - -## Libraries - -A library is mature and reusable code that has been written for use. Each program depends on many basic underlying libraries. - -The library file name is prefixed with lib and suffixed with .so \(dynamic library\) or .a \(static library\). The middle part is the user-defined library file name, for example, libfoo.so or libfoo.a. Because all library files comply with the same specifications, the **lib** prefix can be omitted when the **-l** option specifies the name of the linked library file. That is, when GCC processes **-lfoo**, the library file **libfoo.so** or **libfoo.a** is automatically linked. When creating a library, you must specify the full file name **libfoo.so** or **libfoo.a**. - -Libraries are classified into static libraries and dynamic libraries based on the linking time. The static library links and packs the target file .o generated by assembly and the referenced library into an executable file in the linking phase. The dynamic library is not linked to the target code when the program is compiled, but is loaded when the program is run. The differences are as follows: - -- The resource usage is different. - - The static library is a part of the generated executable file, while the dynamic library is a separate file. Therefore, the sizes and occupied disk space of the executable files of the static library and dynamic library are different, which leads to different resource usage. - -- The scalability and compatibility are different. - - If the implementation of a function in the static library changes, the executable file must be recompiled. For the executable file generated by dynamic linking, only the dynamic library needs to be updated, and the executable file does not need to be recompiled. - -- The dependency is different. - - The executable file of the static library can run without depending on any other contents, while the executable file of the dynamic library must depend on the dynamic library. Therefore, the static library is convenient to migrate. - -- The loading speeds are different. - - Static libraries are linked together with executable files, while dynamic libraries are linked only when they are loaded or run. Therefore, for the same program, static linking is faster than dynamic linking. - -### Dynamic Link Library - -You can use the **-shared** and **-fPIC** options to create a dynamic link library \(DLL\) with the source file, assembly file, or target file. The **-fPIC** option is used in the compilation phase. This option is used when the target file is generated, so as to generate location-independent code. - -Example 1: Generate a DLL from the source file. - -```shell -gcc -fPIC -shared test.c -o libtest.so -``` - -Example 2: Generate a DLL from the target file. - -```shell -gcc -fPIC -c test.c -o test.o -gcc -shared test.o -o libtest.so -``` - -To link a DLL to an executable file, you need to list the name of the DLL in the command line. - -Example: Compile **main.c** and **libtest.so** into **app.out**. When **app.out** is running, the link library **libtest.so** is dynamically loaded. - -```shell -gcc main.c libtest.so -o app.out -``` - -In this mode, the **libtest.so** file in the current directory is used. - -If you choose to search for a DLL, to ensure that the DLL can be linked when the program is running, you must implement by using one of the following methods: - -- Save the DLL to a standard directory, for example, **/usr/lib**. -- Add the DLL path **libraryDIR** to the environment variable **LD\_LIBRARY\_PATH**. - - ```shell - export LD_LIBRARY_PATH=libraryDIR:$LD_LIBRARY_PATH - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >**LD\_LIBRARY\_PATH** is an environment variable of the DLL. If the DLL is not in the default directories \(**/lib** and **/usr/lib**\), you need to specify the environment variable **LD\_LIBRARY\_PATH**. - -- Add the DLL path **libraryDIR** to **/etc/ld.so.conf** and run **ldconfig**, or use the DLL path **libraryDIR** as a parameter to run **ldconfig**. - -```shell -gcc main.c -L libraryDIR -ltest -o app.out -export LD_LIBRARY_PATH=libraryDIR:$LD_LIBRARY_PATH -``` - -### Static Link Library - -To create a static link library \(SLL\), you need to compile the source file to the target file, and then run the **ar** command to compress the target file into an SLL. - -Example: Compile and compress source files **test1.c**, **test2.c**, and **test3.c** into an SLL. - -```shell -gcc -c test1.c test2.c test3.c -ar rcs libtest.a test1.o test2.o test3.o -``` - -The **ar** command is a backup compression command. You can compress multiple files into a backup file \(also called an archive file\) or extract member files from the backup file. The most common use of **ar** is to compress the target files into an SLL. - -The format of the **ar** command to compress the target files into an SLL is as follows: - -ar rcs _Sllfilename_ _Targetfilelist_ - -- _Sllfilename_ : Name of the static library file. -- _Targetfilelist_ : Target file list. -- **r**: replaces the existing target file in the library or adds a new target file. -- **c**: creates a library regardless of whether the library exists. -- **s**: creates the index of the target file. The speed can be improved when a large library is created. - -Example: Create a main.c file to use the SLL. - -```shell -gcc main.c -L libraryDIR -ltest -o test.out -``` - -In the preceding command, **libraryDIR** indicates the path of the libtest.a library. - -## Examples - -### Example for Using GCC to Compile C Programs - -1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. The command is as follows: - - ```shell - cd ~/code - ``` - -2. Compile the Hello World program and save it as **helloworld.c**. The following uses the Hello World program as an example. The command is as follows: - - ```shell - vi helloworld.c - ``` - - Code example: - - ```c - #include - int main() - { - printf("Hello World!\n"); - return 0; - } - ``` - -3. Run the following command to compile the code in the code directory: - - ```shell - gcc helloworld.c -o helloworld - ``` - - If no error is reported, the execution is successful. - -4. After the compilation is complete, the helloworld file is generated. Check the compilation result. The following is an example: - - ```shell - $ ./helloworld - Hello World! - ``` - -### Example for Creating and Using a DLL Using GCC - -1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. Create the **src**, **lib**, and **include** subdirectories in the directory to store the source file, DLL file, and header file, respectively. - - ```shell - cd ~/code - mkdir src lib include - ``` - -2. Run the **cd** command to go to the **~/code/src** directory and create two functions **add.c** and **sub.c** to implement addition and subtraction, respectively. - - ```shell - cd ~/code/src - vi add.c - vi sub.c - ``` - - The following is an example of the **add.c** code: - - ```c - #include "math.h" - int add(int a, int b) - { - return a+b; - } - ``` - - The following is an example of the **sub.c** code: - - ```c - #include "math.h" - int sub(int a, int b) - { - return a-b; - } - ``` - -3. Compile the source files add.c and sub.c into the DLL libmath.so, and store the DLL in the **~/code/lib** directory. - - ```shell - gcc -fPIC -shared add.c sub.c -o ~/code/lib/libmath.so - ``` - -4. Go to the **~/code/include** directory, create a header file **math.h**, and declare the header file of the function. - - ```shell - cd ~/code/include - vi math.h - ``` - - The following is an example of the **math.h** code: - - ```c - #ifndef __MATH_H_ - #define __MATH_H_ - int add(int a, int b); - int sub(int a, int b); - #endif - ``` - -5. Run the **cd** command to go to the **~/code/src** directory and create a **main.c** function that invokes add\(\) and sub\(\). - - ```shell - cd ~/code/src - vi main.c - ``` - - The following is an example of the **math.c** code: - - ```c - #include - #include "math.h" - int main() - { - int a, b; - printf("Please input a and b:\n"); - scanf("%d %d", &a, &b); - printf("The add: %d\n", add(a,b)); - printf("The sub: %d\n", sub(a,b)); - return 0; - } - ``` - -6. Compile **main.c** and **libmath.so** into **math.out**. - - ```shell - gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out - ``` - -7. Add the path of the DLL to the environment variable. - - ```shell - export LD_LIBRARY_PATH=~/code/lib:$LD_LIBRARY_PATH - ``` - -8. Run the following command to execute **math.out**: - - ```shell - ./math.out - ``` - - The command output is as follows: - - ```console - Please input a and b: - 9 2 - The add: 11 - The sub: 7 - ``` - -### Example for Creating and Using an SLL Using GCC - -1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. Create the **src**, **lib**, and **include** subdirectories in the directory to store the source file, SLL file, and header file respectively. - - ```shell - cd ~/code - mkdir src lib include - ``` - -2. Run the **cd** command to go to the **~/code/src** directory and create two functions **add.c** and **sub.c** to implement addition and subtraction, respectively. - - ```shell - cd ~/code/src - vi add.c - vi sub.c - ``` - - The following is an example of the **add.c** code: - - ```c - #include "math.h" - int add(int a, int b) - { - return a+b; - } - ``` - - The following is an example of the **sub.c** code: - - ```c - #include "math.h" - int sub(int a, int b) - { - return a-b; - } - ``` - -3. Compile the source files **add.c** and **sub.c** into the target files **add.o** and **sub.o**. - - ```shell - gcc -c add.c sub.c - ``` - -4. Run the **ar** command to compress the **add.o** and **sub.o** target files into the SLL **libmath.a** and save the SLL to the **~/code/lib** directory. - - ```shell - ar rcs ~/code/lib/libmath.a add.o sub.o - ``` - -5. Go to the **~/code/include** directory, create a header file **math.h**, and declare the header file of the function. - - ```shell - cd ~/code/include - vi math.h - ``` - - The following is an example of the **math.h** code: - - ```c - #ifndef __MATH_H_ - #define __MATH_H_ - int add(int a, int b); - int sub(int a, int b); - #endif - ``` - -6. Run the **cd** command to go to the **~/code/src** directory and create a **main.c** function that invokes add\(\) and sub\(\). - - ```shell - cd ~/code/src - vi main.c - ``` - - The following is an example of the **math.c** code: - - ```c - #include - #include "math.h" - int main() - { - int a, b; - printf("Please input a and b:\n"); - scanf("%d %d", &a, &b); - printf("The add: %d\n", add(a,b)); - printf("The sub: %d\n", sub(a,b)); - return 0; - } - ``` - -7. Compile **main.c** and **libmath.a** into **math.out**. - - ```shell - gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out - ``` - -8. Run the following command to execute **math.out**: - - ```shell - ./math.out - ``` - - The command output is as follows: - - ```console - Please input a and b: - 9 2 - The add: 11 - The sub: 7 - ``` diff --git a/docs/en/docs/ApplicationDev/using-jdk-for-compilation.md b/docs/en/docs/ApplicationDev/using-jdk-for-compilation.md deleted file mode 100644 index 4946b917d12b6bf483f3507c1a327deaf3a04de0..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/using-jdk-for-compilation.md +++ /dev/null @@ -1,531 +0,0 @@ -# Using JDK for Compilation - - -- [Using JDK for Compilation](#using-jdk-for-compilation) - - [Overview](#overview) - - [Basics](#basics) - - [File Type and Tool](#file-type-and-tool) - - [Java Program Generation Process](#java-program-generation-process) - - [Common JDK Options](#common-jdk-options) - - [Class Library](#class-library) - - [Package Declaration](#package-declaration) - - [Package Reference](#package-reference) - - [Examples](#examples) - - [Compiling a Java Program Without a Package](#compiling-a-java-program-without-a-package) - - [Compiling a Java Program with a Package](#compiling-a-java-program-with-a-package) - - - -## Overview - -A Java Development Kit \(JDK\) is a software package required for Java development. It contains the Java Runtime Environment \(JRE\) and compilation and commissioning tools. On the basis of OpenJDK, openEuler optimizes GC, enhances concurrency stability, and enhances security, improving the performance and stability of Java applications on ARM. - -## Basics - - - -### File Type and Tool - -For any given input file, the file type determines which tool to use for processing. The common file types and tools are described in [Table 1](#table634145764320) and [Table 2](#table103504146433). - -**Table 1** Common JDK file types - - - - - - - - - - - - - - - - -

Extension (Suffix)

-

Description

-

.java

-

Java source code file.

-

.class

-

Java bytecode file, which is intermediate code irrelevant to any specific machine or OS environment. It is a binary file, which is the target code file generated after the Java source file is compiled by the Java compiler.

-

.jar

-

JAR package of Java files.

-
- -**Table 2** Common JDK tools - - - - - - - - - - - - - - - - -

Name

-

Description

-

java

-

Java running tool, which is used to run .class bytecode files or .jar files.

-

javac

-

Compiles Java source code files into .class bytecode files.

-

jar

-

Creates and manages JAR files.

-
- -### Java Program Generation Process - -To generate a program from Java source code files and run the program using Java, compilation and run are required. - -1. Compilation: Use the Java compiler \(javac\) to compile Java source code files \(.java files\) into .class bytecode files. -2. Run: Execute the bytecode files on the Java virtual machine \(JVM\). - -### Common JDK Options - -#### Javac Compilation Options - -The command format for javac compilation is as follows: **javac** \[_options_\] \[_sourcefiles_\] \[_classes_\] \[@_argfiles_\] - -In the preceding information: - -_options_: command options. - -_sourcefiles_: one or more source files to be compiled. - -_classes_: one or more classes to be processed as comments. - -@_argfiles_: one or more files that list options and source files. The **-J** option is not allowed in these files. - -Javac is a Java compiler. It has many _options_, but most of them are not commonly used. [Table 3](#table1342946175212) describes the common options values. - -**Table 3** Common javac options - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

options Value

-

Description

-

Example

-

-d path

-

Path for storing the generated class files.

-

By default, the class files generated after compilation are in the same path as the source file. You can use the -d option to export the class files to the specified path.

-

# Use the -d option to export all class files to the bin directory.

-

javac /src/*.java -d /bin

-

-s path

-

Path for storing the generated source files.

-

-

-

-cp path or -classpath path

-

Searches for the class files required for compilation and specifies the location of the class files.

-

# In the Demo, the getLine() method in the GetStringDemo class needs to be invoked. The .class file compiled by the GetStringDemo class is stored in the bin directory.

-

javac -cp bin Demo.java -d bin

-

-verbose

-

Outputs information about the operations being performed by the compiler, such as loaded class information and compiled source file information.

-

# Display information about the operations that are being performed by the compiler.

-

javac -verbose -cp bin Demo.java

-

-source sourceversion

-

Specifies the location of the input source files to be searched for.

-

-

-

-sourcepath path

-

Searches for source files (Java files) required for compilation and specifies the location of the source files to be searched for, for example, JAR, ZIP, or other directories that contain Java files.

-

-

-

-target targetversion

-

Generates class files of a specific JVM version. The value can be 1.1, 1.2, 1.3, 1.4, 1.5 (or 5), 1.6 (or 6), 1.7 (or 7), or 1.8 (or 8). The default value of targetversion is related to sourceversion of the -source option. The options of sourceversion are as follows:

-
  • 1.2, corresponding to target version 1.4
  • 1.3, corresponding to target version 1.4
  • 1.5, 1.6, 1.7, and unspecified, corresponding to target version 1.8
  • For other values, the values of targetversion and sourceversion are the same.
-

-

-
- -#### Java Running Options - -The Java running format is as follows: - -Running class file: **java** \[_options_\] _classesname_ \[args\] - -Running Java file: **java** \[_options_\] -jar _filename_ \[args\] - -In the preceding information: - -_options_: command options, which are separated by spaces. - -_classname_: name of the running .class file. - -_filename_: name of the running .jar file. - -args: parameters transferred to the main\(\) function. The parameters are separated by spaces. - -Java is a tool for running Java applications. It has many _options_, but most of them are not commonly used. [Table 4](#table371918587238) describes the common options. - -**Table 4** Common Java running options - - - - - - - - - - - - - - - - -

options Value

-

Description

-

Example

-

-cp path or -classpath path

-

Specifies the location of the file to be run and the class path to be used, including the .jar, .zip, and class file directories.

-

If there are multiple paths, separate them with colons (:).

-

-

-

-verbose

-

Outputs information about the operations being performed by the compiler, such as loaded class information and compiled source file information.

-

# Display information about the operations that are being performed by the compiler.

-

java -verbose -cp bin Demo.java

-
- -#### JAR Options - -The JAR command format is as follows: **jar** \{c | t | x | u\}\[vfm0M\] \[_jarfile_\] \[_manifest_\] \[-C _dir_\] _file_... - -[Table 5](#table3691718114817) describes the parameters in the **jar** command. - -**Table 5** JAR parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Example

-

c

-

Creates a JAR package.

-

# Compress the hello.class files in the current directory into Hello.jar. The compression process is not displayed. If the Hello.jar files do not exist, create them. Otherwise, clear the directory.

-

jar cf Hello.jar hello.class

-

t

-

Lists the contents of a JAR package.

-

# List the files contained in Hello.jar.

-

jar tf Hello.jar

-

x

-

Decompresses a JAR package.

-

# Decompress Hello.jar to the current directory. No information is displayed.

-

jar xf Hello.jar

-

u

-

Updates the existing JAR package, for example, add files to the JAR package.

-

-

-

v

-

Generates a detailed report and prints it to the standard output.

-

# Compress the hello.class files in the current directory into Hello.jar and display the compression process. If the Hello.jar files do not exist, create them. Otherwise, clear the directory.

-

jar cvf Hello.jar hello.class

-

f

-

Specifies the name of a JAR package. This parameter is mandatory.

-

-

-

m

-

Specifies the manifest file to be contained.

-

-

-

0

-

If this parameter is not set, the generated JAR package is larger but faster than that generated when this parameter is not set.

-

-

-

M

-

If the manifest file of all items is not generated, this parameter will be ignored.

-

# Compress the hello.class files in the current directory into Hello.jar and display the compression process. If the Hello.jar files do not exist, create them. Otherwise, clear the directory. However, the manifest file is not generated when Hello.jar is created.

-

jar cvfM Hello.jar hello.class

-

jarfile

-

JAR package, which is an auxiliary parameter of the f parameter.

-

-

-

manifest

-

Manifest file in .mf format, which is an auxiliary parameter of the m parameter.

-

-

-

-C dir

-

Runs the jar command in the specified dir. This command can be used only with parameters c and t.

-

-

-

file

-

Specifies the file or path list. All files in the file or path (including those in the recursive path) are compressed into the JAR package or the JAR package is decompressed to the path.

-

# Compress all class files in the current directory into Hello.jar and display the compression process. If the Hello.jar files do not exist, create them. Otherwise, clear the directory.

-

jar cvf Hello.jar *.class

-
- -## Class Library - -The Java class library is implemented as a package, which is a collection of classes and interfaces. The Java compiler generates a bytecode file for each class, and the file name is the same as the class name. Therefore, conflicts may occur between classes with the same name. In the Java language, a group of classes and interfaces are encapsulated in a package. Class namespaces can be effectively managed by package. Classes in different packages do not conflict even if they have the same name. This solves the problem of conflicts between classes with the same name and facilitates the management of a large number of classes and interfaces. It also ensures the security of classes and interfaces. - -In addition to many packages provided by Java, developers can customize packages by collecting compiled classes and interfaces into a package for future use. - -Before using a custom package, you need to declare the package. - -### Package Declaration - -The declaration format of a package is package pkg1\[.pkg2\[.pkg3...\]\]. - -To declare a package, you must create a directory. The subdirectory name must be the same as the package name. Then declare the package at the beginning of the class file that needs to be placed in the package, indicating that all classes of the file belong to the package. The dot \(.\) in the package declaration indicates the directory hierarchy. If the source program file does not contain the package statement, the package is specified as an anonymous package. An anonymous package does not have a path. Generally, Java still stores the classes in the source file in the current working directory \(that is, the directory where the Java source files are stored\). - -The package declaration statement must be added to the beginning of the source program file and cannot be preceded by comments or spaces. If you use the same package declaration statement in different source program files, you can include the classes in different source program files in the same package. - -### Package Reference - -In Java, there are two methods to use the common classes in the package provided by Java or the classes in the custom package. - -- Add the package name before the name of the class to be referenced. - - For example, name.A obj=new name.A \(\) - - **name** indicates the package name, **A** indicates the class name, and **obj** indicates the object. This string indicates that class **A** in the **name** package is used to define an object **obj** in the program. - - Example: Create a test object of the Test class in the example package. - - ``` - example.Test test = new example.Test(); - ``` - -- Use **import** at the beginning of the file to import the classes in the package. - - The format of the **import** statement is import pkg1\[.pkg2\[.pkg3...\]\].\(classname | \*\). - - **pkg1\[.pkg2\[.pkg3...\]\]** indicates the package level, and **classname** indicates the class to be imported. If you want to import multiple classes from a package, you can use the wildcard \(\*\) instead. - - Example: Import the **Test** class in the **example** package. - - ``` - import example.Test; - ``` - - Example: Import the entire **example** package. - - ``` - import example.*; - ``` - - -## Examples - - - -### Compiling a Java Program Without a Package - -1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. The command is as follows: - - ``` - $ cd ~/code - ``` - -2. Compile the Hello World program and save it as **HelloWorld.java**. The following uses the Hello World program as an example. The command is as follows: - - ``` - $ vi HelloWorld.java - ``` - - Code example: - - ``` - public class HelloWorld { - public static void main(String[] args) { - System.out.println("Hello World"); - } - } - ``` - -3. Run the following command to compile the code in the code directory: - - ``` - $ javac HelloWorld.java - ``` - - If no error is reported, the execution is successful. - -4. After the compilation is complete, the HelloWorld.class file is generated. You can run the **java** command to view the result. The following is an example: - - ``` - $ java HelloWorld - Hello World - ``` - - -### Compiling a Java Program with a Package - -1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. Create the **~/code/Test/my/example**, **~/code/Hello/world/developers**, and **~/code/Hi/openos/openeuler** subdirectories in the directory to store source files. - - ``` - $ cd ~/code - $ mkdir -p Test/my/example - $ mkdir -p Hello/world/developers - $ mkdir -p Hi/openos/openeuler - ``` - -2. Run the **cd** command to go to the **~/code/Test/my/example** directory and create **Test.java**. - - ``` - $ cd ~/code/Test/my/example - $ vi Test.java - ``` - - The following is an example of the Test.java code: - - ``` - package my.example; - import world.developers.Hello; - import openos.openeuler.Hi; - public class Test { - public static void main(String[] args) { - Hello me = new Hello(); - me.hello(); - Hi you = new Hi(); - you.hi(); - } - } - ``` - -3. Run the **cd** command to go to the **~/code/Hello/world/developers** directory and create **Hello.java**. - - ``` - $ cd ~/code/Hello/world/developers - $ vi Hello.java - ``` - - The following is an example of the Hello.java code: - - ``` - package world.developers; - public class Hello { - public void hello(){ - System.out.println("Hello, openEuler."); - } - } - ``` - -4. Run the **cd** command to go to the **~/code/Hi/openos/openeuler** directory and create **Hi.java**. - - ``` - $ cd ~/code/Hi/openos/openeuler - $ vi Hi.java - ``` - - The following is an example of the Hi.java code: - - ``` - package openos.openeuler; - public class Hi { - public void hi(){ - System.out.println("Hi, the global developers."); - } - } - ``` - -5. Run the **cd** command to go to the **~/code** directory and use javac to compile the source file. - - ``` - $ cd ~/code - $ javac -classpath Hello:Hi Test/my/example/Test.java - ``` - - After the command is executed, the **Test.class**, **Hello.class**, and **Hi.class** files are generated in the **~/code/Test/my/example**, **~/code/Hello/world/developers**, and **~/code/Hi/openos/openeuler** directories. - -6. Run the **cd** command to go to the **~/code** directory and run the **Test** program using Java. - - ``` - $ cd ~/code - $ java -classpath Test:Hello:Hi my/example/Test - ``` - - The command output is as follows: - - ``` - Hello, openEuler. - Hi, the global developers. - ``` diff --git a/docs/en/docs/ApplicationDev/using-make-for-compilation.md b/docs/en/docs/ApplicationDev/using-make-for-compilation.md deleted file mode 100644 index a00b04ec6df7879dbcd1c76bfb0f0e6948961060..0000000000000000000000000000000000000000 --- a/docs/en/docs/ApplicationDev/using-make-for-compilation.md +++ /dev/null @@ -1,372 +0,0 @@ -# Using Make for Compilation - -This chapter describes the basic knowledge of make compilation and provides examples for demonstration. For more information about Make, run the **man make** command. - - -- [Using Make for Compilation](#using-make-for-compilation) - - [Overview](#overview) - - [Basics](#basics) - - [File Type](#file-type) - - [make Work Process](#make-work-process) - - [make Options](#make-options) - - [Makefiles](#makefiles) - - [Makefile Structure](#makefile-structure) - - [Makefile Contents](#makefile-contents) - - [Examples](#examples) - - [Example of Using Makefile to Implement Compilation](#example-of-using-makefile-to-implement-compilation) - - - -## Overview - -The GNU make utility \(usually abbreviated as make\) is a tool for controlling the generation of executable files from source files. make automatically identifies which parts of the complex program have changed and need to be recompiled. Make uses configuration files called makefiles to control how programs are built. - -## Basics - -### File Type - -[Table 1](#table634145764320) describes the file types that may be used in the makefiles file. - -**Table 1** File types - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Extension (Suffix)

-

Description

-

.c

-

C source code file.

-

.C, .cc, or .cxx

-

C++ source code file.

-

.m

-

Objective-C source code file.

-

.s

-

Assembly language source code file.

-

.i

-

Preprocessed C source code file.

-

.ii

-

Preprocessed C++ source code file.

-

.S

-

Pre-processed assembly language source code file.

-

.h

-

Header file contained in the program.

-

.o

-

Target file after compilation.

-

.so

-

Dynamic link library, which is a special target file.

-

.a

-

Static link library.

-

.out

-

Executable files, which do not have a fixed suffix. The system distinguishes executable files from inexecutable files based on file attributes. If the name of an executable file is not given, GCC generates a file named a.out.

-
- -### make Work Process - -The process of deploying make to generate an executable file from the source code file is described as follows: - -1. The make command reads the Makefiles, including the files named GNUmakefile, makefile, and Makefile in the current directory, the included makefile, and the rule files specified by the **-f**, **--file**, and **--makefile** options. -2. Initialize variables. -3. Derive implicit rules, analyze dependencies, and create a dependency chain. -4. Determine which targets need to be regenerated based on the dependency chain. -5. Run a command to generate the final file. - -### make Options - -make command format: **make** \[_option_\]... \[_target_\]... - -In the preceding command: - -_option_ : parameter option. - -_target_ : target specified in Makefile. - -[Table 2](#table261872312343) describes the common make options. - -**Table 2** Common make options - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

options Value

-

Description

-

-C dir, --directory=dir

-

Specifies dir as the working directory after the make command starts to run.

-

When there are multiple -C options, the final working directory of make is the relative path of the first directory.

-

-d

-

Displays all debugging information during execution of the make command. You can use the -d option to display all the information during the construction of the dependency chain and the reconstruction of the target.

-

-e, --environment-overrides

-

Overwrites the variable definition with the same name in Makefile with the environment variable definition.

-

-f file, --file=file,

-

--makefile=file

-

Specifies the file as the Makefile for the make command.

-

-h, --help

-

Displays help information.

-

-i, --ignore-errors

-

Ignores the errors occurred during the execution.

-

-k, --keep-going

-

When an error occurs during command execution, the make command is not terminated. The make command executes all commands as many as possible until a critical error occurs.

-

-n, --just-print, --dry-run

-

Simulates the execution of commands (including the commands starting with @) in the actual execution sequence. This command is used only to display the execution process and has no actual execution effect.

-

-o file, --old-file=file, --assume-old=file

-

The specified file does not need to be rebuilt even if its dependency has expired, and no target of this dependency file is rebuilt.

-

-p, --print-data-base

-

Before the command is executed, all data of Makefile read by make and the version information of make are printed. If you only need to print the data, run the make -qp command to view the preset rules and variables before the make command is executed. You can run the make -p -f /dev/null command.

-

-r, --no-builtin-rules

-

Ignores the use of embedded implicit rules and the implicit suffix list of all suffix rules.

-

-R, --no-builtin-variables

-

Ignores embedded hidden variables.

-

-s, --silent, --quiet

-

Cancels the printing during the command execution.

-

-S, --no-keep-going, --stop

-

Cancels the -k option. In the recursive make process, the sub-make inherits the upper-layer command line option through the MAKEFLAGS variable. You can use the -S option in the sub-make to cancel the -k option transferred by the upper-layer command, or cancel the -k option in the system environment variable MAKEFLAGS.

-

-t, --touch

-

Updates the timestamp of all target files to the current system time. Prevents make from rebuilding all outdated target files.

-

-v,--version

-

Displays the make version.

-
- -## Makefiles - -Make is a tool that uses makefiles for compilation, linking, installation, and cleanup, so as to generate executable files and other related files from source code files. Therefore, makefiles describe the compilation and linking rules of the entire project, including which files need to be compiled, which files do not need to be compiled, which files need to be compiled first, which files need to be compiled later, and which files need to be rebuilt. The makefiles automate project compilation. You do not need to manually enter a large number of source files and parameters each time. - -This chapter describes the structure and main contents of makefiles. For more information about makefiles, run the **info make** command. - -### Makefile Structure - -The makefile file structure is as follows: - -_targets_:_prerequisites_ - -_command_ - -or - -_targets_:_prerequisites_;_command_ - -_command_ - -In the preceding information: - -- _targets_ : targets, which can be target files, executable files, or tags. -- _prerequisites_ : dependency files, which are the files or targets required for generating the _targets_. There can be multiple or none of them. -- _command_ : command \(any shell command\) to be executed by make. Multiple commands are allowed, and each command occupies a line. -- Use colons \(:\) to separate the target files from the dependency files. Press **Tab** at the beginning of each command line. - -The makefile file structure indicates the output target, the object on which the output target depends, and the command to be executed for generating the target. - -### Makefile Contents - -A makefile file consists of the following contents: - -- Explicit rule - - Specify the dependency, such as the file to be generated, dependency file, and generated command. - -- Implicit rule - - Specify the rule that is automatically derived by make. The make command supports the automatic derivation function. - -- Variable definition -- File indicator - - The file indicator consists of three parts: - - - Inclusion of other makefiles, for example, include xx.md - - Selective execution, for example, \#ifdef - - Definition of multiple command lines, for example, define...endef. \(define ... endef\) - -- Comment - - The comment starts with a number sign \(\#\). - -## Examples - -### Example of Using Makefile to Implement Compilation - -1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. - - ```shell - cd ~/code - ``` - -2. Create a header file **hello.h** and two functions **hello.c** and **main.c**. - - ```shell - vi hello.h - vi hello.c - vi main.c - ``` - - The following is an example of the **hello.h** code: - - ```c - #pragma once - #include - void hello(); - ``` - - The following is an example of the **hello.c** code: - - ```c - #include "hello.h" - void hello() - { - int i=1; - while(i<5) - { - printf("The %dth say hello.\n", i); - i++; - } - } - ``` - - The following is an example of the **main.c** code: - - ```c - #include "hello.h" - #include - int main() - { - hello(); - return 0; - } - ``` - -3. Create the makefile. - - ```shell - vi Makefile - ``` - - The following provides an example of the makefile content: - - ```text - main:main.o hello.o - gcc -o main main.o hello.o - main.o:main.c - gcc -c main.c - hello.o:hello.c - gcc -c hello.c - clean: - rm -f hello.o main.o main - ``` - -4. Run the **make** command. - - ```shell - make - ``` - - After the command is executed, the commands executed in makefile are printed. If you do not need to print the information, add the **-s** option to the **make** command. - - ```shell - gcc -c main.c - gcc -c hello.c - gcc -o main main.o hello.o - ``` - -5. Execute the ./main target. - - ```shell - ./main - ``` - - After the command is executed, the following information is displayed: - - The 1th say hello. - - The 2th say hello. - - The 3th say hello. - - The 4th say hello. diff --git a/docs/en/docs/CTinspector/installation-and-deployment.md b/docs/en/docs/CTinspector/installation-and-deployment.md deleted file mode 100644 index 6c03eb21f344939f47667c68389ef3f4e02429fd..0000000000000000000000000000000000000000 --- a/docs/en/docs/CTinspector/installation-and-deployment.md +++ /dev/null @@ -1,41 +0,0 @@ -# Installation and Deployment - -## Software Requirements - -* OS: openEuler 23.09 - -## Hardware Requirements - -* x86_64 architecture - -## Environment Preparation - -* Install openEuler by referring to [Installation Guide](../Installation/installation.md). - -* CTinspector installation requires **root** permissions. - -## CTinspector Installation - -* Install the CTinspector framework software package. - -```shell -yum install ctinspector -``` - -* Check whether the installation is successful. If the corresponding software package is displayed in the output, the installation is successful. - -```shell -rpm -q ctinspector -``` -* Check whether the core dynamic library **libebpf_vm_executor.so** or main program **vm_test** is installed. - -```shell -rpm -ql ctinspector -/usr/bin/vm_test -/usr/include/ctinspector/ebpf_vm_functions.h -/usr/include/ctinspector/ebpf_vm_simulator.h -/usr/include/ctinspector/ebpf_vm_transport_rdma.h -/usr/include/ctinspector/list.h -/usr/include/ctinspector/ub_list.h -/usr/lib64/libebpf_vm_executor.so -``` diff --git a/docs/en/docs/CTinspector/usage.md b/docs/en/docs/CTinspector/usage.md deleted file mode 100644 index f7cbe3ce9656a584c2a9b5eb6cb72bb960fa2dc0..0000000000000000000000000000000000000000 --- a/docs/en/docs/CTinspector/usage.md +++ /dev/null @@ -1,43 +0,0 @@ -# Usage - -## NIC Configuration - -```shell -# Change the MTU of the NIC. -ifconfig ens33 mtu 4200 - -# Add an RXE interface to ens33 for the IB function. -rdma link add rxe_0 type rxe netdev ens33 - -``` - -## Application Development -Use relevant APIs to develop a scenario-specific application. Build the application as a binary ELF file based on the eBPF instruction set. Take **vm_migrate** of the provided **ebpf_example** for example. **vm_migrate** calls the CTinspector framework and can migrate package VMs between nodes in a resumable manner. - -```text -# Compose the Makefile and set the eBPF instruction set. - -CFLAGS=-O2 -fno-inline -emit-llvm -I/usr/include/ctinspector/ -LINKFLAGS=-march=bpf -filetype=obj - -all: vm_migrate.o - -vm_migrate.o: - clang $(CFLAGS) -c migrate.c -o - | llc $(LINKFLAGS) -o vm_migrate.o - -clean: - rm -f vm_migrate.o -``` - -```shell -# Run make to build the application. -make -clang -O2 -fno-inline -emit-llvm -I/usr/include/ctinspector/ -c migrate.c -o - | llc -march=bpf -filetype=obj -o vm_migrate.o -``` - -## Application Running -Running **vm_migrate** on node 1. -![](./figures/migrate_node_1.png) - -Running the CTinspector main prgram on node 2. -![](./figures/migrate_node_2.png) diff --git a/docs/en/docs/CertSignature/figures/cert-tree.png b/docs/en/docs/CertSignature/figures/cert-tree.png deleted file mode 100644 index 930a664600b31140c3939b1abd005cc2572cdbf9..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/CertSignature/figures/cert-tree.png and /dev/null differ diff --git a/docs/en/docs/CertSignature/secure_boot.md b/docs/en/docs/CertSignature/secure_boot.md deleted file mode 100644 index da3a678ef7c3c7a0678d98df08319d98cc097c9d..0000000000000000000000000000000000000000 --- a/docs/en/docs/CertSignature/secure_boot.md +++ /dev/null @@ -1,45 +0,0 @@ -# Secure Boot - -## Overview - -Secure Boot relies on public and private key pairs to sign and verify components in the booting process. During booting, the previous component authenticates the digital signature of the next component. If the authentication is successful, the next component runs. If the authentication fails, the booting stops. Secure Boot ensures the integrity of each component during system booting and prevents unauthenticated components from being loaded and running, preventing security threats to the system and user data. -Components to be authenticated and loaded in sequence in Secure Boot are BIOS, shim, GRUB, and vmlinuz (kernel image). -Related EFI startup components are signed by the openEuler signature platform using signcode. The public key certificate is integrated into the signature database by the BIOS. During the boot, the BIOS verifies the shim. The shim and GRUB components obtain the public key certificate from the signature database of the BIOS and verify the next-level components. - -## Background and Solutions - -In earlier openEuler versions, secure boot components are not signed. Therefore, the secure boot function cannot be directly used to ensure the integrity of system components. -In openEuler 22.03 LTS SP3 and later versions, openEuler uses the community signature platform to sign OS components, including the GRUB and vmlinuz components, and integrates the community signature root certificate in the shim component. -For the shim component, to facilitate end-to-end secure boot, the signature platform of the openEuler community is used for signature. After external CAs officially operate the secure boot component signature service, the signatures of these CAs will be integrated into the shim module of openEuler. - -## Usage - -### Obtaining the openEuler Certificate - -To obtain the openEuler root certificate, visit and download it from the **Certificate Center** directory. -The root certificate name on the web page are **openEuler Shim Default CA** and **default-x509ca.cert**. - -### Operation in the BIOS - -Import the openEuler root certificate to the certificate database of the BIOS and enable secure boot in the BIOS. -For details about how to import the BIOS certificate and enable secure boot, see the documents provided by the BIOS vendor. - -### Operation in the OS - -Check the certificate information in the database: `mokutil –db` -![](./figures/mokutil-db.png) -Note: There is a large amount of certificate information. Only some important information is displayed in the screenshot. -Check the secure boot status: `mokutil --sb` - -- **SecureBoot disabled**: Secure boot is disabled. -![](./figures/mokutil-sb-off.png) -- **SecureBoot enabled**: Secure boot is enabled. -![](./figures/mokutil-sb-on.png) -- **not supported**: The system does not support secure boot. -![](./figures/mokutil-sb-unsupport.png) - -## Constraints - -- **Software**: The OS must be booted in UEFI mode. -- **Architecture**: Arm or x86 -- **Hardware**: The BIOS must support the verification function related to secure boot. diff --git a/docs/en/docs/Container/CRI_API_v1.md b/docs/en/docs/Container/CRI_API_v1.md deleted file mode 100644 index f34be40a1a616a201694c0b9d2c8e903c87e6742..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/CRI_API_v1.md +++ /dev/null @@ -1,202 +0,0 @@ -# CRI API v1 - -## Overview - -Container Runtime Interface (CRI) is the main protocol used by kublet to communicate with container engines. -Kubernetes 1.25 and earlier versions support CRI v1alpha2 and CRI v1. Kubernetes 1.26 and later versions support only CRI v1. - -iSulad supports both [CRI v1alpha2](./CRI_API_v1alpha2.md) and CRI v1. -For CRI v1, iSulad supports the functions described in [CRI v1alpha2](./CRI_API_v1alpha2.md) and new interfaces and fields defined in CRI v1. - -Currently, iSulad supports CRI v1 1.29. The API described on the official website is as follows: - -[https://github.com/kubernetes/cri-api/blob/kubernetes-1.29.0/pkg/apis/runtime/v1/api.proto](https://github.com/kubernetes/cri-api/blob/kubernetes-1.29.0/pkg/apis/runtime/v1/api.proto) - -The API description file used by iSulad is slightly different from the official API. The interfaces in this document prevail. - -## New Fields of CRI v1 - -- **CgroupDriver** - - Enum values for cgroup drivers. - - | Member| Description | - | :----------------: | :----------------: | - | SYSTEMD = 0 | systemd-cgroup driver| - | CGROUPFS = 1 | cgroupfs driver | -- **LinuxRuntimeConfiguration** - - cgroup driver used by the container engine - - | Member | Description | - | :------------------------: | :------------------------------: | - | CgroupDriver cgroup_driver | Enum value for the cgroup driver used by the container engine| -- **ContainerEventType** - - Enum values for container event types - - | Member | Description| - | :-------------------------: | :------------: | - | CONTAINER_CREATED_EVENT = 0 | Container creation event | - | CONTAINER_STARTED_EVENT = 1 | Container startup event | - | CONTAINER_STOPPED_EVENT = 1 | Container stop event | - | CONTAINER_DELETED_EVENT = 1 | Container deletion event | -- **SwapUsage** - - Virtual memory usage - - | Member | Description | - | :------------------------------: | :------------------: | - | int64 timestamp | Timestamp information | - | UInt64Value swap_available_bytes | Available virtual memory bytes| - | UInt64Value swap_usage_bytes | Used virtual memory bytes| - -## New Interfaces - -### RuntimeConfig - -#### Interface Prototype - -```text -rpc RuntimeConfig(RuntimeConfigRequest) returns (RuntimeConfigResponse) {} -``` - -#### Interface Description - -Obtains the cgroup driver configuration (cgroupfs or systemd-cgroup). - -#### Parameter: RuntimeConfigRequest - -No such field - -#### Returns: RuntimeConfigResponse - -| Return | Description | -| :------------------------------ | :------------------------------------------------- | -| LinuxRuntimeConfiguration linux | CgroupDriver enum value for cgroupfs or systemd-cgroup| - -### GetContainerEvents - -#### Interface Prototype - -```text -rpc GetContainerEvents(GetEventsRequest) returns (stream ContainerEventResponse) {} -``` - -#### Interface Description - -Obtains the pod lifecycle event stream. - -#### Parameter: GetEventsRequest - -No such field - -#### Returns: ContainerEventResponse - -| Return | Description | -| :------------------------------------------- | :-------------------------------- | -| string container_id | Container ID | -| ContainerEventType container_event_type | Container event type | -| int64 created_at | Time when the container event is generated | -| PodSandboxStatus pod_sandbox_status | Status of the pod to which the container belongs | -| repeated ContainerStatus containers_statuses | Status of all containers in the pod to which the container belongs| - -## Change Description - -### CRI V1.29 - -#### [Obtaining the cgroup Driver Configuration](https://github.com/kubernetes/kubernetes/pull/118770) - -`RuntimeConfig` obtains the cgroup driver configuration (cgroupfs or systemd-cgroup). - -#### [GetContainerEvents Supports Pod Lifecycle Events](https://github.com/kubernetes/kubernetes/pull/111384) - -`GetContainerEvents` provides event streams related to the pod lifecycle. - -`PodSandboxStatus` is adjusted accordingly. `ContainerStatuses` is added to provide sandbox content status information. - -#### [ContainerStats Virtual Memory Information](https://github.com/kubernetes/kubernetes/pull/118865) - -The virtual memory usage information `SwapUsage` is added to `ContainerStats`. - -#### [OOMKilled Setting in the Reason Field of ContainerStatus](https://github.com/kubernetes/kubernetes/pull/112977) - -The **Reason** field in **ContainerStatus** should be set to OOMKilled when cgroup out-of-memory occurs. - -#### [Modification of PodSecurityContext.SupplementalGroups Description](https://github.com/kubernetes/kubernetes/pull/113047) - -The description is modified to optimize the comments of **PodSecurityContext.SupplementalGroups**. The behavior that the main UID defined by the container image is not in the list is clarified. - -#### [ExecSync Output Restriction](https://github.com/kubernetes/kubernetes/pull/110435) - -The **ExecSync** return value output is less than 16 MB. - -## User Guide - -### Configuring iSulad to Support CRI V1 - -Configure iSulad to support CRI v1 1.29 used by the new Kubernetes version. - -For CRI v1 1.25 or earlier, the functions of V1alpha2 are the same as those of V1. The new features of CRI v1 1.26 or later are supported only in CRI v1. -The functions and features of this upgrade are supported only in CRI v1. Therefore, you need to enable CRI v1as follows. - -Enable CRI v1. - -Set **enable-cri-v1** in **daemon.json** of iSulad to **true** and restart iSulad. - -```json -{ - "group": "isula", - "default-runtime": "runc", - ... - "enable-cri-v1": true -} -``` - -If iSulad is installed from source, enable the **ENABLE_CRI_API_V1** compile option. - -```bash -cmake ../ -D ENABLE_CRI_API_V1=ON -``` - -### Using RuntimeConfig to Obtain the cgroup Driver Configuration - -#### systemd-cgroup Configuration - -iSulad supports both systemd and cgroupfs cgroup drivers. -By default, cgroupfs is used. You can configure iSulad to support systemd-cgroup. -iSulad supports only systemd-cgroup when the runtime is runc. In the iSulad configuration file **daemon.json**, -set **systemd-cgroup** to **true** and restart iSulad to use the systemd-cgroup driver. - -```json -{ - "group": "isula", - "default-runtime": "runc", - ... - "enable-cri-v1": true, - "systemd-cgroup": true -} -``` - -### Using GetContainerEvents to Generate Pod Lifecycle Events - -#### Pod Events Configuration - -In the iSulad configuration file **daemon.json**, -set **enable-pod-events** to **true** and restart iSulad. - -```json -{ - "group": "isula", - "default-runtime": "runc", - ... - "enable-cri-v1": true, - "enable-pod-events": true -} -``` - -## Constraints - -1. The preceding new features are supported by iSulad only when the container runtime is runc. -2. cgroup out-of-memory (OOM) triggers the deletion of the cgroup path of the container. If iSulad processes the OOM event after the cgroup path is deleted, iSulad cannot capture the OOM event of the container. As a result, the **Reason** field in **ContainerStatus** may be incorrect. -3. iSulad does not support the mixed use of different cgroup drivers to manage containers. After a container is started, the cgroup driver configuration in iSulad should not change. diff --git a/docs/en/docs/Container/CRI_API_v1alpha2.md b/docs/en/docs/Container/CRI_API_v1alpha2.md deleted file mode 100644 index 4d5c173f92591366183123e55480d0a5cb3a384d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/CRI_API_v1alpha2.md +++ /dev/null @@ -1,1220 +0,0 @@ -# CRI API v1alpha2 - -## Description - -CRI API is the container runtime APIs provided by Kubernetes. CRI defines service interfaces for containers and images. iSulad uses CRI API to interconnect with Kubernetes. - -The lifecycle of a container is isolated from that of an image. Therefore, two services are required. CRI API is defined using [Protocol Buffers](https://developers.google.com/protocol-buffers/) and is based on [gRPC](https://grpc.io/). - -Currently, the default CRI API version used by iSulad is v1alpha2. The official API description file is as follows: - -[https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto](https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto), - -iSulad uses the API description file of version 1.14 used by Pass, which is slightly different from the official API. The interfaces in this document prevail. - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> For the WebSocket streaming service of CRI API, the listening address of the server is 127.0.0.1, and the port number is 10350. The port number can be configured through the `--websocket-server-listening-port` command option or in the **daemon.json** configuration file. - -## Interfaces - -The following tables list the parameters that may be used by the interfaces. Some parameters cannot be configured. - -### Interface Parameters - -- **DNSConfig** - - Specifies the DNS servers and search domains of a sandbox. - - | Member | Description | - | :----------------------: | :--------------------------------------------------------: | - | repeated string servers | List of DNS servers of the cluster | - | repeated string searches | List of DNS search domains of the cluster | - | repeated string options | List of DNS options. See .| -- **Protocol** - - Enum values of the protocols. - - | Member| Description | - | :------: | :-----: | - | TCP = 0 | TCP| - | UDP = 1 | UDP| -- **PortMapping** - - Specifies the port mapping configurations of a sandbox. - - | Member | Description | - | :------------------: | :----------------: | - | Protocol protocol | Protocol of the port mapping | - | int32 container_port | Port number within the container | - | int32 host_port | Port number on the host | - | string host_ip | Host IP address | -- **MountPropagation** - - Enum values for mount propagation. - - | Member | Description | - | :-------------------------------: | :--------------------------------------------------: | - | PROPAGATION_PRIVATE = 0 | No mount propagation ("rprivate" in Linux) | - | PROPAGATION_HOST_TO_CONTAINER = 1 | Mounts get propagated from the host to the container ("rslave" in Linux) | - | PROPAGATION_BIDIRECTIONAL = 2 | Mounts get propagated from the host to the container and from the container to the host ("rshared" in Linux) | -- **Mount** - - Specifies a host volume to mount into a container. (Only files and folders are supported.) - - | Member | Description | - | :--------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------: | - | string container_path | Path in the container | - | string host_path | Path on the host | - | bool readonly | Whether the configuration is read-only in the container. The default value is **false**. | - | bool selinux_relabel | Whether to set the SELinux label (not supported) | - | MountPropagation propagation | Mount propagation configuration. The value can be **0**, **1**, or **2**, corresponding to **rprivate**, **rslave**, or **rshared**. The default value is **0**. | -- **NamespaceOption** - - | Member | Description | - | :---------------: | :------------------------: | - | bool host_network | Whether to use the network namespace of the host | - | bool host_pid | Whether to use the PID namespace of the host | - | bool host_ipc | Whether to use the IPC namespace of the host | -- **Capability** - - Contains information about the capabilities to add or drop. - - | Member | Description | - | :-------------------------------: | :----------: | - | repeated string add_capabilities | Capabilities to add | - | repeated string drop_capabilities | Capabilities to drop | -- **Int64Value** - - Wrapper of the int64 type. - - | Member| Description| - | :----------------: | :------------: | - | int64 value | Actual int64 value | -- **UInt64Value** - - Wrapper of the uint64 type. - - | Member| Description| - | :----------------: | :------------: | - | uint64 value | Actual uint64 value | -- **LinuxSandboxSecurityContext** - - Specifies Linux security options for a sandbox. - - Note that these security options are not applied to containers in the sandbox and may not be applicable to a sandbox without any running process. - - | Member | Description | - | :--------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | - | NamespaceOption namespace_options | Options for namespaces of the sandbox | - | SELinuxOption selinux_options | SELinux options (not supported) | - | Int64Value run_as_user | UID to run sandbox processes | - | bool readonly_rootfs | Whether the root file system of the sandbox is read-only | - | repeated int64 supplemental_groups | User group information of process 1 in the sandbox besides the primary group | - | bool privileged | Whether the sandbox can run a privileged container | - | string seccomp_profile_path | Path of the seccomp configuration file. Valid values are:
// **unconfined**: seccomp is not used.
// **localhost/****: path of the configuration file installed in the system.
// **:Full path of the configuration file.
//By default, this parameter is not set, which is identical to **unconfined**.| -- **LinuxPodSandboxConfig** - - Sets configurations related to Linux hosts and containers. - - | Member | Description | - | :------------------------------------------: | :-------------------------------------------------------------------------------------: | - | string cgroup_parent | Parent cgroup path of the sandbox. The runtime can convert it to the cgroupfs or systemd semantics as required. (Not configurable)| - | LinuxSandboxSecurityContext security_context | Security attributes of the sandbox | - | map\ sysctls | Linux sysctls configurations of the sandbox | -- **PodSandboxMetadata** - - Stores all necessary information for building the sandbox name. The container runtime is encouraged to expose the metadata in its user interface for better user experience. For example, the runtime can construct a unique sandbox name based on the metadata. - - | Member| Description | - | :----------------: | :----------------------------: | - | string name | Sandbox name | - | string uid | Sandbox UID | - | string namespace | Sandbox namespace | - | uint32 attempt | Number of attempts to create the sandbox. The default value is **0**.| -- **PodSandboxConfig** - - Contains all the required and optional fields for creating a sandbox. - - | Member | Description | - | :--------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------: | - | PodSandboxMetadata metadata | Metadata of the sandbox. This information uniquely identifies the sandbox, and the runtime should leverage this to ensure correct operation. The runtime may also use this information to improve user experience, such as by constructing a readable sandbox name.| - | string hostname | Host name of the sandbox | - | string log_directory | Directory for storing log files of containers in the sandbox | - | DNSConfig dns_config | DNS configuration of the sandbox | - | repeated PortMapping port_mappings | Port mappings of the sandbox | - | map\ labels | Key-value pairs that may be used to identify a single sandbox or a series of sandboxes | - | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified and can be queried by using **PodSandboxStatus**. | - | LinuxPodSandboxConfig linux | Options related to the linux host | -- **PodSandboxNetworkStatus** - - Describes the network status of the sandbox. - - | Member| Description | - | :----------------: | :-------------------: | - | string ip | IP address of the sandbox | - | string name | Name of the network interface in the sandbox | - | string network | Name of the additional network | -- **Namespace** - - Stores namespace options. - - | Member | Description | - | :---------------------: | :----------------: | - | NamespaceOption options | Linux namespace options | -- **LinuxPodSandboxStatus** - - Describes the status of the Linux sandbox. - - | Member | Description| - | :---------------------------: | :-------------: | - | Namespace**namespaces** | Sandbox namespace | -- **PodSandboxState** - - Enum values for sandbox states. - - | Member | Description | - | :------------------: | :--------------------: | - | SANDBOX_READY = 0 | Ready state of the sandbox | - | SANDBOX_NOTREADY = 1 | Non-ready state of the sandbox | -- **PodSandboxStatus** - - Describes the podsandbox status. - - | Member | Description | - | :---------------------------------------: | :-----------------------------------------------: | - | string id | Sandbox ID | - | PodSandboxMetadata metadata | Sandbox metadata | - | PodSandboxState state | Sandbox state | - | int64 created_at | Creation timestamps of the sandbox in nanoseconds | - | repeated PodSandboxNetworkStatus networks | Multi-plane network status of the sandbox | - | LinuxPodSandboxStatus linux | Status specific to Linux sandboxes | - | map\ labels | Key-value pairs that may be used to identify a single sandbox or a series of sandboxes | - | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime.| -- **PodSandboxStateValue** - - Wrapper of **PodSandboxState**. - - | Member | Description| - | :-------------------: | :-------------: | - | PodSandboxState state | Sandbox state | -- **PodSandboxFilter** - - Filtering conditions when listing sandboxes. The intersection of multiple conditions is displayed. - - | Member | Description | - | :--------------------------------: | :--------------------------------------------------: | - | string id | Sandbox ID | - | PodSandboxStateValue state | Sandbox state | - | map\ label_selector | Sandbox labels. Only full match is supported. Regular expressions are not supported.| -- **PodSandbox** - - Minimal data that describes a sandbox. - - | Member | Description | - | :-----------------------------: | :-----------------------------------------------: | - | string id | Sandbox ID | - | PodSandboxMetadata metadata | Sandbox metadata | - | PodSandboxState state | Sandbox state | - | int64 created_at | Creation timestamps of the sandbox in nanoseconds | - | map\ labels | Key-value pairs that may be used to identify a single sandbox or a series of sandboxes | - | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime | -- **KeyValue** - - Wrapper of a key-value pair. - - | Member| Description| - | :----------------: | :------------: | - | string key | Key | - | string value | Value | -- **SELinuxOption** - - SELinux labels to be applied to the container. - - | Member| Description| - | :----------------: | :------------: | - | string user | User | - | string role | Role | - | string type | Type | - | string level | Level | -- **ContainerMetadata** - - ContainerMetadata contains all necessary information for building the container name. The container runtime is encouraged to expose the metadata in its user interface for better user experience. For example, the runtime can construct a unique container name based on the metadata. - - | Member| Description | - | :----------------: | :------------------------------: | - | string name | Name of a container | - | uint32 attempt | Number of attempts to create the container. The default value is **0**.| -- **ContainerState** - - Enum values for container states. - - | Member | Description | - | :-------------------: | :-------------------: | - | CONTAINER_CREATED = 0 | The container is created | - | CONTAINER_RUNNING = 1 | The container is running | - | CONTAINER_EXITED = 2 | The container is in the exit state | - | CONTAINER_UNKNOWN = 3 | The container state is unknown | -- **ContainerStateValue** - - Wrapper of ContainerState. - - | Member | Description| - | :-------------------: | :------------: | - | ContainerState state | Container state value | -- **ContainerFilter** - - Filtering conditions when listing containers. The intersection of multiple conditions is displayed. - - | Member | Description | - | :--------------------------------: | :----------------------------------------------------: | - | string id | Container ID | - | PodSandboxStateValue state | Container state | - | string pod_sandbox_id | Sandbox ID | - | map\ label_selector | Container labels. Only full match is supported. Regular expressions are not supported.| -- **LinuxContainerSecurityContext** - - Security configuration that will be applied to a container. - - | Member | Description | - | :--------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------- | - | Capability capabilities | Capabilities to add or drop | - | bool privileged | Whether the container is in privileged mode. The default value is **false**. | - | NamespaceOption namespace_options | Namespace options of the container | - | SELinuxOption selinux_options | SELinux context to be optionally applied (**not supported currently**) | - | Int64Value run_as_user | UID to run container processes. Only one of **run_as_user** and **run_as_username** can be specified at a time. **run_as_username** takes effect preferentially. | - | string run_as_username | User name to run container processes. If specified, the user must exist in the container image (that is, in **/etc/passwd** inside the image) and be resolved there by the runtime. Otherwise, the runtime must throw an error.| - | bool readonly_rootfs | Whether the root file system in the container is read-only. The default value is configured in **config.json**. | - | repeated int64 supplemental_groups | List of groups of the first process in the container besides the primary group | - | string apparmor_profile | AppArmor configuration file for the container (**not supported currently**) | - | string seccomp_profile_path | Seccomp configuration file for the container | - | bool no_new_privs | Whether to set the **no_new_privs** flag on the container | -- **LinuxContainerResources** - - Resource specification for the Linux container. - - | Member | Description | - | :-------------------------- | :------------------------------------------------------------- | - | int64 cpu_period | CPU Completely Fair Scheduler (CFS) period. The default value is **0**. | - | int64 cpu_quota | CPU CFS quota. The default value is **0**. | - | int64 cpu_shares | CPU shares (weight relative to other containers). The default value is **0**.| - | int64 memory_limit_in_bytes | Memory limit, in bytes. The default value is **0**. | - | int64 oom_score_adj | oom-killer score. The default value is **0**. | - | string cpuset_cpus | CPU cores to be used by the container. The default value is **""**. | - | string cpuset_mems | Memory nodes to be used by the container. The default value is **""**. | -- **Image** - - Basic information about a container image. - - | Member | Description | - | :--------------------------- | :--------------------- | - | string id | Image ID | - | repeated string repo_tags | Image tag name (**repo_tags**) | - | repeated string repo_digests | Image digest information | - | uint64 size | Image size | - | Int64Value uid | UID of the default image user | - | string username | Name of the default image user | -- **ImageSpec** - - Internal data structure that represents an image. Currently, **ImageSpec** wraps only the container image name. - - | Member| Description| - | :----------------: | :------------: | - | string image | Container image name | -- **StorageIdentifier** - - Unique identifier of a storage device. - - | Member| Description| - | :----------------: | :------------: | - | string uuid | UUID of the device | -- **FilesystemUsage** - - | Member | Description | - | :--------------------------- | :------------------------- | - | int64 timestamp | Timestamp at which the information was collected | - | StorageIdentifier storage_id | UUID of the file system that stores the image | - | UInt64Value used_bytes | Space size used for storing image metadata | - | UInt64Value inodes_used | Number of inodes for storing image metadata | -- **AuthConfig** - - | Member | Description | - | :-------------------- | :------------------------------------- | - | string username | User name used for downloading images | - | string password | Password used for downloading images | - | string auth | Base64-encoded authentication information used for downloading images | - | string server_address | Address of the server for downloaded images (not supported currently) | - | string identity_token | Token information used for authentication with the image repository (not supported currently) | - | string registry_token | Token information used for interaction with the image repository (not supported currently) | -- **Container** - - Container description information, such as the ID and state. - - | Member | Description | - | :-----------------------------: | :---------------------------------------------------------: | - | string id | Container ID | - | string pod_sandbox_id | ID of the sandbox to which the container belongs | - | ContainerMetadata metadata | Container metadata | - | ImageSpec image | Image specifications | - | string image_ref | Reference to the image used by the container. For most runtimes, this is an image ID.| - | ContainerState state | Container state | - | int64 created_at | Creation timestamps of the container in nanoseconds | - | map\ labels | Key-value pairs that may be used to identify a single container or a series of containers | - | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime | -- **ContainerStatus** - - Container status information. - - | Member | Description | - | :-----------------------------: | :-----------------------------------------------------------------------: | - | string id | Container ID | - | ContainerMetadata metadata | Container metadata | - | ContainerState state | Container state | - | int64 created_at | Creation timestamps of the container in nanoseconds | - | int64 started_at | Startup timestamps of the container in nanoseconds | - | int64 finished_at | Exit timestamps of the container in nanoseconds | - | int32 exit_code | Container exit code | - | ImageSpec image | Image specifications | - | string image_ref | Reference to the image used by the container. For most runtimes, this is an image ID. | - | string reason | Brief explanation of why the container is in its current state | - | string message | Human-readable message explaining why the container is in its current state | - | map\ labels | Key-value pairs that may be used to identify a single container or a series of containers | - | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime. | - | repeated Mount mounts | Container mount point information | - | string log_path | Container log file path. The file is in the **log_directory** folder configured in **PodSandboxConfig**.| -- **ContainerStatsFilter** - - Filtering conditions when listing container states. The intersection of multiple conditions is displayed. - - | Member | Description | - | :--------------------------------: | :----------------------------------------------------: | - | string id | Container ID | - | string pod_sandbox_id | Sandbox ID | - | map\ label_selector | Container labels. Only full match is supported. Regular expressions are not supported.| -- **ContainerStats** - - Filtering conditions when listing container states. The intersection of multiple conditions is displayed. - - | Member | Description| - | :----------------------------: | :------------: | - | ContainerAttributes attributes | Container Information | - | CpuUsage cpu | CPU usage | - | MemoryUsage memory | Memory usage | - | FilesystemUsage writable_layer | Usage of the writable layer | -- **ContainerAttributes** - - Basic information about the container. - - | Member | Description | - | :----------------------------: | :-----------------------------------------------: | - | string id | Container ID | - | ContainerMetadata metadata | Container metadata | - | map\ labels | Key-value pairs that may be used to identify a single container or a series of containers | - | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime.| -- **CpuUsage** - - Container CPU usage. - - | Member | Description | - | :---------------------------------: | :--------------------: | - | int64 timestamp | Timestamp | - | UInt64Value usage_core_nano_seconds | CPU usage duration, in nanoseconds | -- **MemoryUsage** - - Container memory usage. - - | Member | Description| - | :---------------------------: | :------------: | - | int64 timestamp | Timestamp | - | UInt64Value working_set_bytes | Memory usage | -- **FilesystemUsage** - - Usage of the writable layer of the container. - - | Member | Description | - | :--------------------------: | :-----------------------: | - | int64 timestamp | Timestamp | - | StorageIdentifier storage_id | Writable layer directory | - | UInt64Value used_bytes | Number of bytes occupied by the image at the writable layer | - | UInt64Value inodes_used | Number of inodes occupied by the image at the writable layer | -- **Device** - - Host volume to mount into a container. - - | Member | Description | - | :-------------------- | :--------------------------------------------------------------------------------------------------------- | - | string container_path | Mount path within the container | - | string host_path | Mount path on the host | - | string permissions | cgroup permissions of the device (**r** allows the container to read from the specified device; **w** allows the container to write to the specified device; **m** allows the container to create device files that do not yet exist).| -- **LinuxContainerConfig** - - Configuration specific to Linux containers. - - | Member | Description | - | :--------------------------------------------- | :---------------------- | - | LinuxContainerResources resources | Container resource specifications | - | LinuxContainerSecurityContext security_context | Linux container security configuration | -- **ContainerConfig** - - Required and optional fields for creating a container. - - | Member | Description | - | :------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | ContainerMetadata metadata | Container metadata. This information uniquely identifies the container, and the runtime should leverage this to ensure correct operation. The runtime may also use this information to improve user experience, such as by constructing a readable container name. (**Required**)| - | ImageSpec image | Image used by the container. (Required) | - | repeated string command | Command to be executed. The default value is **"/bin/sh"**. | - | repeated string args | Arguments of the command to be executed | - | string working_dir | Current working directory of the command to be executed | - | repeated KeyValue envs | Environment variables to set in the container | - | repeated Mount mounts | Mount points in the container | - | repeated Device devices | Devices to be mapped in the container | - | mapstring, labels | Key-value pairs that may be used to index and select individual resources | - | mapstring, annotations | Unstructured key-value map that may be used to store and retrieve arbitrary metadata | - | string log_path | Path relative to **PodSandboxConfig.LogDirectory** for container to store the logs (STDOUT and STDERR) on the host - | bool stdin | Whether to enable STDIN of the container | - | bool stdin_once | Whether to immediately disconnect all data streams connected to STDIN when a data stream connected to stdin is disconnected (**not supported currently**) | - | bool tty | Whether to use a pseudo terminal to connect to STDIO of the container | - | LinuxContainerConfig linux | Configuration specific to Linux containers | -- **NetworkConfig** - - Runtime network configuration. - - | Member| Description | - | :----------------- | :-------------------- | - | string pod_cidr | CIDR for pod IP addresses | -- **RuntimeConfig** - - Runtime network configuration. - - | Member | Description | - | :--------------------------- | :---------------- | - | NetworkConfig network_config | Runtime network configuration | -- **RuntimeCondition** - - Runtime condition information. - - | Member| Description | - | :----------------- | :------------------------------------------ | - | string type | Runtime condition type | - | bool status | Runtime status | - | string reason | Brief description of the reason for the runtime condition change | - | string message | Human-readable message describing the reason for the runtime condition change | -- **RuntimeStatus** - - Runtime status. - - | Member | Description | - | :----------------------------------- | :------------------------ | - | repeated RuntimeCondition conditions | Current runtime conditions | - -### Runtime Service - -The runtime service contains interfaces for operating pods and containers, and interfaces for querying the configuration and status of the runtime service. - -#### RunPodSandbox - -#### Interface Prototype - -``` -rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {} -``` - -#### Interface Description - -Creates and starts a pod sandbox. The sandbox is in the ready state on success. - -#### Precautions - -1. The default image for starting the sandbox is **rnd-dockerhub.huawei.com/library/pause-$\{machine\}:3.0**, where **$\{machine\}** indicates the architecture. On x86\_64, the value of **machine** is **amd64**, on ARM64, the value of **machine** is **aarch64**. Currently, only the **amd64** and **aarch64** images can be downloaded from the rnd-dockerhub repository. If the images do not exist on the host, ensure that the host can download them from the rnd-dockerhub repository. -2. The container names use the field in **PodSandboxMetadata** and are separated by underscores (\_). Therefore, the data in metadata cannot contain underscores. Otherwise, the sandbox runs successfully, but the **ListPodSandbox** interface cannot query the sandbox. - -#### Parameter - -| Member | Description | -| :---------------------- | :-------------------------------------------------------------------- | -| PodSandboxConfig config | Sandbox configuration | -| string runtime_handler | Runtime to use for the sandbox. Currently, **lcr** and **kata-runtime** are supported.| - -#### Returns - -| Return | Description | -| :-------------------- | :--------------------- | -| string pod_sandbox_id | The response data is return on success.| - -#### StopPodSandbox - -#### Interface Prototype - -``` -rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {} -``` - -#### Interface Description - -Stops the pod sandbox, stops the sandbox container, and reclaims the network resources (such as IP addresses) allocated to the sandbox. If any running container belongs to the sandbox, the container must be forcibly terminated. - -#### Parameter - -| Member | Description| -| :-------------------- | :------------- | -| string pod_sandbox_id | Sandbox ID | - -#### Returns - -| Return| Description| -| :--------------- | :------------- | -| None | None | - -#### RemovePodSandbox - -#### Interface Prototype - -```text -rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {} -``` - -#### Interface Description - -Removes a sandbox. If there are any running containers in the sandbox, they must be forcibly terminated and removed. This interface must not return an error if the sandbox has already been removed. - -#### Precautions - -1. When a sandbox is deleted, the network resources of the sandbox are not deleted. Before deleting the pod, you must call **StopPodSandbox** to remove the network resources. Ensure that **StopPodSandbox** is called at least once before deleting the sandbox. -2. If the container in a sandbox fails to be deleted when the sandbox is deleted, the sandbox is deleted but the container remains. In this case, you need to manually delete the residual container. - -#### Parameter - -| Member | Description| -| :-------------------- | :------------- | -| string pod_sandbox_id | Sandbox ID | - -#### Returns - -| Return| Description| -| :--------------- | :------------- | -| None | None | - -#### PodSandboxStatus - -#### Interface Prototype - -```text -rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {} -``` - -#### Interface Description - -Queries the status of the sandbox. If the sandbox does not exist, this interface returns an error. - -#### Parameter - -| Member | Description | -| :-------------------- | :-------------------------------------------------- | -| string pod_sandbox_id | Sandbox ID | -| bool verbose | Whether to return extra information about the sandbox (not configurable currently) | - -#### Returns - -| Return | Description | -| :----------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | -| PodSandboxStatus status | Sandbox status information | -| map\ info | Extra information of the sandbox. The **key** can be an arbitrary string, and **value** is in JSON format. **info** can include anything debug information. When **verbose** is set to **true**, **info** cannot be empty (not configurable currently). | - -#### ListPodSandbox - -#### Interface Prototype - -```text -rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {} -``` - -#### Interface Description - -Returns sandbox information. Conditional filtering is supported. - -#### Parameter - -| Member | Description| -| :---------------------- | :------------- | -| PodSandboxFilter filter | Conditional filtering parameters | - -#### Returns - -| Return | Description | -| :------------------------ | :---------------- | -| repeated PodSandbox items | Sandboxes | - -#### CreateContainer - -#### Interface Prototype - -```text -rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {} -``` - -#### Interface Description - -Creates a container in a PodSandbox. - -#### Precautions - -- **sandbox\_config** in **CreateContainerRequest** is the same as the configuration passed to **RunPodSandboxRequest** to create the PodSandbox. It is passed again for reference. **PodSandboxConfig** is immutable and remains unchanged throughout the lifecycle of a pod. -- The container names use the field in **ContainerMetadata** and are separated by underscores (\_). Therefore, the data in metadata cannot contain underscores. Otherwise, the container runs successfully, but the **ListContainers** interface cannot query the container. -- **CreateContainerRequest** does not contain the **runtime\_handler** field. The runtime type of the created container is the same as that of the corresponding sandbox. - -#### Parameter - -| Member | Description | -| :------------------------------ | :--------------------------------- | -| string pod_sandbox_id | ID of the PodSandbox where the container is to be created | -| ContainerConfig config | Container configuration information | -| PodSandboxConfig sandbox_config | PodSandbox configuration information | - -#### Supplementary Information - -Unstructured key-value map that may be used to store and retrieve arbitrary metadata. Some fields can be transferred through this field because CRI does not provide specific parameters. - -- Customization - - | Custom Key:Value| Description | - | :------------------------- | :------------------------------------------------ | - | cgroup.pids.max:int64_t | Limits the number of processes/threads in a container. (Set **-1** for unlimited.)| - -#### Returns - -| Return | Description | -| :------------------ | :--------------- | -| string container_id | ID of the created container | - -#### StartContainer - -#### Interface Prototype - -```text -rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {} -``` - -#### Interface Description - -Starts a container. - -#### Parameter - -| Member | Description| -| :------------------ | :------------- | -| string container_id | Container ID | - -#### Returns - -| Return| Description| -| :--------------- | :------------- | -| None | None | - -#### StopContainer - -#### Interface Prototype - -```text -rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {} -``` - -#### Interface Description - -Stops a running container. The graceful stop timeout can be configured. If the container has been stopped, no error can be returned. - -#### Parameter - -| Member | Description | -| :------------------ | :---------------------------------------------------- | -| string container_id | Container ID | -| int64 timeout | Waiting time before a container is forcibly stopped. The default value is **0**, indicating that the container is forcibly stopped immediately.| - -#### Returns - -None - -#### RemoveContainer - -#### Interface Prototype - -```text -rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {} -``` - -#### Interface Description - -Deletes a container. If the container is running, it must be forcibly stopped. If the container has been deleted, no error can be returned. - -#### Parameter - -| Member | Description| -| :------------------ | :------------- | -| string container_id | Container ID | - -#### Returns - -None - -#### ListContainers - -#### Interface Prototype - -```text -rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {} -``` - -#### Interface Description - -Returns container information. Conditional filtering is supported. - -#### Parameter - -| Member | Description| -| :--------------------- | :------------- | -| ContainerFilter filter | Conditional filtering parameters | - -#### Returns - -| Return | Description| -| :---------------------------- | :------------- | -| repeated Container containers | Containers | - -#### ContainerStatus - -#### Interface Prototype - -```text -rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {} -``` - -#### Interface Description - -Returns container status information. If the container does not exist, an error is returned. - -#### Parameter - -| Member | Description | -| :------------------ | :-------------------------------------------------- | -| string container_id | Container ID | -| bool verbose | Whether to display additional information about the sandbox (not configurable currently) | - -#### Returns - -| Return | Description | -| :----------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | -| ContainerStatus status | Container status information | -| map\ info | Extra information of the sandbox. The **key** can be an arbitrary string, and **value** is in JSON format. **info** can include anything debug information. When **verbose** is set to **true**, **info** cannot be empty (not configurable currently).| - -#### UpdateContainerResources - -#### Interface Prototype - -```text -rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {} -``` - -#### Interface Description - -Updates container resource configurations. - -#### Precautions - -- This interface is used exclusively to update the resource configuration of a container, not a pod. -- Currently, the **oom\_score\_adj** configuration of containers cannot be updated. - -#### Parameter - -| Member | Description | -| :---------------------------- | :---------------- | -| string container_id | Container ID | -| LinuxContainerResources linux | Linux resource configuration information | - -#### Returns - -None - -#### ExecSync - -#### Interface Prototype - -```text -rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {} -``` - -#### Interface Description - -Runs a command synchronously in a container and communicates using gRPC. - -#### Precautions - -This interface runs a single command and cannot open a terminal to interact with the container. - -#### Parameter - -| Member | Description | -| :------------------ | :------------------------------------------------------------------ | -| string container_id | Container ID | -| repeated string cmd | Command to be executed | -| int64 timeout | Timeout interval before a command to be stopped is forcibly terminated, in seconds. The default value is **0**, indicating that there is no timeout limit (**not supported currently**).| - -#### Returns - -| Return| Description | -| :--------------- | :------------------------------------- | -| bytes stdout | Captures the standard output of the command | -| bytes stderr | Captures the standard error output of the command | -| int32 exit_code | Exit code the command finished with. The default value is **0**, indicating success.| - -#### Exec - -#### Interface Prototype - -```text -rpc Exec(ExecRequest) returns (ExecResponse) {} -``` - -#### Interface Description - -Runs a command in the container, obtains the URL from the CRI server using gRPC, and establishes a persistent connection with the WebSocket server based on the obtained URL to interact with the container. - -#### Precautions - -This interface runs a single command and can open a terminal to interact with the container. One of **stdin**, **stdout**, or **stderr** must be true. If **tty** is true, **stderr** must be false. Multiplexing is not supported. In that case, the outputs of **stdout** and **stderr** are combined into a single stream. - -#### Parameter - -| Member | Description | -| :------------------ | :------------------- | -| string container_id | Container ID | -| repeated string cmd | Command to be executed | -| bool tty | Whether to run the command in a TTY | -| bool stdin | Whether to stream standard input | -| bool stdout | Whether to stream standard output | -| bool stderr | Whether to stream standard error output | - -#### Returns - -| Return| Description | -| :--------------- | :------------------------ | -| string url | Fully qualified URL of the exec streaming server | - -#### Attach - -#### Interface Prototype - -```text -rpc Attach(AttachRequest) returns (AttachResponse) {} -``` - -#### Interface Description - -Takes over process 1 of the container, obtains the URL from the CRI server using gRPC, and establishes a persistent connection with the WebSocket server based on the obtained URL to interact with the container. - -#### Parameter - -| Member | Description | -| :------------------ | :------------------ | -| string container_id | Container ID | -| bool tty | Whether to run the command in a TTY | -| bool stdin | Whether to stream standard input | -| bool stdout | Whether to stream standard output | -| bool stderr | Whether to stream standard error output | - -#### Returns - -| Return| Description | -| :--------------- | :-------------------------- | -| string url | Fully qualified URL of the attach streaming server | - -#### ContainerStats - -#### Interface Prototype - -```text -rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {} -``` - -#### Interface Description - -Returns information about the resources occupied by a single container. Only containers whose runtime type is lcr are supported. - -#### Parameter - -| Member | Description| -| :------------------ | :------------- | -| string container_id | Container ID | - -#### Returns - -| Return | Description | -| :------------------- | :------------------------------------------------------ | -| ContainerStats stats | Container information. Information about drives and inodes can be returned only for containers started using images in oci format.| - -#### ListContainerStats - -#### Interface Prototype - -```text -rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {} -``` - -#### Interface Description - -Returns information about resources occupied by multiple containers. Conditional filtering is supported. - -#### Parameter - -| Member | Description| -| :-------------------------- | :------------- | -| ContainerStatsFilter filter | Conditional filtering parameters | - -#### Returns - -| Return | Description | -| :---------------------------- | :-------------------------------------------------------------- | -| repeated ContainerStats stats | List of container information. Information about drives and inodes can be returned only for containers started using images in OCI format.| - -#### UpdateRuntimeConfig - -#### Interface Prototype - -```text -rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse); -``` - -#### Interface Description - -Provides standard CRI for updating pod CIDR of the network plugin. Currently, the CNI network plugins do not need to update the pod CIDR. Therefore, this interface only records access logs. - -#### Precautions - -This interface does not modify the system management information, but only records logs. - -#### Parameter - -| Member | Description | -| :--------------------------- | :---------------------- | -| RuntimeConfig runtime_config | Information to be configured for the runtime | - -#### Returns - -None - -#### Status - -#### Interface Prototype - -```text -rpc Status(StatusRequest) returns (StatusResponse) {}; -``` - -#### Interface Description - -Obtains the network status of the runtime and pod. When the network status is obtained, the network configuration is updated. - -#### Precautions - -If the network configuration fails to be updated, the original configuration is not affected. The original configuration is overwritten only when the network configuration is updated successfully. - -#### Parameter - -| Member| Description | -| :----------------- | :---------------------------------------- | -| bool verbose | Whether to display additional runtime information (not supported currently) | - -#### Returns - -| Return | Description | -| :----------------------- | :---------------------------------------------------------------------------------------------------------- | -| RuntimeStatus status | Runtime status | -| map\ info | Additional runtime information. The key of **info** can be any value, and the **value** is in JSON format and can contain any debug information. Additional information is displayed only when **Verbose** is set to **true**.| - -### Image Service - -Provides gRPC APIs for pulling, viewing, and removing images from the image repository. - -#### ListImages - -#### Interface Prototype - -```text -rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {} -``` - -#### Interface Description - -Lists information about existing images. - -#### Precautions - -This interface is a unified interface. Images of embedded format can be queried using **cri images**. However, because embedded images are not in OCI standard, the query result has the following restrictions: - -- The displayed image ID is **digest** of **config** of the image because embedded images do not have image IDs. -- **digest** cannot be displayed because embedded images have only **digest** of **config**, not **digest** of themselves, and **digest** does not comply with OCI specifications. - -#### Parameter - -| Member| Description| -| :----------------- | :------------- | -| ImageSpec filter | Name of images to be filtered | - -#### Returns - -| Return | Description| -| :-------------------- | :------------- | -| repeated Image images | List of images | - -#### ImageStatus - -#### Interface Prototype - -```text -rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {} -``` - -#### Interface Description - -Queries the details about a specified image. - -#### Precautions - -1. This interface is used to query information about a specified image. If the image does not exist, **ImageStatusResponse** is returned, in which **Image** is **nil**. -2. This interface is a unified interface. Images of embedded format cannot be queried because they do not comply with the OCI specification and lack some fields. - -#### Parameter - -| Member| Description | -| :----------------- | :------------------------------------- | -| ImageSpec image | Image name | -| bool verbose | Queries extra information. This parameter is not supported currently and no extra information is returned.| - -#### Returns - -| Return | Description | -| :----------------------- | :------------------------------------- | -| Image image | Image information | -| map\ info | Extra image information. This parameter is not supported currently and no extra information is returned.| - -#### PullImage - -#### Interface Prototype - -```text - rpc PullImage(PullImageRequest) returns (PullImageResponse) {} -``` - -#### Interface Description - -Downloads an image. - -#### Precautions - -You can download public images or private images using the username, password, and authentication information. The **server_address**, **identity_token**, and **registry_token** fields in **AuthConfig** are not supported. - -#### Parameter - -| Member | Description | -| :------------------------------ | :-------------------------------- | -| ImageSpec image | Name of the image to download | -| AuthConfig auth | Authentication information for downloading a private image | -| PodSandboxConfig sandbox_config | Downloads an Image in the pod context (not supported currently).| - -#### Returns - -| Return| Description | -| :--------------- | :----------------- | -| string image_ref | Information about the downloaded image | - -#### RemoveImage - -#### Interface Prototype - -```text -rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {} -``` - -#### Interface Description - -Deletes a specified image. - -#### Precautions - -This interface is a unified interface. Images of embedded format cannot be deleted based on the image ID because they do not comply with the OCI specification and lack some fields. - -#### Parameter - -| Member| Description | -| :----------------- | :--------------------- | -| ImageSpec image | Name or ID of the image to be deleted | - -#### Returns - -None - -#### ImageFsInfo - -#### Interface Prototype - -```text -rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {} -``` - -#### Interface Description - -Queries information about the file systems of an image. - -#### Precautions - -The queried information is the file system information in the image metadata. - -#### Parameter - -None - -#### Returns - -| Return | Description | -| :----------------------------------------- | :------------------- | -| repeated FilesystemUsage image_filesystems | Image file system information | - -### Constraints - -1. If **log_directory** is configured in **PodSandboxConfig** when a sandbox is created, **log_path** must be specified in **ContainerConfig** when a container of the sandbox is created. Otherwise, the container may fail to be started or even deleted using CRI API. - - The actual **LOGPATH** of the container is **log_directory/log_path**. If **log_path** is not configured, the final **LOGPATH** changes to **log_directory**. - - - If the path does not exist, iSulad creates a soft link pointing to the final path of container logs when starting the container, and **log_directory** becomes a soft link. In this case, there are two situations: - - 1. If **log_path** is not configured for other containers in the sandbox, when other containers are started, **log_directory** is deleted and points to **log_path** of the newly started container. As a result, the logs of the previously started container point to the logs of the container started later. - 2. If **log_path** is configured for other containers in the sandbox, **LOGPATH** of the container is **log_directory/log_path**. Because **log_directory** is a soft link, if **log_directory/log_path** is used as the soft link target to point to the actual log path of the container, the container creation fails. - - If the path exists, iSulad attempts to delete the path (non-recursively) when starting the container. If the path is a folder that contains content, the deletion fails. As a result, the soft link fails to be created and the container fails to be started. When the container is deleted, the same symptom occurs. As a result, the container deletion fails. -2. If **log_directory** is configured in **PodSandboxConfig** when a sandbox is created and **log_path** is configured in **ContainerConfig** when a container is created, the final **LOGPATH** is **log_directory/log_path**. iSulad does not create **LOGPATH** recursively. Therefore, you must ensure that **dirname(LOGPATH)**, that is, the parent directory of the final log directory, exists. -3. If **log_directory** is configured in **PodSandboxConfig** when a sandbox is created, and the same **log_path** is specified in **ContainerConfig** when two or more containers are created or containers in different sandboxes point to the same **LOGPATH**, when the containers are started successfully, the log path of the container that is started later overwrites that of the container that is started earlier. -4. If the image content in the remote image repository changes and the CRI image pulling interface is used to download the image again, the image name and tag of the local original image (if it exists) change to "none." - - Example: - - Local image: - - ```text - IMAGE TAG IMAGE ID SIZE - rnd-dockerhub.huawei.com/pproxyisulad/test latest 99e59f495ffaa 753kB - ``` - - After the **rnd-dockerhub.huawei.com/pproxyisulad/test:latest** image in the remote repository is updated and downloaded again: - - ```text - IMAGE TAG IMAGE ID SIZE - 99e59f495ffaa 753kB - rnd-dockerhub.huawei.com/pproxyisulad/test latest d8233ab899d41 1.42MB - ``` - - Run the `isula images` command. **REF** is displayed as **-**. - - ```text - REF IMAGE ID CREATED SIZE - rnd-dockerhub.huawei.com/pproxyisulad/test:latest d8233ab899d41 2019-02-14 19:19:37 1.42MB - - 99e59f495ffaa 2016-05-04 02:26:41 753kB - ``` - -5. The exec and attach interfaces of iSulad CRI API are implemented using WebSocket. Clients interact with iSulad using the same protocol. When using the exec or attach interface, do not transfer a large amount of data or files over the serial port. The exec or attach interface is used only for basic command interaction. If the user side does not process the data or files in a timely manner, data may be lost. In addition, do not use the exec or attach interface to transfer binary data or files. -6. The iSulad CRI API exec/attach depends on libwebsockets (LWS). It is recommended that the streaming API be used only for persistent connection interaction but not in high-concurrency scenarios, because the connection may fail due to insufficient host resources. It is recommended that the number of concurrent connections be less than or equal to 100. diff --git a/docs/en/docs/Container/appendix-1.md b/docs/en/docs/Container/appendix-1.md deleted file mode 100644 index 88feb8756850a0852848f376b892f699394645f9..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/appendix-1.md +++ /dev/null @@ -1,94 +0,0 @@ -## Appendix - -- [Appendix](#appendix-1) - - [Command Line Interface List](#command-line-interface-list) - - -## Command Line Interface List - -This section lists commands in system containers, which are different from those in common containers. For details about other commands, refer to sections related to the iSulad container engine or run the **isula _XXX_ --help** command. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameters

-

Value Description

-

isula create/run

-

--external-rootfs

-
  • Variable of the string type.
  • Absolute path on the host.
  • Specifies the rootfs of a VM when running a system container.
-

--system-container

-
  • Boolean variable.
  • Specifies whether a container is a system container. In a system container scenario, this function must be enabled.
-

--add-host

-
  • Variable of the string type.
  • Specifies the hosts configuration for a container. The format is hostname:ip. Multiple values can be set.
-

--dns, --dns-option, --dns-search

-
  • Variable of the string type.
  • Specifies the DNS configuration for a container. Multiple values can be set.
-

--ns-change-opt

-
  • Variable of the string type.
  • Container namespace kernel parameter. The value can only be net or ipc. If multiple values are set, separate them with commas (,), for example, --ns-change-opt=net,ipc.
-

--oom-kill-disable

-
  • Boolean variable.
  • Indicates whether to enable the oom-kill-disable function.
-

--shm-size

-
  • Variable of the string type.
  • Sets the size of /dev/shm. The default value is 64 MB. The unit can be byte (B), kilobyte (KB), megabyte (MB), gigabyte (GB), terabyte (TB), or petabyte (PB).
-

--sysctl

-
  • Variable of the string type.
  • Specifies container kernel parameters. The format is key=value. Multiple values can be set. The sysctl whitelist is as follows:
-

kernel.msgmax, kernel.msgmnb, kernel.msgmni, kernel.sem, kernel.shmall, kernel.shmmax, kernel.shmmni, kernel.shm_rmid_forced, kernel.pid_max, net., and fs.mqueue

-
NOTE:

The kernel.pid_max kernel parameter in a container must be able to be namespaced. Otherwise, an error is reported.

-

Parameter restrictions (including the parameter types and value ranges) of the sysctl whitelist in a container must be the same as those of kernel parameters in the physical machine.

-
-

--env-target-file

-
  • Variable of the string type.
  • Specifies the env persistent file path. (The path must be an absolute path and the file must be in the rootfs directory.) The file size cannot exceed 10 MB. If the value of --env conflicts with that of env in the file, the value of --env takes effect.
  • The root directory of the absolute path is the rootfs root directory. That is, to set the file path to /etc/environment in the container, you need to specify env-target-file=/etc/environment only.
-

--cgroup-parent

-
  • Variable of the string type.
  • Specifies the cgroup parent directory of a container. The cgroup root directory is /sys/fs/cgroup/controller.
-

--host-channel

-
  • Variable of the string type.
  • Specifies the memory space shared between the host and a container (tmpfs). The format is as follows:
-

host path:container path:rw/ro:size limit

-

--files-limit

-
  • Variable of the string type.
  • Specifies the maximum number of file handles in a container. The value must be an integer.
-

--user-remap

-
  • Variable of the string type.
  • The parameter format is uid:gid:offset.
-
- diff --git a/docs/en/docs/Container/appendix-2.md b/docs/en/docs/Container/appendix-2.md deleted file mode 100644 index f5342bf482626a31862cf0e1d902874e8bc6f336..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/appendix-2.md +++ /dev/null @@ -1,488 +0,0 @@ -# Appendix - -- [Appendix](#appendix-2) - - [configuration.toml](#configuration-toml) - - [APIs](#apis) - -## configuration.toml - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The value of each field in the **configuration.toml** file is subject to the **configuration.toml** file in the **kata-containers-<**_version_**\>.rpm package**. You cannot set any field in the configuration file. - -``` -[hypervisor.qemu] -path: specifies the execution path of the virtualization QEMU. -kernel: specifies the execution path of the guest kernel. -initrd: specifies the guest initrd execution path. -image: specifies the execution path of the guest image (not applicable). -machine_type: specifies the type of the analog chip. The value is virt for the ARM architecture and pc for the x86 architecture. -kernel_params: specifies the running parameters of the guest kernel. -firmware: specifies the firmware path. If this parameter is left blank, the default firmware is used. -machine_accelerators: specifies an accelerator. -default_vcpus: specifies the default number of vCPUs for each SB/VM. -default_maxvcpus: specifies the default maximum number of vCPUs for each SB/VM. -default_root_ports: specifies the default number of root ports for each SB/VM. -default_bridges: specifies the default number of bridges for each SB/VM. -default_memory: specifies the default memory size of each SB/VM. The default value is 1024 MiB. -memory_slots: specifies the number of memory slots for each SB/VM. The default value is 10. -memory_offset: specifies the memory offset. The default value is 0. -disable_block_device_use: disables the block device from being used by the rootfs of the container. -shared_fs: specifies the type of the shared file system. The default value is virtio-9p. -virtio_fs_daemon: specifies the path of the vhost-user-fs daemon process. -virtio_fs_cache_size: specifies the default size of the DAX cache. -virtio_fs_cache: specifies the cache mode. -block_device_driver: specifies the driver of a block device. -block_device_cache_set: specifies whether to set cache-related options for a block device. The default value is false. -block_device_cache_direct: specifies whether to enable O_DIRECT. The default value is false. -block_device_cache_noflush: specifies whether to ignore device update requests. The default value is false. -enable_iothreads: enables iothreads. -enable_mem_prealloc: enables VM RAM pre-allocation. The default value is false. -enable_hugepages: enables huge pages. The default value is false. -enable_swap: enables the swap function. The default value is false. -enable_debug: enables QEMU debugging. The default value is false. -disable_nesting_checks: disables nested check. -msize_9p = 8192: specifies the number of bytes transmitted in each 9p packet. -use_vsock: uses vsocks to directly communicate with the agent (the prerequisite is that vsocks is supported). The default value is false. -hotplug_vfio_on_root_bus: enables the hot swap of the VFIO device on the root bus. The default value is false. -disable_vhost_net: disables vhost_net. The default value is false. -entropy_source: specifies the default entropy source. -guest_hook_path: specifies the binary path of the guest hook. - -[factory] -enable_template: enables the VM template. The default value is false. -template_path: specifies the template path. -vm_cache_number: specifies the number of VM caches. The default value is 0. -vm_cache_endpoint: specifies the address of the Unix socket used by the VMCache. The default value is /var/run/kata-containers/cache.sock. - -[proxy.kata] -path: specifies the kata-proxy running path. -enable_debug: enables proxy debugging. The default value is false. - -[shim.kata] -path: specifies the running path of kata-shim. -enable_debug: enables shim debugging. The default value is false. -enable_tracing: enables shim opentracing. - -[agent.kata] -enable_debug: enables the agent debugging function. The default value is false. -enable_tracing: enables the agent tracing function. -trace_mode: specifies the trace mode. -trace_type: specifies the trace type. -enable_blk_mount: enables guest mounting of the block device. - -[netmon] -enable_netmon: enables network monitoring. The default value is false. -path: specifies the kata-netmon running path. -enable_debug: enables netmon debugging. The default value is false. - -[runtime] -enable_debug: enables runtime debugging. The default value is false. -enable_cpu_memory_hotplug: enables CPU and memory hot swap. The default value is false. -internetworking_model: specifies the network interconnection mode between VMs and containers. -disable_guest_seccomp: disables the seccemp security mechanism in the guest application. The default value is true. -enable_tracing: enables runtime opentracing. The default value is false. -disable_new_netns: disables network namespace creation for the shim and hypervisor processes. The default value is false. -experimental: enables the experimental feature, which does not support user-defined configurations. -``` - -## APIs - -**Table 1** Commands related to the kata-runtime network - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Subcommand

-

File Example

-

Field

-

Description

-

Remarks

-

kata-network

-
NOTE:
  • The kata-network command must be used in groups. Network devices that are not added using kata-runtime kata-network cannot be deleted or listed using kata-runtime kata-network. The reverse is also true.
  • kata-runtime kata-network imports configuration parameters through a file or stdin.
-
-

add-iface

-
NOTE:
  • An interface can be added to only one container.
  • The execution result is subject to the returned value (non-zero return value).
-
-

  

-

{

-

"device":"tap1",

-

"name":"eth1",

-

"IPAddresses":[{"address":"172.17.1.10","mask":"24"}],

-

"mtu":1300,

-

"hwAddr":"02:42:20:6f:a2:80"

-

"vhostUserSocket":"/usr/local/var/run/openvswitch/vhost-user1"

-

}

-

  

-

device

-

Sets the name of the NIC on a host.

-

Mandatory. The value can contain a maximum of 15 characters, including letters, digits, underscores (\_), hyphens (-), and periods (.). It must start with a letter. The device name must be unique on the same host.

-

name

-

Sets the name of the NIC in the container.

-

Mandatory. The value can contain a maximum of 15 characters, including letters, digits, underscores (\_), hyphens (-), and periods (.). It must start with a letter. Ensure that the name is unique in the same sandbox.

-

IPAddresses

-

Sets the IP address of an NIC.

-

Optional.

-

Currently, one IP address can be configured for each NIC. If no IP address is configured for the NIC, no IP address will be configured in the container, either.

-

mtu

-

Sets the MTU of an NIC.

-

Mandatory.

-

The value ranges from 46 to 9600.

-

hwAddr

-

Sets the MAC address of an NIC.

-

Mandatory.

-

vhostUserSocket

-

Sets the DPDK polling socket path.

-

Optional.

-

The path contains a maximum of 128 bytes. The naming rule can contain digits, letters, and hyphens (-). The path name must start with a letter.

-

del-iface

-

{

-

"name":"eth1"

-

}

-

None

-

Deletes an NIC from a container.

-
NOTE:

When deleting a NIC, you can only delete it based on the name field in the NIC container. Kata does not identify other fields.

-
-

list-ifaces

-

None

-

None

-

Queries the NIC list in a container.

-

None

-

add-route

-

{

-

"dest":"172.17.10.10/24",

-

"gateway":"",

-

"device":"eth1"

-

}

-

dest

-

Sets the network segment corresponding to the route.

-

The value is in the format of <ip>/<mask>. <ip> is mandatory.

-

There are three cases:

-

1. Both IP address and mask are configured.

-

2. If only an IP address is configured, the default mask is 32.

-

3. If "dest":"default" is configured, there is no destination by default. In this case, the gateway needs to be configured.

-

gateway

-

Sets the next-hop gateway of the route.

-

When "dest":"default" is configured, the gateway is mandatory. In other cases, this parameter is optional.

-

device

-

Sets the name of the NIC corresponding to the route.

-

Mandatory.

-

The value contains a maximum of 15 characters.

-

del-route

-

{

-

"dest":"172.17.10.10/24"

-

}

-

None

-

Deletes a container routing rule.

-

dest is mandatory, and both device and gateway are optional.

-
NOTE:

Kata performs fuzzy match based on different fields and deletes the corresponding routing rules.

-
-

list-routes

-

None

-

None

-

Queries the route list in a container.

-

None

-
- -**Table 2** kata-ipvs command line interfaces - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Subcommand

-

Field

-

Parameter

-

Sub-parameter

-

Description

-

Remarks

-

kata-ipvs

-

ipvsadm

-

--parameters

-

-A, --add-service

-

-t, --tcp-service

-

-u, --udp-service

-

Virtual service type.

-

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

-

Example:

-
kata-runtime kata-ipvs ipvsadm --parameters "--add-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 3000" <container-id>
-

-s, --scheduler

-

Load balancing scheduling algorithm.

-

Mandatory. Value range: rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq.

-

-p, --persistent

-

Service duration.

-

Mandatory. The value ranges from 1 to 2678400, in seconds.

-

-E, --edit-service

-

-t, --tcp-service

-

-u, --udp-service

-

Virtual service type.

-

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

-

-s, --scheduler

-

Load balancing scheduling algorithm.

-

Mandatory. Value range: rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq.

-

-p, --persistent

-

Service duration.

-

Mandatory. The value ranges from 1 to 2678400, in seconds.

-

-D, --delete-service

-

-t, --tcp-service

-

-u, --udp-service

-

Virtual service type.

-

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

-

-a, --add-server

-

-t, --tcp-service

-

-u, --udp-service

-

Virtual service type.

-

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

-

Example:

-
kata-runtime kata-ipvs ipvsadm --parameters "--add-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 100" <container-id>
-

-r, --real-server

-

Real server address.

-

Mandatory. The format is ip:port. The value of port ranges from 1 to 65535.

-

-w, --weight

-

Weight

-

Optional. The value ranges from 0 to 65535.

-

-e, --edit-server

-

-t, --tcp-service

-

-u, --udp-service

-

Virtual service type.

-

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

-

-r, --real-server

-

Real server address.

-

Mandatory. The format is ip:port. The value of port ranges from 1 to 65535.

-

-w, --weight

-

Weight

-

Optional. The value ranges from 0 to 65535.

-

-d, --delete-server

-

-t, --tcp-service

-

-u, --udp-service

-

Virtual service type.

-

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

-

-r, --real-server

-

Real server address.

-

Mandatory. The format is ip:port. The value of port ranges from 1 to 65535.

-

-L, --list

-

-t, --tcp-service

-

-u, --udp-service

-

Queries virtual service information.

-

Optional.

-

Example:

-
kata-runtime kata-ipvs ipvsadm --parameters "--list --tcp-service ip:port" <container-id>
-

--set

-

--tcp

-

TCP timeout.

-

Mandatory. The value ranges from 0 to 1296000.

-

Example:

-
kata-runtime kata-ipvs ipvsadm --parameters "--set 100 100 200" <container-id>
-

--tcpfin

-

TCP FIN timeout.

-

Mandatory. The value ranges from 0 to 1296000.

-

--udp

-

UDP timeout.

-

Mandatory. The value ranges from 0 to 1296000.

-

--restore

-

-

-

Imports standard inputs in batches.

-

Rule files can be specified.

-

Example:

-
kata-runtime kata-ipvs ipvsadm --restore - < <rule file path> <container-id>
-
NOTE:

By default, the NAT mode is used for adding a single real server. To add real servers in batches, you need to manually add the -m option to use the NAT mode.

-

The following is an example of the rule file content:

-

-A -t 10.10.11.12:100 -s rr -p 3000

-

-a -t 10.10.11.12:100 -r 172.16.0.1:80 -m

-

-a -t 10.10.11.12:100 -r 172.16.0.1:81 -m

-

-a -t 10.10.11.12:100 -r 172.16.0.1:82 -m

-
-

cleanup

-

--parameters

-

-d, --orig-dst

-

Specifies the IP address.

-

Mandatory.

-

Example:

-
kata-runtime kata-ipvs cleanup --parameters "--orig-dst 172.17.0.4 --protonum tcp" <container-id>
-

-p, --protonum

-

Protocol type.

-

Mandatory. The value can be tcp or udp.

-
- diff --git a/docs/en/docs/Container/appendix.md b/docs/en/docs/Container/appendix.md deleted file mode 100644 index 517681ef8b4be5881a75211d4df506868e47792d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/appendix.md +++ /dev/null @@ -1,867 +0,0 @@ -# Appendix - -- [Appendix](#appendix) - - [Command Line Parameters](#command-line-parameters) - - [CNI Parameters](#cni-parameters) - -## Command Line Parameters - -**Table 1** login command parameters - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

login

-

  

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information.

-

-p, --password

-

Specifies the password for logging in to the registry.

-

--password-stdin

-

Specifies the password for obtaining the registry from standard input.

-

-u, --username

-

Specifies the username for logging in to the registry.

-
- -**Table 2** logout command parameters - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

logout

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information.

-
- -**Table 3** pull command parameters - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

pull

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-
- -**Table 4** rmi command parameters - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

rmi

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-

-f, --force

-

Forcibly removes an image.

-
- -**Table 5** load command parameters - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

load

-

-H, --host (supported only by iSula)

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-

-i, --input

-

Specifies where to import an image. If the image is of the docker type, the value is the image package path. If the image is of the embedded type, the value is the image manifest path.

-

--tag

-

Uses the image name specified by TAG instead of the default image name. This parameter is supported when the type is set to docker.

-
- -**Table 6** images command parameters - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

images

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-

-f, --filter

-

Filters information about specified images.

-

-q, --quit

-

Displays only the image name.

-
- -**Table 7** inspect command parameters - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

inspect

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-

-f, --format

-

Outputs using a template.

-

-t, --time

-

Timeout interval, in seconds. If the inspect command fails to query container information within the specified period, the system stops waiting and reports an error immediately. The default value is 120s. If the value is less than or equal to 0, the inspect command keeps waiting until the container information is obtained successfully.

-
- -**Table 8** tag command parameters - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

tag

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-
- -
- -**Table 9** import command parameters - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

import

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-
- -
- -**Table 10** export command parameters - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

export

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--help

-

Prints help information

-

-D, --debug

-

Enables debugging mode.

-

-o, --output

-

Outputs to a specified file.

-
- -## CNI Parameters - -**Table 1** CNI single network parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Type

-

Mandatory or Not

-

Description

-

cniVersion

-

string

-

Yes

-

CNI version. Only 0.3.0 and 0.3.1 are supported.

-

name

-

string

-

Yes

-

Network name, which is user-defined and must be unique.

-

type

-

string

-

Yes

-

Network type. The following types are supported:

-

underlay_ipvlan

-

overlay_l2

-

underlay_l2

-

vpc-router

-

dpdk-direct

-

phy-direct

-

ipMasq

-

bool

-

No

-

Configures the IP masquerade.

-

ipam

-

structure

-

No

-

For details, see the IPAM parameter definition.

-

ipam.type

-

string

-

No

-

IPAM type. The following types are supported:

-

(1) For underlay_l2, overlay_l2, and vpc-router networking, only the default value distributed_l2 is supported.

-

(2) For underlay_ipvlan networking, the default value is distributed_l2. In the CCN scenario, only null and fixed are supported. In the CCE and FST 5G core scenarios, only null and distributed_l2 are supported.

-

(3) For phy-direct and dpdk-direct networking, the default value is l2, and optional values are null and distributed_l2. In the FST 5G core scenario, only null and distributed_l2 are supported.

-

Description:

-

If the value is out of the range (for example, host-local), Canal automatically sets the value to the default value and no error is returned.

-

null: Canal is not used to manage IP addresses.

-

fixed: fixed IP address, which is used in the CCN scenario.

-

l2: This value is not used in any scenario.

-

distributed_l2: The distributed small subnet is used to manage IP addresses.

-

ipam.subnet

-

string

-

No

-

Subnet information. Canal supports the subnet mask ranging from 8 to 29. The IP address cannot be a multicast address (for example, 224.0.0.0/4), reserved address (240.0.0.0/4), local link address (169.254.0.0/16), or local loop address (127.0.0.0/8).

-

ipam.gateway

-

string

-

No

-

Gateway IP address.

-

ipam.range-start

-

string

-

No

-

Available start IP address.

-

ipam.range-end

-

string

-

No

-

Available end IP address.

-

ipam.routes

-

structure

-

No

-

Subnet list. Each element is a route dictionary. For details, see the route definition.

-

ipam.routes.dst

-

string

-

No

-

Destination network.

-

ipam.routes.gw

-

string

-

No

-

Gateway address.

-

dns

-

structure

-

No

-

Contains some special DNS values.

-

dns.nameservers

-

[]string

-

No

-

NameServers

-

dns.domain

-

string

-

No

-

Domain

-

dns.search

-

[]string

-

No

-

Search

-

dns.options

-

[]string

-

No

-

Options

-

multi_entry

-

int

-

No

-

Number of IP addresses required by a vNIC. The value ranges from 0 to 16. For physical passthrough, a maximum of 128 IP addresses can be applied for a single NIC.

-

backup_mode

-

bool

-

No

-

Active/Standby mode, which is used only for phy-direct and dpdk-direct networking.

-

vlanID

-

int

-

No

-

The value ranges from 0 to 4095. It can be specified through PaaS.

-

vlan_inside

-

bool

-

No

-

The value true indicates that the VLAN function is implemented internally on the node, and the value false indicates that the VLAN function is implemented externally.

-

vxlanID

-

int

-

No

-

The value ranges from 0 to 16777215. It can be specified through PaaS.

-

vxlan_inside

-

bool

-

No

-

The value true indicates that the VLAN function is implemented internally on the node, and the value false indicates that the VLAN function is implemented externally.

-

action

-

string

-

No

-

This parameter can be used only with the special container ID 000000000000.

-

Create: creates a network.

-

Delete: deletes a network.

-

args

-

map[string]interface{}

-

No

-

Key-value pair type.

-

runtimeConfig

-

structure

-

No

-

None

-

capabilities

-

structure

-

No

-

None

-
- -**Table 2** CNI args parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Type

-

Mandatory

-

Description

-

K8S_POD_NAME

-

string

-

No

-

Set this parameter when you apply for a fixed IP address (runtimeConfig.ican_caps.fixed_ip is set to true).

-

K8S_POD_NAMESPACE

-

string

-

No

-

Set this parameter when you apply for a fixed IP address (runtimeConfig.ican_caps.fixed_ip is set to true).

-

SECURE_CONTAINER

-

string

-

No

-

Secure container flag.

-

multi_port

-

int

-

No

-

The value ranges from 1 to 8. The default value is 1. Specifies the number of passthrough NICs. Only phy-direct and dpdk-direct networks are supported.

-

phy-direct

-

string

-

No

-

Specifies the NIC to be connected when you create an SR-IOV container network.

-

dpdk-direct

-

string

-

No

-

Specifies the NIC to be connected when you create a DPDK passthrough container network.

-

tenant_id

-

string

-

No

-

Indicates the tenant ID.

-

Only vpc-router networks are supported.

-

vpc_id

-

string

-

No

-

VPC ID.

-

Only vpc-router networks are supported.

-

secret_name

-

string

-

No

-

Specifies the AK/SK object name on the K8S APIServer.

-

Only vpc-router networks are supported.

-

For details, see the configuration of VPC-Router logical networks.

-

IP

-

string

-

No

-

IP address specified by the user, in the format of 192.168.0.10.

-

K8S_POD_NETWORK_ARGS

-

string

-

No

-

Specifies an IP address, in the format of 192.168.0.10. If both IP and K8S_POD_NETWORK_ARGS in args are not empty, the value of K8S_POD_NETWORK_ARGS prevails.

-

INSTANCE_NAME

-

string

-

No

-

INSTANCE ID.

-

Refer to fixed IP addresses that support containers.

-

dist_gateway_disable

-

bool

-

No

-

The value true indicates that no gateway is created, and the value false indicates that a gateway is created.

-

phynet

-

string or []string

-

No

-

Specifies the name of the physical plane to be added. The physical plane name is predefined and corresponds to that in the SNC system. When two plane names are entered, the active and standby planes are supported. Example: phy_net1 or ["phy_net2","phy_net3"]

-

endpoint_policies

-

struct

-

No

-

"endpoint_policies": [

-

{

-

"Type": "",

-

"ExceptionList": [

-

""

-

],

-

"NeedEncap": true,

-

"DestinationPrefix": ""

-

}

-

]

-

port_map

-

struct

-

No

-

On a NAT network, container ports can be advertised to host ports.

-

"port_map": [

-

{

-

"local_port": number,

-

"host_port": number,

-

"protocol": [string...]

-

}...

-

]

-
- -**Table 3** CNI multiple network parameters - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Type

-

Mandatory

-

Description

-

cniVersion

-

string

-

Yes

-

CNI version. Only 0.3.0 and 0.3.1 are supported.

-

name

-

string

-

Yes

-

Network name, which is user-defined and must be unique.

-

plugins

-

struct

-

Yes

-

For details, see CNI single network parameters

-
diff --git a/docs/en/docs/Container/application-scenarios-2.md b/docs/en/docs/Container/application-scenarios-2.md deleted file mode 100644 index 5346a100ee36da8190c476a4751e0d414e8ca2ec..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/application-scenarios-2.md +++ /dev/null @@ -1,4 +0,0 @@ -# Application Scenarios - -This section describes how to use a secure container. - diff --git a/docs/en/docs/Container/application-scenarios.md b/docs/en/docs/Container/application-scenarios.md deleted file mode 100644 index fe74c96c762fd08199445dbda6c552d38dcce197..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/application-scenarios.md +++ /dev/null @@ -1,5 +0,0 @@ -# Application Scenarios - -This section describes how to use the iSulad. - - diff --git a/docs/en/docs/Container/checking-the-container-health-status.md b/docs/en/docs/Container/checking-the-container-health-status.md deleted file mode 100644 index 2a134c49067912df2121037f47e438d8a6dceb32..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/checking-the-container-health-status.md +++ /dev/null @@ -1,67 +0,0 @@ -# Checking the Container Health Status - -- [Checking the Container Health Status](#checking-the-container-health-status) - - [Scenarios](#scenarios) - - [Configuration Methods](#configuration-methods) - - [Check Rules](#check-rules) - - [Usage Restrictions](#usage-restrictions) - -## Scenarios - -In the production environment, bugs are inevitable in applications provided by developers or services provided by platforms. Therefore, a management system is indispensable for periodically checking and repairing applications. The container health check mechanism adds a user-defined health check function for containers. When a container is created, the **--health-cmd** option is configured so that commands are periodically executed in the container to monitor the health status of the container based on return values. - -## Configuration Methods - -Configurations during container startup: - -```shell -isula run -itd --health-cmd "echo iSulad >> /tmp/health_check_file || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy busybox bash -``` - -The configurable options are as follows: - -- **--health-cmd**: This option is mandatory. If **0** is returned after a command is run in a container, the command execution succeeds. If a value other than **0** is returned, the command execution fails. -- **--health-interval**: interval between two consecutive command executions. The default value is **30s**. The value ranges from **1s** to the maximum value of Int64 \(unit: nanosecond\). If the input parameter is set to **0s**, the default value is used. -- **--health-timeout**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is **30s**. The value ranges from **1s** to the maximum value of Int64 \(unit: nanosecond\). If the input parameter is set to **0s**, the default value is used. Only containers whose runtime is of the LCR type are supported. -- **--health-start-period**: container initialization time. The default value is **0s**. The value ranges from **1s** to the maximum value of Int64 \(unit: nanosecond\). -- **--health-retries**: maximum number of retries for the health check. The default value is **3**. The maximum value is the maximum value of Int32. -- **--health-exit-on-unhealthy**: specifies whether to kill a container when it is unhealthy. The default value is **false**. - -## Check Rules - -1. After a container is started, the container status is **health:starting**. -2. After the period specified by **start-period**, the **cmd** command is periodically executed in the container at the interval specified by **interval**. That is, after the command is executed, the command will be executed again after the specified period. -3. If the **cmd** command is successfully executed within the time specified by **timeout** and the return value is **0**, the check is successful. Otherwise, the check fails. If the check is successful, the container status changes to **health:healthy**. -4. If the **cmd** command fails to be executed for the number of times specified by **retries**, the container status changes to **health:unhealthy**, and the container continues the health check. -5. When the container status is **health:unhealthy**, the container status changes to **health:healthy** if a check succeeds. -6. If **--exit-on-unhealthy** is set, and the container exits due to reasons other than being killed \(the returned exit code is **137**\), the health check takes effect only after the container is restarted. -7. When the **cmd** command execution is complete or times out, iSulad daemon will record the start time, return value, and standard output of the check to the configuration file of the container. A maximum of five records can be recorded. In addition, the configuration file of the container stores health check parameters. -8. When the container is running, the health check status is written into the container configurations. You can run the **isula inspect** command to view the status. - -```text -"Health": { - "Status": "healthy", - "FailingStreak": 0, - "Log": [ - { - "Start": "2018-03-07T07:44:15.481414707-05:00", - "End": "2018-03-07T07:44:15.556908311-05:00", - "ExitCode": 0, - "Output": "" - }, - { - "Start": "2018-03-07T07:44:18.557297462-05:00", - "End": "2018-03-07T07:44:18.63035891-05:00", - "ExitCode": 0, - "Output": "" - }, - ...... -} -``` - -## Usage Restrictions - -- A maximum of five health check status records can be stored in a container. The last five records are saved. -- If health check parameters are set to **0** during container startup, the default values are used. -- After a container with configured health check parameters is started, if iSulad daemon exits, the health check is not executed. After iSulad daemon is restarted, the health status of the running container changes to **starting**. Afterwards, the check rules are the same as above. -- If the health check fails for the first time, the health check status will not change from **starting** to **unhealthy** until the specified number of retries \(**--health-retries**\) is reached, or to **healthy** until the health check succeeds. diff --git a/docs/en/docs/Container/command-reference.md b/docs/en/docs/Container/command-reference.md deleted file mode 100644 index d7578e549bf15d14f944ccd1fda94911fead4829..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/command-reference.md +++ /dev/null @@ -1,7 +0,0 @@ -# Command Reference - -- [Command Reference](#command-reference) - - [Container Engine](#container-engine) - - [Container Management](#container-management-40) - - [Image Management](#image-management-43) - - [Statistics](#statistics) diff --git a/docs/en/docs/Container/configurable-cgroup-path.md b/docs/en/docs/Container/configurable-cgroup-path.md deleted file mode 100644 index d5d24d9d0b195d249fc536e32f5022bfa58f0e39..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/configurable-cgroup-path.md +++ /dev/null @@ -1,99 +0,0 @@ -# Configurable Cgroup Path - -- [Configurable Cgroup Path](#configurable-cgroup-path) - - -## Function Description - -System containers provide the capabilities of isolating and reserving container resources on hosts. You can use the **--cgroup-parent** parameter to specify the cgroup directory used by a container to another directory, thereby flexibly allocating host resources. For example, if the cgroup parent path of containers A, B, and C is set to **/lxc/cgroup1**, and the cgroup parent path of containers D, E, and F is set to **/lxc/cgroup2**, the containers are divided into two groups through the cgroup paths, implementing resource isolation at the cgroup level. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--cgroup-parent

-
  • Variable of the string type.
  • Specifies the cgroup parent path of the container.
-
- -In addition to specifying the cgroup parent path for a system container using commands, you can also specify the cgroup paths of all containers by modifying the startup configuration files of the iSulad container engine. - - - - - - - - - - - - -

Configuration File Path

-

Parameter

-

Description

-

/etc/isulad/daemon.json

-

--cgroup-parent

-
  • Variable of the string type.
  • Specifies the default cgroup parent path of the container.
  • Example: "cgroup-parent": "/lxc/mycgroup"
-
- -## Constraints - -- If the **cgroup parent** parameter is set on both the daemon and client, the value specified on the client takes effect. -- If container A is started before container B, the cgroup parent path of container B is specified as the cgroup path of container A. When deleting a container, you need to delete container B and then container A. Otherwise, residual cgroup resources exist. - -## Example - -Start a system container and specify the **--cgroup-parent** parameter. - -``` -[root@localhost ~]# isula run -tid --cgroup-parent /lxc/cgroup123 --system-container --external-rootfs /root/myrootfs none init -115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -``` - -Check the cgroup information of the init process in the container. - -``` -[root@localhost ~]# isula inspect -f "{{json .State.Pid}}" 11 -22167 -[root@localhost ~]# cat /proc/22167/cgroup -13:blkio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -12:perf_event:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -11:cpuset:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -10:pids:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -9:rdma:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -8:devices:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -7:hugetlb:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -6:memory:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -5:net_cls,net_prio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -4:cpu,cpuacct:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -3:files:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -2:freezer:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -1:name=systemd:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e/init.scope -0::/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e -``` - -The cgroup parent path of the container is set to **/sys/fs/cgroup/**__**/lxc/cgroup123**. - -In addition, you can configure the container daemon file to set the cgroup parent paths for all containers. For example: - -``` -{ - "cgroup-parent": "/lxc/cgroup123", -} -``` - -Restart the container engine for the configuration to take effect. - diff --git a/docs/en/docs/Container/configuring-resources-for-a-secure-container.md b/docs/en/docs/Container/configuring-resources-for-a-secure-container.md deleted file mode 100644 index e6cba396b3d1a8f56c8e5b23e311a4367f5983a6..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/configuring-resources-for-a-secure-container.md +++ /dev/null @@ -1,34 +0,0 @@ -# Configuring Resources for a Secure Container - -- [Configuring Resources for a Secure Container](#configuring-resources-for-a-secure-container) - - [Sharing Resources](#sharing-resources) - - [Limiting Resources](#limiting-resources) - - [Limiting Memory Resources Through the Memory Hotplug Feature](#limiting-memory-resources-through-the-memory-hotplug-feature) - -The secure container runs on a virtualized and isolated lightweight VM. Therefore, resource configuration is divided into two parts: resource configuration for the lightweight VM, that is, host resource configuration; resource configuration for containers in the VM, that is, guest container resource configuration. The following describes resource configuration for the two parts in detail. - - -## Sharing Resources - -Because the secure container runs on a virtualized and isolated lightweight VM, resources in some namespaces on the host cannot be accessed. Therefore, **--net host**, **--ipc host**, **--pid host**, and **--uts host** are not supported during startup. - -When a pod is started, all containers in the pod share the same net namespace and ipc namespace by default. If containers in the same pod need to share the pid namespace, you can use Kubernetes to configure the pid namespace. In Kubernetes 1.11, the pid namespace is disabled by default. - -## Limiting Resources -Limitations on sandbox resources should be configured in **configuration.toml**. -Common fields are: -- **default_vcpus** ：specifies the default number of virtual CPUs. -- **default_maxvcpus** ：specifies the max number of virtual CPUs. -- **default_root_ports** ：specifies the default number of Root Ports in SB/VM. -- **default_bridges** ：specifies the default number of bridges. -- **default_memory** ：specifies the size of memory. The default size is 1024 MiB. -- **memory_slots** ：specifies the number of memory slots. The default number is **10**. - -## Limiting Memory Resources Through the Memory Hotplug Feature -Memory hotplug is a key feature for containers to allocate memory dynamically in deployment. As Kata containers are based on VMs, this feature needs support both from VMM and guest kernel. Luckily, it has been fully supported for the current default version of QEMU and guest kernel used by Kata on ARM64. For other VMMs, e.g, Cloud Hypervisor, the enablement work is on the road. Apart from VMM and guest kernel, memory hotplug also depends on ACPI which depends on firmware. On x86, you can boot a VM using QEMU with ACPI enabled directly, because it boots up with firmware implicitly. For ARM64, however, you need specify firmware explicitly. That is to say, if you are ready to run a normal Kata container on ARM64, what you need extra to do is to install the UEFI ROM before using the memory hotplug feature. - -```shell -$ pushd $GOPATH/src/github.com/kata-containers/tests -$ sudo .ci/aarch64/install_rom_aarch64.sh -$ popd -``` diff --git a/docs/en/docs/Container/container-engine.md b/docs/en/docs/Container/container-engine.md deleted file mode 100644 index c8dec68b4cea7ce554e54f377db8b167ef04263b..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/container-engine.md +++ /dev/null @@ -1,304 +0,0 @@ -# Container Engine - -Docker daemon is a system process that resides in the background. Before you run a docker subcommand, start Docker daemon. - -If the Docker daemon is installed using the RPM package or system package management tool, you can run the **systemctl start docker** command to start the Docker daemon. - -The **docker** command supports the following parameters: - -1. To combine parameters of a single character, run the following command: - - ```shell - docker run -t -i busybox /bin/sh - ``` - - The command can be written as follows: - - ```shell - docker run -ti busybox /bin/sh - ``` - -2. **bool** command parameters such as **--icc=true**, are displayed in the command help. If this parameter is not used, the default value displayed in the command help is used. If this parameter is used, the opposite value of the value displayed in the command help is used. In addition, if **--icc** is not added when Docker daemon is started, **--icc=true** is used by default. Otherwise, **--icc=false** is used. -3. Parameters such as **--attach=\[\]** in the command help indicate that these parameters can be set for multiple times. For example: - - ```shell - docker run --attach=stdin --attach=stdout -i -t busybox /bin/sh - ``` - -4. Parameters such as **-a** and **--attach=\[\]** in the command help indicate that the parameter can be specified using either **-a** _value_ or **--attach=**_value_. For example: - - ```shell - docker run -a stdin --attach=stdout -i -t busybox /bin/sh - ``` - -5. Parameters such as **--name=""** can be configured with a character string and can be configured only once. Parameters such as **-c=** can be configured with an integer and can be configured only once. - -**Table 1** Parameters specified during the Docker daemon startup - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

--api-cors-header

-

CORS header information for enabling remote API calling. This interface supports the secondary development of upper-layer applications, which sets the CORS header for a remote API.

-

--authorization-plugin=[]

-

Authentication plug-in.

-

-b, --bridge=""

-

Existing bridge device mounting to the docker container. Note: none can be used to disable the network in the container.

-

--bip=""

-

Bridge IP address, which is automatically created using the CIDR address. Note: this parameter cannot be used with -b .

-

--cgroup-parent

-

cgroup parent directory configured for all containers.

-

--config-file=/etc/docker/daemon.json

-

Configuration file for starting Docker daemon.

-

--containerd

-

Socket path of containerd.

-

-D, --debug=false

-

Specifies whether to enable the debugging mode.

-

--default-gateway

-

Default gateway of the container IPv4 address.

-

--default-gateway-v6

-

Default gateway of the container IPv6 address.

-

--default-ulimit=[]

-

Default ulimit value of the container.

-

--disable-legacy-registry

-

Disables the original registry.

-

--dns=[]

-

DNS server of the forcibly used container.

-

Example: --dns 8.8.x.x

-

--dns-opt=[]

-

DNS option.

-

--dns-search=[]

-

Forcibly searches DNS search domain name used by a container.

-

Example: --dns-search example.com

-

--exec-opt=[]

-

Parameter to be executed when a container is started.

-

For example, set the native.umask parameter.

-
#The umask value of the started container is 0022.--exec-opt native.umask=normal 
-# The umask value of the started container is 0027 (default value).
---exec-opt  native.umask=secure
-

Note: If native.umask is also configured in docker create or docker run command, the configuration in command is used.

-

--exec-root=/var/run/docker

-

Root directory for storing the execution status file.

-

--fixed-cidr=""

-

Fixed IP address (for example, 10.20.0.0/16) of the subnet. The IP address of the subnet must belong to the network bridge.

-

--fixed-cidr-v6

-

Fixed IPv6 address.

-

-G, --group="docker"

-

Group assigned to the corresponding Unix socket in the background running mode. Note: When an empty string is configured for this parameter, the group information is removed.

-

-g, --graph="/var/lib/docker"

-

The root directory for running docker.

-

-H, --host=[]

-

Socket bound in background mode. One or more sockets can be configured using tcp://host:port, unix:///path to socket, fd://* or fd://socketfd. Example:

-

$ dockerd -H tcp://0.0.0.0:2375

-

or

-

$ export DOCKER_HOST="tcp://0.0.0.0:2375"

-

--insecure-registry=[]

-

Registry for insecure connections. By default, the Docker uses TLS certificates to ensure security for all connections. If the registry does not support HTTPS connections or the certificate is issued by an unknown certificate authority of the Docker daemon, you need to configure --insecure-registry=192.168.1.110:5000 when starting the daemon. This parameter needs to be configured if a private registry is used.

-

--image-layer-check=true

-

Image layer integrity check. To enable the function, set this parameter to true. Otherwise, set this parameter to false. If this parameter is not configured, the function is disabled by default.

-

When Docker is started, the image layer integrity is checked. If the image layer is damaged, the related images are unavailable. Docker cannot verify empty files, directories, or link files. Therefore, if the preceding files are lost due to a power failure, the integrity check of Docker image data may fail. When the Docker version changes, check whether the parameter is supported. If not supported, delete it from the configuration file.

-

--icc=true

-

Enables communication between containers.

-

--ip="0.0.0.0"

-

Default IP address used when a container is bound to a port.

-

--ip-forward=true

-

Starts the net.ipv4.ip_forward process of the container.

-

--ip-masq=true

-

Enables IP spoofing.

-

--iptables=true

-

Starts the iptables rules defined by the Docker container.

-

-l, --log-level=info

-

Log level.

-

--label=[]

-

Daemon label, in key=value format.

-

--log-driver=json-file

-

Default log driver of container logs.

-

--log-opt=map[]

-

Log drive parameters.

-

--mtu=0

-

MTU value of the container network. If this parameter is not configured, value of route MTU is used by default. If the default route is not configured, set this parameter to the constant value 1500.

-

-p, --pidfile="/var/run/docker.pid"

-

PID file path of the background process.

-

--raw-logs

-

Logs with all timestamps and without the ANSI color scheme.

-

--registry-mirror=[]

-

Image registry preferentially used by the dockerd.

-

-s, --storage-driver=""

-

Storage driver used when a container is forcibly run.

-

--selinux-enabled=false

-

Enables SELinux. If the kernel version is 3.10.0-862.14 or later, this parameter cannot be set to true.

-

--storage-opt=[]

-

Storage driver parameter. This parameter is valid only when the storage driver is devicemapper. Example: dockerd --storage-opt dm.blocksize=512K

-

--tls=false

-

Enables the TLS authentication.

-

--tlscacert="/root/.docker/ca.pem"

-

Certificate file path that has been authenticated by the CA.

-

--tlscert="/root/.docker/cert.pem"

-

File path of the TLS certificates.

-

--tlskey="/root/.docker/key.pem"

-

File path of TLS keys.

-

--tlsverify=false

-

Verifies the communication between the background processes and the client using TLS.

-

--insecure-skip-verify-enforce

-

Whether to forcibly skip the verification of the certificate host or domain name. The default value is false.

-

--use-decrypted-key=true

-

Whether to use the decryption private key.

-

--userland-proxy=true

-

Whether to use the userland proxy for the container LO device.

-

--userns-remap

-

User namespace-based user mapping table in the container.

-
NOTE:

This parameter is not supported in the current version.

-
-
diff --git a/docs/en/docs/Container/container-management-1.md b/docs/en/docs/Container/container-management-1.md deleted file mode 100644 index 5dcc5ef03c4a1dec3243b4564ef04dbce3423059..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/container-management-1.md +++ /dev/null @@ -1,706 +0,0 @@ -# Container Management - -- [Container Management](#container-management) - - [Creating a Container](#creating-a-container) - - [Creating Containers Using hook-spec](#creating-containers-using-hook-spec) - - [Configuring Health Check During Container Creation](#configuring-health-check-during-container-creation) - - [Stopping and Deleting a Container](#stopping-and-deleting-a-container) - - [Querying Container Information](#querying-container-information) - - [Modification Operations](#modification-operations) - -## Creating a Container - -### Downloading Images - -Only user **root** can run the **docker** command. If you log in as a common user, you need to use the **sudo** command before running the **docker** command. - -```shell -docker pull busybox -``` - -This command is used to download the **busybox:latest** image from the official Docker registry. \(If no tag is specified in the command, the default tag name **latest** is used.\) During the image download, the system checks whether the dependent layer exists locally. If yes, the image download is skipped. When downloading images from a private registry, specify the registry description. For example, if a private registry containing some common images is created and its IP address is **192.168.1.110:5000**, you can run the following command to download the image from the private registry: - -```shell -docker pull 192.168.1.110:5000/busybox -``` - -The name of the image downloaded from the private registry contains the registry address information, which may be too long. Run the **docker tag** command to generate an image with a shorter name. - -```shell -docker tag 192.168.1.110:5000/busybox busybox -``` - -Run the **docker images** command to view the local image list. - -### Running a Simple Application - -```shell -$ docker run busybox /bin/echo "Hello world" -Hello world -``` - -This command uses the **busybox:latest** image to create a container, and executes the **echo "Hello world"** command in the container. Run the following command to view the created container: - -```shell -$ docker ps -l -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -d8c0a3315bc0 busybox"/bin/echo 'Hello wo..." 5 seconds ago Exited (0) 3 seconds ago practical_franklin -``` - -### Creating an Interactive Container - -```shell -$ docker run -it busybox /bin/bash -root@bf22919af2cf:/# ls -bin boot dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var -root@bf22919af2cf:/# pwd -/ -``` - -The **-ti** option allocates a pseudo terminal to the container and uses standard input \(STDIN\) for interaction. You can run commands in the container. In this case, the container is an independent Linux VM. Run the **exit** command to exit the container. - -### Running a Container in the Background - -Run the following command. **-d** indicates that the container is running in the background. **--name=container1** indicates that the container name is **container1**. - -```shell -$ docker run -d --name=container1 busybox /bin/sh -c "while true;do echo hello world;sleep 1;done" -7804d3e16d69b41aac5f9bf20d5f263e2da081b1de50044105b1e3f536b6db1c -``` - -The command output contains the container ID but does not contain **hello world**. In this case, the container is running in the background. You can run the **docker ps** command to view the running container. - -```shell -$ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -7804d3e16d69 busybox "/bin/sh -c 'while tr" 11 seconds ago Up 10 seconds container1 -``` - -Run the following **docker logs** command to view the output during container running: - -```shell -$ docker logs container1 -hello world -hello world -hello world -... -``` - -### Container Network Connection - -By default, a container can access an external network, while port mapping is required when an external network accesses a container. The following uses how to run the private registry service in Docker as an example. In the following command, **-P** is used to expose open ports in the registry to the host. - -```shell -$ docker run --name=container_registry -d -P registry -cb883f6216c2b08a8c439b3957fb396c847a99079448ca741cc90724de4e4731 -``` - -The container\_registry container has been started, but the mapping between services in the container and ports on the host is not clear. You need to run the **docker port** command to view the port mapping. - -```shell -$ docker port container_registry -5000/tcp -> 0.0.0.0:49155 -``` - -The command output shows that port 5000 in the container is mapped to port 49155 on the host. You can access the registry service by using the host IP address **49155**. Enter **** in the address box of the browser and press **Enter**. The registry version information is displayed. - -When running registry images, you can directly specify the port mapping, as shown in the following: - -```shell -docker run --name=container_registry -d -p 5000:5000 registry -``` - -**-p 5000:5000** is used to map port 5000 in the container to port 5000 on the host. - -### Precautions - -- **Do Not Add -a stdin Independently During Container Startup** - - When starting a container, you must add **-a stdout** or **-a stderr** together with **-a stdin** instead of **-a stdin** only. Otherwise, the device stops responding even after the container exits. - -- **Do Not Use the Long Or Short ID of an Existing Container As the Name of a New Container** - - When creating a container, do not use the long or short ID of the existing container A as the name of the new container B. If the long ID of container A is used as the name of container B, Docker will match container A even though the name of container B is used as the specified target container for operations. If the short ID of container A is used as the name of container B, Docker will match container B even though the short ID of container A is used as the specified target container for operations. This is because Docker matches the long IDs of all containers first. If the matching fails, the system performs exact matching using the value of **container\_name**. If matching failure persists, the container ID is directly matched in fuzzy mode. - -- **Containers That Depend on Standard Input and Output, Such As sh/bash, Must Use the -ti Parameter to Avoid Exceptions** - - Normal case: If you do not use the **-ti** parameter to start a process container such as sh/bash, the container exits immediately. - - The cause of this problem is that Docker creates a stdin that matches services in the container first. If the interactive parameters such as **-ti** are not set, Docker closes pipe after the container is started and the service container process sh/bash exits after stdin is closed. - - Exception: If Docker daemon is forcibly killed in a specific phase \(before pipe is closed\), daemon of the pipe is not closed in time. In this case, the sh/bash process does not exit even without **-ti**. As a result, an exception occurs. You need to manually clear the container. - - After being restarted, daemon takes over the original container stream. Containers without the **-ti** parameter may not be able to process the stream because these containers do not have streams to be taken over in normal cases. In actual services, sh/bash without the **-ti** parameter does not take effect and is seldom used. To avoid this problem, the **-ti** parameter is used to restrict interactive containers. - -- **Container Storage Volumes** - - If you use the **-v** parameter to mount files on the host to a container when the container is started, the inodes of the files may be changed when you run the **vi** or **sed** command to modify the files on the host or in the container. As a result, files on the host and in the container are not synchronized. Do not mount files in the container in this mode \(or do not use together with the **vi** and **sed** commands\). You can also mount the upper-layer directories of the files to avoid exceptions. The **nocopy** option can be used to prevent original files in the mount point directory of a container from being copied to the source directory of the host when Docker mounts volumes. However, this option can be used only when an anonymous volume is mounted and cannot be used in the bind mount scenario. - -- **Do Not Use Options That May Affect the Host** - - The **--privileged** option enables all permissions for a container. On the container, mounting operations can be performed and directories such as **/proc** and **/sys** can be modified, which may affect the host. Therefore, do not use this option for common containers. - - A host-shared namespace, such as the **--pid host**, **--ipc host**, or **--net host** option, can enable a container to share the namespace with the host, which will also affect the host. Therefore, do not use this option. - -- **Do Not Use the Unstable Kernel Memory Cgroup** - - Kernel memory cgroup on the Linux kernel earlier than 4.0 is still in the experimental phase and runs unstably. Therefore, do not use kernel memory cgroup. - - When the **docker run --kernel-memory** command is executed, the following alarm is generated: - - ```text - WARNING: You specified a kernel memory limit on a kernel older than 4.0. Kernel memory limits are experimental on older kernels, it won't work as expected as expected and can cause your system to be unstable. - ``` - -- **blkio-weight Parameter Is Unavailable in the Kernel That Supports blkio Precise Control** - - **--blkio-weight-device** can implement more accurate blkio control in a container. The control requires a specified disk device, which can be implemented through the **--blkio-weight-device** parameter of Docker. In this kernel, Docker does not provide the **--blkio-weight** mode to limit the container blkio. If you use this parameter to create a container, the following error is reported: - - ```text - docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:398: container init caused \"process_linux.go:369: setting cgroup config for ready process caused \\\"blkio.weight not supported, use weight_device instead\\\"\"" - ``` - -- **Using --blkio-weight-device in CFQ Scheduling Policy** - - The **--blkio-weight-device** parameter works only when the disk works in the Completely Fair Queuing \(CFQ\) policy. - - You can view the scheduler file \(**/sys/block/**_disk_**/queue/scheduler**\) to obtain the policies supported by the disk and the current policy. For example, you can run the following command to view **sda**. - - ```shell - # cat /sys/block/sda/queue/scheduler noop [deadline] cfq - ``` - - **sda** supports the following scheduling policies: **noop**, **deadline**, and **cfq**, and the **deadline** policy is being used. You can run the **echo** command to change the policy to **cfq**. - - ```shell - # echo cfq > /sys/block/sda/queue/scheduler - ``` - -- **systemd Usage Restrictions in Basic Container Images** - - When containers created from basic images are used, systemd in basic images is used only for system containers. - -### Concurrent Performance - -- There is an upper limit for the message buffer in Docker. If the number of messages exceeds the upper limit, the messages are discarded. Therefore, it is recommended that the number of commands executed concurrently should not exceed 1000. Otherwise, the internal messages in Docker may be lost and the container may fail to be started. -- When containers are concurrently created and restarted, the error message"oci runtime error: container init still running" is occasionally reported. This is because containerd optimizes the performance of the event waiting queue. When a container is stopped, the **runc delete** command is executed to kill the init processes in the container within 1s. If the init processes are not killed within 1s, runC returns this error message. The garbage collection \(GC\) mechanism of containerd reclaims residual resources after **runc delete** is executed at an interval of 10s. Therefore, operations on the container are not affected. If the preceding error occurs, wait for 4 or 5s and restart the container. - -### Security Feature Interpretation - -1. The following describes default permission configuration analysis of Docker. - - In the default configuration of a native Docker, capabilities carried by each default process are as follows: - - ```text - "CAP_CHOWN", - "CAP_DAC_OVERRIDE", - "CAP_FSETID", - "CAP_FOWNER", - "CAP_MKNOD", - "CAP_NET_RAW", - "CAP_SETGID", - "CAP_SETUID", - "CAP_SETFCAP", - "CAP_SETPCAP", - "CAP_NET_BIND_SERVICE", - "CAP_SYS_CHROOT", - "CAP_KILL", - "CAP_AUDIT_WRITE", - ``` - - The default seccomp configuration is a whitelist. If any syscall is not in the whitelist, **SCMP\_ACT\_ERRNO** is returned by default. Different system invoking is enabled for different caps of Docker. If a capability is not in the whitelist, Docker will not assign it to the container by default. - -2. CAP\_SYS\_MODULE - - CAP\_SYS\_MODULE allows a container to insert or remove ko modules. Adding this capability allows the container to escape or even damage the kernel. Namespace provides the maximum isolation for a container. In the ko module, you only need to point its namespace to **init\_nsproxy**. - -3. CAP\_SYS\_ADMIN - - The sys\_admin permission provides the following capabilities for a container: - - - For file system: **mount**, **umount**, and **quotactl** - - For namespace setting: **setns**, **unshare**, and **clone new namespace** - - driver ioctl - - For PCI control: **pciconfig\_read**, **pciconfig\_write**, and **pciconfig\_iobase** - - **sethostname** - -4. CAP\_NET\_ADMIN - - CAP\_NET\_ADMIN allows a container to access network interfaces and sniff network traffic. The container can obtain the network traffic of all containers including the host, which greatly damages network isolation. - -5. CAP\_DAC\_READ\_SEARCH - - CAP\_DAC\_READ\_SEARCH calls the open\_by\_handle\_at and name\_to\_handle\_at system calls. If the host is not protected by SELinux, the container can perform brute-force search for the inode number of the file\_handle structure to open any file on the host, which affects the isolation of the file system. - -6. CAP\_SYS\_RAWIO - - CAP\_SYS\_RAWIO allows a container to write I/O ports to the host, which may cause the host kernel to crash. - -7. CAP\_SYS\_PTRACE - - The ptrace permission for a container provides ptrace process debugging in the container. RunC has fixed this vulnerability. However, some tools, such as nsenter and docker-enter, are not protected. In a container, processes executed by these tools can be debugged to obtain resource information \(such as namespace and fd\) brought by these tools. In addition, ptrace can bypass seccomp, greatly increasing attack risks of the kernel. - -8. Docker capability interface: --cap-add all - - --cap-add all grants all permissions to a container, including the dangerous permissions mentioned in this section, which allows the container to escape. - -9. Do not disable the seccomp feature of Docker. - - Docker has a default seccomp configuration with a whitelist. **sys\_call** that is not in the whitelist is disabled by seccomp. You can disable the seccomp feature by running **--security-opt 'seccomp:unconfined'**. If seccomp is disabled or the user-defined seccomp configuration is used but the filtering list is incomplete, attack risks of the kernel in the container are increased. - -10. Do not set the **/sys** and **/proc** directories to writable. - - The **/sys** and **/proc** directories contain Linux kernel maintenance parameters and device management interfaces. If the write permission is configured for the directories in a container, the container may escape. - -11. Docker open capability: --CAP\_AUDIT\_CONTROL - - The permission allows a container to control the audit system and run the **AUDIT\_TTY\_GET** and **AUDIT\_TTY\_SET** commands to obtain the TTY execution records \(including the **root** password\) recorded in the audit system. - -12. CAP\_BLOCK\_SUSPEND and CAP\_WAKE\_ALARM - - The permission provides a container the capability to block the system from suspending \(epoll\). - -13. CAP\_IPC\_LOCK - - With this permission, a container can break the max locked memory limit in **ulimit** and use any mlock large memory block to cause DoS attacks. - -14. CAP\_SYS\_LOG - - In a container with this permission, system kernel logs can be read by using dmesg to break through kernel kaslr protection. - -15. CAP\_SYS\_NICE - - In a container with this permission, the scheduling policy and priority of a process can be changed, causing DoS attacks. - -16. CAP\_SYS\_RESOURCE - - With this permission, a container can bypass resource restrictions, such as disk space resource restriction, keymaps quantity restriction, and **pipe-size-max** restriction, causing DoS attacks. - -17. CAP\_SYS\_TIME - - In a container with this capability, the time on the host can be changed. - -18. Risk analysis of Docker default capabilities - - The default capabilities of Docker include CAP\_SETUID and CAP\_FSETID. If the host and a container share a directory, the container can set permissions for the binary file in the shared directory. Common users on the host can use this method to elevate privileges. With the CAP\_AUDIT\_WRITE capability, a container can write logs to the host, and the host must be configured with log anti-explosion measures. - -19. Docker and host share namespace parameters, such as **--pid**, **--ipc**, and **--uts**. - - This parameter indicates that the container and host share the namespace. The container can attack the host as the namespace of the container is not isolated from that of the host. For example, if you use **--pid** to share PID namespace with the host, the PID on the host can be viewed in the container, and processes on the host can be killed at will. - -20. **--device** is used to map the sensitive directories or devices of the host to the container. - - The Docker management plane provides interfaces for mapping directories or devices on a host to the container, such as **--device** and **-v**. Do not map sensitive directories or devices on the host to the container. - -## Creating Containers Using hook-spec - -### Principles and Application Scenarios - -Docker supports the extended features of hooks. The execution of hook applications and underlying runC complies with the OCI standards. For details about the standards, visit - -There are three types of hooks: prestart, poststart, and poststop. They are respectively used before applications in the container are started, after the applications are started, and after the applications are stopped. - -### API Reference - -The **--hook-spec** parameter is added to the **docker run** and **create** commands and is followed by the absolute path of the **spec** file. You can specify the hooks to be added during container startup. These hooks will be automatically appended after the hooks that are dynamically created by Docker \(currently only libnetwork prestart hook\) to execute programs specified by users during the container startup or destruction. - -The structure of **spec** is defined as follows: - -```go -// Hook specifies a command that is run at a particular event in the lifecycle of a container -type Hook struct{ - Path string `json:"path"` - Args []string `json:"args,omitempty"` - Env []string `json:"env,omitempty"` - Timeout *int `json:"timeout,omitempty"` -} -// Hooks for container setup and teardown -type Hooks struct{ - // Prestart is a list of hooks to be run before the container process is executed. - // On Linux, they are run after the container namespaces are created. - Prestart []Hook `json:"prestart,omitempty"` - // Poststart is a list of hooks to be run after the container process is started. - Poststart []Hook `json:"poststart,omitempty"` - // Poststop is a list of hooks to be run after the container process exits. - Poststop []Hook `json:"poststop,omitempty"` -} -``` - -- The **Path**, **Args**, and **Env** parameters are mandatory. -- **Timeout** is optional, while you are advised to set this parameter to a value ranging from 1 to 120. The parameter type is int. Floating point numbers are not allowed. -- The content of the **spec** file must be in JSON format as shown in the preceding example. If the format is incorrect, an error is reported. -- Both **docker run --hook-spec /tmp/hookspec.json**_xxx_, and **docker create --hook-spec /tmp/hookspec.json **_xxx_** && docker start**_xxx_ can be used. - -### Customizing Hooks for a Container - -Take adding a NIC during the startup as an example. The content of the **hook spec** file is as follows: - -```json -{ - "prestart": [ - { - "path": "/var/lib/docker/hooks/network-hook", - "args": ["network-hook", "tap0", "myTap"], - "env": [], - "timeout": 5 - } - ], - "poststart":[], - "poststop":[] -} -``` - -Specify prestart hook to add the configuration of a network hook. The path is **/var/lib/docker/hooks/network-hook**. **args** indicates the program parameters. Generally, the first parameter is the program name, and the second parameter is the parameter accepted by the program. For the network-hook program, two parameters are required. One is the name of the NIC on the host, and the other is the name of the NIC in the container. - -- Precautions - 1. The **hook** path must be in the**hooks** folder in the **graph** directory \(**--graph**\) of Docker. Its default value is **/var/lib/docker/hooks**. You can run the **docker info** command to view the root path. - - ```shell - $ docker info - ... - Docker Root Dir: /var/lib/docker - ... - ``` - - This path may change due to the user's manual configuration and the use of user namespaces \(**daemon --userns-remap**\). After the symbolic link of the path is parsed, the parsed path must start with _Docker Root Dir_**/hooks** \(for example, **/var/lib/docker/hooks**\). Otherwise, an error message is displayed. - - 2. The **hook** path must be an absolute path because daemon cannot properly process a relative path. In addition, an absolute path meets security requirements. - 3. The information output by the hook program to stderr is output to the client and affects the container lifecycle \(for example, the container may fail to be started\). The information output to stdout is ignored. - 4. Do not reversely call Docker instructions in hooks. - 5. The execute permission must have been granted on the configured hook execution file. Otherwise, an error is reported during hook execution. - 6. The execution time of the hook operation must be as short as possible. If the prestart period is too long \(more than 2 minutes\), the container startup times out. If the poststop period is too long \(more than 2 minutes\), the container is abnormal. - - The known exceptions are as follows: When the **docker stop** command is executed to stop a container and the clearing operation is performed after 2 minutes, the hook operation is not complete. Therefore, the system waits until the hook operation is complete \(the process holds a lock\). As a result, all operations related to the container stop responding. The operations can be recovered only after the hook operation is complete. In addition, the two-minute timeout processing of the **docker stop** command is an asynchronous process. Therefore, even if the **docker stop** command is successfully executed, the container status is still **up**. The container status is changed to **exited** only after the hook operation is completed. - -- Suggestions - 1. You are advised to set the hook timeout threshold to a value less than 5s. - 2. You are advised to configure only one prestart hook, one poststart hook, and one poststop hook for each container. If too many hooks are configured, the container startup may take a long time. - 3. You are advised to identify the dependencies between multiple hooks. If required, you need to adjust the sequence of the hook configuration files according to the dependencies. The execution sequence of hooks is based on the sequence in the configured **spec** file. - -### Multiple **hook-spec** - -If multiple hook configuration files are available and you need to run multiple hooks, you must manually combine these files into a configuration file and specify the new configuration file by using the **--hook-spec** parameter. Then all hooks can take effect. If multiple **--hook-spec** parameters are configured, only the last one takes effect. - -Configuration examples: - -The content of the **hook1.json** file is as follows: - -```shell -# cat /var/lib/docker/hooks/hookspec.json -{ - "prestart": [ - { - "path": "/var/lib/docker/hooks/lxcfs-hook", - "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"], - "env": [] - } - ], - "poststart":[], - "poststop":[] -} -``` - -The content of the **hook2.json** file is as follows: - -```shell -# cat /etc/isulad-tools/hookspec.json -{ - "prestart": [ - { - "path": "/docker-root/hooks/docker-hooks", - "args": ["docker-hooks", "--state", "prestart"], - "env": [] - } - ], - "poststart":[], - "poststop":[ - { - "path": "/docker-root/hooks/docker-hooks", - "args": ["docker-hooks", "--state", "poststop"], - "env": [] - } - ] -} -``` - -The content in JSON format after manual combination is as follows: - -```json -{ - "prestart":[ - { - "path": "/var/lib/docker/hooks/lxcfs-hook", - "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"], - "env": [] - }, - { - "path": "/docker-root/hooks/docker-hooks", - "args": ["docker-hooks", "--state", "prestart"], - "env": [] - } - ], - "poststart":[], - "poststop":[ - { - "path": "/docker-root/hooks/docker-hooks", - "args": ["docker-hooks", "--state", "poststop"], - "env": [] - } - ] -} -``` - -Docker daemon reads the binary values of hook in actions such as prestart in the hook configuration files in sequence based on the array sequence and executes the actions. Therefore, you need to identify the dependencies between multiple hooks. If required, you need to adjust the sequence of the hook configuration files according to the dependencies. - -### Customizing Default Hooks for All Containers - -Docker daemon can receive the **--hook-spec** parameter. The semantics of **--hook-spec** is the same as that of **--hook-spec** in **docker create** or **docker run**. You can also add hook configurations to the **/etc/docker/daemon.json** file. - -```json -{ - "hook-spec": "/tmp/hookspec.json" -} -``` - -When a container is running, hooks specified in **--hook-spec** defined by daemon are executed first, and then hooks customized for each container are executed. - -## Configuring Health Check During Container Creation - -Docker provides the user-defined health check function for containers. You can configure the **HEALTHCHECK CMD** option in the Dockerfile, or configure the **--health-cmd** option when a container is created so that commands are periodically executed in the container to monitor the health status of the container based on return values. - -### Configuration Methods - -- Add the following configurations to the Dockerfile file: - - ```text - HEALTHCHECK --interval=5m --timeout=3s --health-exit-on-unhealthy=true \ - CMD curl -f http://localhost/ || exit 1 - ``` - - The configurable options are as follows: - - 1. **--interval=DURATION**: interval between two consecutive command executions. The default value is **30s**. After a container is started, the first check is performed after the interval time. - 2. **--timeout=DURATION**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is **30s**. - 3. **--start-period=DURATION**: container initialization period. The default value is **0s**. During the initialization, the health check is also performed, while the health check failure is not counted into the maximum number of retries. However, if the health check is successful during initialization, the container is considered as started. All subsequent consecutive check failures are counted in the maximum number of retries. - 4. **--retries=N**. maximum number of retries for the health check. The default value is **3**. - 5. **--health-exit-on-unhealthy=BOOLEAN**: whether to kill a container when it is unhealthy. The default value is **false**. - 6. **CMD**: This option is mandatory. If **0** is returned after a command is run in a container, the command execution succeeds. If a value other than **0** is returned, the command execution fails. - - After **HEALTHCHECK** is configured, related configurations are written into the image configurations during image creation. You can run the **docker inspect** command to view the configurations. For example: - - ```json - "Healthcheck": { - "Test": [ - "CMD-SHELL", - "/test.sh" - ] - }, - ``` - -- Configurations during container creation: - - ```shell - docker run -itd --health-cmd "curl -f http://localhost/ || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy centos bash - ``` - - The configurable options are as follows: - - 1. **--health-cmd**: This option is mandatory. If **0** is returned after a command is run in a container, the command execution succeeds. If a value other than **0** is returned, the command execution fails. - 2. **--health-interval**: interval between two consecutive command executions. The default value is **30s**. The upper limit of the value is the maximum value of Int64 \(unit: nanosecond\). - 3. **--health-timeout**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is **30s**. The upper limit of the value is the maximum value of Int64 \(unit: nanosecond\). - 4. **--health-start-period**: container initialization time. The default value is **0s**. The upper limit of the value is the maximum value of Int64 \(unit: nanosecond\). - 5. **--health-retries**: maximum number of retries for the health check. The default value is **3**. The maximum value is the maximum value of Int32. - 6. **--health-exit-on-unhealthy**: specifies whether to kill a container when it is unhealthy. The default value is **false**. - - After the container is started, the **HEALTHCHECK** configurations are written into the container configurations. You can run the **docker inspect** command to view the configurations. For example: - - ```json - "Healthcheck": { - "Test": [ - "CMD-SHELL", - "/test.sh" - ] - }, - ``` - -### Check Rules - -1. After a container is started, the container status is **health:starting**. -2. After the period specified by **start-period**, the **cmd** command is periodically executed in the container at the interval specified by **interval**. That is, after the command is executed, the command will be executed again after the specified period. -3. If the **cmd** command is successfully executed within the time specified by **timeout** and the return value is **0**, the check is successful. Otherwise, the check fails. If the check is successful, the container status changes to **health:healthy**. -4. If the **cmd** command fails to be executed for the number of times specified by **retries**, the container status changes to **health:unhealthy**, and the container continues the health check. -5. When the container status is **health:unhealthy**, the container status changes to **health:healthy** if a check succeeds. -6. If **--health-exit-on-unhealthy** is set, and the container exits due to reasons other than being killed \(the returned exit code is **137**\), the health check takes effect only after the container is restarted. -7. When the **cmd** command execution is complete or times out, Docker daemon will record the start time, return value, and standard output of the check to the configuration file of the container. A maximum of five latest records can be recorded. In addition, the configuration file of the container stores health check parameters. - -Run the **docker ps** command to view the container status. - -```shell -$ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -7de2228674a2 testimg "bash" About an hour ago Up About an hour (unhealthy) cocky_davinci -``` - -When the container is running, the health check status is written into the container configurations. You can run the **docker inspect** command to view the configurations. - -```json -"Health": { - "Status": "healthy", - "FailingStreak": 0, - "Log": [ - { - "Start": "2018-03-07T07:44:15.481414707-05:00", - "End": "2018-03-07T07:44:15.556908311-05:00", - "ExitCode": 0, - "Output": "" - }, - { - "Start": "2018-03-07T07:44:18.557297462-05:00", - "End": "2018-03-07T07:44:18.63035891-05:00", - "ExitCode": 0, - "Output": "" - }, - ...... -} -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- A maximum of five health check status records can be stored in a container. The last five records are saved. ->- Only one health check configuration item can take effect in a container at a time. The later items configured in the Dockerfile will overwrite the earlier ones. Configurations during container creation will overwrite those in images. ->- In the Dockerfile, you can set **HEALTHCHECK NONE** to cancel the health check configuration in a referenced image. When a container is running, you can set **--no-healthcheck** to cancel the health check configuration in an image. Do not configure the health check and **--no-healthcheck** parameters at the same time during the startup. ->- After a container with configured health check parameters is started, if Docker daemon exits, the health check is not executed. After Docker daemon is restarted, the container health status changes to **starting**. Afterwards, the check rules are the same as above. ->- If health check parameters are set to **0** during container image creation, the default values are used. ->- If health check parameters are set to **0** during container startup, the default values are used. - -## Stopping and Deleting a Container - -Run the **docker stop** command to stop the container named **container1**. - -```shell -docker stop container1 -``` - -Or run the **docker kill** command to kill and stop the container. - -```shell -docker kill container1 -``` - -After the container is stopped, run the **docker rm** command to delete the container. - -```shell -docker rm container1 -``` - -Or run the **docker rm -f** command to forcibly delete the container. - -```shell -docker rm -f container1 -``` - -### Precautions - -- Do not run the **docker rm –f**_XXX_ command to delete a container. If you forcibly delete a container, the **docker rm** command ignores errors during the process, which may cause residual metadata of the container. If you delete an image in common mode and an error occurs during the deletion process, the deletion fails and no metadata remains. -- Do not run the **docker kill** command. The **docker kill** command sends related signals to service processes in a container. Depending on the signal processing policies of service processes in the container may cause the result that the signal execution cannot be performed as expected. -- A container in the restarting state may not stop immediately when you run the **docker stop** command. If a container uses the restart rules, when the container is in the restarting state, there is a low probability that the **docker stop** command on the container returns immediately. The container will still be restarted with the impact of the restart rule. -- Do not run the **docker restart** command to restart a container with the **--rm** parameter. When a container with the **--rm** parameter exits, the container is automatically deleted. If the container with the **--rm** parameter is restarted, exceptions may occur. For example, if both the **--rm** and **-ti** parameters are added when the container is started, the restart operation cannot be performed on the container, otherwise, the container may stop responding and cannot exit. - -### When Using docker stop/restart to Specify -t and t<0, Ensure That Applications in the Container Can Process Stop Signal - -Stop Principle: \(The stop process is called by **Restart**.\) - -1. The SIGTERM \(15\) signal can be sent to a container for the first time. -2. Wait for a period of time \(**t** entered by the user\). -3. If the container process still exists, send the SIGKILL \(9\) signal to forcibly kill the process. - -The meaning of the input parameter **t** \(unit: s\) is as follows: - -- **t** < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism. -- **t** = 0: Do not wait and send **kill -9** to the container immediately. -- **t** \> 0: Wait for a specified period and send **kill -9** to the container if the container does not stop within the specified period. - -Therefore, if **t** is set to a value less than 0 \(for example, **t** = **-1**\), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the **docker stop** command is run. - -### Manually Deleting Containers in the Dead State As the Underlying File System May Be Busy - -When Docker deletes a container, it stops related processes of the container, changes the container status to Dead, and then deletes the container rootfs. When the file system or devicemapper is busy, the last step of deleting rootfs fails. Run the **docker ps -a** command. The command output shows that the container is in the Dead state. Containers in the Dead state cannot be started again. Wait until the file system is not busy and run the **docker rm** command again to delete the containers. - -### In PID namespace Shared Containers, If Child Container Is in pause State, Parent Container Stops Responding and the docker run Command Cannot Be Executed - -When the **--pid** parameter is used to create the parent and child containers that share PID namespace, if any process in the child container cannot exit \(for example, it is in the D or pause state\) when the **docker stop** command is executed, the **docker stop** command of the parent container is waiting. You need to manually recover the process so that the command can be executed normally. - -In this case, run the **docker inspect** command on the container in the pause state to check whether the parent container corresponding to **PidMode** is the container that requires **docker stop**. For the required container, run the **docker unpause** command to cancel the pause state of the child container. Then, proceed to the next step. - -Generally, the possible cause is that the PID namespace corresponding to the container cannot be destroyed due to residual processes. If the problem persists, use Linux tools to obtain the residual processes and locate the cause of the process exit failure in PID namespace. After the problem is solved, the container can exit. - -- Obtain PID namespace ID in a container. - - ```shell - docker inspect --format={{.State.Pid}} CONTAINERID | awk '{print "/proc/"$1"/ns/pid"}' |xargs readlink - ``` - -- Obtain threads in the namespace. - - ```shell - ls -l /proc/*/task/*/ns/pid |grep -F PIDNAMESPACE_ID |awk '{print $9}' |awk -F \/ '{print $5}' - ``` - -## Querying Container Information - -In any case, the container status should not be determined based on whether the **docker** command is successfully returned. To view the container status, you are advised to use the following command: - -```shell -docker inspect -``` - -## Modification Operations - -### Precautions for Starting Multiple Processes in Container Using docker exec - -When the first **docker exec** command executed in a container is the **bash** command, ensure that all processes started by **exec** are stopped before you run the **exit** command. Otherwise, the device may stop responding when you run the **exit** command. To ensure that the process started by **exec** is still running in the background when the **exit** command is run, add **nohup** when starting the process. - -### Usage Conflict Between docker rename and docker stats _container\_name_ - -If you run the **docker stats**_container\_name_ command to monitor a container in real time, after the container is renamed by using **docker rename**, the name displayed after **docker stats** is executed is the original name instead of the renamed one. - -### Failed to Perform docker rename Operation on Container in restarting State - -When the rename operation is performed on a container in the restarting state, Docker modifies the container network configuration accordingly. The container in the restarting state may not be started and the network does not exist. As a result, the rename operation reports an error indicating that the sandbox does not exist. You are advised to rename only containers that are not in the restarting state. - -### docker cp - -1. When you run **docker cp** to copy files to a container, all operations on the container can be performed only after the **docker cp** command is executed. -2. When a container runs as a non-**root** user, and you run the **docker cp** command to copy a non-**root** file on the host to the container, the permission role of the file in the container changes to **root**. Different from the **cp** command, the **docker cp** command changes UIDs and GIDs of the files copied to the container to **root**. - -### docker login - -After the **docker login** command is executed, **user/passwd** encrypted by AES \(256-bit\) is saved in **/root/.docker/config.json**. At the same time, _root_**.docker/aeskey** \(permission 0600\) is generated to decrypt **user/passwd** in **/root/.docker/config.json**. Currently, AES key cannot be updated periodically. You need to manually delete the AES key for updating. After AES key is updated, you need to log in to Docker daemon again to push the AES key no matter whether Docker daemon is restarted. For example: - -```text -root@hello:~/workspace/dockerfile# docker login -Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. -Username: example Password: -Login Succeeded -root@hello:~/workspace/dockerfile# docker push example/empty -The push refers to a repository [docker.io/example/empty] -547b6288eb33: Layer already exists -latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524 -root@hello:~/workspace/dockerfile# rm /root/.docker/aeskey -root@hello:~/workspace/dockerfile# docker push example/empty -WARNING: Error loading config file:/root/.docker/config.json - illegal base64 data at input byte 0 -The push refers to a repository [docker.io/example/empty] -547b6288eb33: Layer already exists -errors: -denied: requested access to the resource is denied -unauthorized: authentication required -root@hello:~/workspace/dockerfile# docker login -Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. -Username: example -Password: -Login Succeeded -root@hello:~/workspace/dockerfile# docker push example/empty -The push refers to a repository [docker.io/example/empty] -547b6288eb33: Layer already exists -latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524 -``` diff --git a/docs/en/docs/Container/container-management-2.md b/docs/en/docs/Container/container-management-2.md deleted file mode 100644 index dc3061cccbc065086d1fda0dcab047be930a7fd0..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/container-management-2.md +++ /dev/null @@ -1,1256 +0,0 @@ -# Container Management - -- [Container Management](#container-management) - - [attach](#attach) - - [commit](#commit) - - [cp](#cp) - - [create](#create) - - [diff](#diff) - - [exec](#exec) - - [export](#export) - - [inspect](#inspect) - - [logs](#logs) - - [pause/unpause](#pauseunpause) - - [port](#port) - - [ps](#ps) - - [rename](#rename) - - [restart](#restart) - - [rm](#rm) - - [run](#run) - - [start](#start) - - [stats](#stats) - - [stop](#stop) - - [top](#top) - - [update](#update) - - [wait](#wait) - -Subcommands supported by the current Docker are classified into the following groups by function: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Function

-

Command

-

Description

-

Host environment

-

version

-

Views the Docker version.

-

info

-

Views the Docker system and host environment information.

-

Container-related information

-

Container lifecycle management

-

create

-

Creates a container using an image.

-

run

-

Creates and runs a container using an image.

-

start

-

Starts a stopped container.

-

stop

-

Stops a running container.

-

restart

-

Restarts a container.

-

wait

-

Waits for a container to stop and prints the exit code.

-

rm

-

Deletes a container.

-

Container process management

-

pause

-

Suspends all processes in a container.

-

unpause

-

Resumes a suspended process in a container.

-

top

-

Views processes in a container.

-

exec

-

Executes a process in containers.

-

Container inspection tool

-

ps

-

Views running containers (without attaching any option).

-

logs

-

Displays the log information of a container.

-

attach

-

Connects standard input and output to a container.

-

inspect

-

Returns the bottom-layer information of a container.

-

port

-

Lists the port mappings between containers and hosts.

-

diff

-

Returns the changes made by the container compared with rootfs in the image.

-

cp

-

Copies files between containers and hosts.

-

export

-

Exports the file system in a container in a .tar package.

-

stats

-

Views the resource usage of a container in real time.

-

Images

-

Generates an image.

-

build

-

Creates an image using a Dockerfile.

-

commit

-

Creates an image based on the container rootfs.

-

import

-

Creates an image using the content in the .tar package as the file system.

-

load

-

Loads an image from the .tar package.

-

Image registry

-

login

-

Logs in to a registry.

-

logout

-

Logs out of a registry.

-

pull

-

Pulls an image from the registry.

-

push

-

Pushes an image to the registry.

-

search

-

Searches for an image in the registry.

-

Image management

-

images

-

Displays images in the system.

-

history

-

Displays the change history of an image.

-

rmi

-

Deletes an image.

-

tag

-

Adds a tag to an image.

-

save

-

Saves an image to a .tar package.

-

Others

-

events

-

Obtains real-time events from the Docker daemon.

-

rename

-

Renames a container.

-
- -Some subcommands have some parameters, such as **docker run**. You can run the **docker **_command _**--help** command to view the help information of the command. For details about the command parameters, see the preceding command parameter description. The following sections describe how to use each command. - -## attach - -Syntax: **docker attach \[**_options_**\]** _container_ - -Function: Attaches an option to a running container. - -Parameter description: - -**--no-stdin=false**: Does not attach any STDIN. - -**--sig-proxy=true**: Proxies all signals of the container, except SIGCHLD, SIGKILL, and SIGSTOP. - -Example: - -```shell -$ sudo docker attach attach_test -root@2988b8658669:/# ls bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var -``` - -## commit - -Syntax: **docker commit \[**_options_**\] **_container _**\[**_repository\[:tag\]_**\]** - -Function: creates an image from a container. - -Parameter description: - -**-a**, **--author=""**: specifies an author. - -**-m**, **--message=""**: specifies the submitted information. - -**-p**, **--pause=true**: pauses the container during submission. - -Example: - -Run the following command to start a container and submit the container as a new image: - -```shell -$ sudo docker commit test busybox:test -sha256:be4672959e8bd8a4291fbdd9e99be932912fe80b062fba3c9b16ee83720c33e1 - -$ sudo docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox latest e02e811dd08f 2 years ago 1.09MB -``` - -## cp - -Syntax: **docker cp \[**_options_**\] **_container_**:**_src\_path_ _dest\_path_**|-** - -**docker cp \[**_options_**\]** _src\_path_**|-** _container_**:**_dest\_path_ - -Function: Copies a file or folder from a path in a container to a path on the host or copies a file or folder from the host to the container: - -Precautions: The **docker cp** command does not support the copy of files in virtual file systems such as **/proc**, **/sys**, **/dev**, and **/tmp** in the container and files in the file systems mounted by users in the container. - -Parameter description: - -**-a**, **--archive**: Sets the owner of the file copied to the container to the **container** user \(**--user**\). - -**-L**, **--follow-link**: Parses and traces the symbolic link of a file. - -Example: - -Run the following command to copy the **/test** directory in the registry container to the **/home/**_aaa_ directory on the host: - -```shell -sudo docker cp registry:/test /home/aaa -``` - -## create - -Syntax: **docker create \[**_options_**\]** _image_ **\[**_command_**\] \[**_arg_**...\]** - -Function: Creates a container using an image file and return the ID of the container. After the container is created, run the **docker start** command to start the container. _options_ are used to configure the container during container creation. Some parameters will overwrite the container configuration in the image file. _command_ indicates the command to be executed during container startup. - -Parameter description: - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

-a --attach=[]

-

Attaches the console to the STDIN, STDOUT, and STDERR of the process in the container.

-

--name=""

-

Name of a container.

-

--add-host=[host:ip]

-

Adds a mapping between the host name and IP address to the /etc/hosts in the container.

-

For example, --add-host=test:10.10.10.10.

-

--annotation

-

Sets annotations for the container. For example, set the native.umask parameter.

-
--annotation native.umask=normal #The umask value of the started container is 0022.
---annotation native.umask=secure #The umask value of the started container is 0027.
-

If this parameter is not set, the umask configuration in dockerd is used.

-

--blkio-weight

-

Relative weight of blockio, which ranges from 10 to 1000.

-

--blkio-weight-device=[]

-

Blockio weight, which configures the relative weight.

-

-c, --cpu-shares=0

-

Relative weight of the host CPU obtained by the container. This parameter can be used to obtain a higher priority. By default, all containers obtain the same CPU priority.

-

--cap-add=[]

-

Adds Linux functions.

-

--cap-drop=[]

-

Clears Linux functions.

-

--cgroup-parent

-

cgroup parent directory for the container.

-

--cidfile=""

-

Writes the container ID to a specified file.

-

For example: --cidfile=/home/cidfile-test writes the container ID to the /home/cidfile-test file.

-

--cpu-period

-

CPU CFS period.

-

The default value is 100 ms. Generally, --cpu-period and --cpu-quota are used together. For example, --cpu-period=50000 --cpu-quota=25000 indicates that if there is one CPU, the container can obtain 50% of the CPU every 50 ms.

-

--cpus=0.5 has the same effect.

-

--cpu-quota

-

CPU CFS quota. The default value is 0, indicating that there is no restriction on the quota.

-

--cpuset-cpus

-

Number of CPUs (0-3, 0, 1) that can be used by processes in the container. By default, there is no restriction on this parameter.

-

--cpuset-mems

-

Memory nodes (0-3, 0, 1) for running processes in the container. This parameter is valid only for the NUMA system.

-

--device=[]

-

Adds the host device to a container, for example, --device=/dev/sdc:/dev/xvdc:rwm.

-

--dns=[]

-

Forcibly enables the container to use the specified DNS server. For example, --dns=114.114.xxx.xxx indicates that nameserver 114.114.xxx.xxx is written to /etc/resolv.conf of the created container and the original content is overwritten.

-

--dns-opt=[]

-

DNS options.

-

--dns-search=[]

-

Forcibly searches DNS search domain name used by a container.

-

-e, --env=[]

-

Sets environment variable for the container.

-

--env=[KERNEL_MODULES=]:

-

Inserts a specified module into a container. Currently, only the modules on the host can be inserted. After the container is deleted, the modules still reside on the host, and the --hook-spec option must be configured for the container. The following are valid parameter formats:

-

KERNEL_MODULERS=

-

KERNEL_MODULERS=a

-

KERNEL_MODULERS=a,b

-

KERNEL_MODULERS=a,b,

-

--entrypoint=""

-

Overwrites the original entrypoint in the image. The entrypoint is used to set the command executed when the container is started.

-

--env-file=[]

-

Reads environment variables from a file. Multiple environment variables are separated by lines in the file. For example: --env-file=/home/test/env indicates multiple environment variables are stored in the env file.

-

--expose=[]

-

Enables an internal port of a container. The -P option described in the following section maps the enabled port to a port on the host.

-

--group-add=[]

-

Adds a specified container to an additional group.

-

-h, --hostname=""

-

Host name.

-

--health-cmd

-

Container health check command.

-

--health-interval

-

Interval between two consecutive command executions. The default value is 30s.

-

--health-timeout

-

Maximum duration for executing a single check command. If the execution times out, the command fails to be executed. The default value is 30s.

-

--health-start-period

-

Interval between the time when the container is started and the time when the first health check is performed. The default value is 0s.

-

--health-retries

-

Maximum number of retries after a health check fails. The default value is 3.

-

--health-exit-on-unhealthy

-

Specifies whether to stop a container when the container is unhealthy. The default value is false.

-

--host-channel=[]

-

Sets a channel for communication between processes in the container and the host, in host path:container path:rw/ro:size limit format.

-

-i, --interactive=false

-

Enables STDIN even if it is not attached.

-

--ip

-

IPv4 address of a container.

-

--ip6

-

IPv6 address of a container.

-

--ipc

-

IPC namespace of a container.

-

--isolation

-

Container isolation policy.

-

-l, --label=[]

-

Label of a container.

-

--label-file=[]

-

Obtains the label from the file.

-

--link=[]

-

Links to another container. This parameter adds environment variables of the IP address and port number of the linked container to the container and adds a mapping to the /etc/hosts file, for example, --link=name:alias.

-

--log-driver

-

Log driver of a container.

-

--log-opt=[]

-

Log driver option.

-

-m, --memory=""

-

Memory limit of a container. The format is numberoptional unit, and available units are b, k, m, and g. The minimum value of this parameter is 4m.

-

--mac-address

-

MAC address of a container, for example, 92:d0:c6:0a:xx:xx.

-

--memory-reservation

-

Container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory.

-

--memory-swap

-

Total usage of the common memory and swap partition. -1 indicates no restriction is set on the usage. If this parameter is not set, the swap partition size is twice the value of --memory. That is, the swap partition can use the same amount of memory as --memory.

-

--memory-swappiness=-1

-

Time when the container uses the swap memory. The value ranges from 0 to 100, in percentage.

-

--net="bridge"

-

Network mode of the container. Docker 1.3.0 has the following network modes: bridge, host, none, and container:name|id. The default value is bridge.

-
  • bridge: Creates a network stack on the bridge when the Docker daemon is started.
  • host: Uses the network stack of the host in the container.
  • none: Does not use networks.
  • container:name|id: Reuses the network stack of another container.
-

--no-healthcheck

-

Does not perform health check for a container.

-

--oom-kill-disable

-

Disables the OOM killer. You are advised not to set this parameter if the -m parameter is not set.

-

--oom-score-adj

-

Adjusts the OOM rule of a container. The value ranges from -1000 to 1000.

-

-P, --publish-all=false

-

Maps all enabled ports of a container to host ports. Containers can be accessed through the host ports. You can run the docker port command to view the mapping between container ports and host ports.

-

-p, --publish=[]

-

Maps a port in a container to a port on the host, in IP address:host port:container port | IP address::container port | host port:container port | container port format. If no IP address is configured, accesses of all NICs on the host is listened. If no host port is configured, the host port is automatically allocated.

-

--pid

-

PID namespace of a container.

-

--privileged=false

-

Grants extra permission to a container. If the --privileged option is used, the container can access all devices on the host.

-

--restart=""

-

Configures restart rule when the container exits. Currently, version 1.3.1 supports the following rules:

-
  • no: indicates that the container is not restarted when it is stopped.
  • on-failure: indicates that the container is restarted when the container exit code is not 0. This rule can be used to add the maximum number of restart times, for example, on-failure:5, indicating that the container can be restarted for a maximum of five times.
  • always: indicates the container is exited regardless of the exit code.
-

--read-only

-

Mounts the root file system of the container in read-only mode.

-

--security-opt=[]

-

Container security rule.

-

--shm-size

-

Size of the /dev/shm device. The default value is 64M.

-

--stop-signal=SIGTERM

-

Container stop signal. The default value is SIGTERM.

-

-t, --tty=false

-

Allocates a pseudo terminal.

-

--tmpfs=[]

-

Mounts the tmpfs directory.

-

-u, --user=""

-

User name or user ID.

-

--ulimit=[]

-

ulimit option.

-

--userns

-

User namespace of a container.

-

-v, --volume=[]

-

Mounts a directory of the host to the container, or create a volume in the container. For example, -v /home/test:/home mounts the /home/test directory of the host to the /home directory of the container, and -v /tmp creates the tmp folder in the root directory of the container, the folder can be shared by other containers using the --volumes-from option. The host directory cannot be mounted to the /proc subdirectory of the container. Otherwise, an error is reported when the container is started.

-

--volume-driver

-

Data volume driver of the container. This parameter is optional.

-

--volumes-from=[]

-

Mounts the volume of another container to the current container to share the volume. For example, -volumes-from container_name mounts the volume of container_name to the current container. -v and --volumes-from=[] are two very important options for data backup and live migration.

-

-w, --workdir=""

-

Specifies the working directory of the container.

-
- -Example: - -Run the following command to create a container named **busybox** and run the **docker start** command to start the container. - -```shell -sudo docker create -ti --name=busybox busybox /bin/bash -``` - -## diff - -Syntax: **docker diff** _container_ - -Function: Checks the differences between containers and determines the changes have been made compared with the container creation. - -Parameter description: none. - -Example: - -```shell -$ sudo docker diff registry -C /root -A /root/.bash_history -A /test -``` - -## exec - -Syntax: **docker exec \[**_options_**\]** _container_ _command_ **\[**_arg..._**\]** - -Function: Runs a command in the container. - -Parameter description: - -**-d** and **--detach=false**: Run in the background. - -**-i** and **--interactive=false**: Keep the STDIN of the container enabled. - -**-t** and **--tty=false**: Allocate a virtual terminal. - -**--privileged**: Executes commands in privilege mode. - -**-u** and **--user**: Specifies the user name or UID. - -Example: - -```shell -$ sudo docker exec -ti exec_test ls -bin etc lib media opt root sbin sys tmp var -dev home lib64 mnt proc run srv test usr -``` - -## export - -Syntax: **docker export** _container_ - -Function: Exports the file system content of a container to STDOUT in .tar format. - -Parameter description: none. - -Example: - -Run the following commands to export the contents of the container named **busybox** to the **busybox.tar** package: - -```shell -$ sudo docker export busybox > busybox.tar -$ ls -busybox.tar -``` - -## inspect - -Syntax: **docker inspect \[**_options_**\] **_container_**|**_image_**\[**_container_|_image..._**\]** - -Function: Returns the underlying information about a container or image. - -Parameter description: - -**-f** and **--format=""**: Output information in a specified format. - -**-s** and **--size**: Display the total file size of the container when the query type is container. - -**--type**: Returns the JSON format of the specified type. - -**-t** and **--time=120**: Timeout interval, in seconds. If the **docker inspect** command fails to be executed within the timeout interval, the system stops waiting and immediately reports an error. The default value is **120**. - -Example: - -1. Run the following command to return information about a container: - - ```shell - $ sudo docker inspect busybox_test - [ - { - "Id": "9fbb8649d5a8b6ae106bb0ac7686c40b3cbd67ec2fd1ab03e0c419a70d755577", - "Created": "2019-08-28T07:43:51.27745746Z", - "Path": "bash", - "Args": [], - "State": { - "Status": "running", - "Running": true, - "Paused": false, - "Restarting": false, - "OOMKilled": false, - "Dead": false, - "Pid": 64177, - "ExitCode": 0, - "Error": "", - "StartedAt": "2019-08-28T07:43:53.021226383Z", - "FinishedAt": "0001-01-01T00:00:00Z" - }, - ...... - ``` - -2. Run the following command to return the specified information of a container in a specified format. The following uses the IP address of the busybox\_test container as an example. - - ```shell - $ sudo docker inspect -f {{.NetworkSettings.IPAddress}} busybox_test - 172.17.0.91 - ``` - -## logs - -Syntax: **docker logs \[**_options_**\]** _container_ - -Function: Captures logs in a container that is in the **running** or **stopped** state. - -Parameter description: - -**-f** and **--follow=false**: Print logs in real time. - -**-t** and **--timestamps=false**: Display the log timestamp. - -**--since**: Displays logs generated after the specified time. - -**--tail="all"**: Sets the number of lines to be displayed. By default, all lines are displayed. - -Example: - -1. Run the following command to check the logs of the jaegertracing container where a jaegertracing service runs: - - ```shell - $ sudo docker logs jaegertracing - {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"} - {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}} - {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"} - {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"} - {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"} - {"level":"info","ts":1566979103.4047635,"caller":"all-in-one/main.go:226","msg":"Starting jaeger-collector TChannel server","port":14267} - {"level":"info","ts":1566979103.404901,"caller":"all-in-one/main.go:236","msg":"Starting jaeger-collector HTTP server","http-port":14268} - {"level":"info","ts":1566979103.4577134,"caller":"all-in-one/main.go:256","msg":"Listening for Zipkin HTTP traffic","zipkin.http-port":9411} - ``` - -2. Add **-f** to the command to output the logs of the jaegertracing container in real time. - - ```shell - $ sudo docker logs -f jaegertracing - {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"} - {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}} - {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"} - {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"} - {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"} - ``` - -## pause/unpause - -Syntax: **docker pause** _container_ - -**docker unpause** _container_ - -Function: The two commands are used in pairs. The **docker pause** command suspends all processes in a container, and the **docker unpause** command resumes the suspended processes. - -Parameter description: none. - -Example: - -The following uses a container where the docker registry service runs as an example. After the **docker pause** command is executed to pause the process of the container, access of the registry service by running the **curl** command is blocked. You can run the **docker unpause** command to resume the suspended registry service. The registry service can be accessed by running the **curl** command. - -1. Run the following command to start a registry container: - - ```shell - sudo docker run -d --name pause_test -p 5000:5000 registry - ``` - - Run the **curl** command to access the service. Check whether the status code **200 OK** is returned. - - ```shell - sudo curl -v 127.0.0.1:5000 - ``` - -2. Run the following command to stop the processes in the container: - - ```shell - sudo docker pause pause_test - ``` - - Run the **curl** command to access the service to check whether it is blocked and wait until the service starts. - -3. Run the following command to resume the processes in the container: - - ```shell - sudo docker unpause pause_test - ``` - - The cURL access in step 2 is restored and the request status code **200 OK** is returned. - -## port - -Syntax: **docker port **_container_ **\[**_private\_port\[/proto\]_**\]** - -Function: Lists the port mapping of a container or queries the host port where a specified port resides. - -Parameter description: none. - -Example: - -1. Run the following command to list all port mappings of a container: - - ```shell - $ sudo docker port registry - 5000/tcp -> 0.0.0.0.:5000 - ``` - -2. Run the following command to query the mapping of a specified container port: - - ```shell - $ sudo docker port registry 5000 - 0.0.0.0.:5000 - ``` - -## ps - -Syntax:**docker ps \[**_options_**\]** - -Function: Lists containers in different states based on different parameters. If no parameter is added, all running containers are listed. - -Parameter description: - -**-a** and **--all=false**: Display the container. - -**-f** and **--filter=\[\]**: Filter values. The available options are: **exited=**_int_ \(exit code of the container\) **status=**_restarting|running|paused|exited_ \(status code of the container\), for example, **-f status=running**: lists the running containers. - -**-l** and **--latest=false**: List the latest created container. - -**-n=-1**: Lists the latest created _n_ containers. - -**--no-trunc=false**: Displays all 64-bit container IDs. By default, 12-bit container IDs are displayed. - -**-q** and **--quiet=false**: Display the container ID. - -**-s** and **--size=false**: Display the container size. - -Example: - -1. Run the following command to lists running containers: - - ```shell - sudo docker ps - ``` - -2. Run the following command to display all containers: - - ```shell - sudo docker ps -a - ``` - -## rename - -Syntax: **docker rename OLD\_NAME NEW\_NAME** - -Function: Renames a container. - -Example: - -Run the **docker run** command to create and start a container, run the **docker rename** command to rename the container, and check whether the container name is changed. - -```shell -$ sudo docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -b15976967abb busybox:latest "bash" 3 seconds ago Up 2 seconds festive_morse -$ sudo docker rename pedantic_euler new_name -$ sudo docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -b15976967abb busybox:latest "bash" 34 seconds ago Up 33 seconds new_name -``` - -## restart - -Syntax: **docker restart \[**_options_**\]** _container_ **\[**_container..._**\]** - -Function: Restarts a running container. - -Parameter description: - -**-t** and **--time=10**: Number of seconds to wait for the container to stop before the container is killed. If the container has stopped, restart the container. The default value is **10**. - -Example: - -```shell -sudo docker restart busybox -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->During the container restart, if a process in the **D** or **Z** state exists in the container, the container may fail to be restarted. In this case, you need to analyze the cause of the **D** or **Z** state of the process in the container. Restart the container after the **D** or **Z** state of the process in the container is released. - -## rm - -Syntax: **docker rm \[**_options_**\] **_container_ **\[**_container..._**\]** - -Function: Deletes one or more containers. - -Parameter description: - -**-f** and **--force=false**: Forcibly delete a running container. - -**-l** and **--link=false**: Remove the specified link and do not remove the underlying container. - -**-v** and **--volumes=false**: Remove the volumes associated with the container. - -Example: - -1. Run the following command to delete a stopped container: - - ```shell - sudo docker rm test - ``` - -2. Run the following command to delete a running container: - - ```shell - sudo docker rm -f rm_test - ``` - -## run - -Syntax: **docker run \[**_options_**\] **_image_ **\[**_command_**\] \[**_arg_**...\]** - -Function: Creates a container from a specified image \(if the specified image does not exist, an image is downloaded from the official image registry\), starts the container, and runs the specified command in the container. This command integrates the **docker create**, **docker start**, and **docker exec** commands. - -Parameter description: \(The parameters of this command are the same as those of the **docker create** command. For details, see the parameter description of the **docker create** command. Only the following two parameters are different.\) - -**--rm=false**: Specifies the container to be automatically deleted when it exits. - -**-v**: Mounts a local directory or an anonymous volume to the container. Note: When a local directory is mounted to a container with a SELinux security label, do not add or delete the local directory at the same time. Otherwise, the security label may not take effect. - -**--sig-proxy=true**: Receives proxy of the process signal. SIGCHLD, SIGSTOP, and SIGKILL do not use the proxy. - -Example: - -Run the busybox image to start a container and run the **/bin/sh** command after the container is started: - -```shell -sudo docker run -ti busybox /bin/sh -``` - -## start - -Syntax: **docker start \[**_options_**\]** _container_ **\[**_container_**...\]** - -Function: Starts one or more containers that are not running. - -Parameter description: - -**-a** and **--attach=false**: Attach the standard output and error output of a container to STDOUT and STDERR of the host. - -**-i** and **--interactive=false**: Attach the standard input of the container to the STDIN of the host. - -Example: - -Run the following command to start a container named **busybox** and add the **-i -a** to the command to add standard input and output. After the container is started, directly enter the container. You can exist the container by entering **exit**. - -If **-i -a** is not added to the command when the container is started, the container is started in the background. - -```shell -sudo docker start -i -a busybox -``` - -## stats - -Syntax: **docker stats \[**_options_**\] \[**_container_**...\]** - -Function: Continuously monitors and displays the resource usage of a specified container. \(If no container is specified, the resource usage of all containers is displayed by default.\) - -Parameter description: - -**-a**, and **--all**: Display information about all containers. By default, only running containers are displayed. - -**--no-stream**: Displays only the first result and does not continuously monitor the result. - -Example: - -Run the **docker run** command to start and create a container, and run the **docker stats** command to display the resource usage of the container: - -```shell -$ sudo docker stats -CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS -2e242bcdd682 jaeger 0.00% 77.08MiB / 125.8GiB 0.06% 42B / 1.23kB 97.9MB / 0B 38 -02a06be42b2c relaxed_chandrasekhar 0.01% 8.609MiB / 125.8GiB 0.01% 0B / 0B 0B / 0B 10 -deb9e49fdef1 hardcore_montalcini 0.01% 12.79MiB / 125.8GiB 0.01% 0B / 0B 0B / 0B 9 -``` - -## stop - -Syntax: **docker stop \[**_options_**\]** _container_ **\[**_container_**...\]** - -Function: Sends a SIGTERM signal to a container and then sends a SIGKILL signal to stop the container after a certain period. - -Parameter description: - -**-t** and **--time=10**: Number of seconds that the system waits for the container to exit before the container is killed. The default value is **10**. - -Example: - -```shell -sudo docker stop -t=15 busybox -``` - -## top - -Syntax: **docker top** _container_ **\[**_ps options_**\]** - -Function: Displays the processes running in a container. - -Parameter description: none. - -Example: - -Run the top\_test container and run the **top** command in the container. - -```shell -$ sudo docker top top_test -UID PID PPID C STIME TTY TIME CMD -root 70045 70028 0 15:52 pts/0 00:00:00 bash -``` - -The value of **PID** is the PID of the process in the container on the host. - -## update - -Syntax: **docker update \[**_options_**\]** _container_ **\[**_container_**...\]** - -Function: Hot changes one or more container configurations. - -Parameter description: - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

--accel=[]

-

Configures one or more container accelerators.

-

--blkio-weight

-

Relative weight of the container blockio. The value ranges from 10 to 1000.

-

--cpu-shares

-

Relative weight of the host CPU obtained by the container. This parameter can be used to obtain a higher priority. By default, all containers obtain the same CPU priority.

-

--cpu-period

-

CPU CFS period.

-

The default value is 100 ms. Generally, --cpu-period and --cpu-quota are used together. For example, --cpu-period=50000 --cpu-quota=25000 indicates that if there is one CPU, the container can obtain 50% of the CPU every 50 ms.

-

--cpu-quota

-

CPU CFS quota. The default value is 0, indicating that there is no restriction on the quota.

-

--cpuset-cpus

-

Number of CPUs (0-3, 0, 1) that can be used by processes in the container. By default, there is no restriction on this parameter.

-

--cpuset-mems

-

Memory nodes (0-3, 0, 1) for running processes in the container. This parameter is valid only for the NUMA system.

-

--kernel-memory=""

-

Kernel memory limit of a container. The format is numberoptional unit, and available units are b, k, m, and g.

-

-m, --memory=""

-

Memory limit of a container. The format is numberoptional unit, and available units are b, k, m, and g. The minimum value of this parameter is 4m.

-

--memory-reservation

-

Container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory.

-

--memory-swap

-

Total usage of the common memory and swap partition. -1 indicates no restriction is set on the usage. If this parameter is not set, the swap partition size is twice the value of --memory. That is, the swap partition can use the same amount of memory as --memory.

-

--restart=""

-

Configures restart rule when the container exits. Currently, version 1.3.1 supports the following rules:

-
  • no: indicates that the container is not restarted when it is stopped.
  • on-failure: indicates that the container is restarted when the container exit code is not 0. This rule can be used to add the maximum number of restart times, for example, on-failure:5, indicating that the container can be restarted for a maximum of five times.
  • always: indicates the container is exited regardless of the exit code.
-

--help

-

Help information.

-
- -Example: - -Run the following command to change the CPU and memory configurations of the container named **busybox**, including changing the relative weight of the host CPU obtained by the container to **512**, the CPU cores that can be run by processes in the container to **0,1,2,3**, and the memory limit for running the container to **512 m**. - -```shell -sudo docker update --cpu-shares 512 --cpuset-cpus=0,3 --memory 512m ubuntu -``` - -## wait - -Syntax: **docker wait** _container_ **\[**_container..._**\]** - -Function: Waits for a container to stop and print the exit code of the container: - -Parameter description: none. - -Example: - -Run the following command to start a container named **busybox**: - -```shell -sudo docker start -i -a busybox -``` - -Run the **docker wait** command: - -```shell -$ sudo docker wait busybox -0 -``` - -Wait until the busybox container exits. After the busybox container exits, the exit code **0** is displayed. diff --git a/docs/en/docs/Container/container-management.md b/docs/en/docs/Container/container-management.md deleted file mode 100644 index 019bb44e2db953f8d1a7ae94b64ebfefa0c1a15d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/container-management.md +++ /dev/null @@ -1,2504 +0,0 @@ -# Container Management - -- [Container Management](#container-management) - - [Creating a Container](#creating-a-container) - - [Starting a Container](#starting-a-container) - - [Running a Container](#running-a-container) - - [Stopping a Container](#stopping-a-container) - - [Forcibly Stopping a Container](#forcibly-stopping-a-container) - - [Removing a Container](#removing-a-container) - - [Attaching to a Container](#attaching-to-a-container) - - [Renaming a Container](#renaming-a-container) - - [Executing a Command in a Running Container](#executing-a-command-in-a-running-container) - - [Querying Information About a Single Container](#querying-information-about-a-single-container) - - [Querying Information About All Containers](#querying-information-about-all-containers) - - [Restarting a Container](#restarting-a-container) - - [Waiting for a Container to Exit](#waiting-for-a-container-to-exit) - - [Viewing Process Information in a Container](#viewing-process-information-in-a-container) - - [Displaying Resource Usage Statistics of a Container](#displaying-resource-usage-statistics-of-a-container) - - [Obtaining Container Logs](#obtaining-container-logs) - - [Copying Data Between a Container and a Host](#copying-data-between-a-container-and-a-host) - - [Pausing All Processes in a Container](#pausing-all-processes-in-a-container) - - [Resuming All Processes in a Container](#resuming-all-processes-in-a-container) - - [Obtaining Event Messages from the Server in Real Time](#obtaining-event-messages-from-the-server-in-real-time) - -## Creating a Container - -### Description - -To create a container, run the **isula create** command. The container engine will use the specified container image to create a read/write layer, or use the specified local rootfs as the running environment of the container. After the creation is complete, the container ID is output as standard output. You can run the **isula start** command to start the container. The new container is in the **inited** state. - -### Usage - -```shell -isula create [OPTIONS] IMAGE [COMMAND] [ARG...] -``` - -### Parameters - -The following table lists the parameters supported by the **create** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

create

-

  

-

--add-host

-

Adds the mapping between the custom host and the IP address (host:ip).

-

--annotation

-

Sets annotations for the container. For example, set the native.umask parameter.

-
--annotation native.umask=normal #The umask value of the started container is 0022.
---annotation native.umask=secure #The umask value of the started container is 0027.
-

If this parameter is not set, the umask configuration in iSulad is used.

-

--blkio-weight

-

Specifies the block I/O (relative weight). The value ranges from 10 to 1000. The default value is 0, indicating that this function is disabled.

-

--blkio-weight-device

-

Specifies the block I/O weight (relative device weight). The format is DEVICE_NAME: weight. The weight value ranges from 10 to 1000. The default value is 0, indicating that this function is disabled.

-

--cap-add

-

Adds the Linux permission function.

-

--cap-drop

-

Deletes Linux permissions.

-

--cgroup-parent

-

Specifies the cgroup parent path of the container.

-

--cpu-period

-

Limits the period of CPU CFS.

-

--cpu-quota

-

Limits the CPU CFS quota.

-

--cpu-rt-period

-

Limits the real-time CPU period (in microseconds).

-

--cpu-rt-runtime

-

Limits the real-time running time of the CPU (in microseconds).

-

--cpu-shares

-

CPU share (relative weight).

-

--cpus

-

Specifies the number of CPUs.

-

--cpuset-cpus

-

Specifies the CPU that can be executed. Example values: 0-3, 0, 1.

-

--cpuset-mems

-

Specifies memory that can be executed. Example values: 0-3, 0, 1.

-

--device

-

Adds a device to the container.

-

--device-cgroup-rule

-

Adds a rule to the list of devices allowed by the cgroup.

-

--device-read-bps

-

Limits the read rate (bytes per second) of the device.

-

--device-read-iops

-

Limits the read rate (I/Os per second) of the device.

-

--device-write-bps

-

Limits the write rate (bytes per second) of the device.

-

--device-write-iops

-

Limits the write rate (I/Os per second) of the device.

-

--dns

-

Adds a DNS server.

-

--dns-opt

-

Adds DNS options.

-

--dns-search

-

Sets the search domain of a container.

-

--entrypoint

-

Specifies the entry point to be run when a container is started.

-

-e, --env

-

Sets environment variables.

-

--env-file

-

Configures environment variables using a file.

-

--env-target-file

-

Specifies the target file path in rootfs to which environment variables are exported.

-

--external-rootfs=PATH

-

Specifies a rootfs (a folder or block device) that is not managed by iSulad for the container.

-

--files-limit

-

Limits the number of file handles that can be opened in a container. The value -1 indicates no limit.

-

--group-add=[]

-

Adds additional user groups to the container.

-

--help

-

Prints help information.

-

--health-cmd

-

Command executed in a container.

-

--health-exit-on-unhealthy

-

Determines whether to kill a container when the container is detected unhealthy.

-

--health-interval

-

Interval between two consecutive command executions.

-

--health-retries

-

Maximum number of health check retries.

-

--health-start-period

-

Container initialization interval.

-

--health-timeout

-

Maximum time for executing a single check command.

-

--hook-spec

-

Hook configuration file.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--host-channel

-

Creates the shared memory between the host and the container.

-

-h, --hostname

-

Container host name.

-

--hugetlb-limit=[]

-

Limits the huge page file. For example, --hugetlb-limit 2MB:32MB.

-

-i, --interactive

-

Enables the standard input of the container even if it is not connected to the standard input of the container.

-

--ipc

-

Specifies the IPC namespace.

-

--kernel-memory

-

Limits the kernel memory.

-

-l,--label

-

Sets a label for a container.

-

--lablel-file

-

Sets container labels using files.

-

--log-driver

-

Records the container driver.

-

--log-opt=[]

-

Log driver option. By default, the function of recording container serial port logs is disabled. You can enable it by setting --log-opt disable-log=false.

-

-m, --memory

-

Memory limit.

-

--memory-reservation

-

Sets the container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory. The minimum value is 4 MB.

-

--memory-swap

-

Memory swap space, which should be a positive integer. The value -1 indicates no limit.

-

--memory-swappiness

-

The value of swappiness is a positive integer ranging from 0 to 100. The smaller the value is, the less the swap partition is used and the more the memory is used in the Linux system. The larger the value is, the more the swap space is used by the kernel. The default value is –1, indicating that the default system value is used.

-

--mount

-

Mounts the host directory, volume, or file system to the container.

-

--name=NAME

-

Container name.

-

--net=none

-

Connects the container to the network.

-

--no-healthcheck

-

Disables the health check configuration.

-

--ns-change-opt

-

Namespace kernel parameter option of the system container.

-

--oom-kill-disable

-

Disables OOM.

-

--oom-score-adj

-

Adjusts the OOM preference of the host (from -1000 to 1000).

-

--pid

-

Specifies the PID namespace to be used.

-

--pids-limit

-

Limits the number of processes that can be executed in the container. The value -1 indicates no limit.

-

--privileged

-

Grants container extension privileges.

-

--pull

-

Pulls the image before running.

-

-R, --runtime

-

Container runtime. The parameter value can be lcr, which is case insensitive. Therefore, LCR and lcr are equivalent.

-

--read-only

-

Sets the rootfs of a container to read-only.

-

--restart

-

Restart policy upon container exit.

-

For a system container, --restart on-reboot is supported.

-

--security-opt

-

Security option.

-

--shm-size

-

Size of /dev/shm. The default value is 64MB.

-

--stop-signal

-

Stop signal for a container. The default value is SIGTERM.

-

--storage-opt

-

Configures the storage driver option for a container.

-

--sysctl

-

Sets the sysctl option.

-

--system-container

-

Starts the system container.

-

--tmpfs

-

Mounts the tmpfs directory.

-

-t, --tty

-

Allocates a pseudo terminal.

-

--ulimit

-

Sets the ulimit for a container.

-

-u, --user

-

User name or UID, in the format of [<name|uid>][:<group|gid>].

-

--user-remap

-

Maps users to the system container.

-

--userns

-

Sets the user command space for a container when the user-remap option is enabled.

-

--uts

-

Sets the PID namespace.

-

-v, --volume=[]

-

Mounts a volume.

-

--volumes-from=[]

-

Uses the mounting configuration of the specified container.

-

--workdir

-

Sets the working directory in the container.

-
- -### Constraints - -- When the **--user** or **--group-add** parameter is used to verify the user or group during container startup, if the container uses an OCI image, the verification is performed in the **etc/passwd** and **etc/group** files of the actual rootfs of the image. If a folder or block device is used as the rootfs of the container, the **etc/passwd** and **etc/group** files in the host are verified. The rootfs ignores mounting parameters such as **-v** and **--mount**. That is, when these parameters are used to attempt to overwrite the **etc/passwd** and **etc/group** files, the parameters do not take effect during the search and take effect only when the container is started. The generated configuration is saved in the **iSulad root directory/engine/container ID/start\_generate\_config.json** file. The file format is as follows: - - ```json - { - "uid": 0, - "gid": 8, - "additionalGids": [ - 1234, - 8 - ] - } - ``` - -### Example - -Create a container. - -```shell -$ isula create busybox -fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -$ isula ps -a -STATUS PID IMAGE COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID NAMES -``` - -## Starting a Container - -### Description - -To start one or more containers, run the **isula start** command. - -### Usage - -```shell -isula start [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **start** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

start

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-a, --attach

-

Connects to STDOUT and STDERR of the container.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-
- -### Example - -Start a new container. - -```shell -isula start fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -``` - -## Running a Container - -### Description - -To create and start a container, run the **isula run** command. You can use a specified container image to create a container read/write layer and prepare for running the specified command. After the container is created, run the specified command to start the container. The **run** command is equivalent to creating and starting a container. - -### Usage - -```shell -isula run [OPTIONS] ROOTFS|IMAGE [COMMAND] [ARG...] -``` - -### Parameters - -The following table lists the parameters supported by the **run** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

run

-

--annotation

-

Sets annotations for the container. For example, set the native.umask option.

-
--annotation native.umask=normal #The umask value of the started container is 0022.
---annotation native.umask=secure #The umask value of the started container is 0027.
-

If this parameter is not set, the umask configuration in iSulad is used.

-

--add-host

-

Adds the mapping between the custom host and the IP address (host:ip).

-

--blkio-weight

-

Specifies the block I/O (relative weight). The value ranges from 10 to 1000. The default value is 0, indicating that this function is disabled.

-

--blkio-weight-device

-

Specifies the block I/O weight (relative device weight). The format is DEVICE_NAME: weight. The weight value ranges from 10 to 1000. The default value is 0, indicating that this function is disabled.

-

--cap-add

-

Adds Linux functions.

-

--cap-drop

-

Deletes Linux functions.

-

--cgroup-parent

-

Specifies the cgroup parent path of the container.

-

--cpu-period

-

Limits the period of CPU CFS.

-

--cpu-quota

-

Limits the CPU CFS quota.

-

--cpu-rt-period

-

Limits the real-time CPU period (in microseconds).

-

--cpu-rt-runtime

-

Limits the real-time running time of the CPU (in microseconds).

-

--cpu-shares

-

CPU share (relative weight).

-

--cpus

-

Specifies the number of CPUs.

-

--cpuset-cpus

-

Specifies the CPU that can be executed. Example values: 0-3, 0, 1.

-

--cpuset-mems

-

Specifies memory that can be executed. Example values: 0-3, 0, 1.

-

-d, --detach

-

Runs the container in the background and displays the container ID.

-

--device=[]

-

Adds a device to the container.

-

--device-cgroup-rule

-

Adds a rule to the list of devices allowed by the cgroup.

-

--device-read-bps

-

Limits the read rate (bytes per second) of the device.

-

--device-read-iops

-

Limits the read rate (I/Os per second) of the device.

-

--device-write-bps

-

Limits the write rate (bytes per second) of the device.

-

--device-write-iops

-

Limits the write rate (I/Os per second) of the device.

-

--dns

-

Adds a DNS server.

-

--dns-opt

-

Adds DNS options.

-

--dns-search

-

Sets the search domain of a container.

-

--entrypoint

-

Specifies the entry point to be run when a container is started.

-

-e, --env

-

Sets environment variables.

-

--env-file

-

Configures environment variables using a file.

-

--env-target-file

-

Specifies the target file path in rootfs to which environment variables are exported.

-

--external-rootfs=PATH

-

Specifies a rootfs (a folder or block device) that is not managed by iSulad for the container.

-

--files-limit

-

Limits the number of file handles that can be opened in the container. The value -1 indicates no limit.

-

--group-add=[]

-

Adds additional user groups to the container.

-

--help

-

Prints help information.

-

--health-cmd

-

Command executed in a container.

-

--health-exit-on-unhealthy

-

Determines whether to kill a container when the container is detected unhealthy.

-

--health-interval

-

Interval between two consecutive command executions.

-

--health-retries

-

Maximum number of health check retries.

-

--health-start-period

-

Container initialization interval.

-

--health-timeout

-

Maximum time for executing a single check command.

-

--hook-spec

-

Hook configuration file.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

--host-channel

-

Creates the shared memory between the host and the container.

-

-h, --hostname

-

Container host name.

-

--hugetlb-limit=[]

-

Limits the size of huge-page files, for example, --hugetlb-limit 2MB:32MB.

-

-i, --interactive

-

Enables the standard input of the container even if it is not connected to the standard input of the container.

-

--ipc

-

Specifies the IPC namespace.

-

--kernel-memory

-

Limits the kernel memory.

-

-l, --label

-

Sets a label for a container.

-

--lablel-file

-

Sets the container label through a file.

-

--log-driver

-

Sets the log driver. syslog and json-file are supported.

-

--log-opt=[]

-

Log driver option. By default, the container serial port log function is disabled. You can run the --log-opt disable-log=false command to enable it.

-

-m, --memory

-

Memory limit.

-

--memory-reservation

-

Sets the container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory. The minimum value is 4 MB.

-

--memory-swap

-

Memory swap space, which should be a positive integer. The value -1 indicates no limit.

-

--memory-swappiness

-

The value of swappiness is a positive integer ranging from 0 to 100. The smaller the value is, the less the swap partition is used and the more the memory is used in the Linux system. The larger the value is, the more the swap space is used by the kernel. The default value is –1, indicating that the default system value is used.

-

--mount

-

Mounts a host directory to a container.

-

--name=NAME

-

Container name

-

--net=none

-

Connects a container to the network.

-

--no-healthcheck

-

Disables the health check configuration.

-

--ns-change-opt

-

Namespace kernel parameter option of the system container.

-

--oom-kill-disable

-

Disables OOM.

-

--oom-score-adj

-

Adjusts the OOM preference of the host (from -1000 to 1000).

-

--pid

-

Specifies the PID namespace to be used.

-

--pids-limit

-

Limits the number of processes that can be executed in the container. The value -1 indicates no limit.

-

--privileged

-

Grants container extension privileges.

-

--pull

-

Pulls the image before running.

-

-R, --runtime

-

Container runtime. The parameter value can be lcr, which is case insensitive. Therefore, LCR and lcr are equivalent.

-

--read-only

-

Sets the rootfs of a container to read-only.

-

--restart

-

Restart policy upon container exit.

-

For a system container, --restart on-reboot is supported.

-

--rm

-

Automatically clears a container upon exit.

-

--security-opt

-

Security option.

-

--shm-size

-

Size of /dev/shm. The default value is 64MB.

-

--stop-signal

-

Stop signal for a container. The default value is SIGTERM.

-

--storage-opt

-

Configures the storage driver option of a container.

-

--sysctl

-

Sets the sysctl option.

-

--system-container

-

Starts the system container.

-

--tmpfs

-

Mounts the tmpfs directory.

-

-t, --tty

-

Allocates a pseudo terminal.

-

--ulimit

-

Sets the ulimit for a container.

-

-u, --user

-

User name or UID, in the format of [<name|uid>][:<group|gid>].

-

--user-remap

-

Maps users to the system container.

-

--userns

-

Sets the user command space for a container when the user-remap option is enabled.

-

--uts

-

Sets the PID namespace.

-

-v, --volume=[]

-

Mounts a volume.

-

--volumes-from=[]

-

Uses the mounting configuration of the specified container.

-

--workdir

-

Sets the working directory in the container.

-
- -### Constraints - -- When the parent process of a container exits, the corresponding container automatically exits. -- When a common container is created, the parent process cannot be initiated because the permission of common containers is insufficient. As a result, the container does not respond when you run the **attach** command though it is created successfully. -- If **--net** is not specified when the container is running, the default host name is **localhost**. -- If the **--files-limit** parameter is to transfer a small value, for example, 1, when the container is started, iSulad creates a cgroup, sets the files.limit value, and writes the PID of the container process to the **cgroup.procs** file of the cgroup. At this time, the container process has opened more than one handle. As a result, a write error is reported, and the container fails to be started. -- If both**--mount** and **--volume** exist and their destination paths conflict, **--mount** will be run after **--volume** \(that is, the mount point in **--volume** will be overwritten\). - - Note: The value of the **type** parameter of lightweight containers can be **bind** or **squashfs**. When **type** is set to **squashfs**, **src** is the image path. The value of the **type** parameter of the native Docker can be **bind**, **volume**, and **tmpfs**. - -- The restart policy does not support **unless-stopped**. -- The values returned for Docker and lightweight containers are 127 and 125 respectively in the following three scenarios: - - The host device specified by **--device** does not exist. - - The hook JSON file specified by **--hook-spec** does not exist. - - The entry point specified by **--entrypoint** does not exist. - -- When the **--volume** parameter is used, /dev/ptmx will be deleted and recreated during container startup. Therefore, do not mount the **/dev** directory to that of the container. Use **--device** to mount the devices in **/dev** of the container. -- When the **-it** parameter is used, the **/dev/ptmx** device will be deleted and rebuilt when the container is started. Therefore, do not mount the **/dev** directory to the **/dev** directory of the container. Instead, use **--device** to mount the devices in the **/dev** directory to the container. -- Do not use the echo option to input data to the standard input of the **run** command. Otherwise, the client will be suspended. The echo value should be directly transferred to the container as a command line parameter. - - ```shell - # echo ls | isula run -i busybox /bin/sh - - - ^C - # - ``` - - The client is suspended when the preceding command is executed because the preceding command is equivalent to input **ls** to **stdin**. Then EOF is read and the client does not send data and waits for the server to exit. However, the server cannot determine whether the client needs to continue sending data. As a result, the server is suspended in reading data, and both parties are suspended. - - The correct execution method is as follows: - - ```shell - # isula run -i busybox ls - bin - dev - etc - home - proc - root - sys - tmp - usr - var - # - ``` - -- If the root directory \(/\) of the host is used as the file system of the container, the following situations may occur during the mounting: - - **Table 2** Mounting scenarios - - - - - - - - - - - - - -

Host Path (Source)

-

Container Path (Destination)

-

/home/test1

-

/mnt/

-

/home/test2

-

/mnt/abc

-
- - >![](./public_sys-resources/icon-notice.gif) **NOTICE:** - >Scenario 1: Mount **/home/test1** and then **/home/test2**. In this case, the content in **/home/test1** overwrites the content in **/mnt**. As a result, the **abc** directory does not exist in **/mnt**, and mounting**/home/test2** to **/mnt/abc** fails. - >Scenario 2: Mount **/home/test2** and then **/home/test1**. In this case, the content of **/mnt** is replaced with the content of **/home/test1** during the second mounting. In this way, the content mounted during the first mounting from **/home/test2** to **/mnt/abc** is overwritten. - >The first scenario is not supported. For the second scenario, users need to understand the risk of data access failures. - -- Exercise caution when configuring the **/sys** and **/proc** directories to be writable. - - The **/sys** and **/proc** directories contain the APIs for maintaining Linux kernel parameters and managing devices. If the directories are writable in a container, container escape may occur. - -- Exercise caution when configuring containers to share namespaces with hosts. - - For example, if you use **--pid**, **--ipc**, **--uts**, or **--net** to configure namespace sharing between the container and the host, the namespace isolation between the container and the host is lost, and the host can be attacked from the container. For example, if you use **--pid** to configure PID namespace sharing between the container and the host, the PID of the process on the host can be viewed in the container and the process can be killed in the container. - -- Exercise caution when configuring parameters that can be used to mount host resources, such as **--device** and **-v**. Do not map sensitive directories or devices of the host to containers to prevent leakage of sensitive information. - -- Exercise caution when using the **--privileged** option to start a container. If the **--privileged** option is used, the container will have excessive permissions, affecting the host configuration. - - >![](./public_sys-resources/icon-notice.gif) **NOTICE:** - > - > - In high concurrency scenarios (200 containers are concurrently started), the memory management mechanism of Glibc may cause memory holes and large virtual memory (for example, 10 GB). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set the **MALLOC\_ARENA\_MAX** environment variable to reduce the virtual memory and increase the probability of reducing the physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements. - > - > ```text - > To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance deterioration on the ARM64 server is controlled by less than 10%.) - > Configuration method: - > 1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start the iSulad. - > 2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4. - > ``` - -### Example - -Run a new container. - -```shell -# isula run -itd busybox -9c2c13b6c35f132f49fb7ffad24f9e673a07b7fe9918f97c0591f0d7014c713b -``` - -## Stopping a Container - -### Description - -To stop a container, run the **isula stop** command. The SIGTERM signal is sent to the first process in the container. If the container is not stopped within the specified time \(10s by default\), the SIGKILL signal is sent. - -### Usage - -```shell -isula stop [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **stop** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

stop

-

-f, --force

-

Forcibly stops a running container.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-t, --time

-

Time for graceful stop. If the time exceeds the value of this parameter, the container is forcibly stopped.

-
- -### Constraints - -- If the **t** parameter is specified and the value of **t** is less than 0, ensure that the application in the container can process the stop signal. - - Principle of the Stop command: Send the SIGTERM signal to the container, and then wait for a period of time \(**t** entered by the user\). If the container is still running after the period of time, the SIGKILL signal is sent to forcibly kill the container. - -- The meaning of the input parameter **t** is as follows: - - **t** < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism. - - **t** = 0: Do not wait and send **kill -9** to the container immediately. - - **t** \> 0: Wait for a specified period and send **kill -9** to the container if the container does not stop within the specified period. - - Therefore, if **t** is set to a value less than 0 \(for example, **t** = -1\), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the **isula stop** command is run. - -### Example - -Stop a container. - -```shell -# isula stop fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -``` - -## Forcibly Stopping a Container - -### Description - -To forcibly stop one or more running containers, run the **isula kill** command. - -### Usage - -```shell -isula kill [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **kill** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

kill

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-s, --signal

-

Signal sent to the container.

-
- -### Example - -Kill a container. - -```shell -# isula kill fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -``` - -## Removing a Container - -### Description - -To remove a container, run the **isula rm** command. - -### Usage - -```shell -isula rm [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **rm** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

rm

-

-f, --force

-

Forcibly removes a running container.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-v, --volume

-

Removes a volume mounted to a container. (Note: Currently, iSulad does not use this function.)

-
- -### Constraints - -- In normal I/O scenarios, it takes T1 to delete a running container in an empty environment (with only one container). In an environment with 200 containers (without a large number of I/O operations and with normal host I/O), it takes T2 to delete a running container. The specification of T2 is as follows: T2 = max \{T1 x 3, 5\}s. - -### Example - -Delete a stopped container. - -```shell -# isula rm fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -``` - -## Attaching to a Container - -### Description - -To attach standard input, standard output, and standard error of the current terminal to a running container, run the **isula attach** command. - -### Usage - -```shell -isula attach [OPTIONS] CONTAINER -``` - -### Parameters - -The following table lists the parameters supported by the **attach** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

attach

-

--help

-

Prints help information.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-
- -### Constraints - -- For the native Docker, running the **attach** command will directly enter the container. For the iSulad container, you have to run the **attach** command and press **Enter** to enter the container. - -### Example - -Attach to a running container. - -```shell -# isula attach fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 -/ # -/ # -``` - -## Renaming a Container - -### Description - -To rename a container, run the **isula rename** command. - -### Usage - -```shell -isula rename [OPTIONS] OLD_NAME NEW_NAME -``` - -### Parameters - -The following table lists the parameters supported by the **rename** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

rename

-

--help

-

Prints help information.

-

-H, --host

-

Specifies the path of the iSulad socket file to be connected.

-

-D, --debug

-

Enables the debug mode.

-
- -### Example - -Rename a container. - -```shell -isula rename my_container my_new_container -``` - -## Executing a Command in a Running Container - -### Description - -To execute a command in a running container, run the **isula exec** command. This command is executed in the default directory of the container. If a user-defined directory is specified for the basic image, the user-defined directory is used. - -### Usage - -```shell -isula exec [OPTIONS] CONTAINER COMMAND [ARG...] -``` - -### Parameters - -The following table lists the parameters supported by the **exec** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

exec

-

  

-

-d, --detach

-

Runs a command in the background.

-

-D, --debug

-

Enables the debug mode.

-

-e, --env

-

Sets environment variables. (Note: Currently, iSulad does not use this function.)

-

--help

-

Prints help information.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-i, --interactive

-

Enables the standard input though no connection is set up. (Note: Currently, iSulad does not use this function.)

-

-t, --tty

-

Allocates a pseudo terminal. (Note: Currently, iSulad does not use this function.)

-

-u, --user

-

Logs in to the container as a specified user.

-

--workdir

-

Specifies the working directory for running the command. This function is supported only when runtime is set to lcr.

-
- -### Constraints - -- If no parameter is specified in the **isula exec** command, the **-it** parameter is used by default, indicating that a pseudo terminal is allocated and the container is accessed in interactive mode. -- When you run the **isula exec** command to execute a script and run a background process in the script, you need to use the **nohup** flag to ignore the **SIGHUP** signal. - - When you run the **isula exec** command to execute a script and run a background process in the script, you need to use the **nohup** flag. Otherwise, the kernel sends the **SIGHUP** signal to the process executed in the background when the process \(first process of the session\) exits. As a result, the background process exits and zombie processes occur. - -- After running the **isula exec** command to access the container process, do not run background programs. Otherwise, the system will be suspended. - - To run the **isula exec** command to execute a background process, perform the following steps: - - 1. Run the **isula exec container\_name bash** command to access the container. - 2. After entering the container, run the **script &** command. - 3. Run the **exit** command. The terminal stops responding. - - >After the isula exec command is executed to enter the container, the background program stops responding because the isula exec command is executed to enter the container and run the background while1 program. When the bash command is run to exit the process, the while1 program does not exit and becomes an orphan process, which is taken over by process 1. - >The while1 process is executed by the initial bash process fork &exec of the container. The while1 process copies the file handle of the bash process. As a result, the handle is not completely closed when the bash process exits. - >The console process cannot receive the handle closing event, epoll_wait stops responding, and the process does not exit. - -- Do not run the **isula exec** command in the background. Otherwise, the system may be suspended. - - Run the **isula exec** command in the background as follows: - - Run the **isula exec script &** command in the background, for example, **isula exec container\_name script &,isula exec**. The command is executed in the background. The script continuously displays a file by running the **cat** command. Normally, there is output on the current terminal. If you press **Enter** on the current terminal, the client exits the stdout read operation due to the I/O read failure. As a result, the terminal does not output data. The server continues to write data to the buffer of the FIFO because the process is still displaying files by running the **cat** command. When the buffer is full, the process in the container is suspended in the write operation. - -- When a lightweight container uses the **exec** command to execute commands with pipe operations, you are advised to run the **/bin/bash -c** command. - - Typical application scenarios: - - Run the **isula exec container\_name -it ls /test | grep "xx" | wc -l** command to count the number of xx files in the test directory. The output is processed by **grep** and **wc** through the pipe because **ls /test** is executed with **exec**. The output of **ls /test** executed by **exec** contains line breaks. When the output is processed, the result is incorrect. - - Cause: Run the **ls /test** command using **exec**. The command output contains a line feed character. Run the**| grep "xx" | wc -l** command for the output. The processing result is 2 \(two lines\). - - ```shell - # isula exec -it container ls /test - xx xx10 xx12 xx14 xx3 xx5 xx7 xx9 - xx1 xx11 xx13 xx2 xx4 xx6 xx8 - # - ``` - - Suggestion: When running the **run/exec** command to perform pipe operations, run the **/bin/bash -c** command to perform pipe operations in the container. - - ```shell - # isula exec -it container /bin/sh -c "ls /test | grep "xx" | wc -l" - 15 - # - ``` - -- Do not use the **echo** option to input data to the standard input of the **exec** command. Otherwise, the client will be suspended. The echo value should be directly transferred to the container as a command line parameter. - - ```shell - # echo ls | isula exec 38 /bin/sh - - - ^C - # - ``` - - The client is suspended when the preceding command is executed because the preceding command is equivalent to input **ls** to **stdin**. Then EOF is read and the client does not send data and waits for the server to exit. However, the server cannot determine whether the client needs to continue sending data. As a result, the server is suspended in reading data, and both parties are suspended. - - The correct execution method is as follows: - - ```shell - # isula exec 38 ls - bin dev etc home proc root sys tmp usr var - ``` - -### Example - -Run the echo command in a running container. - -```shell -# isula exec c75284634bee echo "hello,world" -hello,world -``` - -## Querying Information About a Single Container - -### Description - -To query information about a single container, run the **isula inspect** command. - -### Usage - -```shell -isula inspect [OPTIONS] CONTAINER|IMAGE [CONTAINER|IMAGE...] -``` - -### Parameters - -The following table lists the parameters supported by the **inspect** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

inspect

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-f, --format

-

Output format.

-

-t, --time

-

Timeout interval, in seconds. If the inspect command fails to query container information within the specified period, the system stops waiting and reports an error immediately. The default value is 120s. If the value is less than or equal to 0, the inspect command keeps waiting until the container information is obtained successfully.

-
- -### Example - -Query information about a container. - -```shell -# isula inspect -f '{{.State.Status} {{.State.Running}}' c75284634bee -running -true - - -# isula inspect c75284634bee -[ - { - "Id": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a", - "Created": "2019-08-01T22:48:13.993304927-04:00", - "Path": "sh", - "Args": [], - "State": { - "Status": "running", - "Running": true, - "Paused": false, - "Restarting": false, - "Pid": 21164, - "ExitCode": 0, - "Error": "", - "StartedAt": "2019-08-02T06:09:25.535049168-04:00", - "FinishedAt": "2019-08-02T04:28:09.479766839-04:00", - "Health": { - "Status": "", - "FailingStreak": 0, - "Log": [] - } - }, - "Image": "busybox", - "ResolvConfPath": "", - "HostnamePath": "", - "HostsPath": "", - "LogPath": "none", - "Name": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a", - "RestartCount": 0, - "HostConfig": { - "Binds": [], - "NetworkMode": "", - "GroupAdd": [], - "IpcMode": "", - "PidMode": "", - "Privileged": false, - "SystemContainer": false, - "NsChangeFiles": [], - "UserRemap": "", - "ShmSize": 67108864, - "AutoRemove": false, - "AutoRemoveBak": false, - "ReadonlyRootfs": false, - "UTSMode": "", - "UsernsMode": "", - "Sysctls": {}, - "Runtime": "lcr", - "RestartPolicy": { - "Name": "no", - "MaximumRetryCount": 0 - }, - "CapAdd": [], - "CapDrop": [], - "Dns": [], - "DnsOptions": [], - "DnsSearch": [], - "ExtraHosts": [], - "HookSpec": "", - "CPUShares": 0, - "Memory": 0, - "OomScoreAdj": 0, - "BlkioWeight": 0, - "BlkioWeightDevice": [], - "CPUPeriod": 0, - "CPUQuota": 0, - "CPURealtimePeriod": 0, - "CPURealtimeRuntime": 0, - "CpusetCpus": "", - "CpusetMems": "", - "SecurityOpt": [], - "StorageOpt": {}, - "KernelMemory": 0, - "MemoryReservation": 0, - "MemorySwap": 0, - "OomKillDisable": false, - "PidsLimit": 0, - "FilesLimit": 0, - "Ulimits": [], - "Hugetlbs": [], - "HostChannel": { - "PathOnHost": "", - "PathInContainer": "", - "Permissions": "", - "Size": 0 - }, - "EnvTargetFile": "", - "ExternalRootfs": "" - }, - "Mounts": [], - "Config": { - "Hostname": "localhost", - "User": "", - "Env": [ - "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", - "TERM=xterm", - "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" - ], - "Tty": true, - "Cmd": [ - "sh" - ], - "Entrypoint": [], - "Labels": {}, - "Annotations": { - "log.console.file": "none", - "log.console.filerotate": "7", - "log.console.filesize": "1MB", - "rootfs.mount": "/var/lib/isulad/mnt/rootfs", - "native.umask": "secure" - }, - "HealthCheck": { - "Test": [], - "Interval": 0, - "Timeout": 0, - "StartPeriod": 0, - "Retries": 0, - "ExitOnUnhealthy": false - } - }, - "NetworkSettings": { - "IPAddress": "" - } - } -] -``` - -## Querying Information About All Containers - -### Description - -To query information about all containers, run the **isula ps** command. - -### Usage - -```shell -isula ps [OPTIONS] -``` - -### Parameters - -The following table lists the parameters supported by the **ps** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

ps

-

  

-

  

-

  

-

  

-

-a, --all

-

Displays all containers.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-q, --quiet

-

Displays only the container name.

-

-f, --filter

-

Adds filter criteria.

-

--format

-

Output format.

-

--no-trunc

-

Do not truncate the container ID.

-
- -### Example - -Query information about all containers. - -```shell -# isula ps -a - -ID IMAGE STATUS PID COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME NAMES -e84660aa059c rnd-dockerhub.huawei.com/official/busybox running 304765 "sh" 0 0 13 minutes ago - lcr e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016 -# isula ps -a --format "table {{.ID}} {{.Image}}" --no-trunc -ID IMAGE -e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016 rnd-dockerhub.huawei.com/official/busybox -``` - -## Restarting a Container - -### Description - -To restart one or more containers, run the **isula restart** command. - -### Usage - -```shell -isula restart [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **restart** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

restart

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-t, --time

-

Time for graceful stop. If the time exceeds the value of this parameter, the container is forcibly stopped.

-
- -### Constraints - -- If the **t** parameter is specified and the value of **t** is less than 0, ensure that the application in the container can process the stop signal. - - The restart command first calls the stop command to stop the container. Send the SIGTERM signal to the container, and then wait for a period of time \(**t** entered by the user\). If the container is still running after the period of time, the SIGKILL signal is sent to forcibly kill the container. - -- The meaning of the input parameter **t** is as follows: - - **t** < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism. - - **t** = 0: Do not wait and send **kill -9** to the container immediately. - - **t** \> 0: Wait for a specified period and send **kill -9** to the container if the container does not stop within the specified period. - - Therefore, if **t** is set to a value less than 0 \(for example, **t** = -1\), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the **isula stop** command is run. - -### Example - -Restart a container. - -```shell -# isula restart c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a - c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a -``` - -## Waiting for a Container to Exit - -### Description - -To wait for one or more containers to exit, run the **isula wait** command. Only containers whose runtime is of the LCR type are supported. - -### Usage - -```shell -isula wait [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **wait** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

wait

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-
- -### Example - -Wait for a single container to exit. - -```shell -$ isula wait c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a - 137 -``` - -## Viewing Process Information in a Container - -### Description - -To view process information in a container, run the **isula top** command. Only containers whose runtime is of the LCR type are supported. - -### Usage - -```shell -isula top [OPTIONS] container [ps options] -``` - -### Parameters - -The following table lists the parameters supported by the **top** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

top

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-
- -### Example - -Query process information in a container. - -```shell -# isula top 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c -UID PID PPID C STIME TTY TIME CMD -root 22166 22163 0 23:04 pts/1 00:00:00 sh -``` - -## Displaying Resource Usage Statistics of a Container - -### Description - -To display resource usage statistics in real time, run the **isula stats** command. Only containers whose runtime is of the LCR type are supported. - -### Usage - -```shell -isula stats [OPTIONS] [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **stats** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

stats

-

  

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-a, --all

-

Displays all containers. (By default, only running containers are displayed.)

-

--no-stream

-

Display the first result only. Only statistics in non-stream mode are displayed.

-

--original

-

Displays the original data of the container without statistics calculation.

-
- -### Example - -Display resource usage statistics. - -```shell -# isula stats --no-stream 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c CONTAINER CPU % MEM USAGE / LIMIT MEM % BLOCK I / O PIDS -21fac8bb9ea8 0.00 56.00 KiB / 7.45 GiB 0.00 0.00 B / 0.00 B 1 -``` - -## Obtaining Container Logs - -### Description - -To obtain container logs, run the **isula logs** command. - -### Usage - -```shell -isula logs [OPTIONS] [CONTAINER...] -``` - -### Parameters - -The following table lists the parameters supported by the **logs** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

logs

-

  

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-f, --follow

-

Traces log output.

-

--tail

-

Displays the number of log records.

-

-t, --timestamps

-

Displays the timestamp.

-
- -### Constraints - -- By default, the container log function is enabled. To disable this function, run the **isula create --log-opt disable-log=true** or **isula run --log-opt disable-log=true** command. - -### Example - -Obtain container logs. - -```shell -# isula logs 6a144695f5dae81e22700a8a78fac28b19f8bf40e8827568b3329c7d4f742406 -hello, world -hello, world -hello, world -``` - -## Copying Data Between a Container and a Host - -### Description - -To copy data between a host and a container, run the **isula cp** command. Only containers whose runtime is of the LCR type are supported. - -### Usage - -```shell -isula cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH -isula cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH -``` - -### Parameters - -The following table lists the parameters supported by the **cp** command. - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

cp

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-
- -### Constraints - -- When iSulad copies files, note that the **/etc/hostname**, **/etc/resolv.conf**, and **/etc/hosts** files are not mounted to the host, neither the **--volume** and **--mount** parameters. Therefore, the original files in the image instead of the files in the real container are copied. - - ```shell - # isula cp b330e9be717a:/etc/hostname /tmp/hostname - # cat /tmp/hostname - # - ``` - -- When decompressing a file, iSulad does not check the type of the file or folder to be overwritten in the file system. Instead, iSulad directly overwrites the file or folder. Therefore, if the source is a folder, the file with the same name is forcibly overwritten as a folder. If the source file is a file, the folder with the same name will be forcibly overwritten as a file. - - ```shell - # rm -rf /tmp/test_file_to_dir && mkdir /tmp/test_file_to_dir - # isula exec b330e9be717a /bin/sh -c "rm -rf /tmp/test_file_to_dir && touch /tmp/test_file_to_dir" - # isula cp b330e9be717a:/tmp/test_file_to_dir /tmp - # ls -al /tmp | grep test_file_to_dir - -rw-r----- 1 root root 0 Apr 26 09:59 test_file_to_dir - ``` - -- The **cp** command is used only for maintenance and fault locating. Do not use the **cp** command in the production environment. - -### Example - -Copy the **/test/host** directory on the host to the **/test** directory on container 21fac8bb9ea8. - -```shell -isula cp /test/host 21fac8bb9ea8:/test -``` - -Copy the **/www** directory on container 21fac8bb9ea8 to the **/tmp** directory on the host. - -```shell -isula cp 21fac8bb9ea8:/www /tmp/ -``` - -## Pausing All Processes in a Container - -### Description - -The **isula pause** command is used to pause all processes in one or more containers. - -### Usage - -```shell -isula pause [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

pause

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-
- -### Constraints - -- Only containers in the running state can be paused. -- After a container is paused, other lifecycle management operations \(such as **restart**, **exec**, **attach**, **kill**, **stop**, and **rm**\) cannot be performed. -- After a container with health check configurations is paused, the container status changes to unhealthy. - -### Example - -Pause a running container. - -```shell -# isula pause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac - 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac -``` - -## Resuming All Processes in a Container - -### Description - -The **isula unpause** command is used to resume all processes in one or more containers. It is a reversible process of **isula pause**. - -### Usage - -```shell -isula unpause [OPTIONS] CONTAINER [CONTAINER...] -``` - -### Parameters - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

pause

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-
- -### Constraints - -- Only containers in the paused state can be unpaused. - -### Example - -Resume a paused container. - -```shell -# isula unpause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac - 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac -``` - -## Obtaining Event Messages from the Server in Real Time - -### **Description** - -The **isula events** command is used to obtain real-time events from the server. - -### Usage - -```shell -isula events [OPTIONS] -``` - -### Parameter - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Parameter

-

Description

-

events

-

-H, --host

-

Specifies the iSulad socket file path to be accessed.

-

-D, --debug

-

Enables the debug mode.

-

--help

-

Prints help information.

-

-n, --name

-

Obtains event messages of a specified container.

-

-S, --since

-

Obtains event messages generated since a specified time.

-

-U, --until

-

Obtains the event at the specified time point.

-
- -### Constraints - -- Support container-related events: create, start, restart, stop, exec_create, exec_die, attach, kill, top, rename, archive-path, extract-to-dir, update, pause, unpause, export, and resize. -- Supported image-related events: load, remove, pull, login, and logout. - -### Example - -Run the following command to obtain event messages from the server in real time: - -```shell -# isula events -``` diff --git a/docs/en/docs/Container/container-resource-management.md b/docs/en/docs/Container/container-resource-management.md deleted file mode 100644 index 3f1e778f1d41e127494aae254008ed37cd76be29..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/container-resource-management.md +++ /dev/null @@ -1,746 +0,0 @@ -# Container Resource Management - -- [Container Resource Management](#container-resource-management) - - [Description](#description) - - [Sharing Resources](#sharing-resources) - - [Restricting CPU Resources of a Running Container](#restricting-cpu-resources-of-a-running-container) - - [Restricting the Memory Usage of a Running Container](#restricting-the-memory-usage-of-a-running-container) - - [Restricting I/O Resources of a Running Container](#restricting-io-resources-of-a-running-container) - - [Restricting the Rootfs Storage Space of a Container](#restricting-the-rootfs-storage-space-of-a-container) - - [Restricting the Number of File Handles in a Container](#restricting-the-number-of-file-handles-in-a-container) - - [Restricting the Number of Processes or Threads that Can Be Created in a Container](#restricting-the-number-of-processes-or-threads-that-can-be-created-in-a-container) - - [Configuring the ulimit Value in a Container](#configuring-the-ulimit-value-in-a-container) - -## Overview - -You can use namespaces and cgroups to manage container resources. iSula can use cgroup v1 and cgroup v2 to restrict resources. cgroup v2 is an experimental feature and cannot be put into commercial use. When the system is configured to support only cgroup v2 and cgroup v2 is mounted to the **/sys/fs/cgroup** directory, iSula uses cgroup v2 for resource management. No matter cgroup v1 or cgroup v2 is used to manage container resources, iSula provides the same API for users to implement resource restriction. - -## Sharing Resources - -### Description - -Containers or containers and hosts can share namespace information mutually, including PID, network, IPC, and UTS information. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->When namespace information is shared with a host, the namespace isolation mechanism is unavailable. As a result, information on the host can be queried and operated in a container, causing security risks. For example, if **--pid=host** is used to share the PID namespace of a host, information about other processes on the host can be viewed, causing information leakage or even killing the host process. Exercise caution when using the shared host namespace function to ensure security. - -### Usage - -When running the **isula create/run** command, you can set the namespace parameters to share resources. For details, see the following parameter description table. - -### Parameters - -You can specify the following parameters when running the **isula create/run** command: - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--pid

-

Specifies the PID namespace to be shared.

-

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

-

No

-

--net

-

Specifies the network namespace to be shared.

-

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

-

No

-

--ipc

-

Specifies the IPC namespace to be shared.

-

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

-

No

-

--uts

-

Specifies the UTS namespace to be shared.

-

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

-

No

-
- -### Example - -If two containers need to share the same PID namespace, add **--pid container:** when running the container. For example: - -```shell -isula run -tid --name test_pid busybox sh -isula run -tid --name test --pid container:test_pid busybox sh -``` - -## Restricting CPU Resources of a Running Container - -### Description - -You can set parameters to restrict the CPU resources of a container. - -### Usage - -When running the **isula create/run** command, you can set CPU-related parameters to limit the CPU resources of a container. For details about the parameters and values, see the following table. - -### Parameters - -You can specify the following parameters when running the **isula create/run** command: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--cpu-period

-

Limits the CPU CFS period in a container.

-

64-bit integer

-

No

-

--cpu-quota

-

Limits the CPU CFS quota in a container.

-

64-bit integer

-

No

-

--cpu-shares

-

Limit the CPU share (relative weight) in a container.

-

64-bit integer

-

No

-

--cpu-rt-period

-

Limits the real-time CPU period in a container, in microseconds.

-

64-bit integer

-

No

-

--cpu-rt-runtime

-

Limits the real-time running time of the CPU in a container, in microseconds.

-

64-bit integer

-

No

-

--cpuset-cpus

-

Limits the CPU nodes used by a container.

-

Character string. The value is the number of CPUs to be set. For example, the value can be **0-3** or **0,1**.

. -

No

-

--cpuset-mems

-

Limits the memory nodes used by cpuset in a container.

-

Character string. The value is the number of CPUs to be set. For example, the value can be **0-3** or **0,1**.

. -

No

-
- -### Example - -To restrict a container to use a specific CPU, add **--cpuset-cpus number** when running the container. For example: - -```shell -isula run -tid --cpuset-cpus 0,2-3 busybox sh -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->You can check whether the configuration is successful. For details, see "Querying Information About a Single Container." - -## Restricting the Memory Usage of a Running Container - -### Description - -You can set parameters to restrict the memory usage of a container. - -### Usage - -When running the **isula create/run** command, you can set memory-related parameters to restrict memory usage of containers. For details about the parameters and values, see the following table. - -### Parameters - -You can specify the following parameters when running the **isula create/run** command: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--memory

-

Specifies the upper limit of the memory usage of a container.

-

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

-

No

-

--memory-reservation

-

Specifies the soft upper limit of the memory of a container.

-

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

-

No

-

--memory-swap

-

Specifies the upper limit of the swap memory of the container.

-

64-bit integer The value can be -1 or a non-negative number. The value -1 indicates no limit, and the value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

-

No

-

--kernel-memory

-

Specifies the upper limit of the kernel memory of the container.

-

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

-

No

-
- -### Example - -To set the upper limit of the memory of a container, add **--memory \[\]** when running the container. For example: - -```shell -isula run -tid --memory 1G busybox sh -``` - -## Restricting I/O Resources of a Running Container - -### Description - -You can set parameters to limit the read/write speed of devices in the container. - -### Usage - -When running the **isula create/run** command, you can set **--device-read-bps/--device-write-bps :\[\]** to limit the read/write speed of devices in the container. - -### Parameters - -When running the **isula create/run** command, set **--device-read/write-bps**. - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--device-read-bps/--device-write-bps

-

Limits the read/write speed of devices in the container.

-

64-bit integer The value is a positive integer. The value can be 0, indicating that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

-

No

-
- -### Example - -To limit the read/write speed of devices in the container, add **--device-write-bps/--device-read-bps :\[\]** when running the container. For example, to limit the read speed of the device **/dev/sda** in the container **busybox** to 1 Mbit/s, run the following command: - -```shell -isula run -tid --device-read-bps /dev/sda:1mb busybox sh -``` - -To limit the write speed, run the following command: - -```shell -isula run -tid --device-write-bps /dev/sda:1mb busybox sh -``` - -## Restricting the Rootfs Storage Space of a Container - -### Description - -When the overlay2 storage driver is used on the EXT4 file system, the file system quota of a single container can be set. For example, the quota of container A is set to 5 GB, and the quota of container B is set to 10 GB. - -This feature is implemented by the project quota function of the EXT4 file system. If the kernel supports this function, use the syscall SYS\_IOCTL to set the project ID of a directory, and then use the syscall SYS\_QUOTACTL to set the hard limit and soft limit of the corresponding project ID. - -### Usage - -1. Prepare the environment. - - Ensure that the file system supports the **Project ID** and **Project Quota** attributes, the kernel version is 4.19 or later, and the version of the peripheral package e2fsprogs is 1.43.4-2 or later. - -2. Before mounting overlayfs to a container, set different project IDs for the upper and work directories of different containers and set inheritance options. After overlayfs is mounted to a container, the project IDs and inherited attributes cannot be modified. -3. Set the quota as a privileged user outside the container. -4. Add the following configuration to daemon: - - ```shell - -s overlay2 --storage-opt overlay2.override_kernel_check=true - ``` - -5. Daemon supports the following options for setting default restrictions for containers: - - **--storage-opt overlay2.basesize=128M** specifies the default limit. If **--storeage-opt size** is also specified when you run the **isula run** command, the value of this parameter takes effect. If no size is specified during the daemon process or when you run the **isula run** command, the size is not limited. - -6. Enable the **Project ID** and **Project Quota** attributes of the file system. - - Format and mount the file system. - - ```shell - mkfs.ext4 -O quota,project /dev/sdb - mount -o prjquota /dev/sdb /var/lib/isulad - ``` - -### Parameters - -When running the **create/run** command, set **--storage-opt**. - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--storage-opt size=${rootfsSize}

-

Restricts the root file system (rootfs) storage space of the container.

-

The parsed value of rootfsSize is a positive number expressed in bytes within the int64 range. The default unit is B. You can also set the unit to [kKmMgGtTpP])?[iI]?[bB]?$. (The minimum value is 10G in the device mapper scenario.)

-

No

-
- -### Example - -In the **isula run/create** command, use the existing parameter **--storage-opt size=**_value_ to set the quota. The value is a positive number in the unit of **\[kKmMgGtTpP\]?\[iI\]?\[bB\]?**. If the value does not contain a unit, the default unit is byte. - -```console -$ [root@localhost ~]# isula run -ti --storage-opt size=10M busybox -/ # df -h -Filesystem Size Used Available Use% Mounted on -overlay 10.0M 48.0K 10.0M 0% / -none 64.0M 0 64.0M 0% /dev -none 10.0M 0 10.0M 0% /sys/fs/cgroup -tmpfs 64.0M 0 64.0M 0% /dev -shm 64.0M 0 64.0M 0% /dev/shm -/dev/mapper/vg--data-ext41 - 9.8G 51.5M 9.2G 1% /etc/hostname -/dev/mapper/vg--data-ext41 - 9.8G 51.5M 9.2G 1% /etc/resolv.conf -/dev/mapper/vg--data-ext41 - 9.8G 51.5M 9.2G 1% /etc/hosts -tmpfs 3.9G 0 3.9G 0% /proc/acpi -tmpfs 64.0M 0 64.0M 0% /proc/kcore -tmpfs 64.0M 0 64.0M 0% /proc/keys -tmpfs 64.0M 0 64.0M 0% /proc/timer_list -tmpfs 64.0M 0 64.0M 0% /proc/sched_debug -tmpfs 3.9G 0 3.9G 0% /proc/scsi -tmpfs 64.0M 0 64.0M 0% /proc/fdthreshold -tmpfs 64.0M 0 64.0M 0% /proc/fdenable -tmpfs 3.9G 0 3.9G 0% /sys/firmware -/ # -/ # dd if=/dev/zero of=/home/img bs=1M count=12 && sync -dm-4: write failed, project block limit reached. -10+0 records in -9+0 records out -10432512 bytes (9.9MB) copied, 0.011782 seconds, 844.4MB/s -/ # df -h | grep overlay -overlay 10.0M 10.0M 0 100% / -/ # -``` - -### Constraints - -1. The quota applies only to the rw layer. - - The quota of overlay2 is for the rw layer of the container. The image size is not included. - -2. The kernel supports and enables this function. - - The kernel must support the EXT4 project quota function. When running **mkfs**, add **-O quota,project**. When mounting the file system, add **-o prjquota**. If any of the preceding conditions is not met, an error is reported when **--storage-opt size=**_value_ is used. - - ```console - $ [root@localhost ~]# isula run -it --storage-opt size=10Mb busybox df -h - Error response from daemon: Failed to prepare rootfs with error: time="2019-04-09T05:13:52-04:00" level=fatal msg="error creating read- - write layer with ID "a4c0e55e82c55e4ee4b0f4ee07f80cc2261cf31b2c2dfd628fa1fb00db97270f": --storage-opt is supported only for overlay over - xfs or ext4 with 'pquota' mount option" - ``` - -3. Description of the limit of quota: - 1. If the quota is greater than the size of the partition where user **root** of iSulad is located, the file system quota displayed by running the **df** command in the container is the size of the partition where user **root** of iSulad is located, not the specified quota. - 2. **--storage-opt size=0** indicates that the size is not limited and the value cannot be less than 4096. The precision of size is one byte. If the specified precision contains decimal bytes, the decimal part is ignored. For example, if size is set to **0.1**, the size is not limited. \(The value is restricted by the precision of the floating point number stored on the computer. That is, 0.999999999999999999999999999 is equal to 1. The number of digits 9 may vary according to computers. Therefore, 4095.999999999999999999999999999 is equal to 4096.\) Note that running **isula inspect** displays the original command line specified format. If the value contains decimal bytes, you need to ignore the decimal part. - 3. If the quota is too small, for example,**--storage-opt size=4k**, the container may fail to be started because some files need to be created for starting the container. - 4. The **-o prjquota** option is added to the root partition of iSulad when iSulad is started last time. If this option is not added during this startup, the setting of the container with quota created during the last startup does not take effect. - 5. The value range of the daemon quota **--storage-opt overlay2.basesize** is the same as that of **--storage-opt size**. - -4. When **storage-opt** is set to 4 KB, the lightweight container startup is different from that of Docker. - - Use the **storage-opt size=4k** and image **rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest** to run the container. - - Docker fails to be started. - - ```console - [root@localhost ~]# docker run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest - docker: Error response from daemon: symlink /proc/mounts /var/lib/docker/overlay2/e6e12701db1a488636c881b44109a807e187b8db51a50015db34a131294fcf70-init/merged/etc/mtab: disk quota exceeded. - See 'docker run --help'. - ``` - - The lightweight container is started properly and no error is reported. - - ```console - [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest - 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 - [root@localhost ~]# isula ps - STATUS PID IMAGE COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID NAMES - running 17609 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest /bin/bash 0 0 2 seconds ago - lcr 636480b1fc2c 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 - ``` - - During container startup, if you need to create a file in the **rootfs** directory of the container, the image size exceeds 4 KB, and the quota is set to 4 KB, the file creation will fail. - - When Docker starts the container, it creates more mount points than iSulad to mount some directories on the host to the container, such as **/proc/mounts** and **/dev/shm**. If these files do not exist in the image, the creation will fail, therefore, the container fails to be started. - - When a lightweight container uses the default configuration during container startup, there are few mount points. The lightweight container is created only when the directory like **/proc** or **/sys** does not exist. The image **rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest** in the test case contains **/proc** and **/sys**. Therefore, no new file or directory is generated during the container startup. As a result, no error is reported during the lightweight container startup. To verify this process, when the image is replaced with **rnd-dockerhub.huawei.com/official/busybox-aarch64:latest**, an error is reported when the lightweight container is started because **/proc** does not exist in the image. - - ```console - [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/busybox-aarch64:latest - 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4 - Error response from daemon: Start container error: runtime error: 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4:tools/lxc_start.c:main:404 starting container process caused "Failed to setup lxc, - please check the config file." - ``` - -5. Other description: - - When using iSulad with the quota function to switch data disks, ensure that the data disks to be switched are mounted using the **prjquota** option and the mounting mode of the **/var/lib/isulad/storage/overlay2** directory is the same as that of the **/var/lib/isulad** directory. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Before switching the data disk, ensure that the mount point of **/var/lib/isulad/storage/overlay2** is unmounted. - -## Restricting the Number of File Handles in a Container - -### Description - -You can set parameters to limit the number of file handles that can be opened in a container. - -### Usage - -When running the **isula create/run** command, set the **--files-limit** parameter to limit the number of file handles that can be opened in a container. - -### Parameters - -Set the **--files-limit** parameter when running the **isula create/run** command. - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--files-limit

-

Limits the number of file handles that can be opened in a container.

-

64-bit integer The value can be 0 or a negative number, but cannot be greater than 2 to the power of 63 minus 1. The value 0 or a negative number indicates no limit.

-

During container creation, some handles are opened temporarily. Therefore, the value cannot be too small. Otherwise, the container may not be restricted by the file limit. If the value is less than the number of opened handles, the cgroup file cannot be written. It is recommended that the value be greater than 30.

-

No

-
- -### Example - -When running the container, add **--files-limit n**. For example: - -```shell -isula run -ti --files-limit 1024 busybox bash -``` - -### Constraints - -1. If the **--files-limit** parameter is set to a small value, for example, 1, the container may fail to be started. - - ```console - [root@localhost ~]# isula run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64 - 004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4 - Error response from daemon: Start container error: Container is not running:004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4 - ``` - - Docker will be started successfully, and the value of **files.limit cgroup** is **max**. - - ```console - [root@localhost ~]# docker run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64 - ef9694bf4d8e803a1c7de5c17f5d829db409e41a530a245edc2e5367708dbbab - [root@localhost ~]# docker exec -it ef96 cat /sys/fs/cgroup/files/files.limit - max - ``` - - The root cause is that the startup principles of the lxc and runc processes are different. After the lxc process creates the cgroup, the files.limit value is set, and then the PID of the container process is written into the cgroup.procs file of the cgroup. At this time, the process has opened more than one handle. As a result, an error is reported, and the startup fails. After you create a cgroup by running the **runc** command, the PID of the container process is written to the cgroup.procs file of the cgroup, and then the files.limit value is set. Because more than one handle is opened by the process in the cgroup, the file.limit value does not take effect, the kernel does not report any error, and the container is started successfully. - -## Restricting the Number of Processes or Threads that Can Be Created in a Container - -### Description - -You can set parameters to limit the number of processes or threads that can be created in a container. - -### Usage - -When creating or running a container, use the **--pids-limit** parameter to limit the number of processes or threads that can be created in the container. - -### Parameters - -When running the **create/run** command, set the **--pids-limit** parameter. - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--pids-limit

-

Limits the number of file handles that can be opened in a container.

-

64-bit integer The value can be 0 or a negative number, but cannot be greater than 2 to the power of 63 minus 1. The value 0 or a negative number indicates no limit.

-

No

-
- -### Example - -When running the container, add **--pids-limit n**. For example: - -```shell -isula run -ti --pids-limit 1024 busybox bash -``` - -### Constraints - -During container creation, some processes are created temporarily. Therefore, the value cannot be too small. Otherwise, the container may fail to be started. It is recommended that the value be greater than 10. - -## Configuring the ulimit Value in a Container - -### Description - -You can use parameters to control the resources for executed programs. - -### Usage - -Set the **--ulimit** parameter when creating or running a container, or configure the parameter on the daemon to control the resources for executed programs in the container. - -### Parameters - -Use either of the following methods to configure ulimit: - -1. When running the **isula create/run** command, use **--ulimit =\[:\]** to control the resources of the executed shell program. - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value Range

-

Mandatory or Not

-

--ulimit

-

Limits the resources of the executed shell program.

-

64-bit integer The value of the soft limit must be less than or equal to that of the hard limit. If only the soft limit is specified, the value of the hard limit is equal to that of the soft limit. Some types of resources do not support negative numbers. For details, see the following table.

-

No

-
- -2. Use daemon parameters or configuration files. - - For details, see **--default-ulimits** in [Configuration Mode](./installation-configuration.md#configuration-mode). - - **--ulimit** can limit the following types of resources: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Type

-

Description

-

Value Range

-

core

-

limits the core file size (KB)

-

64-bit integer, without unit. The value can be 0 or a negative number. The value -1 indicates no limit. Other negative numbers are forcibly converted into a large positive integer.

-

cpu

-

max CPU time (MIN)

-

data

-

max data size (KB)

-

fsize

-

maximum filesize (KB)

-

locks

-

max number of file locks the user can hold

-

memlock

-

max locked-in-memory address space (KB)

-

msgqueue

-

max memory used by POSIX message queues (bytes)

-

nice

-

nice priority

-

nproc

-

max number of processes

-

rss

-

max resident set size (KB)

-

rtprio

-

max realtime priority

-

rttime

-

realtime timeout

-

sigpending

-

max number of pending signals

-

stack

-

max stack size (KB)

-

nofile

-

max number of open file descriptors

-

64-bit integer, without unit. The value cannot be negative. A negative number is forcibly converted to a large positive number. In addition, "Operation not permitted" is displayed during the setting.

-
- -### Example - -When creating or running a container, add **--ulimit =\[:\]**. For example: - -```shell -isula create/run -tid --ulimit nofile=1024:2048 busybox sh -``` - -### Constraints - -The ulimit cannot be configured in the **daemon.json** and **/etc/sysconfig/iSulad** files \(or the iSulad command line\). Otherwise, an error is reported when iSulad is started. diff --git a/docs/en/docs/Container/container.md b/docs/en/docs/Container/container.md deleted file mode 100644 index d30a929ed1d4839b78b8922d4b4808e755128878..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/container.md +++ /dev/null @@ -1,20 +0,0 @@ -# Container User Guide - -## Overview - -openEuler provides software packages of iSulad and Docker container engines. - -The following container forms are provided for different application scenarios: - -- Common containers applicable to most common scenarios -- Secure containers applicable to strong isolation and multi-tenant scenarios -- System containers applicable to scenarios where the systemd is used to manage services - -This document describes how to install and use the container engines and how to deploy and use containers in different forms. - -## Intended Audience - -This document is intended for openEuler users who need to install containers. You can better understand this document if you: - -- Be familiar with basic Linux operations. -- Have a basic understanding of containers. diff --git a/docs/en/docs/Container/docker-container.md b/docs/en/docs/Container/docker-container.md deleted file mode 100644 index 0c22297da0329b232603efb8bef7f8c1b0d385ae..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/docker-container.md +++ /dev/null @@ -1,4 +0,0 @@ -# Docker Container - -Docker is an open-source Linux container engine that enables quick application packaging, deployment, and delivery. The original meaning of Docker is dork worker, whose job is to pack the goods to the containers, and move containers, and load containers. Similarly, the job of Docker in Linux is to pack applications to containers, and deploy and run applications on various platforms using containers. Docker uses Linux Container technology to turn applications into standardized, portable, and self-managed components, enabling the "build once" and "run everywhere" features of applications. Features of Docker technology include: quick application release, easy application deployment and management, and high application density. - diff --git a/docs/en/docs/Container/dynamically-loading-the-kernel-module.md b/docs/en/docs/Container/dynamically-loading-the-kernel-module.md deleted file mode 100644 index 7c1458075e1501d1422440e86b86ad3f3512467d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/dynamically-loading-the-kernel-module.md +++ /dev/null @@ -1,56 +0,0 @@ -Dynamically Loading the Kernel Module - -- [Dynamically Loading the Kernel Module](#dynamically-loading-the-kernel-module) - - -## Function Description - -Services in a container may depend on some kernel modules. You can set environment variables to dynamically load the kernel modules required by services in the container to the host before the system container starts. This feature must be used together with isulad-hooks. For details, see [Dynamically Managing Container Resources \(syscontainer-tools\)](./dynamically-managing-container-resources-(syscontainer-tools).md). - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

-e KERNEL_MODULES=module_name1,module_name

-
  • Variable of the string type.
  • This parameter can be set to multiple modules. Use commas (,) to separate module names.
-
- -## Constraints - -- If loaded kernel modules are not verified or conflict with existing modules on the host, an unpredictable error may occur on the host. Therefore, exercise caution when loading kernel modules. -- Dynamic kernel module loading transfers kernel modules to be loaded to containers. This function is implemented by capturing environment variables for container startup using isulad-tools. Therefore, this function relies on the proper installation and deployment of isulad-tools. -- Loaded kernel modules need to be manually deleted. - -## Example - -When starting a system container, specify the **-e KERNEL\_MODULES** parameter. After the system container is started, the ip\_vs module is successfully loaded to the kernel. - -``` -[root@localhost ~]# lsmod | grep ip_vs -[root@localhost ~]# isula run -tid -e KERNEL_MODULES=ip_vs,ip_vs_wrr --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/myrootfs none init -ae18c4281d5755a1e153a7bff6b3b4881f36c8e528b9baba8a3278416a5d0980 -[root@localhost ~]# lsmod | grep ip_vs -ip_vs_wrr 16384 0 -ip_vs 176128 2 ip_vs_wrr -nf_conntrack 172032 7 xt_conntrack,nf_nat,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink,ip_vs -nf_defrag_ipv6 20480 2 nf_conntrack,ip_vs -libcrc32c 16384 3 nf_conntrack,nf_nat,ip_vs -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->- isulad-tools must be installed on the host. ->- **--hooks-spec** must be set to **isulad hooks**. - diff --git a/docs/en/docs/Container/dynamically-managing-container-resources-(syscontainer-tools).md b/docs/en/docs/Container/dynamically-managing-container-resources-(syscontainer-tools).md deleted file mode 100644 index 53ea3ff11e0e752e111cf48d6c6d41d075badf95..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/dynamically-managing-container-resources-(syscontainer-tools).md +++ /dev/null @@ -1,495 +0,0 @@ -# Dynamically Managing Container Resources \(syscontainer-tools\) - -- [Dynamically Managing Container Resources \(syscontainer-tools\)](#dynamically-managing-container-resources-(syscontainer-tools)) - - [Device Management](#device-management) - - [NIC Management](#nic-management) - - [Route Management](#route-management) - - [Volume Mounting Management](#volume-mounting-management) - - -Resources in common containers cannot be managed. For example, a block device cannot be added to a common container, and a physical or virtual NIC cannot be inserted to a common container. In the system container scenario, the syscontainer-tools can be used to dynamically mount or unmount block devices, network devices, routes, and volumes for containers. - -To use this function, you need to install the syscontainer-tools first. - -``` -[root@localhost ~]# yum install syscontainer-tools -``` - - -## Device Management - -### Function Description - -isulad-tools allows you to add block devices \(such as disks and logical volume managers\) or character devices \(such as GPUs, binners, and FUSEs\) on the host to a container. The devices can be used in the container. For example, you can run the **fdisk** command to format the disk and write data to the file system. If the devices are not required, isulad-tools allows you to delete them from the container and return them to the host. - -### Command Format - -``` -isulad-tools [COMMAND][OPTIONS] [ARG...] -``` - -In the preceding format: - -**COMMAND**: command related to device management. - -**OPTIONS**: option supported by the device management command. - -**container\_id**: container ID. - -**ARG**: parameter corresponding to the command. - -### Parameter Description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Function Description

-

Option Description

-

Parameter Description

-

add-device

-

Adds block devices or character devices on the host to a container.

-

Supported options are as follows:

-
  • --blkio-weight-device: sets the I/O weight (relative weight, ranging from 10 to 100) of a block device.
  • --device-read-bps: sets the read rate limit for the block device (byte/s).
  • --device-read-iops: sets the read rate limit for the block device (I/O/s).
  • --device-write-bps: sets the write rate limit for the block device (byte/s).
  • --device-write-iops: sets the write rate limit for the block device (I/O/s).
  • --follow-partition: If a block device is a basic block device (primary SCSI block disk), set this parameter to add all partitions of the primary disk.
  • --force: If any block device or character device already exists in the container, use this parameter to overwrite the old block device or character device files.
  • --update-config-only: updates configuration files only and does not add disks.
-

Parameter format: hostdevice[:containerdevice][:permission] [hostdevice[:containerdevice][:permission]]

-

In the preceding format:

-

hostdevice: path on the host for storing a device.

-

containerdevice: path on the container for storing a device.

-

permission: operation permission on a device within the container.

-

remove-device

-

Deletes block devices or character devices from a container and restores them to the host.

-

Supported options are as follows:

-

--follow-partition: If a block device is a basic block device (primary SCSI block disk), set this parameter to delete all partitions of the primary disk in the container, and restore them to the host.

-

Parameter format: hostdevice[:containerdevice] [hostdevice[:containerdevice]]

-

In the preceding format:

-

hostdevice: path on the host for storing a device.

-

containerdevice: path on the container for storing a device.

-

list-device

-

Lists all block devices or character devices in a container.

-

Supported options are as follows:

-
  • --pretty: outputs data in JSON format.
  • --sub-partition: For a primary disk, add this flag to display the primary disk and its sub-partitions.
-

None

-

update-device

-

Updates the disk QoS.

-

Supported options are as follows:

-
  • --device-read-bps: sets the read rate limit for the block device (byte/s). You are advised to set this parameter to a value greater than or equal to 1024.
  • --device-read-iops: sets the read rate limit for the block device (I/O/s).
  • --device-write-bps: sets the write rate limit for the block device (byte/s). You are advised to set this parameter to a value greater than or equal to 1024.
  • --device-write-iops: sets the write rate limit for the block device (I/O/s).
-

None

-
- -### Constraints - -- You can add or delete devices when container instances are not running. After the operation is complete, you can start the container to view the device status. You can also dynamically add a device when the container is running. -- Do not concurrently run the **fdisk** command to format disks in a container and on the host. Otherwise, the container disk usage will be affected. -- When you run the **add-device** command to add a disk to a specific directory of a container, if the parent directory in the container is a multi-level directory \(for example, **/dev/a/b/c/d/e**\) and the directory level does not exist, isulad-tools will automatically create the corresponding directory in the container. When the disk is deleted, the created parent directory is not deleted. If you run the **add-device** command to add a device to this parent directory again, a message is displayed, indicating that a device already exists and cannot be added. -- When you run the** add-device** command to add a disk or update disk parameters, you need to configure the disk QoS. Do not set the write or read rate limit for the block device \(I/O/s or byte/s\) to a small value. If the value is too small, the disk may be unreadable \(the actual reason is the speed is too slow\), affecting service functions. -- When you run the **--blkio-weight-device** command to limit the weight of a specified block device, if the block device supports only the BFQ mode, an error may be reported, prompting you to check whether the current OS environment supports setting the weight of the BFQ block device. - -### Example - -- Start a system container, and set **hook spec** to the isulad hook execution script. - - ``` - [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init - eed1096c8c7a0eca6d92b1b3bc3dd59a2a2adf4ce44f18f5372408ced88f8350 - ``` - - -- Add a block device to a container. - - ``` - [root@localhost ~]# isulad-tools add-device ee /dev/sdb:/dev/sdb123 - Add device (/dev/sdb) to container(ee,/dev/sdb123) done. - [root@localhost ~]# isula exec ee fdisk -l /dev/sdb123 - Disk /dev/sdb123: 50 GiB, 53687091200 bytes, 104857600 sectors - Units: sectors of 1 * 512 = 512 bytes - Sector size (logical/physical): 512 bytes / 512 bytes - I/O size (minimum/optimal): 512 bytes / 512 bytes - Disklabel type: dos - Disk identifier: 0xda58a448 - - Device Boot Start End Sectors Size Id Type - /dev/sdb123p1 2048 104857599 104855552 50G 5 Extended - /dev/sdb123p5 4096 104857599 104853504 50G 83 Linux - ``` - -- Update the device information. - - ``` - [root@localhost ~]# isulad-tools update-device --device-read-bps /dev/sdb:10m ee - Update read bps for device (/dev/sdb,10485760) done. - ``` - -- Delete a device. - - ``` - [root@localhost ~]# isulad-tools remove-device ee /dev/sdb:/dev/sdb123 - Remove device (/dev/sdb) from container(ee,/dev/sdb123) done. - Remove read bps for device (/dev/sdb) done. - ``` - - -## NIC Management - -### Function Description - -isulad-tools allows you to insert physical or virtual NICs on the host to a container. If the NICs are not required, isulad-tools allows you to delete them from the container and return them to the host. In addition, the NIC configurations can be dynamically modified. To insert a physical NIC, add the NIC on the host to the container. To insert a virtual NIC, create a veth pair and insert its one end to the container. - -### Command Format - -``` -isulad-tools [COMMAND][OPTIONS] -``` - -In the preceding format: - -**COMMAND**: command related to NIC management. - -**OPTIONS**: option supported by the NIC management command. - -**container\_id**: container ID. - -### Parameter Description - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Function Description

-

Option Description

-

add-nic

-

Creates an NIC for a container.

-

Supported options are as follows:

-
  • --type: specifies the NIC type. Only eth and veth are supported.
  • --name: specifies the NIC name. The format is [host:]container. If host is not specified, a random value is used.
  • --ip: specifies the NIC IP address.
  • --mac: specifies the NIC MAC address.
  • --bridge: specifies the network bridge bound to the NIC.
  • --mtu: specifies the MTU value of the NIC. The default value is 1500.
  • --update-config-only: If this flag is set, only configuration files are updated and NICs are not added.
  • --qlen: specifies the value of QLEN. The default value is 1000.
-

remove-nic

-

Deletes NICs from a container and restores them to the host.

-

Supported options are as follows:

-
  • --type: specifies the NIC type.
  • --name: specifies the name of the NIC. The format is [host:]container.
-

list-nic

-

Lists all NICs in a container.

-

Supported options are as follows:

-
  • --pretty: outputs data in JSON format.
  • --filter: outputs filtered data in the specific format, for example, --filter' {"ip":"192.168.3.4/24", "Mtu":1500}'.
-

update-nic

-

Modifies configuration parameters of a specified NIC in a container.

-

Supported options are as follows:

-
  • --name: specifies the name of the NIC in the container. This parameter is mandatory.
  • --ip: specifies the NIC IP address.
  • --mac: specifies the NIC MAC address.
  • --bridge: specifies the network bridge bound to the NIC.
  • --mtu: specifies the MTU value of the NIC.
  • --update-config-only: If this flag is set, configuration files are updated and NICs are not updated.
  • --qlen: specifies the value of QLEN.
-
- -### Constraints - -- Physical NICs \(eth\) and virtual NICs \(veth\) can be added. -- When adding a NIC, you can also configure the NIC. The configuration parameters include **--ip**, **--mac**, **--bridge**, **--mtu**, **--qlen**. -- A maximum of eight physical NICs can be added to a container. -- If you run the **isulad-tools add-nic** command to add an eth NIC to a container and do not add a hook, you must manually delete the NIC before the container exits. Otherwise, the name of the eth NIC on the host will be changed to the name of that in the container. -- For a physical NIC \(except 1822 VF NIC\), use the original MAC address when running the **add-nic** command. Do not change the MAC address in the container, or when running the **update-nic** command. -- When using the **isulad-tools add-nic** command, set the MTU value. The value range depends on the NIC model. -- When using isulad-tools to add NICs and routes to containers, you are advised to run the **add-nic** command to add NICs and then run the **add-route** command to add routes. When using isulad-tools to delete NICs and routes from a container, you are advised to run the **remove-route** command to delete routes and then run the **remove-nic** command to delete NICs. -- When using isulad-tools to add NICs, add a NIC to only one container. - -### Example - -- Start a system container, and set **hook spec** to the isulad hook execution script. - - ``` - [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init - 2aaca5c1af7c872798dac1a468528a2ccbaf20b39b73fc0201636936a3c32aa8 - ``` - - -- Add a virtual NIC to a container. - - ``` - [root@localhost ~]# isulad-tools add-nic --type "veth" --name abc2:bcd2 --ip 172.17.28.5/24 --mac 00:ff:48:13:xx:xx --bridge docker0 2aaca5c1af7c - Add network interface to container 2aaca5c1af7c (bcd2,abc2) done - ``` - -- Add a physical NIC to a container. - - ``` - [root@localhost ~]# isulad-tools add-nic --type "eth" --name eth3:eth1 --ip 172.17.28.6/24 --mtu 1300 --qlen 2100 2aaca5c1af7c - Add network interface to container 2aaca5c1af7c (eth3,eth1) done - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >When adding a virtual or physical NIC, ensure that the NIC is in the idle state. Adding a NIC in use will disconnect the system network. - - -## Route Management - -### Function Description - -isulad-tools can be used to dynamically add or delete routing tables for system containers. - -### Command Format - -``` -isulad-tools [COMMAND][OPTIONS] [ARG...] -``` - -In the preceding format: - -**COMMAND**: command related to route management. - -**OPTIONS**: option supported by the route management command. - -**container\_id**: container ID. - -**ARG**: parameter corresponding to the command. - -### API Description - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Function Description

-

Option Description

-

Parameter Description

-

add-route

-

Adds the network routing rules to a container.

-

Supported options are as follows:

-

--update-config-only: If this parameter is configured, configuration files are updated and routing tables are not updated.

-

Parameter format: [{rule1},{rule2}]

-

Example of rule:

-

'[{"dest":"default", "gw":"192.168.10.1"},{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"}]'

-
  • dest: target network. If this parameter is left blank, the default gateway is used.
  • src: source IP address of a route.
  • gw: route gateway.
  • dev: network device.
-

remove-route

-

Deletes a route from a container.

-

Supported options are as follows:

-

--update-config-only: If this parameter is configured, only configuration files are updated and routes are not deleted from the container.

-

Parameter format: [{rule1},{rule2}]

-

Example of rule:

-

'[{"dest":"default", "gw":"192.168.10.1"},{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"}]'

-
  • dest: target network. If this parameter is left blank, the default gateway is used.
  • src: source IP address of a route.
  • gw: route gateway.
  • dev: network device.
-

list-route

-

Lists all routing rules in a container.

-

Supported options are as follows:

-
  • --pretty: outputs data in JSON format.
  • --filter: outputs filtered data in the specific format, for example, --filter' {"ip":"192.168.3.4/24", "Mtu":1500}'.
-

None

-
- -### Constraints - -- When using isulad-tools to add NICs and routes to containers, you are advised to run the **add-nic** command to add NICs and then run the **add-route** command to add routes. When using isulad-tools to delete NICs and routes from a container, you are advised to run the **remove-route** command to delete routes and then run the **remove-nic** command to delete NICs. -- When adding a routing rule to a container, ensure that the added routing rule does not conflict with existing routing rules in the container. - -### Example - -- Start a system container, and set **hook spec** to the isulad hook execution script. - - ``` - [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init - 0d2d68b45aa0c1b8eaf890c06ab2d008eb8c5d91e78b1f8fe4d37b86fd2c190b - ``` - - -- Use isulad-tools to add a physical NIC to the system container. - - ``` - [root@localhost ~]# isulad-tools add-nic --type "eth" --name enp4s0:eth123 --ip 172.17.28.6/24 --mtu 1300 --qlen 2100 0d2d68b45aa0 - Add network interface (enp4s0) to container (0d2d68b45aa0,eth123) done - ``` - - -- isulad-tools adds a routing rule to the system container. Format example: **\[\{"dest":"default", "gw":"192.168.10.1"\},\{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"\}\]**. If **dest** is left blank, its value will be **default**. - - ``` - [root@localhost ~]# isulad-tools add-route 0d2d68b45aa0 '[{"dest":"172.17.28.0/32", "gw":"172.17.28.5","dev":"eth123"}]' - Add route to container 0d2d68b45aa0, route: {dest:172.17.28.0/32,src:,gw:172.17.28.5,dev:eth123} done - ``` - -- Check whether a routing rule is added in the container. - - ``` - [root@localhost ~]# isula exec -it 0d2d68b45aa0 route - Kernel IP routing table - Destination Gateway Genmask Flags Metric Ref Use Iface - 172.17.28.0 172.17.28.5 255.255.255.255 UGH 0 0 0 eth123 - 172.17.28.0 0.0.0.0 255.255.255.0 U 0 0 0 eth123 - ``` - - -## Volume Mounting Management - -### Function Description - -In a common container, you can set the **--volume** parameter during container creation to mount directories or volumes of the host to the container for resource sharing. However, during container running, you cannot unmount directories or volumes that are mounted to the container, or mount directories or volumes of the host to the container. Only the system container can use the isulad-tools tool to dynamically mount directories or volumes of the host to the container and unmount directories or volumes from the container. - -### Command Format - -``` -isulad-tools [COMMAND][OPTIONS] [ARG...] -``` - -In the preceding format: - -**COMMAND**: command related to route management. - -**OPTIONS**: option supported by the route management command. - -**container\_id**: container ID. - -**ARG**: parameter corresponding to the command. - -### API Description - -**Table 1**    - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Function Description

-

Option Description

-

Parameter Description

-

add-path

-

Adds files or directories on the host to a container.

-

None

-

The parameter format is as follows:

-

hostpath:containerpath:permission [hostpath:containerpath:permission ...]

-

In the preceding format:

-

hostpath: path on the host for storing a volume.

-

containerpath: path on the container for storing a volume.

-

permission: operation permission on a mount path within the container.

-

remove-path

-

Deletes directories or files from the container and restores them to the host.

-

None

-

Parameter format: hostpath:containerpath[hostpath:containerpath ]

-

In the preceding format:

-

hostpath: path on the host for storing a volume.

-

containerpath: path on the container for storing a volume.

-

list-path

-

Lists all path directories in a container.

-

Supported options are as follows:

-

--pretty: outputs data in JSON format.

-

None

-
- -### Constraints - -- When running the **add-path** command, specify an absolute path as the mount path. -- The mount point /.sharedpath is generated on the host after the mount path is specified by running the **add-path** command. -- A maximum of 128 volumes can be added to a container. -- Do not overwrite the root directory \(/\) in a container with the host directory by running the **add-path** command. Otherwise, the function is affected. - -### Example - -- Start a system container, and set **hook spec** to the isulad hook execution script. - - ``` - [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init - e45970a522d1ea0e9cfe382c2b868d92e7b6a55be1dd239947dda1ee55f3c7f7 - ``` - - -- Use isulad-tools to mount a directory on the host to a container, implementing resource sharing. - - ``` - [root@localhost ~]# isulad-tools add-path e45970a522d1 /home/test123:/home/test123 - Add path (/home/test123) to container(e45970a522d1,/home/test123) done. - ``` - -- Create a file in the **/home/test123** directory on the host and check whether the file can be accessed in the container. - - ``` - [root@localhost ~]# echo "hello world" > /home/test123/helloworld - [root@localhost ~]# isula exec e45970a522d1 bash - [root@localhost /]# cat /home/test123/helloworld - hello world - ``` - -- Use isulad-tools to delete the mount directory from the container. - - ``` - [root@localhost ~]# isulad-tools remove-path e45970a522d1 /home/test123:/home/test123 - Remove path (/home/test123) from container(e45970a522d1,/home/test123) done - [root@localhost ~]# isula exec e45970a522d1 bash - [root@localhost /]# ls /home/test123/helloworld - ls: cannot access '/home/test123/helloworld': No such file or directory - ``` - - diff --git a/docs/en/docs/Container/environment-variable-persisting.md b/docs/en/docs/Container/environment-variable-persisting.md deleted file mode 100644 index f296c5c144245df505a9256b6373ace2a983f82d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/environment-variable-persisting.md +++ /dev/null @@ -1,51 +0,0 @@ -# Environment Variable Persisting - -- [Environment Variable Persisting](#environment-variable-persisting) - - -## Function Description - -In a system container, you can make the **env** variable persistent to the configuration file in the rootfs directory of the container by specifying the **--env-target-file** interface parameter. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--env-target-file

-
  • Variable of the string type.
  • The env persistent file must be in the rootfs directory and must be an absolute path.
-
- -## Constraints - -- If the target file specified by **--env-target-file** exists, the size cannot exceed 10 MB. -- The parameter specified by **--env-target-file** must be an absolute path in the rootfs directory. -- If the value of **--env** conflicts with that of **env** in the target file, the value of **--env** prevails. - -## Example - -Start a system container and specify the **env** environment variable and **--env-target-file** parameter. - -``` -[root@localhost ~]# isula run -tid -e abc=123 --env-target-file /etc/environment --system-container --external-rootfs /root/myrootfs none init -b75df997a64da74518deb9a01d345e8df13eca6bcc36d6fe40c3e90ea1ee088e -[root@localhost ~]# isula exec b7 cat /etc/environment -PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -TERM=xterm -abc=123 -``` - -The preceding information indicates that the **env** variable \(**abc=123**\) of the container has been made persistent to the **/etc/environment** configuration file. - diff --git a/docs/en/docs/Container/figures/en-us_image_0183048952.png b/docs/en/docs/Container/figures/en-us_image_0183048952.png deleted file mode 100644 index fe9074f8fba969795f1e1d40fb879e21d5fc2a7c..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/en-us_image_0183048952.png and /dev/null differ diff --git a/docs/en/docs/Container/figures/en-us_image_0221924926.png b/docs/en/docs/Container/figures/en-us_image_0221924926.png deleted file mode 100644 index 62ef0decdf6f1e591059904001d712a54f727e68..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/en-us_image_0221924926.png and /dev/null differ diff --git a/docs/en/docs/Container/figures/en-us_image_0221924927.png b/docs/en/docs/Container/figures/en-us_image_0221924927.png deleted file mode 100644 index ad5ed3f7beeb01e6a48707c4806606b41d687e22..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/en-us_image_0221924927.png and /dev/null differ diff --git a/docs/en/docs/Container/figures/isula-build_arch.png b/docs/en/docs/Container/figures/isula-build_arch.png deleted file mode 100644 index f92f15085820ce824bc2ca60ff7d6d25e95f1402..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/isula-build_arch.png and /dev/null differ diff --git a/docs/en/docs/Container/figures/kuasar_arch.png b/docs/en/docs/Container/figures/kuasar_arch.png deleted file mode 100644 index 69fbf889891c2f678590a929d1f91d5139569c27..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/kuasar_arch.png and /dev/null differ diff --git a/docs/en/docs/Container/figures/relationship-between-the-secure-container-and-peripheral-components.png b/docs/en/docs/Container/figures/relationship-between-the-secure-container-and-peripheral-components.png deleted file mode 100644 index 454fc025ecb88fef09472eef7cb29ca7a8164856..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/relationship-between-the-secure-container-and-peripheral-components.png and /dev/null differ diff --git a/docs/en/docs/Container/figures/secure-container.png b/docs/en/docs/Container/figures/secure-container.png deleted file mode 100644 index 2e8b48bdbd0766ec513e0654212cd16613eff826..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Container/figures/secure-container.png and /dev/null differ diff --git a/docs/en/docs/Container/iSulad_support_for_CDI.md b/docs/en/docs/Container/iSulad_support_for_CDI.md deleted file mode 100644 index e7e3cd7ab551a524f2f898fbb3c018d4cc0ea422..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/iSulad_support_for_CDI.md +++ /dev/null @@ -1,120 +0,0 @@ -# iSulad Support for CDI - -## Overview - -Container Device Interface (CDI) is a container runtime specification used to support third-party devices. - -CDI solves the following problems: -In Linux, only one device node needed to be exposed in a container in the past to enable device awareness of the container. However, as devices and software become more complex, vendors want to perform more operations, such as: - -- Exposing multiple device nodes to a container, mounting files from a runtime namespace to a container, or hiding procfs entries. -- Checking the compatibility between containers and devices. For example, checking whether a container can run on a specified device. -- Performing runtime-specific operations, such as virtual machines and Linux container-based runtimes. -- Performing device-specific operations, such as GPU memory cleanup and FPGA re-programming. - -In the absence of third-party device standards, vendors often have to write and maintain multiple plugins for different runtimes, or even contribute vendor-specific code directly in a runtime. In addition, the runtime does not expose the plugin system in a unified manner (or even not at all), resulting in duplication of functionality in higher-level abstractions (such as Kubernetes device plugins). - -To solve the preceding problem, CDI provides the following features: -CDI describes a mechanism that allows third-party vendors to interact with devices without modifying the container runtime. - -The mechanism is exposed as a JSON file (similar to the container network interface CNI), which allows vendors to describe the operations that the container runtime should perform on the OCI-based container. - -Currently, iSulad supports the [CDI v0.6.0](https://github.com/cncf-tags/container-device-interface/blob/v0.6.0/SPEC.md) specification. - -## Configuring iSulad to Support CDI - -Modify the **daemon.json** file as follows and restart iSulad: - -```json -{ - ... - "enable-cri-v1": true, - "cdi-spec-dirs": ["/etc/cdi", "/var/run/cdi"], - "enable-cdi": true -} -``` - -**cdi-spec-dirs** specifies the directory where CDI specifications are stored. If this parameter is not specified, the default value **/etc/cdi** or **/var/run/cdi** is used. - -## Examples - -### CDI Specification Example - -For details about each field, see [CDI v0.6.0](https://github.com/cncf-tags/container-device-interface/blob/v0.6.0/SPEC.md). - -```bash -$ mkdir /etc/cdi -$ cat > /etc/cdi/vendor.json < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

--force-rm=false

-

Deletes containers generated during the build process even if the build is not successful.

-

--no-cache=false

-

Builds cache without using cache.

-

-q, --quiet=false

-

Prevents the redundant information generation during the build.

-

--rm=true

-

Deletes the container generated during the build after the build is successful.

-

-t, --tag=""

-

Tag name of the image generated during the build.

-

--build-arg=[]

-

Configures the build parameters.

-

--label=[]

-

Image-related parameters. The description of each parameter is similar to that of the create command.

-

--isolation

-

Container isolation method.

-

--pull

-

Obtains the latest image during the build.

-
- -**Dockerfile Command** - -Dockerfile is used to describe how to build an image and automatically build a container. The format of all **Dockerfile** commands is _instruction_ _arguments_. - -   - -**FROM Command** - -Syntax: **FROM** _image_ or **FROM** _image_:_tag_ - -Function: Specifies a basic image, which is the first command for all Dockerfile files. If the tag of a basic image is not specified, the default tag name **latest** is used. - -   - -**RUN Command** - -Syntax: **RUN** _command_ \(for example, **run in a shell - \`/bin/sh -c\`**\) or - -**RUN \[**_executable_, _param1_, _param2_ ... **\]** \(in the **exec** command format\) - -Function: Runs any command in the image specified by the **FROM** command and then commits the result. The committed image can be used in later commands. The **RUN** command is equivalent to: - -**docker run** _image_ _command_ - -**docker commit** _container\_id_ - -   - -**Remarks** - -The number sign \(\#\) is used to comment out. - -   - -**MAINTAINER Command** - -Syntax: **MAINTAINER **_name_ - -Function: Specifies the name and contact information of the maintenance personnel. - -   - -**ENTRYPOINT Command** - -Syntax: **ENTRYPOINT cmd **_param1 param2..._ or **ENTRYPOINT \[**_"cmd", "param1", "param2"..._**\]** - -Function: Configures the command to be executed during container startup. - -   - -**USER Command** - -Syntax: **USER **_name_ - -Function: Specifies the running user of memcached. - -   - -**EXPOSE Command** - -Syntax: **EXPOSE **_port_** \[**_port_**...\]** - -Function: Enables one or more ports for images. - -   - -**ENV Command** - -Syntax: **ENV**_ key value_ - -Function: Configures environment variables. After the environment variables are configured, the **RUN** commands can be subsequently used. - -   - -**ADD Command** - -Syntax: **ADD**_ src dst_ - -Function: Copies a file from the _src_ directory to the _dest_ directory of a container. _src_ indicates the relative path of the source directory to be built. It can be the path of a file or directory, or a remote file URL. _dest_ indicates the absolute path of the container. - -   - -**VOLUME Command** - -Syntax: **VOLUME \["**_mountpoint_**"\]** - -Function: Creates a mount point for sharing a directory. - -   - -**WORKDIR Command** - -Syntax: **workdir **_path_ - -Function: Runs the **RUN**, **CMD**, and **ENTRYPOINT** commands to set the current working path. The current working path can be set multiple times. If the current working path is a relative path, it is relative to the previous **WORKDIR** command. - -   - -**CMD command** - -Syntax: **CMD \[**_"executable","param1","param2"_**\]** \(This command is similar to the **exec** command and is preferred.\) - -**CMD \["**_param1_**","**_param2_**"\]** \(The parameters are the default parameters for ENTRYPOINT.\) - -**CMD** _command_ _param1_ _param2_ \(This command is similar to the **shell** command.\) - -Function: A Dockerfile can contain only one CMD command. If there are multiple CMD commands, only the last one takes effect. - -   - -**ONBUILD Commands** - -Syntax: **ONBUILD \[**_other commands_**\]** - -Function: This command is followed by other commands, such as the **RUN** and **COPY** commands. This command is not executed during image build and is executed only when the current image is used as the basic image to build the next-level image. - -The following is a complete example of the Dockerfile command that builds an image with the sshd service installed. - - - - - -
FROM busybox
-ENV  http_proxy http://192.168.0.226:3128
-ENV  https_proxy https://192.168.0.226:3128
-RUN apt-get update && apt-get install -y openssh-server
-RUN mkdir -p /var/run/sshd
-EXPOSE 22
-ENTRYPOINT /usr/sbin/sshd -D
-
- -Example: - -1. Run the following command to build an image using the preceding Dockerfile: - - ``` - $ sudo docker build -t busybox:latest - ``` - -2. Run the following command to view the generated image: - - ``` - docker images | grep busybox - ``` - - -## history - -Syntax: **docker history \[**_options_**\]** _image_ - -Function: Displays the change history of an image. - -Parameter description: - --H, --human=true - -**--no-trunc=false**: Does not delete any output. - -**-q** and **--quiet=false**: Display only IDs. - -Example: - -``` -$ sudo docker history busybox:test -IMAGE CREATED CREATED BY SIZE COMMENT -be4672959e8b 15 minutes ago bash 23B -21970dfada48 4 weeks ago 128MB Imported from - -``` - -   - -## images - -Syntax: **docker images \[**_options_**\] \[**_name_**\]** - -Function: Lists existing images. The intermediate image is not displayed if no parameter is configured. - -Parameter description: - -**-a** and **--all=false**: Display all images. - -**-f** and **--filter=\[\]**: Specify a filtering value, for example, **dangling=true**. - -**--no-trunc=false**: Does not delete any output. - -**-q** and **--quiet=false**: Display only IDs. - -Example: - -``` -$ sudo docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox latest e02e811dd08f 2 years ago 1.09MB -``` - -   - -## import - -Syntax: **docker import URL|- \[**_repository_**\[**_:tag_**\]\]** - -Function: Imports a .tar package that contains rootfs as an image. This parameter corresponds to the **docker export** command. - -Parameter description: none. - -Example: - -Run the following command to generate a new image for **busybox.tar** exported using the **docker export** command: - -``` -$ sudo docker import busybox.tar busybox:test -sha256:a79d8ae1240388fd3f6c49697733c8bac4d87283920defc51fb0fe4469e30a4f -$ sudo docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox test a79d8ae12403 2 seconds ago 1.3MB -``` - -   - -## load - -Syntax: **docker load \[**_options_**\]** - -Function: Reloads an image from .tar package obtained by running the **docker save** command. This parameter corresponds to the **docker save** command. - -Parameter description: - -**-i** and **--input=""** can be used. - -Example: - -``` -$ sudo docker load -i busybox.tar -Loaded image ID: sha256:e02e811dd08fd49e7f6032625495118e63f597eb150403d02e3238af1df240ba -$ sudo docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox latest e02e811dd08f 2 years ago 1.09MB -``` - -## login - -Syntax: **docker login \[**_options_**\] \[**_server_**\]** - -Function: Logs in to an image server. If no server is specified, the system logs in to **https://index.docker.io/v1/** by default. - -Parameter description: - -**-e** and **--email=""**: Email address. - -**-p** and **--password=""**: Password. - -**-u** and **--username=""**: User name. - -Example: - -``` -$ sudo docker login -``` - -## logout - -Syntax: **docker logout \[**_server_**\]** - -Function: Logs out of an image server. If no server is specified, the system logs out of **https://index.docker.io/v1/** by default. - -Parameter description: none. - -Example: - -``` -$ sudo docker logout -``` - -## pull - -Syntax: **docker pull \[**_options_**\]** _name_**\[**_:tag_**\]** - -Function: Pulls an image from an official or private registry. - -Parameter description: - -**-a** and **--all-tags=false**: Download all images in a registry. \(A registry can be tagged with multiple tags. For example, a busybox registry may have multiple tags, such as **busybox:14.04**, **busybox:13.10**, **busybox:latest**. If **-a** is used, all busybox images with tags are pulled.\) - -Example: - -1. Run the following command to obtain the Nginx image from the official registry: - - ``` - $ sudo docker pull nginx - Using default tag: latest - latest: Pulling from official/nginx - 94ed0c431eb5: Pull complete - 9406c100a1c3: Pull complete - aa74daafd50c: Pull complete - Digest: sha256:788fa27763db6d69ad3444e8ba72f947df9e7e163bad7c1f5614f8fd27a311c3 - Status: Downloaded newer image for nginx:latest - ``` - - When an image is pulled, the system checks whether the dependent layer exists. If yes, the local layer is used. - -2. Pull an image from a private registry. - - Run the following command to pull the Fedora image from the private registry, for example, the address of the private registry is **192.168.1.110:5000**: - - ``` - $ sudo docker pull 192.168.1.110:5000/fedora - ``` - - -## push - -Syntax: **docker push** _name_**\[**_:tag_**\]** - -Function: Pushes an image to the image registry. - -Parameter description: none. - -Example: - -1. Run the following command to push an image to the private image registry at 192.168.1.110:5000. -2. Label the image to be pushed. \(The **docker tag** command is described in the following section.\) In this example, the image to be pushed is busybox:sshd. - - ``` - $ sudo docker tag ubuntu:sshd 192.168.1.110:5000/busybox:sshd - ``` - -3. Run the following command to push the tagged image to the private image registry: - - ``` - $ sudo docker push 192.168.1.110:5000/busybox:sshd - ``` - - During the push, the system automatically checks whether the dependent layer exists in the image registry. If yes, the layer is skipped. - - -## rmi - -Syntax: **docker rmi \[**_options_**\] **_image _**\[**_image..._**\]** - -Function: Deletes one or more images. If an image has multiple tags in the image library, only the untag operation is performed when the image is deleted. If the image has only one tag, the dependent layers are deleted in sequence. - -Parameter description: - -**-f** and **--force=false**: Forcibly delete an image. - -**--no-prune=false**: Does not delete parent images without tags. - -Example: - -``` -$ sudo docker rmi 192.168.1.110:5000/busybox:sshd -``` - -## save - -Syntax: **docker save \[**_options_**\] **_image _**\[**_image..._**\]** - -Function: Saves an image to a TAR package. The output is **STDOUT** by default. - -Parameter description: - -**-o** and **--output=""**: Save an image to a file rather than STDOUT. - -Example: - -``` -$ sudo docker save -o nginx.tar nginx:latest -$ ls -nginx.tar -``` - -## search - -Syntax: **docker search **_options_ _TERM_ - -Function: Searches for a specific image in the image registry. - -Parameter description: - -**--automated=false**: Displays the automatically built image. - -**--no-trunc=false**: Does not delete any output. - -**-s** and **--stars=0**: Display only images of a specified star level or higher. - -Example: - -1. Run the following command to search for Nginx in the official image library: - - ``` - $ sudo docker search nginx - NAME DESCRIPTION STARS OFFICIAL AUTOMATED - nginx Official build of Nginx. 11873 [OK] - jwilder/nginx-proxy Automated Nginx reverse proxy for docker con... 1645 [OK] - richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable of... 739 [OK] - linuxserver/nginx An Nginx container, brought to you by LinuxS... 74 - bitnami/nginx Bitnami nginx Docker Image 70 [OK] - tiangolo/nginx-rtmp Docker image with Nginx using the nginx-rtmp... 51 [OK] - ``` - -    - -2. Run the following command to search for busybox in the private image library. The address of the private image library must be added during the search. - - ``` - $ sudo docker search 192.168.1.110:5000/busybox - ``` - - -## tag - -Syntax: **docker tag \[**_options_**\] **_image_**\[**_:tag_**\] \[**_registry host/_**\]\[**_username/_**\]**_name_**\[**_:tag_**\]** - -Function: Tags an image to a registry. - -Parameter description: - -**-f** or **--force=false**: Forcibly replaces the original image when the same tag name exists. - -Example: - -``` -$ sudo docker tag busybox:latest busybox:test -``` - diff --git a/docs/en/docs/Container/image-management.md b/docs/en/docs/Container/image-management.md deleted file mode 100644 index e0455a9616a702e1d7993db9d118c35048f0a5c1..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/image-management.md +++ /dev/null @@ -1,449 +0,0 @@ -# Image Management - -- [Image Management](#image-management) - - [Docker Image Management](#docker-image-management) - - [Logging In to a Registry](#logging-in-to-a-registry) - - [Logging Out of a Registry](#logging-out-of-a-registry) - - [Pulling Images from a Registry](#pulling-images-from-a-registry) - - [Deleting Images](#deleting-images) - - [Adding an Image Tag](#adding-an-image-tag) - - [Loading Images](#loading-images) - - [Listing Images](#listing-images) - - [Inspecting Images](#inspecting-images) - - [Two-Way Authentication](#two-way-authentication) - - [Importing rootfs](#importing-rootfs) - - [Exporting rootfs](#exporting-rootfs) - - [Embedded Image Management](#embedded-image-management) - - [Loading Images](#loading-images-1) - - [Listing Images](#listing-images-1) - - [Inspecting Images](#inspecting-images-1) - - [Deleting Images](#deleting-images-1) - -## Docker Image Management - -### Logging In to a Registry - -#### Description - -The **isula login** command is run to log in to a registry. After successful login, you can run the **isula pull** command to pull images from the registry. If the registry does not require a password, you do not need to run this command before pulling images. - -#### Usage - -```shell -isula login [OPTIONS] SERVER -``` - -#### Parameters - -For details about the parameters in the **login** command, see **Appendix** > **Command Line Parameters** > **Table 1 login command parameters**. - -#### Example - -```shell -$ isula login -u abc my.csp-edge.com:5000 - -Login Succeeded -``` - -### Logging Out of a Registry - -#### Description - -The **isula logout** command is run to log out of a registry. If you run the **isula pull** command to pull images from the registry after logging out of the system, the image will fail to be pulled because you are not authenticated. - -#### Usage - -```shell -isula logout SERVER -``` - -#### Parameters - -For details about the parameters in the **logout** command, see **Appendix** > **Command Line Parameters** > **Table 2 logout command parameters**. - -#### Example - -```shell -$ isula logout my.csp-edge.com:5000 -Logout Succeeded -``` - -### Pulling Images from a Registry - -#### Description - -Pull images from a registry to the local host. - -#### Usage - -```shell -isula pull [OPTIONS] NAME[:TAG] -``` - -#### Parameters - -For details about the parameters in the **pull** command, see **Appendix** > **Command Line Parameters** > **Table 3 pull command parameters**. - -#### Example - -```shell -$ isula pull localhost:5000/official/busybox -Image "localhost:5000/official/busybox" pulling -Image "localhost:5000/official/busybox@sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff" pulled -``` - -### Deleting Images - -#### Description - -Delete one or more images. - -#### Usage - -```shell -isula rmi [OPTIONS] IMAGE [IMAGE...] -``` - -#### Parameters - -For details about the parameters in the **rmi** command, see **Appendix** > **Command Line Parameters** > **Table 4 rmi command parameters**. - -#### Example - -```shell -$ isula rmi rnd-dockerhub.huawei.com/official/busybox -Image "rnd-dockerhub.huawei.com/official/busybox" removed -``` - -### Adding an Image Tag - -#### Description - -Add an image tag. - -#### Usage - -```shell -isula tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG] -``` - -#### Parameters - -For details about the parameters in the **tag** command, see **Appendix** > **Command Line Parameters** > **Table 8 tag command parameters**. - -#### Example - -```shell -isula tag busybox:latest test:latest -``` - -### Loading Images - -#### Description - -Load images from a .tar package. The .tar package must be exported by using the **docker save** command or must be in the same format. - -#### Usage - -```shell -isula load [OPTIONS] -``` - -#### Parameters - -For details about the parameters in the **load** command, see **Appendix** > **Command Line Parameters** > **Table 5 load command parameters**. - -#### Example - -```shell -$ isula load -i busybox.tar -Load image from "/root/busybox.tar" success -``` - -### Listing Images - -#### Description - -List all images in the current environment. - -#### Usage - -```shell -isula images [OPTIONS] -``` - -#### Parameters - -For details about the parameters in the **images** command, see **Appendix** > **Command Line Parameters** > **Table 6 images command parameters**. - -#### Example - -```shell -$ isula images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox latest beae173ccac6 2021-12-31 03:19:41 1.184MB -``` - -### Inspecting Images - -#### Description - -After the configuration information of an image is returned, you can use the **-f** parameter to filter the information as needed. - -#### Usage - -```shell -isula inspect [options] CONTAINER|IMAGE [CONTAINER|IMAGE...] -``` - -#### Parameters - -For details about the parameters in the **inspect** command, see **Appendix** > **Command Line Parameters** > **Table 7 inspect command parameters**. - -#### Example - -```shell -$ isula inspect -f "{{json .image.id}}" rnd-dockerhub.huawei.com/official/busybox -"e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b" -``` - -### Two-Way Authentication - -#### Description - -After this function is enabled, iSulad and image repositories communicate over HTTPS. Both iSulad and image repositories verify the validity of each other. - -#### Usage - -The corresponding registry needs to support this function and iSulad needs to be configured as follows: - -1. Modify iSulad configuration \(default path: **/etc/isulad/daemon.json**\) and set **use-decrypted-key** to **false**. -2. Place related certificates in the folder named after the registry in the **/etc/isulad/certs.d** directory. For details about how to generate certificates, visit the official Docker website: - - [https://docs.docker.com/engine/security/certificates/](https://docs.docker.com/engine/security/certificates/) - - [https://docs.docker.com/engine/security/https/](https://docs.docker.com/engine/security/https/) - -3. Run the **systemctl restart isulad** command to restart iSulad. - -#### Parameters - -Parameters can be configured in the **/etc/isulad/daemon.json** file or carried when iSulad is started. - -```shell -isulad --use-decrypted-key=false -``` - -#### Example - -Set **use-decrypted-key** to **false**. - -```shell -$ cat /etc/isulad/daemon.json -{ - "group": "isulad", - "graph": "/var/lib/isulad", - "state": "/var/run/isulad", - "engine": "lcr", - "log-level": "ERROR", - "pidfile": "/var/run/isulad.pid", - "log-opts": { - "log-file-mode": "0600", - "log-path": "/var/lib/isulad", - "max-file": "1", - "max-size": "30KB" - }, - "log-driver": "stdout", - "hook-spec": "/etc/default/isulad/hooks/default.json", - "start-timeout": "2m", - "storage-driver": "overlay2", - "storage-opts": [ - "overlay2.override_kernel_check=true" - ], - "registry-mirrors": [ - "docker.io" - ], - "insecure-registries": [ - "rnd-dockerhub.huawei.com" - ], - "pod-sandbox-image": "", - "image-opt-timeout": "5m", - "native.umask": "secure", - "network-plugin": "", - "cni-bin-dir": "", - "cni-conf-dir": "", - "image-layer-check": false, - "use-decrypted-key": false, - "insecure-skip-verify-enforce": false -} -``` - -Place the certificate in the corresponding directory. - -```shell -$ pwd -/etc/isulad/certs.d/my.csp-edge.com:5000 -$ ls -ca.crt tls.cert tls.key -``` - -Restart iSulad. - -```shell -systemctl restart isulad -``` - -Run the **pull** command to download images from the registry: - -```shell -$ isula pull my.csp-edge.com:5000/busybox -Image "my.csp-edge.com:5000/busybox" pulling -Image "my.csp-edge.com:5000/busybox@sha256:f1bdc62115dbfe8f54e52e19795ee34b4473babdeb9bc4f83045d85c7b2ad5c0" pulled -``` - -### Importing rootfs - -#### Description - -Import a .tar package that contains rootfs as an image. Generally, the .tar package is exported by running the **export** command or a .tar package that contains rootfs in compatible format. Currently, the .tar, .tar.gz, .tgz, .bzip, .tar.xz, and .txz formats are supported. Do not use the TAR package in other formats for import. - -#### Usage - -```shell -isula import file REPOSITORY[:TAG] -``` - -After the import is successful, the printed character string is the image ID generated by the imported rootfs. - -#### Parameters - -For details about the parameters in the **import** command, see **Appendix** > **Command Line Parameters** > **Table 9 import command parameters**. - -#### Example - -```shell -$ isula import busybox.tar test -sha256:441851e38dad32478e6609a81fac93ca082b64b366643bafb7a8ba398301839d -$ isula images -REPOSITORY TAG IMAGE ID CREATED SIZE -test latest 441851e38dad 2020-09-01 11:14:35 1.168 MB -``` - -### Exporting rootfs - -#### Description - -Export the content of the rootfs of a container as a TAR package. The exported TAR package can be imported as an image by running the **import** command. - -#### Usage - -```shell -isula export [OPTIONS] [ID|NAME] -``` - -#### Parameters - -For details about the parameters in the **export** command, see **Appendix** > **Command Line Parameters** > **Table 10 export command parameters**. - -#### Example - -```shell -$ isula run -tid --name container_test test sh -d7e601c2ef3eb8d378276d2b42f9e58a2f36763539d3bfcaf3a0a77dc668064b -$ isula export -o rootfs.tar d7e601c -$ ls -rootfs.tar -``` - -## Embedded Image Management - -### Loading Images - -#### Description - -Load images based on the **manifest** files of embedded images. The value of **--type** must be set to **embedded**. - -#### Usage - -```shell -isula load [OPTIONS] --input=FILE --type=TYPE -``` - -#### Parameters - -For details about the parameters in the **load** command, see **Appendix** > **Command Line Parameters** > **Table 5 load command parameters**. - -#### Example - -```shell -$ isula load -i test.manifest --type embedded -Load image from "/root/work/bugfix/tmp/ci_testcase_data/embedded/img/test.manifest" success -``` - -### Listing Images - -#### Description - -List all images in the current environment. - -#### Usage - -```shell -isula images [OPTIONS] -``` - -#### Parameters - -For details about the parameters in the **images** command, see **Appendix** > **Command Line Parameters** > **Table 6 images command parameters**. - -#### Example - -```shell -$ isula images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox latest beae173ccac6 2021-12-31 03:19:41 1.184MB -``` - -### Inspecting Images - -#### Description - -After the configuration information of an image is returned, you can use the **-f** parameter to filter the information as needed. - -#### Usage - -```shell -isula inspect [options] CONTAINER|IMAGE [CONTAINER|IMAGE...] -``` - -#### Parameters - -For details about the parameters in the **inspect** command, see **Appendix** > **Command Line Parameters** > **Table 7 inspect command parameters**. - -#### Example - -```shell -$ isula inspect -f "{{json .created}}" test:v1 -"2018-03-01T15:55:44.322987811Z" -``` - -### Deleting Images - -#### Description - -Delete one or more images. - -#### Usage - -```shell -isula rmi [OPTIONS] IMAGE [IMAGE...] -``` - -#### Parameters - -For details about the parameters in the **rmi** command, see **Appendix** > **Command Line Parameters** > **Table 4 rmi command parameters**. - -#### Example - -```shell -$ isula rmi test:v1 -Image "test:v1" removed -``` diff --git a/docs/en/docs/Container/installation-and-configuration-3.md b/docs/en/docs/Container/installation-and-configuration-3.md deleted file mode 100644 index 17277fac22f3490e9477a665e260887cb4b94bb3..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/installation-and-configuration-3.md +++ /dev/null @@ -1,405 +0,0 @@ -# Installation and Configuration - -This chapter describes important configurations related to the installation of the open source container Docker. - -## Precautions - -- The root permission is required for installing a Docker container. -- The **docker-engine** RPM package cannot be installed together with the **containerd**, **runc**, or **podman** RPM package. This is because the **docker-engine** RPM package contains all components required for Docker running, including **containerd**, **runc**, and **docker** binary files. Yet the **containerd**, **runc**, and **podman** RPM packages also contain the corresponding binary files. Software package conflicts may occur due to repeated installation. - -## Basic Installation Configuration - -### Daemon Parameter Configuration - -You can add configuration items to the **/etc/docker/daemon.json** file to customize parameters. You can run the **dockerd --help** command to view related configuration items and their usage methods. A configuration example is as follows: - -```shell -$ cat /etc/docker/daemon.json -{ - "debug": true, - "storage-driver": "overlay2", - "storage-opts": ["overlay2.override_kernel_check=true"] -} -``` - -### Daemon Running Directory Configuration - -Re-configuring various running directories and files \(including **--graph** and **--exec-root**\) may cause directory conflicts or file attribute changes, affecting the normal use of applications. - ->![](./public_sys-resources/icon-notice.gif) **NOTICE:** ->Therefore, the specified directories or files should be used only by Docker to avoid file attribute changes and security issues caused by conflicts. - -- Take **--graph** as an example. When **/new/path/** is used as the new root directory of the daemon, if a file exists in **/new/path/** and the directory or file name conflicts with that required by Docker \(for example, **containers**, **hooks**, and **tmp**\), Docker may update the original directory or file attributes, including the owner and permission. - ->![](./public_sys-resources/icon-notice.gif) **NOTICE:** ->From Docker 17.05, the **--graph** parameter is marked as **Deprecated** and replaced with the **--data-root** parameter. - -### Daemon Network Configuration - -- After the network segment of the docker0 bridge is specified by using the **--bip** parameter on Docker daemon, if the **--bip** parameter is deleted during the next Docker daemon restart, the docker0 bridge uses the previous value of **--bip**, even if the docker0 bridge is deleted before the restart. The reason is that Docker saves the network configuration and restores the previous configuration by default during the next restart. -- When running the **docker network create** command to concurrently create networks, you can create two networks with the same name. The reason is that Docker networks are distinguished by IDs. The name is only an alias that is easy to identify and may not be unique. -- In the Docker bridge network mode, a Docker container establishes external communication through NAT on the host. When Docker daemon starts a Docker container, a docker-proxy process is started for each port mapped on the host to access the proxy. It is recommended that you map only the necessary ports when using userland-proxy to reduce the resources consumed by the port mapping of docker-proxy. - -### Daemon umask Configuration - -The default **umask** value of the main container process and exec process is **0022**. To meet security specifications and prevent containers from being attacked, the default value of **umask** is changed to **0027** after runC implementation is modified. After the modification, the other groups cannot access new files or directories. - -The default value of **umask** is **0027** when Docker starts a container. You can change the value to **0022** by running the **--exec-opt native.umask=normal** command during container startup. - ->![](./public_sys-resources/icon-notice.gif) **NOTICE:** ->If **native.umask** is configured in **docker create** or **docker run** command, its value is used. - -For details, see the parameter description in [docker create](./container-management-2.md#create) and [docker run](./container-management-2.md#run). - -### Daemon Start Time - -The Docker service is managed by systemd, which restricts the startup time of each service. If the Docker service fails to be started within the specified time, the possible causes are as follows: - -- If Docker daemon is started for the first time using devicemapper, the Docker daemon needs to perform the initialization operation on the device. This operation, however, will perform a large number of disk I/O operations. When the disk performance is poor or many I/O conflicts exist, the Docker daemon startup may time out. devicemapper needs to be initialized only once and does not need to be initialized again during later Docker daemon startup. -- If the usage of the current system resources is too high, the system responses slowly, all operations in the system slow down, and the startup of the Docker service may time out. -- During the restart, a daemon traverses and reads configuration files and the init layer and writable layer configurations of each container in the Docker working directory. If there are too many containers \(including the created and exited containers\) in the current system and the disk read and write performance is limited, the startup of the Docker service may time out due to the long-time daemon traversing. - -If the service startup times out, you are advised to rectify the fault as follows: - -- Ensure that the container orchestration layer periodically deletes unnecessary containers, especially the exited containers. -- Based on performance requirements of the solution, adjust the cleanup period of the orchestration layer and the start time of the Docker service. - -### Journald Component - -After systemd-journald is restarted, Docker daemon needs to be restarted. Journald obtains the Docker daemon logs through a pipe. If the journald service is restarted, the pipe is disabled. The write operation of Docker logs triggers the SIGPIPE signal, which causes the Docker daemon crash. If this signal is ignored, the subsequent Docker daemon logs may fail to be recorded. Therefore, you are advised to restart Docker daemon after the journald service is restarted or becomes abnormal, ensuring that Docker logs can be properly recorded and preventing status exceptions caused by daemon crash. - -### Firewalld Component - -You need to restart the Docker service after restarting or starting firewalld. - -- When the firewalld service is started, the iptables rules of the current system are cleared. Therefore, if the firewalld service is restarted during Docker daemon startup, the Docker service may fail to insert iptables rules, causing the Docker service startup failure. -- If the firewalld service is restarted after the Docker service is started, or the status of the firewalld service \(service paused or resumed\) is changed, the iptables rules of the Docker service are deleted. As a result, the container with port mapping fails to be created. - -### Iptables Component - -If the **--icc=false** option is added in Docker, the communication between containers can be restricted. However, if the OS has some rules, the communication between containers may not be restricted. For example: - -```text -Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) -... -0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 -... -0 0 DROP all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0 -... -``` - -In the **Chain FORWARD** command, the ACCEPT icmp rule is added to DROP. As a result, after the **--icc=false** option is added, containers can be pinged, but the peer end is unreachable if UDP or TCP is used. - -Therefore, if you want to add the **--icc=false** option when using Docker in a container OS, you are advised to clear iptables rules on the host first. - -### Audit Component - -You can configure audit for Docker. However, this configuration is not mandatory. For example: - -```text --w /var/lib/docker -k docker --w /etc/docker -k docker --w /usr/lib/systemd/system/docker.service -k docker --w /usr/lib/systemd/system/docker.socket -k docker --w /etc/sysconfig/docker -k docker --w /usr/bin/docker-containerd -k docker --w /usr/bin/docker-runc -k docker --w /etc/docker/daemon.json -k docker -``` - -Configuring audit for Docker brings certain benefits for auditing, while it does not have any substantial effects on attack defense. In addition, the audit configurations cause serious efficiency problems, for example, the system may not respond smoothly. Therefore, exercise caution in the production environment. - -The following uses **-w /var/lib/docker -k docker** as an example to describe how to configure Docker audit. - -```shell -cat /etc/audit/rules.d/audit.rules | grep docker -w /var/lib/docker/ -k docker -auditctl -R /etc/audit/rules.d/audit.rules | grep docker -auditctl -l | grep docker -w /var/lib/docker/ -p rwxa -k docker -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->**-p \[r|w|x|a\]** and **-w** are used together to monitor the read, write, execution, and attribute changes \(such as timestamp changes\) of the directory. In this case, any file or directory operation in the **/var/lib/docker** directory will be recorded in the **audit.log** file. As a result, too many logs will be recorded in the **audit.log** file, which severely affects the memory or CPU usage of the auditd, and further affects the OS. For example, logs similar to the following will be recorded in the **/var/log/audit/audit.log** file each time the **ls /var/lib/docker/containers** command is executed: - -```text -type=SYSCALL msg=audit(1517656451.457:8097): arch=c000003e syscall=257 success=yes exit=3 a0=ffffffffffffff9c a1=1b955b0 a2=90800 a3=0 items=1 ppid=17821 pid=1925 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts6 ses=4 comm="ls" exe="/usr/bin/ls" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="docker"type=CWD msg=audit(1517656451.457:8097): cwd="/root"type=PATH msg=audit(1517656451.457:8097): item=0 name="/var/lib/docker/containers" inode=1049112 dev=fd:00 mode=040700 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:container_var_lib_t:s0 objtype=NORMAL -``` - -### Security Configuration seccomp - -During the container network performance test, it is found that the performance of Docker is lower than that of the native kernel namespace. After seccomp is enabled, system calls \(such as sendto\) are not performed through system\_call\_fastpath. Instead, tracesys is called, which greatly deteriorates the performance. Therefore, you are advised to disable seccomp in container scenarios where services require high performance. For example: - -```shell -docker run -itd --security-opt seccomp=unconfined busybox:latest -``` - -### Do Not Modify Private Directory of Docker Daemon - -Do not modify the root directory used by Docker \(**/var/lib/docker** by default\), the directory during operation \(**/run/docker** by default\), or the files or directories in the two directories. The forbidden operations include deleting files, adding files, creating soft or hard links for the directories or files, or modifying attributes, permissions, or contents of the files. If any modification is required, contact the Euler container team for review. - -### Precautions for Common Users in the Scenario Where a Large Number of Containers Are Deployed - -The maximum number of processes that a common user can create on an OS host can be restricted by creating the **/etc/security/limits.d/20-nproc.conf** file in the system. Similarly, the maximum number of processes that a common user can create in a container is determined by the value in the **/etc/security/limits.d/20-nproc.conf** file in the container image, as shown in the following example: - -```shell -$ cat /etc/security/limits.conf -* soft nproc 4096 -``` - -If an error is reported due to insufficient resources when a large number of containers are deployed by a common user, increase the value **4096** in the **/etc/security/limits.d/20-nproc.conf** file. - -Configure the maximum value based on the maximum capability of the kernel, as shown in the following example: - -```shell -$ sysctl -a | grep pid_max -kernel.pid_max = 32768 -``` - -## Storage Driver Configuration - -This Docker version supports two storage drivers: overlay2 and devicemapper. Since overlay2 has better performance than devicemapper, it is recommended that overlay2 be preferentially used in the production environment. - -### overlay2 Storage Driver Configuration - -#### Configuration Methods - -overlay2 is the default storage driver of Docker. You can also use either of the following methods to explicitly configure the driver: - -- Edit the **/etc/docker/daemon.json** file to explicitly configure the **storage-driver** field. - - ```shell - $ cat /etc/docker/daemon.json - { - "storage-driver": "overlay2" - } - ``` - -- Edit the **/etc/sysconfig/docker-storage** file and explicitly configure the Docker daemon startup parameters. - - ```shell - $ cat /etc/sysconfig/docker-storage - DOCKER_STORAGE_OPTIONS="--storage-driver=overlay2" - ``` - -#### Precautions - -- When you perform lifecycle management operations on some containers, an error may be reported, indicating that the corresponding rootfs or executable file cannot be found. -- If the health check of a container is configured to execute executable files in the container, an error may be reported, which causes the health check failure of the container. - -- When you use overlay2 as the graphdriver and modify an image file in a container for the first time, the modification fails if the file size is greater than the remaining space of the system. Even if a little modification on the file is involved, the whole file must be copied to the upper layer. If the remaining space is insufficient, the modification fails. -- Compared with common file systems, the overlay2 file system has the following behavior differences: - - Kernel version - - overlay2 is compatible only with the native kernel 4.0 or later. You are advised to use the Ext4 file system. - - - Copy-UP performance - - Modifying files at the lower layer triggers file replication to the upper layer. Data block replication and fsync are time-consuming. - - - Rename directories - - The rename system call is allowed only when both the source and the destination paths are at the merged layer. Otherwise, the EXDEV error is reported. - - Kernel 4.10 introduces the redirect directory feature to fix this issue. The corresponding kernel option is **CONFIG\_OVERLAY\_FS\_REDIRECT\_DIR**. - - When overlay2 is used, a file system directory fails to be renamed because the related feature configured in the **/sys/module/overlay/parameters/redirect\_dir** file has been disabled. To use this feature, you need to manually set **/sys/module/overlay/parameters/redirect\_dir** to **Y**. - - - Hard link disconnection - - If there are multiple hard links in the lower-layer directory, writing data to the merged layer will trigger Copy-UP, resulting in hard link disconnection. - - The index feature is introduced in kernel 4.13 to fix this issue. The corresponding kernel option is **CONFIG\_OVERLAY\_FS\_INDEX**. Note that this option is not forward compatible and does not support hot upgrade. - - - Changes of **st\_dev** and **st\_ino** - - After Copy-UP is triggered, you can view only new files at the merged layer, and inodes change. Although **attr** and **xattr** can be replicated, **st\_dev** and **st\_ino** are unique and cannot be replicated. As a result, you can run **stat** and **ls** commands to check inode changes accordingly. - - - fd change - - Before Copy-UP is triggered, you can obtain the descriptor fd1 when opening a file in read-only mode. After Copy-UP is trigger, you can obtain the descriptor fd2 when opening the file with the same name. The two descriptors point to different files. The data written to fd2 is not displayed in fd1. - -#### Abnormal Scenarios - -When a container uses the overlay2 storage driver, mount points may be overwritten. - -#### Abnormal Scenario: Mount Point Being Overwritten - -In the faulty container, there is a mount point in **/var/lib/docker/overlay2**. - -```shell -$ mount -l | grep overlay -overlay on /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/merged type overlay (rw,relatime,seclabel,lowerdir=/var/lib/docker/overlay2/l/JL5PZQLNDCIBU3ZOG3LPPDBHIJ:/var/lib/docker/overlay2/l/ELRPYU4JJG4FDPRLZJCZZE4UO6,upperdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/diff,workdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/work) -/dev/mapper/dm-root on /var/lib/docker/overlay2 type ext4 (rw,relatime,seclabel,data=ordered) -``` - -An error as follows may occur when some Docker commands are executed: - -```shell -$ docker rm 1348136d32 -docker rm: Error response from daemon: driver "overlay2" failed to remove root filesystem for 1348136d32: error while removing /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785: invalid argument -``` - -You will find that the rootfs of the corresponding container cannot be found on the host. However, this does not mean that the rootfs is lost. The rootfs is overwritten by the mount point in **/var/lib/docker/overlay2**, and services are still running properly. The solutions are as follows: - -- Solution 1 - 1. Run the following command to check the graphdriver used by Docker: - - ```shell - docker info | grep "Storage Driver" - ``` - - 2. Run the following commands to query the current mount point: - - ```shell - # Devicemapper - mount -l | grep devicemapper - # Overlay2 - mount -l | grep overlay2 - ``` - - The output format is _A_ on _B_ type _C_ \(_D_\). - - - _A_: block device name or **overlay** - - _B_: mount point - - _C_: file system type - - _D_: mounting attribute - - 3. Run the **umount** command on the mount points \(_B_\) one by one from bottom to top. - 4. Run the **docker restart** command on all the containers or delete all the containers. - 5. Run the following command to restart Docker: - - ```shell - systemctl restart docker - ``` - -- Solution 2 - 1. Migrate services. - 2. Restart nodes. - -### devicemapper Storage Driver Configuration - -If you need to set the storage driver of Docker to devicemapper, you can also use either of the following methods to explicitly configure the driver: - -- Edit the **/etc/docker/daemon.json** file to explicitly configure the **storage-driver** field. - - ```shell - $ cat /etc/docker/daemon.json - { - "storage-driver": "devicemapper" - } - ``` - -- Edit the **/etc/sysconfig/docker-storage** file and explicitly configure the Docker daemon startup parameters. - - ```shell - $ cat /etc/sysconfig/docker-storage - DOCKER_STORAGE_OPTIONS="--storage-driver=devicemapper" - ``` - -#### Precautions - -- To use devicemapper, you must use the direct-lvm mode. For details about the configuration method, refer to . -- When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function. -- Do not set both the following two parameters in the **/etc/lvm/profile/docker-thinpool.profile** file to **100**: - - ```text - activation { - thin_pool_autoextend_threshold=80 - thin_pool_autoextend_percent=20 - } - ``` - -- You are advised to add **--storage-opt dm.use\_deferred\_deletion=true** and **--storage-opt dm.use\_deferred\_removal=true** when using devicemapper. -- When devicemapper is used, you are advised to use Ext4 as the container file system. You need to add **--storage-opt dm.fs=ext4** to the configuration parameters of Docker daemon. -- If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon. -- When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool. - -**Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on Docker Daemon** - -- Generally, the path of the deviceset-metadata file is **/var/lib/docker/devicemapper/metadata/deviceset-metadata** during container startup. -- If user namespaces are used, the path of the deviceset-metadata file is **/var/lib/docker/**_userNSUID.GID_**/devicemapper/metadata/deviceset-metadata**. -- When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the **BaseDeviceUUID** content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the **BaseDeviceUUID** content in the deviceset-metadata file. Otherwise, the Docker service fails to be restarted. - -## Impact of Forcibly Killing Docker Background Processes - -### Semaphores May Be Residual - -When the devicemapper is used as the graphdriver, forcible killing may cause residual semaphores. Docker creates semaphores when performing operations on devicemapper. If daemon is forcibly killed before the semaphores are released, the release may fail. A maximum of one semaphore can be leaked at a time, and the leakage probability is low. However, the Linux OS has an upper limit on semaphores. When the number of semaphore leakage times reaches the upper limit, new semaphores cannot be created. As a result, Docker daemon fails to be started. The troubleshooting method is as follows: - -1. Check the residual semaphores in the system. - - ```shell - $ ipcs - ------ Message Queues -------- - key msqid owner perms used-bytes messages - ------ Shared Memory Segments -------- - key shmid owner perms bytes nattch status - ------ Semaphore Arrays -------- - key semid owner perms nsems - 0x0d4d3358 238977024 root 600 1 - 0x0d4d0ec9 270172161 root 600 1 - 0x0d4dc02e 281640962 root 600 1 - ``` - -2. Run the **dmsetup** command to check semaphores created by devicemapper. The semaphore set is the subset of the system semaphores queried in the previous step. - - ```shell - dmsetup udevcookies - ``` - -3. Check the upper limit of kernel semaphores. The fourth value is the upper limit of the current system semaphores. - - ```shell - $ cat /proc/sys/kernel/sem - 250 32000 32 128 - ``` - - If the number of residual semaphores in step 1 is the same as the upper limit of semaphores in step 3, the number of residual semaphores reaches the upper limit. In this case, Docker daemon cannot be normally started. You can run the following command to increase the upper limit to restart Docker: - - ```shell - echo 250 32000 32 1024 > /proc/sys/kernel/sem - ``` - - You can also run the following command to manually clear the residual devicemapper semaphores. The following describes how to clear the devicemapper semaphores applied one minute ago. - - ```shell - $ dmsetup udevcomplete_all 1 - This operation will destroy all semaphores older than 1 minutes with keys that have a prefix 3405 (0xd4d). - Do you really want to continue? [y/n]: y - 0 semaphores with keys prefixed by 3405 (0xd4d) destroyed. 0 skipped. - ``` - -### NICs May Be Residual - -When a container is started in bridge mode, forcibly killing may cause residual NICs. In bridge network mode, when Docker creates a container, a pair of veths are created on the host, and then the NIC information is saved to the database. If daemon is forcibly killed before the NIC information is saved to the database of Docker, the NIC cannot be associated with Docker and cannot be deleted during the next startup because Docker deletes unused NICs from its database. - -### Failed to Restart a Container - -If container hook takes a long time, and containerd is forcibly killed during container startup, the container start operation may fail. When containerd is forcibly killed during container startup, an error is returned for the Docker start operation. After containerd is restarted, the last startup may still be in the **runc create** execution phase \(executing the user-defined hook may take a long time\). If you run the **docker start** command again to start the container, the following error message may be displayed: - -```text -Error response from daemon: oci runtime error: container with id exists: xxxxxx -``` - -This error is caused by running **runc create** on an existing container \(or being created\). After the **runc create** operation corresponding to the first start operation is complete, the **docker start** command can be successfully executed. - -The execution of hook is not controlled by Docker. In this case, if the container is recycled, the containerd process may be suspended when an unknown hook program is executed. In addition, the risk is controllable \(although the creation of the current container is affected in a short period\). - -- After the first operation is complete, the container can be successfully started again. -- Generally, a new container is created after the container fails to be started. The container that fails to be started cannot be reused. - -In conclusion, this problem has a constraint on scenarios. - -### Failed to Restart the Docker Service - -The Docker service cannot be restarted properly due to frequent startup in a short period The Docker system service is monitored by systemd. If the Docker service is restarted for more than five times within 10s, the systemd service detects the abnormal startup. Therefore, the Docker service is disabled. Docker can respond to the restart command and be normally restarted only when the next period of 10s starts. - -## Impact of System Power-off - -When a system is unexpectedly powered off or system panic occurs, Docker daemon status may not be updated to the disk in time. As a result, Docker daemon is abnormal after the system is restarted. The possible problems include but are not limited to the following: - -- A container is created before the power-off. After the restart, the container is not displayed when the **docker ps –a** command is run, as the file status of the container is not updated to the disk. As a result, daemon cannot obtain the container status after the restart. -- Before the system power-off, a file is being written. After daemon is restarted, the file format is incorrect or the file content is incomplete. As a result, loading fails. -- As Docker database \(DB\) will be damaged during power-off, all DB files in **data-root** will be deleted during node restart. Therefore, the following information created before the restart will be deleted after the restart: - - Network: Resources created through Docker network will be deleted after the node is restarted. - - Volume: Resources created through Docker volume will be deleted after the node is restarted. - - Cache construction: The cache construction information will be deleted after the node is restarted. - - Metadata stored in containerd: Metadata stored in containerd will be recreated when a container is started. Therefore, the metadata stored in containerd will be deleted when the node is restarted. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >If you want to manually clear data and restore the environment, you can set the environment variable **DISABLE\_CRASH\_FILES\_DELETE** to **true** to disable the function of clearing DB files when the daemon process is restarted due to power-off. diff --git a/docs/en/docs/Container/installation-and-deployment-2.md b/docs/en/docs/Container/installation-and-deployment-2.md deleted file mode 100644 index dfb594ccbd6c75205036a8c9cf256ee19ab7f11a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/installation-and-deployment-2.md +++ /dev/null @@ -1,129 +0,0 @@ -# Installation and Deployment - -- [Installation and Deployment](#installation-and-deployment) - - [Installation Methods](#installation-methods) - - [Prerequisites](#prerequisites) - - [Installation Procedure](#installation-procedure) - - [Deployment Configuration](#deployment-configuration) - - [Configuring the Docker Engine](#configuring-the-docker-engine) - - [iSulad Configuration](#isulad-configuration) - - [Configuration.toml](#configurationtoml) - - -## Installation Methods - -### Prerequisites - -- The root permission is required for installing a Kata container. -- For better performance experience, a Kata container needs to run on the bare metal server and cannot run on VMs. -- A Kata container depends on the following components \(openEuler 1.0 version\). Ensure that the required components have been installed in the environment. To install iSulad, refer to [Installation Configuration](./installation-configuration.md). - - docker-engine - - qemu - - -### Installation Procedure - -Released Kata container components are integrated in the **kata-containers-**_version_**.rpm** package. You can run the **rpm** command to install the corresponding software. - -``` -rpm -ivh kata-containers-.rpm -``` - -## Deployment Configuration - -### Configuring the Docker Engine - -To enable the Docker engine to support kata-runtime, perform the following steps to configure the Docker engine: - -1. Ensure that all software packages \(**docker-engine** and **kata-containers**\) have been installed in the environment. -2. Stop the Docker engine. - - ``` - systemctl stop docker - ``` - -3. Modify the configuration file **/etc/docker/daemon.json** of the Docker engine and add the following configuration: - - ``` - { - "runtimes": { - "kata-runtime": { - "path": "/usr/bin/kata-runtime", - "runtimeArgs": [ - "--kata-config", - "/usr/share/defaults/kata-containers/configuration.toml" - ] - } - } - } - ``` - -4. Restart the Docker engine. - - ``` - systemctl start docker - ``` - - -### iSulad Configuration - -To enable the iSulad to support the new container runtime kata-runtime, perform the following steps which are similar to those for the container engine docker-engine: - -1. Ensure that all software packages \(iSulad and kata-containers\) have been installed in the environment. -2. Stop iSulad. - - ``` - systemctl stop isulad - ``` - -3. Modify the **/etc/isulad/daemon.json** configuration file of the iSulad and add the following configurations: - - ``` - { - "runtimes": { - "kata-runtime": { - "path": "/usr/bin/kata-runtime", - "runtime-args": [ - "--kata-config", - "/usr/share/defaults/kata-containers/configuration.toml" - ] - } - } - } - ``` - -4. Restart iSulad. - - ``` - systemctl start isulad - ``` - - -### Configuration.toml - -The Kata container provides a global configuration file configuration.toml. Users can also customize the path and configuration options of the Kata container configuration file. - -In the **runtimeArges** field of Docker engine, you can use **--kata-config** to specify a private file. The default configuration file path is **/usr/share/defaults/kata-containers/configuration.toml**. - -The following lists the common fields in the configuration file. For details about the configuration file options, see [configuration.toml](#configuration-toml-31.md). - -1. hypervisor.qemu - - **path**: specifies the execution path of the virtualization QEMU. - - **kernel**: specifies the execution path of the guest kernel. - - **initrd**: specifies the guest initrd execution path. - - **machine\_type**: specifies the type of the analog chip. The value is **virt** for the ARM architecture and **pc** for the x86 architecture. - - **kernel\_params**: specifies the running parameters of the guest kernel. - -2. proxy.kata - - **path**: specifies the kata-proxy running path. - - **enable\_debug**: enables the debugging function for the kata-proxy process. - -3. agent.kata - - **enable\_blk\_mount**: enables guest mounting of the block device. - - **enable\_debug**: enables the debugging function for the kata-agent process. - -4. runtime - - **enable\_cpu\_memory\_hotplug**: enables CPU and memory hot swap. - - **enable\_debug**: enables debugging for the kata-runtime process. - - diff --git a/docs/en/docs/Container/installation-configuration.md b/docs/en/docs/Container/installation-configuration.md deleted file mode 100644 index c800b0dc550091c057bb26ec7b7038fa39900492..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/installation-configuration.md +++ /dev/null @@ -1,1019 +0,0 @@ -# Installation and Configuration - -- [Installation and Configuration](#installation-and-configuration) - - [Installation Methods](#installation-methods) - - [Deployment Configuration](#deployment-configuration) - -## Installation Methods - -iSulad can be installed by running the **yum** or **rpm** command. The **yum** command is recommended because dependencies can be installed automatically. - -This section describes two installation methods. - -- \(Recommended\) Run the following command to install iSulad: - - ```bash - sudo yum install -y iSulad - ``` - -- If the **rpm** command is used to install iSulad, you need to download and manually install the RMP packages of iSulad and all its dependencies. To install the RPM package of a single iSulad \(the same for installing dependency packages\), run the following command: - - ```bash - # sudo rpm -ihv iSulad-xx.xx.xx-xx.xxx.aarch64.rpm - ``` - -## Deployment Configuration - -After iSulad is installed, you can perform related configurations as required. - -### Configuration Mode - -The iSulad server daemon **isulad** can be configured with a configuration file or by running the **isulad --xxx** command. The priority in descending order is as follows: CLI \> configuration file \> default configuration in code. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->If systemd is used to manage the iSulad process, modify the **OPTIONS** field in the **/etc/sysconfig/iSulad** file, which functions the same as using the CLI. - -- **CLI** - - During service startup, configure iSulad using the CLI. To view the configuration options, run the following command: - - ```bash - # isulad --help - isulad - - lightweight container runtime daemon - - Usage: isulad [global options] - - GLOBAL OPTIONS: - - --authorization-plugin Use authorization plugin - --cgroup-parent Set parent cgroup for all containers - --cni-bin-dir The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin - --cni-conf-dir The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d - --container-log-driver Set default container log driver, such as: json-file - --container-log-opts Set default container log driver options, such as: max-file=7 to set max number of container log files - --default-ulimit Default ulimits for containers (default []) - -e, --engine Select backend engine - -g, --graph Root directory of the iSulad runtime - -G, --group Group for the unix socket(default is isulad) - --help Show help - --hook-spec Default hook spec file applied to all containers - -H, --host The socket name used to create gRPC server - --image-layer-check Check layer intergrity when needed - --insecure-registry Disable TLS verification for the given registry - --insecure-skip-verify-enforce Force to skip the insecure verify(default false) - --log-driver Set daemon log driver, such as: file - -l, --log-level Set log level, the levels can be: FATAL ALERT CRIT ERROR WARN NOTICE INFO DEBUG TRACE - --log-opt Set daemon log driver options, such as: log-path=/tmp/logs/ to set directory where to store daemon logs - --native.umask Default file mode creation mask (umask) for containers - --network-plugin Set network plugin, default is null, support null and cni - -p, --pidfile Save pid into this file - --pod-sandbox-image The image whose network/ipc namespaces containers in each pod will use. (default "pause-${machine}:3.0") - --registry-mirrors Registry to be prepended when pulling unqualified images, can be specified multiple times - --selinux-enabled Enable selinux support - --start-timeout timeout duration for waiting on a container to start before it is killed - -S, --state Root directory for execution state files - --storage-driver Storage driver to use(default overlay2) - -s, --storage-opt Storage driver options - --tls Use TLS; implied by --tlsverify - --tlscacert Trust certs signed only by this CA (default "/root/.iSulad/ca.pem") - --tlscert Path to TLS certificate file (default "/root/.iSulad/cert.pem") - --tlskey Path to TLS key file (default "/root/.iSulad/key.pem") - --tlsverify Use TLS and verify the remote - --use-decrypted-key Use decrypted private key by default(default true) - --userns-remap User/Group setting for user namespaces - -V, --version Print the version - --websocket-server-listening-port CRI websocket streaming service listening port (default 10350) - ``` - - Example: Start iSulad and change the log level to **DEBUG**. - - ```bash - # isulad -l DEBUG - ``` - -- **Configuration file** - - The iSulad configuration files are **/etc/isulad/daemon.json** and **/etc/isulad/daemon_constants.json**. The parameters in the files are described as follows. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Example

-

Description

-

Remarks

-

-e, --engine

-

"engine": "lcr"

-

iSulad runtime, which is Icr by default.

-

None

-

-G, --group

-

"group": "isulad"

-

Socket group.

-

None

-

--hook-spec

-

"hook-spec": "/etc/default/isulad/hooks/default.json"

-

Default hook configuration file for all containers.

-

None

-

-H, --host

-

"hosts": "unix:///var/run/isulad.sock"

-

Communication mode.

-

In addition to the local socket, the tcp://ip:port mode is supported. The port number ranges from 0 to 65535, excluding occupied ports.

-

--log-driver

-

"log-driver": "file"

-

Log driver configuration.

-

None

-

-l, --log-level

-

"log-level": "ERROR"

-

Log output level.

-

None

-

--log-opt

-

"log-opts": {

-

"log-file-mode": "0600",

-

"log-path": "/var/lib/isulad",

-

"max-file": "1",

-

"max-size": "30KB"

-

}

-

Log-related configuration.

-

You can specify max-file, max-size, and log-path. max-file indicates the number of log files. max-size indicates the threshold for triggering log anti-explosion. If max-file is 1, max-size is invalid. log-path specifies the path for storing log files. The log-file-mode command is used to set the permissions to read and write log files. The value must be in octal format, for example, 0666.

-

--container-log-driver

-

"container-log": {

-

"driver": "json-file"

-

}

-

Default driver for serial port logs of the container.

-

Specify the default driver for serial port logs of all containers.

-

--start-timeout

-

"start-timeout": "2m"

-

Time required for starting a container.

-

None

-

None

-

"default-runtime": "lcr"

-

Container runtime, which is lcr by default.

-

If neither the CLI nor the configuration file specifies the runtime, lcr is used by default. The priorities of the three specifying methods are as follows: CLI > configuration file > default value lcr. Currently, lcr and kata-runtime are supported.

-

None

- 
"runtimes":  {
-        "kata-runtime": {
-          "path": "/usr/bin/kata-runtime",
-          "runtime-args": [
-            "--kata-config",
-            "/usr/share/defaults/kata-containers/configuration.toml"
-          ]
-        }
-    }
-

When starting a container, set this parameter to specify multiple runtimes. Runtimes in this set are valid for container startup.

-

Runtime allowlist of a container. The customized runtimes in this set are valid. kata-runtime is used as the example.

-

-p, --pidfile

-

"pidfile": "/var/run/isulad.pid"

-

File for storing PIDs.

-

This parameter is required only when more than two container engines need to be started.

-

-g, --graph

-

"graph": "/var/lib/isulad"

-

Root directory for iSulad runtimes.

-

-S, --state

-

"state": "/var/run/isulad"

-

Root directory of the execution file.

-

--storage-driver

-

"storage-driver": "overlay2"

-

Image storage driver, which is overlay2 by default.

-

Only overlay2 is supported.

-

-s, --storage-opt

-

"storage-opts": [ "overlay2.override_kernel_check=true" ]

-

Image storage driver configuration options.

-

The options are as follows:

- 
overlay2.override_kernel_check=true #Ignore the kernel version check.
-    overlay2.size=${size} #Set the rootfs quota to ${size}.
-    overlay2.basesize=${size} #It is equivalent to overlay2.size.
-

--registry-mirrors

-

"registry-mirrors": [ "docker.io" ]

-

Registry address.

-

None

-

--insecure-registry

-

"insecure-registries": [ ]

-

Registry without TLS verification.

-

None

-

--native.umask

-

"native.umask": "secure"

-

Container umask policy. The default value is secure. The value normal indicates insecure configuration.

-

Set the container umask value.

-

The value can be null (0027 by default), normal, or secure.

- 
normal #The umask value of the started container is 0022.
-    secure #The umask value of the started container is 0027 (default value).
-

--pod-sandbox-image

-

"pod-sandbox-image": "rnd-dockerhub.huawei.com/library/pause-aarch64:3.0"

-

By default, the pod uses the image. The default value is rnd-dockerhub.huawei.com/library/pause-${machine}:3.0.

-

None

-

--network-plugin

-

"network-plugin": ""

-

Specifies a network plug-in. The value is a null character by default, indicating that no network configuration is available and the created sandbox has only the loop NIC.

-

The CNI and null characters are supported. Other invalid values will cause iSulad startup failure.

-

--cni-bin-dir

-

"cni-bin-dir": ""

-

Specifies the storage location of the binary file on which the CNI plug-in depends.

-

The default value is /opt/cni/bin.

-

--cni-conf-dir

-

"cni-conf-dir": ""

-

Specifies the storage location of the CNI network configuration file.

-

The default value is /etc/cni/net.d.

-

--image-layer-check=false

-

"image-layer-check": false

-

Image layer integrity check. To enable the function, set it to true; otherwise, set it to false. It is disabled by default.

-

When iSulad is started, the image layer integrity is checked. If the image layer is damaged, the related images are unavailable. iSulad cannot verify empty files, directories, and link files. Therefore, if the preceding files are lost due to a power failure, the integrity check of iSulad image data may fail to be identified. When the iSulad version changes, check whether the parameter is supported. If not, delete it from the configuration file.

-

--insecure-skip-verify-enforce=false

-

"insecure-skip-verify-enforce": false

-

Indicates whether to forcibly skip the verification of the certificate host name/domain name. The value is of the Boolean type, and the default value is false. If this parameter is set to true, the verification of the certificate host name/domain name is skipped.

-

The default value is false (not skipped). Note: Restricted by the YAJL JSON parsing library, if a non-Boolean value that meets the JSON format requirements is configured in the /etc/isulad/daemon.json configuration file, the default value used by iSulad is false.

-

--use-decrypted-key=true

-

"use-decrypted-key": true

-

Specifies whether to use an unencrypted private key. The value is of the Boolean type. If this parameter is set to true, an unencrypted private key is used. If this parameter is set to false, the encrypted private key is used, that is, two-way authentication is required.

-

The default value is true, indicating that an unencrypted private key is used. Note: Restricted by the YAJL JSON parsing library, if a non-Boolean value that meets the JSON format requirements is configured in the /etc/isulad/daemon.json configuration file, the default value used by iSulad is true.

-

--tls

-

"tls":false

-

Specifies whether to use TLS. The value is of the Boolean type.

-

This parameter is used only in -H tcp://IP:PORT mode. The default value is false.

-

--tlsverify

-

"tlsverify":false

-

Specifies whether to use TLS and verify remote access. The value is of the Boolean type.

-

This parameter is used only in -H tcp://IP:PORT mode.

-

--tlscacert

-

--tlscert

-

--tlskey

-

"tls-config": {

-

"CAFile": "/root/.iSulad/ca.pem",

-

"CertFile": "/root/.iSulad/server-cert.pem",

-

"KeyFile":"/root/.iSulad/server-key.pem"

-

}

-

TLS certificate-related configuration.

-

This parameter is used only in -H tcp://IP:PORT mode.

-

--authorization-plugin

-

"authorization-plugin": "authz-broker"

-

User permission authentication plugin.

-

Only authz-broker is supported.

-

--cgroup-parent

-

"cgroup-parent": "lxc/mycgroup"

-

Default cgroup parent path of a container, which is of the string type.

-

Specifies the cgroup parent path of a container. If --cgroup-parent is specified on the client, the client parameter prevails.

-

Note: If container A is started before container B, the cgroup parent path of container B is specified as the cgroup path of container A. When deleting a container, you need to delete container B and then container A in sequence. Otherwise, residual cgroup resources exist.

-

--default-ulimits

-

"default-ulimits": {

-

"nofile": {

-

"Name": "nofile",

-

"Hard": 6400,

-

"Soft": 3200

-

}

-

}

-

Specifies the ulimit restriction type, soft value, and hard value.

-

Specifies the restricted resource type, for example, nofile. The two field names must be the same, that is, nofile. Otherwise, an error is reported. The value of Hard must be greater than or equal to that of Soft. If the Hard or Soft field is not set, the default value 0 is used.

-

--websocket-server-listening-port

-

"websocket-server-listening-port": 10350

-

Specifies the listening port of the CRI WebSocket streaming service. The default port number is 10350.

-

Specifies the listening port of the CRI websocket streaming service.

-

If the client specifies --websocket-server-listening-port, the specified value is used. The port number ranges from 1024 to 49151.

-

None

-

"cri-runtimes": {

-

"kata": "io.containerd.kata.v2"

-

}

-

Specifies the mapping of custom CRI runtimes.

-

iSulad can convert RuntimeClass to the corresponding runtime through the custom CRI runtime mapping.

-
- - Configuration file **/etc/isulad/daemon_constants.json** - - - - - - - - - - - - - - - - - - - -

Parameter

-

Configuration Example

-

Description

-

Remarks

-

Not supported

-

"default-host": "docker.io"

-

If an image name is prefixed with the image repository name, the image repository name will be removed when the image name is saved and displayed.

-

Generally, this parameter does not need to be modified.

-

Not supported

-

"registry-transformation": {

-

"docker.io": "registry-1.docker.io",

-

"index.docker.io": "registry-1.docker.io"

-

}

-

"key":"value" pair. The image is pulled from the repository specified by "key":"value".

-

Generally, this parameter does not need to be modified.

-
- - Example: - - ```bash - # cat /etc/isulad/daemon.json - { - "group": "isulad", - "default-runtime": "lcr", - "graph": "/var/lib/isulad", - "state": "/var/run/isulad", - "engine": "lcr", - "log-level": "ERROR", - "pidfile": "/var/run/isulad.pid", - "log-opts": { - "log-file-mode": "0600", - "log-path": "/var/lib/isulad", - "max-file": "1", - "max-size": "30KB" - }, - "log-driver": "stdout", - "hook-spec": "/etc/default/isulad/hooks/default.json", - "start-timeout": "2m", - "storage-driver": "overlay2", - "storage-opts": [ - "overlay2.override_kernel_check=true" - ], - "registry-mirrors": [ - "docker.io" - ], - "insecure-registries": [ - "rnd-dockerhub.huawei.com" - ], - "pod-sandbox-image": "", - "native.umask": "secure", - "network-plugin": "", - "cni-bin-dir": "", - "cni-conf-dir": "", - "image-layer-check": false, - "use-decrypted-key": true, - "insecure-skip-verify-enforce": false, - "cri-runtime": { - "kata": "io.containerd.kata.v2" - } - } - - # cat /etc/isulad/daemon.json - { - "default-host": "docker.io", - "registry-transformation":{ - "docker.io": "registry-1.docker.io", - "index.docker.io": "registry-1.docker.io" - } - } - - ``` - - >![](./public_sys-resources/icon-notice.gif) **NOTICE:** - >The default configuration file **/etc/isulad/daemon.json** is for reference only. Configure it based on site requirements. - -### Storage Description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

File

-

Directory

-

Description

-

\*

-

/etc/default/isulad/

-

Stores the OCI configuration file and hook template file of iSulad. The file configuration permission is set to 0640, and the sysmonitor check permission is set to 0550.

-

\*

-

/etc/isulad/

-

Default configuration files of iSulad and seccomp.

-

isulad.sock

-

/var/run/

-

Pipe communication file, which is used for the communication between the client and iSulad.

-

isulad.pid

-

/var/run/

-

File for storing the iSulad PIDs. It is also a file lock to prevent multiple iSulad instances from being started.

-

\*

-

/run/lxc/

-

Lock file, which is created during iSulad running.

-

\*

-

/var/run/isulad/

-

Real-time communication cache file, which is created during iSulad running.

-

\*

-

/var/run/isula/

-

Real-time communication cache file, which is created during iSulad running.

-

\*

-

/var/lib/isulad/

-

Root directory where iSulad runs, which stores the created container configuration, default log path, database file, and mount point.

-

/var/lib/isulad/mnt/: mount point of the container rootfs.

-

/var/lib/isulad/engines/lcr/: directory for storing LCR container configurations. Each container has a directory named after the container.

-
- -### Constraints - -- In high concurrency scenarios \(200 containers are concurrently started\), the memory management mechanism of Glibc may cause memory holes and large virtual memory \(for example, 10 GB\). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set **MALLOC\_ARENA\_MAX** to reducevirtual memory error and increase the rate of reducing physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements. - - ```bash - To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance on the ARM64 server is affected by less than 10%.) - - Configuration method: - 1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start iSulad. - 2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4. - ``` - -- Precautions for specifying the daemon running directories - - Take **--root** as an example. When **/new/path/** is used as the daemon new root directory, if a file exists in **/new/path/** and the directory or file name conflicts with that required by iSulad \(for example, **engines** and **mnt**\), iSulad may update the original directory or file attributes including the owner and permission. - - Therefore, please note the impact of re-specifying various running directories and files on their attributes. You are advised to specify a new directory or file for iSulad to avoid file attribute changes and security issues caused by conflicts. - -- Log file management: - - >![](./public_sys-resources/icon-notice.gif) **NOTICE:** - >Log function interconnection: logs are managed by systemd as iSulad is and then transmitted to rsyslogd. By default, rsyslog restricts the log writing speed. You can add the configuration item **$imjournalRatelimitInterval 0** to the **/etc/rsyslog.conf** file and restart the rsyslogd service. - -- Restrictions on command line parameter parsing - - When the iSulad command line interface is used, the parameter parsing mode is slightly different from that of Docker. For flags with parameters in the command line, regardless of whether a long or short flag is used, only the first space after the flag or the character string after the equal sign \(=\) directly connected to the flag is used as the flag parameter. The details are as follows: - - 1. When a short flag is used, each character in the character string connected to the hyphen \(-\) is considered as a short flag. If there is an equal sign \(=\), the character string following the equal sign \(=\) is considered as the parameter of the short flag before the equal sign \(=\). - - **isula run -du=root busybox** is equivalent to **isula run -du root busybox**, **isula run -d -u=root busybox**, or **isula run -d -u root busybox**. When **isula run -du:root** is used, as **-:** is not a valid short flag, an error is reported. The preceding command is equivalent to **isula run -ud root busybox**. However, this method is not recommended because it may cause semantic problems. - - 2. When a long flag is used, the character string connected to **--** is regarded as a long flag. If the character string contains an equal sign \(=\), the character string before the equal sign \(=\) is a long flag, and the character string after the equal sign \(=\) is a parameter. - - ```bash - isula run --user=root busybox - ``` - - or - - ```bash - isula run --user root busybox - ``` - -- After an iSulad container is started, you cannot run the **isula run -i/-t/-ti** and **isula attach/exec** commands as a non-root user. -- The default path for storing temporary files of iSulad is **/var/lib/isulad/isulad_tmpdir**. If the root directory of iSulad is changed, the path is **\$isulad_root/isulad_tmpdir**. To change the directory for storing temporary files of iSulad, you can configure the **ISULAD_TMPDIR** environment variable before starting iSulad. The **ISULAD_TMPDIR** environment variable is checked during the iSulad startup. If the **ISULAD_TMPDIR** environment variable is configured, the **\$ISULAD_TMPDIR/isulad_tmpdir** directory is used as the path for storing temporary files. Do not store files or folders named **isulad_tmpdir** in **\$ISULAD_TMPDIR** because iSulad recursively deletes the **\$ISULAD_TMPDIR/isulad_tmpdir** directory when it is started to prevent residual data. In addition, ensure that only the **root** user can access the **\$ISULAD_TMPDIR** directory to prevent security problems caused by operations of other users. - -### Daemon Multi-Port Binding - -#### Description - -The daemon can bind multiple UNIX sockets or TCP ports and listen on these ports. The client can interact with the daemon through these ports. - -#### Port - -Users can configure one or more ports in the hosts field in the **/etc/isulad/daemon.json** file, or choose not to specify hosts. - -```json -{ - "hosts": [ - "unix:///var/run/isulad.sock", - "tcp://localhost:5678", - "tcp://127.0.0.1:6789" - ] -} -``` - -Users can also run the **-H** or **--host** command in the **/etc/sysconfig/iSulad** file to configure a port, or choose not to specify hosts. - -```text -OPTIONS='-H unix:///var/run/isulad.sock --host tcp://127.0.0.1:6789' -``` - -If hosts are not specified in the **daemon.json** file and iSulad, the daemon listens on **unix:///var/run/isulad.sock** by default after startup. - -#### Restrictions - -- Users cannot specify hosts in the **/etc/isulad/daemon.json** and **/etc/sysconfig/iSuald** files at the same time. Otherwise, an error will occur and iSulad cannot be started. - - ```bash - unable to configure the isulad with file /etc/isulad/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [unix:///var/run/isulad.sock tcp://127.0.0.1:6789], from file: [unix:///var/run/isulad.sock tcp://localhost:5678 tcp://127.0.0.1:6789]) - ``` - -- If the specified host is a UNIX socket, the socket must start with **unix://** followed by a valid absolute path. -- If the specified host is a TCP port, the TCP port number must start with **tcp://** followed by a valid IP address and port number. The IP address can be that of the local host. -- A maximum of 10 valid ports can be specified. If more than 10 ports are specified, an error will occur and iSulad cannot be started. - -### Configuring TLS Authentication and Enabling Remote Access - -#### Description - -iSulad is designed in C/S mode. By default, the iSulad daemon process listens only on the local/var/run/isulad.sock. Therefore, you can run commands to operate containers only on the local client iSula. To enable iSula's remote access to the container, the iSulad daemon process needs to listen on the remote access port using TCP/IP. However, listening is performed only by simply configuring tcp ip:port. In this case, all IP addresses can communicate with iSulad by calling **isula -H tcp://**_remote server IP address_**:port**, which may cause security problems. Therefore, it is recommended that a more secure version, namely Transport Layer Security \(TLS\), be used for remote access. - -#### Generating TLS Certificate - -- Example of generating a plaintext private key and certificate - - ```bash - #!/bin/bash - set -e - echo -n "Enter pass phrase:" - read password - echo -n "Enter public network ip:" - read publicip - echo -n "Enter host:" - read HOST - - echo " => Using hostname: $publicip, You MUST connect to iSulad using this host!" - - mkdir -p $HOME/.iSulad - cd $HOME/.iSulad - rm -rf $HOME/.iSulad/* - - echo " => Generating CA key" - openssl genrsa -passout pass:$password -aes256 -out ca-key.pem 4096 - echo " => Generating CA certificate" - openssl req -passin pass:$password -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com" - echo " => Generating server key" - openssl genrsa -passout pass:$password -out server-key.pem 4096 - echo " => Generating server CSR" - openssl req -passin pass:$password -subj /CN=$HOST -sha256 -new -key server-key.pem -out server.csr - echo subjectAltName = DNS:$HOST,IP:$publicip,IP:127.0.0.1 >> extfile.cnf - echo extendedKeyUsage = serverAuth >> extfile.cnf - echo " => Signing server CSR with CA" - openssl x509 -req -passin pass:$password -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf - echo " => Generating client key" - openssl genrsa -passout pass:$password -out key.pem 4096 - echo " => Generating client CSR" - openssl req -passin pass:$password -subj '/CN=client' -new -key key.pem -out client.csr - echo " => Creating extended key usage" - echo extendedKeyUsage = clientAuth > extfile-client.cnf - echo " => Signing client CSR with CA" - openssl x509 -req -passin pass:$password -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile extfile-client.cnf - rm -v client.csr server.csr extfile.cnf extfile-client.cnf - chmod -v 0400 ca-key.pem key.pem server-key.pem - chmod -v 0444 ca.pem server-cert.pem cert.pem - ``` - -- Example of generating an encrypted private key and certificate request file - - ```bash - #!/bin/bash - - echo -n "Enter public network ip:" - read publicip - echo -n "Enter pass phrase:" - read password - - # remove certificates from previous execution. - rm -f *.pem *.srl *.csr *.cnf - - - # generate CA private and public keys - echo 01 > ca.srl - openssl genrsa -aes256 -out ca-key.pem -passout pass:$password 2048 - openssl req -subj '/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com' -new -x509 -days $DAYS -passin pass:$password -key ca-key.pem -out ca.pem - - # create a server key and certificate signing request (CSR) - openssl genrsa -aes256 -out server-key.pem -passout pass:$PASS 2048 - openssl req -new -key server-key.pem -out server.csr -passin pass:$password -subj '/CN=iSulad' - - echo subjectAltName = DNS:iSulad,IP:${publicip},IP:127.0.0.1 > extfile.cnf - echo extendedKeyUsage = serverAuth >> extfile.cnf - # sign the server key with our CA - openssl x509 -req -days $DAYS -passin pass:$password -in server.csr -CA ca.pem -CAkey ca-key.pem -out server-cert.pem -extfile extfile.cnf - - # create a client key and certificate signing request (CSR) - openssl genrsa -aes256 -out key.pem -passout pass:$password 2048 - openssl req -subj '/CN=client' -new -key key.pem -out client.csr -passin pass:$password - - # create an extensions config file and sign - echo extendedKeyUsage = clientAuth > extfile.cnf - openssl x509 -req -days 365 -passin pass:$password -in client.csr -CA ca.pem -CAkey ca-key.pem -out cert.pem -extfile extfile.cnf - - # remove the passphrase from the client and server key - openssl rsa -in server-key.pem -out server-key.pem -passin pass:$password - openssl rsa -in key.pem -out key.pem -passin pass:$password - - # remove generated files that are no longer required - rm -f ca-key.pem ca.srl client.csr extfile.cnf server.csr - ``` - -#### APIs - -```json -{ - "tls": true, - "tls-verify": true, - "tls-config": { - "CAFile": "/root/.iSulad/ca.pem", - "CertFile": "/root/.iSulad/server-cert.pem", - "KeyFile":"/root/.iSulad/server-key.pem" - } -} -``` - -#### Restrictions - -The server supports the following modes: - -- Mode 1 \(client verified\): tlsverify, tlscacert, tlscert, tlskey -- Mode 2 \(client not verified\): tls, tlscert, tlskey - -The client supports the following modes: - -- Mode 1 \(verify the identity based on the client certificate, and verify the server based on the specified CA\): tlsverify, tlscacert, tlscert, tlskey -- Mode 2 \(server verified\): tlsverify, tlscacert - -Mode 1 is used for the server, and mode 2 for the client if the two-way authentication mode is used for communication. - -Mode 2 is used for the server and the client if the unidirectional authentication mode is used for communication. - ->![](./public_sys-resources/icon-notice.gif) **NOTICE:** -> ->- If RPM is used for installation, the server configuration can be modified in the **/etc/isulad/daemon.json** and **/etc/sysconfig/iSulad** files. ->- Two-way authentication is recommended as it is more secure than non-authentication or unidirectional authentication. ->- GRPC open-source component logs are not taken over by iSulad. To view gRPC logs, set the environment variables **gRPC\_VERBOSITY** and **gRPC\_TRACE** as required. -> - -#### Example - -On the server: - -```bash - isulad -H=tcp://0.0.0.0:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/server-cert.pem --tlskey ~/.iSulad/server-key.pem -``` - -On the client: - -```bash - isula version -H=tcp://$HOSTIP:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/cert.pem --tlskey ~/.iSulad/key.pem -``` - -### devicemapper Storage Driver Configuration - -To use the devicemapper storage driver, you need to configure a thinpool device which requires an independent block device with sufficient free space. Take the independent block device **/dev/xvdf** as an example. The configuration method is as follows: - -1. Configuring a thinpool - - 1. Stop the iSulad service. - - ```bash - # systemctl stop isulad - ``` - - 2. Create a logical volume manager \(LVM\) volume based on the block device. - - ```bash - # pvcreate /dev/xvdf - ``` - - 3. Create a volume group based on the created physical volume. - - ```bash - # vgcreate isula /dev/xvdf - Volume group "isula" successfully created: - ``` - - 4. Create two logical volumes named **thinpool** and **thinpoolmeta**. - - ```bash - # lvcreate --wipesignatures y -n thinpool isula -l 95%VG - Logical volume "thinpool" created. - ``` - - ```bash - # lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG - Logical volume "thinpoolmeta" created. - ``` - - 5. Convert the two logical volumes into a thinpool and the metadata used by the thinpool. - - ```bash - # lvconvert -y --zero n -c 512K --thinpool isula/thinpool --poolmetadata isula/thinpoolmeta - - WARNING: Converting logical volume isula/thinpool and isula/thinpoolmeta to - thin pool's data and metadata volumes with metadata wiping. - THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) - Converted isula/thinpool to thin pool. - ``` - -2. Modifying the iSulad configuration files - - 1. If iSulad has been used in the environment, back up the running data first. - - ```bash - # mkdir /var/lib/isulad.bk - # mv /var/lib/isulad/* /var/lib/isulad.bk - ``` - - 2. Modify configuration files. - - Two configuration methods are provided. Select one based on site requirements. - - - Edit the **/etc/isulad/daemon.json** file, set **storage-driver** to **devicemapper**, and set parameters related to the **storage-opts** field. For details about related parameters, see [Parameter Description](#parameter-description). The following lists the configuration reference: - - ```json - { - "storage-driver": "devicemapper" - "storage-opts": [ - "dm.thinpooldev=/dev/mapper/isula-thinpool", - "dm.fs=ext4", - "dm.min_free_space=10%" - ] - } - ``` - - - You can also edit **/etc/sysconfig/iSulad** to explicitly specify related iSulad startup parameters. For details about related parameters, see [Parameter Description](#parameter-description). The following lists the configuration reference: - - ```text - OPTIONS="--storage-driver=devicemapper --storage-opt dm.thinpooldev=/dev/mapper/isula-thinpool --storage-opt dm.fs=ext4 --storage-opt dm.min_free_space=10%" - ``` - -3. Start iSulad for the settings to take effect. - - ```bash - # systemctl start isulad - ``` - -#### Parameter Description - -For details about parameters supported by storage-opts, see [Table 1](#en-us_topic_0222861454_table3191161993812). - -**Table 1** Parameter description - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Mandatory or Not

-

Description

-

dm.fs

-

Yes

-

Specifies the type of the file system used by a container. This parameter must be set to ext4, that is, dm.fs=ext4.

-

dm.basesize

-

No

-

Specifies the maximum storage space of a single container. The unit can be k, m, g, t, or p. An uppercase letter can also be used, for example, dm.basesize=50G. This parameter is valid only during the first initialization.

-

dm.mkfsarg

-

No

-

Specifies the additional mkfs parameters when a basic device is created. For example: dm.mkfsarg=-O ^has_journal

-

dm.mountopt

-

No

-

Specifies additional mount parameters when a container is mounted. For example: dm.mountopt=nodiscard

-

dm.thinpooldev

-

No

-

Specifies the thinpool device used for container or image storage.

-

dm.min_free_space

-

No

-

Specifies minimum percentage of reserved space. For example, dm.min_free_space=10% indicates that storage-related operations such as container creation will fail when the remaining storage space falls below 10%.

-
- -#### Precautions - -- When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function. - - To disable automatic capacity expansion, set both **thin\_pool\_autoextend\_threshold** and **thin\_pool\_autoextend\_percent** in the **/etc/lvm/profile/isula-thinpool.profile** file to **100**. - - ```text - activation { - thin_pool_autoextend_threshold=100 - thin_pool_autoextend_percent=100 - } - ``` - -- When devicemapper is used, use Ext4 as the container file system. You need to add **--storage-opt dm.fs=ext4** to the iSulad configuration parameters. -- If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon. -- When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool. - -##### Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on iSula - -- Generally, the path of the deviceset-metadata file is **/var/lib/isulad/devicemapper/metadata/deviceset-metadata** during container startup. -- If user namespaces are used, the path of the deviceset-metadata file is **/var/lib/isulad/**_userNSUID.GID_**/devicemapper/metadata/deviceset-metadata**. -- When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the **BaseDeviceUUID** content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the **BaseDeviceUUID** content in the deviceset-metadata file. Otherwise, the iSulad service fails to be restarted. diff --git a/docs/en/docs/Container/installation-guideline.md b/docs/en/docs/Container/installation-guideline.md deleted file mode 100644 index 738f8861408c2709f8d3250cd2228a9cc584e4f3..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/installation-guideline.md +++ /dev/null @@ -1,28 +0,0 @@ -# Installation Guideline - -1. Install the container engine iSulad. - - ``` - # yum install iSulad - ``` - -2. Install dependent packages of system containers. - - ``` - # yum install isulad-tools authz isulad-lxcfs-toolkit lxcfs - ``` - -3. Run the following command to check whether iSulad is started: - - ``` - # systemctl status isulad - ``` - -4. Enable the lxcfs and authz services. - - ``` - # systemctl start lxcfs - # systemctl start authz - ``` - - diff --git a/docs/en/docs/Container/installation-upgrade-Uninstallation.md b/docs/en/docs/Container/installation-upgrade-Uninstallation.md deleted file mode 100644 index b217abab2632fcf0219fdd7879bf1132230ef335..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/installation-upgrade-Uninstallation.md +++ /dev/null @@ -1,4 +0,0 @@ -# Installation, Upgrade and Uninstallation - -This chapter describes how to install, configure, upgrade, and uninstall iSulad. - diff --git a/docs/en/docs/Container/interconnecting-isula-shim-v2-with-stratovirt.md b/docs/en/docs/Container/interconnecting-isula-shim-v2-with-stratovirt.md deleted file mode 100644 index 649c5ccadabe6183ce2fa013d1640dfb61991c8e..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/interconnecting-isula-shim-v2-with-stratovirt.md +++ /dev/null @@ -1,218 +0,0 @@ -# Interconnecting iSula with the shim v2 Secure Container - -## Overview - -shim v2 is a next-generation shim solution. Compared with shim v1, shim v2 features shorter call chains, clearer architecture, and lower memory overhead in multi-service container scenarios. iSula can run secure containers through isulad-shim or containerd-shim-kata-v2. The isulad-shim component is the implementation of the shim v1 solution, and the containerd-shim-kata-v2 component is the implementation of the shim v2 solution in the secure container scenario. This document describes how to interconnect iSula with containerd-shim-kata-v2. - -## Interconnecting with containerd-shim-kata-v2 - -### Prerequisites - -Before interconnecting iSula with containerd-shim-kata-v2, ensure that the following prerequisites are met: - -- iSulad and kata-containers have been installed. -- StratoVirt supports only the devicemapper storage driver. Therefore, you need to configure the devicemapper environment and ensure that the devicemapper storage driver used by iSulad works properly. - -### Environment Setup - -The following describes how to install and configure iSulad and kata-containers. - -#### Installing Dependencies - -Configure the Yum source based on the OS version and install iSulad and kata-containers as the **root** user. - -```shell -yum install iSulad -yum install kata-containers -``` - -#### Creating and Configuring a Storage Device - -Prepare a drive, for example, **/dev/sdx**. The drive will be formatted. This section uses the block device **/dev/sda** as an example. - -I. Creating devicemapper - -1. Create a physical volume (PV). - - ```shell - $ pvcreate /dev/sda - Physical volume "/dev/loop0" successfully created. - ``` - -2. Create a volume group (VG). - - ```shell - $ vgcreate isula /dev/sda - Volume group "isula" successfully created - ``` - -3. Create the logical volumes **thinpool** and **thinpoolmeta**. - - ```shell - $ lvcreate --wipesignatures y -n thinpool isula -l 95%VG - Logical volume "thinpool" created. - - $ lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG - Logical volume "thinpoolmeta" created. - ``` - -4. Convert the created logical volumes to a thin pool. - - ```shell - $ lvconvert -y --zero n -c 64K \ - --thinpool isula/thinpool \ - --poolmetadata isula/thinpoolmeta - Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data. - WARNING: Converting isula/thinpool and isula/thinpoolmeta to thin pool's data and metadata volumes with metadata wiping. - THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) - Converted isula/thinpool and isula/thinpoolmeta to thin pool. - ``` - -5. Configure automatic extension of the thin pool using lvm. - - ```shell - $ touch /etc/lvm/profile/isula-thinpool.profile - $ cat << EOF > /etc/lvm/profile/isula-thinpool.profile - activation { - thin_pool_autoextend_threshold=80 - thin_pool_autoextend_percent=20 - } - EOF - $ lvchange --metadataprofile isula-thinpool isula/thinpool - Logical volume isula/thinpool changed. - ``` - -II. Changing the iSulad Storage Driver Type and Setting the Default Runtime - -Modify the **/etc/isulad/daemon.json** configuration file. Set **default-runtime** to **io.containerd.kata.v2** and **storage-driver** to **devicemapper**. The modification result is as follows: - -```json - { - "default-runtime": "io.containerd.kata.v2", - "storage-driver": "devicemapper", - "storage-opts": [ - "dm.thinpooldev=/dev/mapper/isula-thinpool", - "dm.fs=ext4", - "dm.min_free_space=10%" - ], -} -``` - -III. Making the Configuration Take Effect - -1. Restart the iSulad for the configuration to take effect. - - ```shell - # systemctl daemon-reload - # systemctl restart isulad - ``` - -2. Check whether the iSula storage driver is successfully configured. - - ```shell - # isula info - ``` - - If the following information is displayed, the configuration is successful: - - ```text - Storage Driver: devicemapper - ``` - -### Interconnection Guide - -This section describes how to interconnect iSula with containerd-shim-kata-v2. - -By default, containerd-shim-kata-v2 uses QEMU as the virtualization component. The following describes how to configure QEMU and StratoVirt. - -#### Using QEMU - -If containerd-shim-kata-v2 uses QEMU as the virtualization component, perform the following operations to interconnect iSula with containerd-shim-kata-v2: - -1. Modify the kata configuration file **/usr/share/defaults/kata-containers/configuration.toml**. - - Set **sandbox_cgroup_with_emulator** to **false**. Currently, shim v2 does not support this function. Other parameters are the same as the kata configuration parameters in shim v1 or use the default values. - - ```text - sandbox_cgroup_with_emulator = false - ``` - -2. Use the BusyBox image to run the secure container and check whether the used runtime is io.containerd.kata.v2. - - ```bash - $ id=`isula run -tid busybox /bin/sh` - $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id - "io.containerd.kata.v2" - ``` - -3. Verify that the QEMU-based VM process is started. If it is started, QEMU is successfully interconnected with the shim v2 secure container. - - ```bash - ps -ef | grep qemu - ``` - -#### Using StratoVirt - -If containerd-shim-kata-v2 uses StratoVirt as the virtualization component, perform the following operations to interconnect iSula with containerd-shim-kata-v2: - -1. Create the **stratovirt.sh** script in any directory (for example, **/home**) and add the execute permission to the file as the **root** user. - - ```shell - # touch /home/stratovirt.sh - # chmod +x /home/stratovirt.sh - ``` - - The content of **stratovirt.sh** is as follows, which is used to specify the path of StratoVirt: - - ```shell - #!/bin/bash - export STRATOVIRT_LOG_LEVEL=info # set log level which includes trace, debug, info, warn and error. - /usr/bin/stratovirt $@ - ``` - -2. Modify the kata configuration file. Set **hypervisor** of the secure container to **stratovirt**, **kernel** to the absolute path of the StratoVirt kernel image, and **initrd** to the initrd image file of kata-containers (if you use YUM to install kata-containers, the initrd image file is downloaded by default and stored in the **/var/lib/kata/** directory). StratoVirt supports only the devicemapper storage mode, prepare the environment in advance and set iSulad to the devicemapper mode. - - The configurations are as follows: - - ```shell - [hypervisor.stratovirt] - path = "/home/stratovirt.sh" - kernel = "/var/lib/kata/vmlinux.bin" - initrd = "/var/lib/kata/kata-containers-initrd.img" - block_device_driver = "virtio-mmio" - use_vsock = true - enable_netmon = true - internetworking_model="tcfilter" - sandbox_cgroup_with_emulator = false - disable_new_netns = false - disable_block_device_use = false - disable_vhost_net = true - ``` - - To use the vsock function in StratoVirt, enable the vhost_vsock kernel module and check whether the module is successfully enabled. - - ```bash - modprobe vhost_vsock - lsmod |grep vhost_vsock - ``` - - Download the kernel of the required version and architecture and save it to the **/var/lib/kata/** directory. For example, download the [openeuler repo]() of the x86 architecture of openEuler 21.03. - - ```bash - cd /var/lib/kata - wget https://repo.openeuler.org/openEuler-21.03/stratovirt_img/x86_64/vmlinux.bin - ``` - -3. Use the BusyBox image to run the secure container and check whether the used runtime is io.containerd.kata.v2. - - ```bash - $ id=`isula run -tid busybox sh` - $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id - "io.containerd.kata.v2" - ``` - -4. Verify that the StratoVirt-based VM process is started. If it is started, StratoVirt is successfully interconnected with the shim v2 secure container. - - ```bash - ps -ef | grep stratovirt - ``` diff --git a/docs/en/docs/Container/interconnection-with-the-cni-network.md b/docs/en/docs/Container/interconnection-with-the-cni-network.md deleted file mode 100644 index 367ea7497103e6dfc5cf219670dd6e207619247d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/interconnection-with-the-cni-network.md +++ /dev/null @@ -1,123 +0,0 @@ -# Interconnection with the CNI Network - -- [Interconnection with the CNI Network](#interconnection-with-the-cni-network) - - [Overview](#overview) - - [Common CNIs](#common-cnis) - - [CNI Network Configuration Description](#cni-network-configuration-description) - - [Adding a Pod to the CNI Network List](#adding-a-pod-to-the-cni-network-list) - - [Removing a Pod from the CNI Network List](#removing-a-pod-from-the-cni-network-list) - - [Usage Restrictions](#usage-restrictions) - -## Overview - -The container runtime interface \(CRI\) is provided to connect to the CNI network, including parsing the CNI network configuration file and adding or removing a pod to or from the CNI network. When a pod needs to support a network through a container network plug-in such as Canal, the CRI needs to be interconnected to Canal so as to provide the network capability for the pod. - -## Common CNIs - -Common CNIs include CNI network configuration items in the CNI network configuration and pod configuration. These CNIs are visible to users. - -- CNI network configuration items in the CNI network configuration refer to those used to specify the path of the CNI network configuration file, path of the binary file of the CNI network plug-in, and network mode. For details, see [Table 1](#en-us_topic_0183259146_table18221919589). -- CNI network configuration items in the pod configuration refer to those used to set the additional CNI network list to which the pod is added. By default, the pod is added only to the default CNI network plane. You can add the pod to multiple CNI network planes as required. - -**Table 1** CNI network configuration items - - - - - - - - - - - - - - - - - - - - - - - - -

Function

-

Command

-

Configuration File

-

Description

-

Path of the binary file of the CNI network plug-in

-

--cni-bin-dir

-

"cni-bin-dir": "",

-

The default value is /opt/cni/bin.

-

Path of the CNI network configuration file

-

--cni-conf-dir

-

"cni-conf-dir": "",

-

The system traverses all files with the extension .conf, .conflist, or .json in the directory. The default value is /etc/cni/net.d.

-

Network mode

-

--network-plugin

-

"network-plugin": "",

-

Specifies a network plug-in. The value is a null character by default, indicating that no network configuration is available and the created sandbox has only the loop NIC. The CNI and null characters are supported. Other invalid values will cause iSulad startup failure.

-
- -Additional CNI network configuration mode: - -Add the network plane configuration item "network.alpha.kubernetes.io/network" to annotations in the pod configuration file. - -The network plane is configured in JSON format, including: - -- **name**: specifies the name of the CNI network plane. -- **interface**: specifies the name of a network interface. - -The following is an example of the CNI network configuration method: - -```json -"annotations" : { - "network.alpha.kubernetes.io/network": "{\"name\": \"mynet\", \"interface\": \"eth1\"}" - } -``` - -### CNI Network Configuration Description - -The CNI network configuration includes two types, both of which are in the .json file format. - -- Single-network plane configuration file with the file name extension .conf or .json. For details about the configuration items, see Table 1 in the appendix. -- Multi-network plane configuration file with the file name extension .conflist. For details about the configuration items, see Table 3 in the appendix. - -### Adding a Pod to the CNI Network List - -If **--network-plugin=cni** is configured for iSulad and the default network plane is configured, a pod is automatically added to the default network plane when the pod is started. If the additional network configuration is configured in the pod configuration, the pod is added to these additional network planes when the pod is started. - -**port\_mappings** in the pod configuration is also a network configuration item, which is used to set the port mapping of the pod. To set port mapping, perform the following steps: - -```json -"port_mappings":[ - { - "protocol": 1, - "container_port": 80, - "host_port": 8080 - } -] -``` - -- **protocol**: protocol used for mapping. The value can be **tcp** \(identified by 0\) or **udp** \(identified by 1\). -- **container\_port**: port through which the container is mapped. -- **host\_port**: port mapped to the host. - -### Removing a Pod from the CNI Network List - -When StopPodSandbox is called, the interface for removing a pod from the CNI network list will be called to clear network resources. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->1. Before calling the RemovePodSandbox interface, you must call the StopPodSandbox interface at least once. ->2. If StopPodSandbox fails to call the CNI, residual network resources will be cleaned by the CNI network plugin. - -## Usage Restrictions - -- Currently, only CNI 0.3.0 and CNI 0.3.1 are supported. In later versions, CNI 0.1.0 and CNI 0.2.0 may need to be supported. Therefore, when error logs are displayed, the information about CNI 0.1.0 and CNI 0.2.0 is reserved. -- name: The value must contain lowercase letters, digits, hyphens \(-\), and periods \(.\) and cannot be started or ended with a hyphen or period. The value can contain a maximum of 200 characters. -- The number of configuration files cannot exceed 200, and the size of a single configuration file cannot exceed 1 MB. -- The extended parameters need to be configured based on the actual network requirements. Optional parameters do not need to be written into the netconf.json file. diff --git a/docs/en/docs/Container/isula-build.md b/docs/en/docs/Container/isula-build.md deleted file mode 100644 index a26ac516a723e384b765671803b5aafcb0a6be6a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/isula-build.md +++ /dev/null @@ -1,13 +0,0 @@ -# Container Image Building - -## Overview - -isula-build is a container image build tool developed by the iSula container team. It allows you to quickly build container images using Dockerfiles. - -The isula-build uses the server/client mode. The isula-build functions as a client and provides a group of command line tools for image build and management. The isula-builder functions as the server to process client management requests, and runs as a daemon process in the background. - -![isula-build architecture](./figures/isula-build_arch.png) - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> Currently, isula-build supports OCI image format ([OCI Image Format Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md/)) and Docker image format ([Image Manifest Version 2, Schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/)). Use the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command to enable the experimental feature for supporting OCI image format. When the experimental feature is disabled, isula-build will take Docker image format as the default image format. Otherwise, isula-build will take OCI image format as the default image format. diff --git a/docs/en/docs/Container/isula-build_user_guide.md b/docs/en/docs/Container/isula-build_user_guide.md deleted file mode 100644 index 078424ace9c2969abe3306ab7799af6398bd45a8..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/isula-build_user_guide.md +++ /dev/null @@ -1,1030 +0,0 @@ -# Installation - -## Preparations - -To ensure that isula-build can be successfully installed, the following software and hardware requirements must be met: - -- Supported architectures: x86_64 and AArch64 -- Supported OS: openEuler -- You have the permissions of the root user. - -### Installing isula-build - -Before using isula-build to build a container image, you need to install the following software packages: - -#### (Recommended) Method 1: Using Yum - -1. Configure the openEuler Yum source. - -2. Log in to the target server as the root user and install isula-build. - - ```shell - sudo yum install -y isula-build - ``` - -#### Method 2: Using the RPM Package - -1. Obtain an **isula-build-*.rpm** installation package from the openEuler Yum source, for example, **isula-build-0.9.6-4.oe1.x86_64.rpm**. - -2. Upload the obtained RPM software package to any directory on the target server, for example, **/home/**. - -3. Log in to the target server as the root user and run the following command to install isula-build: - - ```shell - sudo rpm -ivh /home/isula-build-*.rpm - ``` - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> After the installation is complete, you need to manually start the isula-build service. For details about how to start the service, see [Managing the isula-build Service](#managing-the-isula-build-service). - -# Configuring and Managing the isula-build Service - -## Configuring the isula-build Service - -After the isula-build software package is installed, the systemd starts the isula-build service based on the default configuration contained in the isula-build software package on the isula-build server. If the default configuration file on the isula-build server cannot meet your requirements, perform the following operations to customize the configuration file: After the default configuration is modified, restart the isula-build server for the new configuration to take effect. For details, see [Managing the isula-build Service](#managing-the-isula-build-service). - -Currently, the isula-build server contains the following configuration file: - -- **/etc/isula-build/configuration.toml**: general isula-builder configuration file, which is used to set the isula-builder log level, persistency directory, runtime directory, and OCI runtime. Parameters in the configuration file are described as follows: - -| Configuration Item | Mandatory or Optional | Description | Value | -| --------- | -------- | --------------------------------- | ----------------------------------------------- | -| debug | Optional | Indicates whether to enable the debug log function. | **true**: Enables the debug log function. **false**: Disables the debug log function. | -| loglevel | Optional | Sets the log level. | debug
info
warn
error | -| run_root | Mandatory | Sets the root directory of runtime data. | For example, **/var/run/isula-build/** | -| data_root | Mandatory | Sets the local persistency directory. | For example, **/var/lib/isula-build/** | -| runtime | Optional | Sets the runtime type. Currently, only **runc** is supported. | runc | -| group | Optional | Sets the owner group for the local socket file **isula_build.sock** so that non-privileged users in the group can use isula-build. | isula | -| experimental | Optional | Indicates whether to enable experimental features. | **true**: Enables experimental features. **false**: Disables experimental features. | - -- **/etc/isula-build/storage.toml**: configuration file for local persistent storage, including the configuration of the storage driver in use. - -| Configuration Item | Mandatory or Optional | Description | -| ------ | -------- | ------------------------------ | -| driver | Optional | Storage driver type. Currently, **overlay2** is supported. | - - For more settings, see [containers-storage.conf.5](https://github.com/containers/storage/blob/main/docs/containers-storage.conf.5.md). - -- **/etc/isula-build/registries.toml**: configuration file for each image repository. - -| Configuration Item | Mandatory or Optional | Description | -| ------------------- | -------- | ------------------------------------------------------------ | -| registries.search | Optional | Search domain of the image repository. Only listed image repositories can be found. | -| registries.insecure | Optional | Accessible insecure image repositories. Listed image repositories cannot pass the authentication and are not recommended. | - - For more settings, see [containers-registries.conf.5](https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md). - -- **/etc/isula-build/policy.json**: image pull/push policy file. Currently, this file cannot be configured. - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> - isula-build supports the preceding configuration file with the maximum size of 1 MB. -> - The persistent working directory dataroot cannot be configured on the memory disk, for example, tmpfs. -> - Currently, only overlay2 can be used as the underlying storage driver. -> - Before setting the `--group` option, ensure that the corresponding user group has been created on a local OS and non-privileged users have been added to the group. After isula-builder is restarted, non-privileged users in the group can use the isula-build function. In addition, to ensure permission consistency, the owner group of the isula-build configuration file directory **/etc/isula-build** is set to the group specified by `--group`. - -## Managing the isula-build Service - -Currently, openEuler uses systemd to manage the isula-build service. The isula-build software package contains the systemd service files. After installing the isula-build software package, you can use the systemd tool to start or stop the isula-build service. You can also manually start the isula-builder software. Note that only one isula-builder process can be started on a node at a time. - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> Only one isula-builder process can be started on a node at a time. - -### (Recommended) Using systemd for Management - -You can run the following systemd commands to start, stop, and restart the isula-build service: - -- Run the following command to start the isula-build service: - - ```sh - sudo systemctl start isula-build.service - ``` - -- Run the following command to stop the isula-build service: - - ```sh - sudo systemctl stop isula-build.service - ``` - -- Run the following command to restart the isula-build service: - - ```sh - sudo systemctl restart isula-build.service - ``` - -The systemd service file of the isula-build software installation package is stored in the `/usr/lib/systemd/system/isula-build.service` directory. If you need to modify the systemd configuration of the isula-build service, modify the file and run the following command to make the modification take effect. Then restart the isula-build service based on the systemd management command. - -```sh -sudo systemctl daemon-reload -``` - -### Directly Running isula-builder - -You can also run the `isula-builder` command on the server to start the service. The `isula-builder` command can contain flags for service startup. The following flags are supported: - -- `-D, --debug`: whether to enable the debugging mode. -- `--log-level`: log level. The options are **debug**, **info**, **warn**, and **error**. The default value is **info**. -- `--dataroot`: local persistency directory. The default value is **/var/lib/isula-build/**. -- `--runroot`: runtime directory. The default value is **/var/run/isula-build/**. -- `--storage-driver`: underlying storage driver type. -- `--storage-opt`: underlying storage driver configuration. -- `--group`: sets the owner group for the local socket file **isula_build.sock** so that non-privileged users in the group can use isula-build. The default owner group is **isula**. -- `--experimental`: whether to enable experimental features. - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> If the command line parameters contain the same configuration items as those in the configuration file, the command line parameters are preferentially used for startup. - -Start the isula-build service. For example, to specify the local persistency directory **/var/lib/isula-build** and disable debugging, run the following command: - -```sh -sudo isula-builder --dataroot "/var/lib/isula-build" --debug=false -``` - -# Usage Guidelines - -## Prerequisites - -isula-build depends on the executable file **runc** to build the **RUN** instruction in the Dockerfile. Therefore, runc must be pre-installed in the running environment of isula-build. The installation method depends on the application scenario. If you do not need to use the complete docker-engine tool chain, you can install only the docker-runc RPM package. - -```sh -sudo yum install -y docker-runc -``` - -If you need to use a complete docker-engine tool chain, install the docker-engine RPM package, which contains the executable file **runc** by default. - -```sh -sudo yum install -y docker-engine -``` - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> Ensure the security of OCI runtime (runc) executable files to prevent malicious replacement. - -## Overview - -The isula-build client provides a series of commands for building and managing container images. Currently, the isula-build client provides the following commands: - -- `ctr-img`: manages container images. The `ctr-img` command contains the following subcommands: - - `build`: builds a container image based on the specified Dockerfile. - - `images`: lists local container images. - - `import`: imports a basic container image. - - `load`: imports a cascade image. - - `rm`: deletes a local container image. - - `save`: exports a cascade image to a local disk. - - `tag`: adds a tag to a local container image. - - `pull`: pulls an image to a local host. - - `push`: pushes a local image to a remote repository. -- `info`: displays the running environment and system information of isula-build. -- `login`: logs in to the remote container image repository. -- `logout`: logs out of the remote container image repository. -- `version`: displays the versions of isula-build and isula-builder. -- `manifest` (experimental): manages the manifest list. - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> - The `isula-build completion` and `isula-builder completion` commands are used to generate the bash command completion script. These commands are implicitly provided by the command line framework and is not displayed in the help information. -> - isula-build client does not have any configuration file. To use isula-build experimental features, enable the environment variable **ISULABUILD_CLI_EXPERIMENTAL** on the client using the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command. - -The following describes how to use these commands in detail. - -## ctr-img: Container Image Management - -The isula-build command groups all container image management commands into the `ctr-img` command. The command format is as follows: - -```shell -isula-build ctr-img [command] -``` - -### build: Container Image Build - -The subcommand build of the `ctr-img` command is used to build container images. The command format is as follows: - -```shell -isula-build ctr-img build [flags] -``` - -The `build` command contains the following flags: - -- `--build-arg`: string list containing variables required during the build process. -- `--build-static`: key value, which is used to build binary equivalence. Currently, the following key values are included: - `- build-time`: string indicating that a container image is built at a specified timestamp. The timestamp format is *YYYY-MM-DD HH-MM-SS*. -- `-f, --filename`: string indicating the path of the Dockerfiles. If this parameter is not specified, the current path is used. -- `--format`: string indicating the image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled). -- `--iidfile`: string indicating a local file to which the ID of the image is output. -- `-o, --output`: string indicating the image export mode and path. -- `--proxy`: boolean, which inherits the proxy environment variable on the host. The default value is **true**. -- `--tag`: string indicating the tag value of the image that is successfully built. -- `--cap-add`: string list containing permissions required by the **RUN** instruction during the build process. - -**The following describes the flags in detail.** - -#### \--build-arg - -Parameters in the Dockerfile are inherited from the commands. The usage is as follows: - -```sh -$ echo "This is bar file" > bar.txt -$ cat Dockerfile_arg -FROM busybox -ARG foo -ADD ${foo}.txt . -RUN cat ${foo}.txt -$ sudo isula-build ctr-img build --build-arg foo=bar -f Dockerfile_arg -STEP 1: FROM busybox -Getting image source signatures -Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad -Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b -Writing manifest to image destination -Storing signatures -STEP 2: ARG foo -STEP 3: ADD ${foo}.txt . -STEP 4: RUN cat ${foo}.txt -This is bar file -Getting image source signatures -Copying blob sha256:6194458b07fcf01f1483d96cd6c34302ffff7f382bb151a6d023c4e80ba3050a -Copying blob sha256:6bb56e4a46f563b20542171b998cb4556af4745efc9516820eabee7a08b7b869 -Copying config sha256:39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2 -Writing manifest to image destination -Storing signatures -Build success with image id: 39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2 -``` - -#### \--build-static - -Specifies a static build. That is, when isula-build is used to build a container image, differences between all timestamps and other build factors (such as the container ID and hostname) are eliminated. Finally, a container image that meets the static requirements is built. - -When isula-build is used to build a container image, assume that a fixed timestamp is given to the build subcommand and the following conditions are met: - -- The build environment is consistent before and after the upgrade. -- The Dockerfile is consistent before and after the build. -- The intermediate data generated before and after the build is consistent. -- The build commands are the same. -- The versions of the third-party libraries are the same. - -For container image build, isula-build supports the same Dockerfile. If the build environments are the same, the image content and image ID generated in multiple builds are the same. - -`--build-static` supports the key-value pair option in the *key=value* format. Currently, the following options are supported: - -- build-time: string, which indicates the fixed timestamp for creating a static image. The value is in the format of *YYYY-MM-DD HH-MM-SS*. The timestamp affects the attribute of the file for creating and modifying the time at the diff layer. - - Example: - - ```sh - sudo isula-build ctr-img build -f Dockerfile --build-static='build-time=2020-05-23 10:55:33' . - ``` - - In this way, the container images and image IDs built in the same environment for multiple times are the same. - -#### \--format - -This option can be used when the experiment feature is enabled. The default image format is **oci**. You can specify the image format to build. For example, the following commands are used to build an OCI image and a Docker image, respectively. - - ```sh - export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format oci . - ``` - - ```sh - export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format docker . - ``` - -#### \--iidfile - -Run the following command to output the ID of the built image to a file: - -```shell -isula-build ctr-img build --iidfile filename -``` - -For example, to export the container image ID to the **testfile** file, run the following command: - - ```sh -sudo isula-build ctr-img build -f Dockerfile_arg --iidfile testfile - ``` - - Check the container image ID in the **testfile** file. - - ```sh -$ cat testfile -76cbeed38a8e716e22b68988a76410eaf83327963c3b29ff648296d5cd15ce7b - ``` - -#### \-o, --output - -Currently, `-o` and `--output` support the following formats: - -- `isulad:image:tag`: directly pushes the image that is successfully built to iSulad, for example, `-o isulad:busybox:latest`. The following restrictions apply: - - - isula-build and iSulad must be on the same node. - - The tag must be configured. - - On the isula-build client, you need to temporarily save the successfully built image as **/var/tmp/isula-build-tmp-%v.tar** and then import it to iSulad. Ensure that the **/var/tmp/** directory has sufficient disk space. - -- `docker-daemon:image:tag`: directly pushes the successfully built image to Docker daemon, for example, `-o docker-daemon:busybox:latest`. The following restrictions apply: -- isula-build and Docker must be on the same node. - - The tag must be configured. - -- `docker://registry.example.com/repository:tag`: directly pushes the successfully built image to the remote image repository in Docker image format, for example, `-o docker://localhost:5000/library/busybox:latest`. - -- `docker-archive:/:image:tag`: saves the successfully built image to the local host in Docker image format, for example, `-o docker-archive:/root/image.tar:busybox:latest`. - -When experiment feature is enabled, you can build image in OCI image format with: - -- `oci://registry.example.com/repository:tag`: directly pushes the successfully built image to the remote image repository in OCI image format(OCI image format should be supported by the remote repository), for example, `-o oci://localhost:5000/library/busybox:latest`. - -- `oci-archive:/:image:tag`：saves the successfully built image to the local host in OCI image format, for example, `-o oci-archive:/root/image.tar:busybox:latest`。 - -In addition to the flags, the `build` subcommand also supports an argument whose type is string and meaning is context, that is, the context of the Dockerfile build environment. The default value of this parameter is the current path where isula-build is executed. This path affects the path retrieved by the **ADD** and **COPY** instructions of the .dockerignore file and Dockerfile. - -#### \--proxy - -Specifies whether the container started by the **RUN** instruction inherits the proxy-related environment variables **http_proxy**, **https_proxy**, **ftp_proxy**, **no_proxy**, **HTTP_PROXY**, **HTTPS_PROXY**, and **FTP_PROXY**. The default value is **true**. - -When a user configures proxy-related **ARG** or **ENV** in the Dockerfile, the inherited environment variables will be overwritten. - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> If the client and daemon are running on different terminals, the environment variables of the terminal where the daemon is running are inherited. - -#### \--tag - -Specifies the tag of the image stored on the local disk after the image is successfully built. - -#### \--cap-add - -Run the following command to add the permission required by the **RUN** instruction during the build process: - -```shell -isula-build ctr-img build --cap-add ${CAP} -``` - -Example: - -```sh -sudo isula-build ctr-img build --cap-add CAP_SYS_ADMIN --cap-add CAP_SYS_PTRACE -f Dockerfile -``` - -> **Note:** -> -> - A maximum of 100 container images can be concurrently built. -> - isula-build supports Dockerfiles with a maximum size of 1 MB. -> - isula-build supports a .dockerignore file with a maximum size of 1 MB. -> - Ensure that only the current user has the read and write permissions on the Dockerfiles to prevent other users from tampering with the files. -> - During the build, the **RUN** instruction starts the container to build in the container. Currently, isula-build supports the host network only. -> - isula-build only supports the tar compression format. -> - isula-build commits once after each image build stage is complete, instead of each time a Dockerfile line is executed. -> - isula-build does not support cache build. -> - isula-build starts the build container only when the **RUN** instruction is built. -> - Currently, the history function of Docker images is not supported. -> - The stage name can start with a digit. -> - The stage name can contain a maximum of 64 characters. -> - isula-build does not support resource restriction on a single Dockerfile build. If resource restriction is required, you can configure a resource limit on isula-builder. -> - Currently, isula-build does not support a remote URL as the data source of the **ADD** instruction in the Dockerfile. -> - The local tar package exported using the **docker-archive** and **oci-archive** types are not compressed, you can manually compress the file as required. - -### image: Viewing Local Persistent Build Images - -You can run the `images` command to view the images in the local persistent storage. - -```sh -$ sudo isula-build ctr-img images ---------------------------------------- ----------- ----------------- ------------------------ ------------ -REPOSITORY TAG IMAGE ID CREATED SIZE ---------------------------------------- ----------- ----------------- ------------------------ ------------ -localhost:5000/library/alpine latest a24bb4013296 2022-01-17 10:02:19 5.85 MB - 39b62a3342ee 2022-01-17 10:01:12 1.45 MB ---------------------------------------- ----------- ----------------- ------------------------ ------------ -``` - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> The image size displayed by running the `isula-build ctr-img images` command may be different from that displayed by running the `docker images` command. When calculating the image size, `isula-build` directly calculates the total size of .tar packages at each layer, while `docker` calculates the total size of files by decompressing the .tar packages and traversing the diff directory. Therefore, the statistics are different. - -### import: Importing a Basic Container Image - -A tar file in rootfs form can be imported into isula-build via the `ctr-img import` command. - -The command format is as follows: - -```shell -isula-build ctr-img import [flags] -``` - -Example: - -```sh -$ sudo isula-build ctr-img import busybox.tar mybusybox:latest -Getting image source signatures -Copying blob sha256:7b8667757578df68ec57bfc9fb7754801ec87df7de389a24a26a7bf2ebc04d8d -Copying config sha256:173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1 -Writing manifest to image destination -Storing signatures -Import success with image id: "173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1" -$ sudo isula-build ctr-img images ---------------------------------------- ----------- ----------------- ------------------------ ------------ -REPOSITORY TAG IMAGE ID CREATED SIZE ---------------------------------------- ----------- ----------------- ------------------------ ------------ -mybusybox latest 173b3cf612f8 2022-01-12 16:02:31 1.47 MB ---------------------------------------- ----------- ----------------- ------------------------ ------------ -``` - ->![](./public_sys-resources/icon-note.gif) **Note** -> -> isula-build supports the import of container basic images with a maximum size of 1 GB. - -### load: Importing Cascade Images - -Cascade images are images that are saved to the local computer by running the `docker save` or `isula-build ctr-img save` command. The compressed image package contains a layer-by-layer image package named **layer.tar**. You can run the `ctr-img load` command to import the image to isula-build. - -The command format is as follows: - -```shell -isula-build ctr-img load [flags] -``` - -Currently, the following flags are supported: - -- `-i, --input`: path of the local .tar package. - -Example: - -```sh -$ sudo isula-build ctr-img load -i ubuntu.tar -Getting image source signatures -Copying blob sha256:cf612f747e0fbcc1674f88712b7bc1cd8b91cf0be8f9e9771235169f139d507c -Copying blob sha256:f934e33a54a60630267df295a5c232ceb15b2938ebb0476364192b1537449093 -Copying blob sha256:943edb549a8300092a714190dfe633341c0ffb483784c4fdfe884b9019f6a0b4 -Copying blob sha256:e7ebc6e16708285bee3917ae12bf8d172ee0d7684a7830751ab9a1c070e7a125 -Copying blob sha256:bf6751561805be7d07d66f6acb2a33e99cf0cc0a20f5fd5d94a3c7f8ae55c2a1 -Copying blob sha256:c1bd37d01c89de343d68867518b1155cb297d8e03942066ecb44ae8f46b608a3 -Copying blob sha256:a84e57b779297b72428fc7308e63d13b4df99140f78565be92fc9dbe03fc6e69 -Copying blob sha256:14dd68f4c7e23d6a2363c2320747ab88986dfd43ba0489d139eeac3ac75323b2 -Copying blob sha256:a2092d776649ea2301f60265f378a02405539a2a68093b2612792cc65d00d161 -Copying blob sha256:879119e879f682c04d0784c9ae7bc6f421e206b95d20b32ce1cb8a49bfdef202 -Copying blob sha256:e615448af51b848ecec00caeaffd1e30e8bf5cffd464747d159f80e346b7a150 -Copying blob sha256:f610bd1e9ac6aa9326d61713d552eeefef47d2bd49fc16140aa9bf3db38c30a4 -Copying blob sha256:bfe0a1336d031bf5ff3ce381e354be7b2bf310574cc0cd1949ad94dda020cd27 -Copying blob sha256:f0f15db85788c1260c6aa8ad225823f45c89700781c4c793361ac5fa58d204c7 -Copying config sha256:c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e -Writing manifest to image destination -Storing signatures -Loaded image as c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e -``` - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> - isula-build allows you to import a container image with a maximum size of 50 GB. -> - isula-build automatically recognizes the image format and loads it from the cascade image file. - -### rm: Deleting a Local Persistent Image - -You can run the `rm` command to delete an image from the local persistent storage. The command format is as follows: - -```shell -isula-build ctr-img rm IMAGE [IMAGE...] [FLAGS] -``` - -Currently, the following flags are supported: - -- `-a, --all`: deletes all images stored locally. -- `-p, --prune`: deletes all images that are stored locally and do not have tags. - -Example: - -```sh -$ sudo isula-build ctr-img rm -p -Deleted: sha256:78731c1dde25361f539555edaf8f0b24132085b7cab6ecb90de63d72fa00c01d -Deleted: sha256:eeba1bfe9fca569a894d525ed291bdaef389d28a88c288914c1a9db7261ad12c -``` - -### save: Exporting Cascade Images - -You can run the `save` command to export the cascade images to the local disk. The command format is as follows: - -```shell -isula-build ctr-img save [REPOSITORY:TAG]|imageID -o xx.tar -``` - -Currently, the following flags are supported: - -- `-f, --format`: which indicates the exported image format: **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled) -- `-o, --output`: which indicates the local path for storing the exported images. - -The following example shows how to export an image using *image/tag*: - -```sh -$ sudo isula-build ctr-img save busybox:latest -o busybox.tar -Getting image source signatures -Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a -Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f -Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef -Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6 -Writing manifest to image destination -Storing signatures -Save success with image: busybox:latest -``` - -The following example shows how to export an image using *ImageID*: - -```sh -$ sudo isula-build ctr-img save 21c3e96ac411 -o busybox.tar -Getting image source signatures -Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a -Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f -Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef -Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6 -Writing manifest to image destination -Storing signatures -Save success with image: 21c3e96ac411 -``` - -The following example shows how to export multiple images to the same tarball: - -```sh -$ sudo isula-build ctr-img save busybox:latest nginx:latest -o all.tar -Getting image source signatures -Copying blob sha256:eb78099fbf7fdc70c65f286f4edc6659fcda510b3d1cfe1caa6452cc671427bf -Copying blob sha256:29f11c413898c5aad8ed89ad5446e89e439e8cfa217cbb404ef2dbd6e1e8d6a5 -Copying blob sha256:af5bd3938f60ece203cd76358d8bde91968e56491daf3030f6415f103de26820 -Copying config sha256:b8efb18f159bd948486f18bd8940b56fd2298b438229f5bd2bcf4cedcf037448 -Writing manifest to image destination -Storing signatures -Getting image source signatures -Copying blob sha256:e2d6930974a28887b15367769d9666116027c411b7e6c4025f7c850df1e45038 -Copying config sha256:a33de3c85292c9e65681c2e19b8298d12087749b71a504a23c576090891eedd6 -Writing manifest to image destination -Storing signatures -Save success with image: [busybox:latest nginx:latest] -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- Save exports an image in .tar format by default. If necessary, you can save the image and then manually compress it. ->- When exporting an image using image name, specify the entire image name in the *REPOSITORY:TAG* format. - -### tag: Tagging Local Persistent Images - -You can run the `tag` command to add a tag to a local persistent container image. The command format is as follows: - -```shell -isula-build ctr-img tag / busybox:latest -``` - -Example: - -```sh -$ sudo isula-build ctr-img images ---------------------------------------- ----------- ----------------- -------------------------- ------------ -REPOSITORY TAG IMAGE ID CREATED SIZE ---------------------------------------- ----------- ----------------- -------------------------- ------------ -alpine latest a24bb4013296 2020-05-29 21:19:46 5.85 MB ---------------------------------------- ----------- ----------------- -------------------------- ------------ -$ sudo isula-build ctr-img tag a24bb4013296 alpine:v1 -$ sudo isula-build ctr-img images ---------------------------------------- ----------- ----------------- ------------------------ ------------ -REPOSITORY TAG IMAGE ID CREATED SIZE ---------------------------------------- ----------- ----------------- ------------------------ ------------ -alpine latest a24bb4013296 2020-05-29 21:19:46 5.85 MB -alpine v1 a24bb4013296 2020-05-29 21:19:46 5.85 MB ---------------------------------------- ----------- ----------------- ------------------------ ------------ -``` - -### pull: Pulling an Image To a Local Host - -Run the `pull` command to pull an image from a remote image repository to a local host. Command format: - -```shell -isula-build ctr-img pull REPOSITORY[:TAG] -``` - -Example: - -```sh -$ sudo isula-build ctr-img pull example-registry/library/alpine:latest -Getting image source signatures -Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad -Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b -Writing manifest to image destination -Storing signatures -Pull success with image: example-registry/library/alpine:latest -``` - -### push: Pushing a Local Image to a Remote Repository - -Run the `push` command to push a local image to a remote repository. Command format: - -```shell -isula-build ctr-img push REPOSITORY[:TAG] -``` - -Currently, the following flags are supported: - -- `-f, --format`: indicates the pushed image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled) - -Example: - -```sh -$ sudo isula-build ctr-img push example-registry/library/mybusybox:latest -Getting image source signatures -Copying blob sha256:d2421964bad195c959ba147ad21626ccddc73a4f2638664ad1c07bd9df48a675 -Copying config sha256:f0b02e9d092d905d0d87a8455a1ae3e9bb47b4aa3dc125125ca5cd10d6441c9f -Writing manifest to image destination -Storing signatures -Push success with image: example-registry/library/mybusybox:latest -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- Before pushing an image, log in to the corresponding image repository. - -## info: Viewing the Operating Environment and System Information - -You can run the `isula-build info` command to view the running environment and system information of isula-build. The command format is as follows: - -```shell - isula-build info [flags] -``` - -The following flags are supported: - -- `-H, --human-readable`: Boolean. The memory information is printed in the common memory format. The value is 1000 power. -- `-V, --verbose`: Boolean. The memory usage is displayed during system running. - -Example: - -```sh -$ sudo isula-build info -H - General: - MemTotal: 7.63 GB - MemFree: 757 MB - SwapTotal: 8.3 GB - SwapFree: 8.25 GB - OCI Runtime: runc - DataRoot: /var/lib/isula-build/ - RunRoot: /var/run/isula-build/ - Builders: 0 - Goroutines: 12 - Store: - Storage Driver: overlay - Backing Filesystem: extfs - Registry: - Search Registries: - oepkgs.net - Insecure Registries: - localhost:5000 - oepkgs.net - Runtime: - MemSys: 68.4 MB - HeapSys: 63.3 MB - HeapAlloc: 7.41 MB - MemHeapInUse: 8.98 MB - MemHeapIdle: 54.4 MB - MemHeapReleased: 52.1 MB -``` - -## login: Logging In to the Remote Image Repository - -You can run the `login` command to log in to the remote image repository. The command format is as follows: - -```shell - isula-build login SERVER [FLAGS] -``` - -Currently, the following flags are supported: - -```shell - Flags: - -p, --password-stdin Read password from stdin - -u, --username string Username to access registry -``` - -Enter the password through stdin. In the following example, the password in creds.txt is transferred to the stdin of isula-build through a pipe for input. - -```sh - $ cat creds.txt | sudo isula-build login -u cooper -p mydockerhub.io - Login Succeeded -``` - -Enter the password in interactive mode. - -```sh - $ sudo isula-build login mydockerhub.io -u cooper - Password: - Login Succeeded -``` - -## logout: Logging Out of the Remote Image Repository - -You can run the `logout` command to log out of the remote image repository. The command format is as follows: - -```shell - isula-build logout [SERVER] [FLAGS] -``` - -Currently, the following flags are supported: - -```shell - Flags: - -a, --all Logout all registries -``` - -Example: - -```sh - $ sudo isula-build logout -a - Removed authentications -``` - -## version: Querying the isula-build Version - -You can run the `version` command to view the current version information. - -```sh -$ sudo isula-build version -Client: - Version: 0.9.6-4 - Go Version: go1.15.7 - Git Commit: 83274e0 - Built: Wed Jan 12 15:32:55 2022 - OS/Arch: linux/amd64 - -Server: - Version: 0.9.6-4 - Go Version: go1.15.7 - Git Commit: 83274e0 - Built: Wed Jan 12 15:32:55 2022 - OS/Arch: linux/amd64 -``` - -## manifest: Manifest List Management - -The manifest list contains the image information corresponding to different system architectures. You can use the same manifest (for example, **openeuler:latest**) in different architectures to obtain the image of the corresponding architecture. The manifest contains the create, annotate, inspect, and push subcommands. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> manifest is an experiment feature. When using this feature, you need to enable the experiment options on the client and server. For details, see Client Overview and Configuring Services. - -### create: Manifest List Creation - -The create subcommand of the `manifest` command is used to create a manifest list. The command format is as follows: - -```shell -isula-build manifest create MANIFEST_LIST MANIFEST [MANIFEST...] -``` - -You can specify the name of the manifest list and the remote images to be added to the list. If no remote image is specified, an empty manifest list is created. - -Example: - -```sh -sudo isula-build manifest create openeuler localhost:5000/openeuler_x86:latest localhost:5000/openeuler_aarch64:latest -``` - -### annotate: Manifest List Update - -The `annotate` subcommand of the `manifest` command is used to update the manifest list. The command format is as follows: - -```shell -isula-build manifest annotate MANIFEST_LIST MANIFEST [flags] -``` - -You can specify the manifest list to be updated and the images in the manifest list, and use flags to specify the options to be updated. This command can also be used to add new images to the manifest list. - -Currently, the following flags are supported: - -- --arch: Applicable architecture of the rewritten image. The value is a string. -- --os: Indicates the applicable system of the image. The value is a string. -- --os-features: Specifies the OS features required by the image. This parameter is a string and rarely used. -- --variant: Variable of the image recorded in the list. The value is a string. - -Example: - -```sh -sudo isula-build manifest annotate --os linux --arch arm64 openeuler:latest localhost:5000/openeuler_aarch64:latest -``` - -### inspect: Manifest List Inspect - -The `inspect` subcommand of the `manifest` command is used to query the manifest list. The command format is as follows: - -```shell -isula-build manifest inspect MANIFEST_LIST -``` - -Example: - -```sh -$ sudo isula-build manifest inspect openeuler:latest -{ - "schemaVersion": 2, - "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", - "manifests": [ - { - "mediaType": "application/vnd.docker.distribution.manifest.v2+json", - "size": 527, - "digest": "sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff", - "platform": { - "architecture": "amd64", - "os": "linux" - } - }, - { - "mediaType": "application/vnd.docker.distribution.manifest.v2+json", - "size": 527, - "digest": "sha256:f814888b4bb6149bd39ba8375a1932fb15071b4dbffc7f76c7b602b06abbb820", - "platform": { - "architecture": "arm64", - "os": "linux" - } - } - ] -} -``` - -### push: Manifest List Push to the Remote Repository - -The manifest subcommand `push` is used to push the manifest list to the remote repository. The command format is as follows: - -```shell -isula-build manifest push MANIFEST_LIST DESTINATION -``` - -Example: - -```sh -sudo isula-build manifest push openeuler:latest localhost:5000/openeuler:latest -``` - -# Directly Integrating a Container Engine - -isula-build can be integrated with iSulad or Docker to import the built container image to the local storage of the container engine. - -## Integration with iSulad - -Images that are successfully built can be directly exported to the iSulad. - -Example: - -```sh -sudo isula-build ctr-img build -f Dockerfile -o isulad:busybox:2.0 -``` - -Specify iSulad in the -o parameter to export the built container image to iSulad. You can query the image using isula images. - -```sh -$ sudo isula images -isula images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox 2.0 2d414a5cad6d 2020-08-01 06:41:36 5.577 MB -``` - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> - It is required that isula-build and iSulad be on the same node. -> - When an image is directly exported to the iSulad, the isula-build client needs to temporarily store the successfully built image as `/var/lib/isula-build/tmp/[build_id]/isula-build-tmp-%v.tar` and then import it to the iSulad. Ensure that the /var/tmp/ directory has sufficient disk space. If the isula-build client process is killed or Ctrl+C is pressed during the export, you need to manually clear the `/var/lib/isula-build/tmp/[build_id]/isula-build-tmp-%v.tar` file. - -## Integration with Docker - -Images that are successfully built can be directly exported to the Docker daemon. - -Example: - -```sh -sudo isula-build ctr-img build -f Dockerfile -o docker-daemon:busybox:2.0 -``` - -Specify docker-daemon in the -o parameter to export the built container image to Docker. You can run the `docker images` command to query the image. - -```sh -$ sudo docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -busybox 2.0 2d414a5cad6d 2 months ago 5.22MB -``` - ->![](./public_sys-resources/icon-note.gif) **Note:** -> -> The isula-build and Docker must be on the same node. - -# Precautions - -This chapter is something about constraints, limitations and differences with `docker build` when use isula-builder build images. - -## Constraints or Limitations - -1. When export an image to [iSulad](https://gitee.com/openeuler/iSulad/blob/master/README.md/), a tag is necessary. -2. Because the OCI runtime, for example, **runc**, will be called by isula-builder when executing the **RUN** instruction, the integrity of the runtime binary should be guaranteed by the user. -3. DataRoot should not be set to **tmpfs**. -4. **Overlay2** is the only storage driver supported by isula-builder currently. -5. Docker image is the only image format supported by isula-builder currently. -6. You are advised to set file permission of the Dockerfile to **0600** to avoid tampering by other users. -7. Only host network is supported by the **RUN** instruction currently. -8. When export image to a tar package, only tar compression format is supported by isula-builder currently. -9. The base image size is limited to 1 GB when importing a base image using `import`. - -## Differences with "docker build" - -`isula-build` complies with [Dockerfile specification](https://docs.docker.com/engine/reference/builder), but there are also some subtle differences between `isula-builder` and `docker build` as follows: - -1. isula-builder commits after each build stage, but not every line. -2. Build cache is not supported by isula-builder. -3. Only **RUN** instruction will be executed in the build container. -4. Build history is not supported currently. -5. Stage name can be start with a number. -6. The length of the stage name is limited to 64 in `isula-builder`. -7. **ADD** instruction source can not be a remote URL currently. -8. Resource restriction on a single build is not supported. If resource restriction is required, you can configure a resource limit on isula-builder. -9. `isula-builder` add each origin layer tar size to get the image size, but docker only uses the diff content of each layer. So the image size listed by `isula-builder images` is different. -10. Image name should be in the *NAME:TAG* format. For example **busybox:latest**, where **latest** must not be omitted. - -# Appendix - -## Command Line Parameters - -**Table 1** Parameters of the `ctr-img build` command - -| **Command** | **Parameter** | **Description** | -| ------------- | -------------- | ------------------------------------------------------------ | -| ctr-img build | --build-arg | String list, which contains variables required during the build. | -| | --build-static | Key value, which is used to build binary equivalence. Currently, the following key values are included: - build-time: string, which indicates that a fixed timestamp is used to build a container image. The timestamp format is YYYY-MM-DD HH-MM-SS. | -| | -f, --filename | String, which indicates the path of the Dockerfiles. If this parameter is not specified, the current path is used. | -| | --format | String, which indicates the image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled). | -| | --iidfile | String, which indicates the ID of the image output to a local file. | -| | -o, --output | String, which indicates the image export mode and path.| -| | --proxy | Boolean, which inherits the proxy environment variable on the host. The default value is true. | -| | --tag | String, which indicates the tag value of the image that is successfully built. | -| | --cap-add | String list, which contains permissions required by the **RUN** instruction during the build process.| - -**Table 2** Parameters of the `ctr-img load` command - -| **Command** | **Parameter** | **Description** | -| ------------ | ----------- | --------------------------------- | -| ctr-img load | -i, --input | String, path of the local .tar package to be imported.| - -**Table 3** Parameters of the `ctr-img push` command - -| **Command** | **Parameter** | **Description** | -| ------------ | ----------- | --------------------------------- | -| ctr-img push | -f, --format | String, which indicates the pushed image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled).| - -**Table 4** Parameters of the `ctr-img rm` command - -| **Command** | **Parameter** | **Description** | -| ---------- | ----------- | --------------------------------------------- | -| ctr-img rm | -a, --all | Boolean, which is used to delete all local persistent images. | -| | -p, --prune | Boolean, which is used to delete all images that are stored persistently on the local host and do not have tags. | - -**Table 5** Parameters of the `ctr-img save` command - -| **Command** | **Parameter** | **Description** | -| ------------ | ------------ | ---------------------------------- | -| ctr-img save | -o, --output | String, which indicates the local path for storing the exported images.| -| ctr-img save | -f, --format | String, which indicates the exported image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled).| - -**Table 6** Parameters of the `login` command - -| **Command** | **Parameter** | **Description** | -| -------- | -------------------- | ------------------------------------------------------- | -| login | -p, --password-stdin | Boolean, which indicates whether to read the password through stdin. or enter the password in interactive mode. | -| | -u, --username | String, which indicates the username for logging in to the image repository.| - -**Table 7** Parameters of the `logout` command - -| **Command** | **Parameter** | **Description** | -| -------- | --------- | ------------------------------------ | -| logout | -a, --all | Boolean, which indicates whether to log out of all logged-in image repositories. | - -**Table 8** Parameters of the `manifest annotate` command - -| **Command** | **Parameter** | **Description** | -| ----------------- | ------------- | ---------------------------- | -| manifest annotate | --arch | Set architecture | -| | --os | Set operating system | -| | --os-features | Set operating system feature | -| | --variant | Set architecture variant | - -## Communication Matrix - -The isula-build component processes communicate with each other through the Unix socket file. No port is used for communication. - -## File and Permission - -- All isula-build operations must be performed by the **root** user. To perform operations as a non-privileged user, you need to configure the `--group` option. - -- The following table lists the file permissions involved in the running of isula-build. - -| **File Path** | **File/Folder Permission** | **Description** | -| ------------------------------------------- | ------------------- | ------------------------------------------------------------ | -| /usr/bin/isula-build | 550 | Binary file of the command line tool. | -| /usr/bin/isula-builder | 550 | Binary file of the isula-builder process. | -| /usr/lib/systemd/system/isula-build.service | 640 | systemd configuration file, which is used to manage the isula-build service. | -| /usr/isula-build | 650 | Root directory of the isula-builder configuration file. | -| /etc/isula-build/configuration.toml | 600 | General isula-builder configuration file, including the settings of the isula-builder log level, persistency directory, runtime directory, and OCI runtime. | -| /etc/isula-build/policy.json | 600 | Syntax file of the signature verification policy file. | -| /etc/isula-build/registries.toml | 600 | Configuration file of each image repository, including the available image repository list and image repository blacklist. | -| /etc/isula-build/storage.toml | 600 | Configuration file of the local persistent storage, including the configuration of the used storage driver. | -| /etc/isula-build/isula-build.pub | 400 | Asymmetric encryption public key file. | -| /var/run/isula_build.sock | 660 | Local socket of isula-builder. | -| /var/lib/isula-build | 700 | Local persistency directory. | -| /var/run/isula-build | 700 | Local runtime directory. | -| /var/lib/isula-build/tmp/\[build_id\]/isula-build-tmp-*.tar | 644 | Local temporary directory for storing the images when they are exported to iSulad. | diff --git a/docs/en/docs/Container/isulad-container-engine.md b/docs/en/docs/Container/isulad-container-engine.md deleted file mode 100644 index 54cd5ca2112776a9d584b4eb2e5132607a5dd743..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/isulad-container-engine.md +++ /dev/null @@ -1,12 +0,0 @@ -# iSulad Container Engine - - -Compared with Docker, iSulad is a new container solution with a unified architecture design to meet different requirements in the CT and IT fields. Lightweight containers are implemented using C/C++. They are smart, fast, and not restricted by hardware and architecture. With less noise floor overhead, the containers can be widely used. - -[Figure 1](#en-us_topic_0182207099_fig10763114141217) shows the unified container architecture. - -**Figure 1** Unified container architecture - - -![](./figures/en-us_image_0183048952.png) - diff --git a/docs/en/docs/Container/kuasar-install-config.md b/docs/en/docs/Container/kuasar-install-config.md deleted file mode 100644 index c4633b45e5fdaf9bccff8e788dc48d8ea00409dc..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/kuasar-install-config.md +++ /dev/null @@ -1,126 +0,0 @@ -# Installation and Configuration - -## Installation - -### Prerequisites - -- To obtain better performance experience, Kuasar must run on bare metal servers. **Currently, Kuasar cannot run on VMs.** -- The running of Kuasar depends on the following openEuler components. Ensure that the dependent components of the required versions have been installed in the environment. - - iSulad (See [Installation and Configuration](./installation-configuration) of iSulad.) - - StratoVirt (See [Installing StratoVirt](../StratoVirt/Install_StratoVirt.md)) - -### Procedure - -1. The Kuasar deliverables are included in the **kuasar** RPM package. Run the `yum` command to directly install Kuasar. - - ```sh - yum install kuasar - ``` - -2. Install the CRI command line tool crictl required for starting sandboxes and containers. - - ```sh - # Arm environment - $ wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.25.0/crictl-v1.25.0-linux-arm64.tar.gz - $ tar -zxvf crictl-v1.25.0-linux-arm64.tar.gz -C /usr/local/bin - # x86 environment - $ wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.25.0/crictl-v1.25.0-linux-amd64.tar.gz - $ tar -zxvf crictl-v1.25.0-linux-amd64.tar.gz -C /usr/local/bin - ``` - -3. Install the CNI plugins required for CRI to configure the network. - - ```sh - $ mkdir -p /opt/cni/bin && mkdir -p /etc/cni/net.d - - # Arm environment - $ wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-arm64-v1.3.0.tgz - $ tar -zxvf cni-plugins-linux-arm64-v1.3.0.tgz -C /opt/cni/bin/ - # x86 environment - $ wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-amd64-v1.3.0.tgz - $ tar -zxvf cni-plugins-linux-amd64-v1.3.0.tgz -C /opt/cni/bin/ - ``` - -## Configuration - -### Configuring iSulad - -Modify the iSulad configuration file **/etc/isulad/daemon.json** so that iSulad can invoke the container runtime of the Kuasar VMM type. Add the following information: - -```sh -$ cat /etc/isulad/daemon.json -... - "cri-sandboxers": { - "vmm": { - "name": "vmm", - "address": "/run/vmm-sandboxer.sock" - } - }, - "cri-runtimes": { - "vmm": "io.containerd.vmm.v1" - }, -... -``` - -Restart iSulad. - -```sh -systemctl restart isulad -``` - -### crictl Configuration - -Modify the crictl configuration file **/etc/crictl.yaml** to connect to iSulad. - -```sh -$ cat /etc/crictl.yaml -runtime-endpoint: unix:///var/run/isulad.sock -image-endpoint: unix:///var/run/isulad.sock -timeout: 10 -``` - -### Kuasar configuration - -Modify the configuration file to connect Kuasar to StratoVirt. (You can use the default configuration. For details about the fields in the configuration file, see [附录](./kuasar-appendix.md ).) - -```sh -$ cat /var/lib/kuasar/config_stratovirt.toml -[sandbox] -log_level = "info" - -[hypervisor] -path = "/usr/bin/stratovirt" -machine_type = "virt,mem-share=on" -kernel_path = "/var/lib/kuasar/vmlinux.bin" -image_path = "" -initrd_path = "/var/lib/kuasar/kuasar.initrd" -kernel_params = "task.log_level=debug task.sharefs_type=virtiofs" -vcpus = 1 -memory_in_mb = 1024 -block_device_driver = "virtio-blk" -debug = true -enable_mem_prealloc = false - -[hypervisor.virtiofsd_conf] -path = "/usr/bin/vhost_user_fs" -``` - -Start the kuasar-vmm service. - -```sh -systemctl start kuasar-vmm -``` - -Check whether the service is running. - -```sh -$ systemctl status kuasar-vmm -● kuasar-vmm.service - Kuasar microVM type sandboxer daemon process - Loaded: loaded (/usr/lib/systemd/system/kuasar-vmm.service; disabled; vendor preset: disabled) - Active: active (running) since Sat 2023-08-26 14:57:08 CST; 1h 25min ago - Main PID: 1000445 (vmm-sandboxer) - Tasks: 99 (limit: 814372) - Memory: 226.4M - CGroup: /system.slice/kuasar-vmm.service - └─ 1000445 /usr/local/bin/vmm-sandboxer --listen /run/vmm-sandboxer.sock --dir /run/kuasar-vmm -``` diff --git a/docs/en/docs/Container/kuasar-usage.md b/docs/en/docs/Container/kuasar-usage.md deleted file mode 100644 index 211dc468e9a5bab6c2298fe02dfddb7122276370..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/kuasar-usage.md +++ /dev/null @@ -1,92 +0,0 @@ -# Usage Instructions - -Start a Kuasar sandbox. - -1. Ensure that Kuasar and related components have been correctly installed and configured. - -2. Prepare the service container image. Assume that the container image is **busybox**. Use the iSula container engine to download the container image. - - ```sh - isula pull busybox - ``` - -3. Prepare the YAML files for the pod and container. The file examples are as follows: - - ```sh - $ cat podsandbox.yaml - metadata: - name: busybox-sandbox - namespace: default - uid: hdishd83djaidwnduwk28bcsc - log_directory: /tmp - linux: - namespaces: - options: {} - - $ cat pod-container.yaml - metadata: - name: busybox - image: - image: docker.io/library/busybox:latest - command: - - top - log_path: busybox.log - ``` - -4. Start a pod. - - ```sh - $ crictl runp --runtime=vmm podsandbox.yaml - 5cbcf744949d8500e7159d6bd1e3894211f475549c0be15d9c60d3c502c7ede3 - ``` - - Check the pod list. The pod is in the **Ready** state. - - ```sh - $ crictl pods - POD ID CREATED STATE NAME NAMESPACE ATTEMPT - 5cbcf744949d8 About a minute ago Ready busybox-sandbox default 1 - ``` - -5. Create a service container in the pod. - - ```sh - $ crictl create 5cbcf744949d8500e7159d6bd1e3894211f475549c0be15d9c60d3c502c7ede3 pod-container.yaml podsandbox.yaml - c11df540f913e57d1e28372334c028fd6550a2ba73208a3991fbcdb421804a50 - ``` - - View the container list. The container is in the **Created** state. - - ```sh - $ crictl ps -a - CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID - c11df540f913e docker.io/library/busybox:latest 15 seconds ago Created busybox 0 5cbcf744949d - ``` - -6. Start the service container. - - ```sh - crictl start c11df540f913e57d1e28372334c028fd6550a2ba73208a3991fbcdb421804a50 - ``` - - Check the container list. The container is in the **Running** state. - - ```sh - $ crictl ps - CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID - c11df540f913e docker.io/library/busybox:latest 2 minutes ago Running busybox 0 5cbcf744949d8 - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >You can also run a `crictl run` command to start a pod with a service container. - > - >```sh - >$ crictl run -r vmm --no-pull container-config.yaml podsandbox-config.yaml - >``` - -7. Stop and delete the container and the pod. - - ```sh - crictl rm -f c11df540f913e - crictl rmp -f 5cbcf744949d8 - ``` diff --git a/docs/en/docs/Container/kuasar.md b/docs/en/docs/Container/kuasar.md deleted file mode 100644 index b39468efdfbe793bd087ce1abb7a2058ea0d479f..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/kuasar.md +++ /dev/null @@ -1,12 +0,0 @@ -# Kuasar Multi-Sandbox Container Runtime - -## Overview - -Kuasar is a container runtime that supports unified management of multiple types of sandboxes. It supports multiple mainstream sandbox isolation technologies, including the kernel-based native container sandbox, lightweight virtualization technology-based microVM sandbox, application kernel sandbox based on process-level virtualization, and the emerging WebAssembly sandbox. -Based on the Kuasar unified container runtime combined with the iSulad container engine and StratoVirt virtualization engine, openEuler builds lightweight full-stack self-developed secure containers for cloud native scenarios, delivering key competitiveness of ultra-low overhead and ultra-fast startup. - -**Figure 1** Kuasar architecture -![](./figures/kuasar_arch.png) - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The root permission is required for installing and using Kuasar. diff --git a/docs/en/docs/Container/local-volume-management.md b/docs/en/docs/Container/local-volume-management.md deleted file mode 100644 index 705a361e14127625d93299099c0c4433984171ff..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/local-volume-management.md +++ /dev/null @@ -1,234 +0,0 @@ -# Local Volume Management - -- [Local Volume Management](#local-volume-management) - - [Overview](#overview) - - [Precautions](#precautions) - - [Usage](#usage) - - [Using the -v Option to Mount Data](#using-the--v-option-to-mount-data) - - [Format](#format) - - [Functions](#functions) - - [Parameter Description](#parameter-description) - - [Examples](#examples) - - [Using the --mount Option to Mount Data](#using-the---mount-option-to-mount-data) - - [Format](#format-1) - - [Functions](#functions-1) - - [Parameter Description](#parameter-description-1) - - [Examples](#examples-1) - - [Reusing the Mounting Configuration in Other Containers](#reusing-the-mounting-configuration-in-other-containers) - - [Format](#format-2) - - [Functions](#functions-2) - - [Parameter Description](#parameter-description-2) - - [Examples](#examples-2) - - [Using the Anonymous Volume in an Image](#using-the-anonymous-volume-in-an-image) - - [Querying a Volume](#querying-a-volume) - - [Format](#format-3) - - [Functions](#functions-3) - - [Parameter Description](#parameter-description-3) - - [Examples](#examples-3) - - [Deleting a Volume](#deleting-a-volume) - - [Format](#format-4) - - [Functions](#functions-4) - - [Parameter Description](#parameter-description-4) - - [Examples](#examples-4) - - [Precautions](#precautions-1) - - [Conflict Combination Rules](#conflict-combination-rules) - - [Differences Between iSula and Docker](#differences-between-isula-and-docker) - -## Overview - -After a container managed by iSula is destroyed, all data in the container is destroyed. If you want to retain data after the container is destroyed, a data persistence mechanism is required. iSula allows files, directories, or volumes on a host to be mounted to a container at runtime. You can write the data to be persisted to the mount point in the container. After the container is destroyed, the files, directories, and volumes on the host are retained. If you need to delete a file, directory, or volume on the host, you can manually delete the file or directory, or run the iSula command to delete the volume. Currently, the iSula supports only local volume management. Local volumes are classified into named volumes and anonymous volumes. A volume whose name is specified by a user is called a named volume. If a user does not specify a name for a volume, iSula automatically generates a name (a 64-bit random number) for the volume, that is, an anonymous volume. - -The following describes how to use iSula to manage local volumes. - -## Precautions - -- The volume name contains 2 to 64 characters and complies with the regular expression ^\[a-zA-Z0-9\]\[a-zA-Z0-9_.-\]{1,63}$. That is, the first character of the volume name must be a letter or digit, and other characters can be letters, digits, underscores (_), periods (.), and hyphens (-). -- During container creation, if data exists at the mount point of the container corresponding to the volume, the data is copied to the volume by default. If the iSula breaks down or restarts or the system is powered off during the copy process, the data in the volume may be incomplete. In this case, you need to manually delete the volume or the data in the volume to ensure that the data is correct and complete. - -## Usage - -### Using the -v Option to Mount Data - -#### Format - -```shell -isula run -v [SRC:]DST[:MODE,MODE...] IMAGE -``` - -#### Functions - -When you create and run a container, use the -v/--volume option to mount the files, directories, or volumes on the host to the container for data persistence. - -#### Parameter Description - -- SRC: Path of the file, directory, or volume to be mounted on the host. If the value is an absolute path, a file or folder on the host is mounted. If the value is a volume name, a volume is mounted. If this parameter is not specified, an anonymous volume is mounted. If a folder or volume does not exist, iSula creates a folder or volume and then mounts it. -- DST: Mount path in the container. The value must be an absolute path. -- MODE: When the source to be mounted is a directory or file, the valid parameters are ro, rw, z, Z, private, rprivate, slave, rslave, shared, and rshared. Only one parameter of the same type can be configured. If the source is a volume, the valid parameters are ro, rw, z, Z, and nocopy. Only one parameter of the same type can be configured. Use commas (,) to separate multiple attributes. The parameters are described as follows: - -| Parameter | Description | -| -------- | -----------------------------------------------| -| ro | The mount point in the container is mounted in read-only mode. | -| rw | The mount point in the container is mounted in read/write mode. | -| z | If SELinux is enabled, add the SELinux share label during mounting. | -| Z | If SELinux is enabled, add the SELinux private label during mounting. | -| private | The mount point in the container is mounted in private propagation mode. | -| rprivate | The mount point in the container is recursively mounted in private propagation mode. | -| slave | The mount point in the container is mounted in subordinate propagation mode. | -| rslave | The mount point in the container is recursively mounted in subordinate propagation mode. | -| shared | The mount point in the container is mounted in shared propagation mode. | -| rshared | The mount point in the container is recursively mounted in shared propagation mode. | -| nocopy | Data at the mount point is not copied. If this parameter is not set, data is copied by default. In addition, if data already exists in the volume, the data will not be copied. | - -#### Examples - -Run the container based on BusyBox, create or mount a volume named vol to the /vol directory of the container, and set the mount point to read-only. In addition, if data exists at the mount point in the container, the data is not copied. - -```shell -isula run -v vol:/vol:ro,nocopy busybox -``` - -### Using the --mount Option to Mount Data - -#### Format - -```shell -isula run --mount [type=TYPE,][src=SRC,]dst=DST[,KEY=VALUE] busybox -``` - -#### Functions - -When you create and run a container, use the --mount option to mount the files, directories, or volumes on the host to the container for data persistence. - -#### Parameter Description - -- type: Type of data mounted to the container. The value can be **bind**, **volume**, **squashfs**, or **tmpfs**. If this parameter is not specified, the default value is **volume**. -- src: Path of the file, directory, or volume to be mounted on the host. If the value is an absolute path, the file or directory on the host is mounted. If the value is a volume name, a volume is mounted. If this parameter is not specified, the volume is an anonymous volume. If a folder or volume does not exist, iSula creates a file or volume and then mounts it. The keyword src is also called source. -- dst: Mount path in the container. The value must be an absolute path. The keyword dst is also called destination or target. -- KEY=VALUE: Parameter of `--mount`. The values are as follows: - -| Key | Value | -| ------------------------------ | --------------------------------------------------------------------------- | -| selinux-opts/bind-selinux-opts | z or Z. z indicates that if SELinux is enabled, the SELinux share label is added during mounting. Z indicates that if SELinux is enabled, the SELinux private label is added during mounting.| -| ro/readonly | 0/false indicates that the mount is read/write. 1/true indicates that the mount is read-only. If this parameter is not specified, the mount is read-only. The parameter is supported only when type is set to bind. | -| bind-propagation | The value is **private/rprivate/slave/rslave/shared/rshared** and functions similar to the value of `-v`. This is valid only when **type** is **bind**. | -| volume-nocopy | Data at the mount point is not copied. If this parameter is not specified, data is copied by default. In addition, if data already exists in the volume, the data will not be copied. This parameter is supported only when type is set to volume. | -| tmpfs-size | Maximum sizeof tmpfs. This is not limited by default. | -| tmpfs-mode | Permission for the mounted tmpfs. By default, the value is **777**. | - -#### Examples - -Run the container based on BusyBox, create or mount a volume named vol to the /vol directory of the container, and set the mount point to read-only. In addition, if data exists at the mount point in the container, the data is not copied. - -```shell -isula run --mount type=volume,src=vol,dst=/vol,ro=true,volume-nocopy=true busybox -``` - -### Reusing the Mounting Configuration in Other Containers - -#### Format - -```shell -isula run --volumes-from CON1[:MODE] busybox -``` - -#### Functions - -When you create and run a container, use the --volumes-from option to indicate that the mount point configuration includes that of the CON1 container. You can set multiple --volumes-from options. - -#### Parameter Description - -- CON1: Name or ID of the container whose mount point is reused. -- MODE: If the value is ro, the mount point is read-only. If the value is rw, the mount point is read/write. - -#### Examples - -Assume that a container named container1 has been configured with a volume vol1 to the container directory /vol1, and a container named container2 has been configured with a volume vol2 to the container directory /vol2. Run a new container to reuse the mounting configuration of container1 and container2. That is, volume vol1 is mounted to the /vol1 directory of the container, and volume vol2 is mounted to the /vol2 directory of the container. - -```shell -isula run --volumes-from container1 --volumes-from container2 busbyox -``` - -### Using the Anonymous Volume in an Image - -You do not need to perform any configuration to use the anonymous volume in the image. If an anonymous volume is configured in the image, iSula automatically creates an anonymous volume and mounts it to the specified path in the image at container runtime. You can write data to the mount point of an anonymous volume in a container for data persistence. - -### Querying a Volume - -#### Format - -```shell -isula volume ls [OPTIONS] -``` - -#### Functions - -This command is used to query all volumes managed by iSula. - -#### Parameter Description - -Option: - -- -q,--quiet: If this parameter is not specified, only the volume driver information and volume name are queried by default. If this parameter is specified, only the volume name is queried. - -#### Examples - -This command is used to query all volumes managed by iSula and return only the volume name. - -```shell -isula volume ls -q -``` - -### Deleting a Volume - -#### Format - -```shell -isula volume rm [OPTIONS] VOLUME [VOLUME...] -isula volume prune [OPTIONS] -``` - -#### Functions - -- rm: deletes a specified volume. If the volume is used by a container, the volume fails to be deleted. -- prune: deletes all volumes that are not used by containers. - -#### Parameter Description - -OPTIONS in the prune command: - -- -f,--force: specifies that the system does not display a message asking you whether to delete the volume. By default, a risk message is displayed. You need to enter y to continue the operation. - -#### Examples - -Delete volumes vol1 and vol2. - -```shell -isula volume rm vol1 vol2 -``` - -Delete all unused volumes in the following format. No risk message is displayed. - -```shell -isula volume prune -f -``` - -### Precautions - -#### Conflict Combination Rules - -If a volume mount point conflict occurs, perform the following operations: - -- If configurations of -v and --mount conflict, a failure message is returned. -- If the configuration obtained from --volumes-from conflicts with the -v or --mount configuration, the configuration is discarded. -- If the anonymous volume configuration in the image conflicts with the -v, --mount, or --volumes-from configuration, the configuration is discarded. - -#### Differences Between iSula and Docker - -| iSula Behavior | Docker Behavior | -| ------------------------------------------- | ------------------------------------------- | -| The volume name can contain a maximum of 64 characters. | The length of the volume name is not limited. | -| If the source to be mounted does not exist, the --mount parameter is created. | If the source to be mounted does not exist, an error is reported. | -| The --mount parameter supports the z or Z parameter configuration in bind-selinux-opts and selinux-opts. | The --mount parameter does not support the parameter configuration in the bind-selinux-opts and selinux-opts. | -| Rules for combining mount point conflicts are not processed. | The anonymous volume specified by -v is processed as the anonymous volume in the image. | -| The volume prune command displays the space that has been reclaimed. | The volume prune command does not display the space that has been reclaimed. | -| -v, --mount, and --volumes-from are configured in hostconfig, and the anonymous volume is configured in config. | The anonymous volume specified by -v is configured in config, and other configurations are configured in hostconfig. | diff --git a/docs/en/docs/Container/managing-the-lifecycle-of-a-secure-container.md b/docs/en/docs/Container/managing-the-lifecycle-of-a-secure-container.md deleted file mode 100644 index c16a2ee9a8a087af3d2b3643aba0a162d85f8b2a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/managing-the-lifecycle-of-a-secure-container.md +++ /dev/null @@ -1,107 +0,0 @@ -# Managing the Lifecycle of a Secure Container - -- [Managing the Lifecycle of a Secure Container](#managing-the-lifecycle-of-a-secure-container) - - [Starting a Secure Container](#starting-a-secure-container) - - [Stopping a Secure Container](#stopping-a-secure-container) - - [Deleting a Secure Container](#deleting-a-secure-container) - - [Running a New Command in the Container](#running-a-new-command-in-the-container) - - - - -## Starting a Secure Container - -You can use the Docker engine or iSulad as the container engine of the secure container. The invoking methods of the two engines are similar. You can select either of them to start a secure container. - -To start a secure container, perform the following steps: - -1. Ensure that the secure container component has been correctly installed and deployed. -2. Prepare the container image. If the container image is busybox, run the following commands to download the container image using the Docker engine or iSulad: - - ``` - docker pull busybox - ``` - - ``` - isula pull busybox - ``` - -3. Start a secure container. Run the following commands to start a secure container using the Docker engine and iSulad: - - ``` - docker run -tid --runtime kata-runtime --network none busybox - ``` - - ``` - isula run -tid --runtime kata-runtime --network none busybox - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The secure container supports the CNI network only and does not support the CNM network. The **-p** and **--expose** options cannot be used to expose container ports. When using a secure container, you need to specify the **--net=none** option. - -4. Start a pod. - 1. Start the pause container and obtain the sandbox ID of the pod based on the command output. Run the following commands to start a pause container using the Docker engine and iSulad: - - ``` - docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox - ``` - - ``` - isula run -tid --runtime kata-runtime --network none --annotation io.kubernetes.cri.container-type=sandbox - ``` - -    - - 1. Create a service container and add it to the pod. Run the following commands to create a service container using the Docker engine and iSulad: - - ``` - docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=container --annotation io.kubernetes.sandbox.id= busybox - ``` - - ``` - isula run -tid --runtime kata-runtime --network none --annotation io.kubernetes.cri.container-type=container --annotation io.kubernetes.cri.sandbox-id= busybox - ``` - - **--annotation** is used to mark the container type, which is provided by the Docker engine and iSulad, but not provided by the open-source Docker engine in the upstream community. - - - -## Stopping a Secure Container - -- Run the following command to stop a secure container: - - ``` - docker stop - ``` - -- Stop a pod. - - When stopping a pod, note that the lifecycle of the pause container is the same as that of the pod. Therefore, stop service containers before the pause container. - - -## Deleting a Secure Container - -Ensure that the container has been stopped. - -``` -docker rm -``` - -To forcibly delete a running container, run the **-f** command. - -``` -docker rm -f -``` - -## Running a New Command in the Container - -The pause container functions only as a placeholder container. Therefore, if you start a pod, run a new command in the service container. The pause container does not execute the corresponding command. If only one container is started, run the following command directly: - -``` -docker exec -ti -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->1. If the preceding command has no response because another host runs the **docker restart** or **docker stop** command to access the same container, you can press **Ctrl**+**P**+**Q** to exit the operation. ->2. If the **-d** option is used, the command is executed in the background and no error information is displayed. The exit code cannot be used to determine whether the command is executed correctly. - diff --git a/docs/en/docs/Container/maximum-number-of-handles.md b/docs/en/docs/Container/maximum-number-of-handles.md deleted file mode 100644 index a8cdb1d40bf2a63c78e36d75b8bc8207b02aeff2..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/maximum-number-of-handles.md +++ /dev/null @@ -1,59 +0,0 @@ -# Maximum Number of Handles - -- [Maximum Number of Handles](#maximum-number-of-handles) - - -## Function Description - -System containers support limit on the number of file handles. File handles include common file handles and network sockets. When starting a container, you can specify the **--files-limit** parameter to limit the maximum number of handles opened in the container. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--files-limit

-

  

-
  • The value cannot be negative and must be an integer.
  • The value 0 indicates that the number is not limited by the parameter. The maximum number is determined by the current kernel files cgroup.
-
- -## Constraints - -- If the value of **--files-limit** is too small, the system container may fail to run the **exec** command and the error "open temporary files" is reported. Therefore, you are advised to set the parameter to a large value. -- File handles include common file handles and network sockets. - -## Example - -To use **--files-limit** to limit the number of file handles opened in a container, run the following command to check whether the kernel supports files cgroup: - -``` -[root@localhost ~]# cat /proc/1/cgroup | grep files -10:files:/ -``` - -If **files** is displayed, files cgroup is supported. - -Start the container, specify the **--files-limit** parameter, and check whether the **files.limit** parameter is successfully written. - -``` -[root@localhost ~]# isula run -tid --files-limit 1024 --system-container --external-rootfs /tmp/root-fs empty init 01e82fcf97d4937aa1d96eb8067f9f23e4707b92de152328c3fc0ecb5f64e91d -[root@localhost ~]# isula exec -it 01e82fcf97d4 bash -[root@localhost ~]# cat /sys/fs/cgroup/files/files.limit -1024 - -``` - -The preceding information indicates that the number of file handles is successfully limited in the container. - diff --git a/docs/en/docs/Container/monitoring-secure-containers.md b/docs/en/docs/Container/monitoring-secure-containers.md deleted file mode 100644 index 4bea45be650d38a74aea8f86d1478e8833de4f4e..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/monitoring-secure-containers.md +++ /dev/null @@ -1,56 +0,0 @@ -# Monitoring Secure Containers - -- [Monitoring Secure Containers](#monitoring-secure-containers) - - -## Description - -In kata 2.x, events subcommand is removed and replaced by **kata-runtime metrics**, which can be used to gather metrics associated with infrastructure used to run a sandbox, including virtual machine stats, shim v2 CPU seconds and CPU stat of guest OS and so on. Metrics are organized in a Prometheus compatible format so that they can be easily uploaded to Prometheus when work with kata-monitor. - -## Usage - -``` -kata-runtime metrics -``` - -## Prerequisites - -The sandbox ID must be the full ID. The sandbox to be queried must be in the **running** state. Otherwise, the following error message will be displayed: "Container ID \(\) does not exist". - -When using annotation to make a container run in a specific sandbox, clients should not use kata-runtime metrics to gather metrics of that container. The correct way is to query the corresponding sandbox. - -This command can be used to query the status of only one container. - -## Example -```shell -$ kata-runtime metrics e2270357d23f9d3dd424011e1e70aa8defb267d813c3d451db58f35aeac97a04 - -# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. -# TYPE go_gc_duration_seconds summary -go_gc_duration_seconds{quantile="0"} 2.656e-05 -go_gc_duration_seconds{quantile="0.25"} 3.345e-05 -go_gc_duration_seconds{quantile="0.5"} 3.778e-05 -go_gc_duration_seconds{quantile="0.75"} 4.657e-05 -go_gc_duration_seconds{quantile="1"} 0.00023001 -go_gc_duration_seconds_sum 0.00898126 -go_gc_duration_seconds_count 195 -# HELP go_goroutines Number of goroutines that currently exist. -# TYPE go_goroutines gauge -go_goroutines 27 -# HELP go_info Information about the Go environment. -# TYPE go_info gauge -go_info{version="go1.17.3"} 1 -# HELP kata_hypervisor_netdev Net devices statistics. -# TYPE kata_hypervisor_netdev gauge -kata_hypervisor_netdev{interface="lo",item="recv_bytes"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_compressed"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_drop"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_errs"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_fifo"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_frame"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_multicast"} 0 -kata_hypervisor_netdev{interface="lo",item="recv_packets"} 0 -kata_hypervisor_netdev{interface="lo",item="sent_bytes"} 0 -kata_hypervisor_netdev{interface="lo",item="sent_carrier"} 0 -kata_hypervisor_netdev{interface="lo",item="sent_colls"} 0 -``` \ No newline at end of file diff --git a/docs/en/docs/Container/privileged-container.md b/docs/en/docs/Container/privileged-container.md deleted file mode 100644 index cd19dffa22c7fe8b214beb9b5227c1bb2db47cf6..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/privileged-container.md +++ /dev/null @@ -1,239 +0,0 @@ -# Privileged Container - -- [Privileged Container](#privileged-container.) - - [Scenarios](#scenarios) - - [Usage Restrictions](#usage-restrictions-1) - - [Usage Guide](#usage-guide) - - -## Scenarios - -By default, iSulad starts common containers that are suitable for starting common processes. However, common containers have only the default permissions defined by capabilities in the **/etc/default/isulad/config.json** directory. To perform privileged operations \(such as use devices in the **/sys** directory\), a privileged container is required. By using this feature, user **root** in the container has **root** permissions of the host. Otherwise, user **root** in the container has only common user permissions of the host. - -## Usage Restrictions - -Privileged containers provide all functions for containers and remove all restrictions enforced by the device cgroup controller. A privileged container has the following features: - -- Secomp does not block any system call. -- The **/sys** and **/proc** directories are writable. -- All devices on the host can be accessed in the container. - -- All system capabilities will be enabled. - -Default capabilities of a common container are as follows: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Capability Key

-

Description

-

SETPCAP

-

Modifies the process capabilities.

-

MKNOD

-

Allows using the system call mknod() to create special files.

-

AUDIT_WRITE

-

Writes records to kernel auditing logs.

-

CHOWN

-

Modifies UIDs and GIDs of files. For details, see the chown(2).

-

NET_RAW

-

Uses RAW and PACKET sockets and binds any IP address to the transparent proxy.

-

DAC_OVERRIDE

-

Ignores the discretionary access control (DAC) restrictions on files.

-

FOWNER

-

Ignores the restriction that the file owner ID must be the same as the process user ID.

-

FSETID

-

Allows setting setuid bits of files.

-

KILL

-

Allows sending signals to processes that do not belong to itself.

-

SETGID

-

Allows the change of the process group ID.

-

SETUID

-

Allows the change of the process user ID.

-

NET_BIND_SERVICE

-

Allows bounding to a port whose number is smaller than 1024.

-

SYS_CHROOT

-

Allows using the system call chroot().

-

SETFCAP

-

Allows transferring and deleting capabilities to other processes.

-
- -When a privileged container is enabled, the following capabilities are added: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Capability Key

-

Description

-

SYS_MODULE

-

Loads and unloads kernel modules.

-

SYS_RAWIO

-

Allows direct access to /devport, /dev/mem, /dev/kmem, and original block devices.

-

SYS_PACCT

-

Allows the process BSD audit.

-

SYS_ADMIN

-

Allows executing system management tasks, such as loading or unloading file systems and setting disk quotas.

-

SYS_NICE

-

Allows increasing the priority and setting the priorities of other processes.

-

SYS_RESOURCE

-

Ignores resource restrictions.

-

SYS_TIME

-

Allows changing the system clock.

-

SYS_TTY_CONFIG

-

Allows configuring TTY devices.

-

AUDIT_CONTROL

-

Enables and disables kernel auditing, modifies audit filter rules, and extracts audit status and filtering rules.

-

MAC_ADMIN

-

Overrides the mandatory access control (MAC), which is implemented for the Smack Linux Security Module (LSM).

-

MAC_OVERRIDE

-

Allows MAC configuration or status change, which is implemented for Smack LSM.

-

NET_ADMIN

-

Allows executing network management tasks.

-

SYSLOG

-

Performs the privileged syslog(2) operation.

-

DAC_READ_SEARCH

-

Ignores the DAC access restrictions on file reading and catalog search.

-

LINUX_IMMUTABLE

-

Allows modifying the IMMUTABLE and APPEND attributes of a file.

-

NET_BROADCAST

-

Allows network broadcast and multicast access.

-

IPC_LOCK

-

Allows locking shared memory segments.

-

IPC_OWNER

-

Ignores the IPC ownership check.

-

SYS_PTRACE

-

Allows tracing any process.

-

SYS_BOOT

-

Allows restarting the OS.

-

LEASE

-

Allows modifying the FL_LEASE flag of a file lock.

-

WAKE_ALARM

-

Triggers the function of waking up the system, for example, sets the CLOCK_REALTIME_ALARM and CLOCK_BOOTTIME_ALARM timers.

-

BLOCK_SUSPEND

-

Allows blocking system suspension.

-
- -## Usage Guide - -iSulad runs the **--privileged** command to enable the privilege mode for containers. Do not add privileges to containers unless necessary. Comply with the principle of least privilege to reduce security risks. - -``` -isula run --rm -it --privileged busybox -``` - diff --git a/docs/en/docs/Container/querying-information.md b/docs/en/docs/Container/querying-information.md deleted file mode 100644 index 34bc33a21945d246c518b75c690b3b68b74d536c..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/querying-information.md +++ /dev/null @@ -1,95 +0,0 @@ -# Querying Information - -- [Querying Information](#querying-information) - - [Querying the Service Version](#querying-the-service-version) - - [Querying System-level Information](#querying-system-level-information) - - -## Querying the Service Version - -### Description - -The **isula version** command is run to query the version of the iSulad service. - -### Usage - -``` -isula version -``` - -### Example - -Query the version information. - -``` -isula version -``` - -If the iSulad service is running properly, you can view the information about versions of the client, server, and **OCI config**. - -``` -Client: - Version: 1.0.31 - Git commit: fa7f9902738e8b3d7f2eb22768b9a1372ddd1199 - Built: 2019-07-30T04:21:48.521198248-04:00 - -Server: - Version: 1.0.31 - Git commit: fa7f9902738e8b3d7f2eb22768b9a1372ddd1199 - Built: 2019-07-30T04:21:48.521198248-04:00 - -OCI config: - Version: 1.0.0-rc5-dev - Default file: /etc/default/isulad/config.json -``` - -If the iSulad service is not running, only the client information is queried and a message is displayed indicating that the connection times out. - -``` -Client: - Version: 1.0.31 - Git commit: fa7f9902738e8b3d7f2eb22768b9a1372ddd1199 - Built: 2019-07-30T04:21:48.521198248-04:00 - -Can not connect with server.Is the iSulad daemon running on the host? -``` - -Therefore, the **isula version** command is often used to check whether the iSulad service is running properly. - -## Querying System-level Information - -### Description - -The **isula info** command is run to query the system-level information, number of containers, and number of images. - -### Usage - -``` -isula info -``` - -### Example - -Query system-level information, including the number of containers, number of images, kernel version, and operating system \(OS\). - -``` -$ isula info -Containers: 2 - Running: 0 - Paused: 0 - Stopped: 2 -Images: 8 -Server Version: 1.0.31 -Logging Driver: json-file -Cgroup Driverr: cgroupfs -Hugetlb Pagesize: 2MB -Kernel Version: 4.19 -Operating System: Fedora 29 (Twenty Nine) -OSType: Linux -Architecture: x86_64 -CPUs: 8 -Total Memory: 7 GB -Name: localhost.localdomain -iSulad Root Dir: /var/lib/isulad -``` - diff --git a/docs/en/docs/Container/reboot-or-shutdown-in-a-container.md b/docs/en/docs/Container/reboot-or-shutdown-in-a-container.md deleted file mode 100644 index 84d5c380070ef178a6e525732c4f5cbf21619d0c..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/reboot-or-shutdown-in-a-container.md +++ /dev/null @@ -1,82 +0,0 @@ -# Reboot or Shutdown in a Container - -- [Reboot or Shutdown in a Container](#reboot-or-shutdown-in-a-container) - - -## Function Description - -The **reboot** and **shutdown** commands can be executed in a system container. You can run the **reboot** command to restart a container, and run the **shutdown** command to stop a container. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--restart

-
  • Variable of the string type.
  • Supported option is as follows:

    on-reboot: restarts the system container.

    -

      

    -
-
- -## Constraints - -- The shutdown function relies on the actual OS of the container running environment. -- When you run the **shutdown -h now** command to shut down the system, do not open multiple consoles. For example, if you run the **isula run -ti** command to open a console and run the **isula attach** command for the container in another host bash, another console is opened. In this case, the **shutdown** command fails to be executed. - -## Example - -- Specify the **--restart on-reboot** parameter when starting a container. For example: - - ``` - [root@localhost ~]# isula run -tid --restart on-reboot --system-container --external-rootfs /root/myrootfs none init - 106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f - ``` - - -- In the container, run the **reboot** command. - - ``` - [root@localhost ~]# isula exec -it 10 bash - [root@localhost /]# reboot - ``` - - Check whether the container is restarted. - - ``` - [root@localhost ~]# isula exec -it 10 ps aux - USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND - root 1 0.1 0.0 21588 9504 ? Ss 12:11 0:00 init - root 14 0.1 0.0 27024 9376 ? Ss 12:11 0:00 /usr/lib/system - root 17 0.0 0.0 18700 5876 ? Ss 12:11 0:00 /usr/lib/system - dbus 22 0.0 0.0 9048 3624 ? Ss 12:11 0:00 /usr/bin/dbus-d - root 26 0.0 0.0 8092 3012 ? Rs+ 12:13 0:00 ps aux - ``` - -- In the container, run the **shutdown** command. - - ``` - [root@localhost ~]# isula exec -it 10 bash - [root@localhost /]# shutdown -h now - [root@localhost /]# [root@localhost ~]# - ``` - - Check whether the container is stopped. - - ``` - [root@localhost ~]# isula exec -it 10 bash - Error response from daemon: Exec container error;Container is not running:106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f - ``` - - diff --git a/docs/en/docs/Container/secure-container.md b/docs/en/docs/Container/secure-container.md deleted file mode 100644 index 760d0296ff0c680eeba4c07dc4be7e6683a7d750..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/secure-container.md +++ /dev/null @@ -1,29 +0,0 @@ -# Secure Container - - -## Overview - -The secure container technology is an organic combination of virtualization and container technologies. Compared with a common Linux container, a secure container has better isolation performance. - -Common Linux containers use namespaces to isolate the running environment between processes and use cgroups to limit resources. Essentially, these common Linux containers share the same kernel. Therefore, if a single container affects the kernel intentionally or unintentionally, the containers on the same host will be affected. - -Secure containers are isolated by the virtualization layers. Containers on the same host do not affect each other. - -**Figure 1** Secure container architecture - - -![](./figures/secure-container.png) - -Secure containers are closely related to the concept of pod in Kubernetes. Kubernetes is the open-source ecosystem standard for the container scheduling management platform. It defines a group of container runtime interfaces \(CRIs\). - -In the CRI standards, a pod is a logical grouping of one or more containers, which are scheduled together and share interprocess communication \(IPC\) and network namespaces. As the smallest unit for scheduling, a pod must contain a pause container and one or more service containers. The lifecycle of a pause container is the same as that of the pod. - -A lightweight virtual machine \(VM\) in a secure container is a pod. The first container started in the VM is the pause container, and the containers started later are service containers. - -In a secure container, you can start a single container or start a pod. - -[Figure 2](#fig17734185518269) shows the relationship between the secure container and peripheral components. - -**Figure 2** Relationship between the secure container and peripheral components -![](./figures/relationship-between-the-secure-container-and-peripheral-components.png "relationship-between-the-secure-container-and-peripheral-components") - diff --git a/docs/en/docs/Container/security-and-isolation.md b/docs/en/docs/Container/security-and-isolation.md deleted file mode 100644 index 740d6295972c57c7a64b17dee7d54a173522ec18..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/security-and-isolation.md +++ /dev/null @@ -1,339 +0,0 @@ -# Security and Isolation - -- [Security and Isolation](#security-and-isolation) - - [Many-to-Many User Namespaces](#many-to-many-user-namespaces) - - [User Permission Control](#user-permission-control) - - [proc File System Isolation](#proc-file-system-isolation) - -## Many-to-Many User Namespaces - -### Function Description - -User namespaces are used to map user **root** of a container to a common user of the host and allow the processes and user in the container \(that are unprivileged on the host\) to have privilege. This can prevent the processes in the container from escaping to the host and performing unauthorized operations. In addition, after user namespaces are used, the container and host use different UIDs and GIDs. This ensures that user resources in the container such as file descriptors are isolated from those on the host. - -In system containers, you can configure the **--user-remap** API parameter to map user namespaces of different containers to different user namespaces on the host, isolating the user namespaces of containers. - -### Parameter Description - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--user-remap

-

The parameter format is uid:gid:offset. The parameter is described as follows:

-
  • uid and gid must be integers greater than or equal to 0.
  • offset must be an integer greater than 0 and less than 65536. The value cannot be too small. Otherwise, the container cannot be started.
  • Either the sum of uid and offset or the sum of gid and offset must be less than or equal to 232 - 1. Otherwise, an error is reported during container startup.
-
- -### Constraints - -- If **--user-remap** is specified in a system container, the rootfs directory must be accessible to users specified by _uid_ or _gid_ in **--user-remap**. Otherwise, user namespaces of containers cannot access rootfs. As a result, the containers fail to be started. -- All IDs in the container can be mapped to the host rootfs. Some directories or files may be mounted from the host to containers, for example, device files in the **/dev/pts** directory. If _offset_ is too small, the mounting may fail. -- _uid_, _gid_, and _offset_ are controlled by the upper-layer scheduling platform. The container engine only checks the validity of them. -- **--user-remap** is available only in system containers. -- **--user-remap** and **--privileged** cannot be set simultaneously. Otherwise, an error is reported during container startup. -- If _uid_ or _gid_ is set to **0**, **--user-remap** does not take effect. -- If **--user-map** is specified for a system container, ensure that the user corresponding to the specified UID or GID can access the isulad metadata directories (**/var/lib/isulad/**, **/var/lib/isulad/engines/**, and **/var/lib/isulad/engines/lcr**). -- **--user-remap** and **--userns** cannot be specified at the same time. - -### Usage Guide - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Before specifying the **--user-remap** parameter, configure an offset value for UIDs and GIDs of all directories and files in rootfs. The offset value should be equal to that for _uid_ and _gid_ in **--user-remap**. ->For example, run the following command to offset UIDs and GIDs of all files in the **dev** directory with 100000: ->chown 100000:100000 dev - -Specify the **--user-remap** parameter when the system container is started. - -```shell -[root@localhost ~]# chmod 751 /var/lib/isulad/ -[root@localhost ~]# chmod 751 /var/lib/isulad/engines/ -[root@localhost ~]# chmod 751 /var/lib/isulad/engines/lcr -[root@localhost ~]# isula run -tid --user-remap 100000:100000:65535 --system-container --external-rootfs /home/root-fs none /sbin/init -eb9605b3b56dfae9e0b696a729d5e1805af900af6ce24428fde63f3b0a443f4a -``` - -Check the /sbin/init process information on the host and in a container. - -```shell -[root@localhost ~]# isula exec eb ps aux | grep /sbin/init -root 1 0.6 0.0 21624 9624 ? Ss 15:47 0:00 /sbin/init -[root@localhost ~]# ps aux | grep /sbin/init -100000 4861 0.5 0.0 21624 9624 ? Ss 15:47 0:00 /sbin/init -root 4948 0.0 0.0 213032 808 pts/0 S+ 15:48 0:00 grep --color=auto /sbin/init -``` - -The owner of the /sbin/init process in the container is user **root**, but the owner of the host is the user whose UID is **100000**. - -Create a file in a container and view the file owner on the host. - -```shell -[root@localhost ~]# isula exec -it eb bash -[root@localhost /]# echo test123 >> /test123 -[root@localhost /]# exit -exit -[root@localhost ~]# ll /home/root-fs/test123 --rw-------. 1 100000 100000 8 Aug 2 15:52 /home/root-fs/test123 -``` - -The owner of the file that is generated in the container is user **root**, but the file owner displayed on the host is the user whose ID is **100000**. - -## User Permission Control - -### Function Description - -A container engine supports TLS for user identity authentication, which is used to control user permissions. Currently, container engines can connect to the authz plug-in to implement permission control. - -### API Description - -You can configure the startup parameters of the iSulad container engine to specify the permission control plug-in. The default daemon configuration file is **/etc/isulad/daemon.json**. - - - - - - - - - - - -

Parameter

-

Example

-

Description

-

--authorization-plugin

-

"authorization-plugin": "authz-broker"

-

User permission authentication plug-in. Currently, only authz-broker is supported.

-
- -### Constraints - -- User permission policies need to be configured for authz. The default policy file is **/var/lib/authz-broker/policy.json**. This file can be dynamically modified and the modification will take effect immediately without restarting the plug-in service. -- A container engine can be started by user **root**. If some commands used are enabled for by common users, common users may obtain excessive permissions. Therefore, exercise caution when performing such operations. Currently, running the **container\_attach**, **container\_create**, and **container\_exec\_create** commands may cause risks. -- Some compound operations, such as running **isula exec** and **isula inspect** or running and **isula attach** and **isula inspect**, depend on the permission of **isula inspect**. If a user does not have this permission, an error is reported. -- Using SSL/TLS encryption channels hardens security but also reduces performance. For example, the delay increases, more CPU resources are consumed, and encryption and decryption require higher throughput. Therefore, the number of concurrent executions decreases compared with non-TLS communication. According to the test result, when the ARM server \(Cortex-A72 64-core\) is almost unloaded, TLS is used to concurrently start a container. The maximum number of concurrent executions is 200 to 250. -- If **--tlsverify** is specified on the server, the default path where authentication files store is **/etc/isulad**. The default file names are **ca.pem**, **cert.pem**, and **key.pem**. - -### Example - -1. Ensure that the authz plug-in is installed on the host. If the authz plug-in is not installed, run the following command to install and start the authz plug-in service: - - ```shell - [root@localhost ~]# yum install authz - [root@localhost ~]# systemctl start authz - ``` - -2. To enable this function, configure the container engine and TLS certificate. You can use OpenSSL to generate the required certificate. - - ```shell - #SERVERSIDE - - # Generate CA key - openssl genrsa -aes256 -passout "pass:$PASSWORD" -out "ca-key.pem" 4096 - # Generate CA - openssl req -new -x509 -days $VALIDITY -key "ca-key.pem" -sha256 -out "ca.pem" -passin "pass:$PASSWORD" -subj "/C=$COUNTRY/ST=$STATE/L=$CITY/O=$ORGANIZATION/OU=$ORGANIZATIONAL_UNIT/CN=$COMMON_NAME/emailAddress=$EMAIL" - # Generate Server key - openssl genrsa -out "server-key.pem" 4096 - - # Generate Server Certs. - openssl req -subj "/CN=$COMMON_NAME" -sha256 -new -key "server-key.pem" -out server.csr - - echo "subjectAltName = DNS:localhost,IP:127.0.0.1" > extfile.cnf - echo "extendedKeyUsage = serverAuth" >> extfile.cnf - - openssl x509 -req -days $VALIDITY -sha256 -in server.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "server-cert.pem" -extfile extfile.cnf - - #CLIENTSIDE - - openssl genrsa -out "key.pem" 4096 - openssl req -subj "/CN=$CLIENT_NAME" -new -key "key.pem" -out client.csr - echo "extendedKeyUsage = clientAuth" > extfile.cnf - openssl x509 -req -days $VALIDITY -sha256 -in client.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "cert.pem" -extfile extfile.cnf - ``` - - If you want to use the preceding content as the script, replace the variables with the configured values. If the parameter used for generating the CA is empty, set it to **"**. **PASSWORD**, **COMMON\_NAME**, **CLIENT\_NAME**, and **VALIDITY** are mandatory. - -3. When starting the container engine, add parameters related to the TLS and authentication plug-in and ensure that the authentication plug-in is running properly. In addition, to use TLS authentication, the container engine must be started in TCP listening mode instead of the Unix socket mode. The configuration on the container daemon is as follows: - - ```json - { - "tls": true, - "tls-verify": true, - "tls-config": { - "CAFile": "/root/.iSulad/ca.pem", - "CertFile": "/root/.iSulad/server-cert.pem", - "KeyFile":"/root/.iSulad/server-key.pem" - }, - "authorization-plugin": "authz-broker" - } - ``` - -4. Configure policies. For the basic authorization process, all policies are stored in the **/var/lib/authz-broker/policy.json** configuration file. The configuration file can be dynamically modified without restarting the plug-in. Only the SIGHUP signal needs to be sent to the authz process. In the file, a line contains one JSON policy object. The following provides policy configuration examples: - - - All users can run all iSuald commands: **\{"name":"policy\_0","users":\[""\],"actions":\[""\]\}** - - Alice can run all iSulad commands: **\{"name":"policy\_1","users":\["alice"\],"actions":\[""\]\}** - - A blank user can run all iSulad commands: **\{"name":"policy\_2","users":\[""\],"actions":\[""\]\}** - - Alice and Bob can create new containers: **\{"name":"policy\_3","users":\["alice","bob"\],"actions":\["container\_create"\]\}** - - service\_account can read logs and run **docker top**: **\{"name":"policy\_4","users":\["service\_account"\],"actions":\["container\_logs","container\_top"\]\}** - - Alice can perform any container operations: **\{"name":"policy\_5","users":\["alice"\],"actions":\["container"\]\}** - - Alice can perform any container operations, but the request type can only be **get**: **\{"name":"policy\_5","users":\["alice"\],"actions":\["container"\], "readonly":true\}** - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >- **actions** supports regular expressions. - >- **users** does not support regular expressions. - >- A users cannot be repeatedly specified by **users**. That is, a user cannot match multiple rules. - -5. After updating the configurations, configure TLS parameters on the client to connect to the container engine. That is, access the container engine with restricted permissions. - - ```shell - [root@localhost ~]# isula version --tlsverify --tlscacert=/root/.iSulad/ca.pem --tlscert=/root/.iSulad/cert.pem --tlskey=/root/.iSulad/key.pem -H=tcp://127.0.0.1:2375 - ``` - - If you want to use the TLS authentication for default client connection, move the configuration file to **\~/.iSulad** and set the **ISULAD\_HOST** and **ISULAD\_TLS\_VERIFY** variables \(rather than transferring **-H=tcp://$HOST:2375** and -**-tlsverify** during each call\). - - ```shell - [root@localhost ~]# mkdir -pv ~/.iSulad - [root@localhost ~]# cp -v {ca,cert,key}.pem ~/.iSulad - [root@localhost ~]# export ISULAD_HOST=localhost:2375 ISULAD_TLS_VERIFY=1 - [root@localhost ~]# isula version - ``` - -## proc File System Isolation - -### Application Scenario - -Container virtualization is lightweight and efficient, and can be quickly deployed. However, containers are not strongly isolated, which causes great inconvenience to users. Containers have some defects in isolation because the namespace feature of the Linux kernel is not perfect. For example, you can view the proc information on the host \(such as meminfo, cpuinfo, stat, and uptime\) in the proc file system of a container. You can use the lxcfs tool to replace the /proc content of instances in the container with the content in the /proc file system of the host so that services in the container can obtain the correct resource value. - -### API Description - -A system container provides two tool packages: lxcfs and lxcfs-toolkit, which are used together. Lxcfs resides on the host as the daemon process. lxcfs-toolkit mounts the lxcfs file system of the host to containers through the hook mechanism. - -The command line of lxcfs-toolkit is as follows: - -```shell -lxcfs-toolkit [OPTIONS] COMMAND [COMMAND_OPTIONS] -``` - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Function

-

Parameter

-

remount

-

Remounts lxcfs to containers.

-

--all: remounts lxcfs to all containers.

-

--container-id: remounts lxcfs to a specified container.

-

umount

-

Unmounts lxcfs from containers.

-

--all: unmounts lxcfs from all containers.

-

--container-id: unmounts lxcfs from a specified container.

-

check-lxcfs

-

Checks whether the lxcfs service is running properly.

-

None

-

prestart

-

Mounts the /var/lib/lxcfs directory to the container before the lxcfs service starts.

-

None

-
- -### Constraints - -- Currently, only the **cpuinfo**, **meminfo**, **stat**, **diskstats**, **partitions**, **swaps**, and **uptime** files in the proc file system are supported. Other files are not isolated from other kernel API file systems \(such as sysfs\). -- After an RPM package is installed, a sample JSON file is generated in **/var/lib/lcrd/hooks/hookspec.json**. To add the log function, you need to add the **--log** configuration during customization. -- The **diskstats** file displays only information about disks that support CFQ scheduling, instead of partition information. Devices in containers are displayed as names in the **/dev** directory. If a device name does not exist, the information is left blank. In addition, the device where the container root directory is located is displayed as **sda**. -- The **slave** parameter is required when lxcfs is mounted. If the **shared** parameter is used, the mount point in containers may be leaked to the host, affecting the host running. -- Lxcfs supports graceful service degradation. If the lxcfs service crashes or becomes unavailable, the **cpuinfo**, **meminfo**, **stat**, **diskstats**, **partitions**, **swaps**and **uptime** files in containers are about host information, and other service functions of containers are not affected. -- Bottom layer of lxcfs depends on the FUSE kernel module and libfuse library. Therefore, the kernel needs to support FUSE. -- Lxcfs supports only the running of 64-bit applications in containers. If a 32-bit application is running in a container, the CPU information \(**cpuinfo**\) read by the application may fail to meet expectations. -- Lxcfs simulates the resource view only of container control groups \(cgroups\). Therefore, system calls \(such as sysconf\) in containers can obtain only host information. Lxcfs cannot implement the kernel isolation. -- The CPU information \(**cpuinfo**\) displayed after lxcfs implements the isolation has the following features: - - **processor**: The value increases from 0. - - **physical id**: The value increases from 0. - - **sibliing**: It has a fixed value of **1**. - - **core id**: It has a fixed value of **0**. - - **cpu cores**: It has a fixed value of **1**. - -### Example - -1. Install the lxcfs and lxcfs-toolkit packages and start the lxcfs service. - - ```shell - [root@localhost ~]# yum install lxcfs lxcfs-toolkit - [root@localhost ~]# systemctl start lxcfs - ``` - -2. After a container is started, check whether the lxcfs mount point exists in the container. - - ```shell - [root@localhost ~]# isula run -tid -v /var/lib/lxc:/var/lib/lxc --hook-spec /var/lib/isulad/hooks/hookspec.json --system-container --external-rootfs /home/root-fs none init - a8acea9fea1337d9fd8270f41c1a3de5bceb77966e03751346576716eefa9782 - [root@localhost ~]# isula exec a8 mount | grep lxcfs - lxcfs on /var/lib/lxc/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/partitions type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) - ``` - -3. Run the **update** command to update the CPU and memory resource configurations of the container and check the container resources. As shown in the following command output, the container resource view displays the actual container resource data instead of data of the host. - - ```shell - [root@localhost ~]# isula update --cpuset-cpus 0-1 --memory 1G a8 - a8 - [root@localhost ~]# isula exec a8 cat /proc/cpuinfo - processor : 0 - BogoMIPS : 100.00 - cpu MHz : 2400.000 - Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid - CPU implementer : 0x41 - CPU architecture: 8 - CPU variant : 0x0 - CPU part : 0xd08 - CPU revision : 2 - - processor : 1 - BogoMIPS : 100.00 - cpu MHz : 2400.000 - Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid - CPU implementer : 0x41 - CPU architecture: 8 - CPU variant : 0x0 - CPU part : 0xd08 - CPU revision : 2 - - [root@localhost ~]# isula exec a8 free -m - total used free shared buff/cache available - Mem: 1024 17 997 7 8 1006 - Swap: 4095 0 4095 - ``` diff --git a/docs/en/docs/Container/security-features.md b/docs/en/docs/Container/security-features.md deleted file mode 100644 index 65436a67383285c7832d39d9fe42cd9176a571b9..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/security-features.md +++ /dev/null @@ -1,255 +0,0 @@ -# Security Features - -- [Security Features](#security-features) - - [Seccomp Security Configuration](#seccomp-security-configuration) - - [Scenarios](#scenarios) - - [Usage Restrictions](#usage-restrictions) - - [Usage Guide](#usage-guide) - - [capabilities Security Configuration](#capabilities-security-configuration) - - [Scenarios](#scenarios-1) - - [Usage Restrictions](#usage-restrictions-1) - - [Usage Guide](#usage-guide-1) - - [SELinux Security Configuration](#selinux-security-configuration) - - [Scenarios](#scenarios-2) - - [Usage Restrictions](#usage-restrictions-2) - - [Usage Guide](#usage-guide-2) - -## Seccomp Security Configuration - -### Scenarios - -Secure computing mode \(Seccomp\) is a simple sandboxing mechanism introduced to the Linux kernel from version 2.6.23. In some specific scenarios, you may want to perform some privileged operations in a container without starting the privileged container. You can add **--cap-add** at runtime to obtain some small-scope permissions. For container instances with strict security requirements, th capability granularity may not meet the requirements. You can use some methods to control the permission scope in a refined manner. - -- Example - - In a common container scenario, you can use the **-v** flag to map a directory \(including a binary file that cannot be executed by common users\) on the host to the container. - - In the container, you can add chmod 4777 \(the modification permission of the binary file\) to the S flag bit. In this way, on the host, common users who cannot run the binary file \(or whose running permission is restricted\) can obtain the permissions of the binary file \(such as the root permission\) when running the binary file after the action added to the S flag bit is performed, so as to escalate the permission or access other files. - - In this scenario, if strict security requirements are required, the chmod, fchmod, and fchmodat system calls need to be tailored by using Seccomp. - -### Usage Restrictions - -- Do not disable the Seccomp feature of iSulad. - By default, Seccomp uses an allowlist to disable system calls that are not in the list. **--security-opt 'seccomp:unconfined'** disables the Seccomp feature. If Seccomp is disabled or the allowlist is not correctly configured, the attack surface is increased. - -- By default, Seccomp returns **SCMP_ACT_ERRNO** to system calls that are not in the allow list and opens system calls based on capabilities. Capabilities that are not in the allowlist will not be added to the container. - -### Usage Guide - -Use **--security-opt** to transfer the configuration file to the container where system calls need to be filtered. - -```shell -isula run -itd --security-opt seccomp=/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->1. When the configuration file is transferred to the container by using **--security-opt** during container creation, the default configuration file \(**/etc/isulad/seccomp\_default.json**\) is used. ->2. When **--security-opt** is set to **unconfined** during container creation, system calls are not filtered for the container. ->3. **/path/to/seccomp/profile.json** must be an absolute path. - -#### Obtaining the Default Seccomp Configuration of a Common Container - -- Start a common container \(or a container with **--cap-add**\) and check its default permission configuration. - - ```shell - cat /etc/isulad/seccomp_default.json | python -m json.tool > profile.json - ``` - - The **seccomp** field contains many **syscalls** fields. Then extract only the **syscalls** fields and perform the customization by referring to the customization of the seccomp configuration file. - - ```text - "defaultAction": "SCMP_ACT_ERRNO", - "syscalls": [ - { - "action": "SCMP_ACT_ALLOW", - "name": "accept" - }, - { - "action": "SCMP_ACT_ALLOW", - "name": "accept4" - }, - { - "action": "SCMP_ACT_ALLOW", - "name": "access" - }, - { - "action": "SCMP_ACT_ALLOW", - "name": "alarm" - }, - { - "action": "SCMP_ACT_ALLOW", - "name": "bind" - }, - ]... - ``` - -- Check the Seccomp configuration that can be identified by the LXC. - - ```shell - cat /var/lib/isulad/engines/lcr/74353e38021c29314188e29ba8c1830a4677ffe5c4decda77a1e0853ec8197cd/seccomp - ``` - - ```console - ... - waitpid allow - write allow - writev allow - ptrace allow - personality allow [0,0,SCMP_CMP_EQ,0] - personality allow [0,8,SCMP_CMP_EQ,0] - personality allow [0,131072,SCMP_CMP_EQ,0] - personality allow [0,131080,SCMP_CMP_EQ,0] - personality allow [0,4294967295,SCMP_CMP_EQ,0] - ... - ``` - -#### Customizing the Seccomp Configuration File - -When starting a container, use **--security-opt** to introduce the Seccomp configuration file. Container instances will restrict the running of system APIs based on the configuration file. Obtain the default Seccomp configuration of common containers, obtain the complete template, and customize the configuration file by referring to this section to start the container. - -```shell -isula run --rm -it --security-opt seccomp:/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox -``` - -The configuration file template is as follows: - -```text -{ -"defaultAction": "SCMP_ACT_ALLOW", -"syscalls": [ -{ -"name": "syscall-name", -"action": "SCMP_ACT_ERRNO", -"args": null -} -] -} -``` - ->![](./public_sys-resources/icon-notice.gif) **NOTICE:** -> ->- **defaultAction** and **syscalls**: The types of their corresponding actions are the same, but their values must be different. The purpose is to ensure that each syscall has a default action. Clear definitions in the syscall array shall prevail. As long as the values of **defaultAction** and **action** are different, no action conflicts will occur. The following actions are supported: -> **SCMP\_ACT\_ERRNO**: forbids calling syscalls and displays error information. -> **SCMP\_ACT\_ALLOW**: allows calling syscalls. ->- **syscalls**: array, which can contain one or more syscalls. **args** is optional. ->- **name**: syscalls to be filtered. ->- **args**: array. The definition of each object in the array is as follows: -> -> ```c -> type Arg struct { -> Index uint `json:"index"` // Parameter ID. Take open(fd, buf, len) as an example. The fd corresponds to 0 and buf corresponds to 1. -> Value uint64 `json:"value"` // Value to be compared with the parameter. -> ValueTwo uint64 `json:"value_two"` // It is valid only when Op is set to MaskEqualTo. After the bitwise AND operation is performed on the user-defined value and the value of Value, the result is compared with the value of ValueTwo. If they are the same, the action is executed. -> Op Operator `json:"op"` -> } -> ``` -> -> The value of **Op** in **args** can be any of the following: -> "SCMP\_CMP\_NE": NotEqualTo -> "SCMP\_CMP\_LT": LessThan -> "SCMP\_CMP\_LE": LessThanOrEqualTo -> "SCMP\_CMP\_EQ": EqualTo -> "SCMP\_CMP\_GE": GreaterThanOrEqualTo -> "SCMP\_CMP\_GT": GreaterThan -> "SCMP\_CMP\_MASKED\_EQ": MaskEqualTo - -## capabilities Security Configuration - -### Scenarios - -The capability mechanism is a security feature introduced to Linux kernel after version 2.2. The super administrator permission is controlled at a smaller granularity to prevent the root permission from being used. The root permission is divided based on different domains so that the divided permissions can be enabled or disabled separately. For details about capabilities, see the _Linux Programmer's Manual_ \([capabilities\(7\) - Linux man page](http://man7.org/linux/man-pages/man7/capabilities.7.html)\). - -```shell -man capabilities -``` - -### Usage Restrictions - -- The default capability list \(whitelist\) of the iSulad service, which is carried by common container processes by default, are as follows: - - ```text - "CAP_CHOWN", - "CAP_DAC_OVERRIDE", - "CAP_FSETID", - "CAP_FOWNER", - "CAP_MKNOD", - "CAP_NET_RAW", - "CAP_SETGID", - "CAP_SETUID", - "CAP_SETFCAP", - "CAP_SETPCAP", - "CAP_NET_BIND_SERVICE", - "CAP_SYS_CHROOT", - "CAP_KILL", - "CAP_AUDIT_WRITE" - ``` - -- Default configurations of capabilities include **CAP\_SETUID** and **CAP\_FSETID**. If the host and a container share a directory, the container can set permissions for the binary file in the shared directory. Common users on the host can use this feature to elevate privileges. The container can write **CAP\_AUDIT\_WRITE** to the host, which may cause risks. If the application scenario does not require this capability, you are advised to use **--cap-drop** to delete the capability when starting the container. -- Adding capabilities means that the container process has greater capabilities than before. In addition, more system call APIs are opened. - -### Usage Guide - -iSulad uses **--cap-add** or **--cap-drop** to add or delete specific permissions for a container. Do not add extra permissions to the container unless necessary. You are advised to remove the default but unnecessary permissions from the container. - -```shell -isula run --rm -it --cap-add all --cap-drop SYS_ADMIN rnd-dockerhub.huawei.com/official/busybox -``` - -## SELinux Security Configuration - -### Scenarios - -Security-Enhanced Linux \(SELinux\) is a Linux kernel security module that provides a mechanism for supporting access control security policies. Through Multi-Category Security \(MCS\), iSulad labels processes in containers to control containers' access to resources, reducing privilege escalation risks and preventing further damage. - -### Usage Restrictions - -- Ensure that SELinux is enabled for the host and daemon \(the **selinux-enabled** field in the **/etc/isulad/daemon.json** file is set to **true** or **--selinux-enabled** is added to command line parameters\). -- Ensure that a proper SELinux policy has been configured on the host. container-selinux is recommended. -- The introduction of SELinux affects the performance. Therefore, evaluate the scenario before setting SELinux. Enable the SELinux function for the daemon and set the SELinux configuration in the container only when necessary. -- When you configure labels for a mounted volume, the source directory cannot be a subdirectory of **/**, **/usr**, **/etc**, **/tmp**, **/home**, **/run**, **/var**, **/root**, or **/usr**. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- iSulad does not support labeling the container file system. To ensure that the container file system and configuration directory are labeled with the container access permission, run the **chcon** command to label them. ->- If SELinux access control is enabled for iSulad, you are advised to add a label to the **/var/lib/isulad** directory before starting daemon. Files and folders generated in the directory during container creation inherit the label by default. For example: -> -> ```shell -> chcon -R system_u:object_r:container_file_t:s0 /var/lib/isulad -> ``` - -### Usage Guide - -- Enable SELinux for daemon. - - ```shell - isulad --selinux-enabled - ``` - -- Configure SELinux security context labels during container startup. - - **--security-opt="label=user:USER"**: Set the label user for the container. - - **--security-opt="label=role:ROLE"**: Set the label role for the container. - - **--security-opt="label=type:TYPE"**: Set the label type for the container. - - **--security-opt="label=level:LEVEL"**: Set the label level for the container. - - **--security-opt="label=disable"**: Disable the SELinux configuration for the container. - - ```shell - $ isula run -itd --security-opt label=type:container_t --security-opt label=level:s0:c1,c2 rnd-dockerhub.huawei.com/official/centos - 9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370 - ``` - -- Add the selinux label to a mounted volume \(**z** indicates the shared mode\). - - ```shell - $ isula run -itd -v /test:/test:z rnd-dockerhub.huawei.com/official/centos - 9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370 - - $ ls -Z /test - system_u:object_r:container_file_t:s0 file - ``` diff --git a/docs/en/docs/Container/shared-memory-channels.md b/docs/en/docs/Container/shared-memory-channels.md deleted file mode 100644 index f00335a8fe96cb4b9e08c181566800601a15d63a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/shared-memory-channels.md +++ /dev/null @@ -1,59 +0,0 @@ -# Shared Memory Channels - -- [Shared Memory Channels](#shared-memory-channels) - - -## Function Description - -System containers enable the communication between container and host processes through shared memory. You can set the **--host-channel** parameter when creating a container to allow the host to share the same tmpfs with the container so that they can communicate with each other. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--host-channel

-
  • Variable of the string type. Its format is as follows:
    <host path>:<container path>:<rw/ro>:<size limit>
    -
  • The parameter is described as follows:

    <host path>: path to which tmpfs is mounted on the host, which must be an absolute path.

    -

    <container path>: path to which tmpfs is mounted in a container, which must be an absolute path.

    -

    <rw/ro>: permissions on the file system mounted to the container. The value can only be rw (read and write) or ro (read only). The default value is rw.

    -

    <size limit>: maximum size used by the mounted tmpfs. The minimum value is one 4 KB physical page, and the maximum value is half of the total physical memory in the system. The default value is 64MB.

    -
-
- -## Constraints - -- The lifecycle of tmpfs mounted on the host starts from the container startup to the container deletion. After a container is deleted and its occupied space is released, the space is removed. -- When a container is deleted, the path to which tmpfs is mounted on the host is deleted. Therefore, an existing directory on the host cannot be used as the mount path. -- To ensure that processes running by non-root users on the host can communicate with containers, the permission for tmpfs mounted on the host is 1777. - -## Example - -Specify the **--host-channel** parameter when creating a container. - -``` -[root@localhost ~]# isula run --rm -it --host-channel /testdir:/testdir:rw:32M --system-container --external-rootfs /root/myrootfs none init -root@3b947668eb54:/# dd if=/dev/zero of=/testdir/test.file bs=1024 count=64K -dd: error writing '/testdir/test.file': No space left on device -32769+0 records in -32768+0 records out -33554432 bytes (34 MB, 32 MiB) copied, 0.0766899 s, 438 MB/s -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->- If **--host-channel** is used for size limit, the file size is constrained by the memory limit in the container. \(The OOM error may occur when the memory usage reaches the upper limit.\) ->- If a user creates a shared file on the host, the file size is not constrained by the memory limit in the container. ->- If you need to create a shared file in the container and the service is memory-intensive, you can add the value of **--host-channel** to the original value of the container memory limit, eliminating the impact. - diff --git a/docs/en/docs/Container/specifying-rootfs-to-create-a-container.md b/docs/en/docs/Container/specifying-rootfs-to-create-a-container.md deleted file mode 100644 index 6a2dbfae7a81e4ef6dafbb5ce1a951e067c93d5d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/specifying-rootfs-to-create-a-container.md +++ /dev/null @@ -1,46 +0,0 @@ -# Specifying Rootfs to Create a Container - -## Function Description - -Different from a common container that needs to be started by specifying a container image, a system container is started by specifying a local root file system \(rootfs\) using the **--external-rootfs** parameter. The rootfs contains the operating system environment on which the container depends during running. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--external-rootfs

-
  • Variable of the string type.
  • Absolute path in the root file system of the container, that is, the path of the rootfs.
-
- -## Constraints - -- The rootfs directory specified using the **--external-rootfs** parameter must be an absolute path. -- The rootfs directory specified using the **--external-rootfs** parameter must be a complete OS environment including **systemd** package. Otherwise, the container fails to be started. -- When a container is deleted, the rootfs directory specified using **--external-rootfs** is not deleted. -- Containers based on an ARM rootfs cannot run in the x86 environment. Containers based on an x86 rootfs cannot run in the ARM environment. -- You are advised not to start multiple container instances in the same rootfs. That is, one rootfs is used by only one container instance that is in the lifecycle. - -## Example - -Assuming the local rootfs path is **/root/myrootfs**, run the following command to start a system container: - -``` -# isula run -tid --system-container --external-rootfs /root/myrootfs none init -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The rootfs is a user-defined file system. Prepare it by yourself. For example, a rootfs is generated after the TAR package of a container image is decompressed. - diff --git a/docs/en/docs/Container/statistics.md b/docs/en/docs/Container/statistics.md deleted file mode 100644 index bda570d8c5a4741abd61abedcfcadf7ad012cc83..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/statistics.md +++ /dev/null @@ -1,110 +0,0 @@ -# Statistics - -- [Statistics](#statistics) - - [events](#events) - - [info](#info) - - [version](#version) - - -## events - -Syntax: **docker events \[**_options_**\]** - -Function: Obtains real-time events from the docker daemon. - -Parameter description: - -**--since=""**: Displays events generated after the specified timestamp. - -**--until=""**: Displays events generated before the specified timestamp. - -Example: - -After the **docker events** command is executed, a container is created and started by running the **docker run** command. create and start events are output. - -``` -$ sudo docker events -2019-08-28T16:23:09.338838795+08:00 container create 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu) -2019-08-28T16:23:09.339909205+08:00 container attach 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu) -2019-08-28T16:23:09.397717518+08:00 network connect e2e20f52662f1ee2b01545da3b02e5ec7ff9c85adf688dce89a9eb73661dedaa (container=53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e, name=bridge, type=bridge) -2019-08-28T16:23:09.922224724+08:00 container start 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu) -2019-08-28T16:23:09.924121158+08:00 container resize 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (height=48, image=busybox:latest, name=eager_wu, width=210) -``` - -   - -## info - -Syntax: **docker info** - -Function: Displays the Docker system information, including the number of containers, number of images, image storage driver, container execution driver, kernel version, and host OS version. - -Parameter description: none. - -Example: - -``` -$ sudo docker info -Containers: 4 - Running: 3 - Paused: 0 - Stopped: 1 -Images: 45 -Server Version: 18.09.0 -Storage Driver: devicemapper - Pool Name: docker-thinpool - Pool Blocksize: 524.3kB - Base Device Size: 10.74GB - Backing Filesystem: ext4 - Udev Sync Supported: true - Data Space Used: 11GB - Data Space Total: 51GB - Data Space Available: 39.99GB - Metadata Space Used: 5.083MB - Metadata Space Total: 532.7MB - Metadata Space Available: 527.6MB - Thin Pool Minimum Free Space: 5.1GB - Deferred Removal Enabled: true - Deferred Deletion Enabled: true - Deferred Deleted Device Count: 0 -...... -``` - -   - -## version - -Syntax: **docker version** - -Function: Displays the Docker version information, including the client version, server version, Go version, and OS and Arch information. - -Parameter description: none. - -Example: - -``` -$ sudo docker version -Client: - Version: 18.09.0 - EulerVersion: 18.09.0.48 - API version: 1.39 - Go version: go1.11 - Git commit: cbf6283 - Built: Mon Apr 1 00:00:00 2019 - OS/Arch: linux/arm64 - Experimental: false - -Server: - Engine: - Version: 18.09.0 - EulerVersion: 18.09.0.48 - API version: 1.39 (minimum version 1.12) - Go version: go1.11 - Git commit: cbf6283 - Built: Mon Apr 1 00:00:00 2019 - OS/Arch: linux/arm64 - Experimental: false -``` - -   - diff --git a/docs/en/docs/Container/supporting-oci-hooks.md b/docs/en/docs/Container/supporting-oci-hooks.md deleted file mode 100644 index 76ce5759967a23f6ded80ac49bfc30c429b402a8..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/supporting-oci-hooks.md +++ /dev/null @@ -1,87 +0,0 @@ -# Supporting OCI hooks - -- [Supporting OCI hooks](#supporting-oci-hooks) - - [Description](#description) - - [APIs](#apis) - - [Usage Restrictions](#usage-restrictions) - - -## Description - -The running of standard OCI hooks within the lifecycle of a container is supported. There are three types of standard hooks: - -- prestart hook: executed after the **isula start** command is executed and before the init process of the container is started. -- poststart hook: executed after the init process is started and before the **isula start** command is returned. -- poststop hook: executed after the container is stopped and before the stop command is returned. - -The configuration format specifications of OCI hooks are as follows: - -- **path**: \(Mandatory\) The value must be a character string and must be an absolute path. The specified file must have the execute permission. -- **args**: \(Optional\) The value must be a character string array. The syntax is the same as that of **args** in **execv**. -- **env**: \(Optional\) The value must be a character string array. The syntax is the same as that of environment variables. The content is a key-value pair, for example, **PATH=/usr/bin**. -- **timeout**: \(Optional\) The value must be an integer that is greater than 0. It indicates the timeout interval for hook execution. If the running time of the hook process exceeds the configured time, the hook process is killed. - -The hook configuration is in JSON format and usually stored in a file ended with **json**. An example is as follows: - -``` -{ - "prestart": [ - { - "path": "/usr/bin/echo", - "args": ["arg1", "arg2"], - "env": [ "key1=value1"], - "timeout": 30 - }, - { - "path": "/usr/bin/ls", - "args": ["/tmp"] - } - ], - "poststart": [ - { - "path": "/usr/bin/ls", - "args": ["/tmp"], - "timeout": 5 - } - ], - "poststop": [ - { - "path": "/tmp/cleanup.sh", - "args": ["cleanup.sh", "-f"] - } - ] -} -``` - -## APIs - -Both iSulad and iSula provide the hook APIs. The default hook configurations provided by iSulad apply to all containers. The hook APIs provided by iSula apply only to the currently created container. - -The default OCI hook configurations provided by iSulad are as follows: - -- Set the configuration item **hook-spec** in the **/etc/isulad/daemon.json** configuration file to specify the path of the hook configuration file. Example: **"hook-spec": "/etc/default/isulad/hooks/default.json"** -- Use the **isulad --hook-spec** parameter to set the path of the hook configuration file. - -The OCI hook configurations provided by iSula are as follows: - -- **isula create --hook-spec**: specifies the path of the hook configuration file in JSON format. -- **isula run --hook-spec**: specifies the path of the hook configuration file in JSON format. - -The configuration for **run** takes effect in the creation phase. - -## Usage Restrictions - -- The path specified by **hook-spec** must be an absolute path. -- The file specified by **hook-spec** must exist. -- The path specified by **hook-spec** must contain a common text file in JSON format. -- The file specified by **hook-spec** cannot exceed 10 MB. -- **path** configured for hooks must be an absolute path. -- The file that is designated by **path** configured for hooks must exist. -- The file that is designated by **path** configured for hooks must have the execute permission. -- The owner of the file that is designated by **path** configured for hooks must be user **root**. -- Only user **root** has the write permission on the file that is designated by **path** configured for hooks. -- The value of **timeout** configured for hooks must be greater than **0**. - -    - - diff --git a/docs/en/docs/Container/system-container.md b/docs/en/docs/Container/system-container.md deleted file mode 100644 index 37861583d3becff0c5d5cb8134bcf5d44af561e7..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/system-container.md +++ /dev/null @@ -1,3 +0,0 @@ -# System Container - -System containers are used for heavyweight applications and cloud-based services in scenarios with re-computing, high performance, and high concurrency. Compared with the VM technology, system containers can directly inherit physical machine features and has better performance and less overhead. In addition, system containers can be allocated more computing units of limited resources, reducing costs. Therefore, system containers can be used to build differentiated product competitiveness and provide computing unit instances with higher computing density, lower price, and better performance. diff --git a/docs/en/docs/Container/uninstallation.md b/docs/en/docs/Container/uninstallation.md deleted file mode 100644 index 192e53f5508cc82384d3d474111029f0d67cc802..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/uninstallation.md +++ /dev/null @@ -1,24 +0,0 @@ -# Uninstallation - -To uninstall iSulad, perform the following operations: - -1. Uninstall iSulad and its dependent software packages. - - If the **yum** command is used to install iSulad, run the following command to uninstall iSulad: - - ``` - $ sudo yum remove iSulad - ``` - - - If the **rpm** command is used to install iSulad, uninstall iSulad and its dependent software packages. Run the following command to uninstall an RPM package. - - ``` - sudo rpm -e iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm - ``` - -2. Images, containers, volumes, and related configuration files are not automatically deleted. The reference command is as follows: - - ``` - $ sudo rm -rf /var/lib/iSulad - ``` - - diff --git a/docs/en/docs/Container/upgrade-methods.md b/docs/en/docs/Container/upgrade-methods.md deleted file mode 100644 index 5294263ed82402538f59fb9cfe43f950e9b367e8..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/upgrade-methods.md +++ /dev/null @@ -1,21 +0,0 @@ -# Upgrade Methods - -- For an upgrade between patch versions of a major version, for example, upgrading 2.x.x to 2.x.x, run the following command: - - ``` - $ sudo yum update -y iSulad - ``` - -- For an upgrade between major versions, for example, upgrading 1.x.x to 2.x.x, save the current configuration file **/etc/isulad/daemon.json**, uninstall the existing iSulad software package, install the iSulad software package to be upgraded, and restore the configuration file. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->- You can run the **sudo rpm -qa |grep iSulad** or **isula version** command to check the iSulad version. ->- If you want to manually perform upgrade between patch versions of a major version, run the following command to download the RPM packages of iSulad and all its dependent libraries: -> ``` -> $ sudo rpm -Uhv iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm -> ``` -> If the upgrade fails, run the following command to forcibly perform the upgrade: -> ``` -> $ sudo rpm -Uhv --force iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm -> ``` - diff --git a/docs/en/docs/Container/usage-guide.md b/docs/en/docs/Container/usage-guide.md deleted file mode 100644 index df5b305b523c1d4c60c33fd4d270dc58d613be0a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/usage-guide.md +++ /dev/null @@ -1,19 +0,0 @@ -# Usage Guide - - -System container functions are enhanced based on the iSula container engine. The container management function and the command format of the function provided by system containers are the same as those provided by the iSula container engine. - -The following sections describe how to use the enhanced functions provided by system containers. For details about other command operations, see [iSulad Container Engine](#isulad-container-engine.md#EN-US_TOPIC_0184808037). - -The system container functions involve only the **isula create/run** command. Unless otherwise specified, this command is used for all functions. The command format is as follows: - -``` -isula create/run [OPTIONS] [COMMAND] [ARG...] -``` - -In the preceding format: - -- **OPTIONS**: one or more command parameters. For details about supported parameters, see [iSulad Container Engine](#isulad-container-engine.md#EN-US_TOPIC_0184808037) \> [Appendix](#appendix.md#EN-US_TOPIC_0184808158) \> [Command Line Parameters](#command-line-parameters.md#EN-US_TOPIC_0189976936). -- **COMMAND**: command executed after a system container is started. -- **ARG**: parameter corresponding to the command executed after a system container is started. - diff --git a/docs/en/docs/Container/using-systemd-to-start-a-container.md b/docs/en/docs/Container/using-systemd-to-start-a-container.md deleted file mode 100644 index 69dc5d7bf4c77b31cf594c51af66ca0664f9fe4d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/using-systemd-to-start-a-container.md +++ /dev/null @@ -1,87 +0,0 @@ -# Using systemd to Start a Container - -- [Using systemd to Start a Container](#using-systemd-to-start-a-container) - -## Function Description - -The init process started in system containers differs from that in common containers. Common containers cannot start system services through systemd. However, system containers have this capability. You can enable the systemd service by specifying the **--system-container** parameter when starting a system container. - -## Parameter Description - - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--system-container

-
  • The value is of a Boolean data type and can be true or false. The default value is true.
  • Specifies whether it is a system container. This function must be enabled.
-
- -## Constraints - -- The systemd service needs to call some special system APIs, including mount, umount2, unshare, reboot, and name\_to\_handle\_at. Therefore, permissions to call the preceding APIs are enabled for system containers when the privileged container tag is disabled. -- All system containers are started by the init process. The init process does not respond to the SIGTERM signal which indicates normal exit. By default, the **stop** command forcibly kills the container 10 seconds later. If you need a quicker stop, you can manually specify the timeout duration of the **stop** command. -- **--system-container** must be used together with **--external-rootfs**. -- Various services can run in a system container. The **systemctl** command is used to manage the service starting and stopping. Services may depend on each other. As a result, when an exception occurs, some service processes are in the D or Z state so that the container cannot exit properly. -- Some service processes in a system container may affect other operation results. For example, if the NetworkManager service is running in the container, adding NICs to the container may be affected \(the NICs are successfully added but then stopped by the NetworkManger\), resulting in unexpected results. -- Currently, system containers and hosts cannot be isolated by using udev events. Therefore, the **fstab** file cannot be configured. -- The systemd service may conflict with the cgconfig service provided by libcgroup. You are advised to delete the libcgroup-related packages from a container or set **Delegate** of the cgconfig service to **no**. - -## Example - -- Specify the **--system-container** and **--external-rootfs** parameters to start a system container. - - ```shell - [root@localhost ~]# isula run -tid -n systest01 --system-container --external-rootfs /root/myrootfs none init - ``` - -- After the preceding commands are executed, the container is running properly. You can run the **exec** command to access the container and view the process information. The command output indicates that the systemd service has been started. - - ```shell - [root@localhost ~]# isula exec -it systest01 bash - [root@localhost /]# ps -ef - UID PID PPID C STIME TTY TIME CMD - root 1 0 2 06:49 ? 00:00:00 init - root 14 1 2 06:49 ? 00:00:00 /usr/lib/systemd/systemd-journal - root 16 1 0 06:49 ? 00:00:00 /usr/lib/systemd/systemd-network - dbus 23 1 0 06:49 ? 00:00:00 /usr/bin/dbus-daemon --system -- - root 25 0 0 06:49 ? 00:00:00 bash - root 59 25 0 06:49 ? 00:00:00 ps –ef - ``` - -- Run the **systemctl** command in the container to check the service status. The command output indicates that the service is managed by systemd. - - ```shell - [root@localhost /]# systemctl status dbus - ● dbus.service - D-Bus System Message Bus - Loaded: loaded (/usr/lib/systemd/system/dbus.service; static; vendor preset: - disabled) - Active: active (running) since Mon 2019-07-22 06:49:38 UTC; 2min 5 - 8s ago - Docs: man:dbus-daemon(1) - Main PID: 23 (dbus-daemon) - CGroup: /system.slice/dbus.service - └─23 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidf - ile --systemd-activation --syslog-only - - Jul 22 06:49:38 localhost systemd[1]: Started D-Bus System Message Bus. - ``` - -- Run the **systemctl** command in the container to stop or start the service. The command output indicates that the service is managed by systemd. - - ```shell - [root@localhost /]# systemctl stop dbus - Warning: Stopping dbus.service, but it can still be activated by: - dbus.socket - [root@localhost /]# systemctl start dbus - ``` diff --git a/docs/en/docs/Container/writable-namespace-kernel-parameters.md b/docs/en/docs/Container/writable-namespace-kernel-parameters.md deleted file mode 100644 index 6154883b5d8133b6ab3d55f1e610064517256226..0000000000000000000000000000000000000000 --- a/docs/en/docs/Container/writable-namespace-kernel-parameters.md +++ /dev/null @@ -1,88 +0,0 @@ -# Writable Namespace Kernel Parameters - -- [Writable Namespace Kernel Parameters](#writable-namespace-kernel-parameters) - -## Function Description - -For services running in containers, such as databases, big data, and common applications, some kernel parameters need to be set and adjusted to obtain the optimal performance and reliability. The modification permission of all kernel parameters must be disabled or enabled simultaneously \(by using privileged container\). - -When the modification permission is disabled, only the --sysctl external interface is provided and parameters cannot be flexibly modified in a container. - -When the modification permission is enabled, some kernel parameters are globally valid. If some parameters are modified in a container, all programs on the host will be affected, harming security. - -System containers provide the **--ns-change-opt** parameter, which can be used to dynamically set namespace kernel parameters in a container. The parameter value can be **net** or **ipc**. - -## Parameter Description - - - - - - - - - - - -

Command

-

Parameter

-

Value Description

-

isula create/run

-

--ns-change-opt

-
  • Variable of the string type.
  • The parameter value can be net or ipc.

    net: All namespace parameters in the /proc/sys/net directory are supported.

    -

    ipc: Supported namespace parameters are as follows:

    -

    /proc/sys/kernel/msgmax

    -

    /proc/sys/kernel/msgmnb

    -

    /proc/sys/kernel/msgmni

    -

    /proc/sys/kernel/sem

    -

    /proc/sys/kernel/shmall

    -

    /proc/sys/kernel/shmmax

    -

    /proc/sys/kernel/shmmni

    -

    /proc/sys/kernel/shm_rmid_forced

    -

    /proc/sys/fs/mqueue/msg_default

    -

    /proc/sys/fs/mqueue/msg_max

    -

    /proc/sys/fs/mqueue/msgsize_default

    -

    /proc/sys/fs/mqueue/msgsize_max

    -

    /proc/sys/fs/mqueue/queues_max

    -
  • You can specify multiple namespace configurations and separate them with commas (,). For example, --ns-change-opt=net,ipc.
-
- -## Constraints - -- If both **--privileged** \(privileged container\) and **--ns-change-opt** are specified during container startup, **--ns-change-opt** does not take effect. - -## Example - -Start a container and set **--ns-change-opt** to **net**. - -```shell -[root@localhost ~]# isula run -tid --ns-change-opt net --system-container --external-rootfs /root/myrootfs none init -4bf44a42b4a14fdaf127616c90defa64b4b532b18efd15b62a71cbf99ebc12d2 -[root@localhost ~]# isula exec -it 4b mount | grep /proc/sys -proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime) -proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime) -proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime) -``` - -The mount point **/proc/sys/net** in the container has the **rw** option, indicating that the **net**-related namespace kernel parameters have the read and write permissions. - -Start another container and set **--ns-change-opt** to **ipc**. - -```shell -[root@localhost ~]# isula run -tid --ns-change-opt ipc --system-container --external-rootfs /root/myrootfs none init -c62e5e5686d390500dab2fa76b6c44f5f8da383a4cbbeac12cfada1b07d6c47f -[root@localhost ~]# isula exec -it c6 mount | grep /proc/sys -proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime) -proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/shmmax type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/shmmni type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/shmall type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/shm_rmid_forced type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/msgmax type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/msgmni type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/msgmnb type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/kernel/sem type proc (rw,nosuid,nodev,noexec,relatime) -proc on /proc/sys/fs/mqueue type proc (rw,nosuid,nodev,noexec,relatime) -``` - -The mount point information of **ipc**-related kernel parameters in the container contains the **rw** option, indicating that the **ipc**-related namespace kernel parameters have the read and write permissions. diff --git a/docs/en/docs/DPU-OS/dpu-os-background-and-requirements.md b/docs/en/docs/DPU-OS/dpu-os-background-and-requirements.md deleted file mode 100644 index 7995cc738b125ced2a435f44d69586006f5dacbe..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPU-OS/dpu-os-background-and-requirements.md +++ /dev/null @@ -1,67 +0,0 @@ -# Background and Requirements - -## Overview - -In data center and cloud scenarios, Moore's Law fails. The CPU computing power growth rate of general processing units slows down, while the network I/O speed and performance keep increasing. The processing capability of current general-purpose processors cannot keep up with the I/O processing requirements of networks and drives. In traditional data centers, more and more general-purpose CPU computing power is consumed by I/O and management plane. Such resource loss is called "datacenter tax". According to AWS statistics, datacenter taxes may account for more than 30% of the computing power of data centers, and even more in specific scenarios. - -The data processing unit (DPU) is developed to free the computing resources from the datacenter taxes. The management plane, network, storage, and security capabilities are offloaded to the DPU for processing acceleration, reducing costs and improving efficiency. Currently, mainstream cloud vendors, such as AWS, Alibaba Cloud, and Huawei Cloud, offload the management plane and related data planes to self-developed processors, achieving 100% sales of data center computing resources. - -Currently, DPU is undergoing rapid development. Cloud and big data scenarios have strong demands for DPUs. Many DPU startups have launched different DPU products. To meet such requirements, cloud and big data vendors need to consider how to integrate and use different DPU products. DPU vendors also need to adapt device drivers to user OSs. openEuler builds DPU-OS to solve the problem of DPU adaptation for DPU vendors and users. In addition, the OS on the DPU is used to accelerate some services. Therefore, the performance of DPU-OS is optimized and accelerated. DPU-related acceleration capabilities can be built based on openEuler and embedded in DPU-OS to build a DPU software ecosystem. - -## DPU-OS Requirement Analysis and Design - -### DPU Status Quo and OS Requirements - -DPUs typically have the following characteristics and problems: - -* Limited general processing capabilities and resources - - Currently, the DPU is still in the early stage of development, and the hardware is still evolving. In addition, the current hardware specifications are low due to the limitation of the DPU power supply. Mainstream DPUs often have 8 to 24 general-purpose processor cores with weak single-core capability and limited amount of memory, usually 16 to 32 GB. And the local storage space of a DPU ranges from dozens of GB to hundreds of GB. These restrictions need to be taken into consideration during the design of an OS that runs on DPUs. - -* Various DPU OS Installation Modes - - The variety of DPU vendors and products result in different installation and deployment modes of OSs, including PXE network installation, USB flash drive installation, and custom installation (using an image delivered by the host). - -* DPU Performance Requirements - - DPUs are required to have strong performance in their application scenarios. Compared with common server OSs, a DPU OS may have to support some specific features, such as vDPU for device passthrough and hot migration, vendor-specific driver support, seamless offload of DPU processes, custom optimized user mode data plane acceleration tools (for example, DPDK, SPDK, and OVS), and DPU management and monitoring tools. - -Based on the preceding DPU status, the requirements for DPU-OS are as follows: - -* Lightweight installation package - - The openEuler system image is tailored to reduce the space occupied by unnecessary software packages. System services are optimized to reduce the resource overhead. - -* Tailoring Configurations and Tool Support - - Tailoring configuration and tool support are provided. Users and DPU vendors can customize the tailoring based on their requirements. openEuler provides ISO reference implementation. - -* Customized kernel and system for ultimate performance - - The customized kernel and related drivers provide competitive DPU kernel features. The customized acceleration components enable DPU hardware acceleration. The optimized system configurations deliver better performance. DPU management and control tools facilitate unified management. - -### DPU-OS Design - -**Figure 1** DPU-OS overall design - -![dpuos-arch](./figures/dpuos-arch.png) - -As shown in Figure 1, DPU-OS contains five layers: - -* Kernel layer: The kernel configurations are customized to be lightweight, with unnecessary kernel features and modules removed. Specific kernel features are enabled to provide high-performance DPU kernel capabilities. - -* Driver layer: The native openEuler drivers are tailored to keep the minimum collection. Some DPU vendor drivers are integrated to natively support related DPU hardware. - -* System configuration layer: sysctl and proc are configured for optimal performance of DPU-related services. - -* Peripheral package layer: openEuler peripheral packages are tailored to keep the minimum collection. A set of DPU-related customization tools are provided. - -* System service layer: The native service startup items are optimized to reduce unnecessary system services and minimize the system running overhead. - -The five-layer design makes DPU-OS lightweight with ultimate performance. This solution is a long-term design and strongly depends on the software and hardware ecosystem of DPUs. In the first phase, this solution is tailored based on openEuler imageTailor. - -For details about how to tailor DPU-OS, see [DPU-OS Tailoring Guide](./dpu-os-tailoring-guide.md). For details about how to verify and deploy DPU-OS, see [DPU-OS Deployment Verification Guide](./verification-and-deployment.md). - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> Currently, DPU-OS is tailored with imageTailor based on the existing openEuler kernel and peripheral packages to provide a lightweight OS installation image. In the future, related kernel and peripheral package features can be developed and integrated based on actual requirements. diff --git a/docs/en/docs/DPU-OS/dpu-os-tailoring-guide.md b/docs/en/docs/DPU-OS/dpu-os-tailoring-guide.md deleted file mode 100644 index c79768919ed12fbc5ec6cf4144b864e8e1fd8244..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPU-OS/dpu-os-tailoring-guide.md +++ /dev/null @@ -1,65 +0,0 @@ -# DPU-OS Tailoring Guide - -This section describes how to use imageTailor to obtain the DPU-OS installation image based on the dpuos configuration file of the [dpu-utilities repository](https://gitee.com/openeuler/dpu-utilities/tree/master/dpuos). The procedure is as follows. - -## Preparing imageTailor and Required RPM Packages - -Install the imageTailor tool and prepare the RPM packages required for tailoring. For details, see the [imageTailor User Guide](../TailorCustom/imageTailor-user-guide.md). - -You can use the installation image provided by openEuler as the RPM package source for image tailoring. **openEuler-22.03-LTS-everything-debug-aarch64-dvd.iso** have a complete collection of RPM packages but the image is large in size. You can use the RPM packages in **openEuler-22.03-LTS-aarch64-dvd.iso** and **install-scripts.noarch**. - -You can obtain **install-scripts.noarch** from the everything package or download it using `yum`. - -```bash -yum install -y --downloadonly --downloaddir=./ install-scripts -``` - -## Copying DPU-OS Configuration Files - -The imageTailor tool is installed in the **/opt/imageTailor** directory by default. Run the following command to copy the DPU-OS configuration files to the corresponding directories (select the directories of the target architecture): Currently, the DPU-OS tailoring configuration library supports the x86_64 and AArch64 architectures. - -```bash -cp -rf custom/cfg_dpuos /opt/imageTailor/custom -cp -rf kiwi/minios/cfg_dpuos /opt/imageTailor/kiwi/minios/cfg_dpuos -``` - -## Modifying Other Configuration Files - -* Add the **dpuos** configuration to the **kiwi/eulerkiwi/product.conf** file. - -```text -dpuos PANGEA EMBEDDED DISK GRUB2 install_mode=install install_media=CD install_repo=CD selinux=0 -``` - -* Add the **dpuos** configuration to the **kiwi/eulerkiwi/minios.conf** file. - -```text -dpuos kiwi/minios/cfg_dpuos yes -``` - -* Add the **dpuos** configuration to the **repos/RepositoryRule.conf** file. - -```text -dpuos 1 rpm-dir euler_base -``` - -## Setting a Password - -Go to the subdirectories of **/opt/imageTailor** and change the passwords in the following files: - -* `custom/cfg_dpuos/usr_file/etc/default/grub` - -* `custom/cfg_dpuos/rpm.conf` - -* `kiwi/minios/cfg_dpuos/rpm.conf` - -For details about how to generate and modify password, see [Configuring Initial Passwords](../TailorCustom/imageTailor-user-guide.md#configuring-initial-passwords) in the _openEuler imageTailor User Guide_. - -## Running the Tailoring Command - -Run the following command to tailor the ISO file. The tailored ISO file is stored in the **/opt/imageTailor/result** directory. - -```bash -cd /opt/imageTailor -./mkdliso -p dpuos -c custom/cfg_dpuos --sec --minios force -``` diff --git a/docs/en/docs/DPU-OS/overview.md b/docs/en/docs/DPU-OS/overview.md deleted file mode 100644 index d2b955531051f2e52b194f54ad63d1e753486765..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPU-OS/overview.md +++ /dev/null @@ -1,11 +0,0 @@ -# Overview - -This document introduces the background requirements and overall design of DPU-OS, and describes how to create, deploy, and verify a DPU-OS image on openEuler. DPU-OS is built based on the openEuler ecosystem. It delivers ultimate performance and provides an implementation reference of DPU OSs for DPU scenarios and users. - -This article is intended for community developers, DPU vendors, and users who use the openEuler OS and want to learn and use DPUs. Users must: - -- Know basic Linux operations. - -- Be familiar with the basic knowledge and operations related to build and deployment on Linux. - -- Understand openEuler ImageTailor. diff --git a/docs/en/docs/DPU-OS/verification-and-deployment.md b/docs/en/docs/DPU-OS/verification-and-deployment.md deleted file mode 100644 index 2c1581d72014721571a2f965c1b0d953228fd931..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPU-OS/verification-and-deployment.md +++ /dev/null @@ -1,38 +0,0 @@ -# Verification and Deployment - -After the DPU-OS image is created, you can install and deploy DPU-OS for verification. Currently, the DPU hardware is not mature enough. Therefore, you can use VirtualBox to start a VM for deployment and verification. - -## Deploying DPU-OS on VirtualBox - -This section describes how to install and deploy DPU-OS on VirtualBox. - -### Verification Preparation - -Before deploying DPU-OS, make the following preparations: - -- Obtain the DPU-OS ISO. -- Prepare a host machine with VirtualBox installed. - -### Initial Installation and Startup - -#### Creating a VM - -Creating a VM on VirtualBox. - -- Select the VM configuration. Two CPUs and at least 4 GB memory are recommended. - -- Create a VM disk. At least 60 GB is recommended. - -- In the system extended attributes, enable EFI. - -- In the storage settings, select the local DPU-OS ISO file as the CD-ROM file. - -- Other network or display settings can be custom. - -#### Starting the VM - -Start the created VM and select **Install from ISO** to install DPU-OS. After DPU-OS is automatically installed, the VM restarts. - -Select **Boot From Local Disk** to boot into DPU-OS. The password is the one specified when the DPU-OS image is created. - -After the preceding steps are performed, the local deployment of the DPU-OS is verified. diff --git a/docs/en/docs/DPUOffload/imperceptible-container-management-plane-offload.md b/docs/en/docs/DPUOffload/imperceptible-container-management-plane-offload.md deleted file mode 100644 index dc6425474e43fdbc924921f3a9c73c4cfe01333e..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPUOffload/imperceptible-container-management-plane-offload.md +++ /dev/null @@ -1,36 +0,0 @@ -# Imperceptible Container Management Plane Offload - -## Overview - -Moore's law ceases to apply in data center and cloud scenarios. The CPU computing power growth rate of general processing units is slowing down, while the network I/O speed and performance keep increasing. As a result, the processing capability of current general-purpose processors cannot meet the I/O processing requirements of the network and drives. In traditional data centers, more and more general-purpose CPU computing power is occupied by I/O and management planes. This part of resource loss is called data center tax. According to AWS statistics, the data center tax may account for more than 30% of the computing power of the data center. - -The data processing unit (DPU) is developed to release the computing resources from the host CPU. The management plane, network, storage, and security capabilities are offloaded to DPUs for acceleration, reducing costs and improving efficiency. Mainstream cloud vendors, such as AWS, Alibaba Cloud, and Huawei Cloud, use self-developed processors to offload the management plane and related data plane, achieving 100% utilization of data center computing resources. - -The management plane processes can be offloaded to the DPU by splitting the component source code. The source code is split into two parts that run independently on the host and DPU based on the function logic. In this way, the component is offloaded. However, this method has the following problems: - -1. The software compatibility of the component is affected. You need to maintain the component and related patches in subsequent version upgrades, which increases the maintenance workload. -2. The offload cannot be inherited by other components. You need to split each component based on code logic analysis. - -To solve these problems, openEuler introduces imperceptible DPU offload. The abstraction layer provided by the OS shields the cross-host access differences between the host and DPU, and enables service processes to be offloaded to the DPU with virtually zero modification. This part of work at the common layer of the OS and is irrelevant to upper-layer services. Other services can also inherit the offload to DPU. - -## Architecture - -### Imperceptible Container Management Plane DPU Offload Architecture - -**Figure 1** Imperceptible Container Management Plane DPU Offload Architecture - -![offload-arch](./figures/offload-arch.png) - -As shown in Figure 1, after the container management plane is offloaded, management processes such as dockerd and kubelet run on the DPU side, and container processes run on the host. The interaction between processes is ensured by the system layer. - -* Communication layer: DPUs and hosts can communicate with each other through PCIe interfaces or networks. A communication interface layer is provided based on underlying physical connections to provide communication interfaces for upper-layer services. - -* qtfs kernel shared file system: The container management plane components kubelet and dockerd interact with container processes through file systems. Management plane tools need to prepare data plane paths to rootfs and volume for container processes. In addition, the proc and cgroup file systems need to be used to control and monitor the resources and status of container processes. For details about qtfs, see [qtfs Shared File System Introduction and Usage](./qtfs-architecture-and-usage.md). - -* User-mode offload environment: You need to use qtfs to prepare the runtime environment for the offloaded management plane, and remotely mount the container management and runtime directories of the host to the DPU. System management file systems such as proc, sys, and cgroup need to be mounted. To prevent damage to the native system functions of the DPU, the preceding mounting operations are performed in the chroot environment. In addition, the management plane (running on the DPU) and container processes (running on the host) have invoking relationships. The rexec remote binary execution tool needs to be used to provide corresponding functions. - -For details about how to offload container management plane, see the [Deployment Guide](./offload-deployment-guide.md). - -> ![](public_sys-resources/icon-note.gif) **NOTE**: -> -> In this user guide, modifications are performed to the container management plane components and the rexec tool of a specific version. You can modify other versions based on the actual execution environment. The patch provided in this document is for verification only and is not for commercial use. diff --git a/docs/en/docs/DPUOffload/libvirt-direct-connection-aggregation-environment-establishment.md b/docs/en/docs/DPUOffload/libvirt-direct-connection-aggregation-environment-establishment.md deleted file mode 100644 index 118467766ef52f4d4d6a2869cc0a3fc3ee6ad1e3..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPUOffload/libvirt-direct-connection-aggregation-environment-establishment.md +++ /dev/null @@ -1,356 +0,0 @@ -# 1 Hardware Preparation - -## Test Mode - -Prepare two physical machines (VMs have not been tested) that can communicate with each other. - -One physical machine functions as the DPU, and the other functions as the host. In this document, DPU and HOST refer to the two physical machines. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->In the test mode, network ports are exposed without connection authentication, which is risky and should be used only for internal tests and verification. Do not use this mode in the production environment. - -## vsock mode - -The DPU and HOST are required. The DPU must be able to provide vsock communication through virtio. - -This document describes only the test mode usage. - -# 2 libvirt offload architecture - -![arch](./figures/arch.png) - -# 3 Environment Setup - -## 3.1 qtfs File System Deployment - -For details, visit . - -To establish a qtfs connection, you need to disable the firewall. - -## 3.2 Deploying the udsproxyd Service - -### 3.2.1 Introduction - -udsproxyd is a cross-host Unix domain socket (UDS) proxy service, which needs to be deployed on both the host and DPU. The udsproxyd components on the host and dpu are peers. They implement seamless UDS communication between the host and DPU, which means that if two processes can communicate with each other through UDSs on the same host, they can do the same between the host and DPU. The code of the processes does not need to be modified, only that the client process needs to run with the **LD_PRELOAD=libudsproxy.so** environment variable. As a cross-host Unix socket service, udsproxyd can be used by running with `LD_PRELOAD=libudsproxy.so`. With the support of qtfs, udsproxyd can also be used transparently. You need to configure the allowlist in advance. The specific operations are described later. - -### 3.2.2 Deploying udsproxyd - -Build udsproxyd in the dpu-utilities project: - -```bash -cd qtfs/ipc -make -j UDS_TEST_MODE=1 && make install -``` - -The engine service on the qtfs server has incorporated the udsproxyd feature. You do not need to manually start udsproxyd if the qtfs server is deployed. However, you need to start udsproxyd on the client by running the following command: - -```bash -nohup /usr/bin/udsproxyd 2>&1 & -``` - -Parameters: - -```bash -thread num: number of threads. Currently, only one thread is supported. -addr: IP address of the host. -port: Port used on the host. -peer addr: IP address of the udsproxyd peer. -peer port: port used on the udsproxyd peer. -``` - -Example: - -```bash -nohup /usr/bin/udsproxyd 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 & -``` - -If the qtfs engine service is not started, you can start udsproxyd on the server to test udsproxyd separately. Run the following command: - -```bash -nohup /usr/bin/udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 & -``` - -### 3.2.3 Using udsproxyd - -#### 3.2.3.1 Using udsproxyd Independently - -When starting the client process of the Unix socket application that uses the UDS service, add the **LD_PRELOAD=libudsproxy.so** environment variable to intercept the **connect** API of glibc for UDS interconnection. In the libvirt offload scenario, you can copy **libudsproxy.so**, which will be used by the libvirtd service, to the **/usr/lib64** directory in the chroot directory of libvirt. - -#### 3.2.3.2 Using the udsproxyd Service Transparently - -Configure the UDS service allowlist for qtfs. The allowlist is the sock file address bound to the Unix socket server. For example, the files of the Unix socket server created by the libvirt VM are in the **/var/lib/libvirt** directory. In this case, add the directory path to the allowlist in either of the following ways: - -* Load the allowlist by using the `qtcfg` utility. First compile the utility in **qtfs/qtinfo**. - -Run the following command on the qtfs client: - -```bash -make role=client -make install -``` - -Run the following command on the qtfs server: - -```bash -make role=server -make install -``` - -After `qtcfg` is installed automatically, run `qtcfg` to configure the allowlist. Assume that **/var/lib/libvirt** needs to be added to the allowlist: - -```bash -qtcfg -x /var/lib/libvirt/ -``` - -Query the allowlist: - -```bash -qtcfg -z -``` - -Delete an allowlist entry: - -```bash -qtcfg -y 0 -``` - -The parameter is the index number listed when you query the allowlist. - -* Add an allowlist entry through the configuration file. The configuration file needs to be set before the qtfs or qtfs_server kernel module is loaded. The allowlist is loaded when the kernel modules are initialized. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The allowlist prevents irrelevant Unix sockets from establishing remote connections, causing errors or wasting resources. You are advised to set the allowlist as precisely as possible. For example, in this document, **/var/lib/libvirt** is set in the libvirt scenario. It would be risky to directly add **/var/lib**, **/var**, or the root directory. - -## 3.3 rexec Service Deployment - -### 3.3.1 Introduction - -rexec is a remote execution component developed using the C language. It consists of the rexec client and rexec server. The server is a daemon process, and the client is a binary file. After being started, the client establishes a UDS connection with the server using the udsproxyd service, and the server daemon process starts a specified program on the server machine. During libvirt virtualization offload, libvirtd is offloaded to the DPU. When libvirtd needs to start the QEMU process on the HOST, the rexec client is invoked to remotely start the process. - -### 3.3.2 Deploying rexec - -#### 3.3.2.1 Configuring the Environment Variables and Allowlist - -Configure the rexec server allowlist on the host. Put the **whitelist** file in the **/etc/rexec** directory, and change the file permission to read-only. - -```bash -chmod 400 /etc/rexec/whitelist -``` - -In the test environment, the allowlist is not mandatory. You can disable the allowlist by deleting the **whitelist** file and restarting the rexec_server process. - -After downloading the dpu-utilities code, go to the **qtfs/rexec** directory and run `make && make install` to install all binary files required by rexec (**rexec** and **rexec_server**) to the **/usr/bin** directory. - -Before starting the rexec_server service on the server, check whether the **/var/run/rexec** directory exists. If not, create it. - -```bash -mkdir /var/run/rexec -``` - -#### 3.3.2.2 Starting the Service - -You can start the rexec_server service on the server in either of the following ways. - -* Method 1: -Configure rexec as a systemd service. - -Add the **[rexec.service](./config/rexec.service)** file to **/usr/lib/systemd/system**. - -Then, use `systemctl` to manage the rexec service. - -Start the service for the first time: - -```bash -systemctl daemon-reload - -systemctl enable --now rexec -``` - -Restart the service: - -```bash -systemctl stop rexec - -systemctl start rexec -``` - -* Method 2: -Manually start the service in the background. - -```bash -nohup /usr/bin/rexec_server 2>&1 & -``` - -## 3.4 libvirt Service Deployment - -### 3.4.1 Deploying on the HOST - -Install the VM runtime and libvirt. (libvirt is installed to create related directories.) - -```bash -yum install -y qemu libvirt edk2-aarch64 # (required for starting VMs in the Arm environment) -``` - -Put the VM image on the HOST. The VM image will be mounted to the client through qtfs and shared with libvirt. - -### 3.4.2 Deploying on the DPU - -#### 3.4.2.1 Creating the Chroot Environment - -(a) Download the QCOW image from the openEuler official website, for example, openEuler 23.09: . - -(b) Mount the QCOW2 image. - -```bash -cd /root/ - -mkdir p2 new_root_origin new_root - -modprobe nbd maxport=8 - -qemu-nbd -c /dev/nbd0 xxx.qcow2 - -mount /dev/nbd0p2 /root/p2 - -cp -rf /root/p2/* /root/new_root_origin/ - -umount /root/p2 - -qemu-nbd -d /dev/nbd0 -``` - -(c) Now, the root directory of the image is decompressed in **new_root_origin**. Bind mount **new_root** to **new_root_origin** as the mount point for chroot. - -```bash -mount --bind /root/new_root_origin /root/new_root -``` - -#### 3.4.2.2 Installing libvirt - -Compile the source code with a patch. - -(a) Go to the chroot environment and install the compilation environment and common tools. - -```bash -yum groupinstall "Development tools" -y -yum install -y vim meson qemu qemu-img strace edk2-aarch64 tar -``` - -**edk2-aarch64** is required for starting VMs in the Arm environment. - -(b) Install the dependency packages required for libvirt compilation. - -```bash - yum install -y rpcgen python3-docutils glib2-devel gnutls-devel libxml2-devel libpciaccess-devel libtirpc-devel yajl-devel systemd-devel dmidecode glusterfs-api numactl -``` - -(c) Download the libvirt-x.x.x source code package . - -(d) Obtain the libvirt patch: - -. - -(e) Decompress the source code package to a directory in the chroot environment, for example, **/home**, and apply the patch. - -(f) Go to the **libvirt-x.x.x** directory and run the following command: - -```bash -meson build --prefix=/usr -Ddriver_remote=enabled -Ddriver_network=enabled -Ddriver_qemu=enabled -Dtests=disabled -Ddocs=enabled -Ddriver_libxl=disabled -Ddriver_esx=disabled -Dsecdriver_selinux=disabled -Dselinux=disabled -``` - -(g) Complete the installation. - -```bash -ninja -C build install -``` - -#### 3.4.2.3 Starting the libvirtd Service - -To use libvirt direct connection aggregation, you need to start the libvirtd service in the chroot environment, which requires the libvirtd service outside the chroot environment to be stopped. - -(a) Put the [VM jumper script](./scripts/qemu-kvm) in **/usr/bin** and **/usr/libexec** in the chroot environment to replace the **qemu-kvm** binary file. The jumper script will call rexec to start a remote VM. ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->In the XML file of virsh, set **\** under **\** to **qemu-kvm**. If you set **\** to another value, change it to **qemu-kvm** or replace the binary file specified by **\** with the jumper script. The content of the jumper script also needs to be modified accordingly. - -(b) Copy the **libudsproxy.so** file generated during udsproxyd compilation to the **/usr/lib64** directory in the chroot directory. If the udsproxyd service is used by configuring the UDS allowlist of qtfs, you do not need to copy the **libudsproxy.so** file. - -(c) Save the **rexec** binary file generated during rexec compilation to the **/usr/bin** directory of the chroot environment. - -(d) To configure the chroot mounting environment, you need to mount some directories. Use the following scripts: - -* [virt_start.sh](./scripts/virt_start.sh) is the configuration script. In the script, you need to manually change the **qtfs.ko** path to the path of the compiled **.ko** file and set the correct HOST IP address. -* [virt_umount.sh](./scripts/virt_umount.sh) is the configuration revert script. - -(e) The mount directories in the script are based on the examples in this document. You can modify the paths in the script as required. - -(f) After the chroot environment is configured, enter the chroot environment and manually start libvirtd. - -If qtfs is not configured to use the udsproxyd allowlist, run the following commands: - -```bash -LD_PRELOAD=/usr/lib64/libudsproxy.so virtlogd -d -LD_PRELOAD=/usr/lib64/libudsproxy.so libvirtd -d -``` - -If qtfs is configured to use the udsproxyd allowlist, the LD_PRELOAD prefix is not required: - -```bash -virtlogd -d -libvirtd -d -``` - -To check whether the allowlist is configured, run the following command in another terminal that is not in the chroot environment: - -```bash -qtcfg -z -``` - -Check whether the allowlist contains **/var/lib/libvirt**. - -## 3.5 VM Startup - -After the service is deployed, you can manage the VM life cycle from the DPU. - -### 3.5.1 Defining the VM - -(a) Place the VM boot image in a directory on the HOST, for example: - -```bash -/home/VMs/Domain_name -``` - -(b) Use qtfs to mount the directory to the DPU. - -```bash -mount -t qtfs /home/VMs /home/VMs -``` - -(c) In the XML file, **/home/VMs/Domain_name** is used as the boot image. In this way, the same image file is presented to the DPU and HOST (**Domain_name** is the VM **domain**). - -(d) Check whether **\** in the XML file points to the jumper script. - -(e) Define the VM. - -```bash -virsh define xxx.xml -``` - -### 3.5.2 Starting the VM - -```bash -virsh start domain -``` - -# 4 Environment Reset - -Some libvirt directories are shared between the DPU and the HOST. Therefore, you need to unmount these directories before uninstalling the environment. Generally, stop the libvirtd and virtlogd processes and run the **virt_umount.sh** script. If a VM is running on the HOST, stop the VM before unmounting the directories. - -# 5 Common Errors - -1. libvirt compilation failure: Check whether the dependency packages are installed. If an external directory or HOST directory is mounted to the chroot environment, the compilation may fail. In this case, unmount the directory first. - -2. qtfs mounting failure: The engine process on the server is not started or the firewall is not disabled. As a result, the qtfs connection fails. - -3. VM definition failure: Check whether the emulator in the XML file points to the jumper script, whether the VM image has been mounted to the DPU through qtfs, and whether the path is the same as that on the HOST. - -4. VM startup failure: Check whether the libvirtd and virtlogd services are started, whether the rexec service is started, whether the jumper process is started, and whether an error is reported when qemu-kvm is started. diff --git a/docs/en/docs/DPUOffload/offload-deployment-guide.md b/docs/en/docs/DPUOffload/offload-deployment-guide.md deleted file mode 100644 index 094150a113179bfbf8e19661df28d5c2808ffb99..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPUOffload/offload-deployment-guide.md +++ /dev/null @@ -1,168 +0,0 @@ - -# Imperceptible Container Management Plane Offload Deployment Guide - -> ![](./public_sys-resources/icon-note.gif) **NOTE**: -> -> In this user guide, modifications are performed to the container management plane components and the rexec tool of a specific version. You can modify other versions based on the actual execution environment. The patch provided in this document is for verification only and is not for commercial use. -> ![](./public_sys-resources/icon-note.gif) **NOTE**: -> -> The communication between shared file systems is implemented through the network. You can perform a simulated offload using two physical machines or VMs connected through the network. -> -> Before the verification, you are advised to set up a Kubernetes cluster and container running environment that can be used properly and offload the management plane process of a single node. You can use a physical machine or VM that is connected to the network as an emulated DPU. - -## Introduction - -Container management plane, that is, management tools of containers such as Kubernetes, dockerd, containerd, and isulad. Container management plane offload is to offload the container management plane from the host where the container is located to another host, that is, the DPU, a set of hardware that has an independent running environment. - -By mounting directories related to container running on the host to the DPU through qtfs, the container management plane tool running on the DPU can access these directories and prepare the running environment for the containers running on the host. To remotely mount the special file systems such as proc and sys, a dedicated rootfs is created as the running environment of Kubernetes and dockerd (referred to as **/another_rootfs**). - -In addition, rexec is used to start and delete containers so that the container management plane and containers can run on two different hosts for remote container management. - -## Related Component Patches - -### rexec - -rexec is a remote execution tool written in the Go language based on the [rexec](https://github.com/docker/libchan/tree/master/examples/rexec) example tool of Docker/libchan. rexec is used to remotely invoke binary files. For ease of use, capabilities such as transferring environment variables and monitoring the exit of original processes are added to rexec. - -To use the rexec tool, run the `CMD_NET_ADDR=tcp://0.0.0.0: rexec_server` command on the server to start the rexec service process, and then run the `CMD_NET_ADDR=tcp://: rexec [command]` on the client`. This instructs rexec_server to execute the command. - -### dockerd - -The changes to dockerd are based on version 18.09. - -In containerd, the part that invokes libnetwork-setkey through hook is commented out. This does not affect container startup. In addition, to ensure the normal use of `docker load`, an error in the `mount` function in **mounter_linux.go** is commented out. - -In the running environment of the container management plane, **/proc** is mounted to the proc file system on the server, and the local proc file system is mounted to **/local_proc**. In dockerd and containerd, **/proc** is changed to **/local_proc** for accessing **/proc/self/xxx**, **/proc/getpid()/xxx**, or related file systems. - -### containerd - -The changes to containerd are based on containerd-1.2-rc.1. - -When obtaining mounting information, **/proc/self/mountinfo** can obtain only the local mounting information of dockerd but cannot obtain that on the server. Therefore, **/proc/self/mountinfo** is changed to **/proc/1/mountinfo** to obtain the mounting information on the server by obtaining the mounting information of process 1 on the server. - -In containerd-shim, the Unix socket that communicates with containerd is changed to TCP. containerd obtains the IP address of the running environment of containerd-shim through the **SHIM_HOST** environment variable, that is, the IP address of the server. The has value of shim is used to generate a port number, which is used as the communication port to start containerd-shim. - -In addition, the original method of sending signals to containerd-shim is changed to the method of remotely invoking the `kill` command to send signals to shim, ensuring that Docker can correctly kill containers. - -### Kubernetes - -kubelet is not modified. The container QoS manager may fail to be configured for the first time. This error does not affect the subsequent pod startup process. - -## Container Management Plane Offload Operation Guide - -Start rexec_server on both the server and client. rexec_server on the server is used to invoke rexec to stat containerd-shim. rexec_server on the client is used to execute invoking of dockerd and containerd by containerd-shim. - -### Server - -Create a folder required by the container management plane, insert **qtfs_server.ko**, and start the engine process. - -In addition, you need to create the rexec script **/usr/bin/dockerd** on the server. - -``` shell -#!/bin/bash -CMD_NET_ADDR=tcp://: rexec /usr/bin/dockerd $* -``` - -### Client - -Prepare a rootfs as the running environment of dockerd and containerd. Use the following script to mount the server directories required by dockerd and containerd to the client. Ensure that the remote directories mounted in the script exist on both the server and client. - -``` shell -#!/bin/bash -mkdir -p /another_rootfs/var/run/docker/containerd -iptables -t nat -N DOCKER -echo "---------insmod qtfs ko----------" -insmod /YOUR/QTFS/PATH/qtfs.ko qtfs_server_ip= qtfs_log_level=INFO - -# The proc file system in the chroot environment is replaced by the proc shared file system of the DPU. The actual proc file system of the local host needs to be mounted to **/local_proc**. -mount -t proc proc /another_rootfs/local_proc/ - -# Bind the chroot internal environment to the external environment to facilitate configuration and running. -mount --bind /var/run/ /another_rootfs/var/run/ -mount --bind /var/lib/ /another_rootfs/var/lib/ -mount --bind /etc /another_rootfs/etc - -mkdir -p /another_rootfs/var/lib/isulad - -# Create and mount the dev, sys, and cgroup file systems in the chroot environment. -mount -t devtmpfs devtmpfs /another_rootfs/dev/ -mount -t sysfs sysfs /another_rootfs/sys -mkdir -p /another_rootfs/sys/fs/cgroup -mount -t tmpfs tmpfs /another_rootfs/sys/fs/cgroup -list="perf_event freezer files net_cls,net_prio hugetlb pids rdma cpu,cpuacct memory devices blkio cpuset" -for i in $list -do - echo $i - mkdir -p /another_rootfs/sys/fs/cgroup/$i - mount -t cgroup cgroup -o rw,nosuid,nodev,noexec,relatime,$i /another_rootfs/sys/fs/cgroup/$i -done - -## common system dir -mount -t qtfs -o proc /proc /another_rootfs/proc -echo "proc" -mount -t qtfs /sys /another_rootfs/sys -echo "cgroup" - -# Mount the shared directory required by the container management plane. -mount -t qtfs /var/lib/docker/containers /another_rootfs/var/lib/docker/containers -mount -t qtfs /var/lib/docker/containerd /another_rootfs/var/lib/docker/containerd -mount -t qtfs /var/lib/docker/overlay2 /another_rootfs/var/lib/docker/overlay2 -mount -t qtfs /var/lib/docker/image /another_rootfs/var/lib/docker/image -mount -t qtfs /var/lib/docker/tmp /another_rootfs/var/lib/docker/tmp -mkdir -p /another_rootfs/run/containerd/io.containerd.runtime.v1.linux/ -mount -t qtfs /run/containerd/io.containerd.runtime.v1.linux/ /another_rootfs/run/containerd/io.containerd.runtime.v1.linux/ -mkdir -p /another_rootfs/var/run/docker/containerd -mount -t qtfs /var/run/docker/containerd /another_rootfs/var/run/docker/containerd -mount -t qtfs /var/lib/kubelet/pods /another_rootfs/var/lib/kubelet/pods -``` - -In**/another_rootfs**, create the following script to support cross-host operations: - -* /another_rootfs/usr/local/bin/containerd-shim - -``` shell -#!/bin/bash -CMD_NET_ADDR=tcp://: /usr/bin/rexec /usr/bin/containerd-shim $* -``` - -* /another_rootfs/usr/local/bin/remote_kill - -``` shell -#!/bin/bash -CMD_NET_ADDR=tcp://: /usr/bin/rexec /usr/bin/kill $* -``` - -* /another_rootfs/usr/sbin/modprobe - -``` shell -#!/bin/bash -CMD_NET_ADDR=tcp://: /usr/bin/rexec /usr/sbin/modprobe $* -``` - -After changing the root directories of dockerd and containerd to the required rootfs, run the following command to start dockerd and containerd: - -* containerd - -``` shell -#!/bin/bash -SHIM_HOST= containerd --config /var/run/docker/containerd/containerd.toml --address /var/run/containerd/containerd.sock -``` - -* dockerd - -``` shell -#!/bin/bash -SHIM_HOST=CMD_NET_ADDR=tcp://: /usr/bin/dockerd --containerd /var/run/containerd/containerd.sock -``` - -* kubelet - -Use the original parameters to start kubelet in the chroot environment. - -Because **/var/run/** is bound to **/another_rootfs/var/run/**, you can use Docker to access the **docker.sock** interface for container management in the regular rootfs. - -The container management plane is offloaded to the DPU. You can run `docker` commands to create and delete containers, or use `kubectl` on the current node to schedule and destroy pods. The actual container service process runs on the host. - -> ![](./public_sys-resources/icon-note.gif) **NOTE**: -> -> This guide describes only the container management plane offload. The offload of container network and data volumes requires additional offload capabilities, which are not included. You can perform cross-node startup of containers that are not configured with network and storage by referring to this guide. diff --git a/docs/en/docs/DPUOffload/qtfs-architecture-and-usage.md b/docs/en/docs/DPUOffload/qtfs-architecture-and-usage.md deleted file mode 100644 index 7fe0544418ae42a58daab9eb2e091f945d944416..0000000000000000000000000000000000000000 --- a/docs/en/docs/DPUOffload/qtfs-architecture-and-usage.md +++ /dev/null @@ -1,77 +0,0 @@ -# qtfs Shared File System Architecture and Usage - -## Introduction - -qtfs is a shared file system project. It can be deployed on either a host-DPU hardware architecture or on two hosts. qtfs works in client-server mode, allowing the client to access specified file systems on the server in the same way that local files are accessed. - -qtfs provides the following features: - -+ Mount point propagation - -+ Sharing of special file systems such as proc, sys, and cgroup - -+ Shared read and write of remote files - -+ Remote mounting of server file systems on the client - -+ Customized processing of special files - -+ Remote FIFO, Unix sockets, and epoll that allow the client and server to access the files as if they were like local - -+ Bottom-layer host-DPU communication over the PCIe protocol, outperforming the network - -+ Kernel module development, preventing intrusive modification to the kernel - -## Software Architecture - -![qtfs-arch](./figures/qtfs-arch.png) - -## Installation - -Perform operations in the following qtfs-related directories: - -+ **qtfs**: code of the client kernel module. Compile the client **.ko** file in this directory. - -+ **qtfs_server**: code of the server kernel module. Compile the server **.ko** files and related programs in this directory. - -+ **qtinfo**: diagnosis tool that is used to check the status of file systems and change the log level. - -+ **demo**, **test**, and **doc**: demo programs, test programs, and project documents. - -+ Root directory: code of common modules used by the client and server. - -Configure the kernel compilation environment on two servers (or VMs). - -1. The kernel version must be 5.10 or later. -2. Install the kernel development package by running `yum install kernel-devel`. -3. Assume that the host IP address is 192.168.10.10 and the DPU IP address is 192.168.10.11. - -Install the qtfs server. - -```bash - 1. cd qtfs_server - 2. make clean && make - 3. insmod qtfs_server.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN - 4. nohup ./engine 16 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 & -``` - -Install the qtfs client. - -```bash - 1. cd qtfs - 2. make clean && make - 3. insmod qtfs.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN - 4. cd ../ipc/ - 5. make clean && make && make install - 6. nohup udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 & -``` - -## Usage - -After the installation is complete, mount the server file system to the client. For example: - -```bash - mount -t qtfs / /root/mnt/ -``` - -The file system is visible to the client. Access **/root/mnt** on the client to view and operate files on the server. diff --git a/docs/en/docs/Embedded/application-development-using-sdk.md b/docs/en/docs/Embedded/application-development-using-sdk.md deleted file mode 100644 index d05ff8b8634f5a515703a003d9abc9fcd57c0feb..0000000000000000000000000000000000000000 --- a/docs/en/docs/Embedded/application-development-using-sdk.md +++ /dev/null @@ -1,181 +0,0 @@ -# Application Development Using openEuler Embedded SDK - -In addition to the basic functions of openEuler Embedded, you can also develop applications, that is, running your own programs on openEuler Embedded. This chapter describes how to develop applications using openEuler Embedded SDK. - - -- [Application Development Using openEuler Embedded SDK](#application-development-using-openeuler-embedded-sdk) - - [Installing the SDK](#installing-the-sdk) - - [Using the SDK to Build a Hello World Example](#using-the-sdk-to-build-a-hello-world-example) - - [Using the SDK to Build a Kernel Module Example](#using-the-sdk-to-build-a-kernel-module-example) - - -### Installing the SDK - -1. **Install dependent software packages.** - - To use the SDK to develop the kernel module, you need to install some necessary software packages. Run the following command: - - ``` - Install on openEuler: - yum install make gcc g++ flex bison gmp-devel libmpc-devel openssl-devel - - Install on Ubuntu: - apt-get install make gcc g++ flex bison libgmp3-dev libmpc-dev libssl-dev - ``` - -2. **Run the self-extracting installation script of the SDK.** - - Run the following command: - - ``` - sh openeuler-glibc-x86_64-openeuler-image-aarch64-qemu-aarch64-toolchain-22.09.sh - ``` - - Enter the installation path of the toolchain as prompted. The default path is **/opt/openeuler/\/**. You can also set the path to a relative or absolute path. - - The following is an example: - - ``` - sh ./openeuler-glibc-x86_64-openeuler-image-armv7a-qemu-arm-toolchain-22.09.sh - openEuler embedded(openEuler Embedded Reference Distro) SDK installer version 22.09 - ================================================================ - Enter target directory for SDK (default: /opt/openeuler/22.09): sdk - You are about to install the SDK to "/usr1/openeuler/sdk". Proceed [Y/n]? y - Extracting SDK...............................................done - Setting it up...SDK has been successfully set up and is ready to be used. - Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g. - $ . /usr1/openeuler/sdk/environment-setup-armv7a-openeuler-linux-gnueabi - ``` - -3. **Set the environment variable of the SDK.** - - Run the `source` command. The `source` command is displayed in the output of the previous step. Run the command. - - ``` - . /usr1/openeuler/myfiles/sdk/environment-setup-armv7a-openeuler-linux-gnueabi - ``` - -4. **Check whether the installation is successful.** - - Run the following command to check whether the installation and environment configuration are successful: - - ``` - arm-openeuler-linux-gnueabi-gcc -v - ``` - -### Using the SDK to Build a Hello World Example - -1. **Prepare the code.** - - The following describes how to build a hello world program that runs in the image of the openEuler Embedded root file system. - - Create a **hello.c** file. The source code is as follows: - - ``` c - #include - - int main(void) - { - printf("hello world\n"); - } - ``` - - Compose a **CMakeLists.txt** file as follows and place it in the same directory as the **hello.c** file. - - ``` - project(hello C) - - add_executable(hello hello.c) - ``` - -2. **Compile and generate a binary file.** - - Go to the directory where the **hello.c** file is stored and run the following commands to compile the file using the toolchain: - - ``` - cmake .. - make - ``` - - Copy the compiled **hello** program to a sub-directory of **/tmp/** (for example, **/tmp/myfiles/**) on openEuler Embedded. For details about how to copy the file, see [Shared File System Enabled Scenario](./installation-and-running.md#shared-file-system-enabled-scenario). - -3. **Run the user-mode program.** - - Run the **hello** program on openEuler Embedded. - - ``` - cd /tmp/myfiles/ - ./hello - ``` - - If the running is successful, the message **hello world** is displayed. - -### Using the SDK to Build a Kernel Module Example - -1. **Prepare the code.** - - The following describes how to build a kernel module that runs in the kernel of openEuler Embedded. - - Create a **hello.c** file. The source code is as follows: - - ```c - #include - #include - - static int hello_init(void) - { - printk("Hello, openEuler Embedded!\r\n"); - return 0; - } - - static void hello_exit(void) - { - printk("Byebye!"); - } - - module_init(hello_init); - module_exit(hello_exit); - - MODULE_LICENSE("GPL"); - ``` - - Compose a Makefile as follows and place it in the same directory as the **hello.c** file. - - ``` - KERNELDIR := ${KERNEL_SRC_DIR} - CURRENT_PATH := $(shell pwd) - - target := hello - obj-m := $(target).o - - build := kernel_modules - - kernel_modules: - $(MAKE) -C $(KERNELDIR) M=$(CURRENT_PATH) modules - clean: - $(MAKE) -C $(KERNELDIR) M=$(CURRENT_PATH) clean - ``` - - `KERNEL_SRC_DIR` indicates the directory of the kernel source tree. This variable is automatically configured after the SDK is installed. - - ![](./public_sys-resources/icon-note.gif) **NOTE** - - - `KERNEL_SRC_DIR` indicates the directory of the kernel source tree. This variable is automatically configured after the SDK is installed. - - - The `$(MAKE) -C $(KERNELDIR) M=$(CURRENT_PATH) modules` and `$(MAKE) -C $(KERNELDIR) M=$(CURRENT_PATH) clean` codes are preceded by the Tab key instead of the Space key. - -2. **Compile and generate a kernel module.** - - Go to the directory where the **hello.c** file is stored and run the following command to compile the file using the toolchain: - - make - - Copy the compiled **hello.ko** file to a directory on openEuler Embedded. For details about how to copy the file, see the [Shared File System Enabled Scenario](./installation-and-running.md#shared-file-system-enabled-scenario). - -3. **Insert the kernel module.** - - Insert the kernel module to openEuler Embedded: - - insmod hello.ko - - If the running is successful, the message **Hello, openEuler Embedded!** is output to the kernel logs. \ No newline at end of file diff --git a/docs/en/docs/Embedded/container-build-guide.md b/docs/en/docs/Embedded/container-build-guide.md deleted file mode 100644 index 1414c94c4e9af1cddb684dfd0aac37d27b516754..0000000000000000000000000000000000000000 --- a/docs/en/docs/Embedded/container-build-guide.md +++ /dev/null @@ -1,197 +0,0 @@ -Container Build Guide -============================== - -The openEuler Embedded build process is based on the openEuler OS, and requires many system tools and build tools to be installed. To help developers quickly set up a build environment, the OS and tools on which the build process depends are encapsulated into a container. In this way, developers can avoid the time-consuming environment preparation process and focus on development. - - - - - [Environment Preparation](#environment-preparation) - - [Installing Docker](#installing-docker) - - [Obtaining the Container Image](#obtaining-the-container-image) - - [Preparing the Container Build Environment](#preparing-the-container-build-environment) - - [Version Build](#version-build) - - [Downloading Source Code](#downloading-source-code) - - [Compiling the Build](#compiling-the-build) - - [Build Result](#build-result) - - -## Environment Preparation - -Use Docker to create a container environment. The software and hardware requirements of Docker are as follows: - -- OS: openEuler 20.03/22.03, Ubuntu 20.04/22.04, Debian 11, and SUSE 12.05 are recommended. -- Kernel: Linux 3.8 or later is recommended. -- Driver: The kernel must include a proper storage driver, for example, Device Mapper, AUFS, vfs, btrfs, or ZFS. -- Architecture: 64-bit architecture (currently only x86-64 and AMD64). - -### Installing Docker - -------------- - -1. Check whether Docker has been installed in the current environment. - - Run the following command. If the Docker version is displayed, Docker has been installed in the current environment. You can use it directly. - - ``` {.sourceCode .console} - docker version - ``` - -2. If Docker is not installed, install it by referring to the [official document](http://docs.docker.com/engine/install/). - - Install Docker on openEuler by referring to the installation guide for CentOS. - - For example, run the following command to install Docker on openEuler: - - ``` - sudo yum install docker - ``` - -### Obtaining the Container Image - ---------------- - -Run the `docker pull` command to pull the image from Huawei Cloud to the host machine: - -``` -docker pull swr.cn-north-4.myhuaweicloud.com/openeuler-embedded/openeuler-container -``` - -By default, the latest image is downloaded. You can also specify the image version to be downloaded based on the version to be compiled. The command is as follows: - -``` -docker pull [Container Image Name]:[Tag] -# example: docker pull swr.cn-north-4.myhuaweicloud.com/openeuler-embedded/openeuler-container:latest -``` - -Container images - -| Container Image Name | Tag | For Image Branch | Kernel Version | Libc Version | -| ------------------------------------------------------------ | --------- | ------------------- | -------------- | ------------ | -| swr.cn-north-4.myhuaweicloud.com/openeuler -embedded/openeuler-container | latest | master | 21.03 | 2.31 | -| swr.cn-north-4.myhuaweicloud.com/openeuler -embedded/openeuler-container | 22.09 | openEuler-22.09 | 21.03 | 2.31 | -| swr.cn-north-4.myhuaweicloud.com/openeuler -embedded/openeuler-container | 22.03-lts | openEuler-22.03-LTS | 22.03 LTS | 2.34 | -| swr.cn-north-4.myhuaweicloud.com/openeuler -embedded/openeuler-container | 21.09 | openEuler-21.09 | 21.03 | 2.31 | - -> ![](./public_sys-resources/icon-note.gif) **NOTE** -> -> - To build openEuler images of different branches or versions, you need to use different containers. The **For Image Branch** column shows the mapping. -> - To be compatible with the host tool and native SDK of Yocto Poky, the built-in libc 2.31 container is used. Therefore, the C library version is earlier than 22.03. - -### Preparing the Container Build Environment - -------------------- - -#### 1. Start a container. - -Run the `docker run` command to start a container. To ensure that the container can run in the background and access the Internet after being started, you are advised to run the following command to start the container: - -``` -docker run -idt --network host swr.cn-north-4.myhuaweicloud.com/openeuler-embedded/openeuler-container bash -``` - -Parameter description: - -- **-i**: keeps the standard input open. -- **-d**: starts a container in daemon mode in the background. -- **-t**: allocates a pseudo-tty and binds it to the standard input of the container. -- **\--network**: connects a container to the network of the host machine. -- **swr.cn-north-4.myhuaweicloud.com/openeuler-embedded/openeuler-container**: specifies the name of the image (image_name:image_version). -- **bash**: method for accessing a container. - -#### 2. Check the ID of the started container. - -``` -docker ps -``` - -#### 3. Enter the container. - -``` -docker exec -it bash -``` - -After the build environment is ready, you can build in the container. - -## Version Build - -### Downloading Source Code - -1. Obtain the source code download script. - - ``` - git clone https://gitee.com/openeuler/yocto-meta-openeuler.git -b -v /usr1/openeuler/src/yocto-meta-openeuler - #example: git clone https://gitee.com/openeuler/yocto-meta-openeuler.git -b master -v /usr1/openeuler/src/yocto-meta-openeuler - ``` - - > ![](./public_sys-resources/icon-note.gif) **NOTE** - > - > - For details about **<For Image Branch>**, see the content in the third column of the container image list. - > - > - The full code required for build is obtained from the yocto-meta-openeuler repository. Therefore, if you want to build the code of the corresponding version (such as openEuler 22.09 or openEuler 22.03 LTS), download the yocto-meta-openeuler repository of the corresponding branch. - > - Different containers are required for building openEuler images of different branches or versions. - -2. Download the source code using the script. - - ``` - cd /usr1/openeuler/src/yocto-meta-openeuler/scripts - sh download_code.sh /usr1/openeuler/src - ``` - - > ![](./public_sys-resources/icon-note.gif) **NOTE** - > - > 22.09 and later versions support **/usr1/openeuler/src/yocto-meta-openeuler/script/oe_helper.sh**. You can run the **source oe_helper.sh** command to download the code by referring to the **usage** description. - -### Compiling the Build - -- Compilation architecture: aarch64-std, aarch64-pro, arm-std, or raspberrypi4-64 -- Build directory: **/usr1/build** -- Source code directory: **/usr1/openeuler/src** -- Path of the compiler: **/usr1/openeuler/gcc/openeuler\_gcc\_arm64le** - ->![](./public_sys-resources/icon-note.gif) **NOTE** ->- Use different compilers for different compilation architectures. aarch64-std, aarch64-pro, and raspberrypi4-64 use the openeuler\_gcc\_arm64le compiler, and arm-std uses the openeuler\_gcc\_arm32le compiler. ->- The following uses the aarch64-std architecture as an example. - -1. Change the owner group of the **/usr1** directory to **openeuler**. Otherwise, permission issues may occur when switching to the **openeuler** user. - - ``` - chown -R openeuler:users /usr1 - ``` - -2. Switch to the **openeuler** user. - - ``` - su openeuler - ``` - -3. Go to the path where the build script is stored and run the script. - - ``` - # Go to the directory where the compilation initialization scripts are stored. - cd /usr1/openeuler/src/yocto-meta-openeuler/scripts - ``` - ``` - # For versions earlier than 22.03, skip this command. (You must run this command in 22.09 and later versions.) - # Initialize the container build dependency tool (poky nativesdk). - . /opt/buildtools/nativesdk/environment-setup-x86_64-pokysdk-linux - ``` - ``` - # Initialize the compilation environment using the compilation initialization script. - source compile.sh aarch64-std /usr1/build /usr1/openeuler/gcc/openeuler_gcc_arm64le - bitbake openeuler-image - ``` - - > ![](./public_sys-resources/icon-note.gif) **NOTE** - > - > 22.09 and later versions support **/usr1/openeuler/src/yocto-meta-openeuler/script/oe_helper.sh**. You can run the **source oe_helper.sh** command to download the code by referring to the **usage** description. - -### Build Result - -By default, the files are generated in the **output** directory of the build directory. For example, the built files of the aarch64-std example are generated in the **/usr1/build/output** directory, as shown in the following table: - -| Filename | Description | -| --------------------------------------------------------- | ----------------------------------- | -| Image-\* | openEuler Embedded image | -| openeuler-glibc-x86\_64-openeuler-image-\*-toolchain-\*.sh | openEuler Embedded SDK toolchain | -| openeuler-image-qemu-aarch64-\*.rootfs.cpio.gz | openEuler Embedded file system | -| zImage | openEuler Embedded compressed image | diff --git a/docs/en/docs/Embedded/embedded.md b/docs/en/docs/Embedded/embedded.md deleted file mode 100644 index 9077b34292e25b5504254863c07c64e0dcb0232f..0000000000000000000000000000000000000000 --- a/docs/en/docs/Embedded/embedded.md +++ /dev/null @@ -1,5 +0,0 @@ -# Embedded - -This document includes the following content: - -- openEuler Embedded User Guide: describes how to use, build, and develop relevant programs. diff --git a/docs/en/docs/Embedded/installation-and-running.md b/docs/en/docs/Embedded/installation-and-running.md deleted file mode 100644 index 1ec121c66fcff0a053b0fa4f99f6054235afdfd1..0000000000000000000000000000000000000000 --- a/docs/en/docs/Embedded/installation-and-running.md +++ /dev/null @@ -1,186 +0,0 @@ -# Installation and Running - -This chapter describes how to obtain a pre-built image and how to run an image. - - - -- [Installation and Running](#installation-and-running) - - [Obtaining the Image](#obtaining-the-image) - - [Image Content](#image-content) - - [Running the Image](#running-the-image) - - [Simplified Running Scenario](#simplified-running-scenario) - - [Shared File System Enabled Scenario](#shared-file-system-enabled-scenario) - - [Network Enabled Scenario](#network-enabled-scenario) - - -## Obtaining the Image -The released pre-built images support only the ARM and AArch64 architectures, and are compatible only with the ARM virt-4.0 platform of QEMU. You can obtain the images through the following links: - -- [qemu_arm](https://repo.openeuler.org/openEuler-22.09/embedded_img/arm32/arm-std/) for ARM Cortex A15 processor of 32-bit ARM architecture. -- [qemu_aarch64](https://repo.openeuler.org/openEuler-22.09/embedded_img/arm64/aarch64-std/) for ARM Cortex A57 processor of 64-bit AArch64 architecture. - -You can deploy an openEuler Embedded image on a physical bare-metal server, cloud server, container, or VM as long as the environment supports QEMU 5.0 or later. - -## Image Content - -The downloaded image consists of the following parts: - -- Kernel image **zImage**, which is built based on Linux 5.10 of the openEuler community. You can obtain the kernel configurations through the following links: - - [ARM(Cortex a15)](https://gitee.com/openeuler/yocto-embedded-tools/blob/openEuler-22.09/config/arm/defconfig-kernel) - - [ARM(Cortex a57)](https://gitee.com/openeuler/yocto-embedded-tools/blob/openEuler-22.09/config/arm64/defconfig-kernel) for the AArch64 architecture. The image provides the image self-decompression function in addition. For details, see the corresponding [patch](https://gitee.com/openeuler/yocto-embedded-tools/blob/openEuler-22.09/patches/arm64/0001-arm64-add-zImage-support-for-arm64.patch). - -- Root file system image: - - **openeuler-image-qemu-xxx.cpio.gz**, which is the image of the standard root file system. It has received necessary security hardening and includes various software packages, such as audit, cracklib, OpenSSH, Linux PAM, shadow and software packages supported by iSula. - -- Software Development Kit (SDK) - - - **openeuler-glibc-x86_64-xxxxx.sh**: The self-extracting installation package of openEuler Embedded SDK. The SDK contains tools, libraries, and header files for developing user-mode applications and kernel modules. - -## Running the Image - -You can run the image to experience the functions of openEuler Embedded, and develop basic embedded Linux applications. - -![](./public_sys-resources/icon-note.gif) **Note:** - -- You are advised to use QEMU 5.0 or later to run the image. Some additional functions (the network and shared file system) depend on the virtio-net and virtio-fs features of QEMU. If these features are not enabled in QEMU, errors may occur during image running. In this case, you may need to recompile QEMU from the source code. - -- When running the image, you are advised to place the kernel image and root file system image in the same directory. - -Download and install QEMU by referring to the [QEMU official website](https://www.qemu.org/download/#linux), or download and build from [source](https://www.qemu.org/download/#source). Use the following command to verify the installation: - -``` -qemu-system-aarch64 --version -``` - -### Simplified Running Scenario - -In this scenario, the network and shared file system are not enabled in QEMU. You can use this scenario to experience the functions. - -1. **Start QEMU.** - - For the ARM architecture (ARM Cortex A15), run the following command: - - ``` - qemu-system-arm -M virt-4.0 -m 1G -cpu cortex-a15 -nographic -kernel zImage -initrd - ``` - - For the AArch64 architecture (ARM Cortex A57), run the following command: - - ``` - qemu-system-aarch64 -M virt-4.0 -m 1G -cpu cortex-a57 -nographic -kernel zImage -initrd - ``` - - ![](./public_sys-resources/icon-note.gif) **Note:** - - -The standard root file system image is securely hardened and requires you to set a password for the **root** user during the first startup. The password must comply with the follow requirements: - -1. Must contain at least eight characters. - -2. Must contain digits, letters, and special characters. - - @#$%^&*+|\\=~`!?,.:;-_'"(){}[]/>< - - For example, **openEuler@2021**. - - - -2. **Check whether QEMU is started successfully.** - - The shell of openEuler Embedded will be displayed after QEMU is successfully started and logged in. - -### Shared File System Enabled Scenario - -The shared file system allows the host machine of QEMU to share files with openEuler Embedded. In this way, programs that are cross-compiled on the host machine can run on openEuler Embedded after being copied to the shared directory. - -Assume that the **/tmp** directory of the host machine is used as the shared directory, and a **hello_openeuler.txt** file is created in the directory in advance. To enable the shared file system function, perform the following steps: - -1. **Start QEMU.** - - For the ARM architecture (ARM Cortex A15), run the following command: - - ``` - qemu-system-arm -M virt-4.0 -m 1G -cpu cortex-a15 -nographic -kernel zImage -initrd -device virtio-9p-device,fsdev=fs1,mount_tag=host -fsdev local,security_model=passthrough,id=fs1,path=/tmp - ``` - - For the AArch64 architecture (ARM Cortex A57), run the following command: - - ``` - qemu-system-aarch64 -M virt-4.0 -m 1G -cpu cortex-a57 -nographic -kernel zImage -initrd -device virtio-9p-device,fsdev=fs1,mount_tag=host -fsdev local,security_model=passthrough,id=fs1,path=/tmp - ``` - -2. **Mount the file system.** - - After you start and log in to openEuler Embedded, run the following commands to mount the shared file system: - - ``` - cd /tmp - mkdir host - mount -t 9p -o trans=virtio,version=9p2000.L host /tmp/host - ``` - - That is, mount the 9p file system to the **/tmp/host** directory of openEuler Embedded to implement sharing mapping. - -3. **Check whether the file system is shared successfully.** - - In openEuler Embedded, run the following commands: - - ``` - cd /tmp/host - ls - ``` - If **hello_openeuler.txt** is discovered, the file system is shared successfully. - -### Network Enabled Scenario - -The virtio-net of QEMU and the virtual NIC of the host machine allow for the network communication between the host machine and openEuler Embedded. In addition to sharing files using virtio-fs, you can transfer files between the host machine and openEuler Embedded using the network, for example, the **scp** command. - -1. **Start QEMU.** - - For the ARM architecture (ARM Cortex A15), run the following command: - - ``` - qemu-system-arm -M virt-4.0 -m 1G -cpu cortex-a15 -nographic -kernel zImage -initrd -device virtio-net-device,netdev=tap0 -netdev tap,id=tap0,script=/etc/qemu-ifup - ``` - - For the AArch64 architecture (ARM Cortex A57), run the following command: - - ``` - qemu-system-aarch64 -M virt-4.0 -m 1G -cpu cortex-a57 -nographic -kernel zImage -initrd -device virtio-net-device,netdev=tap0 -netdev tap,id=tap0,script=/etc/qemu-ifup - ``` - -2. **Create a vNIC on the host machine.** - - You can create a **/qemu-ifup** script in the **/etc** directory and run the script to create a **tap0** vNIC on the host machine. The script details are as follows: - - ``` - #!/bin/bash - ifconfig $1 192.168.10.1 up - ``` - **root** permissions are required for running the script. - - ``` - chmod a+x qemu-ifup - ``` - - Use the **qemu-ifup** script to create a **tap0** vNIC on the host machine. The IP address of the vNIC is **192.168.10.1**. - -3. **Configure the NIC of openEuler Embedded.** - - Log in to openEuler Embedded and run the following command: - ``` - ifconfig eth0 192.168.10.2 - ``` - -4. **Check whether the network connection is normal.** - - In openEuler Embedded, run the following command: - ``` - ping 192.168.10.1 - ``` - - If the IP address can be pinged, the network connection between the host machine and openEuler Embedded is normal. - - >![](./public_sys-resources/icon-note.gif) **Note:** - > - >If you need openEuler Embedded to access the Internet through the host machine, create a bridge on the host machine. For details, see the related documents. diff --git a/docs/en/docs/Embedded/openEuler-Embedded-22.09-release-notes.md b/docs/en/docs/Embedded/openEuler-Embedded-22.09-release-notes.md deleted file mode 100644 index b5148b4ed7321cd034ba25f4dd9bc3278b8f9fa7..0000000000000000000000000000000000000000 --- a/docs/en/docs/Embedded/openEuler-Embedded-22.09-release-notes.md +++ /dev/null @@ -1,29 +0,0 @@ -# openEuler Embedded 22.09 Release Notes - -openEuler Embedded 22.09 is the second innovation release of openEuler Embedded. This section describes the main features of this version. - -## Kernel - -- The kernel is upgraded to 5.10.0-106.18.0. - -- The kernel supports Preempt-RT patches. - -- The kernel supports Raspberry Pi 4B patches. - -## Software Packages - -- More than 140 software packages are supported. For details, see [Supported Software Packages](https://openeuler.gitee.io/yocto-meta-openeuler/features/software_package_description.html). - -## Feature Highlights - -- The multi-OS hybrid deployment capability is enhanced. The Raspberry Pi 4B hybrid deployment instance is added. The service-oriented hybrid deployment function is added. Zephyr can be accessed through the Linux shell CLI. For details, see [Multi-OS Hybrid Deployment Framework](https://openeuler.gitee.io/yocto-meta-openeuler/features/mcs.html). - -- The distributed soft bus capability is enhanced. The distributed soft bus-based openEuler and OpenHarmony device authentication and interconnection are supported. Southbound Wi-Fi transmission media are supported. For details, see [Distributed Soft Bus](https://openeuler.gitee.io/yocto-meta-openeuler/features/distributed_soft_bus.html). - -- Security hardening. For details, see [Security Hardening Description](https://openeuler.gitee.io/yocto-meta-openeuler/security_hardening/index.html). - -- Preempt-RT-based soft real-time. For details, see [Soft Real-Time System Introduction](https://openeuler.gitee.io/yocto-meta-openeuler/features/preempt_rt.html). - -## Build System - -- The NativeSDK is added for containerized build. For details, see [Container Build Guide](./container-build-guide.md). diff --git a/docs/en/docs/Embedded/public_sys-resources/hosttools.png b/docs/en/docs/Embedded/public_sys-resources/hosttools.png deleted file mode 100644 index 1b154b40fc76ead162cdd7c7d32303f581d9cfa8..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Embedded/public_sys-resources/hosttools.png and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/mac-content.jpg b/docs/en/docs/EulerLauncher/images/mac-content.jpg deleted file mode 100644 index 7db445ef7e6b068454e510bfd74f4e56ff523063..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/mac-content.jpg and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/mac-install.jpg b/docs/en/docs/EulerLauncher/images/mac-install.jpg deleted file mode 100644 index 07945fa95430a28993b65410881a248a62845bcb..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/mac-install.jpg and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/mac-start.jpg b/docs/en/docs/EulerLauncher/images/mac-start.jpg deleted file mode 100644 index d950b29c99438703a5dd670953c9afc30feb9494..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/mac-start.jpg and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/tray-icon.png b/docs/en/docs/EulerLauncher/images/tray-icon.png deleted file mode 100644 index be4dcb7853cd90e02c9cb37f4f6ee2c75da13469..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/tray-icon.png and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/win-install.jpg b/docs/en/docs/EulerLauncher/images/win-install.jpg deleted file mode 100644 index 715655ba9b3ed1cb037d385668aea931ed5c7c29..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/win-install.jpg and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/win-terminal-1.jpg b/docs/en/docs/EulerLauncher/images/win-terminal-1.jpg deleted file mode 100644 index 21dbb721c45068b146c84bbb46cd2845e987f7ce..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/win-terminal-1.jpg and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/images/win-terminal-2.jpg b/docs/en/docs/EulerLauncher/images/win-terminal-2.jpg deleted file mode 100644 index 74893803d48e5c7de52325be3f96fdd8c0400b27..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/EulerLauncher/images/win-terminal-2.jpg and /dev/null differ diff --git a/docs/en/docs/EulerLauncher/mac-user-manual.md b/docs/en/docs/EulerLauncher/mac-user-manual.md deleted file mode 100644 index 45d8d71d6660794279d0e1e44dea9726ff4fffc3..0000000000000000000000000000000000000000 --- a/docs/en/docs/EulerLauncher/mac-user-manual.md +++ /dev/null @@ -1,231 +0,0 @@ -# Installing and Running EulerLauncher on macOS - -## Preparations - -### Installing Homebrew - -Homebrew is the software package manager for macOS, which provides many practical functions, such as installation, uninstallation, update, viewing, and search. It allows you to use a simple command to manage packages without considering dependencies and file paths. - -On the macOS desktop, press **Shift**+**Command**+**U** to open the **Utilities** folder in **Go** and find **Terminal.app**. - - -Install Homebrew based on the network status. - -Run the following command to install Homebrew: - -``` Shell -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -``` - -Alternatively, run the following command to install Homebrew: - -``` Shell -/bin/zsh -c "$(curl -fsSL https://gitee.com/cunkai/HomebrewCN/raw/master/Homebrew.sh)" -``` - -### Installing QEMU and Wget - -EulerLauncher depends on QEMU to run on macOS, and image download depends on Wget. Homebrew can be used to easily download and manage such software. Run the following command to install QEMU and Wget: - -``` Shell -brew install qemu -brew install wget -``` - -### Configuring the sudo Password-Free Permission - -EulerLauncher depends on QEMU to run on macOS. To improve user network experience, [vmnet framework][1] of macOS is used to provide VM network capabilities. Currently, administrator permissions are required for using vmnet. When using the QEMU backend to create VMs with vmnet network devices, you need to enable the administrator permission. EulerLauncher automatically uses the `sudo` command to implement this process during startup. Therefore, you need to configure the `sudo` password-free permission for the current user. If you do not want to perform this configuration, please stop using EulerLauncher. - -1. On the macOS desktop, press **Shift**+**Command**+**U** to open the **Utilities** folder in **Go** and find **Terminal.app**. - - -2. Enter `sudo visudo` in the terminal to modify the sudo configuration file. Note that you may be required to enter the password in this step. Enter the password as prompted. - -3. Find and replace `%admin ALL=(ALL) ALL` with `%admin ALL=(ALL) NOPASSWD: ALL`. - - -4. Press **ESC** and enter **:wq** to save the settings. - -## Installing EulerLauncher - -EulerLauncher supports macOS Ventura for Apple Silicon and x86 architectures. [Download the latest version of EulerLauncher][1] for macOS and decompress it to the desired location. - -The directory generated after the decompression contains the following files: - - - -**install.exe** is the installation file, which is used to install the support files required by EulerLauncher to the specified location. **EulerLauncher.dmg** is the disk image of the main program. - -1. This operation requires the `sudo` permission. You need to [configure the sudo password-free permission](#configuring-the-sudo-password-free-permission) first. -2. Install the support files. Double-click the **install** file and wait until the program execution is completed. - -3. Configure EulerLauncher. - - - Check the locations of QEMU and Wget. The name of the QEMU binary file varies according to the architecture. Select the correct name (Apple Silicon: **qemu-system-aarch64**; Intel: **qemu-system-x86_64**) as required. - - ``` Shell - which wget - which qemu-system-{host_arch} - ``` - - Reference output: - - ```shell - /opt/homebrew/bin/wget - /opt/homebrew/bin/qemu-system-aarch64 - ``` - - Record the paths, which will be used in the following steps. - - - Open the **eulerlauncher.conf** file and configure it. - - ```shell - sudo vi /Library/Application\ Support/org.openeuler.eulerlauncher/eulerlauncher.conf - ``` - - EulerLauncher configurations are as follows: - - ```shell - [default] - log_dir = # Log file location (xxx.log) - work_dir = # EulerLauncher working directory, which is used to store VM images and VM files. - wget_dir = # Path of the Wget executable file. Set this parameter based on the previous step. - qemu_dir = # Path of the QEMU executable file. Set this parameter based on the previous step. - debug = True - - [vm] - cpu_num = 1 # Number of CPUs of the VM. - memory = 1024 #Memory size of the VM, in MB. Do not set this parameter to a value greater than 2048 for M1 users. - ``` - - Save the modifications and exit. - -4. Install **EulerLauncher.app**. - - - Double-click **EulerLauncher.dmg**. In the displayed window, drag **EulerLauncher.app** to **Applications** to complete the installation. You can find **EulerLauncher.app** in applications. - - - -## Using EulerLauncher - -1. Find **EulerLauncher.app** in applications and click to start the program. - -2. EulerLauncher needs to access the network. When the following dialog box is displayed, click **Allow**. - - - -3. Currently, EulerLauncher can be accessed only in CLI mode. Open **Terminal.app** and use the CLI to perform operations. - -### Operations on Images - -1. List available images. - - ```Shell - eulerlauncher images - ``` - - There are two types of EulerLauncher images: remote images and local images. Only local images in the **Ready** state can be used to create VMs. Remote images can be used only after being downloaded. You can also load a downloaded local image to EulerLauncher. For details, see the following sections. - -2. Download a remote image. - - ```Shell - eulerlauncher download-image 23.09 - - Downloading: 23.09, this might take a while, please check image status with "images" command. - ``` - - The image download request is an asynchronous request. The download is completed in the background. The time required depends on your network status. The overall image download process includes download, decompression, and format conversion. During the download, you can run the `image` command to view the download progress and image status at any time. - - ```Shell - eulerlauncher images - ``` - - When the image status changes to **Ready**, the image is downloaded successfully. The image in the **Ready** state can be used to create VMs. - - ```Shell - eulerlauncher images - ``` - -3. Load a local image. - - Load a custom image or an image downloaded to the local host to EulerLauncher to create a custom VM. - - ```Shell - eulerlauncher load-image --path {image_file_path} IMAGE_NAME - ``` - - The supported image formats are *xxx***.qcow2.xz** and *xxx***.qcow2**. - - Example: - - ```Shell - eulerlauncher load-image --path /opt/openEuler-23.09-x86_64.qcow2.xz 2309-load - - Loading: 2309-load, this might take a while, please check image status with "images" command. - ``` - - Load the **openEuler-23.09-x86_64.qcow2.xz** file in the **/opt** directory to the EulerLauncher system and name it **2309-load**. Similar to the download command, the load command is also an asynchronous command. You need to run the image list command to query the image status until the image status is **Ready**. Compared with directly downloading an image, loading an image is much faster. - - ```Shell - eulerlauncher images - ...... - - eulerlauncher images - ...... - ``` - -4. Delete an image. - - Run the following command to delete an image from the EulerLauncher system: - - ```Shell - eulerlauncher delete-image 2309-load - - Image: 2309-load has been successfully deleted. - ``` - -### Operations on VMs - -1. List VMs. - - ```shell - eulerlauncher list - - +----------+-----------+---------+---------------+ - | Name | Image | State | IP | - +----------+-----------+---------+---------------+ - | test1 | 2309-load | Running | 172.22.57.220 | - +----------+-----------+---------+---------------+ - | test2 | 2309-load | Running | N/A | - +----------+-----------+---------+---------------+ - ``` - - If the VM IP address is **N/A** and the VM status is **Running**, the VM is newly created and the network configuration is not complete. Configuring the network takes several seconds. You can obtain the VM information again later. - -2. Log in to a VM. - - If an IP address has been assigned to a VM, you can run the `ssh` command to log in to the VM. - - ```Shell - ssh root@{instance_ip} - ``` - - If the official image provided by the openEuler community is used, the default username is **root** and the default password is **openEuler12#$**. - -3. Create a VM. - - ```Shell - eulerlauncher launch --image {image_name} {instance_name} - ``` - - Use `--image` to specify an image and a VM name. - -4. Delete a VM. - - ```Shell - eulerlauncher delete-instance {instance_name} - ``` - -Delete a specified VM based on the VM name. - -[1]: https://developer.apple.com/documentation/vmnet diff --git a/docs/en/docs/EulerLauncher/win-user-manual.md b/docs/en/docs/EulerLauncher/win-user-manual.md deleted file mode 100644 index be313bc7f06526d68e88e4586be4266cdc361a3d..0000000000000000000000000000000000000000 --- a/docs/en/docs/EulerLauncher/win-user-manual.md +++ /dev/null @@ -1,159 +0,0 @@ -# Installing and Running EulerLauncher on Windows - -EulerLauncher currently supports Windows 10/11. [Download the latest version of EulerLauncher][1] for Windows and decompress it to the desired location. -Right-click **config-env.bat** and choose **Run as administrator** from the shortcut menu. This script adds the current directory to the system environment variable `PATH`. If you know how to configure environment variables or the configuration script is incorrect, you can manually add the directory where the current script is located and the **qemu-img** subdirectory to the system environment variable `PATH`. - -To run EulerLauncher on Windows, you need to connect EulerLauncher to the Hyper-V virtualization backend. Hyper-V is a Microsoft hardware virtualization product that provides better performance for VMs on Windows. Before running EulerLauncher, check whether Hyper-V is enabled in your system. For details about how to check and enable Hyper-V, see [Install Hyper-V][2] or other network resources. - -The directory generated after the decompression contains the following files: - -- **eulerlauncherd.exe**: main process of EulerLauncher. It is a daemon process running in the background. It interacts with various virtualization backends and manages the life cycles of VMs, containers, and images. -- **eulerlauncher.exe**: EulerLauncher CLI client. You can use this client to interact with the eulerlauncherd daemon process and perform operations on VMs and images. -- **eulerlauncher.conf**: EulerLauncher configuration file, which must be stored in the same directory as **eulerlauncherd.exe**. Configure the file as follows: - -```text -[default] -# Configure the directory for storing log files. -log_dir = D:\eulerlauncher-workdir\logs -# Whether to enable the debug log level. -debug = True -# Configure the EulerLauncher working directory. -work_dir = D:\eulerlauncher-workdir -# Configure the EulerLauncher image directory. The image directory is relative to the working directory. -image_dir = images -# Configure the VM file directory of EulerLauncher. The VM file directory is relative to the working directory. -instance_dir = instances -``` - -After the configuration is complete, right-click **eulerlauncherd.exe** and choose **Run as administrator** from the shortcut menu. **eulerlauncherd.exe** runs as a daemon process in the background. - -Start PowerShell or Command Prompt to prepare for the corresponding operation. - -## Exiting the eulerlauncherd Background Process on Windows - -After **eulerlauncherd.exe** is executed, the eulerlauncherd icon is displayed in the system tray in the lower right corner. Right-click the icon and choose **Exit EulerLauncher** from the shortcut menu to exit the eulerlauncherd background process. - -## Operations on Images - -1. List available images. - - ```PowerShell - eulerlauncher.exe images - - +-----------+----------+--------------+ - | Images | Location | Status | - +-----------+----------+--------------+ - | 22.03-LTS | Remote | Downloadable | - | 21.09 | Remote | Downloadable | - | 2203-load | Local | Ready | - +-----------+----------+--------------+ - ``` - - There are two types of EulerLauncher images: remote images and local images. Only local images in the **Ready** state can be used to create VMs. Remote images can be used only after being downloaded. You can also load a downloaded local image to EulerLauncher. For details, see the following sections. - -2. Download a remote image. - - ```PowerShell - eulerlauncher.exe download-image 22.03-LTS - - Downloading: 22.03-LTS, this might take a while, please check image status with "images" command. - ``` - - The image download request is an asynchronous request. The download is completed in the background. The time required depends on your network status. The overall image download process includes download, decompression, and format conversion. During the download, you can run the `image` command to view the download progress and image status at any time. - - ```PowerShell - eulerlauncher.exe images - ``` - - When the image status changes to **Ready**, the image is downloaded successfully. The image in the **Ready** state can be used to create VMs. - - ```PowerShell - eulerlauncher.exe images - ``` - -3. Load a local image. - - Load a custom image or an image downloaded to the local host to EulerLauncher to create a custom VM. - - ```PowerShell - eulerlauncher.exe load-image --path {image_file_path} IMAGE_NAME - ``` - - The supported image formats are *xxx***.qcow2.xz** and *xxx***.qcow2**. - - Example: - - ```PowerShell - eulerlauncher.exe load-image --path D:\openEuler-23.09-x86_64.qcow2.xz 2309-load - - Loading: 2309-load, this might take a while, please check image status with "images" command. - ``` - - Load the **openEuler-22.03-LTS-x86_64.qcow2.xz** file in the **D:\\** directory to the EulerLauncher system and name it **2203-load**. Similar to the download command, the load command is also an asynchronous command. You need to run the image list command to query the image status until the image status is **Ready**. Compared with directly downloading an image, loading an image is much faster. - - ```PowerShell - eulerlauncher.exe images - - ...... - - eulerlauncher images - - ...... - ``` - -4. Delete an image. - - Run the following command to delete an image from the EulerLauncher system: - - ```PowerShell - eulerlauncher.exe delete-image 2309-load - - Image: 2309-load has been successfully deleted. - ``` - -## Operations on VMs - -1. List VMs. - - ```Powershell - eulerlauncher.exe list - - +----------+-----------+---------+---------------+ - | Name | Image | State | IP | - +----------+-----------+---------+---------------+ - | test1 | 2309-load | Running | 172.22.57.220 | - +----------+-----------+---------+---------------+ - | test2 | 2309-load | Running | N/A | - +----------+-----------+---------+---------------+ - ``` - - If the VM IP address is **N/A** and the VM status is **Running**, the VM is newly created and the network configuration is not complete. Configuring the network takes several seconds. You can obtain the VM information again later. - -2. Log in to a VM. - - If an IP address has been assigned to a VM, you can run the `ssh` command to log in to the VM. - - ```PowerShell - ssh root@{instance_ip} - ``` - - If the official image provided by the openEuler community is used, the default username is **root** and the default password is **openEuler12#$**. - -3. Create a VM. - - ```PowerShell - eulerlauncher.exe launch --image {image_name} {instance_name} - ``` - - Use `--image` to specify an image and a VM name. - -4. Delete a VM. - - ```PowerShell - eulerlauncher.exe delete-instance {instance_name} - ``` - - Delete a specified VM based on the VM name. - -[1]: https://gitee.com/openeuler/eulerlauncher/releases -[2]: https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v diff --git a/docs/en/docs/GCC/kernel_FDO_user_guide.md b/docs/en/docs/GCC/kernel_FDO_user_guide.md deleted file mode 100644 index 02c9ec2b151b0e06e05168bd6de78374be9acfd8..0000000000000000000000000000000000000000 --- a/docs/en/docs/GCC/kernel_FDO_user_guide.md +++ /dev/null @@ -1,69 +0,0 @@ -# Kernel FDO User Guide - -## Overview - -The feedback-directed optimization (FDO) of the kernel allows users to build optimized kernels for different applications to improve the application performance in single-application scenarios. In addition, FDO is integrated GCC for openEuler, and A-FOT provides automatic optimization, enabling users to easily enable FDO. - -## Installation and Deployment - -### Software Requirements - -* OS: openEuler 23.09 - -### Hardware Requirements - -* Architecture: AArch64 or x86_64 - -### Software Installation - -#### Downloading the Kernel Source Code - -Download the kernel source code, A-FOT, and dependency packages. - -```shell -yum install -y kernel-source A-FOT make gcc flex bison elfutils-libelf-devel diffutils openssl-devel dwarves -``` - -Copy the kernel source code. - -```shell -cp -r /usr/src/linux-6.4.0-8.0.0.16.oe2309.aarch64 . -``` - -**Note: Change the version number as required.** - -## Usage - -You can use A-FOT to enable kernel FDO and obtain the optimized kernel by specifying **opt_mode** as **Auto_kernel_PGO**. Other configuration items can be specified on the CLI, for example, `./a-fot --pgo_phase 1`. `-s` and `-n` options can be specified on CLI only. Options related to kernel FDO are as follows. - -| No.| Option (Configuration File)| Description | Default Value | -| ---- | -------------------- | ------------------------------------------------------------ | ------------------------ | -| 1 | config_file | Path of the configuration file. User configurations are read from this file. | ${afot_path}/a-fot.ini | -| 2 | opt_mode | Optimization mode to be executed by the tool. The value can be **AutoFDO**, **AutoPrefetch**, **AutoBOLT**, or **Auto_kernel_PGO**.| AutoPrefetch | -| 3 | pgo_mode | Kernel FDO mode, which can be **arc** (GCOV, using only the arc profile) or **all** (full PGO, using the arc and value profiles). | all | -| 4 | pgo_phase | FDO execution phase. The value can be **1** (kernel instrumentation phase) or **2** (data collection and kernel optimization phase). | 1 | -| 5 | kernel_src | Kernel source code directory. If this option is not specified, the tool automatically downloads the source code. | None (optional) | -| 6 | kernel_name | File name of the kernel build. The tool will add the **-pgoing** or **-pgoed** suffix depending on the phase. | kernel | -| 7 | work_path | Script working directory, which is used to store log files, wrappers, and profiles. | **/opt** (**/tmp** cannot be used.)| -| 8 | run_script | Application execution script. The user needs to write the script, which will be used by the tool to execute the target application.| /root/run.sh | -| 9 | gcc_path | GCC path. | /usr | -| 10 | -s | Silent mode. The tool automatically reboots and switches the kernel to execute the second phase. | None | -| 11 | -n | The tool is not used to compile the kernel. This option applies to the scenario where the execution environment and kernel compilation environment are different. | None | - -After configuring the compilation options, run the following command to use A-FOT to automatically optimize the kernel: - -```shell -a-fot --config_file ./a-fot.ini -s -``` - -**Note: The `-s` option instructs A-FOT to automatically reboot into the compiled kernel. If you do not want the tool to automatically perform this sensitive operation, omit this option. However, you need to manually reboot and perform the second phase (`--pgo_phase 2`).** - -**Note: All paths must be absolute paths.** - -**Note: The kernel of openEuler 23.09 does not support full PGO. Change the value of pgo_mode to arc.** - -## Compatibility - -This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and issues will be fixed as soon as possible. Developers are welcome to join this project. - -* The kernel of openEuler 23.09 does not support full PGO. Change the value of pgo_mode to arc. diff --git a/docs/en/docs/GCC/overview.md b/docs/en/docs/GCC/overview.md deleted file mode 100644 index 9ac7d9b17c68f7897a0675a7116e671dbc94aafe..0000000000000000000000000000000000000000 --- a/docs/en/docs/GCC/overview.md +++ /dev/null @@ -1,3 +0,0 @@ -# GCC for openEuler User Guide - -The GCC for openEuler compiler is developed based on the open source GNU compiler Collection (GCC). The open source GCC is the de facto standard of cross-platform compilers, and it complies with the GPLv3 license, becoming the most widely used C/C++ compiler on Linux. GCC for openEuler inherits capabilities of the open source GCC. It also has optimizations on C, C++, and Fortran languages and delivers enhanced features such as automatic feedback-directed optimization (FDO), software and hardware collaboration, memory optimization, and automatic vectorization. GCC for openEuler is compatible with a wide range of hardware platforms such as Kunpeng, Phytium, and Loongson, fully unleashing the computing power of these hardware platforms. diff --git a/docs/en/docs/GMEM/install_deploy.md b/docs/en/docs/GMEM/install_deploy.md deleted file mode 100644 index 0a9991c8a70919e8a69fd3b8820d4bdb0b8f6984..0000000000000000000000000000000000000000 --- a/docs/en/docs/GMEM/install_deploy.md +++ /dev/null @@ -1,116 +0,0 @@ -# Installation and Deployment - -This section describes how to install and deploy GMEM. - -## Software and Hardware Requirements - -* Kunpeng 920 CPU -* Ascend 910 processor -* openEuler 23.09 - -## Environment Requirements - -* The root permission is required for using and configuring GMEM. -* GMEM can be enabled or disabled only at the system level. -* The administrator must ensure that the GMEM configuration is safe and available. - -## Installing GMEM - -* Prepare files. - - [CANN Community Version History - Ascend Community (hiascend.com)](https://www.hiascend.com/en/software/cann/community-history) - - [Firmware and Driver - Ascend Community (hiascend.com)](https://www.hiascend.com/en/hardware/firmware-drivers/community?product=2&model=19&cann=6.0.1.alpha001&driver=1.0.18.alpha) - - | Source | Software Package | - | ------------------------------------------------------------ | ------------------------------------------------------------ | - | openEuler 23.09 | kernel-6.4.0-xxx.aarch64.rpm
kernel-devel-6.4.0-xxx.aarch64.rpm
libgmem-xxx.aarch64.rpm
libgmem-devel-xxx.aarch64.rpm | - | Ascend community | CANN package:
Ascend-cann-toolkit-xxx-linux.aarch64.rpm
NPU firmware and driver:
Ascend-hdk-910-npu-driver-xxx.aarch64.rpm
Ascend-hdk-910-npu-firmware-xxx.noarch.rpm | - | Contact the maintainers of the GMEM community.
[@yang_yanchao](https://gitee.com/yang_yanchao) email:
[@LemmyHuang](https://gitee.com/LemmyHuang) email: | gmem-example-xxx.aarch64.rpm
mindspore-xxx-linux_aarch64.whl | - -* Install the kernel. - - Ensure that GMEM compilation options are enabled (enabled by default) for the openEuler kernel. - - ```sh - [root@localhost ~]# cat /boot/config-`uname -r` | grep CONFIG_GMEM - CONFIG_GMEM=y - CONFIG_GMEM_DEV=m - - [root@localhost ~]# cat /boot/config-`uname -r` | grep CONFIG_REMOTE_PAGER - CONFIG_REMOTE_PAGER=m - CONFIG_REMOTE_PAGER_MASTER=m - ``` - - Add **gmem=on** to the boot options. - - ```sh - [root@localhost gmem]# cat /proc/cmdline - BOOT_IMAGE=/vmlinuz-xxx root=/dev/mapper/openeuler-root ... gmem=on - ``` - - Configure **transparent_hugepage**. - - ```sh - echo always > /sys/kernel/mm/transparent_hugepage/enabled - ``` - -* Install the user-mode dynamic library libgmem. - - ```sh - yum install libgmem libgmem-devel - ``` - -* Install the CANN framework. - - Install the matching CANN, including the toolkit, driver, and firmware. After the installation is complete, restart the system. - - ```sh - rpm -ivh Ascend-cann-toolkit-xxx-linux.aarch64.rpm - # Use the tool provided by libgmem to install the NPU driver. - sh /usr/local/gmem/install_npu_driver.sh Ascend-hdk-910-npu-driver-xxx.aarch64.rpm - rpm -ivh Ascend-hdk-910-npu-firmware-xxx.noarch.rpm - ``` - - Run the environment configuration script in the **Ascend** directory to configure environment variables. - - ```sh - source /usr/local/Ascend/ascend-toolkit/set_env.sh - ``` - - Check whether the NPU is working properly. - - ```sh - [root@localhost ~]# npu-smi info - +-------------------------------------------------------------------------------------------+ - | npu-smi 22.0.4.1 Version: 22.0.4.1 | - +----------------------+---------------+----------------------------------------------------+ - | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)| - | Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) | - +======================+===============+====================================================+ - | 0 910B | OK | 79.4 82 0 / 0 | - | 0 | 0000:81:00.0 | 0 1979 / 15039 0 / 32768 | - +======================+===============+====================================================+ - ``` - -* Install the gmem-example software package. - - gmem-example updates the host driver, NPU driver, and NPU kernel. After the installation is complete, restart the system for the driver to take effect. - - ```sh - rpm -ivh gmem-example-xxx.aarch64.rpm - ``` - -* Install MindSpore. - - Obtain the correct MindSpore version and install it. After the installation, run the following command to check whether MindSpore functions are normal: - - ```sh - python -c "import mindspore;mindspore.run_check()" - MindSpore version: x.x.x - The result of multiplication calculation is correct, MindSpore has been installed on platform [Ascend] successfully! - ``` - -## Performing Training or Inference - -After installation is complete, you can execute MindSpore-based training or inference directly without any adaptation. diff --git a/docs/en/docs/GMEM/usage.md b/docs/en/docs/GMEM/usage.md deleted file mode 100644 index 5203c52e2146c04277e135b8a0de40eec7337d5d..0000000000000000000000000000000000000000 --- a/docs/en/docs/GMEM/usage.md +++ /dev/null @@ -1,66 +0,0 @@ -# Usage Instructions - -## Introduction - -GMEM applies for virtual memory that is peer-to-peer accessible through a specific flag and provides some memory optimization semantics externally. Performance can be optimized through the memory semantics. -libgmem is the abstraction layer of the GMEM user API. It encapsulates the preceding memory semantics to simplify user operations. - -## Available APIs - -* Memory application - - GMEM extends `mmap` semantics and adds a MAP_PEER_SHARED flag to apply for heterogeneous memory. When the flag is used, 2 MB-aligned virtual addresses are returned by default. - - ```c - addr = mmap(NULL , size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_PEER_SHARED, -1, 0); - ``` - -* Memory release - - `munmap` is used to release memory of hosts and devices. - - ```c - munmap(addr, size); - ``` - -* Memory semantics - - `FreeEager`: For an address segment within the specified range \[**addr**, **addr** + **size**], `FreeEager` releases a complete page that aligns the page size inwards (the default page size is 2 MB). If no complete page exists in the range, a success message is returned. - - If the API is invoked successfully, 0 is returned. Otherwise, an error code is returned. - - ```c - Prototype: `int gmemFreeEager(unsigned long addr, size_t size, void *stream);` - Usage: `ret = gmemFreeEager(addr, size, stream);` - ``` - - `Prefetch`: For an address segment with the specified range \[**addr**, **addr** + **size**], `Prefetch` prefetches a complete page (covering the entire address segment) whose range is outward aligned with the page size. This ensures that the subsequent access to the virtual memory area initiated by the specified computing unit device **hnid** does not trigger a page fault. - - If the API is invoked successfully, 0 is returned. Otherwise, an error code is returned. - - ```c - Prototype: `int gmemPrefetch(unsigned long addr, size_t size, int hnid, void *stream);` - Usage: `ret = gmemPrefetch(addr, size, hnid, stream);` - ``` - - If the value of **stream** is empty, the invocation is synchronous. Otherwise, the invocation is asynchronous. - -* Other APIs - - Obtain the NUMA ID of the current device. If the API is invoked successfully, the NUMA ID is returned. Otherwise, an error code is returned. - - ```c - Prototype: `int gmemGetNumaId (void);` - Usage: `numaid = gmemGetNumaId ();` - ``` - - Obtain the GMEM statistics of the kernel. - - ```sh - cat /proc/gmemstat - ``` - -## Constraints - -1. GMEM supports only 2 MB huge pages. Therefore, transparent huge pages of the host OS and NPU OS must be enabled to use GMEM. -2. The heterogeneous memory obtained using `MAP_PEER_SHARED` cannot be inherited during forking. diff --git a/docs/en/docs/Gazelle/Gazelle.md b/docs/en/docs/Gazelle/Gazelle.md deleted file mode 100644 index c93925431235edb7093763d32c66211903a4dee4..0000000000000000000000000000000000000000 --- a/docs/en/docs/Gazelle/Gazelle.md +++ /dev/null @@ -1,285 +0,0 @@ -# Gazelle User Guide - -## Overview - -Gazelle is a high-performance user-mode protocol stack. It directly reads and writes NIC packets in user mode based on DPDK and transmit the packets through shared hugepage memory, and uses the LwIP protocol stack. Gazelle greatly improves the network I/O throughput of applications and accelerates the network for the databases, such as MySQL and Redis. - -- High Performance -Zero-copy and lock-free packets that can be flexibly scaled out and scheduled adaptively. -- Universality -Compatible with POSIX without modification, and applicable to different types of applications. - -In the single-process scenario where the NIC supports multiple queues, use **liblstack.so** only to shorten the packet path. - -## Installation - -Configure the Yum source of openEuler and run the`yum` command to install Gazelle. - -```sh -yum install dpdk -yum install libconfig -yum install numactl -yum install libboundscheck -yum install libpcap -yum install gazelle -``` - ->NOTE: -The version of dpdk must be 21.11-2 or later. - -## How to Use - -To configure the operating environment and use Gazelle to accelerate applications, perform the following steps: - -### 1. Installing the .ko File as the root User - -Install the .ko files based on the site requirements to bind NICs to the user-mode driver. - -Bind the NIC from the kernel driver to the user-mode driver. Choose one of the following .ko files based on the site requirements. - -```sh -#If the IOMMU is available -modprobe vfio-pci - -#If the IOMMU is not available and the VFIO supports the no-IOMMU mode -modprobe vfio enable_unsafe_noiommu_mode=1 -modprobe vfio-pci - -#Other cases -modprobe igb_uio -``` - ->NOTE: -You can check whether the IOMMU is enabled based on the BIOS configuration. - -### 2. Binding the NIC Using DPDK - -Bind the NIC to the driver selected in Step 1 to provide an interface for the user-mode NIC driver to access the NIC resources. - -```sh -#Using vfio-pci -dpdk-devbind -b vfio-pci enp3s0 - -#Using igb_uio -dpdk-devbind -b igb_uio enp3s0 -``` - -### 3. Configuring Memory Huge Pages - -Gazelle uses hugepage memory to improve efficiency. You can configure any size for the memory huge pages reserved by the system using the **root** permissions. Each memory huge page requires a file descriptor. If the memory is large, you are advised to use 1 GB huge pages to avoid occupying too many file descriptors. -Select a page size based on the site requirements and configure sufficient memory huge pages. Run the following commands to configure huge pages: - -```sh -#Configuring 1024 2 MB huge pages on node0. The total memory is 2 GB. -echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages - -#Configuring 5 1 GB huge pages on node0. The total memory is 5 GB. -echo 5 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages -``` - ->NOTE: -Run the **cat** command to query the actual number of reserved pages. If the continuous memory is insufficient, the number may be less than expected. - -### 4. Mounting Memory Huge Pages - -Create a directory for the lstack process to access the memory huge pages. Run the following commands: - -```sh -mkdir -p /mnt/hugepages-lstack -chmod -R 700 /mnt/hugepages-lstack - -mount -t hugetlbfs nodev /mnt/hugepages-lstack -o pagesize=2M -``` - -### 5. Enabling Gazelle for an Application - -Enable Gazelle for an application using either of the following methods as required. - -- Recompile the application and replace the sockets interface. - -```sh -#Add the Makefile of Gazelle to the application makefile. --include /etc/gazelle/lstack.Makefile - -#Add the LSTACK_LIBS variable when compiling the source code. -gcc test.c -o test ${LSTACK_LIBS} -``` - -- Use the **LD_PRELOAD** environment variable to load the Gazelle library. -Use the **GAZELLE_BIND_PROCNAME** environment variable to specify the process name, and **LD_PRELOAD** to specify the Gazelle library path. - -```sh -GAZELLE_BIND_PROCNAME=test LD_PRELOAD=/usr/lib64/liblstack.so ./test -``` - -- Use the **GAZELLE_THREAD_NAME** environment variable to specify the thread bound to Gazelle. -If only one thread of a multi-thread process meets the conditions for using Gazelle, use **GAZELLE_THREAD_NAME** to specify the thread for using Gazelle. Other threads use kernel-mode protocol stack. - -```sh -GAZELLE_BIND_PROCNAME=test GAZELLE_THREAD_NAME=test_thread LD_PRELOAD=/usr/lib64/liblstack.so ./test -``` - -### 6. Configuring Gazelle - -- The **lstack.conf** file is used to specify the startup parameters of lstack. The default path is **/etc/gazelle/lstack.conf**. The parameters in the configuration file are as follows: - -|Options|Value|Remarks| -|:---|:---|:---| -|dpdk_args|--socket-mem (mandatory)
--huge-dir (mandatory)
--proc-type (mandatory)
--legacy-mem
--map-perfect
-d|DPDK initialization parameter. For details, see the DPDK description.
**--map-perfect** is an extended feature. It is used to prevent the DPDK from occupying excessive address space and ensure that extra address space is available for lstack.
The **-d** option is used to load the specified .so library file.| -|listen_shadow| 0/1 | Whether to use the shadow file descriptor for listening. This function is enabled when there is a single listen thread and multiple protocol stack threads.| -|use_ltran| 0/1 | Whether to use ltran. This parameter is no longer supported.| -|num_cpus|"0,2,4 ..."|IDs of the CPUs bound to the lstack threads. The number of IDs is the number of lstack threads (less than or equal to the number of NIC queues). You can select CPUs by NUMA nodes.| -|low_power_mode|0/1|Whether to enable the low-power mode. This parameter is not supported currently.| -|kni_switch|0/1|Whether to enable the rte_kni module. The default value is **0**. This parameter is no longer supported.| -|unix_prefix|"string"|Prefix string of the Unix socket file used for communication between Gazelle processes. By default, this parameter is left blank. The value must be the same as the value of **unix_prefix** in **ltran.conf** of the ltran process that participates in communication, or the value of the **-u** option for `gazellectl`. The value cannot contain special characters and can contain a maximum of 128 characters.| -|host_addr|"192.168.xx.xx"|IP address of the protocol stack, which is also the IP address of the application.| -|mask_addr|"255.255.xx.xx"|Subnet mask.| -|gateway_addr|"192.168.xx.1"|Gateway address.| -|devices|"aa:bb:cc:dd:ee:ff"|MAC address for NIC communication. The NIC is used as the primary bond NIC in bond 1 mode. | -|app_bind_numa|0/1|Whether to bind the epoll and poll threads of an application to the NUMA node where the protocol stack is located. The default value is 1, indicating that the threads are bound.| -|send_connect_number|4|Number of connections for sending packets in each protocol stack loop. The value is a positive integer.| -|read_connect_number|4|Number of connections for receiving packets in each protocol stack loop. The value is a positive integer.| -|rpc_number|4|Number of RPC messages processed in each protocol stack loop. The value is a positive integer.| -|nic_read_num|128|Number of data packets read from the NIC in each protocol stack cycle. The value is a positive integer.| -|bond_mode|-1|Bond mode. Currently, two network ports can be bonded. The default value is -1, indicating that the bond mode is disabled. bond1/4/6 is supported.| -|bond_slave_mac|"aa:bb:cc:dd:ee:ff;AA:BB:CC:DD:EE:FF"|MAC addresses of the bond network ports. Separate the MAC addresses with semicolons (;).| -|bond_miimon|10|Listening interval in bond mode. The default value is 10. The value ranges from 0 to 1500.| -|udp_enable|0/1|Whether to enable the UDP function. The default value is 1.| -|nic_vlan_mode|-1|Whether to enable the VLAN mode. The default value is -1, indicating that the VLAN mode is disabled. The value ranges from -1 to 4095. IDs 0 and 4095 are commonly reserved in the industry and have no actual effect.| -|tcp_conn_count|1500|Maximum number of TCP connections. The value of this parameter multiplied by **mbuf_count_per_conn** is the size of the mbuf pool applied for during initialization. If the value is too small, the startup fails. The value of (**tcp_conn_count** x **mbuf_count_per_conn** x 2048) cannot be greater than the huge page size.| -|mbuf_count_per_conn|170|Number of mbuf required by each TCP connection. The value of this parameter multiplied by **tcp_conn_count** is the size of the mbuf address pool applied for during initialization. If the value is too small, the startup fails. The value of (**tcp_conn_count** x **mbuf_count_per_conn** x 2048) cannot be greater than the huge page size.| - -lstack.conf example: - -```sh -dpdk_args=["--socket-mem", "2048,0,0,0", "--huge-dir", "/mnt/hugepages-lstack", "--proc-type", "primary", "--legacy-mem", "--map-perfect"] - -use_ltran=1 -kni_switch=0 - -low_power_mode=0 - -num_cpus="2,22" -num_wakeup="3,23" - -host_addr="192.168.1.10" -mask_addr="255.255.255.0" -gateway_addr="192.168.1.1" -devices="aa:bb:cc:dd:ee:ff" - -send_connect_number=4 -read_connect_number=4 -rpc_number=4 -nic_read_num=128 -mbuf_pool_size=1024000 -bond_mode=1 -bond_slave_mac="aa:bb:cc:dd:ee:ff;AA:BB:CC:DD:EE:FF" -udp_enable=1 -nic_vlan_mode=-1 -``` - -- The ltran mode is deprecated. If multiple processes are required, try the virtual network mode using SR-IOV network hardware. - -### 7. Starting an Application - -- Start the application. -If the environment variable **LSTACK_CONF_PATH** is not used to specify the configuration file before the application is started, the default configuration file path **/etc/gazelle/lstack.conf** is used. - -```sh -export LSTACK_CONF_PATH=./lstack.conf -LD_PRELOAD=/usr/lib64/liblstack.so GAZELLE_BIND_PROCNAME=redis-server redis-server redis.conf -``` - -### 8. APIs - -Gazelle wraps the POSIX interfaces of the application. The code of the application does not need to be modified. - -### 9. Debugging Commands - -```sh -Usage: gazellectl [-h | help] - or: gazellectl lstack {show | set} {ip | pid} [LSTACK_OPTIONS] [time] [-u UNIX_PREFIX] - - where LSTACK_OPTIONS := - show lstack all statistics - -r, rate show lstack statistics per second - -s, snmp show lstack snmp - -c, connetct show lstack connect - -l, latency show lstack latency - -x, xstats show lstack xstats - -k, nic-features show state of protocol offload and other feature - -a, aggregatin [time] show lstack send/recv aggregation - set: - loglevel {error | info | debug} set lstack loglevel - lowpower {0 | 1} set lowpower enable - [time] measure latency time default 1S -``` - -The `-u` option specifies the prefix of the Unix socket for communication between Gazelle processes. The value of this parameter must be the same as that of **unix_prefix** in the **lstack.conf** file. - -**Packet Capturing Tool** -The NIC used by Gazelle is managed by DPDK. Therefore, tcpdump cannot capture Gazelle packets. As a substitute, Gazelle uses gazelle-pdump provided in the dpdk-tools software package as the packet capturing tool. gazelle-pdump uses the multi-process mode of DPDK to share memory with the lstack process. -[Usage](https://gitee.com/openeuler/gazelle/blob/master/doc/pdump/pdump.md) - -**Thread Binding** -When the starting a lstack process, you can specify a thread bound to lstack using the environment variable **GAZELLE_THREAD_NAME**. When there are multiple threads in the service process, you can use this variable to specify the thread whose network interface needs to be managed by lstack. Other threads will use the kernel-mode protocol stack. By default, this parameter is left blank, that is, all threads in the process are bound. - -### 10. Precautions - -### 1. Location of the DPDK Configuration File - -For the **root** user, the configuration file is stored in the **/var/run/dpdk** directory after the DPDK is started. -For a non-root user, the path of the DPDK configuration file is determined by the environment variable **XDG_RUNTIME_DIR**. - -- If **XDG_RUNTIME_DIR** is not set, the DPDK configuration file is stored in **/tmp/dpdk**. -- If **XDG_RUNTIME_DIR** is set, the DPDK configuration file is stored in the path specified by **XDG_RUNTIME_DIR**. -- Note that **XDG_RUNTIME_DIR** is set by default on some servers. - -## Restrictions - -Restrictions of Gazelle are as follows: - -### Function Restrictions - -- Blocking **accept()** or **connect()** is not supported. -- A maximum of 1500 TCP connections are supported. -- Currently, only TCP, ICMP, ARP, IPv4, and UDP are supported. -- When a peer end pings Gazelle, the specified packet length must be less than or equal to 14,000 bytes. -- Transparent huge pages are not supported. -- VM NICs do not support multiple queues. - -### Operation Restrictions - -- By default, the command lines and configuration files provided by Gazelle requires **root** permissions. Privilege escalation and changing of file owner are required for non-root users. -- To bind the NIC from user-mode driver back to the kernel driver, you must exit Gazelle first. -- Memory huge pages cannot be remounted to subdirectories created in the mount point. -- The minimum hugepage memory of each application instance protocol stack thread is 800 MB. -- Gazelle supports only 64-bit OSs. -- The `-march=native` option is used when building the x86 version of Gazelle to optimize Gazelle based on the CPU instruction set of the build environment (Intel® Xeon® Gold 5118 CPU @ 2.30GHz). Therefore, the CPU of the operating environment must support the SSE4.2, AVX, AVX2, and AVX-512 instruction set extensions. -- The maximum number of IP fragments is 10 (the maximum ping packet length is 14,790 bytes). TCP does not use IP fragments. -- You are advised to set the **rp_filter** parameter of the NIC to 1 using the `sysctl` command. Otherwise, the Gazelle protocol stack may not be used as expected. Instead, the kernel protocol stack is used. -- The hybrid bonding of multiple types of NICs is not supported. -- The active/standby mode (bond1 mode) supports active/standby switchover only when a fault occurs at the link layer (for example, the network cable is disconnected), but does not support active/standby switchover when a fault occurs at the physical layer (for example, the NIC is powered off or removed). -- If the length of UDP packets to be sent exceeds 45952 (32 x 1436) bytes, increase the value of **send_ring_size** to at least 64. - -## Precautions - -You need to evaluate the use of Gazelle based on application scenarios. - -The ltran mode and kni module is no longer supported due to changes in the dependencies and upstream community. - -**Shared Memory** - -- Current situation: - The memory huge pages are mounted to the **/mnt/hugepages-lstack** directory. During process initialization, files are created in the **/mnt/hugepages-lstack** directory. Each file corresponds to a huge page, and the mmap function is performed on the files. -- Current mitigation measures - The huge page file permission is **600**. Only the owner can access the files. The default owner is the **root** user. Other users can be configured. - Huge page files are locked by DPDK and cannot be directly written or mapped. -- Caution - Malicious processes belonging to the same user imitate the DPDK implementation logic to share huge page memory using huge page files and perform write operations to damage the huge page memory. As a result, the Gazelle program crashes. It is recommended that the processes of a user belong to the same trust domain. - -**Traffic Limit** -Gazelle does not limit the traffic. Users can send packets at the maximum NIC line rate to the network, which may congest the network. - -**Process Spoofing** -Ensure that all lstack processes are trusted. diff --git a/docs/en/docs/HSAK/develop_with_hsak.md b/docs/en/docs/HSAK/develop_with_hsak.md deleted file mode 100644 index cbd2bb70c1b451fafd4baade6878ffcd30d7ee95..0000000000000000000000000000000000000000 --- a/docs/en/docs/HSAK/develop_with_hsak.md +++ /dev/null @@ -1,229 +0,0 @@ -# Instructions - -## **nvme.conf.in** Configuration File - -By default, the HSAK configuration file is located in **/etc/spdk/nvme.conf.in**. You can modify the configuration file based on service requirements. The content of the configuration file is as follows: - -- \[Global\] - -1. **ReactorMask**: cores used for I/O polling. The value is a hexadecimal number and cannot be set to core 0. The bits from the least significant one to the most significant one indicate different CPU cores. For example, 0x1 indicates core 0, and 0x6 indicates cores 1 and 2. This parameter supports a maximum of 34 characters, including the hexadecimal flag **0x**. Each hexadecimal character can be F at most, indicating four cores. Therefore, a maximum of 128 (32 x 4) cores are supported. -2. **LogLevel**: HSAK log print level (**0**: error; **1**: warning; **2**: notice; **3**: info; **4**: debug). -3. **MemSize**: memory occupied by HSAK (The minimum value is 500 MB.) -4. **MultiQ**: whether to enable multi-queue on the same block device. -5. **E2eDif**: DIF type (**1**: half-way protection; **2**: full protection). Drives from different vendors may have different DIF support capabilities. For details, see the documents provided by hardware vendors. -6. **IoStat**: whether to enable the I/O statistics function. The options are **Yes** and **No**. -7. **RpcServer**: whether to start the RPC listening thread. The options are **Yes** and **No**. -8. **NvmeCUSE**: whether to enable the CUSE function. The options are **Yes** and **No**. After the function is enabled, the NVMe character device is generated in the **/dev/spdk** directory. - -- \[Nvme\] - -1. **TransportID**: PCI address and name of the NVMe controller. The format is **TransportID "trtype:PCIe traddr:0000:09:00.0" nvme0**. -2. **RetryCount**: number of retries upon an I/O failure. The value **0** indicates no retry. The maximum value is **255**. -3. **TimeoutUsec**: I/O timeout interval. If this parameter is set to **0** or left blank, no timeout interval is set. The unit is μs. -4. **ActionOnTimeout**: I/O timeout behavior (**None**: prints information only; **Reset**: resets the controller; **abort**: aborts the command). The default value is **None**. - -- \[Reactor\] - -1. **BatchSize**: number of I/Os that can be submitted in batches. The default value is **8**, and the maximum value is **32**. - -## Header File Reference - -HSAK provides two external header files. Include the two files when using HSAK for development. - -1. **bdev_rw.h**: defines the macros, enumerations, data structures, and APIs of the user-mode I/O operations on the data plane. -2. **ublock.h**: defines macros, enumerations, data structures, and APIs for functions such as device management and information obtaining on the management plane. - -## Service Running - -After software development and compilation, you must run the **setup.sh** script to rebind the NVMe drive driver to the user mode before running the software. The script is located in **/opt/spdk** by default. -Run the following commands to change the drive driver's binding mode from kernel to user and reserve 1024 x 2 MB huge pages: - -```shell -[root@localhost ~]# cd /opt/spdk -[root@localhost spdk]# ./setup.sh -0000:3f:00.0 (8086 2701): nvme -> uio_pci_generic -0000:40:00.0 (8086 2701): nvme -> uio_pci_generic -``` - -Run the following commands to restore the drive driver's mode from user to kernel and free the reserved huge pages: - -```shell -[root@localhost ~]# cd /opt/spdk -[root@localhost spdk]# ./setup.sh reset -0000:3f:00.0 (8086 2701): uio_pci_generic -> nvme -0000:40:00.0 (8086 2701): uio_pci_generic -> nvme -``` - -## User-Mode I/O Read and Write Scenarios - -Call HSAK APIs in the following sequence to read and write service data through the user-mode I/O channel: - -1. Initialize the HSAK UIO module. - Call **libstorage_init_module** to initialize the HSAK user-mode I/O channel. - -2. Open a drive block device. - Call **libstorage_open** to open a specified block device. If multiple block devices need to be opened, call this API repeatedly. - -3. Allocate I/O memory. - Call **libstorage_alloc_io_buf** or **libstorage_mem_reserve** to allocate memory. **libstorage_alloc_io_buf** can allocate a maximum of 65 KB I/Os, and **libstorage_mem_reserve** can allocate unlimited memory unless there is no available space. - -4. Perform read and write operations on a drive. - You can call the following APIs to perform read and write operations based on service requirements: - - - libstorage_async_read - - libstorage_async_readv - - libstorage_async_write - - libstorage_async_writev - - libstorage_sync_read - - libstorage_sync_write - -5. Free I/O memory. - Call **libstorage_free_io_buf** or **libstorage_mem_free** to free memory, which must correspond to the API used to allocate memory. - -6. Close a drive block device. - Call **libstorage_close** to close a specified block device. If multiple block devices are opened, call this API repeatedly to close them. - - | API | Description | - | ----------------------- | ------------------------------------------------------------ | - | libstorage_init_module | Initializes the HSAK module. | - | libstorage_open | Opens a block device. | - | libstorage_alloc_io_buf | Allocates memory from buf_small_pool or buf_large_pool of SPDK. | - | libstorage_mem_reserve | Allocates memory space from the huge page memory reserved by DPDK. | - | libstorage_async_read | Delivers asynchronous I/O read requests (the read buffer is a contiguous buffer). | - | libstorage_async_readv | Delivers asynchronous I/O read requests (the read buffer is a discrete buffer). | - | libstorage_async_write | Delivers asynchronous I/O write requests (the write buffer is a contiguous buffer). | - | libstorage_async_wrtiev | Delivers asynchronous I/O write requests (the write buffer is a discrete buffer). | - | libstorage_sync_read | Delivers synchronous I/O read requests (the read buffer is a contiguous buffer). | - | libstorage_sync_write | Delivers synchronous I/O write requests (the write buffer is a contiguous buffer). | - | libstorage_free_io_buf | Frees the allocated memory to buf_small_pool or buf_large_pool of SPDK. | - | libstorage_mem_free | Frees the memory space that libstorage_mem_reserve allocates. | - | libstorage_close | Closes a block device. | - | libstorage_exit_module | Exits the HSAK module. | - -## Drive Management Scenarios - -HSAK contains a group of C APIs, which can be used to format drives and create and delete namespaces. - -1. Call the C API to initialize the HSAK UIO component. If the HSAK UIO component has been initialized, skip this operation. - - libstorage_init_module - -2. Call corresponding APIs to perform drive operations based on service requirements. The following APIs can be called separately: - - - libstorage_create_namespace - - - libstorage_delete_namespace - - - libstorage_delete_all_namespace - - - libstorage_nvme_create_ctrlr - - - libstorage_nvme_delete_ctrlr - - - libstorage_nvme_reload_ctrlr - - - libstorage_low_level_format_nvm - - - libstorage_deallocate_block - -3. If you exit the program, destroy the HSAK UIO. If other services are using the HSAK UIO, you do not need to exit the program and destroy the HSAK UIO. - - libstorage_exit_module - - | API | Description | - | ------------------------------- | ------------------------------------------------------------ | - | libstorage_create_namespace | Creates a namespace on a specified controller (the prerequisite is that the controller supports namespace management). | - | libstorage_delete_namespace | Deletes a namespace from a specified controller. | - | libstorage_delete_all_namespace | Deletes all namespaces from a specified controller. | - | libstorage_nvme_create_ctrlr | Creates an NVMe controller based on the PCI address. | - | libstorage_nvme_delete_ctrlr | Destroys an NVMe controller based on the controller name. | - | libstorage_nvme_reload_ctrlr | Automatically creates or destroys the NVMe controller based on the input configuration file. | - | libstorage_low_level_format_nvm | Low-level formats an NVMe drive. | - | libstorage_deallocate_block | Notifies NVMe drives of blocks that can be freed for garbage collection. | - -## Data-Plane Drive Information Query - -The I/O data plane of HSAK provides a group of C APIs for querying drive information. Upper-layer services can process service logic based on the queried information. - -1. Call the C API to initialize the HSAK UIO component. If the HSAK UIO component has been initialized, skip this operation. - - libstorage_init_module - -2. Call corresponding APIs to query information based on service requirements. The following APIs can be called separately: - - - libstorage_get_nvme_ctrlr_info - - - libstorage_get_mgr_info_by_esn - - - libstorage_get_mgr_smart_by_esn - - - libstorage_get_bdev_ns_info - - - libstorage_get_ctrl_ns_info - -3. If you exit the program, destroy the HSAK UIO. If other services are using the HSAK UIO, you do not need to exit the program and destroy the HSAK UIO. - - libstorage_exit_module - - | API | Description | - | ------------------------------- | ------------------------------------------------------------ | - | libstorage_get_nvme_ctrlr_info | Obtains information about all controllers. | - | libstorage_get_mgr_info_by_esn | Obtains the management information of the drive corresponding to an ESN. | - | libstorage_get_mgr_smart_by_esn | Obtains the S.M.A.R.T. information of the drive corresponding to an ESN. | - | libstorage_get_bdev_ns_info | Obtains namespace information based on the device name. | - | libstorage_get_ctrl_ns_info | Obtains information about all namespaces based on the controller name. | - -## Management-Plane Drive Information Query - -The management plane component Ublock of HSAK provides a group of C APIs for querying drive information on the management plane. - -1. Call the C API to initialize the HSAK Ublock server. - -2. Call the HSAK UIO component initialization API in another process based on service requirements. - -3. If multiple processes are required to query drive information, initialize the Ublock client. - -4. Call the APIs listed in the following table on the Ublock server process or client process to query information. - -5. After obtaining the block device list, call the APIs listed in the following table to free resources. - -6. If you exit the program, destroy the HSAK Ublock module (the destruction method on the server is the same as that on the client). - - | API | Description | - | ---------------------------- | ------------------------------------------------------------ | - | init_ublock | Initializes the Ublock function module. This API must be called before the other Ublock APIs. A process can be initialized only once because the init_ublock API initializes DPDK. The initial memory allocated by DPDK is bound to the process PID. One PID can be bound to only one memory. In addition, DPDK does not provide an API for freeing the memory. The memory can be freed only by exiting the process. | - | ublock_init | It is the macro definition of the init_ublock API. It can be considered as initializing Ublock to an RPC service. | - | ublock_init_norpc | It is the macro definition of the init_ublock API. It can be considered as initializing Ublock to a non-RPC service. | - | ublock_get_bdevs | Obtains the device list. The obtained device list contains only PCI addresses and does not contain specific device information. To obtain specific device information, call the ublock_get_bdev API. | - | ublock_get_bdev | Obtains information about a specific device, including the device serial number, model, and firmware version. The information is stored in character arrays instead of character strings. | - | ublock_get_bdev_by_esn | Obtains the device information based on the specified ESN, including the serial number, model, and firmware version. | - | ublock_get_SMART_info | Obtains the S.M.A.R.T. information of a specified device. | - | ublock_get_SMART_info_by_esn | Obtains the S.M.A.R.T. information of the device corresponding to an ESN. | - | ublock_get_error_log_info | Obtains the error log information of a device. | - | ublock_get_log_page | Obtains information about a specified log page of a specified device. | - | ublock_free_bdevs | Frees the device list. | - | ublock_free_bdev | Frees device resources. | - | ublock_fini | Destroys the Ublock module. This API destroys the Ublock module and internally created resources. This API must be used together with the Ublock initialization API. | - -## Log Management - -HSAK logs are exported to **/var/log/messages** through syslog by default and managed by the rsyslog service of the OS. If a custom log directory is required, use rsyslog to configure the log directory. - -1. Modify the **/etc/rsyslog.conf** configuration file. - -2. Restart the rsyslog service: - - ```shell - if ($programname == 'LibStorage') then { - action(type="omfile" fileCreateMode="0600" file="/var/log/HSAK/run.log") - stop - } - ``` - -3. Start the HSAK process. The log information is redirected to the target directory. - - ```shell - sysemctl restart rsyslog - ``` - -4. If redirected logs need to be dumped, manually configure log dump in the **/etc/logrotate.d/syslog** file. diff --git a/docs/en/docs/HSAK/hsak_interface.md b/docs/en/docs/HSAK/hsak_interface.md deleted file mode 100644 index c1e45123bf7ca1bf14ec9bbdac6420f5182321ba..0000000000000000000000000000000000000000 --- a/docs/en/docs/HSAK/hsak_interface.md +++ /dev/null @@ -1,2551 +0,0 @@ -## C APIs - -### Macro Definition and Enumeration - -#### bdev_rw.h - -##### enum libstorage_ns_lba_size - -1. Prototype - -``` -enum libstorage_ns_lba_size -{ -LIBSTORAGE_NVME_NS_LBA_SIZE_512 = 0x9, -LIBSTORAGE_NVME_NS_LBA_SIZE_4K = 0xc -}; -``` - -2. Description - -Sector (data) size of a drive. - -##### enum libstorage_ns_md_size - -1. Prototype - -``` -enum libstorage_ns_md_size -{ -LIBSTORAGE_METADATA_SIZE_0 = 0, -LIBSTORAGE_METADATA_SIZE_8 = 8, -LIBSTORAGE_METADATA_SIZE_64 = 64 -}; -``` - -2. Description - -Metadata size of a drive. - -3. Remarks - -- ES3000 V3 (single-port) supports formatting of five sector types (512+0, 512+8, 4K+64, 4K, and 4K+8). - -- ES3000 V3 (dual-port) supports formatting of four sector types (512+0, 512+8, 4K+64, and 4K). - -- ES3000 V5 supports formatting of five sector types (512+0, 512+8, 4K+64, 4K, and 4K+8). - -- Optane drives support formatting of seven sector types (512+0, 512+8, 512+16,4K, 4K+8, 4K+64, and 4K+128). - - -##### enum libstorage_ns_pi_type - -1. Prototype - -``` -enum libstorage_ns_pi_type -{ -LIBSTORAGE_FMT_NVM_PROTECTION_DISABLE = 0x0, -LIBSTORAGE_FMT_NVM_PROTECTION_TYPE1 = 0x1, -LIBSTORAGE_FMT_NVM_PROTECTION_TYPE2 = 0x2, -LIBSTORAGE_FMT_NVM_PROTECTION_TYPE3 = 0x3, -}; -``` - -2. Description - -Protection type supported by drives. - -3. Remarks - -ES3000 supports only protection types 0 and 3. Optane drives support only protection types 0 and 1. - -##### enum libstorage_crc_and_prchk - -1. Prototype - -``` -enum libstorage_crc_and_prchk -{ -LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK = 0x0, -LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK = 0x1, -LIBSTORAGE_LIB_CRC_AND_DISABLE_PRCHK = 0x2, -LIBSTORAGE_LIB_CRC_AND_ENABLE_PRCHK = 0x3, -#define NVME_NO_REF 0x4 -LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK_NO_REF = LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK | NVME_NO_REF, -LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK_NO_REF = LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK | NVME_NO_REF, -}; -``` - -2. Description - -- **LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK**: Cyclic redundancy check (CRC) is performed for the application layer, but not for HSAK. CRC is disabled for drives. - -- **LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK**: CRC is performed for the application layer, but not for HSAK. CRC is enabled for drives. - -- **LIBSTORAGE_LIB_CRC_AND_DISABLE_PRCHK**: CRC is performed for HSAK, but not for the application layer. CRC is disabled for drives. - -- **LIBSTORAGE_LIB_CRC_AND_ENABLE_PRCHK**: CRC is performed for HSAK, but not for the application layer. CRC is enabled for drives. - -- **LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK_NO_REF**: CRC is performed for the application layer, but not for HSAK. CRC is disabled for drives. REF tag verification is disabled for drives whose PI TYPE is 1 (Intel Optane P4800). - -- **LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK_NO_REF**: CRC is performed for the application layer, but not for HSAK. CRC is enabled for drives. REF tag verification is disabled for drives whose PI TYPE is 1 (Intel Optane P4800). - -- If PI TYPE of an Intel Optane P4800 drive is 1, the CRC and REF tag of the metadata area are verified by default. - -- Intel Optane P4800 drives support DIF in 512+8 format but does not support DIF in 4096+64 format. - -- For ES3000 V3 and ES3000 V5, PI TYPE of the drives is 3. By default, only the CRC of the metadata area is performed. - -- ES3000 V3 supports DIF in 512+8 format but does not support DIF in 4096+64 format. ES3000 V5 supports DIF in both 512+8 and 4096+64 formats. - - -The summary is as follows: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
E2E Verification ModeCtrl FlagCRC Generator Write ProcessRead Process
Application VerificationCRC for HSAKCRC for DrivesApplication VerificationCRC for HSAKCRC for Drives
Halfway protection0ControllerXXXXXX
1ControllerXXXXX√
2ControllerXXXXXX
3ControllerXXXXX√
Full protection0App√XX√XX
1App√X√√X√
2HSAKX√XX√X
3HSAKX√√X√√
- - - - - -##### enum libstorage_print_log_level - -1. Prototype - -``` -enum libstorage_print_log_level -{ -LIBSTORAGE_PRINT_LOG_ERROR, -LIBSTORAGE_PRINT_LOG_WARN, -LIBSTORAGE_PRINT_LOG_NOTICE, -LIBSTORAGE_PRINT_LOG_INFO, -LIBSTORAGE_PRINT_LOG_DEBUG, -}; -``` - -2. Description - -Storage Performance Development Kit (SPDK) log print levels: ERROR, WARN, NOTICE, INFO, and DEBUG, corresponding to 0 to 4 in the configuration file. - -##### MAX_BDEV_NAME_LEN - -1. Prototype - -``` -#define MAX_BDEV_NAME_LEN 24 -``` - -2. Description - -Maximum length of a block device name. - -##### MAX_CTRL_NAME_LEN - -1. Prototype - -``` -#define MAX_CTRL_NAME_LEN 16 -``` - -2. Description - -Maximum length of a controller. - -##### LBA_FORMAT_NUM - -1. Prototype - -``` -#define LBA_FORMAT_NUM 16 -``` - -2. Description - -Number of LBA formats supported by a controller. - -##### LIBSTORAGE_MAX_DSM_RANGE_DESC_COUNT - -1. Prototype - -``` -#define LIBSTORAGE_MAX_DSM_RANGE_DESC_COUNT 256 -``` - -2. Description - -Maximum number of 16-byte sets in the dataset management command. - -#### ublock.h - -##### UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO - -1. Prototype - -``` -#define UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO 1 -``` - -2. Description - -This macro is used to define that the subsystem corresponding to the uevent event is the userspace I/O subsystem (UIO) provided by the kernel. When the service receives the uevent event, this macro is used to determine whether the event is a UIO event that needs to be processed. - -The value of the int subsystem member in struct ublock_uevent is **UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO**. Currently, only this value is available. - -##### UBLOCK_TRADDR_MAX_LEN - -1. Prototype - -``` -#define UBLOCK_TRADDR_MAX_LEN 256 -``` - -2. Description - -The *Domain:Bus:Device.Function* (**%04x:%02x:%02x.%x**) format indicates the maximum length of the PCI address character string. The actual length is far less than 256 bytes. - -##### UBLOCK_PCI_ADDR_MAX_LEN - -1. Prototype - -``` -#define UBLOCK_PCI_ADDR_MAX_LEN 256 -``` - -2. Description - -Maximum length of the PCI address character string. The actual length is far less than 256 bytes. The possible formats of the PCI address are as follows: - -- Full address: **%x:%x:%x.%x** or **%x.%x.%x.%x** - -- When the **Function** value is **0**: **%x:%x:%x** - -- When the **Domain** value is **0**: **%x:%x.%x** or **%x.%x.%x** - -- When the **Domain** and **Function** values are **0**: **%x:%x** or **%x.%x** - -##### UBLOCK_SMART_INFO_LEN - -1. Prototype - -``` -#define UBLOCK_SMART_INFO_LEN 512 -``` - -2. Description - -Size of the structure for the S.M.A.R.T. information of an NVMe drive, which is 512 bytes. - -##### enum ublock_rpc_server_status - -1. Prototype - -``` -enum ublock_rpc_server_status { -// start rpc server or not -UBLOCK_RPC_SERVER_DISABLE = 0, -UBLOCK_RPC_SERVER_ENABLE = 1, -}; -``` - -2. Description - -Status of the RPC service in HSAK. The status can be enabled or disabled. - -##### enum ublock_nvme_uevent_action - -1. Prototype - -``` -enum ublock_nvme_uevent_action { -UBLOCK_NVME_UEVENT_ADD = 0, -UBLOCK_NVME_UEVENT_REMOVE = 1, -UBLOCK_NVME_UEVENT_INVALID, -}; -``` - -2. Description - -Indicates whether the uevent hot swap event is to insert or remove a drive. - -##### enum ublock_subsystem_type - -1. Prototype - -``` -enum ublock_subsystem_type { -SUBSYSTEM_UIO = 0, -SUBSYSTEM_NVME = 1, -SUBSYSTEM_TOP -}; -``` - -2. Description - -Type of the callback function, which is used to determine whether the callback function is registered for the UIO driver or kernel NVMe driver. - -### Data Structure - -#### bdev_rw.h - -##### struct libstorage_namespace_info - -1. Prototype - -``` -struct libstorage_namespace_info -{ -char name[MAX_BDEV_NAME_LEN]; -uint64_t size; /** namespace size in bytes */ -uint64_t sectors; /** number of sectors */ -uint32_t sector_size; /** sector size in bytes */ -uint32_t md_size; /** metadata size in bytes */ -uint32_t max_io_xfer_size; /** maximum i/o size in bytes */ -uint16_t id; /** namespace id */ -uint8_t pi_type; /** end-to-end data protection information type */ -uint8_t is_active :1; /** namespace is active or not */ -uint8_t ext_lba :1; /** namespace support extending LBA size or not */ -uint8_t dsm :1; /** namespace supports Dataset Management or not */ -uint8_t pad :3; -uint64_t reserved; -}; -``` - -2. Description - -This data structure contains the namespace information of a drive. - -3. Struct members - -| Member | Description | -| ---------------------------- | ------------------------------------------------------------ | -| char name[MAX_BDEV_NAME_LEN] | Name of the namespace. | -| uint64_t size | Size of the drive space allocated to the namespace, in bytes. | -| uint64_t sectors | Number of sectors. | -| uint32_t sector_size | Size of each sector, in bytes. | -| uint32_t md_size | Metadata size, in bytes. | -| uint32_t max_io_xfer_size | Maximum size of data in a single I/O operation, in bytes. | -| uint16_t id | Namespace ID. | -| uint8_t pi_type | Data protection type. The value is obtained from enum libstorage_ns_pi_type. | -| uint8_t is_active :1 | Namespace active or not. | -| uint8_t ext_lba :1 | Whether the namespace supports logical block addressing (LBA) in extended mode. | -| uint8_t dsm :1 | Whether the namespace supports dataset management. | -| uint8_t pad :3 | Reserved parameter. | -| uint64_t reserved | Reserved parameter. | - - - - -##### struct libstorage_nvme_ctrlr_info - -1. Prototype - -``` -struct libstorage_nvme_ctrlr_info -{ -char name[MAX_CTRL_NAME_LEN]; -char address[24]; -struct -{ -uint32_t domain; -uint8_t bus; -uint8_t dev; -uint8_t func; -} pci_addr; -uint64_t totalcap; /* Total NVM Capacity in bytes */ -uint64_t unusecap; /* Unallocated NVM Capacity in bytes */ -int8_t sn[20]; /* Serial number */ -uint8_t fr[8]; /* Firmware revision */ -uint32_t max_num_ns; /* Number of namespaces */ -uint32_t version; -uint16_t num_io_queues; /* num of io queues */ -uint16_t io_queue_size; /* io queue size */ -uint16_t ctrlid; /* Controller id */ -uint16_t pad1; -struct -{ -struct -{ -/** metadata size */ -uint32_t ms : 16; -/** lba data size */ -uint32_t lbads : 8; -uint32_t reserved : 8; -} lbaf[LBA_FORMAT_NUM]; -uint8_t nlbaf; -uint8_t pad2[3]; -uint32_t cur_format : 4; -uint32_t cur_extended : 1; -uint32_t cur_pi : 3; -uint32_t cur_pil : 1; -uint32_t cur_can_share : 1; -uint32_t mc_extented : 1; -uint32_t mc_pointer : 1; -uint32_t pi_type1 : 1; -uint32_t pi_type2 : 1; -uint32_t pi_type3 : 1; -uint32_t md_start : 1; -uint32_t md_end : 1; -uint32_t ns_manage : 1; /* Supports the Namespace Management and Namespace Attachment commands */ -uint32_t directives : 1; /* Controller support Directives or not */ -uint32_t streams : 1; /* Controller support Streams Directives or not */ -uint32_t dsm : 1; /* Controller support Dataset Management or not */ -uint32_t reserved : 11; -} cap_info; -}; -``` - -1. Description - -This data structure contains the controller information of a drive. - -2. Struct members - - -| Member | Description | -| ------------------------------------------------------------ | ------------------------------------------------------------ | -| char name[MAX_CTRL_NAME_LEN] | Controller name. | -| char address[24] | PCI address, which is a character string. | -| struct
{
uint32_t domain;
uint8_t bus;
uint8_t dev;
uint8_t func;
} pci_addr | PCI address, in segments. | -| uint64_t totalcap | Total capacity of the controller, in bytes. Optane drives are based on the NVMe 1.0 protocol and do not support this parameter. | -| uint64_t unusecap | Free capacity of the controller, in bytes. Optane drives are based on the NVMe 1.0 protocol and do not support this parameter. | -| int8_t sn[20]; | Serial number of a drive, which is an ASCII character string without **0**. | -| uint8_t fr[8]; | Drive firmware version, which is an ASCII character string without **0**. | -| uint32_t max_num_ns | Maximum number of namespaces. | -| uint32_t version | NVMe protocol version supported by the controller. | -| uint16_t num_io_queues | Number of I/O queues supported by a drive. | -| uint16_t io_queue_size | Maximum length of an I/O queue. | -| uint16_t ctrlid | Controller ID. | -| uint16_t pad1 | Reserved parameter. | - -Members of the struct cap_info substructure: - -| Member | Description | -| ------------------------------------------------------------ | ------------------------------------------------------------ | -| struct
{
uint32_t ms : 16;
uint32_t lbads : 8;
uint32_t reserved : 8;
}lbaf[LBA_FORMAT_NUM] | **ms**: metadata size. The minimum value is 8 bytes.
**lbads**: The LBA size is 2^lbads, and the value of **lbads** is greater than or equal to 9. | -| uint8_t nlbaf | Number of LBA formats supported by the controller. | -| uint8_t pad2[3] | Reserved parameter. | -| uint32_t cur_format : 4 | Current LBA format of the controller. | -| uint32_t cur_extended : 1 | Whether the controller supports LBA in extended mode. | -| uint32_t cur_pi : 3 | Current protection type of the controller. | -| uint32_t cur_pil : 1 | The current protection information (PI) of the controller is located in the first or last eight bytes of the metadata. | -| uint32_t cur_can_share : 1 | Whether the namespace supports multi-path transmission. | -| uint32_t mc_extented : 1 | Whether metadata is transmitted as part of the data buffer. | -| uint32_t mc_pointer : 1 | Whether metadata is separated from the data buffer. | -| uint32_t pi_type1 : 1 | Whether the controller supports protection type 1. | -| uint32_t pi_type2 : 1 | Whether the controller supports protection type 2. | -| uint32_t pi_type3 : 1 | Whether the controller supports protection type 3. | -| uint32_t md_start : 1 | Whether the controller supports protection information in the first eight bytes of metadata. | -| uint32_t md_end : 1 | Whether the controller supports protection information in the last eight bytes of metadata. | -| uint32_t ns_manage : 1 | Whether the controller supports namespace management. | -| uint32_t directives : 1 | Whether the Directives command set is supported. | -| uint32_t streams : 1 | Whether Streams Directives is supported. | -| uint32_t dsm : 1 | Whether Dataset Management commands are supported. | -| uint32_t reserved : 11 | Reserved parameter. | - -##### struct libstorage_dsm_range_desc - -1. Prototype - -``` -struct libstorage_dsm_range_desc -{ -/* RESERVED */ -uint32_t reserved; - -/* NUMBER OF LOGICAL BLOCKS */ -uint32_t block_count; - -/* UNMAP LOGICAL BLOCK ADDRESS */uint64_t lba;}; -``` - -2. Description - -Definition of a single 16-byte set in the data management command set. - -3. Struct members - -| Member | Description | -| -------------------- | ------------------------ | -| uint32_t reserved | Reserved parameter. | -| uint32_t block_count | Number of LBAs per unit. | -| uint64_t lba | Start LBA. | - -##### struct libstorage_ctrl_streams_param - -1. Prototype - -``` -struct libstorage_ctrl_streams_param -{ -/* MAX Streams Limit */ -uint16_t msl; - -/* NVM Subsystem Streams Available */ -uint16_t nssa; - -/* NVM Subsystem Streams Open */uint16_t nsso; - -uint16_t pad; -}; -``` - -2. Description - -Streams attribute value supported by NVMe drives. - -3. Struct members - -| Member | Description | -| ------------- | ------------------------------------------------------------ | -| uint16_t msl | Maximum number of Streams resources supported by a drive. | -| uint16_t nssa | Number of Streams resources that can be used by each NVM subsystem. | -| uint16_t nsso | Number of Streams resources used by each NVM subsystem. | -| uint16_t pad | Reserved parameter. | - - - -##### struct libstorage_bdev_streams_param - -1. Prototype - -``` -struct libstorage_bdev_streams_param -{ -/* Stream Write Size */ -uint32_t sws; - -/* Stream Granularity Size */ -uint16_t sgs; - -/* Namespace Streams Allocated */ -uint16_t nsa; - -/* Namespace Streams Open */ -uint16_t nso; - -uint16_t reserved[3]; -}; -``` - -2. Description - -Streams attribute value of the namespace. - -3. Struct members - -| Member | Description | -| -------------------- | ------------------------------------------------------------ | -| uint32_t sws | Write granularity with the optimal performance, in sectors. | -| uint16_t sgs | Write granularity allocated to Streams, in sws. | -| uint16_t nsa | Number of private Streams resources that can be used by a namespace. | -| uint16_t nso | Number of private Streams resources used by a namespace. | -| uint16_t reserved[3] | Reserved parameter. | - -##### struct libstorage_mgr_info - -1. Prototype - -``` -struct libstorage_mgr_info -{ -char pci[24]; -char ctrlName[MAX_CTRL_NAME_LEN]; -uint64_t sector_size; -uint64_t cap_size; -uint16_t device_id; -uint16_t subsystem_device_id; -uint16_t vendor_id; -uint16_t subsystem_vendor_id; -uint16_t controller_id; -int8_t serial_number[20]; -int8_t model_number[40]; -uint8_t firmware_revision[8]; -}; -``` - -2. Description - -Drive management information (consistent with the drive information used by the management plane). - -3. Struct members - -| Member | Description | -| -------------------------------- | ---------------------------------------------- | -| char pci[24] | Character string of the drive PCI address. | -| char ctrlName[MAX_CTRL_NAME_LEN] | Character string of the drive controller name. | -| uint64_t sector_size | Drive sector size. | -| uint64_t cap_size | Drive capacity, in bytes. | -| uint16_t device_id | Drive device ID. | -| uint16_t subsystem_device_id | Drive subsystem device ID. | -| uint16­*t vendor*id | Drive vendor ID. | -| uint16_t subsystem_vendor_id | Drive subsystem vendor ID. | -| uint16_t controller_id | Drive controller ID. | -| int8_t serial_number[20] | Drive serial number. | -| int8_t model_number[40] | Device model. | -| uint8_t firmware_revision[8] | Firmware version. | - -##### struct **attribute**((packed)) libstorage_smart_info - -1. Prototype - -``` -/* same with struct spdk_nvme_health_information_page in nvme_spec.h */ -struct __attribute__((packed)) libstorage_smart_info { -/* details of uint8_t critical_warning - -union spdk_nvme_critical_warning_state { - -uint8_t raw; -* - -struct { - -uint8_t available_spare : 1; - -uint8_t temperature : 1; - -uint8_t device_reliability : 1; - -uint8_t read_only : 1; - -uint8_t volatile_memory_backup : 1; - -uint8_t reserved : 3; - -} bits; - -}; -*/ -uint8_t critical_warning; -uint16_t temperature; -uint8_t available_spare; -uint8_t available_spare_threshold; -uint8_t percentage_used; -uint8_t reserved[26]; - -/* - -Note that the following are 128-bit values, but are - -defined as an array of 2 64-bit values. -*/ -/* Data Units Read is always in 512-byte units. */ -uint64_t data_units_read[2]; -/* Data Units Written is always in 512-byte units. */ -uint64_t data_units_written[2]; -/* For NVM command set, this includes Compare commands. */ -uint64_t host_read_commands[2]; -uint64_t host_write_commands[2]; -/* Controller Busy Time is reported in minutes. */ -uint64_t controller_busy_time[2]; -uint64_t power_cycles[2]; -uint64_t power_on_hours[2]; -uint64_t unsafe_shutdowns[2]; -uint64_t media_errors[2]; -uint64_t num_error_info_log_entries[2]; -/* Controller temperature related. */ -uint32_t warning_temp_time; -uint32_t critical_temp_time; -uint16_t temp_sensor[8]; -uint8_t reserved2[296]; -}; -``` - -1. Description - -This data structure defines the S.M.A.R.T. information of a drive. - -2. Struct members - -| Member | **Description (For details, see the NVMe protocol.)** | -| -------------------------------------- | ------------------------------------------------------------ | -| uint8_t critical_warning | Critical alarm of the controller status. If a bit is set to 1, the bit is valid. You can set multiple bits to be valid. Critical alarms are returned to the host through asynchronous events.
Bit 0: When this bit is set to 1, the redundant space is less than the specified threshold.
Bit 1: When this bit is set to 1, the temperature is higher or lower than a major threshold.
Bit 2: When this bit is set to 1, component reliability is reduced due to major media errors or internal errors.
Bit 3: When this bit is set to 1, the medium has been set to the read-only mode.
Bit 4: When this bit is set to 1, the volatile component of the controller fails. This parameter is valid only when the volatile component exists in the controller.
Bits 5-7: reserved. | -| uint16_t temperature | Temperature of a component. The unit is Kelvin. | -| uint8_t available_spare | Percentage of the available redundant space (0 to 100%). | -| uint8_t available_spare_threshold | Threshold of the available redundant space. An asynchronous event is reported when the available redundant space is lower than the threshold. | -| uint8_t percentage_used | Percentage of the actual service life of a component to the service life of the component expected by the manufacturer. The value **100** indicates that the actual service life of the component has reached to the expected service life, but the component can still be used. The value can be greater than 100, but any value greater than 254 will be set to 255. | -| uint8_t reserved[26] | Reserved. | -| uint64_t data_units_read[2] | Number of 512 bytes read by the host from the controller. The value **1** indicates that 1000 x 512 bytes are read, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | -| uint64_t data_units_written[2] | Number of 512 bytes written by the host to the controller. The value **1** indicates that 1000 x 512 bytes are written, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | -| uint64_t host_read_commands[2] | Number of read commands delivered to the controller. | -| uint64_t host_write_commands[2]; | Number of write commands delivered to the controller. | -| uint64_t controller_busy_time[2] | Busy time for the controller to process I/O commands. The process from the time the commands are delivered to the time the results are returned to the CQ is busy. The time is expressed in minutes. | -| uint64_t power_cycles[2] | Number of machine on/off cycles. | -| uint64_t power_on_hours[2] | Power-on duration, in hours. | -| uint64_t unsafe_shutdowns[2] | Number of abnormal power-off times. The value is incremented by 1 when CC.SHN is not received during power-off. | -| uint64_t media_errors[2] | Number of times that the controller detects unrecoverable data integrity errors, including uncorrectable ECC errors, CRC errors, and LBA tag mismatch. | -| uint64_t num_error_info_log_entries[2] | Number of entries in the error information log within the controller lifecycle. | -| uint32_t warning_temp_time | Accumulated time when the temperature exceeds the warning alarm threshold, in minutes. | -| uint32_t critical_temp_time | Accumulated time when the temperature exceeds the critical alarm threshold, in minutes. | -| uint16_t temp_sensor[8] | Temperature of temperature sensors 1–8. The unit is Kelvin. | -| uint8_t reserved2[296] | Reserved. | - -##### libstorage_dpdk_contig_mem - -1. Prototype - -``` -struct libstorage_dpdk_contig_mem { -uint64_t virtAddr; -uint64_t memLen; -uint64_t allocLen; -}; -``` - -2. Description - -Description about a contiguous virtual memory segment in the parameters of the callback function that notifies the service layer of initialization completion after the DPDK memory is initialized. - -Currently, 800 MB memory is reserved for HSAK. Other memory is returned to the service layer through **allocLen** in this struct for the service layer to allocate memory for self-management. - -The total memory to be reserved for HSAK is about 800 MB. The memory reserved for each memory segment is calculated based on the number of NUMA nodes in the environment. When there are too many NUMA nodes, the memory reserved on each memory segment is too small. As a result, HSAK initialization fails. Therefore, HSAK supports only the environment with a maximum of four NUMA nodes. - -3. Struct members - -| Member | Description | -| ----------------- | -------------------------------------------------------- | -| uint64_t virtAddr | Start address of the virtual memory. | -| uint64_t memLen | Length of the virtual memory, in bytes. | -| uint64_t allocLen | Available memory length in the memory segment, in bytes. | - -##### struct libstorage_dpdk_init_notify_arg - -1. Prototype - -``` -struct libstorage_dpdk_init_notify_arg { -uint64_t baseAddr; -uint16_t memsegCount; -struct libstorage_dpdk_contig_mem *memseg; -}; -``` - -2. Description - -Callback function parameter used to notify the service layer of initialization completion after DPDK memory initialization, indicating information about all virtual memory segments. - -3. Struct members - -| Member | Description | -| ----------------------------------------- | ------------------------------------------------------------ | -| uint64_t baseAddr | Start address of the virtual memory. | -| uint16_t memsegCount | Number of valid **memseg** array members, that is, the number of contiguous virtual memory segments. | -| struct libstorage_dpdk_contig_mem *memseg | Pointer to the memory segment array. Each array element is a contiguous virtual memory segment, and every two elements are discontiguous. | - -##### struct libstorage_dpdk_init_notify - -1. Prototype - -``` -struct libstorage_dpdk_init_notify { -const char *name; -void (*notifyFunc)(const struct libstorage_dpdk_init_notify_arg *arg); -TAILQ_ENTRY(libstorage_dpdk_init_notify) tailq; -}; -``` - -2. Description - -Struct used to notify the service layer of the callback function registration after the DPDK memory is initialized. - -3. Struct members - -| Member | Description | -| ------------------------------------------------------------ | ------------------------------------------------------------ | -| const char *name | Name of the service-layer module of the registered callback function. | -| void (*notifyFunc)(const struct libstorage_dpdk_init_notify_arg *arg) | Callback function parameter used to notify the service layer of initialization completion after the DPDK memory is initialized. | -| TAILQ_ENTRY(libstorage_dpdk_init_notify) tailq | Linked list that stores registered callback functions. | - -#### ublock.h - -##### struct ublock_bdev_info - -1. Prototype - -``` -struct ublock_bdev_info { -uint64_t sector_size; -uint64_t cap_size; // cap_size -uint16_t device_id; -uint16_t subsystem_device_id; // subsystem device id of nvme control -uint16_t vendor_id; -uint16_t subsystem_vendor_id; -uint16_t controller_id; -int8_t serial_number[20]; -int8_t model_number[40]; -int8_t firmware_revision[8]; -}; -``` - -2. Description - -This data structure contains the device information of a drive. - -3. Struct members - -| Member | Description | -| ---------------------------- | ----------------------------------------------- | -| uint64_t sector_size | Sector size of a drive, for example, 512 bytes. | -| uint64_t cap_size | Total drive capacity, in bytes. | -| uint16_t device_id | Device ID. | -| uint16_t subsystem_device_id | Device ID of a subsystem. | -| uint16_t vendor_id | Main ID of the device vendor. | -| uint16_t subsystem_vendor_id | Sub-ID of the device vendor. | -| uint16_t controller_id | ID of the device controller. | -| int8_t serial_number[20] | Device serial number. | -| int8_t model_number[40] | Device model. | -| int8_t firmware_revision[8] | Firmware version. | - -##### struct ublock_bdev - -1. Prototype - -``` -struct ublock_bdev { -char pci[UBLOCK_PCI_ADDR_MAX_LEN]; -struct ublock_bdev_info info; -struct spdk_nvme_ctrlr *ctrlr; -TAILQ_ENTRY(ublock_bdev) link; -}; -``` - -2. Description - -The data structure contains the drive information of the specified PCI address, and the structure itself is a node of the queue. - -3. Struct members - -| Member | Description | -| --------------------------------- | ------------------------------------------------------------ | -| char pci[UBLOCK_PCI_ADDR_MAX_LEN] | PCI address. | -| struct ublock_bdev_info info | Drive information. | -| struct spdk_nvme_ctrlr *ctrlr | Data structure of the device controller. The members in this structure are not open to external systems. External services can obtain the corresponding member data through the SPDK open source interface. | -| TAILQ_ENTRY(ublock_bdev) link | Structure of the pointers before and after a queue. | - -##### struct ublock_bdev_mgr - -1. Prototype - -``` -struct ublock_bdev_mgr { -TAILQ_HEAD(, ublock_bdev) bdevs; -}; -``` - -2. Description - -This data structure defines the header structure of a ublock_bdev queue. - -3. Struct members - -| Member | Description | -| -------------------------------- | ----------------------- | -| TAILQ_HEAD(, ublock_bdev) bdevs; | Queue header structure. | - -##### struct **attribute**((packed)) ublock_SMART_info - -1. Prototype - -``` -struct __attribute__((packed)) ublock_SMART_info { -uint8_t critical_warning; -uint16_t temperature; -uint8_t available_spare; -uint8_t available_spare_threshold; -uint8_t percentage_used; -uint8_t reserved[26]; -/* - -Note that the following are 128-bit values, but are - -defined as an array of 2 64-bit values. -*/ -/* Data Units Read is always in 512-byte units. */ -uint64_t data_units_read[2]; -/* Data Units Written is always in 512-byte units. */ -uint64_t data_units_written[2]; -/* For NVM command set, this includes Compare commands. */ -uint64_t host_read_commands[2]; -uint64_t host_write_commands[2]; -/* Controller Busy Time is reported in minutes. */ -uint64_t controller_busy_time[2]; -uint64_t power_cycles[2]; -uint64_t power_on_hours[2]; -uint64_t unsafe_shutdowns[2]; -uint64_t media_errors[2]; -uint64_t num_error_info_log_entries[2]; -/* Controller temperature related. */ -uint32_t warning_temp_time; -uint32_t critical_temp_time; -uint16_t temp_sensor[8]; -uint8_t reserved2[296]; -}; -``` - -2. Description - -This data structure defines the S.M.A.R.T. information of a drive. - -3. Struct members - -| Member | Description (For details, see the NVMe protocol.) | -| -------------------------------------- | ------------------------------------------------------------ | -| uint8_t critical_warning | Critical alarm of the controller status. If a bit is set to 1, the bit is valid. You can set multiple bits to be valid. Critical alarms are returned to the host through asynchronous events.
Bit 0: When this bit is set to 1, the redundant space is less than the specified threshold.
Bit 1: When this bit is set to 1, the temperature is higher or lower than a major threshold.
Bit 2: When this bit is set to 1, component reliability is reduced due to major media errors or internal errors.
Bit 3: When this bit is set to 1, the medium has been set to the read-only mode.
Bit 4: When this bit is set to 1, the volatile component of the controller fails. This parameter is valid only when the volatile component exists in the controller.
Bits 5-7: reserved. | -| uint16_t temperature | Temperature of a component. The unit is Kelvin. | -| uint8_t available_spare | Percentage of the available redundant space (0 to 100%). | -| uint8_t available_spare_threshold | Threshold of the available redundant space. An asynchronous event is reported when the available redundant space is lower than the threshold. | -| uint8_t percentage_used | Percentage of the actual service life of a component to the service life of the component expected by the manufacturer. The value **100** indicates that the actual service life of the component has reached to the expected service life, but the component can still be used. The value can be greater than 100, but any value greater than 254 will be set to 255. | -| uint8_t reserved[26] | Reserved. | -| uint64_t data_units_read[2] | Number of 512 bytes read by the host from the controller. The value **1** indicates that 1000 x 512 bytes are read, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | -| uint64_t data_units_written[2] | Number of 512 bytes written by the host to the controller. The value **1** indicates that 1000 x 512 bytes are written, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | -| uint64_t host_read_commands[2] | Number of read commands delivered to the controller. | -| uint64_t host_write_commands[2]; | Number of write commands delivered to the controller. | -| uint64_t controller_busy_time[2] | Busy time for the controller to process I/O commands. The process from the time the commands are delivered to the time the results are returned to the CQ is busy. The value is expressed in minutes. | -| uint64_t power_cycles[2] | Number of machine on/off cycles. | -| uint64_t power_on_hours[2] | Power-on duration, in hours. | -| uint64_t unsafe_shutdowns[2] | Number of abnormal power-off times. The value is incremented by 1 when CC.SHN is not received during power-off. | -| uint64_t media_errors[2] | Number of unrecoverable data integrity errors detected by the controller, including uncorrectable ECC errors, CRC errors, and LBA tag mismatch. | -| uint64_t num_error_info_log_entries[2] | Number of entries in the error information log within the controller lifecycle. | -| uint32_t warning_temp_time | Accumulated time when the temperature exceeds the warning alarm threshold, in minutes. | -| uint32_t critical_temp_time | Accumulated time when the temperature exceeds the critical alarm threshold, in minutes. | -| uint16_t temp_sensor[8] | Temperature of temperature sensors 1–8. The unit is Kelvin. | -| uint8_t reserved2[296] | Reserved. | - -##### struct ublock_nvme_error_info - -1. Prototype - -``` -struct ublock_nvme_error_info { -uint64_t error_count; -uint16_t sqid; -uint16_t cid; -uint16_t status; -uint16_t error_location; -uint64_t lba; -uint32_t nsid; -uint8_t vendor_specific; -uint8_t reserved[35]; -}; -``` - -2. Description - -This data structure contains the content of a single error message in the device controller. The number of errors supported by different controllers may vary. - -3. Struct members - -| Member | Description (For details, see the NVMe protocol.) | -| ----------------------- | ------------------------------------------------------------ | -| uint64_t error_count | Error sequence number, which increases in ascending order. | -| uint16_t sqid | Submission queue identifier for the command associated with an error message. If an error cannot be associated with a specific command, this parameter should be set to **FFFFh**. | -| uint16_t cid | Command identifier associated with an error message. If an error cannot be associated with a specific command, this parameter should be set to **FFFFh**. | -| uint16_t status | Status of a completed command. | -| uint16_t error_location | Command parameter associated with an error message. | -| uint64_t lba | First LBA when an error occurs. | -| uint32_t nsid | Namespace where an error occurs. | -| uint8_t vendor_specific | Log page identifier associated with the page if other vendor-specific error messages are available. The value **00h** indicates that no additional information is available. The valid value ranges from 80h to FFh. | -| uint8_t reserved[35] | Reserved. | - -##### struct ublock_uevent - -1. Prototype - -``` -struct ublock_uevent { -enum ublock_nvme_uevent_action action; -int subsystem; -char traddr[UBLOCK_TRADDR_MAX_LEN + 1]; -}; -``` - -2. Description - -This data structure contains parameters related to the uevent event. - -3. Struct members - -| Member | Description | -| -------------------------------------- | ------------------------------------------------------------ | -| enum ublock_nvme_uevent_action action | Whether the uevent event type is drive insertion or removal through enumeration. | -| int subsystem | Subsystem type of the uevent event. Currently, only **UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO** is supported. If the application receives other values, no processing is required. | -| char traddr[UBLOCK_TRADDR_MAX_LEN + 1] | PCI address character string in the *Domain:Bus:Device.Function* (**%04x:%02x:%02x.%x**) format. | - -##### struct ublock_hook - -1. Prototype - -``` -struct ublock_hook -{ -ublock_callback_func ublock_callback; -void *user_data; -}; -``` - -2. Description - -This data structure is used to register callback functions. - -3. Struct members - -| Member | Description | -| ------------------------------------ | ------------------------------------------------------------ | -| ublock_callback_func ublock_callback | Function executed during callback. The type is bool func(void *info, void *user_data). | -| void *user_data | User parameter transferred to the callback function. | - -##### struct ublock_ctrl_iostat_info - -1. Prototype - -``` -struct ublock_ctrl_iostat_info -{ -uint64_t num_read_ops; -uint64_t num_write_ops; -uint64_t read_latency_ms; -uint64_t write_latency_ms; -uint64_t io_outstanding; -uint64_t num_poll_timeout; -uint64_t io_ticks_ms; -}; -``` - -2. Description - -This data structure is used to obtain the I/O statistics of a controller. - -3. Struct members - -| Member | Description | -| ------------------------- | ------------------------------------------------------------ | -| uint64_t num_read_ops | Accumulated number of read I/Os of the controller. | -| uint64_t num_write_ops | Accumulated number of write I/Os of the controller. | -| uint64_t read_latency_ms | Accumulated read latency of the controller, in ms. | -| uint64_t write_latency_ms | Accumulated write latency of the controller, in ms. | -| uint64_t io_outstanding | Queue depth of the controller. | -| uint64_t num_poll_timeout | Accumulated number of polling timeouts of the controller. | -| uint64_t io_ticks_ms | Accumulated I/O processing latency of the controller, in ms. | - -### API - -#### bdev_rw.h - -##### libstorage_get_nvme_ctrlr_info - -1. Prototype - -uint32_t libstorage_get_nvme_ctrlr_info(struct libstorage_nvme_ctrlr_info** ppCtrlrInfo); - -2. Description - -Obtains information about all controllers. - -3. Parameters - -| Parameter | Description | -| ----------------------------------------------- | ------------------------------------------------------------ | -| struct libstorage_nvme_ctrlr_info** ppCtrlrInfo | Output parameter, which returns all obtained controller information.
Note:
Free the memory using the free API in a timely manner. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Failed to obtain controller information or no controller information is obtained. | -| > 0 | Number of obtained controllers. | - -##### libstorage_get_mgr_info_by_esn - -1. Prototype - -``` -int32_t libstorage_get_mgr_info_by_esn(const char *esn, struct libstorage_mgr_info *mgr_info); -``` - -2. Description - -Obtains the management information about the NVMe drive corresponding to the ESN. - -3. Parameters - -| Parameter | Description | -| ------------------------------------ | ------------------------------------------------------------ | -| const char *esn | ESN of the target device.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string.
| -| struct libstorage_mgr_info *mgr_info | Output parameter, which returns all obtained NVMe drive management information. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Succeeded in querying the NVMe drive management information corresponding to an ESN. | -| -1 | Failed to query the NVMe drive management information corresponding to an ESN. | -| -2 | No NVMe drive matching an ESN is obtained. | - -##### libstorage_get_mgr_smart_by_esn - -1. Prototype - -``` -int32_t libstorage_get_mgr_smart_by_esn(const char *esn, uint32_t nsid, struct libstorage_smart_info *mgr_smart_info); -``` - -2. Description - -Obtains the S.M.A.R.T. information of the NVMe drive corresponding to an ESN. - -3. Parameters - -| Parameter | Description | -| ------------------------------------ | ------------------------------------------------------------ | -| const char *esn | ESN of the target device.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string.
| -| uint32_t nsid | Specified namespace. | -| struct libstorage_mgr_info *mgr_info | Output parameter, which returns all obtained S.M.A.R.T. information of NVMe drives. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Succeeded in querying the S.M.A.R.T. information of the NVMe drive corresponding to an ESN. | -| -1 | Failed to query the S.M.A.R.T. information of the NVMe drive corresponding to an ESN. | -| -2 | No NVMe drive matching an ESN is obtained. | - -##### libstorage_get_bdev_ns_info - -1. Prototype - -``` -uint32_t libstorage_get_bdev_ns_info(const char* bdevName, struct libstorage_namespace_info** ppNsInfo); -``` - -2. Description - -Obtains namespace information based on the device name. - -3. Parameters - -| Parameter | Description | -| ------------------------------------------- | ------------------------------------------------------------ | -| const char* bdevName | Device name. | -| struct libstorage_namespace_info** ppNsInfo | Output parameter, which returns namespace information.
Note:
Free the memory using the free API in a timely manner. | - -4. Return value - -| Return Value | Description | -| ------------ | ---------------------------- | -| 0 | The operation failed. | -| 1 | The operation is successful. | - -##### libstorage_get_ctrl_ns_info - -1. Prototype - -``` -uint32_t libstorage_get_ctrl_ns_info(const char* ctrlName, struct libstorage_namespace_info** ppNsInfo); -``` - -2. Description - -Obtains information about all namespaces based on the controller name. - -3. Parameters - -| Parameter | Description | -| ------------------------------------------- | ------------------------------------------------------------ | -| const char* ctrlName | Controller name. | -| struct libstorage_namespace_info** ppNsInfo | Output parameter, which returns information about all namespaces.
Note:
Free the memory using the free API in a timely manner. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Failed to obtain the namespace information or no namespace information is obtained. | -| > 0 | Number of namespaces obtained. | - -##### libstorage_create_namespace - -1. Prototype - -``` -int32_t libstorage_create_namespace(const char* ctrlName, uint64_t ns_size, char** outputName); -``` - -2. Description - -Creates a namespace on a specified controller (the prerequisite is that the controller supports namespace management). - -Optane drives are based on the NVMe 1.0 protocol and do not support namespace management. Therefore, this API is not supported. - -ES3000 V3 and V5 support only one namespace by default. By default, a namespace exists on the controller. To create a namespace, delete the original namespace. - -3. Parameters - -| Parameter | Description | -| -------------------- | ------------------------------------------------------------ | -| const char* ctrlName | Controller name. | -| uint64_t ns_size | Size of the namespace to be created (unit: sector_size). | -| char** outputName | Output parameter, which indicates the name of the created namespace.
Note:
Free the memory using the free API in a timely manner. | - -4. Return value - -| Return Value | Description | -| ------------ | ---------------------------------------------- | -| ≤ 0 | Failed to create the namespace. | -| > 0 | ID of the created namespace (starting from 1). | - -##### libstorage_delete_namespace - -1. Prototype - -``` -int32_t libstorage_delete_namespace(const char* ctrlName, uint32_t ns_id); -``` - -2. Description - -Deletes a namespace from a specified controller. Optane drives are based on the NVMe 1.0 protocol and do not support namespace management. Therefore, this API is not supported. - -3. Parameters - -| Parameter | Description | -| -------------------- | ---------------- | -| const char* ctrlName | Controller name. | -| uint32_t ns_id | Namespace ID | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Deletion succeeded. | -| Other values | Deletion failed.
Note:
Before deleting a namespace, stop I/O operations. Otherwise, the namespace fails to be deleted. | - -##### libstorage_delete_all_namespace - -1. Prototype - -``` -int32_t libstorage_delete_all_namespace(const char* ctrlName); -``` - -2. Description - -Deletes all namespaces from a specified controller. Optane drives are based on the NVMe 1.0 protocol and do not support namespace management. Therefore, this API is not supported. - -3. Parameters - -| Parameter | Description | -| -------------------- | ---------------- | -| const char* ctrlName | Controller name. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Deletion succeeded. | -| Other values | Deletion failed.
Note:
Before deleting a namespace, stop I/O operations. Otherwise, the namespace fails to be deleted. | - -##### libstorage_nvme_create_ctrlr - -1. Prototype - -``` -int32_t libstorage_nvme_create_ctrlr(const char *pci_addr, const char *ctrlr_name); -``` - -2. Description - -Creates an NVMe controller based on the PCI address. - -3. Parameters - -| Parameter | Description | -| ---------------- | ---------------- | -| char *pci_addr | PCI address. | -| char *ctrlr_name | Controller name. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------- | -| < 0 | Creation failed. | -| 0 | Creation succeeded. | - -##### libstorage_nvme_delete_ctrlr - -1. Prototype - -``` -int32_t libstorage_nvme_delete_ctrlr(const char *ctrlr_name); -``` - -1. Description - -Destroys an NVMe controller based on the controller name. - -2. Parameters - -| Parameter | Description | -| ---------------------- | ---------------- | -| const char *ctrlr_name | Controller name. | - -This API can be called only after all delivered I/Os are returned. - -3. Return value - -| Return Value | Description | -| ------------ | ---------------------- | -| < 0 | Destruction failed. | -| 0 | Destruction succeeded. | - -##### libstorage_nvme_reload_ctrlr - -1. Prototype - -``` -int32_t libstorage_nvme_reload_ctrlr(const char *cfgfile); -``` - -2. Description - -Adds or deletes an NVMe controller based on the configuration file. - -3. Parameters - -| Parameter | Description | -| ------------------- | ------------------------------- | -| const char *cfgfile | Path of the configuration file. | - - -Before using this API to delete a drive, ensure that all delivered I/Os have been returned. - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| < 0 | Failed to add or delete drives based on the configuration file. (Drives may be successfully added or deleted for some controllers.) | -| 0 | Drives are successfully added or deleted based on the configuration file. | - -> Constraints - -- Currently, a maximum of 36 controllers can be configured in the configuration file. - -- The reload API creates as many controllers as possible. If a controller fails to be created, the creation of other controllers is not affected. - -- In concurrency scenarios, the final drive initialization status may be inconsistent with the input configuration file. - -- If you delete a drive that is delivering I/Os by reloading the drive, I/Os fail. - -- After the controller name (for example, **nvme0**) corresponding to the PCI address in the configuration file is modified, the modification does not take effect after this interface is called. - -- The reload function is valid only when drives are added or deleted. Other configuration items in the configuration file cannot be reloaded. - -##### libstorage_low_level_format_nvm - -1. Prototype - -``` -int8_t libstorage_low_level_format_nvm(const char* ctrlName, uint8_t lbaf, -enum libstorage_ns_pi_type piType, -bool pil_start, bool ms_extented, uint8_t ses); -``` - -2. Description - -Low-level formats NVMe drives. - -3. Parameters - -| Parameter | Description | -| --------------------------------- | ------------------------------------------------------------ | -| const char* ctrlName | Controller name. | -| uint8_t lbaf | LBA format to be used. | -| enum libstorage_ns_pi_type piType | Protection type to be used. | -| bool pil_start | The protection information is stored in first eight bytes (1) or last eight bytes (0) of the metadata. | -| bool ms_extented | Whether to format to the extended type. | -| uint8_t ses | Whether to perform secure erase during formatting. Currently, only the value **0** (no-secure erase) is supported. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------- | -| < 0 | Formatting failed. | -| ≥ 0 | LBA format generated after successful formatting. | - -> Constraints - -- This low-level formatting API will clear the data and metadata of the drive namespace. Exercise caution when using this API. - -- It takes several seconds to format an ES3000 drive and several minutes to format an Intel Optane drive. Before using this API, wait until the formatting is complete. If the formatting process is forcibly stopped, the formatting fails. - -- Before formatting, stop the I/O operations on the data plane. If the drive is processing I/O requests, the formatting may fail occasionally. If the formatting is successful, the drive may discard the I/O requests that are being processed. Therefore, before formatting the drive, ensure that the I/O operations on the data plane are stopped. - -- During the formatting, the controller is reset. As a result, the initialized drive resources are unavailable. Therefore, after the formatting is complete, restart the I/O process on the data plane. - -- ES3000 V3 supports protection types 0 and 3, PI start and PI end, and mc extended. ES3000 V3 supports DIF in 512+8 format but does not support DIF in 4096+64 format. - -- ES3000 V5 supports protection types 0 and 3, PI start and PI end, mc extended, and mc pointer. ES3000 V5 supports DIF in both 512+8 and 4096+64 formats. - -- Optane drives support protection types 0 and 1, PI end, and mc extended. Optane drives support DIF in 512+8 format but does not support DIF in 4096+64 format. - -| **Drive Type** | **LBA Format** | **Drive Type** | **LBA Format** | -| ------------------ | ------------------------------------------------------------ | -------------- | ------------------------------------------------------------ | -| Intel Optane P4800 | lbaf0:512+0
lbaf1:512+8
lbaf2:512+16
lbaf3:4096+0
lbaf4:4096+8
lbaf5:4096+64
lbaf6:4096+128 | ES3000 V3, V5 | lbaf0:512+0
lbaf1:512+8
lbaf2:4096+64
lbaf3:4096+0
lbaf4:4096+8 | - -##### LIBSTORAGE_CALLBACK_FUNC - -1. Prototype - -``` -typedef void (*LIBSTORAGE_CALLBACK_FUNC)(int32_t cb_status, int32_t sct_code, void* cb_arg); -``` - -2. Description - -Registered HSAK I/O completion callback function. - -3. Parameters - -| Parameter | Description | -| ----------------- | ------------------------------------------------------------ | -| int32_t cb_status | I/O status code. The value **0** indicates success, a negative value indicates system error code, and a positive value indicates drive error code (for different error codes,
see [Appendixes](#Appendixes)). | -| int32_t sct_code | I/O status code type:
0: [GENERIC](#generic)
1: [COMMAND_SPECIFIC](#command_specific)
2: [MEDIA_DATA_INTERGRITY_ERROR](#media_data_intergrity_error)
7: VENDOR_SPECIFIC | -| void* cb_arg | Input parameter of the callback function. | - -4. Return value - -None. - -##### libstorage_deallocate_block - -1. Prototype - -``` -int32_t libstorage_deallocate_block(int32_t fd, struct libstorage_dsm_range_desc *range, uint16_t range_count, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); -``` - -2. Description - -Notifies NVMe drives of the blocks that can be released. - -3. Parameters - -| Parameter | Description | -| --------------------------------------- | ------------------------------------------------------------ | -| int32_t fd | Open drive file descriptor. | -| struct libstorage_dsm_range_desc *range | Description of blocks that can be released on NVMe drives.
Note:
This parameter requires **libstorage_mem_reserve** to allocate huge page memory. 4 KB alignment is required during memory allocation, that is, align is set to 4096.
The TRIM range of drives is restricted based on different drives. Exceeding the maximum TRIM range on the drives may cause data exceptions. | -| uint16_t range_count | Number of members in the array range. | -| LIBSTORAGE_CALLBACK_FUNC cb | Callback function. | -| void* cb_arg | Callback function parameter. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------- | -| < 0 | Failed to deliver the request. | -| 0 | Request submitted successfully. | - -##### libstorage_async_write - -1. Prototype - -``` -int32_t libstorage_async_write(int32_t fd, void *buf, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); -``` - -2. Description - -Delivers asynchronous I/O write requests (the write buffer is a contiguous buffer). - -3. Parameters - -| Parameter | Description | -| -------------------------------------- | ------------------------------------------------------------ | -| int32_t fd | File descriptor of the block device. | -| void *buf | Buffer for I/O write data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
The LBA in extended mode must contain the metadata memory size. | -| size_t nbytes | Size of a single write I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| off64_t offset | Write offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.) | -| size_t md_len | Buffer length of metadata. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.) | -| enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | -| LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | -| void* cb_arg | Parameters of the callback function. | - -4. Return value - -| Return Value | Description | -| ------------ | ---------------------------------------------- | -| 0 | I/O write requests are submitted successfully. | -| Other values | Failed to submit I/O write requests. | - -##### libstorage_async_read - -1. Prototype - -``` -int32_t libstorage_async_read(int32_t fd, void *buf, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); -``` - -2. Description - -Delivers asynchronous I/O read requests (the read buffer is a contiguous buffer). - -3. Parameters - -| Parameter | Description | -| -------------------------------------- | ------------------------------------------------------------ | -| int32_t fd | File descriptor of the block device. | -| void *buf | Buffer for I/O read data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
LBAs in extended mode must contain the metadata memory size. | -| size_t nbytes | Size of a single read I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| off64_t offset | Read offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. The LBA in extended mode does not include the metadata size. | -| void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.). | -| size_t md_len | Buffer length of metadata. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.). | -| enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | -| LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | -| void* cb_arg | Parameters of the callback function. | - -4. Return value - -| Return Value | Description | -| ------------ | --------------------------------------------- | -| 0 | I/O read requests are submitted successfully. | -| Other values | Failed to submit I/O read requests. | - -##### libstorage_async_writev - -1. Prototype - -``` -int32_t libstorage_async_writev(int32_t fd, struct iovec *iov, int iovcnt, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); -``` - -2. Description - -Delivers asynchronous I/O write requests (the write buffer is a discrete buffer). - -3. Parameters - -| Parameter | Description | -| -------------------------------------- | ------------------------------------------------------------ | -| int32_t fd | File descriptor of the block device. | -| struct iovec *iov | Buffer for I/O write data.
Note:
LBAs in extended mode must contain the metadata size.
The address must be 4-byte-aligned and the length cannot exceed 4 GB. | -| int iovcnt | Number of buffers for I/O write data. | -| size_t nbytes | Size of a single write I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| off64_t offset | Write offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.) | -| size_t md_len | Length of the metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.) | -| enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | -| LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | -| void* cb_arg | Parameters of the callback function. | - -4. Return value - -| Return Value | Description | -| ------------ | ---------------------------------------------- | -| 0 | I/O write requests are submitted successfully. | -| Other values | Failed to submit I/O write requests. | - -##### libstorage_async_readv - -1. Prototype - -``` -int32_t libstorage_async_readv(int32_t fd, struct iovec *iov, int iovcnt, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); -``` - -2. Description - -Delivers asynchronous I/O read requests (the read buffer is a discrete buffer). - -3. Parameters - -| Parameter | Description | -| -------------------------------------- | ------------------------------------------------------------ | -| int32_t fd | File descriptor of the block device. | -| struct iovec *iov | Buffer for I/O read data.
Note:
LBAs in extended mode must contain the metadata size.
The address must be 4-byte-aligned and the length cannot exceed 4 GB. | -| int iovcnt | Number of buffers for I/O read data. | -| size_t nbytes | Size of a single read I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| off64_t offset | Read offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.) | -| size_t md_len | Length of the metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.) | -| enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | -| LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | -| void* cb_arg | Parameters of the callback function. | - -4. Return value - -| Return Value | Description | -| ------------ | --------------------------------------------- | -| 0 | I/O read requests are submitted successfully. | -| Other values | Failed to submit I/O read requests. | - -##### libstorage_sync_write - -1. Prototype - -``` -int32_t libstorage_sync_write(int fd, const void *buf, size_t nbytes, off_t offset); -``` - -2. Description - -Delivers synchronous I/O write requests (the write buffer is a contiguous buffer). - -3. Parameters - -| Parameter | Description | -| -------------- | ------------------------------------------------------------ | -| int32_t fd | File descriptor of the block device. | -| void *buf | Buffer for I/O write data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
LBAs in extended mode must contain the metadata memory size. | -| size_t nbytes | Size of a single write I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| off64_t offset | Write offset of the LBA, in bytes. (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | - -4. Return value - -| Return Value | Description | -| ------------ | ---------------------------------------------- | -| 0 | I/O write requests are submitted successfully. | -| Other values | Failed to submit I/O write requests. | - -##### libstorage_sync_read - -1. Prototype - -``` -int32_t libstorage_sync_read(int fd, const void *buf, size_t nbytes, off_t offset); -``` - -2. Description - -Delivers synchronous I/O read requests (the read buffer is a contiguous buffer). - -3. Parameters - -| Parameter | Description | -| -------------- | ------------------------------------------------------------ | -| int32_t fd | File descriptor of the block device. | -| void *buf | Buffer for I/O read data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
LBAs in extended mode must contain the metadata memory size. | -| size_t nbytes | Size of a single read I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | -| off64_t offset | Read offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | - -4. Return value - -| Return Value | Description | -| ------------ | --------------------------------------------- | -| 0 | I/O read requests are submitted successfully. | -| Other values | Failed to submit I/O read requests. | - -##### libstorage_open - -1. Prototype - -``` -int32_t libstorage_open(const char* devfullname); -``` - -2. Description - -Opens a block device. - -3. Parameters - -| Parameter | Description | -| ----------------------- | ---------------------------------------- | -| const char* devfullname | Block device name (format: **nvme0n1**). | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| -1 | Opening failed. For example, the device name is incorrect, or the number of opened FDs is greater than the number of available channels of the NVMe drive. | -| > 0 | File descriptor of the block device. | - -After the MultiQ function in **nvme.conf.in** is enabled, different FDs are returned if a thread opens the same device for multiple times. Otherwise, the same FD is returned. This attribute applies only to the NVMe device. - -##### libstorage_close - -1. Prototype - -``` -int32_t libstorage_close(int32_t fd); -``` - -2. Description - -Closes a block device. - -3. Parameters - -| Parameter | Description | -| ---------- | ------------------------------------------ | -| int32_t fd | File descriptor of an opened block device. | - -4. Return value - -| Return Value | Description | -| ------------ | ----------------------------------------------- | -| -1 | Invalid file descriptor. | -| -16 | The file descriptor is busy. Retry is required. | -| 0 | Close succeeded. | - -##### libstorage_mem_reserve - -1. Prototype - -``` -void* libstorage_mem_reserve(size_t size, size_t align); -``` - -2. Description - -Allocates memory space from the huge page memory reserved by the DPDK. - -3. Parameters - -| Parameter | Description | -| ------------ | ----------------------------------- | -| size_t size | Size of the memory to be allocated. | -| size_t align | Aligns allocated memory space. | - -4. Return value - -| Return Value | Description | -| ------------ | -------------------------------------- | -| NULL | Allocation failed. | -| Other values | Address of the allocated memory space. | - -##### libstorage_mem_free - -1. Prototype - -``` -void libstorage_mem_free(void* ptr); -``` - -2. Description - -Frees the memory space pointed to by **ptr**. - -3. Parameters - -| Parameter | Description | -| --------- | ---------------------------------------- | -| void* ptr | Address of the memory space to be freed. | - -4. Return value - -None. - -##### libstorage_alloc_io_buf - -1. Prototype - -``` -void* libstorage_alloc_io_buf(size_t nbytes); -``` - -2. Description - -Allocates memory from buf_small_pool or buf_large_pool of the SPDK. - -3. Parameters - -| Parameter | Description | -| ------------- | ----------------------------------- | -| size_t nbytes | Size of the buffer to be allocated. | - -4. Return value - -| Return Value | Description | -| ------------ | -------------------------------------- | -| Other values | Start address of the allocated buffer. | - -##### libstorage_free_io_buf - -1. Prototype - -``` -int32_t libstorage_free_io_buf(void *buf, size_t nbytes); -``` - -2. Description - -Frees the allocated memory to buf_small_pool or buf_large_pool of the SPDK. - -3. Parameters - -| Parameter | Description | -| ------------- | ---------------------------------------- | -| void *buf | Start address of the buffer to be freed. | -| size_t nbytes | Size of the buffer to be freed. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------ | -| -1 | Freeing failed. | -| 0 | Freeing succeeded. | - -##### libstorage_init_module - -1. Prototype - -``` -int32_t libstorage_init_module(const char* cfgfile); -``` - -2. Description - -Initializes the HSAK module. - -3. Parameters - -| Parameter | Description | -| ------------------- | ------------------------------------ | -| const char* cfgfile | Name of the HSAK configuration file. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------- | -| Other values | Initialization failed. | -| 0 | Initialization succeeded. | - -##### libstorage_exit_module - -1. Prototype - -``` -int32_t libstorage_exit_module(void); -``` - -2. Description - -Exits the HSAK module. - -3. Parameters - -None. - -4. Return value - -| Return Value | Description | -| ------------ | --------------------------------- | -| Other values | Failed to exit the cleanup. | -| 0 | Succeeded in exiting the cleanup. | - -##### LIBSTORAGE_REGISTER_DPDK_INIT_NOTIFY - -1. Prototype - -``` -LIBSTORAGE_REGISTER_DPDK_INIT_NOTIFY(_name, _notify) -``` - -2. Description - -Service layer registration function, which is used to register the callback function when the DPDK initialization is complete. - -3. Parameters - -| Parameter | Description | -| --------- | ------------------------------------------------------------ | -| _name | Name of a module at the service layer. | -| _notify | Prototype of the callback function registered at the service layer: **void (*notifyFunc)(const struct libstorage_dpdk_init_notify_arg *arg);** | - -4. Return value - -None - -#### ublock.h - -##### init_ublock - -1. Prototype - -``` -int init_ublock(const char *name, enum ublock_rpc_server_status flg); -``` - -2. Description - -Initializes the Ublock module. This API must be called before other Ublock APIs. If the flag is set to **UBLOCK_RPC_SERVER_ENABLE**, that is, Ublock functions as the RPC server, the same process can be initialized only once. - -When Ublock is started as the RPC server, the monitor thread of a server is started at the same time. When the monitor thread detects that the RPC server thread is abnormal (for example, thread suspended), the monitor thread calls the exit function to trigger the process to exit. - -In this case, the product script is used to start the process again. - -3. Parameters - -| Parameter | Description | -| ------------------------------------ | ------------------------------------------------------------ | -| const char *name | Module name. The default value is **ublock**. You are advised to set this parameter to **NULL**. | -| enum ublock_rpc_server_status
flg | Whether to enable RPC. The value can be **UBLOCK_RPC_SERVER_DISABLE** or **UBLOCK_RPC_SERVER_ENABLE**.
If RPC is disabled and the drive is occupied by service processes, the Ublock module cannot obtain the drive information. | - -4. Return value - -| Return Value | Description | -| ------------- | ------------------------------------------------------------ | -| 0 | Initialization succeeded. | -| -1 | Initialization failed. Possible cause: The Ublock module has been initialized. | -| Process exits | Ublock considers that the following exceptions cannot be rectified and directly calls the exit API to exit the process:
- The RPC service needs to be created, but it fails to be created onsite.
- Failed to create a hot swap monitoring thread. | - -##### ublock_init - -1. Prototype - -``` -#define ublock_init(name) init_ublock(name, UBLOCK_RPC_SERVER_ENABLE) -``` - -2. Description - -It is the macro definition of the init_ublock API. It can be regarded as initializing Ublock into the required RPC service. - -3. Parameters - -| Parameter | Description | -| --------- | ------------------------------------------------------------ | -| name | Module name. The default value is **ublock**. You are advised to set this parameter to **NULL**. | - -4. Return value - -| Return Value | Description | -| ------------- | ------------------------------------------------------------ | -| 0 | Initialization succeeded. | -| -1 | Initialization failed. Possible cause: The Ublock RPC server module has been initialized. | -| Process exits | Ublock considers that the following exceptions cannot be rectified and directly calls the exit API to exit the process:
- The RPC service needs to be created, but it fails to be created onsite.
- Failed to create a hot swap monitoring thread. | - -##### ublock_init_norpc - -1. Prototype - -``` -#define ublock_init_norpc(name) init_ublock(name, UBLOCK_RPC_SERVER_DISABLE) -``` - -2. Description - -It is the macro definition of the init_ublock API and can be considered as initializing Ublock into a non-RPC service. - -3. Parameters - -| Parameter | Description | -| --------- | ------------------------------------------------------------ | -| name | Module name. The default value is **ublock**. You are advised to set this parameter to **NULL**. | - -4. Return value - -| Return Value | Description | -| ------------- | ------------------------------------------------------------ | -| 0 | Initialization succeeded. | -| -1 | Initialization failed. Possible cause: The Ublock client module has been initialized. | -| Process exits | Ublock considers that the following exceptions cannot be rectified and directly calls the exit API to exit the process:
- The RPC service needs to be created, but it fails to be created onsite.
- Failed to create a hot swap monitoring thread. | - -##### ublock_fini - -1. Prototype - -``` -void ublock_fini(void); -``` - -2. Description - -Destroys the Ublock module and internally created resources. This API must be used together with the Ublock initialization API. - -3. Parameters - -None. - -4. Return value - -None. - -##### ublock_get_bdevs - -1. Prototype - -``` -int ublock_get_bdevs(struct ublock_bdev_mgr* bdev_list); -``` - -2. Description - -Obtains the device list (all NVMe devices in the environment, including kernel-mode and user-mode drivers). The obtained NVMe device list contains only PCI addresses and does not contain specific device information. To obtain specific device information, call ublock_get_bdev. - -3. Parameters - -| Parameter | Description | -| --------------------------------- | ------------------------------------------------------------ | -| struct ublock_bdev_mgr* bdev_list | Output parameter, which returns the device queue. The **bdev_list** pointer must be allocated externally. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------ | -| 0 | The device queue is obtained successfully. | -| -2 | No NVMe device exists in the environment. | -| Other values | Failed to obtain the device list. | - -##### ublock_free_bdevs - -1. Prototype - -``` -void ublock_free_bdevs(struct ublock_bdev_mgr* bdev_list); -``` - -2. Description - -Releases a device list. - -3. Parameters - -| Parameter | Description | -| --------------------------------- | ------------------------------------------------------------ | -| struct ublock_bdev_mgr* bdev_list | Head pointer of the device queue. After the device queue is cleared, the **bdev_list** pointer is not released. | - -4. Return value - -None. - -##### ublock_get_bdev - -1. Prototype - -``` -int ublock_get_bdev(const char *pci, struct ublock_bdev *bdev); -``` - -2. Description - -Obtains information about a specific device. In the device information, the serial number, model, and firmware version of the NVMe device are saved as character arrays instead of character strings. (The return format varies depending on the drive controller, and the arrays may not end with 0.) - -After this API is called, the corresponding device is occupied by Ublock. Therefore, call ublock_free_bdev to free resources immediately after the required service operation is complete. - -3. Parameters - -| Parameter | Description | -| ------------------------ | ------------------------------------------------------------ | -| const char *pci | PCI address of the device whose information needs to be obtained. | -| struct ublock_bdev *bdev | Output parameter, which returns the device information. The **bdev** pointer must be allocated externally. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | The device information is obtained successfully. | -| -1 | Failed to obtain device information due to incorrect parameters. | -| -11(EAGAIN) | Failed to obtain device information due to the RPC query failure. A retry is required (3s sleep is recommended). | - -##### ublock_get_bdev_by_esn - -1. Prototype - -``` -int ublock_get_bdev_by_esn(const char *esn, struct ublock_bdev *bdev); -``` - -2. Description - -Obtains information about the device corresponding to an ESN. In the device information, the serial number, model, and firmware version of the NVMe device are saved as character arrays instead of character strings. (The return format varies depending on the drive controller, and the arrays may not end with 0.) - -After this API is called, the corresponding device is occupied by Ublock. Therefore, call ublock_free_bdev to free resources immediately after the required service operation is complete. - -3. Parameters - -| Parameter | Description | -| ------------------------ | ------------------------------------------------------------ | -| const char *esn | ESN of the device whose information is to be obtained.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string. | -| struct ublock_bdev *bdev | Output parameter, which returns the device information. The **bdev** pointer must be allocated externally. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | The device information is obtained successfully. | -| -1 | Failed to obtain device information due to incorrect parameters. | -| -11(EAGAIN) | Failed to obtain device information due to the RPC query failure. A retry is required (3s sleep is recommended). | - -##### ublock_free_bdev - -1. Prototype - -``` -void ublock_free_bdev(struct ublock_bdev *bdev); -``` - -2. Description - -Frees device resources. - -3. Parameters - -| Parameter | Description | -| ------------------------ | ------------------------------------------------------------ | -| struct ublock_bdev *bdev | Pointer to the device information. After the data in the pointer is cleared, the **bdev** pointer is not freed. | - -4. Return value - -None. - -##### TAILQ_FOREACH_SAFE - -1. Prototype - -``` -#define TAILQ_FOREACH_SAFE(var, head, field, tvar) -for ((var) = TAILQ_FIRST((head)); -(var) && ((tvar) = TAILQ_NEXT((var), field), 1); -(var) = (tvar)) -``` - -2. Description - -Provides a macro definition for each member of the secure access queue. - -3. Parameters - -| Parameter | Description | -| --------- | ------------------------------------------------------------ | -| var | Queue node member on which you are performing operations. | -| head | Queue head pointer. Generally, it refers to the object address defined by **TAILQ_HEAD(xx, xx) obj**. | -| field | Name of the struct used to store the pointers before and after the queue in the queue node. Generally, it is the name defined by **TAILQ_ENTRY (xx) name**. | -| tvar | Next queue node member. | - -4. Return value - -None. - -##### ublock_get_SMART_info - -1. Prototype - -``` -int ublock_get_SMART_info(const char *pci, uint32_t nsid, struct ublock_SMART_info *smart_info); -``` - -2. Description - -Obtains the S.M.A.R.T. information of a specified device. - -3. Parameters - -| Parameter | Description | -| ------------------------------------ | ------------------------------------------------------------ | -| const char *pci | Device PCI address. | -| uint32_t nsid | Specified namespace. | -| struct ublock_SMART_info *smart_info | Output parameter, which returns the S.M.A.R.T. information of the device. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | The S.M.A.R.T. information is obtained successfully. | -| -1 | Failed to obtain S.M.A.R.T. information due to incorrect parameters. | -| -11(EAGAIN) | Failed to obtain S.M.A.R.T. information due to the RPC query failure. A retry is required (3s sleep is recommended). | - -##### ublock_get_SMART_info_by_esn - -1. Prototype - -``` -int ublock_get_SMART_info_by_esn(const char *esn, uint32_t nsid, struct ublock_SMART_info *smart_info); -``` - -2. Description - -Obtains the S.M.A.R.T. information of the device corresponding to an ESN. - -3. Parameters - -| Parameter | Description | -| --------------------------------------- | ------------------------------------------------------------ | -| const char *esn | Device ESN.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string. | -| uint32_t nsid | Specified namespace. | -| struct ublock_SMART_info
*smart_info | Output parameter, which returns the S.M.A.R.T. information of the device. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | The S.M.A.R.T. information is obtained successfully. | -| -1 | Failed to obtain SMART information due to incorrect parameters. | -| -11(EAGAIN) | Failed to obtain S.M.A.R.T. information due to the RPC query failure. A retry is required (3s sleep is recommended). | - -##### ublock_get_error_log_info - -1. Prototype - -``` -int ublock_get_error_log_info(const char *pci, uint32_t err_entries, struct ublock_nvme_error_info *errlog_info); -``` - -2. Description - -Obtains the error log information of a specified device. - -3. Parameters - -| Parameter | Description | -| ------------------------------------------ | ------------------------------------------------------------ | -| const char *pci | Device PCI address. | -| uint32_t err_entries | Number of error logs to be obtained. A maximum of 256 error logs can be obtained. | -| struct ublock_nvme_error_info *errlog_info | Output parameter, which returns the error log information of the device. For the **errlog_info** pointer, the caller needs to apply for space and ensure that the obtained space is greater than or equal to err_entries x size of (struct ublock_nvme_error_info). | - -4. Return value - -| Return Value | Description | -| ------------------------------------------------------------ | ------------------------------------------------------------ | -| Number of obtained error logs. The value is greater than or equal to 0. | Error logs are obtained successfully. | -| -1 | Failed to obtain error logs due to incorrect parameters. | -| -11(EAGAIN) | Failed to obtain error logs due to the RPC query failure. A retry is required (3s sleep is recommended). | - -##### ublock_get_log_page - -1. Prototype - -``` -int ublock_get_log_page(const char *pci, uint8_t log_page, uint32_t nsid, void *payload, uint32_t payload_size); -``` - -2. Description - -Obtains information about a specified device and log page. - -3. Parameters - -| Parameter | Description | -| --------------------- | ------------------------------------------------------------ | -| const char *pci | Device PCI address. | -| uint8_t log_page | ID of the log page to be obtained. For example, **0xC0** and **0xCA** indicate the customized S.M.A.R.T. information of ES3000 V5 drives. | -| uint32_t nsid | Namespace ID. Some log pages support obtaining by namespace while some do not. If obtaining by namespace is not supported, the caller must transfer **0XFFFFFFFF**. | -| void *payload | Output parameter, which stores log page information. The caller is responsible for allocating memory. | -| uint32_t payload_size | Size of the applied payload, which cannot be greater than 4096 bytes. | - -4. Return value - -| Return Value | Description | -| ------------ | ---------------------------------------------------- | -| 0 | The log page is obtained successfully. | -| -1 | Failed to obtain error logs due to parameter errors. | - -##### ublock_info_get_pci_addr - -1. Prototype - -``` -char *ublock_info_get_pci_addr(const void *info); -``` - -2. Description - -Obtains the PCI address of the hot swap device. - -The memory occupied by info and the memory occupied by the returned PCI address do not need to be freed by the service process. - -3. Parameters - -| Parameter | Description | -| ---------------- | ------------------------------------------------------------ | -| const void *info | Hot swap event information transferred by the hot swap monitoring thread to the callback function. | - -4. Return value - -| Return Value | Description | -| ------------ | --------------------------------- | -| NULL | Failed to obtain the information. | -| Other values | Obtained PCI address. | - -##### ublock_info_get_action - -1. Prototype - -``` -enum ublock_nvme_uevent_action ublock_info_get_action(const void *info); -``` - -2. Description - -Obtains the type of the hot swap event. - -The memory occupied by info does not need to be freed by service process. - -3. Parameters - -| Parameter | Description | -| ---------------- | ------------------------------------------------------------ | -| const void *info | Hot swap event information transferred by the hot swap monitoring thread to the callback function. | - -4. Return value - -| Return Value | Description | -| -------------------------- | ------------------------------------------------------------ | -| Type of the hot swap event | Type of the event that triggers the callback function. For details, see the definition in **5.1.2.6 enum ublock_nvme_uevent_action**. | - -##### ublock_get_ctrl_iostat - -1. Prototype - -``` -int ublock_get_ctrl_iostat(const char* pci, struct ublock_ctrl_iostat_info *ctrl_iostat); -``` - -2. Description - -Obtains the I/O statistics of a controller. - -3. Parameters - -| Parameter | Description | -| ------------------------------------------- | ------------------------------------------------------------ | -| const char* pci | PCI address of the controller whose I/O statistics are to be obtained. | -| struct ublock_ctrl_iostat_info *ctrl_iostat | Output parameter, which returns I/O statistics. The **ctrl_iostat** pointer must be allocated externally. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------------------------ | -| 0 | Succeeded in obtaining I/O statistics. | -| -1 | Failed to obtain I/O statistics due to invalid parameters or RPC errors. | -| -2 | Failed to obtain I/O statistics because the NVMe drive is not taken over by the I/O process. | -| -3 | Failed to obtain I/O statistics because the I/O statistics function is disabled. | - -##### ublock_nvme_admin_passthru - -1. Prototype - -``` -int32_t ublock_nvme_admin_passthru(const char *pci, void *cmd, void *buf, size_t nbytes); -``` - -2. Description - -Transparently transmits the **nvme admin** command to the NVMe device. Currently, only the **nvme admin** command for obtaining the identify parameter is supported. - -3. Parameters - -| Parameter | Description | -| --------------- | ------------------------------------------------------------ | -| const char *pci | PCI address of the destination controller of the **nvme admin** command. | -| void *cmd | Pointer to the **nvme admin** command struct. The struct size is 64 bytes. For details, see the NVMe specifications. Currently, only the command for obtaining the identify parameter is supported. | -| void *buf | Saves the output of the **nvme admin** command. The space is allocated by users and the size is expressed in nbytes. | -| size_t nbytes | Size of the user buffer. The buffer for the identify parameter is 4096 bytes, and that for the command to obtain the identify parameter is 4096 nbytes. | - -4. Return value - -| Return Value | Description | -| ------------ | ------------------------------------------ | -| 0 | The user command is executed successfully. | -| -1 | Failed to execute the user command. | - -## Appendixes - -### GENERIC - -Generic Error Code Reference - -| sc | value | -| ------------------------------------------ | ----- | -| NVME_SC_SUCCESS | 0x00 | -| NVME_SC_INVALID_OPCODE | 0x01 | -| NVME_SC_INVALID_FIELD | 0x02 | -| NVME_SC_COMMAND_ID_CONFLICT | 0x03 | -| NVME_SC_DATA_TRANSFER_ERROR | 0x04 | -| NVME_SC_ABORTED_POWER_LOSS | 0x05 | -| NVME_SC_INTERNAL_DEVICE_ERROR | 0x06 | -| NVME_SC_ABORTED_BY_REQUEST | 0x07 | -| NVME_SC_ABORTED_SQ_DELETION | 0x08 | -| NVME_SC_ABORTED_FAILED_FUSED | 0x09 | -| NVME_SC_ABORTED_MISSING_FUSED | 0x0a | -| NVME_SC_INVALID_NAMESPACE_OR_FORMAT | 0x0b | -| NVME_SC_COMMAND_SEQUENCE_ERROR | 0x0c | -| NVME_SC_INVALID_SGL_SEG_DESCRIPTOR | 0x0d | -| NVME_SC_INVALID_NUM_SGL_DESCIRPTORS | 0x0e | -| NVME_SC_DATA_SGL_LENGTH_INVALID | 0x0f | -| NVME_SC_METADATA_SGL_LENGTH_INVALID | 0x10 | -| NVME_SC_SGL_DESCRIPTOR_TYPE_INVALID | 0x11 | -| NVME_SC_INVALID_CONTROLLER_MEM_BUF | 0x12 | -| NVME_SC_INVALID_PRP_OFFSET | 0x13 | -| NVME_SC_ATOMIC_WRITE_UNIT_EXCEEDED | 0x14 | -| NVME_SC_OPERATION_DENIED | 0x15 | -| NVME_SC_INVALID_SGL_OFFSET | 0x16 | -| NVME_SC_INVALID_SGL_SUBTYPE | 0x17 | -| NVME_SC_HOSTID_INCONSISTENT_FORMAT | 0x18 | -| NVME_SC_KEEP_ALIVE_EXPIRED | 0x19 | -| NVME_SC_KEEP_ALIVE_INVALID | 0x1a | -| NVME_SC_ABORTED_PREEMPT | 0x1b | -| NVME_SC_SANITIZE_FAILED | 0x1c | -| NVME_SC_SANITIZE_IN_PROGRESS | 0x1d | -| NVME_SC_SGL_DATA_BLOCK_GRANULARITY_INVALID | 0x1e | -| NVME_SC_COMMAND_INVALID_IN_CMB | 0x1f | -| NVME_SC_LBA_OUT_OF_RANGE | 0x80 | -| NVME_SC_CAPACITY_EXCEEDED | 0x81 | -| NVME_SC_NAMESPACE_NOT_READY | 0x82 | -| NVME_SC_RESERVATION_CONFLICT | 0x83 | -| NVME_SC_FORMAT_IN_PROGRESS | 0x84 | - -### COMMAND_SPECIFIC - -Error Code Reference for Specific Commands - -| sc | value | -| ------------------------------------------ | ----- | -| NVME_SC_COMPLETION_QUEUE_INVALID | 0x00 | -| NVME_SC_INVALID_QUEUE_IDENTIFIER | 0x01 | -| NVME_SC_MAXIMUM_QUEUE_SIZE_EXCEEDED | 0x02 | -| NVME_SC_ABORT_COMMAND_LIMIT_EXCEEDED | 0x03 | -| NVME_SC_ASYNC_EVENT_REQUEST_LIMIT_EXCEEDED | 0x05 | -| NVME_SC_INVALID_FIRMWARE_SLOT | 0x06 | -| NVME_SC_INVALID_FIRMWARE_IMAGE | 0x07 | -| NVME_SC_INVALID_INTERRUPT_VECTOR | 0x08 | -| NVME_SC_INVALID_LOG_PAGE | 0x09 | -| NVME_SC_INVALID_FORMAT | 0x0a | -| NVME_SC_FIRMWARE_REQ_CONVENTIONAL_RESET | 0x0b | -| NVME_SC_INVALID_QUEUE_DELETION | 0x0c | -| NVME_SC_FEATURE_ID_NOT_SAVEABLE | 0x0d | -| NVME_SC_FEATURE_NOT_CHANGEABLE | 0x0e | -| NVME_SC_FEATURE_NOT_NAMESPACE_SPECIFIC | 0x0f | -| NVME_SC_FIRMWARE_REQ_NVM_RESET | 0x10 | -| NVME_SC_FIRMWARE_REQ_RESET | 0x11 | -| NVME_SC_FIRMWARE_REQ_MAX_TIME_VIOLATION | 0x12 | -| NVME_SC_FIRMWARE_ACTIVATION_PROHIBITED | 0x13 | -| NVME_SC_OVERLAPPING_RANGE | 0x14 | -| NVME_SC_NAMESPACE_INSUFFICIENT_CAPACITY | 0x15 | -| NVME_SC_NAMESPACE_ID_UNAVAILABLE | 0x16 | -| NVME_SC_NAMESPACE_ALREADY_ATTACHED | 0x18 | -| NVME_SC_NAMESPACE_IS_PRIVATE | 0x19 | -| NVME_SC_NAMESPACE_NOT_ATTACHED | 0x1a | -| NVME_SC_THINPROVISIONING_NOT_SUPPORTED | 0x1b | -| NVME_SC_CONTROLLER_LIST_INVALID | 0x1c | -| NVME_SC_DEVICE_SELF_TEST_IN_PROGRESS | 0x1d | -| NVME_SC_BOOT_PARTITION_WRITE_PROHIBITED | 0x1e | -| NVME_SC_INVALID_CTRLR_ID | 0x1f | -| NVME_SC_INVALID_SECONDARY_CTRLR_STATE | 0x20 | -| NVME_SC_INVALID_NUM_CTRLR_RESOURCES | 0x21 | -| NVME_SC_INVALID_RESOURCE_ID | 0x22 | -| NVME_SC_CONFLICTING_ATTRIBUTES | 0x80 | -| NVME_SC_INVALID_PROTECTION_INFO | 0x81 | -| NVME_SC_ATTEMPTED_WRITE_TO_RO_PAGE | 0x82 | - -### MEDIA_DATA_INTERGRITY_ERROR - -Error Code Reference for Medium Exceptions - -| sc | value | -| -------------------------------------- | ----- | -| NVME_SC_WRITE_FAULTS | 0x80 | -| NVME_SC_UNRECOVERED_READ_ERROR | 0x81 | -| NVME_SC_GUARD_CHECK_ERROR | 0x82 | -| NVME_SC_APPLICATION_TAG_CHECK_ERROR | 0x83 | -| NVME_SC_REFERENCE_TAG_CHECK_ERROR | 0x84 | -| NVME_SC_COMPARE_FAILURE | 0x85 | -| NVME_SC_ACCESS_DENIED | 0x86 | -| NVME_SC_DEALLOCATED_OR_UNWRITTEN_BLOCK | 0x87 | \ No newline at end of file diff --git a/docs/en/docs/HSAK/hsak_tools_usage.md b/docs/en/docs/HSAK/hsak_tools_usage.md deleted file mode 100644 index e7268ba81652164dee22c13c8c7efbada4329ebe..0000000000000000000000000000000000000000 --- a/docs/en/docs/HSAK/hsak_tools_usage.md +++ /dev/null @@ -1,123 +0,0 @@ -# Command-Line Interface - -## Command for Querying Drive Information - -### Format - -```shell -libstorage-list [] [] -``` - -### Parameters - -- *commands*: Only **help** is available. **libstorage-list help** is used to display the help information. - -- *device*: specifies the PCI address. The format is **0000:09:00.0**. Multiple PCI addresses are allowed and separated by spaces. If no specific PCI address is set, the command line lists all enumerated device information. - -### Precautions - -- The fault injection function applies only to development, debugging, and test scenarios. Do not use this function on live networks. Otherwise, service and security risks may occur. - -- Before running this command, ensure that the management component (Ublock) server has been started, and the user-mode I/O component (UIO) has not been started or has been correctly started. - -- Drives that are not occupied by the Ublock or UIO component will be occupied during the command execution. If the Ublock or UIO component attempts to obtain the drive control permission, a storage device access conflict may occur. As a result, the command execution fails. - -## Command for Switching Drivers for Drives - -### Format - -```shell -libstorage-shutdown reset [ ...] -``` - -### Parameters - -- **reset**: switches the UIO driver to the kernel-mode driver for a specific drive. - -- *device*: specifies the PCI address, for example, **0000:09:00.0**. Multiple PCI addresses are allowed and separated by spaces. - -### Precautions - -- The **libstorage-shutdown reset** command is used to switch a drive from the user-mode UIO driver to the kernel-mode NVMe driver. - -- Before running this command, ensure that the Ublock server has been started, and the UIO component has not been started or has been correctly started. - -- The **libstoage-shutdown reset** command is risky. Before switching to the NVMe driver, ensure that the user-mode instance has stopped delivering I/Os to the NVMe device, all FDs on the NVMe device have been disabled, and the instance that accesses the NVMe device has exited. - -## Command for Obtaining I/O Statistics - -### Format - -```shell -libstorage-iostat [-t ] [-i ] [-d ] -``` - -### Parameters - -- -**t**: interval, in seconds. The value ranges from 1 to 3600. This parameter is of the int type. If the input parameter value exceeds the upper limit of the int type, the value is truncated to a negative or positive number. - -- -**i**: number of collection times. The minimum value is **1** and the maximum value is *MAX_INT*. If this parameter is not set, information is collected at an interval by default. This parameter is of the int type. If the input parameter value exceeds the upper limit of the int type, the value is truncated to a negative or positive number. - -- -**d**: name of a block device (for example, **nvme0n1**, which depends on the controller name configured in **/etc/spdk/nvme.conf.in**). You can use this parameter to collect performance data of one or more specified devices. If this parameter is not set, performance data of all detected devices is collected. - -### Precautions - -- The I/O statistics configuration is enabled. - -- The process has delivered I/O operations to the drive whose performance information needs to be queried through the UIO component. - -- If no device in the current environment is occupied by service processes to deliver I/Os, the command exits after the message "You cannot get iostat info for nvme device no deliver io" is displayed. - -- When multiple queues are enabled on a drive, the I/O statistics tool summarizes the performance data of multiple queues on the drive and outputs the data in a unified manner. - -- The I/O statistics tool supports data records of a maximum of 8192 drive queues. - -- The I/O statistics are as follows: - - | Device | r/s | w/s | rKB/s | wKB/s | avgrq-sz | avgqu-sz | r_await | w_await | await | svctm | util% | poll-n | - | ----------- | ------------------------------ | ------------------------------- | ----------------------------------- | ------------------------------------ | -------------------------------------- | -------------------------- | --------------------- | ---------------------- | ------------------------------- | --------------------------------------- | ------------------ | -------------------------- | - | Device name | Number of read I/Os per second | Number of write I/Os per second | Number of read I/O bytes per second | Number of write I/O bytes per second | Average size of delivered I/Os (bytes) | I/O depth of a drive queue | I/O read latency (μs) | I/O write latency (μs) | Average read/write latency (μs) | Processing latency of a single I/O (μs) | Device utilization | Number of polling timeouts | - -## Commands for Drive Read/Write Operations - -### Format - -```shell -libstorage-rw [OPTIONS...] -``` - -### Parameters - -1. **COMMAND** parameters - - - **read**: reads a specified logical block from the device to the data buffer (standard output by default). - - - **write**: writes data in a data buffer (standard input by default) to a specified logical block of the NVMe device. - - - **help**: displays the help information about the command line. - -2. **device**: specifies the PCI address, for example, **0000:09:00.0**. - -3. **OPTIONS** parameters - - - **--start-block, -s**: indicates the 64-bit start address of the logical block to be read or written. The default value is **0**. - - - **--block-count, -c**: indicates the number of the logical blocks to be read or written (counted from 0). - - - **--data-size, -z**: indicates the number of bytes of the data to be read or written. - - - **--namespace-id, -n**: indicates the namespace ID of the device. The default value is **1**. - - - **--data, -d**: indicates the data file used for read and write operations (The read data is saved during read operations and the written data is provided during write operations.) - - - **--limited-retry, -l**: indicates that the device controller restarts for a limited number of times to complete device read and write operations. - - - **--force-unit-access, -f**: ensures that read and write operations are completed from the nonvolatile media before the instruction is completed. - - - **--show-command, -v**: displays instruction information before sending a read/write command. - - - **--dry-run, -w**: displays only information about read and write instructions but does not perform actual read and write operations. - - - **--latency. -t**: collects statistics on the end-to-end read and write latency of the CLI. - - - **--help, -h**: displays the help information about related commands. diff --git a/docs/en/docs/HSAK/introduce_hsak.md b/docs/en/docs/HSAK/introduce_hsak.md deleted file mode 100644 index 031efb9c3fd3b9658c100ef6864970b77c80b073..0000000000000000000000000000000000000000 --- a/docs/en/docs/HSAK/introduce_hsak.md +++ /dev/null @@ -1,47 +0,0 @@ -# HSAK Developer Guide - -## Overview - -As the performance of storage media such as NVMe SSDs and SCMs is continuously improved, the latency overhead of the media layer in the I/O stack continuously reduces, and the overhead of the software stack becomes a bottleneck. Therefore, the kernel I/O data plane needs to be reconstructed to reduce the overhead of the software stack. HSAK provides a high-bandwidth and low-latency I/O software stack for new storage media, which reduces the overhead by more than 50% compared with the traditional I/O software stack. -The HSAK user-mode I/O engine is developed based on the open-source SPDK. - -1. A unified interface is provided for external systems to shield the differences between open-source interfaces. -2. Enhanced I/O data plane features are added, such as DIF, drive formatting, batch I/O delivery, trim, and dynamic drive addition and deletion. -3. Features such as drive device management, drive I/O monitoring, and maintenance and test tools are provided. - -## Compilation Tutorial - -1. Download the HSAK source code. - - $ git clone - -2. Install the compilation and running dependencies. - - The compilation and running of HSAK depend on components such as Storage Performance Development Kit (SPDK), Data Plane Development Kit (DPDK), and libboundscheck. - -3. Start the compilation. - - $ cd hsak - - $ mkdir build - - $ cd build - - $ cmake .. - - $ make - -## Precautions - -### Constraints - -- A maximum of 512 NVMe devices can be used and managed on the same machine. -- When HSAK is enabled to execute I/O-related services, ensure that the system has at least 500 MB continuous idle huge page memory. -- Before enabling the user-mode I/O component to execute services, ensure that the drive management component (Ublock) has been enabled. -- When the drive management component (Ublock) is enabled to execute services, ensure that the system has sufficient continuous idle memory. Each time the Ublock component is initialized, 20 MB huge page memory is allocated. -- Before HSAK is run, **setup.sh** is called to configure huge page memory and unbind the kernel-mode driver of the NVMe device. -- Other interfaces provided by the HSAK module can be used only after libstorage_init_module is successfully executed. Each process can call libstorage_init_module only once. -- After the libstorage_exit_module function is executed, other interfaces provided by HSAK cannot be used. In multi-thread scenarios, exit HSAK after all threads end. -- Only one service can be started for the HSAK Ublock component on a server and supports concurrent access of a maximum of 64 Ublock clients. The Ublock server can process a maximum of 20 client requests per second. -- The HSAK Ublock component must be started earlier than the data plane I/O component and Ublock clients. The command line tool provided by HSAK can be executed only after the Ublock server is started. -- Do not register the function for processing the SIGBUS signal. SPDK has an independent processing function for the signal. If the processing function is overwritten, the registered signal processing function becomes invalid and a core dump occurs. diff --git a/docs/en/docs/Installation/FAQ1.md b/docs/en/docs/Installation/FAQ1.md deleted file mode 100644 index eaab85269015e7eddab588cda34550db5ecb09f3..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/FAQ1.md +++ /dev/null @@ -1,66 +0,0 @@ -# FAQs - -## Failed to Start the Raspberry Pi - -### Symptom - -After the Raspberry Pi image released by the openEuler is written to the SD card, the Raspberry Pi fails to be started. - -### Cause Analysis - -The possible causes are as follows: - -1. The downloaded image file is incomplete. To avoid this problem, ensure that the image passes the integrity verification. -2. An error occurs when the image is written to the SD card. In most cases, the error occurs when the image is written to the SD card in the Windows environment using the application software. - -### Solution - -Re-write the complete image to the SD card. - -## Failed to Connect to Wi-Fi by Running the nmcli Command - -### Symptom - -Failed to connect to the Wi-Fi network by running the `nmcli dev wifi connect SSID password PWD` command. An error message, for example, `Error: Connection activation failed: (7) Secrets were required, but not provided.`, is displayed. - -### Cause Analysis - -The command to be executed does not have a password. Note that if the password contains special characters, use single quotation marks to quote the password. If you fail to connect to the Wi-Fi network by running the `nmcli` command line, you are advised to use the nmtui utility for connection. - -### Solution - -Run the `nmtui` command to enter the nmtui utility. Perform the following steps to connect to the Wi-Fi network: - -1. Select **Edit a connection** and press **Enter**. The window for editing network connections is displayed. -2. Press the right arrow key on the keyboard to select **Add**, and then press **Enter**. The window for creating a network connection is displayed. -3. Set the connection type to **Wi-Fi**, press the right arrow key on the keyboard to select **Create**, and press **Enter**. The page for editing Wi-Fi connection information is displayed. -4. On the Wi-Fi connection information page, edit the following information. Other information depends on the specific requirements. After the editing is complete, select **OK** and press **Enter** to return to the window for editing network connections. - 1. Enter the name of the Wi-Fi connection in the **Profile name** text box. You can use the default name, for example, **Wi-Fi connection 1**. - 2. Enter **wlan0** in the **Device** text box. - 3. Enter the SSID of the Wi-Fi network to be connected in the **SSID** text box. - 4. In the **Security** area, select the Wi-Fi password encryption mode as required, for example, **WPA & WPA2 Personal**. - 5. Enter the Wi-Fi password in the **Password** text box. - -5. Select **Back** to return to the home screen of the nmtui utility. -6. Select **Activate a connection** and press **Enter**. The window for activating network connections is displayed. -7. Check whether the added Wi-Fi connection is activated. The name of an activated Wi-Fi connection is marked with an asterisk (*). If the Wi-Fi connection is not activated, select the Wi-Fi connection, press the right arrow key on the keyboard to select **Activate**, and press **Enter** to activate the connection. After the activation is complete, select **Back** and press **Enter** to return to the home screen of the nmtui utility. -8. Select **Quit**, press the right arrow key on the keyboard to select **OK**, and press **Enter** to exit the nmtui utility. - -## Failed to Install the TensorFlow and Related Packages - -### Symptom - -Failed to install the TensorFlow and related packages using **yum**. - -### Cause Analysis - -The dependencies of TensorFlow have not been upgraded to the version that adapts to TensorFlow 2.12.1. You need to manually install the dependencies using **pip**. - -### Solution - -1. Run `yumdownloader python3-tensorflow` to download the TensorFlow RPM package. -2. Run `rpm -ivh --nodeps python3-tensorflow` to install the package. -3. Install TensorFlow dependencies. - 1. Use **pip** to install dependencies: `pip3 install tensorflow-estimator==2.12.0 keras==2.12.0 protobuf==3.20.3` - 2. Use **yum** to install other dependencies: `yum install python3-termcolor python3-future python3-numpy python3-six python3-astunparse python3-google-pasta python3-opt-einsum python3-typing-extensions python3-wrapt python3-h5py python3-grpcio python3-absl-py python3-flatbuffers python3-gast` -4. Use **yum** to install related packages. For example, run `yum install python-keras-rl2` to install python-keras-rl2. diff --git a/docs/en/docs/Installation/Installation-Guide1.md b/docs/en/docs/Installation/Installation-Guide1.md deleted file mode 100644 index db8031a38bcbe874ab2004e36cc2375f89e7d90e..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/Installation-Guide1.md +++ /dev/null @@ -1,188 +0,0 @@ -# Installation Guide - -This section describes how to enable the Raspberry Pi function after [Writing Raspberry Pi Images into the SD card](./Installation-Modes1.md). - - -- [Installation Guide](#installation-guide) - - [Starting the System](#starting-the-system) - - [Logging in to the System](#logging-in-to-the-system) - - [Configuring the System](#configuring-the-system) - - [Expanding the Root Directory Partition](#expanding-the-root-directory-partition) - - [Connecting to the Wi-Fi Network](#connecting-to-the-wi-fi-network) - - -## Starting the System - -After an image is written into the SD card, insert the SD card into the Raspberry Pi and power on the SD card. - -For details about the Raspberry Pi hardware, visit the [Raspberry Pi official website](https://www.raspberrypi.org/). - -## Logging in to the System - -You can log in to the Raspberry Pi in either of the following ways: - -1. Local login - - Connect the Raspberry Pi to the monitor (the Raspberry Pi video output interface is Micro HDMI), keyboard, and mouse, and start the Raspberry Pi. The Raspberry Pi startup log is displayed on the monitor. After Raspberry Pi is started, enter the user name **root** and password **openeuler** to log in. - -2. SSH remote login - - By default, the Raspberry Pi uses the DHCP mode to automatically obtain the IP address. If the Raspberry Pi is connected to a known router, you can log in to the router to check the IP address. The new IP address is the Raspberry Pi IP address. - - **Figure 1** Obtain the IP address - ![](./figures/Obtain the IP address) - - According to the preceding figure, the IP address of the Raspberry Pi is **192.168.31.109**. You can run the `ssh root@192.168.31.109` command and enter the password `openeuler` to remotely log in to the Raspberry Pi. - -## Configuring the System - -### Expanding the Root Directory Partition - -The space of the default root directory partition is small. Therefore, you need to expand the partition capacity before using it. - -To expand the root directory partition capacity, perform the following procedure: - -1. Run the `fdisk -l` command as the root user to check the drive partition information. The command output is as follows: - - ```shell - # fdisk -l - Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors - Units: sectors of 1 * 512 = 512 bytes - Sector size (logical/physical): 512 bytes / 512 bytes - I/O size (minimum/optimal): 512 bytes / 512 bytes - Disklabel type: dos - Disk identifier: 0xf2dc3842 - - Device Boot Start End Sectors Size Id Type - /dev/mmcblk0p1 * 8192 593919 585728 286M c W95 FAT32 (LBA) - /dev/mmcblk0p2 593920 1593343 999424 488M 82 Linux swap / Solaris - /dev/mmcblk0p3 1593344 5044223 3450880 1.7G 83 Linux - ``` - - The drive letter of the SD card is **/dev/mmcblk0**, which contains three partitions: - - - **/dev/mmcblk0p1**: boot partition - - **/dev/mmcblk0p2**: swap partition - - **/dev/mmcblk0p3**: root directory partition - - Here, we need to expand the capacity of `/dev/mmcblk0p3`. - -2. Run the `fdisk /dev/mmcblk0` command as the root user and the interactive command line interface (CLI) is displayed. To expand the partition capacity, perform the following procedure as shown in [Figure 2](#zh-cn_topic_0151920806_f6ff7658b349942ea87f4521c0256c315). - - 1. Enter `p` to check the partition information. - - Record the start sector number of `/dev/mmcblk0p3`. That is, the value in the `Start` column of the `/dev/mmcblk0p3` information. In the example, the start sector number is `1593344`. - - 2. Enter `d` to delete the partition. - - 3. Enter `3` or press `Enter` to delete the partition whose number is `3`. That is, the `/dev/mmcblk0p3`. - - 4. Enter `n` to create a partition. - - 5. Enter `p` or press `Enter` to create a partition of the `Primary` type. - - 6. Enter `3` or press `Enter` to create a partition whose number is `3`. That is, the `/dev/mmcblk0p3`. - - 7. Enter the start sector number of the new partition. That is, the start sector number recorded in Step `1`. In the example, the start sector number is `1593344`. - - > ![](./public_sys-resources/icon-notice.gif) **NOTE:** -Do not press **Enter** or use the default parameters. - - 8. Press `Enter` to use the last sector number by default as the end sector number of the new partition. - - 9. Enter `N` without changing the sector ID. - - 10. Enter `w` to save the partition settings and exit the interactive CLI. - - **Figure 2** Expand the partition capacity -![](./figures/Expand the partition capacity) - -3. Run the `fdisk -l` command as the root user to check the drive partition information and ensure that the drive partition is correct. The command output is as follows: - - ```shell - # fdisk -l - Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors - Units: sectors of 1 * 512 = 512 bytes - Sector size (logical/physical): 512 bytes / 512 bytes - I/O size (minimum/optimal): 512 bytes / 512 bytes - Disklabel type: dos - Disk identifier: 0xf2dc3842 - - Device Boot Start End Sectors Size Id Type - /dev/mmcblk0p1 * 8192 593919 585728 286M c W95 FAT32 (LBA) - /dev/mmcblk0p2 593920 1593343 999424 488M 82 Linux swap / Solaris - /dev/mmcblk0p3 1593344 31116287 29522944 14.1G 83 Linux - ``` - -4. Run the `resize2fs /dev/mmcblk0p3` command as the root user to increase the size of the unloaded file system. - -5. Run the `df -lh` command to check the drive space information and ensure that the root directory partition has been expanded. - - > ![](./public_sys-resources/icon-notice.gif) **NOTE:** -If the root directory partition is not expanded, run the `reboot` command to restart the Raspberry Pi and then run the `resize2fs /dev/mmcblk0p3` command as the root user. - -### Connecting to the Wi-Fi Network - -To connect to the Wi-Fi network, perform the following procedure: - -1. Check the IP address and network adapter information. - - `ip a` - - Obtain information about the wireless network adapter **wlan0**: - - ```text - 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - inet6 ::1/128 scope host - valid_lft forever preferred_lft forever - 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 - link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff - inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0 - valid_lft 41570sec preferred_lft 41570sec - inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute - valid_lft forever preferred_lft forever - 3: wlan0: mtu 1500 qdisc fq_codel state DOWN group default qlen 1000 - link/ether e2:e6:99:89:47:0c brd ff:ff:ff:ff:ff:ff - ``` - -2. Scan information about available Wi-Fi networks. - - `nmcli dev wifi` - -3. Connect to the Wi-Fi network. - - Run the `nmcli dev wifi connect SSID password PWD` command as the root user to connect to the Wi-Fi network. - - In the command, `SSID` indicates the SSID of the available Wi-Fi network scanned in the preceding step, and `PWD` indicates the password of the Wi-Fi network. For example, if the `SSID` is `openEuler-wifi`and the password is `12345678`, the command for connecting to the Wi-Fi network is `nmcli dev wifi connect openEuler-wifi password 12345678`. The connection is successful. - - ```text - Device 'wlan0' successfully activated with '26becaab-4adc-4c8e-9bf0-1d63cf5fa3f1'. - ``` - -4. Check the IP address and wireless network adapter information. - - `ip a` - - ```text - 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - inet6 ::1/128 scope host - valid_lft forever preferred_lft forever - 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 - link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff - inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0 - valid_lft 41386sec preferred_lft 41386sec - inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute - valid_lft forever preferred_lft forever - 3: wlan0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 - link/ether dc:a6:32:50:de:58 brd ff:ff:ff:ff:ff:ff - inet 192.168.31.110/24 brd 192.168.31.255 scope global dynamic noprefixroute wlan0 - valid_lft 43094sec preferred_lft 43094sec - inet6 fe80::394:d086:27fa:deba/64 scope link noprefixroute - valid_lft forever preferred_lft forever - ``` diff --git a/docs/en/docs/Installation/Installation-Modes1.md b/docs/en/docs/Installation/Installation-Modes1.md deleted file mode 100644 index 4b5318c7de3dbe690af342b540044312a13a5648..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/Installation-Modes1.md +++ /dev/null @@ -1,117 +0,0 @@ -# Installation Modes - -> ![](./public_sys-resources/icon-notice.gif) **NOTE** -> -> - The hardware supports only Raspberry Pi 3B/3B+/4B. -> - The installation is performed by writing images to the SD card. This section describes how to write images on Windows, Linux, and Mac. -> - The image used in this section is the Raspberry Pi image of openEuler. For details about how to obtain the image, see [Installation Preparations](Installation-Preparations1.md). - - - -- [Installation Modes](#installation-modes) - - [Writing Images on Windows](#writing-images-on-windows) - - [Formatting the SD Card](#formatting-the-sd-card) - - [Writing Images to the SD Card](#writing-images-to-the-sd-card) - - [Writing Images on Linux](#writing-images-on-linux) - - [Checking Drive Partition Information](#checking-drive-partition-information) - - [Unmounting the SD Card](#unmounting-the-sd-card) - - [Writing Images to the SD Card](#writing-images-to-the-sd-card-1) - - [Writing Images on Mac](#writing-images-on-mac) - - [Checking Drive Partition Information](#checking-drive-partition-information-1) - - [Unmounting the SD Card](#unmounting-the-sd-card-1) - - [Writing Images to the SD Card](#writing-images-to-the-sd-card-2) - - -## Writing Images on Windows - -This section uses Windows 10 as an example to describe how to write images to the SD card in the Windows environment. - -### Formatting the SD Card - -To format the SD card, perform the following procedures: - -1. Download and install a SD card formatting tool. The following operations use SD Card Formatter as an example. - -2. Start SD Card Formatter. In **Select card**, select the drive letter of the SD card to be formatted. - - If no image has been installed in the SD card, only one drive letter exists. In **Select card**, select the drive letter of the SD card to be formatted. - - If an image has been installed in the SD card, one or more drive letters exist. For example, the SD card corresponds to three drive letters: E, G, and H. In **Select card**, you can select the drive letter E of the boot partition. - -3. In **Formatting options**, select a formatting mode. The default mode is **Quick format**. - -4. Click **Format** to start formatting. A progress bar is displayed to show the formatting progress. - -5. After the formatting is completed, the message "Formatting was successfully completed" is displayed. Click **OK**. - -### Writing Images to the SD Card - -> ![](./public_sys-resources/icon-notice.gif) **NOTE** -If the compressed image file **openEuler-21.09-raspi-aarch64.img.xz** is obtained, decompress the file to obtain the **openEuler-21.09-raspi-aarch64.img** image file. - -To write the **openEuler-21.09-raspi-aarch64.img** image file to the SD card, perform the following procedures: - -1. Download and install a tool for writing images. The following operations use Win32 Disk Imager as an example. -2. Start Win32 Disk Imager and right-click **Run as administrator**. -3. Select the path of the image file in IMG format from the **Image File** drop-down list box. -4. In **Device**, select the drive letter of the SD card to which the image is written. -5. Click **Write**. A progress bar is displayed to show the progress of writing the image to the SD card. -6. After the write operation is completed, a dialog box is displayed, indicating that the write operation is successfully completed. Click **OK**. - -## Writing Images on Linux - -This section describes how to write images to the SD card in the Linux environment. - -### Checking Drive Partition Information - -Run the ` **fdisk -l** ` command as the **root** user to obtain the drive information of the SD card. For example, the drive partition corresponding to the SD card can be **/dev/sdb**. - -### Unmounting the SD Card - -1. Run the ` **df -lh** ` command to check the mounted volumes. - -2. If the partitions corresponding to the SD card are not mounted, skip this step. If the partitions (for example, /dev/sdb1 and /dev/sdb3) are mounted, run the following commands as the **root** user to unmount them: - - `umount /dev/sdb1` - - `umount /dev/sdb3` - -### Writing Images to the SD Card - -1. If the image obtained is compressed, run the ` **xz -d openEuler-21.09-raspi-aarch64.img.xz** ` command to decompress the compressed file to obtain the **openEuler-21.09-raspi-aarch64.img** image file. Otherwise, skip this step. - -2. Run the following command as the **root** user to write the `openEuler-21.09-raspi-aarch64.img` image to the SD card: - - `dd bs=4M if=openEuler-21.09-raspi-aarch64.img of=/dev/sdb` - - > ![](./public_sys-resources/icon-note.gif) **NOTE** - Generally, the block size is set to 4 MB. If the write operation fails or the written image cannot be used, you can set the block size to 1 MB and try again. However, the write operation is time-consuming when the block size is set to 1 MB. - -## Writing Images on Mac - -This section describes how to flash images to the SD card in the Mac environment. - -### Checking Drive Partition Information - -Run the ` **diskutil list** ` command as the **root** user to obtain the drive information of the SD card. For example, the drive partition corresponding to the SD card can be **/dev/disk3**. - -### Unmounting the SD Card - -1. Run the ` **df -lh** ` command to check the mounted volumes. - -2. If the partitions corresponding to the SD card are not mounted, skip this step. If the partitions (for example, dev/disk3s1 and /dev/disk3s3) are mounted, run the following commands as the **root** user to unmount them: - - `diskutil umount /dev/disk3s1` - - `diskutil umount /dev/disk3s3` - -### Writing Images to the SD Card - -1. If the image obtained is compressed, run the `xz -d openEuler-21.09-raspi-aarch64.img.xz` command to decompress the compressed file to obtain the **openEuler-21.09-raspi-aarch64.img** image file. Otherwise, skip this step. - -2. Run the following command as the **root** user to write the image `openEuler-21.09-raspi-aarch64.img` to the SD card: - - `dd bs=4m if=openEuler-21.09-raspi-aarch64.img of=/dev/disk3` - - > ![](./public_sys-resources/icon-note.gif) **NOTE** - > Generally, the block size is set to 4 MB. If the write operation fails or the written image cannot be used, you can set the block size to 1 MB and try again. However, the write operation is time-consuming when the block size is set to 1 MB. diff --git a/docs/en/docs/Installation/Installation-Preparations1.md b/docs/en/docs/Installation/Installation-Preparations1.md deleted file mode 100644 index e8bf87cbda5ae891dae8721501b8dab9003b00fb..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/Installation-Preparations1.md +++ /dev/null @@ -1,102 +0,0 @@ -# Installation Preparations - -This section describes the compatibility of the hardware and software and the related configurations and preparations required for the installation. - -## Obtaining the Installation Source - -Before installation, obtain the openEuler Raspberry Pi image and its verification file. - -1. Visit [openEuler Repo](https://repo.openeuler.org/). -2. Choose **openEuler 22.03 LTS SP2**. -3. Click **raspi_img**. The download list of Raspberry Pi images is displayed. -4. Click **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz** to download the openEuler Raspberry Pi image to the local PC. -5. Click **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum** to download the verification file of the openEuler Raspberry Pi image to the local PC. - -## Verifying the Image Integrity - -### Overview - -During package transmission, to prevent software packages from being incompletely downloaded due to network or storage device problems, you need to verify the integrity of the software packages after obtaining them. Only the software packages that pass the verification can be deployed. - -Compare the verification value recorded in the verification file with the verification value that is manually calculated to determine whether the software package is complete. If the two values are the same, the downloaded file is complete. Otherwise, the downloaded file is incomplete and you need to obtain the software package again. - -### Prerequisites - -Before verifying the integrity of the image file, ensure that the following files are available: - -Image file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz** -Image file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz** - -Verification file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum** -Verification file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum** - -### Procedures - -To verify the file integrity, perform the following procedures: - -1. Obtain the verification value from the verification file. Run the following command: - - ```shell - cat openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum - ``` - -2. Calculate the SHA256 verification value of the file. Run the following command: - - ```shell - sha256sum openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz - ``` - - After the command is executed, the verification value is displayed. - -3. Check whether the verification values obtained from the step 1 and step 2 are consistent. - - If they are consistent, the downloaded file is not damaged. Otherwise, the downloaded file is incomplete and you need to obtain the file again. - -## Installation Requirements - -If the openEuler OS is installed in the Raspberry Pi environment, the Raspberry Pi environment must meet the following requirements. - -### Hardware Compatibility - -Currently, the openEuler Raspberry Pi image supports the 3B, 3B+, and 4B versions. - -### Minimum Hardware Specifications - -[Table 1](#tff48b99c9bf24b84bb602c53229e2542) lists the minimum hardware specifications for the openEuler Raspberry Pi image. - -**Table 1** Minimum hardware specifications - - - - - - - - - - - - - - - - - - - - - - -

Component Name

-

Minimum Hardware Specifications

-

Description

-

Raspberry Pi version

-
  • Raspberry Pi 3B
  • Raspberry Pi 3B+
  • Raspberry Pi 4B
  • Raspberry Pi 400
-

-

-

Memory

-

≥ 2 GB (4 GB or higher recommended for better user experience)

-

-

-

Drive

-

8 GB or higher recommended for better user experience

-

-

-
diff --git a/docs/en/docs/Installation/Installation.md b/docs/en/docs/Installation/Installation.md deleted file mode 100644 index 1b71043c393ad163228adad09aea51ac3590dddb..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/Installation.md +++ /dev/null @@ -1,5 +0,0 @@ -# Installation Guide - -This guide describes how to install openEuler. - -This guide is intended for openEuler users with a basic understanding of Linux system management, and is also recommended for administrators, system engineers, and maintenance personnel. diff --git a/docs/en/docs/Installation/More-Resources.md b/docs/en/docs/Installation/More-Resources.md deleted file mode 100644 index fab0df5d93e9e7c5283d8d46368d3c4d6402c887..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/More-Resources.md +++ /dev/null @@ -1,4 +0,0 @@ -# Reference - -- How to Create a Raspberry Pi Image File -- How to Use Raspberry Pi diff --git a/docs/en/docs/Installation/RISC-V-LicheePi4A.md b/docs/en/docs/Installation/RISC-V-LicheePi4A.md deleted file mode 100644 index d10c5bf708ea9b9dda285614659b81ef937b8072..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/RISC-V-LicheePi4A.md +++ /dev/null @@ -1,96 +0,0 @@ -# Installing on Licheepi4A - -## Hardware Preparation - -- `Sipeed LicheePi 4A` device (either `8 GB` or `16 GB` version) -- Monitor -- `USB` keyboard and mouse -- Equipment/components required for serial operation (optional) -- `RJ45` network cable and router/switch for wired network connection - -## Device Firmware - -Different memory versions of `LicheePi4A` require different firmware: - -- `u-boot-with-spl-lpi4a.bin` is the u-boot file for the 8 GB version. -- `u-boot-with-spl-lpi4a-16g.bin` is the u-boot file for the 16 GB version. - -The following flashing method uses the `16GB + 128GB` core board as an example, assuming the user already downloaded `base` image and the corresponding u-boot file. - -## Flashing Method - -### Flashing Tools - -Please use `fastboot` command for flashing. You can download `burn_tools.zip` from `https://dl.sipeed.com/shareURL/LICHEE/licheepi4a/07_Tools`. The archive contains flashing tools for `Windows`, `macOS`, and `Linux`. - -### Set Hardware to Enter Flashing Mode - -> Please first check that the DIP switch on the baseboard is set to EMMC boot mode. After confirming, you can proceed with the flashing. - -Hold down the `BOOT` button on the board, then insert the `USB-C` cable to power on the device (the other end of the cable should be connected to a `PC`), entering `USB` flashing mode. -On `Windows`, check the device manager for the `USB download gadget` device. -On `Linux`, use `lsusb` to check for the device, showing: `ID 2345:7654 T-HEAD USB download gadget`. - -### Driver Installation on Windows - -> Note: -> The provided image does not include Windows drivers. You can download `burn_tools.zip` [here](https://dl.sipeed.com/shareURL/LICHEE/licheepi4a/07_Tools) and find the `windows/usb_driver-fullmask` folder inside. This folder contains the drivers needed for Windows. - -To flash on Windows, you need to enter advanced startup mode and disable digital signature enforcement. Follow the steps below to disable digital signature enforcement: - -#### Windows 10 - -1. Go to `Settings` -> `Update & Security` -2. Click `Recovery` on the left, then click `Restart now` under `Advanced startup`. Your computer will restart. Save any ongoing work before proceeding. - -#### Windows 11 - -1. Go to `Settings` -> `System` -> `Recovery` -2. Click `Restart now` under `Advanced startup`. Your computer will restart. Save any ongoing work before proceeding. - -##### After Restart - -1. Click `Troubleshoot`, then `Advanced options` -> `Startup Settings`. The system will restart again. -2. After restarting, select `Disable driver signature enforcement`. This option is usually number 7 but may vary. After selecting the appropriate option, the system will restart again. -3. After rebooting into the system, install the driver. Open `Device Manager`, find `USB download gadget` under `Other devices`, and double-click it. -4. Click `Update driver` under the `General` tab. -5. On the `Browse my computer for drivers` page, paste the path to the `usb_driver-fullmask` directory. -6. Click `Next` to install the driver. - -### Flashing the Image - -After entering flashing mode, use fastboot to flash the image. On macOS or Linux, if fastboot is self-installed, you may need to give it executable permissions. - -#### Windows Steps - -First, add `fastboot` to the system environment variable `PATH`, or place `fastboot` in the same directory. Also, extract the image files. Open `PowerShell` and execute the following commands: - -```bash -# Replace with the u-boot file corresponding to your board version -fastboot flash ram u-boot-with-spl-lpi4a-16g.bin -fastboot reboot -# After rebooting, wait 5 seconds before continuing -# Replace with the u-boot file corresponding to your board version -fastboot flash uboot u-boot-with-spl-lpi4a-16g.bin -fastboot flash boot openEuler-24.03-LTS-riscv64-lpi4a-base-boot.ext4 -fastboot flash root openEuler-24.03-LTS-riscv64-lpi4a-base-root.ext4 -``` - -#### Linux/macOS Steps - -You may need to prefix the fastboot commands with `sudo`. - -```bash -# Replace with the u-boot file corresponding to your board version -sudo fastboot flash ram u-boot-with-spl-lpi4a-16g.bin -sudo fastboot reboot -# After rebooting, wait 5 seconds before continuing -# Replace with the u-boot file corresponding to your board version -sudo fastboot flash uboot u-boot-with-spl-lpi4a-16g.bin -sudo fastboot flash boot openEuler-24.03-LTS-riscv64-lpi4a-base-boot.ext4 -sudo fastboot flash root openEuler-24.03-LTS-riscv64-lpi4a-base-root.ext4 -``` - -## Hardware Availability - -The official release is based on the [`openEuler kernel6.6`](./RISCV-OLK6.6.md) version, and not all kernel modules are fully supported. This version emphasizes a consistent official ecosystem experience. For more complete hardware functionality, use third-party releases. diff --git a/docs/en/docs/Installation/RISC-V-Pioneer1.3.md b/docs/en/docs/Installation/RISC-V-Pioneer1.3.md deleted file mode 100644 index feb321f4e7886cdbaf76dc8070f001c05367e8f2..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/RISC-V-Pioneer1.3.md +++ /dev/null @@ -1,104 +0,0 @@ -# Installing on Pioneer Box - -## Hardware Preparation - -- `Milk-V Pioneer v1.3` device or motherboard (with necessary peripherals) - 1 set - -- `m.2 NVMe` solid-state drive - 1 unit - -> If it contains data, format it to clear the data (make sure to back up personal files) -> -> If you have a `PCIe` adapter card, place it in the first `PCIe` slot of the device (recommended) -> -> If no `PCIe` adapter card, use the onboard `NVMe` interface - -- `AMD R5 230` graphics card - 1 unit - -> Place it in the second `PCIe` slot of the device - -- `USB` flash drive - 1 unit - -> Should be at least `16GiB` - -- `microSD card` - 1 unit - -> Should be at least `4GiB` - -- Monitor (the display interface should match the graphics card) - -- `USB` keyboard and mouse - 1 set - -- Equipment/components required for serial operation (optional) - -- `RJ45` network cable - at least 1, and router/switch for wired network connection - -> It is recommended to use the device's onboard `RJ45` network port rather than the manufacturer's provided `PCIe` network card. -> -> The device does not come with a `WiFi` network card and does not support WiFi or Bluetooth connectivity. Please prepare the corresponding equipment if needed. - -## Types of Images - -### ISO - -> `ISO` images support booting via `UEFI`, corresponding to the **UEFI version** firmware described below. - -Download the `ISO` file (e.g., `openEuler-24.03-LTS-riscv64-dvd.iso`) from the official download page and burn it to a **USB flash drive**. - -- It is recommended to use the `Balena Etcher` software for graphical burning [download from `https://github.com/balena-io/etcher/releases/latest`]. The burning process is not detailed here. -- In a command-line environment, you can also use the `dd` method to burn the image. Refer to the following command: - -```txt -~$ sudo dd if=openEuler-24.03-LTS-riscv64-dvd.iso of=/dev/sda bs=512K iflag=fullblock oflag=direct conv=fsync status=progress -``` - -### Image - -> `Image` images support booting via `Legacy`, corresponding to the **non-UEFI version** firmware described below. - -Download the `Zip` archive containing the image (e.g., `openEuler-24.03-LTS-riscv64-sg2042.img.zip`) from the official download page and burn it directly to an **SDCARD** or **solid-state drive**. - -## Device Firmware - -> Since the device's factory firmware currently does not support `UEFI`, users of the `ISO` version need to manually replace the firmware with the `UEFI` version based on `EDK2`. - -Download the device firmware archive `sg2042_firmware_uefi.zip` from the official download page under the **Embedded category**, extract it, and burn the `img` file to an **SDCARD**. - -```txt -~$ sudo dd if=firmware_single_sg2042-master.img of=/dev/sda bs=512K iflag=fullblock oflag=direct conv=fsync status=progress -261619712 bytes (262 MB, 250 MiB) copied, 20 s, 13.1 MB/s -268435456 bytes (268 MB, 256 MiB) copied, 20.561 s, 13.1 MB/s - -512+0 records in -512+0 records out -268435456 bytes (268 MB, 256 MiB) copied, 20.5611 s, 13.1 MB/s -``` - -> Due to the older firmware version shipped with the device, Image users who want to use a newer version of the firmware can update to the **non-UEFI version** of the firmware. - -Download the device firmware package `sg2042_firmware_uboot.zip` from the embedded category on the download page, and follow the same procedure as the UEFI firmware to extract and flash the img file to the **SDCARD**. - -After burning, insert the **SDCARD** into the device's card slot. - -## Pre-Startup Checks - -For `ISO` version users: - -- Ensure the `microSD card` with the `UEFI` firmware is inserted into the device's card slot. - - > The current `UEFI` firmware cannot manually adjust or specify the boot order, please understand. - -- If using the factory-provided solid-state drive, or if another bootable `RISC-V` operating system exists on the drive, remove the solid-state drive for formatting or replace it with another empty solid-state drive to avoid interference with the boot order. - -For `Image` version users: - -- If using the factory-provided solid-state drive, or if another bootable `RISC-V` operating system exists on the drive, remove the solid-state drive for formatting or replace it with another empty solid-state drive to avoid interference with the boot order. - -## Notes for Use - -For `ISO` version users: - -- Due to the limitations of the current version of the `UEFI` firmware, the `Grub2` boot menu may take a long time (~15s) to load and respond slowly if the graphics card is inserted into the `PCIe` slot during startup. - -For `Image` version users: - -- Due to the limitations of the current factory firmware, the `RISC-V` serial output is incomplete during device startup, and the serial output will be closed before the operating system is fully loaded. The graphics card needs to be inserted into the `PCIe` slot and connected to a monitor to observe the complete startup process. diff --git a/docs/en/docs/Installation/RISC-V-QEMU.md b/docs/en/docs/Installation/RISC-V-QEMU.md deleted file mode 100644 index b44198e923609dfeab014337d49719e5dcb1984e..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/RISC-V-QEMU.md +++ /dev/null @@ -1,70 +0,0 @@ -# Installing on QEMU - -## Firmware - -### Standard EFI Firmware - -Download the following binaries from the download page: - -``` text -RISCV_VIRT_CODE.fd -RISCV_VIRT_VARS.fd -``` - -Alternatively, compile the latest EDK2 OVMF firmware locally according to the [official documentation](https://github.com/tianocore/edk2/tree/master/OvmfPkg/RiscVVirt) - -### EFI Firmware with Penglai TEE Support - -Download the following binary from the download page: - -``` text -fw_dynamic_oe_2403_penglai.bin -``` - -## QEMU Version - ->To support UEFI, QEMU version 8.1 or above is required. -> ->During compilation, the libslirp dependency needs to be installed (package name varies by distribution, for openEuler it's libslirp-devel) and the --enable-slirp parameter should be added. - -``` bash -~$ qemu-system-riscv64 --version -QEMU emulator version 8.2.2 -Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers -``` - -## qcow2 Image - -Obtain the ISO file (e.g., `openEuler-24.03-LTS-riscv64.qcow2.xz`) - -``` bash -~$ ls *.qcow2.xz -openEuler-24.03-LTS-riscv64.qcow2.xz -``` - -## Getting the Startup Script - -Obtain the startup script from the download page: - -- `start_vm.sh`: Default script, requires manual installation of the desktop environment. -- `start_vm_penglai.sh`: Script supporting Penglai TEE functionality. - -Script Parameters: - -- `ssh_port`: Local SSH forwarding port, default is 12055. -- `vcpu`: Number of threads for QEMU execution, default is 8 cores, adjustable as needed. -- `memory`: Amount of memory allocated for QEMU execution, default is 8GiB, adjustable as needed. -- `fw`: Firmware payload for booting. -- `drive`: Path to the virtual disk, adjustable as needed. -- `bios` (optional): Boot firmware, can be used to load firmware enabled with Penglai TEE. - -## Creating a Virtual Hard Disk File - -Create a new virtual hard disk file, in the example below, the size of the virtual hard disk is 40GiB. -> Do not use qcow2 virtual hard disk files that already contain data to avoid unexpected situations during the boot process. -> -> Ensure that there is only one qcow2 virtual hard disk file in the current directory to avoid errors in script recognition. - -``` bash -~$ qemu-img create -f qcow2 qemu.qcow2 40G -``` diff --git a/docs/en/docs/Installation/faqs.md b/docs/en/docs/Installation/faqs.md deleted file mode 100644 index 636edc6921458bdcf581580ba2037049da8539fe..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/faqs.md +++ /dev/null @@ -1,346 +0,0 @@ -# FAQs - -## openEuler Fails to Start After It Is Installed to the Second Drive - -### Symptom - -The OS is installed on the second drive **sdb** during the installation, causing startup failure. - -### Possible Causes - -When openEuler is installed to the second drive, MBR and GRUB are installed to the second drive **sdb** by default. The following two situations may occur: - -1. openEuler installed on the first drive is loaded and started if it is complete. -2. openEuler installed on the first drive fails to be started from hard drives if it is incomplete. - -The preceding two situations occur because the first drive **sda** is booted by default to start openEuler in the BIOS window. If openEuler is not installed on the **sda** drive, system restart fails. - -### Solutions - -This problem can be solved using either of the following two methods: - -- During the openEuler installation, select the first drive or both drives, and install the boot loader on the first drive **sda**. -- After installing openEuler, restart it by modifying the boot option in the BIOS window. - -## openEuler Enters Emergency Mode After It Is Started - -### Symptom - -openEuler enters emergency mode after it is powered on. - -![fig](./figures/en-us_image_0229291264.jpg) - -### Possible Causes - -Damaged OS files result in drive mounting failure, or overpressured I/O results in drive mounting timeout \(threshold: 90s\). - -An unexpected system power-off and low I/O performance of drives may also cause the problem. - -### Solutions - -1. Log in to openEuler as the **root** user. -2. Check and restore files by using the file system check \(fsck\) tool, and restart openEuler. - - >![fig](./public_sys-resources/icon-note.gif) **NOTE:** - >The fsck tool checks and maintains inconsistent file systems. If the system is powered off or a drive is faulty, run the **fsck** command to check file systems. Run the **fsck.ext3 -h** and **fsck.ext4 -h** commands to view the usage method of the fsck tool. - -If you want to disable the timeout mechanism of drive mounting, add **x-systemd.device-timeout=0** to the **etc/fstab** file. For example: - -```sh -# -# /etc/fstab -# Created by anaconda on Mon Sep 14 17:25:48 2015 -# -# Accessible filesystems, by reference, are maintained under '/dev/drive' -# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info -# -/dev/mapper/openEuler-root / ext4 defaults,x-systemd.device-timeout=0 0 0 -UUID=afcc811f-4b20-42fc-9d31-7307a8cfe0df /boot ext4 defaults,x-systemd.device-timeout=0 0 0 -/dev/mapper/openEuler-home /home ext4 defaults 0 0 -/dev/mapper/openEuler-swap swap swap defaults 0 0 -``` - -## openEuler Fails to Be Reinstalled When an Unactivated Logical Volume Group Exists - -### Symptom - -After a drive fails, openEuler fails to be reinstalled because a logical volume group that cannot be activated exists in openEuler. - -### Possible Causes - -During the installation of openEuler, a logical volume group cannot be activated. - -### Solutions - -Before reinstalling openEuler, restore the abnormal logical volume group to the normal status or clear it. For example: - -- Restore the logical volume group. - 1. Run the following command to clear the active status of the abnormal logical volume group to ensure that the error message "Can't open /dev/sdc exclusively mounted filesystem" is not displayed: - - ```sh - vgchange -a n testvg32947 - ``` - - 2. Run the following command to recreate a physical volume based on the backup file: - - ```sh - pvcreate --uuid JT7zlL-K5G4-izjB-3i5L-e94f-7yuX-rhkLjL --restorefile /etc/lvm/backup/testvg32947 /dev/sdc - ``` - - 3. Run the following command to restore the logical volume group information: - - ```sh - vgcfgrestore testvg32947 - ``` - - 4. Run the following command to reactivate the logical volume group: - - ```sh - vgchange -ay testvg32947 - ``` - -- Run the following commands to clear the logical volume group: - - ```sh - vgchange -a n testvg32947 - vgremove -y testvg32947 - ``` - -## An Exception Occurs During the Selection of the Installation Source - -### Symptom - -After the installation source is selected, the message "Error checking software selection" is displayed. - -### Possible Causes - -This is because the software package dependency in the installation source is abnormal. - -### Solutions - -Check whether the installation source is abnormal. Use the new installation source. - -## Kdump Service Fails to Be Enabled - -### Symptom - -Run the **systemctl status kdump** command. The following information is displayed, indicating that no memory is reserved. - -![fig](./figures/en-us_image_0229291280.png) - -### Possible Causes - -The kdump service requires the system to reserve memory for running the kdump kernel. However, the system does not reserve memory for the kdump service. As a result, the kdump service cannot be started. - -### Solutions - -For the scenario where the OS has been installed - -1. Add **crashkernel=1024M,high** to **/boot/efi/EFI/openEuler/grub.cfg**. -2. Restart the system for configuration to take effect. -3. Run the following command to check the kdump status: - - ```sh - systemctl status kdump - ``` - - If the following information is displayed, the kdump status is **active**, indicating that the kdump service is enabled. No further action is required. - - ![fig](./figures/en-us_image_0229291272.png) - -### Parameter Description - -The following table describes the parameters of the memory reserved for the kdump kernel. - -**Table 1** crashkernel parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Kernel Boot Parameter

-

Description

-

Default Value

-

Remarks

-

crashkernel=X

-

Reserve X of the physical memory for kdump when the physical memory is less than 4 GB.

-

None. You can adjust the value as required.

-

This configuration method is used only when the memory is less than 4 GB. Ensure that the continuous available memory is sufficient.

-

crashkernel=X@Y

-

Reserve X of the memory at the start address Y for kdump.

-

None. You can adjust the value as required.

-

Ensure that the X of the memory at the start address Y is not reserved for other modules.

-

crashkernel=X,high

-

Reserve 256 MB of the physical memory for kdump when the physical memory is less than 4 GB, and X of the physical memory for kdump when the physical memory is greater than or equal to 4 GB.

-

None. You can adjust the value based as required. The recommended value is 1024M,high.

-

Ensure that 256 MB of the memory is reserved for continuous use when the physical memory is less than 4 GB and X of the memory is reserved when the physical memory is greater than or equal to 4 GB. The actual reserved memory size equals 256 MB plus X.

-

crashkernel=X,low

-

crashkernel=Y,high

-

Reserve X of the physical memory for kdump when the physical memory is less than 4 GB and Y of the physical memory for kdump when the physical memory is greater than or equal to 4 GB.

-

None. You can adjust the value as required.

-

Ensure that X of the memory is reserved for continuous use when the physical memory is less than 4 GB and Y of the memory is reserved when the physical memory is greater than or equal to 4 GB. The actual reserved memory size equals X plus Y.

-
- -## Fails to Select Only One Drive for Reinstallation When openEuler Is Installed on a Logical Volume Consisting of Multiple Drives - -### Symptom - -If openEuler is installed on a logical volume consisting of multiple drives, an error message will be displayed as shown in [Figure 1](#fig115949762617) when you attempt to select one of the drives for reinstallation. - -**Figure 1** Error message -![fig](./figures/error-message.png "error-message") - -### Possible Causes - -The previous logical volume contains multiple drives. If you select one of the drives for reinstallation, the logical volume will be damaged. - -### Solutions - -The logical volume formed by multiple drives is equivalent to a volume group. Therefore, you only need to delete the corresponding volume group. - -1. Press **Ctrl**+**Alt**+**F2** to switch to the CLI and run the following command to find the volume group: - - ```sh - vgs - ``` - - ![fig](./figures/en-us_image_0231657950.png) - -2. Run the following command to delete the volume group: - - ```sh - vgremove euleros - ``` - -3. Run the following command to restart the installation program for the modification to take effect: - - ```sh - systemctl restart anaconda - ``` - - >![fig](./public_sys-resources/icon-note.gif) **NOTE:** - >You can also press **Ctrl**+**Alt**+**F6** to return to the GUI and click **Refresh** in the lower right corner to refresh the storage configuration. - -## openEuler Fails to Be Installed on an x86 PM in UEFI Mode due to Secure Boot Option Setting - -### Symptom - -During the installation of openEuler on an x86 PM in UEFI mode, the system stays at the "No bootable device" page and the installation cannot continue because **secure boot** is set to **enabled** \(by default, it is set to **disabled**\), as shown in [Figure 2](#fig115949762618). - -**Figure 2** Dialog box showing "No bootable device" -![fig](./figures/dialog-box-showing-no-bootable-device.png "dialog-box-showing-no-bootable-device") - -### Possible Causes - -After **Secure Boot** is set to **Enabled**, the mainboard verifies the boot program and OS. If the boot program and OS are not signed using the corresponding private key, they cannot pass the authentication of the built-in public key on the mainboard. - -### Solutions - -Access the BIOS, set **Secure Boot** to **Disabled**, and reinstall the openEuler. - -1. During the system startup, press **F11** and enter the password **Admin@9000** to access the BIOS. - - ![fig](./figures/bios.png) - -2. Choose **Administer Secure Boot**. - - ![fig](./figures/security.png) - -3. Set **Enforce Secure Boot** to **Disabled**. - - ![fig](./figures/enforce-secure-boot.png) - - >![fig](./public_sys-resources/icon-note.gif) **NOTE:** - >After **Enforce Secure Boot** is set to **Disabled**, save the settings and exit. Then, reinstall the system. - -## pmie_check Is Reported in the messages Log During openEuler Installation - -### Symptom - -During the OS installation, if you click **Server > Performance tool**, PCP is installed. After the OS is installed and restarted, an error "pmie_check failed in /usr/share/pcp/lib/pmie" is displayed in the **/var/log/messages** log. - -### Possible Causes - -anaconda does not support the installation of SELinux policy module in the chroot environment. During the pcp-selinux installation, the postin script fails to execute the PCP-related SELinux policy module. As a result, an error is reported after the OS is restarted. - -### Solutions - -After the OS is installed and restarted, perform either of the following two operations: - -1. Install SElinux policy module pcpupstream. - - ```sh - /usr/libexec/pcp/bin/selinux-setup /var/lib/pcp/selinux install "pcpupstream" - - ```sh - -2. Reinstall pcp-selinux - - ```sh - sudo dnf reinstall pcp-selinux - - ```sh - -## Installation Fails when a User Selects Two Drives with OS Installed and Customizes Partitioning - -### Symptom - -During the OS installation, the OS has been installed on two drives. In this case, if you select one drive for custom partitioning, and click **Cancel** to perform custom partitioning on the other drive, the installation fails. - -![fig](./figures/cancle_disk.png) - -![fig](./figures/custom_paratition.png) - -### Possible Causes - -A user selects a drive for partitioning twice. After the user clicks **Cancel** and then selects the other drive, the drive information is incorrect. As a result, the installation fails. - -### Solutions - -Select the target drive for custom partitioning. Do not frequently cancel the operation. If you have to cancel and select another drive, you are advised to reinstall the OS. - -### Learn More About the Issue at - - - -## vmcore Fails to Be Generated by Kdump on the PM with LSI MegaRAID Card Installed - -### Symptom - -After the Kdump service is deployed, kernel breaks down due to the manual execution of the **echo c > /proc/sysrq-trigger** command or kernel fault. When Kdump enables second kernel, an error "BRCM Debug mfi stat 0x2d, data len requested/completed 0x200/0x0" is reported in the MegaRAID driver, as shown in the following figure. As a result, vmcore fails to be generated. - -![Error information](./figures/Megaraid_IO_Request_uncompleted.png) - -### Possible Causes - -The **reset_devices** parameter is configured by default and is enabled during second kernel startup, making MegaRAID driver or drive faulty. An error is reported when the vmcore file is dumped ana accesses the MegaRAID card. As a result, vmcore fails to be generated. - -### Solutions - -Delete the **reset_devices** parameter in the **etc/sysconfig/kdump** file on a PM, as shown in the following figure. Therefore, the I/O request will be responded when the MegaRAID driver resets the device during the second kernel startup, and vmcore will be successfully generated. - -![Deleting reset_devices](./figures/reset_devices.png) diff --git a/docs/en/docs/Installation/figures/Figure-18.png b/docs/en/docs/Installation/figures/Figure-18.png deleted file mode 100644 index b5463dddcc1a892edc09d9ff9ac12a7d1751e6ae..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/Figure-18.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/Installation_source.png b/docs/en/docs/Installation/figures/Installation_source.png deleted file mode 100644 index 558374e3260e5218b6528ddd8d021606bf790787..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/Installation_source.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/Target_installation_position - 01.png b/docs/en/docs/Installation/figures/Target_installation_position - 01.png deleted file mode 100644 index 339d3d96f469f54f5b9c0f3b40fb0cd78935180c..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/Target_installation_position - 01.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/cancle_disk.png b/docs/en/docs/Installation/figures/cancle_disk.png deleted file mode 100644 index f1db0f2c524695303f0d8791fcb3c256c75507ad..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/cancle_disk.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/confignetwork.png b/docs/en/docs/Installation/figures/confignetwork.png deleted file mode 100644 index 79903b72948a06d3fceff97c11f49d12f7571b94..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/confignetwork.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/custom_paratition.png b/docs/en/docs/Installation/figures/custom_paratition.png deleted file mode 100644 index d2e8c68e6af866e96bf5dd2a2f532de81c59a9d9..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/custom_paratition.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/disk-encryption-password.png b/docs/en/docs/Installation/figures/disk-encryption-password.png deleted file mode 100644 index ba84e060133644910ff199376e11d2929cfe8d47..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/disk-encryption-password.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/en-us_image_0213178479.png b/docs/en/docs/Installation/figures/en-us_image_0213178479.png deleted file mode 100644 index 62ef0decdf6f1e591059904001d712a54f727e68..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/en-us_image_0213178479.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/en-us_image_0229291229.png b/docs/en/docs/Installation/figures/en-us_image_0229291229.png deleted file mode 100644 index b315531ca7f99d2a045b7933351af96cadc1ad77..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/en-us_image_0229291229.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/en-us_image_0229291236.png b/docs/en/docs/Installation/figures/en-us_image_0229291236.png deleted file mode 100644 index bf466a3d751df4a4c6fd99aecf620ec9adf540a3..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/en-us_image_0229291236.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/error-message.png b/docs/en/docs/Installation/figures/error-message.png deleted file mode 100644 index c5802a2b7a750eed8429ec06c7e4919a3d161a9e..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/error-message.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-10.png b/docs/en/docs/Installation/figures/figure-10.png deleted file mode 100644 index 45a84ab55ce7c5e51e12bbb6888ccf671a5539ef..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-10.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-11.png b/docs/en/docs/Installation/figures/figure-11.png deleted file mode 100644 index d50310a29b8087365b93e41b3a5f4aa1880b1b85..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-11.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-12.png b/docs/en/docs/Installation/figures/figure-12.png deleted file mode 100644 index 339d3d96f469f54f5b9c0f3b40fb0cd78935180c..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-12.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-13.png b/docs/en/docs/Installation/figures/figure-13.png deleted file mode 100644 index 3288b925f49c12b0a6c3a96d5b1ca27fad23120a..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-13.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-14.png b/docs/en/docs/Installation/figures/figure-14.png deleted file mode 100644 index 2113c78fa2977772c55be8ca3b32ddb09ba65f1d..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-14.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-15.png b/docs/en/docs/Installation/figures/figure-15.png deleted file mode 100644 index 9fd26d5e6247d8e843b79cef0c05ef971c32a4ad..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-15.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-16.png b/docs/en/docs/Installation/figures/figure-16.png deleted file mode 100644 index 2c3231a61e2ea97c2d6b74868373a1176d3dfc05..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-16.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-17.png b/docs/en/docs/Installation/figures/figure-17.png deleted file mode 100644 index 8862e92a973e1f52bc7db887f223da880a33d4a3..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-17.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-19.png b/docs/en/docs/Installation/figures/figure-19.png deleted file mode 100644 index ed519042a32bc742d5210f55163de526b9cf17f0..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-19.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-4.png b/docs/en/docs/Installation/figures/figure-4.png deleted file mode 100644 index 329cc06b70cc26cde85106d40db2c8a17947f937..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-4.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-5.png b/docs/en/docs/Installation/figures/figure-5.png deleted file mode 100644 index aac62bc3aa30d89fa3a133a8262ef926bb9319b7..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-5.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-6.png b/docs/en/docs/Installation/figures/figure-6.png deleted file mode 100644 index 440167f076a2b9e6c4882a808461698d12088e65..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-6.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-7.png b/docs/en/docs/Installation/figures/figure-7.png deleted file mode 100644 index a6a6b00852e044d4e624bd1ad3494807bffa1854..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-7.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-8.png b/docs/en/docs/Installation/figures/figure-8.png deleted file mode 100644 index eaf7defbe291deae7d5a67993db33b2213a8ea75..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-8.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/figure-9.png b/docs/en/docs/Installation/figures/figure-9.png deleted file mode 100644 index fd3e344ab1a8e53ebbf6c8dbb11e5bbf67977a3b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/figure-9.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/ftp-mode.png b/docs/en/docs/Installation/figures/ftp-mode.png deleted file mode 100644 index 14b45c69f0b553d55ebcb01064b8f9e2ee591ed8..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/ftp-mode.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/host_env8.png b/docs/en/docs/Installation/figures/host_env8.png deleted file mode 100644 index d08dcc89f40e1671a55a42fbcb02f26e987a461e..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/host_env8.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/http-mode.png b/docs/en/docs/Installation/figures/http-mode.png deleted file mode 100644 index 1414062bbd161ac75998d1c098933807a3268412..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/http-mode.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/installsourceen.png b/docs/en/docs/Installation/figures/installsourceen.png deleted file mode 100644 index 43e59b694ec1afcf8591e8272390da927da9a3fe..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/installsourceen.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/manual-partitioning-page.png b/docs/en/docs/Installation/figures/manual-partitioning-page.png deleted file mode 100644 index 7f3debff53c167acc15dd95c5face0c30e9e8ec3..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/manual-partitioning-page.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/nfs-mode.png b/docs/en/docs/Installation/figures/nfs-mode.png deleted file mode 100644 index da913146067340b0db9a94baa4691c5b8497ebae..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/nfs-mode.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/setting-a-system-language.png b/docs/en/docs/Installation/figures/setting-a-system-language.png deleted file mode 100644 index e8e6faa69580e707657cba3f2f589918321a4b4d..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/setting-a-system-language.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/setting-date-and-time.png b/docs/en/docs/Installation/figures/setting-date-and-time.png deleted file mode 100644 index 6e366072db2ca698ae2bc317a361e9d38877a2d0..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/setting-date-and-time.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/setting-the-keyboard-layout.png b/docs/en/docs/Installation/figures/setting-the-keyboard-layout.png deleted file mode 100644 index 62b0074220b8e2c8ebca37dceecc92e0c2fcdffc..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/setting-the-keyboard-layout.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/setting-the-network-and-host-name.png b/docs/en/docs/Installation/figures/setting-the-network-and-host-name.png deleted file mode 100644 index b17ebdaafeaa2228ddbe0d8135fee3eabdc1cb76..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/setting-the-network-and-host-name.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/sourceftp.png b/docs/en/docs/Installation/figures/sourceftp.png deleted file mode 100644 index 2e18d3f5c6d999c8a637ebed36ccb740a96d8449..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/sourceftp.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/sourcenfs.png b/docs/en/docs/Installation/figures/sourcenfs.png deleted file mode 100644 index 3a4564871319deb546776b2542575ed43f2f2a35..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/sourcenfs.png and /dev/null differ diff --git a/docs/en/docs/Installation/figures/startparam.png b/docs/en/docs/Installation/figures/startparam.png deleted file mode 100644 index b197f4d492213513edf84a99cdb14f186630a828..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Installation/figures/startparam.png and /dev/null differ diff --git a/docs/en/docs/Installation/installation-guideline.md b/docs/en/docs/Installation/installation-guideline.md deleted file mode 100644 index 8daaf4b7d4efa72da7f340df04af0eeba61adf90..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/installation-guideline.md +++ /dev/null @@ -1,329 +0,0 @@ -# Installation Guideline - -This section describes how to install openEuler using a CD/DVD-ROM. The installation process is the same for other installation modes except the boot option. - -## Starting the Installation - -### Booting from the CD/DVD-ROM Drive - -Load the ISO image of openEuler from the CD/DVD-ROM drive of the server and restart the server. The procedure is as follows: - ->![](./public_sys-resources/icon-note.gif) **Note** ->Before the installation, ensure that the server boots from the CD/DVD-ROM drive preferentially. The following steps describe how to install openEuler using a virtual CD/DVD-ROM drive on the baseboard management controller (BMC). The procedure for installing openEuler from a physical drive is the same as that of a virtual drive. - -1. On the toolbar, click the icon shown in the following figure. - - **Figure 1** Drive icon - ![](./figures/drive-icon.png) - - An image dialog box is displayed, as shown in the following figure. - - **Figure 2** Image dialog box - ![](./figures/image-dialog-box.png) - -2. Select **Image File** and then click **...**. The **Open** dialog box is displayed. -3. Select the image file and click **Open**. In the image dialog box, click **Connect**. If **Connect** changes to **Disconnect**, the virtual CD/DVD-ROM drive is connected to the server. -4. On the toolbar, click the restart icon shown in the following figure to restart the device. - - **Figure 3** Restart icon - ![](./figures/restart-icon.png) - -### Installation Wizard - -A boot menu is displayed after the system is booted using the boot medium. In addition to options for starting the installation program, some other options are available on the boot menu. During system installation, the **Test this media & install openEuler 21.09** mode is used by default. Press the arrow keys on the keyboard to change the selection, and press **Enter** when the desired option is highlighted. - ->![](./public_sys-resources/icon-note.gif) **Note** -> ->- If you do not perform any operations within 1 minute, the system automatically selects the default option **Test this media & install openEuler 21.09** and enters the installation page. ->- During physical machine installation, if you cannot use the arrow keys to select boot options and the system does not respond after you press **Enter**, click ![](./figures/en-us_image_0229420473.png) on the BMC page and configure **Key & Mouse Reset**. - -**Figure 4** Installation Wizard -![](./figures/figure-4.png) - -Installation wizard options are described as follows: - -- **Install openEuler 21.09**: Install openEuler on your server in GUI mode. - -- **Test this media & install openEuler 21.09**: Default option. Install openEuler on your server in GUI mode. The integrity of the installation medium is checked before the installation program is started. - -- **Troubleshooting**: Troubleshooting mode, which is used when the system cannot be installed properly. In troubleshooting mode, the following options are available: - - **Install openEuler 21.09 in basic graphics mode**: Basic graphics installation mode. In this mode, the video driver is not started before the system starts and runs. - - **Rescue the openEuler system**: Rescue mode, which is used to restore the system. In rescue mode, the installation process is printed to the Virtual Network Computing (VNC) or BMC interface, and the serial port is unavailable. -On the installation wizard screen, press **e** to go to the parameter editing screen of the selected option, and press **c** to go to the command line interface (CLI). - -### Installation in GUI Mode - -On the installation wizard page, select **Test this media & install openEuler 21.09** to enter the GUI installation mode. - -Perform graphical installation operations using a keyboard. - -- Press **Tab** or **Shift+Tab** to move between GUI controls (such as buttons, area boxes, and check boxes). -- Press the up or down arrow key to move a target in the list. -- Press the left or right arrow key to move between the horizontal toolbar and watch bar. -- Press the spacebar or **Enter** to select or delete highlighted options, expand or collapse a drop-down list. -- Press **Alt+a shortcut key** (the shortcut key varies for different pages) to select the control where the shortcut key is located. The shortcut key can be highlighted (underlined) by holding down **Alt**. - -## Configuring an Installation Program Language - -After the installation starts, the system will prompt the language that is used during the installation process. English is configured by default, as shown in the following figure. Configure another language as required. - -**Figure 5** Selecting a language -![](./figures/figure-5.png) - -After the language is set, click **Continue**. The installation page is displayed. - -If you want to exit the installation, click **Exit**. The message **Are you sure you want to exit the installation program?** is displayed. Click **Yes** in the dialog box to go back to the installation wizard page. - -## Entering the Installation Page - -After the installation program starts, the installation page is displayed, as shown in the following figure. On the page, you can configure the time, language, installation source, network, and storage device. - -Some configuration items are matched with safety symbols. A safety symbol will disappear after the item is configured. Start the installation only when all the safety symbols disappear from the page. - -If you want to exit the installation, click **Exit**. The message **Are you sure you want to exit the installation program?** is displayed. Click **Yes** in the dialog box to go back to the installation wizard page. - -**Figure 6** Installation summary -![](./figures/figure-6.png) - -## Setting the Keyboard Layout - -On the **INSTALLATION SUMMARY** page, click **KEYBOARD**. You can add or delete multiple keyboard layouts in the system. - -- To view the keyboard layout: Select a keyboard layout in the left box and click **keyboard** under the box. -- To test the keyboard layout: Select the keyboard layout in the left box and click keyboard icon in the upper right corner to switch to the desired layout, and then type in the right text box to ensure that the keyboard layout can work properly. - -**Figure 7** Setting the keyboard layout -![](./figures/figure-7.png) - -After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Setting a System Language - -On the **INSTALLATION SUMMARY** page, click **LANGUAGE SUPPORT** to set the system language, as shown in the following figure. Set another language as required, such as Chinese. - ->![](./public_sys-resources/icon-note.gif) **Note** ->If you select **Chinese**, the system does not support the display of Chinese characters when you log in to the system using VNC, but supports the display of Chinese characters when you log in to the system using a serial port. When you log in to the system using SSH, whether the system supports the display of Chinese characters depends on the SSH client. If you select **English**, the display is not affected. - -**Figure 8** Setting a system language - -![](./figures/figure-8.png) - -After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Setting Date and Time - -On the **INSTALLATION SUMMARY** page, click **TIME & DATE**. On the **TIME & DATE** page, set the system time zone, date, and time. - -When setting the time zone, select a region from the drop-down list of **Region** and a city from the drop-down list of **City** at the top of the page, as shown in the following figure. - -If your city is not displayed in the drop-down list, select the nearest city in the same time zone. - ->![](./public_sys-resources/icon-note.gif) **Note** -> ->- Before manually setting the time zone, disable the network time synchronization function in the upper right corner. ->- If you want to use the network time, ensure that the network can connect to the remote NTP server. For details about how to set the network, see [Setting the Network and Host Name](#setting-the-network-and-host-name). - -**Figure 9** Setting date and time - -![](./figures/figure-9.png) - -After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Setting the Installation Source - -On the **INSTALLATION SUMMARY** page, click **INSTALLATION SOURCE** to locate the installation source. - -- When you use a complete CD/DVD-ROM for installation, the installation program automatically detects and displays the installation source information. You can use the default settings, as shown in the following figure. - - **Figure 10** Installation source - ![](./figures/figure-10.png) - -- When the network source is used for installation, you need to set the URL of the network source. - - - HTTP or HTTPS mode - - The following figure shows the installation source in HTTP or HTTPS mode: - - ![](./figures/http-mode.png) - - If the HTTPS server uses a private certificate, press **e** on the installation wizard page to go to the parameter editing page of the selected option, and add the **inst.noverifyssl** parameter. - - Enter the actual installation source address, for example, ****, where **openEuler-21.09** indicates the version number, and **x86-64** indicates the CPU architecture. Use the actual version number and CPU architecture. - - - FTP mode - - The following figure shows the installation source in FTP mode. Enter the FTP address in the text box. - - ![](./figures/ftp-mode.png) - - You need to set up an FTP server, mount the **openEuler-21.09-x86_64-dvd.iso** image, and copy the mounted files to the shared directory on the FTP server. **x86_64** indicates the CPU architecture. Use the actual image. - - - NFS mode - - The following figure shows the installation source in NFS mode. Enter the NFS address in the text box. - - ![](./figures/nfs-mode.png) - - You need to set up an NFS server, mount the **openEuler-21.09-x86_64-dvd.iso** image, and copy the mounted file to the shared directory on the NFS server. **x86_64** indicates the CPU architecture. Use the actual image. - -During the installation, if you have any questions about configuring the installation source, see [An Exception Occurs During the Selection of the Installation Source](https://gitee.com/openeuler/docs/blob/5232a58d1e76f59c50d68183bdfd3f6dc1603390/docs/en/docs/Installation/faqs.md#an-exception-occurs-during-the-selection-of-the-installation-source). - -After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Selecting Installation Software - -On the **INSTALLATION SUMMARY** page, click **SOFTWARE SELECTION** to specify the software package to be installed. - -Based on the actual requirements, select **Minimal Install** in the left box and select an add-on in the **Additional software for Selected Environment** area in the right box, as shown in the following figure. - -**Figure 11** Selecting installation software -![](./figures/figure-11.png) - ->![](./public_sys-resources/icon-note.gif) **Note** -> ->- In **Minimal Install** mode, not all packages in the installation source will be installed. If the required package is not installed, you can mount the installation source to the local host as a repo source, and use DNF to install the package. ->- If you select **Virtualization Host**, the virtualization components QEMU, libvirt, and edk2 are installed by default. You can select whether to install the OVS component in the add-on area. - -After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Setting the Installation Destination - -On the **INSTALLATION SUMMARY** page, click **INSTALLATION DESTINATION** to select the OS installation disk and partition. - -You can view available local storage devices in the following figure. - -**Figure 12** Setting the installation destination -![](./figures/figure-12.png) - -### Storage Configuration - -On the **INSTALLATION DESTINATION** page, configure storage for system partition. You can either manually configure partitions or select **Automatic** for automatic partitioning. - ->![](./public_sys-resources/icon-note.gif) **Note** -> ->- During partitioning, to ensure system security and performance, you are advised to divide the device into the following partitions: **/boot**, **/var**, **/var/log**, **/var/log/audit**, **/home**, and **/tmp**. See [Table 1](#table1). ->- If the system is configured with the **swap** partition, the **swap** partition is used when the physical memory of the system is insufficient. Although the **swap** partition can be used to expand the physical memory, if it is used due to insufficient memory, the system response slows and the system performance deteriorates. Therefore, you are not advised to configure it in the system with sufficient physical memory or in the performance sensitive system. ->- If you need to split a logical volume group, select **Custom** to manually partition the logical volume group. On the **MANUAL PARTITIONING** page, click **Modify** in the **Volume Group** area to reconfigure the logical volume group. - -**Table 1** Suggested disk partitions - -| Partition Type | Partition Type | Partition Size | Description | -| --- | --- | --- | --- | -| Primary partition | / | 20 GB | Root directory used to install the OS. | -| Primary partition | /boot | 1 GB | Boot partition. | -| Primary partition | /swap | 16 GB | Swap partition. | -| Logical partition | /home | 1 GB | Stores local user data. | -| Logical partition | /tmp | 10 GB | Stores temporary files. | -| Logical partition | /var | 5 GB | Stores the dynamic data of the daemon process and other system service processes. | -| Logical partition | /var/log | 10 GB | Stores system log data. | -| Logical partition | /var/log/audit | 2 GB | Stores system audit log data. | -| Logical partition | /usr| 5 GB | Stores shared and read-only applications. | -| Logical partition | /var/tmp | 5 GB | Stores temporary files that can be retained during system reboot. | -| Logical partition | /opt | Size of the remaining disk space. | Used to install application software. | - -**Automatic** - -Select **Automatic** if the software is installed in a new storage device or the data in the storage device is not required. After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -**Custom** - -If you need to manually partition the disk, click **Custom** and click **Done** in the upper left corner. The following page is displayed. - -On the **MANUAL PARTITIONING** page, you can partition the disk in either of the following ways. After the partitioning is completed, the window shown in the following figure is displayed. - -- Automatic creation: Click **Click here to create them automatically**. The system automatically assigns four mount points according to the available storage space: **/boot**, **/**, **/home**, **/boot/efi**, and **swap**. - -- Manual creation: Click ![](./figures/en-us_image_0229291243.png) to add a mount point. It is recommended that the expected capacity of each mount point not exceed the available space. - - >![](./public_sys-resources/icon-note.gif) **Note** - >If the expected capacity of the mount point exceeds the available space, the system allocates the remaining available space to the mount point. - -**Figure 13** MANUAL PARTITIONING page -![](./figures/figure-13.png) - ->![](./public_sys-resources/icon-note.gif) **Note** ->If non-UEFI mode is selected, the **/boot/efi** partition is not required. Otherwise, it is required. - -After the setting is complete, click **Done** in the upper left corner to go back to the **SUMMARY OF CHANGES** page. - -Click **Accept Changes** to go back to the **INSTALLATION SUMMARY** page. - -## Setting the Network and Host Name - -On the **INSTALLATION SUMMARY** page, select **NETWORK & HOST NAME** to configure the system network functions. - -The installation program automatically detects a local access interface. The detected interface is listed in the left box, and the interface details are displayed in the right area, as shown in [Figure 14](#zh-cn_topic_0186390264_zh-cn_topic_0122145831_fig123700157297). You can enable or disable a network interface by clicking the switch in the upper right corner of the page. The switch is turned off by default. If the installation source is set to network, turn on the switch. You can also click **Configure** to configure the selected interface. Select **Connect automatically with priority** to enable the NIC automatic startup upon system startup, as shown in [Figure 15](#zh-cn_topic_0186390264_zh-cn_topic_0122145831_fig6). - -In the lower left box, enter the host name. The host name can be the fully quantified domain name (FQDN) in the format of *hostname.domain_name* or the brief host name in the format of *hostname*. - -**Figure 14** Setting the network and host name -![](./figures/figure-14.png) - -**Figure 15** Configuring the network -![](./figures/figure-15.png) - -After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Setting the Root Password - -Select Root Password on the **INSTALLATION SUMMARY** page. The **Root Password** page is displayed, as shown in the following figure. Enter a password based on [Password Complexity](#password-complexity) requirements and confirm the password. - ->![](./public_sys-resources/icon-note.gif) **Note** -> ->- The **root** account is used to perform key system management tasks. You are not advised to use the **root** account for daily work or system access. ->- If you select **Lock root account** on the **Root Password** page, the **root** account will be disabled. - -**Figure 16** root password -![](./figures/figure-16.png) - -### Password Complexity - -The password of the **root** user or the password of the new user must meet the password complexity requirements. Otherwise, the password configuration or user creation will fail. The password complexity requirements are as follows: - -1. A password must contain at least eight characters. - -2. A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters. - -3. A password must be different from the account name. - -4. A password cannot contain words in the dictionary. - - >![](./public_sys-resources/icon-note.gif) **Note** - >In the installed openEuler environment, you can run the`cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt` command to export the dictionary library file **dictionary.txt**, and then check whether the password is in the dictionary. - -After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - -## Creating a User - -Click **User Creation**. [Figure 17](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319) shows the page for creating a user. Enter a username and set a password. By clicking **Advanced**, you can also configure a home directory and a user group, as shown in [Figure 18](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig128716531312). - -**Figure 17** Creating a user -![](./figures/figure-17.png) - -**Figure 18** Advanced user configuration -![](./figures/figure-18.png) - -After configuration, click **Done** in the left-upper corner to switch back to the **INSTALLATION SUMMARY** page. - -## Starting Installation - -On the installation page, after all the mandatory items are configured, the alarm symbols will disappear. Then, you can click **Begin Installation** to install the system. - -## Installation Procedure - -After the installation starts, the overall installation progress and the progress of writing the software package to the system are displayed. See the following figure. - ->![](./figures/en-us_image_0213178479.png) ->If you click **Exit** or reset or power off the server during the installation, the installation is interrupted and the system is unavailable. In this case, you need to reinstall the system. - -**Figure 19** Installation process -![](./figures/figure-19.png) - -## Completing the Installation - -After openEuler is installed, Click **Reboot** to restart the system. - ->![](./public_sys-resources/icon-note.gif) **Note** -> ->- If a physical CD/DVD-ROM is used for installation and it is not automatically ejected during the restart, manually remove it. Then, the openEuler CLI login page is displayed. ->- If a virtual CD/DVD-ROM is used for installation, change the server boot option to **Hard Disk** and restart the server. Then, the openEuler CLI login page is displayed. diff --git a/docs/en/docs/Installation/installation-modes.md b/docs/en/docs/Installation/installation-modes.md deleted file mode 100644 index ca5179265e12d8ce7aecddbc67f6d2e74cea010c..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/installation-modes.md +++ /dev/null @@ -1,199 +0,0 @@ -# Installation Modes - ->![](./public_sys-resources/icon-notice.gif) **NOTICE** -> ->- Only TaiShan 200 servers and FusionServer Pro rack server are supported. For details about the supported server models, see [Hardware Compatibility](./installation-preparations.md#hardware-compatibility). Only a virtualization platform created by the virtualization components \(openEuler as the host OS and QEMU and KVM provided in the release package\) of openEuler and the x86 virtualization platform of Huawei public cloud are supported. ->- Currently, only installation modes such as DVD-ROM, USB flash drive, network, QCOW2 image, and private image are supported. In addition, only the x86 virtualization platform of Huawei public cloud supports the private image installation mode. - - - -- [Installation Modes](#installation-modes) - - [Installation Through a DVD-ROM](#installation-through-a-dvd-rom) - - [Preparing the Installation Source](#preparing-the-installation-source) - - [Starting the Installation](#starting-the-installation) - - [Installation Through a USB Flash Drive](#installation-through-a-usb-flash-drive) - - [Preparing the Installation Source](#preparing-the-installation-source-1) - - [Starting the Installation](#starting-the-installation-1) - - [Installation Through the Network Using PXE](#installation-through-the-network-using-pxe) - - [Installation Through a QCOW2 Image](#installation-through-a-qcow2-image) - - [Creating a QCOW2 Image](#creating-a-qcow2-image) - - [Starting the Installation](#starting-the-installation-2) - - [Installation Through a Private Image](#installation-through-a-private-image) - - [Creating a Private Image](#creating-a-private-image) - - [Starting the Installation](#starting-the-installation-3) - - - -## Installation Through a DVD-ROM - -This section describes how to create or use a DVD-ROM to install the openEuler. - -### Preparing the Installation Source - -If you have obtained a DVD-ROM, directly install the OS using the DVD-ROM. If you have obtained an ISO file, record the ISO file to a DVD and install the OS using the obtained DVD. - -### Starting the Installation - -Perform the following operations to start the installation: - ->![](./public_sys-resources/icon-note.gif) **NOTE** ->Set the system to preferentially boot from the DVD-ROM drive. Take BIOS as an example. You need to move the **CD/DVD-ROM Drive** option under **Boot Type Order** to the top. - -1. Disconnect all drives that are not required, such as USB drives. -2. Start your computer system. -3. Insert the installation DVD-ROM into the computer. -4. Restart the computer system. - -After a short delay, a graphical wizard page is displayed, which contains different boot options. If you do not perform any operation within one minute, the installation starts automatically with the default option. - -## Installation Through a USB Flash Drive - -This section describes how to create or use a USB flash drive to install the openEuler. - -### Preparing the Installation Source - -Pay attention to the capacity of the USB flash drive. The USB flash drive must have sufficient space to store the entire image. It is recommended that the USB flash drive have more than 16 GB space. - -1. Connect the USB flash drive to the system and run the **dmesg** command to view related log. At the end of the log, you can view the information generated by the USB flash drive that is just connected. The information is similar to the following: - - ```text - [ 170.171135] sd 5:0:0:0: [sdb] Attached SCSI removable disk - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE** - >Take the **sdb** USB flash drive as an example. - -2. Switch to user **root**. When running the **su** command, you need to enter the password. - - ```bash - su - root - ``` - -3. Ensure that the USB flash drive is not mounted. - - ```bash - findmnt /dev/sdb - ``` - - - If no command output is displayed, the file system is not mounted. Go to the next step. - - - If the following information is displayed, the USB flash drive is automatically mounted. - - ```bash - $ findmnt /dev/sdb - TARGET SOURCE FSTYPE OPTIONS - /mnt/iso /dev/sdb iso9660 ro,relatime - ``` - - In this case, you need to run the **umount** command to uninstall the device. - - ```bash - umount /mnt/iso - ``` - -4. Run the **dd** command to write the ISO image to the USB flash drive. - - ```bash - dd if=/path/to/image.iso of=/dev/device bs=blocksize - ``` - - Replace **/path/to/image.iso** with the complete path of the downloaded ISO image file, replace **device** with the device name provided by the **dmesg** command, and set a proper block size \(for example, 512 KB\) to replace **blocksize** to accelerate the write progress. - - For example, if the ISO image file name is **/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso** and the detected device name is **sdb**, run the following command: - - ```bash - dd if=/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso of=/dev/sdb bs=512k - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE** - >As described in ISOLINUX, the ISO 9660 file system created by the **mkisofs** command will boot through BIOS firmware, but only from the CD-ROM, DVD-ROM, or BD. In this case, you need to run the **isohybrid -u your.iso** command to process the ISO file and then run the **dd** command to write the ISO file to the USB flash drive. (This problem affects only the x86 architecture.) - -5. After the image is written, remove the USB flash drive. - - No progress is displayed during the image write process. When the number sign (#) appears again, run the following command to write the data to the drive. Then exit the **root** account and remove the USB flash drive. In this case, you can use the USB drive as the installation source of the system. - - ```bash - sync - ``` - -### Starting the Installation - -Perform the following operations to start the installation: - ->![](./public_sys-resources/icon-note.gif) **NOTE** ->Set the system to preferentially boot from the USB flash drive. Take the BIOS as an example. You need to move the **USB** option under **Boot Type Order** to the top. - -1. Disconnect all drives that are not required. -2. Open your computer system. -3. Insert the USB flash drive into the computer. -4. Restart the computer system. - -After a short delay, a graphical wizard page is displayed, which contains different boot options. If you do not perform any operation within one minute, the installation program automatically starts the installation. - -## Installation Through the Network Using PXE - -To boot with PXE, you need to properly configure the server and your computer's network interface shall support PXE. - -If the target hardware is installed with a PXE-enabled NIC, configure it to boot the computer from network system files rather than local media \(such as DVD-ROMs\) and execute the Anaconda installation program. - -For installation through the network using PXE, the client uses a PXE-enabled NIC to send a broadcast request for DHCP information and IP address to the network. The DHCP server provides the client with an IP address and other network information, such as the IP address or host name of the DNS and FTP server \(which provides the files required for starting the installation program\), and the location of the files on the server. - ->![](./public_sys-resources/icon-note.gif) **NOTE** ->The TFTP, DHCP, and HTTP server configurations are not described here. For details, see [Full-automatic Installation Guide](./using-kickstart-for-automatic-installation.md#full-automatic-installation-guide). - -## Installation Through a QCOW2 Image - -This section describes how to create or use a QCOW2 image to install the openEuler. - -### Creating a QCOW2 Image - -1. Install the **qemu-img** software package. - - ```bash - dnf install -y qemu-img - ``` - -2. Run the **create** command of the qemu-img tool to create an image file. The command format is as follows: - - ```bash - qemu-img create -f -o - ``` - - The parameters are described as follows: - - - _imgFormat_: Image format. The value can be **raw** or **qcow2**. - - _fileOption_: File option, which is used to set features of an image file, such as specifying a backend image file, compressing, and encrypting. - - _fileName_: File name. - - _diskSize_: Disk size, which specifies the size of a block disk. The unit can be K, M, G, or T, indicating KiB, MiB, GiB, or TiB. - - For example, to create an image file **openEuler-image.qcow2** whose disk size is 32 GB and format is qcow2, the command and output are as follows: - - ```bash - $ qemu-img create -f qcow2 openEuler-image.qcow2 32G - Formatting 'openEuler-image.qcow2', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 - ``` - -### Starting the Installation - -Perform the following operations to start the installation: - -1. Prepare a QCOW2 image file. -2. Prepare the VM network. -3. Prepare the UEFI boot tool set EDK II. -4. Prepare the VM XML configuration file. -5. Create a VM. -6. Start the VM. - -For details, see the [*Virtualization User Guide*](./../Virtualization/virtualization.md). - -## Installation Through a Private Image - -This section describes how to create or use a private image to install the openEuler. - -### Creating a Private Image - -For instructions about how to create a private image, see [*Image Management Service User Guide*](https://support.huaweicloud.com/intl/en-us/usermanual-ims/en-us_topic_0013901628.html). - -### Starting the Installation - -For details about how to start the x86 virtualization platform of Huawei public cloud, see [Elastic Cloud Server User Guide](https://support.huaweicloud.com/intl/en-us/wtsnew-ims/index.html). diff --git a/docs/en/docs/Installation/installation-preparations.md b/docs/en/docs/Installation/installation-preparations.md deleted file mode 100644 index 0ad55d5d963d4f962a44cfbaf6137aa4c1f9e9ae..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/installation-preparations.md +++ /dev/null @@ -1,121 +0,0 @@ -# Installation Preparations - -This section describes the compatibility of the hardware and software and the related configurations and preparations required for the installation. - -## Obtaining the Installation Source - -Obtain the openEuler release package and verification file before the installation. - -Perform the following operations to obtain the openEuler release package: - -1. Visit the [openEuler](https://www.openeuler.org/en/) website. -2. Click **Downloads**. -3. Click **Community Editions**. The version list is displayed. -4. Click **Download** on the right of **openEuler 22.03 LTS SP2**. -5. Download the required openEuler release package and the corresponding verification file based on the architecture and scenario. - 1. If the architecture is AArch64: - 1. Click **AArch64**. - 2. If you install openEuler from a local source, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-(everything-)aarch64-dvd.iso** to the local host. - 3. If you install openEuler through the network, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-aarch64-dvd.iso** to the local host. - 2. If the architecture is x86_64: - 1. Click **x86_64**. - 2. If you install openEuler from a local source, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-(everything-)x86_64-dvd.iso** to the local host. - 3. If you install openEuler through the network, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-x86_64-dvd.iso** to the local host. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> - When the network is available, install the environment on the network because the ISO release package is small. -> - The release package of the AArch64 architecture supports the UEFI mode, while the release package of the x86\_64 architecture supports both the UEFI and Legacy modes. - -## Release Package Integrity Check - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> This section describes how to verify the integrity of the release package in the AArch64 architecture. The procedure for verifying the integrity of the release package in the x86_64 architecture is the same. - -### Introduction - -To prevent the software package from being incompletely downloaded due to network or storage device faults during transmission, you need to verify the integrity of the software package after obtaining it. Only the software package that passes the verification can be installed. - -Compare the verification value recorded in the verification file with the .iso file verification value calculated manually to check whether the software package passes the verification. If the values are consistent, the .iso file is not damaged. Otherwise, the file is damaged and you need to obtain it again. - -### Prerequisites - -Before verifying the integrity of the release package, prepare the following files: - -ISO file: **openEuler-22.03-LTS-SP2-aarch64-dvd.iso** - -Verification file: Copy and save the **Integrity Check** SHA256 value to a local file. - -### Procedures - -To verify the file integrity, perform the following operations: - -1. Calculate the SHA256 verification value of the file. Run the following command: - - ```shell - sha256sum openEuler-22.03-LTS-SP2-aarch64-dvd.iso - ``` - - After the command is run, the verification value is displayed. - -2. Check whether the calculated value is the same as that of the saved SHA256 value. - - If the verification values are consistent, the .iso file is not damaged. If they are inconsistent, you can confirm that the file is damaged and you need to obtain the file again. - -## Installation Requirements for PMs - -To install the openEuler OS on a PM, the PM must meet the following requirements. - -### Hardware Compatibility - -You need to take hardware compatibility into account during openEuler installation. [Table 1](#table14948632047) describes the types of supported servers. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- TaiShan 200 servers are backed by Huawei Kunpeng 920 processors. ->- Currently, only Huawei TaiShan and FusionServer Pro servers are supported. More servers from other vendors will be supported in the future. - -**Table 1** Supported servers - -| Server Type | Server Name | Server Model | -| :---- | :---- | :---- | -| Rack server | TaiShan 200 | 2280 balanced model | -| Rack server | FusionServer Pro | FusionServer Pro 2288H V5
NOTE：
The server must be configured with the Avago SAS3508 RAID controller card and the LOM-X722 NIC.| - -### Minimum Hardware Specifications - -[Table 2](#tff48b99c9bf24b84bb602c53229e2541) lists the minimum hardware specifications supported by openEuler. - -**Table 2** Minimum hardware specifications - -| Component | Minimum Hardware Specifications | -| :---- | :---- | -| Architecture | AArch64 or x86_64 | -| CPU | Two CPUs | -| Memory | ≥ 4 GB (8 GB or higher recommended for better user experience) | -| Hard disk | ≥ 32 GB (120 GB or higher recommended for better user experience) | - -## Installation Requirements for VMs - -To install the openEuler OS on a VM, the PM must meet the following requirements. - -### Virtualization Platform Compatibility - -When installing openEuler, pay attention to the compatibility of the virtualization platform. Currently, the following virtualization platforms are supported: - -- A virtualization platform created by the virtualization components \(openEuler as the host OS and QEMU and KVM provided in the release package\) of openEuler -- x86 virtualization platform of Huawei public cloud - -### Minimum Virtualization Space - -[Table 3](#tff48b99c9bf24b84bb602c53229e2541) lists the minimum virtualization space required by openEuler. - -**Table 3** Minimum virtualization space - -| Component | Minimum Virtualization Space | -| :---- | :---- | -| Architecture | AArch64 or x86_64 | -| CPU | Two CPUs| -| Memory | ≥ 4 GB (8 GB or higher recommended for better user experience) | -| Hard disk | ≥ 32 GB (120 GB or higher recommended for better user experience) | diff --git a/docs/en/docs/Installation/using-kickstart-for-automatic-installation.md b/docs/en/docs/Installation/using-kickstart-for-automatic-installation.md deleted file mode 100644 index e731b9de0638a5da691139f48e0c8f2300db7279..0000000000000000000000000000000000000000 --- a/docs/en/docs/Installation/using-kickstart-for-automatic-installation.md +++ /dev/null @@ -1,365 +0,0 @@ -# Using Kickstart for Automatic Installation - - -- [Using Kickstart for Automatic Installation](#using-kickstart-for-automatic-installation) - - [Introduction](#introduction) - - [Overview](#overview) - - [Advantages and Disadvantages](#advantages-and-disadvantages) - - [Background](#background) - - [Semi-automatic Installation Guide](#semi-automatic-installation-guide) - - [Environment Requirements](#environment-requirements) - - [Procedure](#procedure) - - [Full-automatic Installation Guide](#full-automatic-installation-guide) - - [Environment Requirements](#environment-requirements-1) - - [Procedure](#procedure-1) - - - -## Introduction - -### Overview - -You can use the kickstart tool to automatically install the openEuler OS in either of the following ways: - -- Semi-automatic installation: You only need to specify the location of the kickstart file. Kickstart automatically configures OS attributes such as keyboard, language, and partitions. -- Automatic installation: The OS is automatically installed. - -### Advantages and Disadvantages - -[Table 1](#table1388812373315) lists the advantages and disadvantages of semi-automatic installation and full-automatic installation using kickstart. You can select an installation mode as required. - -**Table 1** Advantages and disadvantages - - - - - - - - - - - - - - - - -

Installation Mode

-

Advantage

-

Disadvantage

-

Semi-automatic installation

-

Services such as TFTP, PXE, and DHCP do not need to be prepared.

-

You need to manually specify the path of the kickstart file.

-

Full-automatic installation

-

The OS is installed automatically.

-

Services, such as TFTP, DHCPD, and PXE, need to be configured.

-
- -### Background - -**Kickstart** - -Kickstart is an unattended installation mode. The principle of kickstart is to record typical parameters that need to be manually entered during the installation and generate the configuration file **ks.cfg**. During the installation, the installation program searches the **ks.cfg** configuration file first for required parameters. If no matching parameters are found, you need to manually configure these parameters. If all required parameters are covered by the kickstart file, automatic installation can be achieved by only specifying the path of the kickstart file. - -Both full-automatic or semi-automatic installation can be achieved by kickstart. - -**PXE** - -Pre-boot Execution Environment \(PXE\)\) works in client/server network mode. The PXE client can obtain an IP address from the DHCP server during the startup and implement client boot and installation through the network based on protocols such as trivial file transfer protocol \(TFTP\). - -**TFTP** - -TFTP is used to transfer simple and trivial files between clients and the server. - -## Semi-automatic Installation Guide - -### Environment Requirements - -The environment requirements for semi-automatic installation of openEuler OS using kickstart are as follows: - -- PM/VM \(For details about how to create VMs, see the documents from corresponding vendors\): includes the computer where kickstart is used for automatic installation and the computer where the kickstart tool is installed. -- Httpd: stores the kickstart file. -- ISO: openEuler-21.09-aarch64-dvd.iso - -### Procedure - -To use kickstart to perform semi-automatic installation of openEuler, perform the following steps: - -**Environment Preparation** - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Before the installation, ensure that the firewall of the HTTP server is disabled. Run the following command to disable the firewall: -> ->``` ->iptables -F ->``` - -1. Install httpd and start the service. - - ```shell - # dnf install httpd -y - # systemctl start httpd - # systemctl enable httpd - ``` - -2. Run the following commands to prepare the kickstart file: - - ```shell - # mkdir /var/www/html/ks - #vim /var/www/html/ks/openEuler-ks.cfg ===>The file can be obtained by modifying the **anaconda-ks.cfg** file automatically generated from openEuler OS. - ==================================== - ***Modify the following information as required.*** - #version=DEVEL - ignoredisk --only-use=sda - autopart --type=lvm - # Partition clearing information - clearpart --none --initlabel - # Use graphical install - graphical - # Use CDROM installation media - cdrom - # Keyboard layouts - keyboard --vckeymap=cn --xlayouts='cn' - # System language - lang zh_CN.UTF-8 - - # Network information - network --bootproto=dhcp --device=enp4s0 --ipv6=auto --activate - network --hostname=openeuler.com - # Root password - rootpw --iscrypted $6$fQE83lxEZ48Or4zc$j7/PlUMHn29yTjCD4Fi44WTZL/RzVGxJ/7MGsZMl6QfE3KjIVT7M4UrhFXbafvRq2lUddAFcyWHd5WRmXfEK20 - # Run the Setup Agent on first boot - firstboot --enable - # Do not configure the X Window System - skipx - # System services - services --disabled="chronyd" - # System timezone - timezone Asia/Shanghai --utc --nontp - - %packages - @^minimal-environment - @standard - - %end - - %anaconda - pwpolicy root --minlen=8 --minquality=1 --notstrict --nochanges --notempty - pwpolicy user --minlen=8 --minquality=1 --notstrict --nochanges --emptyok - pwpolicy luks --minlen=8 --minquality=1 --notstrict --nochanges --notempty - %end - - %post - #enable kdump - sed -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg - %end - ===================================== - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >The method of generating the password ciphertext is as follows: - > - >``` - ># python3 - >Python 3.7.0 (default, Apr 1 2019, 00:00:00) - >[GCC 7.3.0] on linux - >Type "help", "copyright", "credits" or "license" for more information. - >>>> import crypt - >>>> passwd = crypt.crypt("myPasswd") - >>>> print (passwd) - >$6$63c4tDmQGn5SDayV$mZoZC4pa9Jdt6/ALgaaDq6mIExiOO2EjzomB.Rf6V1BkEMJDcMddZeGdp17cMyc9l9ML9ldthytBEPVcnboR/0 - >``` - -3. Mount the ISO image file to the CD-ROM drive of the computer where openEuler is to be installed. - - If you want to install openEuler through the NFS, specify the path \(which is **cdrom** by default\) of installation source in the kickstart file. - -**Installing the System** - -1. The installation selection dialog box is displayed. - 1. On the installation wizard page in [Starting the Installation](installation-modes.md#starting-the-installation), select **Install openEuler 21.09** and press **e**. - 2. Add **inst.ks=** to the startup parameters. - - ![startparam.png](https://gitee.com/openeuler/docs/raw/master/docs/zh/docs/Installation/figures/startparam.png "startparam.png") - - 3. Press **Ctrl**+**x** to start the automatic installation. - -2. Verify that the installation is complete. - - After the installation is complete, the system automatically reboots. If the first boot option of the system is set to the CD_ROM, the installation page is displayed again. Shut down the computer and change startup option to start from the hard disk preferentially. - - ![](./figures/completing-the-automatic-installation.png) - -## Full-automatic Installation Guide - -### Environment Requirements - -The environment requirements for full-automatic installation of openEuler using kickstart are as follows: - -- PM/VM \(For details about how to create VMs, see the documents from corresponding vendors\): includes the computer where kickstart is used for automatic installation and the computer where the kickstart tool is installed. -- Httpd: stores the kickstart file. -- TFTP: provides vmlinuz and initrd files. -- DHCPD/PXE: provides the DHCP service. -- ISO: openEuler-21.09-aarch64-dvd.iso - -### Procedure - -To use kickstart to perform full-automatic installation of openEuler, perform the following steps: - -**Environment Preparation** - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Before the installation, ensure that the firewall of the HTTP server is disabled. Run the following command to disable the firewall: -> ->``` ->iptables -F ->``` - -1. Install httpd and start the service. - - ```shell - # dnf install httpd -y - # systemctl start httpd - # systemctl enable httpd - ``` - -2. Install and configure TFTP. - - ```shell - # dnf install tftp-server -y - # vim /etc/xinetd.d/tftp - service tftp - { - socket_type = dgram - protocol = udp - wait = yes - user = root - server = /usr/sbin/in.tftpd - server_args = -s /var/lib/tftpboot - disable = no - per_source = 11 - cps = 100 2 - flags = IPv4 - } - # systemctl start tftp - # systemctl enable tftp - # systemctl start xinetd - # systemctl status xinetd - # systemctl enable xinetd - ``` - -3. Prepare the installation source. - - ```shell - # mount openEuler-21.09-aarch64-dvd.iso /mnt - # cp -r /mnt/* /var/www/html/openEuler/ - ``` - -4. Set and modify the kickstart configuration file **openEuler-ks.cfg**. Select the HTTP installation source by referring to [3](#en-us_topic_0229291289_l1692f6b9284e493683ffa2ef804bc7ca). - - ```shell - #vim /var/www/html/ks/openEuler-ks.cfg - ==================================== - ***Modify the following information as required.*** - #version=DEVEL - ignoredisk --only-use=sda - autopart --type=lvm - # Partition clearing information - clearpart --none --initlabel - # Use graphical install - graphical - # Keyboard layouts - keyboard --vckeymap=cn --xlayouts='cn' - # System language - lang zh_CN.UTF-8 - #Use http installation source - url --url=//192.168.122.1/openEuler/ - %post - #enable kdump - sed -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg - %end - ... - ``` - -5. Modify the PXE configuration file **grub.cfg** as follows. (Note: Currently, openEuler does not support the cfg file in bls format.) - - ```shell - # cp -r /mnt/images/pxeboot/* /var/lib/tftpboot/ - # cp /mnt/EFI/BOOT/grubaa64.efi /var/lib/tftpboot/ - # cp /mnt/EFI/BOOT/grub.cfg /var/lib/tftpboot/ - # ls /var/lib/tftpboot/ - grubaa64.efi grub.cfg initrd.img TRANS.TBL vmlinuz - # vim /var/lib/tftpboot/grub.cfg - set default="1" - - function load_video { - if [ x$feature_all_video_module = xy ]; then - insmod all_video - else - insmod efi_gop - insmod efi_uga - insmod ieee1275_fb - insmod vbe - insmod vga - insmod video_bochs - insmod video_cirrus - fi - } - - load_video - set gfxpayload=keep - insmod gzio - insmod part_gpt - insmod ext2 - - set timeout=60 - - - ### BEGIN /etc/grub.d/10_linux ### - menuentry 'Install openEuler 21.03 ' --class red --class gnu-linux --class gnu --class os { - set root=(tftp,192.168.1.1) - linux /vmlinuz ro inst.geoloc=0 console=ttyAMA0 console=tty0 rd.iscsi.waitnet=0 inst.ks=http://192.168.122.1/ks/openEuler-ks.cfg - initrd /initrd.img - } - ``` - -6. Configure DHCP \(which can be replaced by DNSmasq\). - - ```shell - # dnf install dhcp -y - # - # DHCP Server Configuration file. - # see /usr/share/doc/dhcp-server/dhcpd.conf.example - # see dhcpd.conf(5) man page - # - # vim /etc/dhcp/dhcpd.conf - ddns-update-style interim; - ignore client-updates; - filename "grubaa64.efi"; # location of the pxelinux startup file; - next-server 192.168.122.1; # (IMPORTANT) IP address of the TFTP server; - subnet 192.168.122.0 netmask 255.255.255.0 { - option routers 192.168.111.1; # Gateway address - option subnet-mask 255.255.255.0; # Subnet mask - range dynamic-bootp 192.168.122.50 192.168.122.200; # Dynamic IP address range - default-lease-time 21600; - max-lease-time 43200; - } - # systemctl start dhcpd - # systemctl enable dhcpd - ``` - -**Installing the System** - -1. On the **Start boot option** screen, press **F2** to boot from the PXE and start automatic installation. - - ![](./figures/en-us_image_0229291270.png) - - ![](./figures/en-us_image_0229291286.png) - - ![](./figures/en-us_image_0229291247.png) - -2. The automatic installation window is displayed. -3. Verify that the installation is complete. - - ![](./figures/completing-the-automatic-installation.png) diff --git a/docs/en/docs/K3s/K3s-deployment-guide.md b/docs/en/docs/K3s/K3s-deployment-guide.md deleted file mode 100644 index e61d7082278eaae6b3dcc246d7a60517a524450e..0000000000000000000000000000000000000000 --- a/docs/en/docs/K3s/K3s-deployment-guide.md +++ /dev/null @@ -1,86 +0,0 @@ -# K3s Deployment Guide - -### What Is K3s? -K3s is a lightweight Kubernetes distribution that is optimized for edge computing and IoT scenarios. The K3s provides the following enhanced features: -- Packaged as a single binary file. -- Uses SQLite3-based lightweight storage backend as the default storage mechanism and supports etcd3, MySQL, and PostgreSQL. -- Encapsulated in a simple launcher that handles various complex TLS and options. -- Secure by default and has reasonable default values for lightweight environments. -- Batteries included, providing simple but powerful functions such as local storage providers, service load balancers, Helm controllers, and Traefik Ingress controllers. -- Encapsulates all operations of the Kubernetes control plane in a single binary file and process, capable of automating and managing complex cluster operations including certificate distribution. -- Minimizes external dependencies and requires only kernel and cgroup mounting. - -### Application Scenarios -K3s is applicable to the following scenarios: - -- Edge computing -- IoT -- Continuous integration -- Development -- ARM -- Embedded Kubernetes - -The resources required for running K3s are small. Therefore, K3s is also suitable for development and test scenarios. In these scenarios, K3s facilitates function verification and problem reproduction by shortening cluster startup time and reducing resources consumed by the cluster. - -### Deploying K3s - -#### Preparations - -- Ensure that the host names of the server node and agent node are different. - -You can run the `hostnamectl set-hostname "host name"` command to change the host name. - -![1661829534335](./figures/set-hostname.png) - -- Install K3s on each node using Yum. - - The K3s official website provides binary executable files of different architectures and the **install.sh** script for offline installation. The openEuler community migrates the compilation process of the binary file to the community and releases the compiled RPM package. You can run the `yum` command to download and install K3s. - -![1661830441538](./figures/yum-install.png) - -#### Deploying the Server Node - -To install K3s on a single server, run the following command on the server node: -``` -INSTALL_K3S_SKIP_DOWNLOAD=true k3s-install.sh -``` - -![1661825352724](./figures/server-install.png) - -#### Checking Server Deployment - -![1661825403705](./figures/check-server.png) - -#### Deploying the Agent Node - -Query the token value of the server node. The token is stored in the **/var/lib/rancher/k3s/server/node-token** file on the server node. - -> **Note**: -> -> Only the second half of the token is used. - -![1661825538264](./figures/token.png) - -Add agents. Run the following command on each agent node: - -``` -INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken k3s-install.sh -``` - -> **Note:** -> -> Replace **myserver** with the IP address of the server or a valid DNS, and replace **mynodetoken** with the token of the server node. - -![1661829392357](./figures/agent-install.png) - -#### Checking Agent Deployment - -After the installation is complete, run `kubectl get nodes` on the server node to check if the agent node is successfully registered. - -![1661826797319](./figures/check-agent.png) - -A basic K3S cluster is set up. - -#### More - -For details about how to use K3s, visit the K3s [official website](https://rancher.com/docs/k3s/latest/en/). diff --git a/docs/en/docs/Kernel/how-to-use.md b/docs/en/docs/Kernel/how-to-use.md deleted file mode 100644 index 6079a2bb92663f33297627a8e0f3009acf1d1a25..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kernel/how-to-use.md +++ /dev/null @@ -1,255 +0,0 @@ -## How to Use - -### Tiered-Reliability Memory Management for the OS - -**Overview** - -Memory is divided into two ranges based on high and low reliability. Therefore, memory allocation and release must be managed separately based on the reliability. The OS must be able to control the memory allocation path. User-mode processes use low reliable memory, and kernel-mode processes use highly reliable memory. When the highly reliable memory is insufficient, the allocation needs to fall back to the low reliable memory range or the allocation fails. - -In addition, according to the reliability requirements and types of processes, on-demand allocation of highly reliable and low reliable memory is required. For example, specify highly reliable memory for key processes to reduce the probability of memory errors encountered by key processes. Currently, the kernel uses highly reliable memory, and user-mode processes use low reliable memory. As a result, some key or core services, such as the service forwarding process, are unstable. If an exception occurs, I/Os are interrupted, affecting service stability. Therefore, these key services must use highly reliable memory to improve the stability of key processes. - -When a memory error occurs in the system, the OS overwrites the unallocated low reliable memory to clear the undetected memory error. - -**Restrictions** - -- **High-reliability memory for key processes** - - 1. The abuse of the `/proc//reliable` API may cause excessive use of highly reliable memory. - 2. The `reliable` attribute of a user-mode process can be modified by using the proc API or directly inherited from its parent process only after the process is started. `systemd (pid=1)` uses highly reliable memory. Its `reliable` attribute is useless and is not inherited. The `reliable` attribute of kernel-mode threads is invalid. - 3. The program and data segments of processes use highly reliable memory. Because the highly reliable memory is insufficient, the low reliable memory is used for startup. - 4. Common processes also use highly reliable memory in some scenarios, such as HugeTLB, page cache, vDSO, and TMPFS. - -- **Overwrite of unallocated memory** - - The overwrite of the unallocated memory can be executed only once and does not support concurrent operations. If this feature is executed, it will have the following impacts: - - 1. This feature takes a long time. When one CPU of each node is occupied by the overwrite thread, other tasks cannot be scheduled on the CPU. - 2. During the overwrite process, the zone lock needs to be obtained. Other service processes need to wait until the overwrite is complete. As a result, the memory may not be allocated in time. - 3. In the case of concurrent execution, queuing is blocked, resulting in a longer delay. - - If the machine performance is poor, the kernel RCU stall or soft lockup alarm may be triggered, and the process memory allocation may be blocked. Therefore, this feature can be used only on physical machines if necessary. There is a high probability that the preceding problem occurs on VMs. - - The following table lists the reference data of physical machines. (The actual time required depends on the hardware performance and system load.) - - -Table 1 Test data when the TaiShan 2280 V2 server is unloaded - -| Test Item | Node 0 | Node 1 | Node 2 | Node 3 | -| ------------- | ------ | ------ | ------ | ------ | -| Free Mem (MB) | 109290 | 81218 | 107365 | 112053 | - -The total time is 3.2s. - -**Usage** - -This sub-feature provides multiple APIs. You only need to perform steps 1 to 6 to enable and verify the sub-feature. - -1. Configure `kernelcore=reliable` to enable tiered-reliability memory management. `CONFIG_MEMORY_RELIABLE` is mandatory. Otherwise, tiered-reliability memory management is disabled for the entire system. - -2. You can use the startup parameter `reliable_debug=[F][,S][,P]` to disable the fallback function (`F`), disable the TMPFS to use highly reliable memory (`S`), or disable the read/write cache to use highly reliable memory (`P`). By default, all the preceding functions are enabled. - -3. Based on the address range reported by the BIOS, the system searches for and marks the highly reliable memory. For the NUMA system, not every node needs to reserve reliable memory. However, the lower 4 GB physical space on node 0 must be highly reliable memory. During the system startup, the system allocates memory. If the highly reliable memory cannot be allocated, the low reliable memory is allocated (based on the fallback logic of the mirroring function) or the system cannot be started. If low reliable memory is used, the entire system is unstable. Therefore, the highly reliable memory on node 0 must be retained and the lower 4 GB physical space must be highly reliable memory. - -4. After the startup, you can check whether memory tiering is enabled based on the startup log. If it is enabled, the following information is displayed: - - ``` - mem reliable: init succeed, mirrored memory - ``` - -5. The physical address range corresponding to the highly reliable memory can be queried in the startup log. Observe the attributes in the memory map reported by the EFI. The memory range with `MR` is the highly reliable memory range. The following is an excerpt of the startup log. The memory range `mem06` is the highly reliable memory, and `mem07` is the low reliable memory. Their physical address ranges are also listed (the highly and low reliable memory address ranges cannot be directly queried in other modes). - - ``` - [ 0.000000] efi: mem06: [Conventional Memory| |MR| | | | | | |WB| | | ] range=[0x0000000100000000-0x000000013fffffff] (1024MB) - [ 0.000000] efi: mem07: [Conventional Memory| | | | | | | | |WB| | | ] range=[0x0000000140000000-0x000000083eb6cfff] (28651MB) - ``` - -6. During kernel-mode development, a page struct page can be determined based on the zone where the page is located. `ZONE_MOVABLE` indicates a low reliable memory zone. If the zone ID is smaller than `ZONE_MOVABLE`, the zone is a highly reliable memory zone. The following is an example: - - ``` - bool page_reliable(struct page *page) - { - if (!mem_reliable_status() || !page) - return false; - return page_zonenum(page) < ZONE_MOVABLE; - } - ``` - - In addition, the provided APIs are classified based on their functions: - - 1. **Checking whether the reliability function is enabled at the code layer**: In the kernel module, use the following API to check whether the tiered-reliability memory management function is enabled. If `true` is returned, the function is enabled. If `false` is returned, the function is disabled. - - ``` - #include - bool mem_reliable_status(void); - ``` - - 2. **Memory hot swap**: If the kernel enables the memory hot swap operation (Logical Memory hot-add), the highly and low reliable memories also support this operation. The operation unit is the memory block, which is the same as the native process. - - ``` - # Bring the memory online to the highly reliable memory range. - echo online_kernel > /sys/devices/system/memory/auto_online_blocks - # Bring the memory online to the low reliable memory range. - echo online_movable > /sys/devices/system/memory/auto_online_blocks - ``` - - 3. **Dynamically disabling a tiered management function**: The long type is used to determine whether to enable or disable the tiered-reliability memory management function based on each bit. - - - `bit0`: enables tiered-reliability memory management. - - `bit1`: disables fallback to the low reliable memory range. - - `bit2`: disables TMPFS to use highly reliable memory. - - `bit3`: disables the page cache to use highly reliable memory. - - Other bits are reserved for extension. If you need to change the value, call the following proc API (the permission is 600). The value range is 0-15. (The subsequent functions are processed only when `bit 0` of the general function is `1`. Otherwise, all functions are disabled.) - - ``` - echo 15 > /proc/sys/vm/reliable_debug - # All functions are disabled because bit0 is 0. - echo 14 > /proc/sys/vm/reliable_debug - ``` - - This command can only be used to disable the function. This command cannot be used to enable a function that has been disabled or is disabled during running. - - Note: This function is used for escape and is configured only when the tiered-reliability memory management feature needs to be disabled in abnormal scenarios or during commissioning. Do not use this function as a common function. - - 4. **Viewing highly reliable memory statistics**: Call the native `/proc/meminfo` API. - - - `ReliableTotal`: total size of reliable memory managed by the kernel. - - `ReliableUsed`: total size of reliable memory used by the system, including the reserved memory used in the system. - - `ReliableBuddyMem`: remaining reliable memory of the partner system. - - `ReliableTaskUsed`: highly reliable memory used by systemd and key user processes, including anonymous pages and file pages. - - `ReliableShmem`: highly reliable memory usage of the shared memory, including the total highly reliable memory used by the shared memory, TMPFS, and rootfs. - - `ReliableFileCache`: highly reliable memory usage of the read/write cache. - - 5. **Overwrite of unallocated memory**: This function requires the configuration item to be enabled. - - Enable `CONFIG_CLEAR_FREELIST_PAGE` and add the startup parameter `clear_freelist`. Call the proc API. The value can only be `1` (the permission is 0200). - - ``` - echo 1 > /proc/sys/vm/clear_freelist_pages - ``` - - Note: This feature depends on the startup parameter `clear_freelist`. The kernel matches only the prefix of the startup parameter. Therefore, this feature also takes effect for parameters with misspelled suffix, such as `clear_freelisttt`. - - To prevent misoperations, add the kernel module parameter `cfp_timeout_ms` to indicate the maximum execution duration of the overwrite function. If the overwrite function times out, the system exits even if the overwrite operation is not complete. The default value is `2000` ms (the permission is 0644). - - ``` - echo 500 > /sys/module/clear_freelist_page/parameters/cfp_timeout_ms # Set the timeout to 500 ms. - ``` - - 6. **Checking and modifying the high and low reliability attribute of the current process**: Call the `/proc//reliable` API to check whether the process is a highly reliable process. If the process is running and written, the attribute is inherited. If the subprocess does not require the attribute, manually modify the subprocess attribute. The systemd and kernel threads do not support the read and write of the attribute. The value can be `0` or `1`. The default value is `0`, indicating a low reliable process (the permission is 0644). - - ``` - # Change the process whose PID is 1024 to a highly reliable process. After the change, the process applies for memory from the highly reliable memory range. If the memory fails to be allocated, the allocation may fall back to the low reliable memory range. - echo 1 > /proc/1024/reliable - ``` - - 7. **Setting the upper limit of highly reliable memory requested by user-mode processes**: Call `/proc/sys/vm/task_reliable_limit` to modify the upper limit of highly reliable memory requested by user-mode processes. The value range is [`ReliableTaskUsed`, `ReliableTotal`], and the unit is byte (the permission is 0644). Notes: - - - The default value is `ulong_max`, indicating that there is no limit. - - If the value is `0`, the reliable process cannot use the highly reliable memory. In fallback mode, the allocation falls back to the low reliable memory range. Otherwise, OOM occurs. - - If the value is not `0` and the limit is triggered, the fallback function is enabled. The allocation falls back to the low reliable memory range. If the fallback function is disabled, OOM is returned. - -### Highly Reliable Memory for Read and Write Cache - -**Overview** - -A page cache is also called a file cache. When Linux reads or writes files, the page cache is used to cache the logical content of the files to accelerate the access to images and data on disks. If low reliable memory is allocated to page caches, UCE may be triggered during the access, causing system exceptions. Therefore, the read/write cache (page cache) needs to be placed in the highly reliable memory zone. In addition, to prevent the highly reliable memory from being exhausted due to excessive page cache allocations (unlimited by default), the total number of page caches and the total amount of reliable memory need to be limited. - -**Restrictions** - -1. When the page cache exceeds the limit, it is reclaimed periodically. If the generation speed of the page cache is faster than the reclamation speed, the number of page caches may be higher than the specified limit. -2. The usage of `/proc/sys/vm/reliable_pagecache_max_bytes` has certain restrictions. In some scenarios, the page cache forcibly uses reliable memory. For example, when metadata (such as inode and dentry) of the file system is read, the reliable memory used by the page cache exceeds the API limit. In this case, you can run `echo 2 \> /proc/sys/vm/drop_caches` to release inode and dentry. -3. When the highly reliable memory used by the page cache exceeds the `reliable_pagecache_max_bytes` limit, the low reliable memory is allocated by default. If the low reliable memory cannot be allocated, the native process is used. -4. FileCache statistics are first collected in the percpu cache. When the value in the cache exceeds the threshold, the cache is added to the entire system and then displayed in `/proc/meminfo`. `ReliableFileCache` does not have the preceding threshold in `/proc/meminfo`. As a result, the value of `ReliableFileCache` may be greater than that of `FileCache`. -5. Write cache scenarios are restricted by `dirty_limit` (restricted by /`proc/sys/vm/dirty_ratio`, indicating the percentage of dirty pages on a single memory node). If the threshold is exceeded, the current zone is skipped. For tiered-reliability memory, because highly and low reliable memories are in different zones, the write cache may trigger fallback of the local node and use the low reliable memory of the local node. You can run `echo 100 > /proc/sys/vm/dirty_ratio` to cancel the restriction. -6. The highly reliable memory feature for the read/write cache limits the page cache usage. The system performance is affected in the following scenarios: - - If the upper limit of the page cache is too small, the I/O increases and the system performance is affected. - - If the page cache is reclaimed too frequently, system freezing may occur. - - If a large amount of page cache is reclaimed each time after the page cache exceeds the limit, system freezing may occur. - -**Usage** - -The highly reliable memory is enabled by default for the read/write cache. To disable the highly reliable memory, configure `reliable_debug=P`. In addition, the page cache cannot be used unlimitedly. The function of limiting the page cache size depends on the `CONFIG_SHRINK_PAGECACHE` configuration item. - -`FileCache` in `/proc/meminfo` can be used to query the usage of the page cache, and `ReliableFileCache` can be used to query the usage of the reliable memory in the page cache. - -The function of limiting the page cache size depends on several proc APIs, which are defined in `/proc/sys/vm/` to control the page cache usage. For details, see the following table. - -| API Name (Native/New) | Permission| Description | Default Value | -| ------------------------------------ | ---- | ------------------------------------------------------------ | ------------------------------------------ | -| `cache_reclaim_enable` (native) | 644 | Whether to enable the page cache restriction function.
Value range: `0` or `1`. If an invalid value is input, an error is returned.
Example: `echo 1 > cache_reclaim_enable`| 1 | -| `cache_limit_mbytes` (new) | 644 | Upper limit of the cache, in MB.
Value range: The minimum value is 0, indicating that the restriction function is disabled. The maximum value is the memory size in MB, for example, the value displayed by running the `free –m` command (the value of `MemTotal` in `meminfo` converted in MB).
Example: `echo 1024 \> cache_limit_mbytes`
Others: It is recommended that the cache upper limit be greater than or equal to half of the total memory. Otherwise, the I/O performance may be affected if the cache is too small.| 0 | -| `cache_reclaim_s` (native) | 644 | Interval for triggering cache reclamation, in seconds. The system creates work queues based on the number of online CPUs. If there are *n* CPUs, the system creates *n* work queues. Each work queue performs reclamation every `cache_reclaim_s` seconds. This parameter is compatible with the CPU online and offline functions. If the CPU is offline, the number of work queues decreases. If the CPU is online, the number of work queues increases.
Value range: The minimum value is `0` (indicating that the periodic reclamation function is disabled) and the maximum value is `43200`. If an invalid value is input, an error is returned.
Example: `echo 120 \> cache_reclaim_s`
Others: You are advised to set the reclamation interval to several minutes (for example, 2 minutes). Otherwise, frequent reclamation may cause system freezing.| 0 | -| `cache_reclaim_weight` (native) | 644 | Weight of each reclamation. Each CPU of the kernel expects to reclaim `32 x cache_reclaim_weight` pages each time. This weight applies to both reclamation triggered by the page upper limit and periodic page cache reclamation.
Value range: 1 to 100. If an invalid value is input, an error is returned.
Example: `echo 10 \> cache_reclaim_weight`
Others: You are advised to set this parameter to `10` or a smaller value. Otherwise, the system may freeze each time too much memory is reclaimed.| 1 | -| `reliable_pagecache_max_bytes` (new)| 644 | Total amount of highly reliable memory in the page cache.
Value range: 0 to the maximum highly reliable memory, in bytes. You can call `/proc/meminfo` to query the maximum highly reliable memory. If an invalid value is input, an error is returned.
Example: `echo 4096000 \> reliable_pagecache_max_bytes`| Maximum value of the unsigned long type, indicating that the usage is not limited.| - -### Highly Reliable Memory for entered - -**Overview** - -If TMPFS is used as rootfs, it stores core files and data used by the OS. However, TMPFS uses low reliable memory by default, which makes core files and data unreliable. Therefore, TMPFS must use highly reliable memory. - -**Usage** - -By default, the highly reliable memory is enabled for TMPFS. To disable it, configure `reliable_debug=S`. You can dynamically disable it by calling `/proc/sys/vm/reliable_debug`, but cannot dynamically enable it. - -When enabling TMPFS to use highly reliable memory, you can check `ReliableShmem` in `/proc/meminfo` to view the highly reliable memory that has been used by TMPFS. - -By default, the upper limit for TMPFS to use highly reliable memory is half of the physical memory (except when TMPFS is used as rootfs). The conventional SYS V shared memory is restricted by `/proc/sys/kernel/shmmax and /proc/sys/kernel/shmall` and can be dynamically configured. It is also restricted by the highly reliable memory used by TMPFS. For details, see the following table. - -| **Parameter** | **Description** | -|---------------------------------|--------------------------------| -| `/proc/sys/kernel/shmmax` (native)| Size of a single SYS V shared memory range.| -| `/proc/sys/kernel/shmall` (native)| Total size of the SYS V shared memory that can be used. | - -The `/proc/sys/vm/shmem_reliable_bytes_limit` API is added for you to set the available highly reliable size (in bytes) of the system-level TMPFS. The default value is `LONG_MAX`, indicating that the usage is not limited. The value ranges from 0 to the total reliable memory size of the system. The permission is 644. When fallback is disabled and the memory usage reaches the upper limit, an error indicating that no memory is available is returned. When fallback is enabled, the system attempts to allocate memory from the low reliable memory zone. Example: - -``` -echo 10000000 > /proc/sys/vm/shmem_reliable_bytes_limit -``` - -### UCE Does Not Reset After the Switch from the User Mode to Kernel Mode - -**Overview** - -Based on the tiered-reliability memory management solution, the kernel and key processes use highly reliable memory. Most user-mode processes use low reliable memory. When the system is running, a large amount of data needs to be exchanged between the user mode and kernel mode. When data is transferred to the kernel mode, data in the low reliable memory zone is copied to the highly reliable memory zone. The copy operation is performed in kernel mode. If a UCE occurs when the user-mode data is read, that is, the kernel-mode memory consumption UCE occurs, the system triggers a panic. This sub-feature provides solutions for scenarios where UCEs occurred in the switch from the user mode to kernel mode to avoid system reset, including copy-on-write (COW), copy_to_user, copy_from_user, get_user, put_user, and core dump scenarios. Other scenarios are not supported. - -**Restrictions** - -1. ARMv8.2 or later that supports the RAS feature. -2. This feature changes the synchronization exception handling policy. Therefore, this feature takes effect only when the kernel receives a synchronization exception reported by the firmware. -3. The kernel processing depends on the error type reported by the BIOS. The kernel cannot process fatal hardware errors but can process recoverable hardware errors. -4. Only the COW, copy_to_user (including the read page cache), copy_from_user, get_user, put_user, and core dump scenarios are supported. -5. In the core dump scenario, UCE tolerance needs to be implemented on the write API of the file system. This feature supports only three common file systems: ext4, TMPFS, and PipeFS. The corresponding error tolerance APIs are as follows: - - PipeFS: `copy_page_from_iter` - - ext4/TMPFS: `iov_iter_copy_from_user_atomic` - -**Usage** - -Ensure that `CONFIG_ARCH_HAS_COPY_MC` is enabled in the kernel. If `/proc/sys/kernel/machine_check_safe` is set to `1`, this feature is enabled for all scenarios. If `/proc/sys/kernel/machine_check_safe` is set to `0`, this feature is disabled for all scenarios. Other values are invalid. - -The fault tolerance mechanism in each scenario is as follows: - -| **No.**| **Scenario** | **Symptom** | **Mitigation Measure** | -| ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| 1 | `copy_from/to_user`: basic switch to the user mode, involving syscall, sysctl, and procfs operations| If a UCE occurs during the copy, the kernel is reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset. | -| 2 | `get/put_user`: simple variable copy, mainly in netlink scenarios.| If a UCE occurs during the copy, the kernel is reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset. | -| 3 | COW: fork subprocess, which triggers COW. | COW is triggered. If a UCE occurs, the kernel is reset. | If a UCE occurs, kill related processes. The kernel does not automatically reset.| -| 4 | Read cache: The user mode uses low reliable memory. When a user-mode program reads or writes files, the OS uses idle memory to cache hard disk files, improving performance. However, when the user-mode program reads a file, the kernel accesses the cache first.| A UCE occurs, causing the kernel to reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset.| -| 5 | UCE is triggered by memory access during a core dump. | A UCE occurs, causing the kernel to reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset. | -| 6 | Write cache: When the write cache is flushed back to the disk, a UCE is triggered. | Cache flushing is actually disk DMA data migration. If a UCE is triggered during this process, page write fails after timeout. As a result, data inconsistency occurs and the file system becomes unavailable. If the data is key data, the kernel resets.| No solution is available. The kernel will be reset. | -| 7 | Kernel startup parameters and module parameters use highly reliable memory. | / | Not supported. The risk is reduced. | -| 8 | relayfs: a file system that quickly forwards data from the kernel mode to the user mode.| / | Not supported. The risk is reduced. | -| 9 | `seq_file`: transfers kernel data to the user mode as a file. | / | Not supported. The risk is reduced. | - -Most user-mode data uses low reliable memory. Therefore, this project involves only the scenario where user-mode data is read in kernel mode. In Linux, data can be exchanged between the user space and kernel space in nine modes, including kernel startup parameters, module parameters, sysfs, sysctl, syscall (system call), netlink, procfs, seq_file, debugfs, and relayfs. There are two other cases: COW and read/write file cache (page cache) when a process is created. - -In sysfs, syscall, netlink, and procfs modes, data is transferred from the user mode to the kernel mode in copy_from_user or get_user mode. - -The user mode can be switched to the kernel mode in the following scenarios: - -copy_from_user, get_user, COW, read cache, and write cache flushing. - -The kernel mode can be switched to the user mode in the following scenarios: - -relayfs, seq_file, copy_to_user, and put_user diff --git a/docs/en/docs/Kernel/overview.md b/docs/en/docs/Kernel/overview.md deleted file mode 100644 index 2751a20e048a4a77e19609d6f9cf9a47f090fb12..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kernel/overview.md +++ /dev/null @@ -1,3 +0,0 @@ -## Overview - -This feature allows you to allocate memory with corresponding reliability as required and mitigates the impact of some possible UCEs or CEs to some extent. In this way, the overall service reliability does not deteriorate when partial memory mirroring (a RAS feature called address range mirroring) is used. \ No newline at end of file diff --git a/docs/en/docs/Kernel/restrictions.md b/docs/en/docs/Kernel/restrictions.md deleted file mode 100644 index 3656a25694c047dbc18979d9b1847c36c0d5bdf2..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kernel/restrictions.md +++ /dev/null @@ -1,87 +0,0 @@ -## Restrictions - -This section describes the general constraints of this feature. Each subfeature has specific constraints, which are described in the corresponding section. - -**Compatibility** - -1. Currently, this feature applies only to ARM64. -2. The hardware needs to support partial memory mirroring (address range mirroring), that is, the memory whose attribute is `EFI_MEMORY_MORE_RELIABLE` is reported through the UEFI standard API. Common memory does not need to be set. The mirrored memory is the highly reliable memory, and the common memory is the low reliable memory. -3. High-reliability and low reliable memory tiering is implemented by using the memory management zones of the kernel. They cannot dynamically flow (that is, pages cannot move between different zones). -4. Continuous physical memory with different reliability is divided into different memblocks. As a result, the allocation of large continuous physical memory blocks may be restricted after memory tiering is enabled. -5. To enable this feature, the value of `kernelcore` must be `reliable`, which is incompatible with other values of this parameter. - - -**Design Specifications** - -1. During kernel-mode development, pay attention to the following points when allocating memory: - - - If the memory allocation API supports the specified `gfp_flag`, only the memory allocation whose `gfp_flag` contains `__GFP_HIGHMEM` and `__GFP_MOVABLE` forcibly allocates the common memory range or redirects to the reliable memory range. Other `gfp_flags` do not intervene. - - - High-reliability memory is allocated from slab, slub, and slob. (If the memory allocated at a time is greater than `KMALLOC_MAX_CACHE_SIZE` and `gfp_flag` is set to a common memory range, low reliable memory may be allocated.) - -2. During user-mode development, pay attention to the following points when allocating memory: - - - After the attribute of a common process is changed to a key process, the highly reliable memory is used only in the actual physical memory allocation phase (page fault). The attribute of the previously allocated memory does not change, and vice versa. Therefore, the memory allocated when a common process is started and changed to a key process may not be highly reliable memory. Whether the configuration takes effect can be verified by querying whether the physical address corresponding to the virtual address belongs to the highly reliable memory range. - - Similar mechanisms (ptmalloc, tcmalloc, and dpdk) in libc libraries, such as chunks in glibc, use cache logic to improve performance. However, memory cache causes inconsistency between the memory allocation logics of the user and the kernel. When a common process becomes a key process, this feature cannot be enabled (it is enabled only when the kernel allocates memory). - -3. When an upper-layer service applies for memory, if the highly reliable memory is insufficient (triggering the native min waterline of the zone) or the corresponding limit is triggered, the page cache is preferentially released to attempt to reclaim the highly reliable memory. If the memory still cannot be allocated, the kernel selects OOM or fallback to the low reliable memory range based on the fallback switch to complete memory allocation. (Fallback indicates that when the memory of a memory management zone or node is insufficient, memory is allocated from other memory management zones or nodes.) - -4. The dynamic memory migration mechanism similar to `NUMA_BALANCING` may cause the allocated highly reliable or low reliable memory to be migrated to another node. Because the migration operation loses the memory allocation context and the target node may not have the corresponding reliable memory, the memory reliability after the migration may not be as expected. - -5. The following configuration files are introduced based on the usage of the user-mode highly reliable memory: - - - **/proc/sys/vm/task_reliable_limit**: upper limit of the highly reliable memory used by key processes (including systemd). It contains anonymous pages and file pages. The SHMEM used by the process is also counted (included in anonymous pages). - - - **/proc/sys/vm/reliable_pagecache_max_bytes**: soft upper limit of the highly reliable memory used by the global page cache. The number of highly reliable page caches used by common processes is limited. By default, the system does not limit the highly reliable memory used by page caches. This restriction does not apply to scenarios such as highly reliable processes and file system metadata. Regardless of whether fallback is enabled, when a common process triggers the upper limit, the low reliable memory is allocated by default. If the low reliable memory cannot be allocated, the native process is used. - - - **/proc/sys/vm/shmem_reliable_bytes_limit**: soft upper limit of the highly reliable memory used by the global SHMEM. It limits the amount of highly reliable memory used by the SHMEM of common processes. By default, the system does not limit the amount of highly reliable memory used by SHMEM. High-reliability processes are not subject to this restriction. When fallback is disabled, if a common process triggers the upper limit, memory allocation fails, but OOM does not occur (consistent with the native process). - - If the above limits are reached, memory allocation fallback or OOM may occur. - - Memory allocation caused by page faults generated by key processes in the TMPFS or page cache may trigger multiple limits. For details about the interaction between multiple limits, see the following table. - - | Whether task_reliable_limit Is Reached| Whether reliable_pagecache_max_bytes or shmem_reliable_bytes_limit Is Reached| Memory Allocation Processing Policy | - | --------------------------- | ------------------------------------------------------------ | ------------------------------------------------ | - | Yes | Yes | The page cache is reclaimed first to meet the allocation requirements. Otherwise, fallback or OOM occurs.| - | Yes | No | The page cache is reclaimed first to meet the allocation requirements. Otherwise, fallback or OOM occurs.| - | No | No | High-reliability memory is allocated first. Otherwise, fallback or OOM occurs. | - | No | Yes | High-reliability memory is allocated first. Otherwise, fallback or OOM occurs. | - - Key processes comply with `task_reliable_limit`. If `task_reliable_limit` is greater than `tmpfs` or `pagecachelimit`, page cache and TMPFS generated by key processes still use highly reliable memory. As a result, the highly reliable memory used by page cache and TMPFS is greater than the corresponding limit. - - When `task_reliable_limit` is triggered, if the size of the highly reliable file cache is less than 4 MB, the file cache will not be reclaimed synchronously. If the highly reliable file cache is less than 4 MB when the page cache is generated, the allocation will fall back to the low reliable memory range. If the highly reliable file cache is greater than 4 MB, the page cache is reclaimed preferentially for allocation. However, when the size is close to 4 MB, direct cache reclamation is triggered more frequently. Because the lock overhead of direct cache reclamation is high, the CPU usage is high. In this case, the file read/write performance is close to the raw disk performance. - -6. Even if the system has sufficient highly reliable memory, the allocation may fall back to the low reliable memory range. - - - If the memory cannot be migrated to another node for allocation, the allocation falls back to the low reliable memory range of the current node. The common scenarios are as follows: - - If the memory allocation contains `__GFP_THISNODE` (for example, transparent huge page allocation), memory can be allocated only from the current node. If the highly reliable memory of the node does not meet the allocation requirements, the system attempts to allocate memory from the low reliable memory range of the memory node. - - A process runs on a node that contains common memory by running commands such as `taskset` and `numactl`. - - A process is scheduled to a common memory node under the native scheduling mechanism of the system memory. - - High-reliability memory allocation triggers the highly reliable memory usage threshold, which also causes fallback to the low reliable memory range. - -7. If tiered-reliability memory fallback is disabled, highly reliable memory cannot be expanded to low reliable memory. As a result, user-mode applications may not be compatible with this feature in determining the memory usage, for example, determining the available memory based on MemFree. - -8. If tiered-reliability memory fallback is enabled, the native fallback is affected. The main difference lies in the selection of the memory management zone and NUMA node. - - - Fallback process of **common user processes**: low reliable memory of the local node -> low reliable memory of the remote node. - - Fallback process of **key user processes**: highly reliable memory of the local node -> highly reliable memory of the remote node. If no memory is allocated and the fallback function of `reliable` is enabled, the system retries as follows: low reliable memory of the local node -> low reliable memory of the remote node. - -**Scenarios** - -1. The default page size (`PAGE_SIZE`) is 4 KB. -2. The lower 4 GB memory of the NUMA node 0 must be highly reliable, and the highly reliable memory size and low reliable memory size must meet the kernel requirements. Otherwise, the system may fail to start. There is no requirement on the highly reliable memory size of other nodes. However, - if a node does not have highly reliable memory or the highly reliable memory is insufficient, the per-node management structure may be located in the highly reliable memory of other nodes (because the per-node management structure is a kernel data structure and needs to be located in the highly reliable memory zone). As a result, a kernel warning is generated, for example, `vmemmap_verify` alarms are generated and the performance is affected. -3. Some statistics (such as the total amount of highly reliable memory for TMPFS) of this feature are collected using the percpu technology, which causes extra overhead. To reduce the impact on performance, there is a certain error when calculating the sum. It is normal that the error is less than 10%. -4. Huge page limit: - - In the startup phase, static huge pages are low reliable memory. By default, static huge pages allocated during running are low reliable memory. If memory allocation occurs in the context of a key process, the allocated huge pages are highly reliable memory. - - In the transparent huge page (THP) scenario, if one of the 512 4 KB pages to be combined (2 MB for example) is a highly reliable page, the newly allocated 2 MB huge page uses highly reliable memory. That is, the THP uses more highly reliable memory. - - The allocation of the reserved 2 MB huge page complies with the native fallback process. If the current node lacks low reliable memory, the allocation falls back to the highly reliable range. - - In the startup phase, 2 MB huge pages are reserved. If no memory node is specified, the load is balanced to each memory node for huge page reservation. If a memory node lacks low reliable memory, highly reliable memory is used according to the native process. -5. Currently, only the normal system startup scenario is supported. In some abnormal scenarios, kernel startup may be incompatible with the memory tiering function, for example, the kdump startup phase. (Currently, kdump can be automatically disabled. In other scenarios, it needs to be disabled by upper-layer services.) -6. In the swap-in and swap-out, memory offline, KSM, cma, and gigantic page processes, the newly allocated page types are not considered based on the tiered-reliability memory. As a result, the page types may not be defined (for example, the highly reliable memory usage statistics are inaccurate and the reliability level of the allocated memory is not as expected). - -**Impact on Performance** - -- Due to the introduction of tiered-reliability memory management, the judgment logic is added for physical page allocation, which affects the performance. The impact depends on the system status, memory type, and high- and low reliable memory margin of each node. -- This feature introduces highly reliable memory usage statistics, which affects system performance. -- When `task_reliable_limit` is triggered, the cache in the highly reliable zone is reclaimed synchronously, which increases the CPU usage. In the scenario where `task_reliable_limit` is triggered by page cache allocation (file read/write operations, such as dd), if the available highly reliable memory (ReliableFileCache is considered as available memory) is close to 4 MB, cache reclamation is triggered more frequently. The overhead of direct cache reclamation is high, causing high CPU usage. In this case, the file read/write performance is close to the raw disk performance. \ No newline at end of file diff --git a/docs/en/docs/Kernel/tiered-reliability-memory-management-user-guide.md b/docs/en/docs/Kernel/tiered-reliability-memory-management-user-guide.md deleted file mode 100644 index 47598fab7930fbc361a22a9f391c8f24a219dd4d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kernel/tiered-reliability-memory-management-user-guide.md +++ /dev/null @@ -1,353 +0,0 @@ -# Tiered-Reliability Memory Management - -[1.1 Overview](#overview) - -[1.2 Restrictions](#restrictions) - -[1.3 How to Use](#how-to-use) - -## Overview - -This feature allows you to allocate memory with corresponding reliability as required and mitigates the impact of some possible UCEs or CEs to some extent. In this way, the overall service reliability does not deteriorate when partial memory mirroring (a RAS feature called address range mirroring) is used. - -## Restrictions - -This section describes the general constraints of this feature. Each subfeature has specific constraints, which are described in the corresponding section. - -### Compatibility - -1. Currently, this feature applies only to ARM64. -2. The hardware needs to support partial memory mirroring (address range mirroring), that is, the memory whose attribute is `EFI_MEMORY_MORE_RELIABLE` is reported through the UEFI standard API. Common memory does not need to be set. The mirrored memory is the highly reliable memory, and the common memory is the low reliable memory. -3. High-reliability and low reliable memory tiering is implemented by using the memory management zones of the kernel. They cannot dynamically flow (that is, pages cannot move between different zones). -4. Continuous physical memory with different reliability is divided into different memblocks. As a result, the allocation of large continuous physical memory blocks may be restricted after memory tiering is enabled. -5. To enable this feature, the value of `kernelcore` must be `reliable`, which is incompatible with other values of this parameter. - -### Design Specifications - -1. During kernel-mode development, pay attention to the following points when allocating memory: - - - If the memory allocation API supports the specified `gfp_flag`, only the memory allocation whose `gfp_flag` contains `__GFP_HIGHMEM` and `__GFP_MOVABLE` forcibly allocates the common memory range or redirects to the reliable memory range. Other `gfp_flags` do not intervene. - - - High-reliability memory is allocated from slab, slub, and slob. (If the memory allocated at a time is greater than `KMALLOC_MAX_CACHE_SIZE` and `gfp_flag` is set to a common memory range, low reliable memory may be allocated.) - -2. During user-mode development, pay attention to the following points when allocating memory: - - - After the attribute of a common process is changed to a key process, the highly reliable memory is used only in the actual physical memory allocation phase (page fault). The attribute of the previously allocated memory does not change, and vice versa. Therefore, the memory allocated when a common process is started and changed to a key process may not be highly reliable memory. Whether the configuration takes effect can be verified by querying whether the physical address corresponding to the virtual address belongs to the highly reliable memory range. - - Similar mechanisms (ptmalloc, tcmalloc, and dpdk) in libc libraries, such as chunks in glibc, use cache logic to improve performance. However, memory cache causes inconsistency between the memory allocation logics of the user and the kernel. When a common process becomes a key process, this feature cannot be enabled (it is enabled only when the kernel allocates memory). - -3. When an upper-layer service applies for memory, if the highly reliable memory is insufficient (triggering the native min waterline of the zone) or the corresponding limit is triggered, the page cache is preferentially released to attempt to reclaim the highly reliable memory. If the memory still cannot be allocated, the kernel selects OOM or fallback to the low reliable memory range based on the fallback switch to complete memory allocation. (Fallback indicates that when the memory of a memory management zone or node is insufficient, memory is allocated from other memory management zones or nodes.) - -4. The dynamic memory migration mechanism similar to `NUMA_BALANCING` may cause the allocated highly reliable or low reliable memory to be migrated to another node. Because the migration operation loses the memory allocation context and the target node may not have the corresponding reliable memory, the memory reliability after the migration may not be as expected. - -5. The following configuration files are introduced based on the usage of the user-mode highly reliable memory: - - - **/proc/sys/vm/task_reliable_limit**: upper limit of the highly reliable memory used by key processes (including systemd). It contains anonymous pages and file pages. The SHMEM used by the process is also counted (included in anonymous pages). - - - **/proc/sys/vm/reliable_pagecache_max_bytes**: soft upper limit of the highly reliable memory used by the global page cache. The number of highly reliable page caches used by common processes is limited. By default, the system does not limit the highly reliable memory used by page caches. This restriction does not apply to scenarios such as highly reliable processes and file system metadata. Regardless of whether fallback is enabled, when a common process triggers the upper limit, the low reliable memory is allocated by default. If the low reliable memory cannot be allocated, the native process is used. - - - **/proc/sys/vm/shmem_reliable_bytes_limit**: soft upper limit of the highly reliable memory used by the global SHMEM. It limits the amount of highly reliable memory used by the SHMEM of common processes. By default, the system does not limit the amount of highly reliable memory used by SHMEM. High-reliability processes are not subject to this restriction. When fallback is disabled, if a common process triggers the upper limit, memory allocation fails, but OOM does not occur (consistent with the native process). - - If the above limits are reached, memory allocation fallback or OOM may occur. - - Memory allocation caused by page faults generated by key processes in the TMPFS or page cache may trigger multiple limits. For details about the interaction between multiple limits, see the following table. - - | Whether task_reliable_limit Is Reached| Whether reliable_pagecache_max_bytes or shmem_reliable_bytes_limit Is Reached| Memory Allocation Processing Policy | - | --------------------------- | ------------------------------------------------------------ | ------------------------------------------------ | - | Yes | Yes | The page cache is reclaimed first to meet the allocation requirements. Otherwise, fallback or OOM occurs.| - | Yes | No | The page cache is reclaimed first to meet the allocation requirements. Otherwise, fallback or OOM occurs.| - | No | No | High-reliability memory is allocated first. Otherwise, fallback or OOM occurs. | - | No | Yes | High-reliability memory is allocated first. Otherwise, fallback or OOM occurs. | - - Key processes comply with `task_reliable_limit`. If `task_reliable_limit` is greater than `tmpfs` or `pagecachelimit`, page cache and TMPFS generated by key processes still use highly reliable memory. As a result, the highly reliable memory used by page cache and TMPFS is greater than the corresponding limit. - - When `task_reliable_limit` is triggered, if the size of the highly reliable file cache is less than 4 MB, the file cache will not be reclaimed synchronously. If the highly reliable file cache is less than 4 MB when the page cache is generated, the allocation will fall back to the low reliable memory range. If the highly reliable file cache is greater than 4 MB, the page cache is reclaimed preferentially for allocation. However, when the size is close to 4 MB, direct cache reclamation is triggered more frequently. Because the lock overhead of direct cache reclamation is high, the CPU usage is high. In this case, the file read/write performance is close to the raw disk performance. - -6. Even if the system has sufficient highly reliable memory, the allocation may fall back to the low reliable memory range. - - - If the memory cannot be migrated to another node for allocation, the allocation falls back to the low reliable memory range of the current node. The common scenarios are as follows: - - If the memory allocation contains `__GFP_THISNODE` (for example, transparent huge page allocation), memory can be allocated only from the current node. If the highly reliable memory of the node does not meet the allocation requirements, the system attempts to allocate memory from the low reliable memory range of the memory node. - - A process runs on a node that contains common memory by running commands such as `taskset` and `numactl`. - - A process is scheduled to a common memory node under the native scheduling mechanism of the system memory. - - High-reliability memory allocation triggers the highly reliable memory usage threshold, which also causes fallback to the low reliable memory range. - -7. If tiered-reliability memory fallback is disabled, highly reliable memory cannot be expanded to low reliable memory. As a result, user-mode applications may not be compatible with this feature in determining the memory usage, for example, determining the available memory based on MemFree. - -8. If tiered-reliability memory fallback is enabled, the native fallback is affected. The main difference lies in the selection of the memory management zone and NUMA node. - -- Fallback process of **common user processes**: low reliable memory of the local node -> low reliable memory of the remote node. -- Fallback process of **key user processes**: highly reliable memory of the local node -> highly reliable memory of the remote node. If no memory is allocated and the fallback function of `reliable` is enabled, the system retries as follows: low reliable memory of the local node -> low reliable memory of the remote node. - -### Scenarios - -1. The default page size (`PAGE_SIZE`) is 4 KB. -2. The lower 4 GB memory of the NUMA node 0 must be highly reliable, and the highly reliable memory size and low reliable memory size must meet the kernel requirements. Otherwise, the system may fail to start. There is no requirement on the highly reliable memory size of other nodes. However, - if a node does not have highly reliable memory or the highly reliable memory is insufficient, the per-node management structure may be located in the highly reliable memory of other nodes (because the per-node management structure is a kernel data structure and needs to be located in the highly reliable memory zone). As a result, a kernel warning is generated, for example, `vmemmap_verify` alarms are generated and the performance is affected. -3. Some statistics (such as the total amount of highly reliable memory for TMPFS) of this feature are collected using the percpu technology, which causes extra overhead. To reduce the impact on performance, there is a certain error when calculating the sum. It is normal that the error is less than 10%. -4. Huge page limit: - - In the startup phase, static huge pages are low reliable memory. By default, static huge pages allocated during running are low reliable memory. If memory allocation occurs in the context of a key process, the allocated huge pages are highly reliable memory. - - In the transparent huge page (THP) scenario, if one of the 512 4 KB pages to be combined (2 MB for example) is a highly reliable page, the newly allocated 2 MB huge page uses highly reliable memory. That is, the THP uses more highly reliable memory. - - The allocation of the reserved 2 MB huge page complies with the native fallback process. If the current node lacks low reliable memory, the allocation falls back to the highly reliable range. - - In the startup phase, 2 MB huge pages are reserved. If no memory node is specified, the load is balanced to each memory node for huge page reservation. If a memory node lacks low reliable memory, highly reliable memory is used according to the native process. -5. Currently, only the normal system startup scenario is supported. In some abnormal scenarios, kernel startup may be incompatible with the memory tiering function, for example, the kdump startup phase. (Currently, kdump can be automatically disabled. In other scenarios, it needs to be disabled by upper-layer services.) -6. In the swap-in and swap-out, memory offline, KSM, cma, and gigantic page processes, the newly allocated page types are not considered based on the tiered-reliability memory. As a result, the page types may not be defined (for example, the highly reliable memory usage statistics are inaccurate and the reliability level of the allocated memory is not as expected). - -### Impact on Performance - -- Due to the introduction of tiered-reliability memory management, the judgment logic is added for physical page allocation, which affects the performance. The impact depends on the system status, memory type, and high- and low reliable memory margin of each node. -- This feature introduces highly reliable memory usage statistics, which affects system performance. -- When `task_reliable_limit` is triggered, the cache in the highly reliable zone is reclaimed synchronously, which increases the CPU usage. In the scenario where `task_reliable_limit` is triggered by page cache allocation (file read/write operations, such as dd), if the available highly reliable memory (ReliableFileCache is considered as available memory) is close to 4 MB, cache reclamation is triggered more frequently. The overhead of direct cache reclamation is high, causing high CPU usage. In this case, the file read/write performance is close to the raw disk performance. - -## How to Use - -### Tiered-Reliability Memory Management for the OS - -#### Overview - -Memory is divided into two ranges based on high and low reliability. Therefore, memory allocation and release must be managed separately based on the reliability. The OS must be able to control the memory allocation path. User-mode processes use low reliable memory, and kernel-mode processes use highly reliable memory. When the highly reliable memory is insufficient, the allocation needs to fall back to the low reliable memory range or the allocation fails. - -In addition, according to the reliability requirements and types of processes, on-demand allocation of highly reliable and low reliable memory is required. For example, specify highly reliable memory for key processes to reduce the probability of memory errors encountered by key processes. Currently, the kernel uses highly reliable memory, and user-mode processes use low reliable memory. As a result, some key or core services, such as the service forwarding process, are unstable. If an exception occurs, I/Os are interrupted, affecting service stability. Therefore, these key services must use highly reliable memory to improve the stability of key processes. - -When a memory error occurs in the system, the OS overwrites the unallocated low reliable memory to clear the undetected memory error. - -#### Restrictions - -- **High-reliability memory for key processes** - - 1. The abuse of the `/proc//reliable` API may cause excessive use of highly reliable memory. - 2. The `reliable` attribute of a user-mode process can be modified by using the proc API or directly inherited from its parent process only after the process is started. `systemd (pid=1)` uses highly reliable memory. Its `reliable` attribute is useless and is not inherited. The `reliable` attribute of kernel-mode threads is invalid. - 3. The program and data segments of processes use highly reliable memory. Because the highly reliable memory is insufficient, the low reliable memory is used for startup. - 4. Common processes also use highly reliable memory in some scenarios, such as HugeTLB, page cache, vDSO, and TMPFS. - -- **Overwrite of unallocated memory** - - The overwrite of the unallocated memory can be executed only once and does not support concurrent operations. If this feature is executed, it will have the following impacts: - - 1. This feature takes a long time. When one CPU of each node is occupied by the overwrite thread, other tasks cannot be scheduled on the CPU. - 2. During the overwrite process, the zone lock needs to be obtained. Other service processes need to wait until the overwrite is complete. As a result, the memory may not be allocated in time. - 3. In the case of concurrent execution, queuing is blocked, resulting in a longer delay. - - If the machine performance is poor, the kernel RCU stall or soft lockup alarm may be triggered, and the process memory allocation may be blocked. Therefore, this feature can be used only on physical machines if necessary. There is a high probability that the preceding problem occurs on VMs. - - The following table lists the reference data of physical machines. (The actual time required depends on the hardware performance and system load.) - -Table 1 Test data when the TaiShan 2280 V2 server is unloaded - -| Test Item | Node 0 | Node 1 | Node 2 | Node 3 | -| ------------- | ------ | ------ | ------ | ------ | -| Free Mem (MB) | 109290 | 81218 | 107365 | 112053 | - -The total time is 3.2s. - -#### Usage - -This sub-feature provides multiple APIs. You only need to perform steps 1 to 6 to enable and verify the sub-feature. - -1. Configure `kernelcore=reliable` to enable tiered-reliability memory management. `CONFIG_MEMORY_RELIABLE` is mandatory. Otherwise, tiered-reliability memory management is disabled for the entire system. - -2. You can use the startup parameter `reliable_debug=[F][,S][,P]` to disable the fallback function (`F`), disable the TMPFS to use highly reliable memory (`S`), or disable the read/write cache to use highly reliable memory (`P`). By default, all the preceding functions are enabled. - -3. Based on the address range reported by the BIOS, the system searches for and marks the highly reliable memory. For the NUMA system, not every node needs to reserve reliable memory. However, the lower 4 GB physical space on node 0 must be highly reliable memory. During the system startup, the system allocates memory. If the highly reliable memory cannot be allocated, the low reliable memory is allocated (based on the fallback logic of the mirroring function) or the system cannot be started. If low reliable memory is used, the entire system is unstable. Therefore, the highly reliable memory on node 0 must be retained and the lower 4 GB physical space must be highly reliable memory. - -4. After the startup, you can check whether memory tiering is enabled based on the startup log. If it is enabled, the following information is displayed: - - ```shell - mem reliable: init succeed, mirrored memory - ``` - -5. The physical address range corresponding to the highly reliable memory can be queried in the startup log. Observe the attributes in the memory map reported by the EFI. The memory range with `MR` is the highly reliable memory range. The following is an excerpt of the startup log. The memory range `mem06` is the highly reliable memory, and `mem07` is the low reliable memory. Their physical address ranges are also listed (the highly and low reliable memory address ranges cannot be directly queried in other modes). - - ```text - [ 0.000000] efi: mem06: [Conventional Memory| |MR| | | | | | |WB| | | ] range=[0x0000000100000000-0x000000013fffffff] (1024MB) - [ 0.000000] efi: mem07: [Conventional Memory| | | | | | | | |WB| | | ] range=[0x0000000140000000-0x000000083eb6cfff] (28651MB) - ``` - -6. During kernel-mode development, a page struct page can be determined based on the zone where the page is located. `ZONE_MOVABLE` indicates a low reliable memory zone. If the zone ID is smaller than `ZONE_MOVABLE`, the zone is a highly reliable memory zone. The following is an example: - - ```c - bool page_reliable(struct page *page) - { - if (!mem_reliable_status() || !page) - return false; - return page_zonenum(page) < ZONE_MOVABLE; - } - ``` - - In addition, the provided APIs are classified based on their functions: - - 1. **Checking whether the reliability function is enabled at the code layer**: In the kernel module, use the following API to check whether the tiered-reliability memory management function is enabled. If `true` is returned, the function is enabled. If `false` is returned, the function is disabled. - - ```c - #include - bool mem_reliable_status(void); - ``` - - 2. **Memory hot swap**: If the kernel enables the memory hot swap operation (Logical Memory hot-add), the highly and low reliable memories also support this operation. The operation unit is the memory block, which is the same as the native process. - - ```shell - # Bring the memory online to the highly reliable memory range. - echo online_kernel > /sys/devices/system/memory/auto_online_blocks - # Bring the memory online to the low reliable memory range. - echo online_movable > /sys/devices/system/memory/auto_online_blocks - ``` - - 3. **Dynamically disabling a tiered management function**: The long type is used to determine whether to enable or disable the tiered-reliability memory management function based on each bit. - - - `bit0`: enables tiered-reliability memory management. - - `bit1`: disables fallback to the low reliable memory range. - - `bit2`: disables TMPFS to use highly reliable memory. - - `bit3`: disables the page cache to use highly reliable memory. - - Other bits are reserved for extension. If you need to change the value, call the following proc API (the permission is 600). The value range is 0-15. (The subsequent functions are processed only when `bit 0` of the general function is `1`. Otherwise, all functions are disabled.) - - ```shell - echo 15 > /proc/sys/vm/reliable_debug - # All functions are disabled because bit0 is 0. - echo 14 > /proc/sys/vm/reliable_debug - ``` - - This command can only be used to disable the function. This command cannot be used to enable a function that has been disabled or is disabled during running. - - Note: This function is used for escape and is configured only when the tiered-reliability memory management feature needs to be disabled in abnormal scenarios or during commissioning. Do not use this function as a common function. - - 4. **Viewing highly reliable memory statistics**: Call the native `/proc/meminfo` API. - - - `ReliableTotal`: total size of reliable memory managed by the kernel. - - `ReliableUsed`: total size of reliable memory used by the system, including the reserved memory used in the system. - - `ReliableBuddyMem`: remaining reliable memory of the partner system. - - `ReliableTaskUsed`: highly reliable memory used by systemd and key user processes, including anonymous pages and file pages. - - `ReliableShmem`: highly reliable memory usage of the shared memory, including the total highly reliable memory used by the shared memory, TMPFS, and rootfs. - - `ReliableFileCache`: highly reliable memory usage of the read/write cache. - - 5. **Overwrite of unallocated memory**: This function requires the configuration item to be enabled. - - Enable `CONFIG_CLEAR_FREELIST_PAGE` and add the startup parameter `clear_freelist`. Call the proc API. The value can only be `1` (the permission is 0200). - - ```shell - echo 1 > /proc/sys/vm/clear_freelist_pages - ``` - - Note: This feature depends on the startup parameter `clear_freelist`. The kernel matches only the prefix of the startup parameter. Therefore, this feature also takes effect for parameters with misspelled suffix, such as `clear_freelisttt`. - - To prevent misoperations, add the kernel module parameter `cfp_timeout_ms` to indicate the maximum execution duration of the overwrite function. If the overwrite function times out, the system exits even if the overwrite operation is not complete. The default value is `2000` ms (the permission is 0644). - - ```shell - echo 500 > /sys/module/clear_freelist_page/parameters/cfp_timeout_ms # Set the timeout to 500 ms. - ``` - - 6. **Checking and modifying the high and low reliability attribute of the current process**: Call the `/proc//reliable` API to check whether the process is a highly reliable process. If the process is running and written, the attribute is inherited. If the subprocess does not require the attribute, manually modify the subprocess attribute. The systemd and kernel threads do not support the read and write of the attribute. The value can be `0` or `1`. The default value is `0`, indicating a low reliable process (the permission is 0644). - - ```shell - # Change the process whose PID is 1024 to a highly reliable process. After the change, the process applies for memory from the highly reliable memory range. If the memory fails to be allocated, the allocation may fall back to the low reliable memory range. - echo 1 > /proc/1024/reliable - ``` - - 7. **Setting the upper limit of highly reliable memory requested by user-mode processes**: Call `/proc/sys/vm/task_reliable_limit` to modify the upper limit of highly reliable memory requested by user-mode processes. The value range is \[`ReliableTaskUsed`, `ReliableTotal`], and the unit is byte (the permission is 0644). Notes: - - - The default value is `ulong_max`, indicating that there is no limit. - - If the value is `0`, the reliable process cannot use the highly reliable memory. In fallback mode, the allocation falls back to the low reliable memory range. Otherwise, OOM occurs. - - If the value is not `0` and the limit is triggered, the fallback function is enabled. The allocation falls back to the low reliable memory range. If the fallback function is disabled, OOM is returned. - -### Highly Reliable Memory for Read and Write Cache - -#### Overview - -A page cache is also called a file cache. When Linux reads or writes files, the page cache is used to cache the logical content of the files to accelerate the access to images and data on disks. If low reliable memory is allocated to page caches, UCE may be triggered during the access, causing system exceptions. Therefore, the read/write cache (page cache) needs to be placed in the highly reliable memory zone. In addition, to prevent the highly reliable memory from being exhausted due to excessive page cache allocations (unlimited by default), the total number of page caches and the total amount of reliable memory need to be limited. - -#### Restrictions - -1. When the page cache exceeds the limit, it is reclaimed periodically. If the generation speed of the page cache is faster than the reclamation speed, the number of page caches may be higher than the specified limit. -2. The usage of `/proc/sys/vm/reliable_pagecache_max_bytes` has certain restrictions. In some scenarios, the page cache forcibly uses reliable memory. For example, when metadata (such as inode and dentry) of the file system is read, the reliable memory used by the page cache exceeds the API limit. In this case, you can run `echo 2 \> /proc/sys/vm/drop_caches` to release inode and dentry. -3. When the highly reliable memory used by the page cache exceeds the `reliable_pagecache_max_bytes` limit, the low reliable memory is allocated by default. If the low reliable memory cannot be allocated, the native process is used. -4. FileCache statistics are first collected in the percpu cache. When the value in the cache exceeds the threshold, the cache is added to the entire system and then displayed in `/proc/meminfo`. `ReliableFileCache` does not have the preceding threshold in `/proc/meminfo`. As a result, the value of `ReliableFileCache` may be greater than that of `FileCache`. -5. Write cache scenarios are restricted by `dirty_limit` (restricted by /`proc/sys/vm/dirty_ratio`, indicating the percentage of dirty pages on a single memory node). If the threshold is exceeded, the current zone is skipped. For tiered-reliability memory, because highly and low reliable memories are in different zones, the write cache may trigger fallback of the local node and use the low reliable memory of the local node. You can run `echo 100 > /proc/sys/vm/dirty_ratio` to cancel the restriction. -6. The highly reliable memory feature for the read/write cache limits the page cache usage. The system performance is affected in the following scenarios: - - If the upper limit of the page cache is too small, the I/O increases and the system performance is affected. - - If the page cache is reclaimed too frequently, system freezing may occur. - - If a large amount of page cache is reclaimed each time after the page cache exceeds the limit, system freezing may occur. - -#### Usage - -The highly reliable memory is enabled by default for the read/write cache. To disable the highly reliable memory, configure `reliable_debug=P`. In addition, the page cache cannot be used unlimitedly. The function of limiting the page cache size depends on the `CONFIG_SHRINK_PAGECACHE` configuration item. - -`FileCache` in `/proc/meminfo` can be used to query the usage of the page cache, and `ReliableFileCache` can be used to query the usage of the reliable memory in the page cache. - -The function of limiting the page cache size depends on several proc APIs, which are defined in `/proc/sys/vm/` to control the page cache usage. For details, see the following table. - -| API Name (Native/New) | Permission| Description | Default Value | -| ------------------------------------ | ---- | ------------------------------------------------------------ | ------------------------------------------ | -| `cache_reclaim_enable` (native) | 644 | Whether to enable the page cache restriction function.
Value range: `0` or `1`. If an invalid value is input, an error is returned.
Example: `echo 1 > cache_reclaim_enable`| 1 | -| `cache_limit_mbytes` (new) | 644 | Upper limit of the cache, in MB.
Value range: The minimum value is 0, indicating that the restriction function is disabled. The maximum value is the memory size in MB, for example, the value displayed by running the `free –m` command (the value of `MemTotal` in `meminfo` converted in MB).
Example: `echo 1024 \> cache_limit_mbytes`
Others: It is recommended that the cache upper limit be greater than or equal to half of the total memory. Otherwise, the I/O performance may be affected if the cache is too small.| 0 | -| `cache_reclaim_s` (native) | 644 | Interval for triggering cache reclamation, in seconds. The system creates work queues based on the number of online CPUs. If there are *n* CPUs, the system creates *n* work queues. Each work queue performs reclamation every `cache_reclaim_s` seconds. This parameter is compatible with the CPU online and offline functions. If the CPU is offline, the number of work queues decreases. If the CPU is online, the number of work queues increases.
Value range: The minimum value is `0` (indicating that the periodic reclamation function is disabled) and the maximum value is `43200`. If an invalid value is input, an error is returned.
Example: `echo 120 \> cache_reclaim_s`
Others: You are advised to set the reclamation interval to several minutes (for example, 2 minutes). Otherwise, frequent reclamation may cause system freezing.| 0 | -| `cache_reclaim_weight` (native) | 644 | Weight of each reclamation. Each CPU of the kernel expects to reclaim `32 x cache_reclaim_weight` pages each time. This weight applies to both reclamation triggered by the page upper limit and periodic page cache reclamation.
Value range: 1 to 100. If an invalid value is input, an error is returned.
Example: `echo 10 \> cache_reclaim_weight`
Others: You are advised to set this parameter to `10` or a smaller value. Otherwise, the system may freeze each time too much memory is reclaimed.| 1 | -| `reliable_pagecache_max_bytes` (new)| 644 | Total amount of highly reliable memory in the page cache.
Value range: 0 to the maximum highly reliable memory, in bytes. You can call `/proc/meminfo` to query the maximum highly reliable memory. If an invalid value is input, an error is returned.
Example: `echo 4096000 \> reliable_pagecache_max_bytes`| Maximum value of the unsigned long type, indicating that the usage is not limited.| - -### Highly Reliable Memory for TMPFS - -#### Overview - -If TMPFS is used as rootfs, it stores core files and data used by the OS. However, TMPFS uses low reliable memory by default, which makes core files and data unreliable. Therefore, TMPFS must use highly reliable memory. - -#### Usage - -By default, the highly reliable memory is enabled for TMPFS. To disable it, configure `reliable_debug=S`. You can dynamically disable it by calling `/proc/sys/vm/reliable_debug`, but cannot dynamically enable it. - -When enabling TMPFS to use highly reliable memory, you can check `ReliableShmem` in `/proc/meminfo` to view the highly reliable memory that has been used by TMPFS. - -By default, the upper limit for TMPFS to use highly reliable memory is half of the physical memory (except when TMPFS is used as rootfs). The conventional SYS V shared memory is restricted by `/proc/sys/kernel/shmmax and /proc/sys/kernel/shmall` and can be dynamically configured. It is also restricted by the highly reliable memory used by TMPFS. For details, see the following table. - -| **Parameter** | **Description** | -|---------------------------------|--------------------------------| -| `/proc/sys/kernel/shmmax` (native)| Size of a single SYS V shared memory range.| -| `/proc/sys/kernel/shmall` (native)| Total size of the SYS V shared memory that can be used. | - -The `/proc/sys/vm/shmem_reliable_bytes_limit` API is added for you to set the available highly reliable size (in bytes) of the system-level TMPFS. The default value is `LONG_MAX`, indicating that the usage is not limited. The value ranges from 0 to the total reliable memory size of the system. The permission is 644. When fallback is disabled and the memory usage reaches the upper limit, an error indicating that no memory is available is returned. When fallback is enabled, the system attempts to allocate memory from the low reliable memory zone. Example: - -```shell -echo 10000000 > /proc/sys/vm/shmem_reliable_bytes_limit -``` - -### UCE Does Not Reset After the Switch from the User Mode to Kernel Mode - -#### Overview - -Based on the tiered-reliability memory management solution, the kernel and key processes use highly reliable memory. Most user-mode processes use low reliable memory. When the system is running, a large amount of data needs to be exchanged between the user mode and kernel mode. When data is transferred to the kernel mode, data in the low reliable memory zone is copied to the highly reliable memory zone. The copy operation is performed in kernel mode. If a UCE occurs when the user-mode data is read, that is, the kernel-mode memory consumption UCE occurs, the system triggers a panic. This sub-feature provides solutions for scenarios where UCEs occurred in the switch from the user mode to kernel mode to avoid system reset, including copy-on-write (COW), copy_to_user, copy_from_user, get_user, put_user, and core dump scenarios. Other scenarios are not supported. - -#### Restrictions - -1. ARMv8.2 or later that supports the RAS feature. -2. This feature changes the synchronization exception handling policy. Therefore, this feature takes effect only when the kernel receives a synchronization exception reported by the firmware. -3. The kernel processing depends on the error type reported by the BIOS. The kernel cannot process fatal hardware errors but can process recoverable hardware errors. -4. Only the COW, copy_to_user (including the read page cache), copy_from_user, get_user, put_user, and core dump scenarios are supported. -5. In the core dump scenario, UCE tolerance needs to be implemented on the write API of the file system. This feature supports only three common file systems: ext4, TMPFS, and PipeFS. The corresponding error tolerance APIs are as follows: - - PipeFS: `copy_page_from_iter` - - ext4/TMPFS: `iov_iter_copy_from_user_atomic` - -#### Usage - -Ensure that `CONFIG_ARCH_HAS_COPY_MC` is enabled in the kernel. If `/proc/sys/kernel/machine_check_safe` is set to `1`, this feature is enabled for all scenarios. If `/proc/sys/kernel/machine_check_safe` is set to `0`, this feature is disabled for all scenarios. Other values are invalid. - -The fault tolerance mechanism in each scenario is as follows: - -| **No.**| **Scenario** | **Symptom** | **Mitigation Measure** | -| ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| 1 | `copy_from/to_user`: basic switch to the user mode, involving syscall, sysctl, and procfs operations| If a UCE occurs during the copy, the kernel is reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset. | -| 2 | `get/put_user`: simple variable copy, mainly in netlink scenarios.| If a UCE occurs during the copy, the kernel is reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset. | -| 3 | COW: fork subprocess, which triggers COW. | COW is triggered. If a UCE occurs, the kernel is reset. | If a UCE occurs, kill related processes. The kernel does not automatically reset.| -| 4 | Read cache: The user mode uses low reliable memory. When a user-mode program reads or writes files, the OS uses idle memory to cache hard disk files, improving performance. However, when the user-mode program reads a file, the kernel accesses the cache first.| A UCE occurs, causing the kernel to reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset.| -| 5 | UCE is triggered by memory access during a core dump. | A UCE occurs, causing the kernel to reset. | If a UCE occurs, kill the current process. The kernel does not automatically reset. | -| 6 | Write cache: When the write cache is flushed back to the disk, a UCE is triggered. | Cache flushing is actually disk DMA data migration. If a UCE is triggered during this process, page write fails after timeout. As a result, data inconsistency occurs and the file system becomes unavailable. If the data is key data, the kernel resets.| No solution is available. The kernel will be reset. | -| 7 | Kernel startup parameters and module parameters use highly reliable memory. | / | Not supported. The risk is reduced. | -| 8 | relayfs: a file system that quickly forwards data from the kernel mode to the user mode.| / | Not supported. The risk is reduced. | -| 9 | `seq_file`: transfers kernel data to the user mode as a file. | / | Not supported. The risk is reduced. | - -Most user-mode data uses low reliable memory. Therefore, this project involves only the scenario where user-mode data is read in kernel mode. In Linux, data can be exchanged between the user space and kernel space in nine modes, including kernel startup parameters, module parameters, sysfs, sysctl, syscall (system call), netlink, procfs, seq_file, debugfs, and relayfs. There are two other cases: COW and read/write file cache (page cache) when a process is created. - -In sysfs, syscall, netlink, and procfs modes, data is transferred from the user mode to the kernel mode in copy_from_user or get_user mode. - -The user mode can be switched to the kernel mode in the following scenarios: - -copy_from_user, get_user, COW, read cache, and write cache flushing. - -The kernel mode can be switched to the user mode in the following scenarios: - -relayfs, seq_file, copy_to_user, and put_user diff --git a/docs/en/docs/KernelLiveUpgrade/KernelLiveUpgrade.md b/docs/en/docs/KernelLiveUpgrade/KernelLiveUpgrade.md deleted file mode 100644 index 401fae920ffdd7d4b27361040483b6dfcfaa9559..0000000000000000000000000000000000000000 --- a/docs/en/docs/KernelLiveUpgrade/KernelLiveUpgrade.md +++ /dev/null @@ -1 +0,0 @@ -# Kernel Hot Upgrade Guide This document describes how to install, deploy, and use the kernel hot upgrade feature on openEuler. This kernel hot upgrade feature on openEuler is implemented through quick kernel restart and hot program migration. A user-mode tool is provided to automate this process. This document is intended for community developers, open-source enthusiasts, and partners who want to learn about and use the openEuler system and kernel hot upgrade. The users are expected to know basics about the Linux operating system. ## Application Scenario The kernel hot upgrade is to save and restore the process running data with the second-level end-to-end latency. The following two conditions must be met: 1. The kernel needs to be restarted due to vulnerability fixing or version update. 2. Services running on the kernel can be quickly recovered after the kernel is restarted. \ No newline at end of file diff --git a/docs/en/docs/KernelLiveUpgrade/common-problems-and-solutions.md b/docs/en/docs/KernelLiveUpgrade/common-problems-and-solutions.md deleted file mode 100644 index a529bf7d454c91ecd9fcd7090feb075b000cc5fc..0000000000000000000000000000000000000000 --- a/docs/en/docs/KernelLiveUpgrade/common-problems-and-solutions.md +++ /dev/null @@ -1,29 +0,0 @@ -# Common Problems and Solutions - -1. After the `nvwa update` command is executed, the system is not upgraded. - - Cause: An error occurs when the running information is retained or the kernel is replaced. - - Solution: View logs to find the error cause. - -2. After the acceleration feature is enabled, the `nvwa` command fails to be executed. - - Cause: NVWA provides many acceleration features, including quick kexec, pin memory, and cpu park. These features involve the cmdline configuration and memory allocation. When selecting the memory, run cat /proc/iomemory to ensure that the selected memory does not conflict with that of other programs. If necessary, run the dmesg command to check whether error logs exist after the feature is enabled. - -3. After the hot upgrade, the related process is not recovered. - - Cause: Check whether the nvwa service is running. If the nvwa service is running, the service or process may fail to be recovered. - - Solution: Run the service `nvwa status` command to view the NVWA logs. If the service fails to be started, check whether the service is enabled, and then run the `systemd` command to view the logs of the corresponding service. Further logs are stored in the process or service folder named after the path specified by **criu_dir**. The dump.log file stores the logs generated when the running information is retained, and the restore.log file restores the logs generated for process recovery. - -4. The recovery fails, and the log displays "Can't fork for 948: File exists." - - Cause: The kernel hot upgrade tool finds that the PID of the program is occupied during program recovery. - - Solution: The current kernel does not provide a mechanism for retaining PIDs. Related policies are being developed. This restriction will be resolved in later kernel versions. Currently, you can only manually restart related processes. - -5. When the `nvwa` command is used to save and recover a simple program (hello world), the system displays a message indicating that the operation fails or the program is not running. - - Cause: There are many restrictions on the use of CRIU. - - Solution: View the NVWA logs. If the error is related to the CRIU, check the dump.log or restore.log file in the corresponding directory. For details about the usage restrictions related to the CRIU, see [CRIU WiKi](https://criu.org/What_cannot_be_checkpointed). diff --git a/docs/en/docs/KernelLiveUpgrade/how-to-run.md b/docs/en/docs/KernelLiveUpgrade/how-to-run.md deleted file mode 100644 index c37c3710214cf45e452fc574591613ca611784c9..0000000000000000000000000000000000000000 --- a/docs/en/docs/KernelLiveUpgrade/how-to-run.md +++ /dev/null @@ -1,113 +0,0 @@ -# How to Run - - - -- [How to Run](#how-to-run) - - [Command](#command) - - [Restrictions](#restrictions) - - [NVWA Acceleration Feature Description and Usage](#nvwa-acceleration-feature-description-and-usage) - - [Generated Log Information](#generated-log-information) - - - -## Command - -- `nvwa help` - - Prints the help information. The printed information is as follows: - - ```text - NAME: - nvwa - a tool used for openEuler kernel update. - - USAGE: - nvwa [global options] command [command options] [arguments...] - - VERSION: - 0.1 - - COMMANDS: - update specify kernel version for nvwa to update - init init nvwa running environment - help, h Shows a list of commands or help for one command - - GLOBAL OPTIONS: - --help, -h show help (default: false) - --version, -v print the version (default: false) - ``` - - Note that global options such as `nvwa --help` and `nvwa --version` are not supported due to parameter parsing restrictions. This problem will be resolved in later versions. - To view the NVWA version installed in the system, run the `rpm -qa nvwa` command. - -- `nvwa update ` - - When the kernel is hot upgraded to a version, the NVWA searches for the kernel image and ramfs in the /boot directory. The kernel must be named in the vmlinuz-\ format, and rootfs in the initramfs-\.img format. - - Note that the upgrade may fail. If the upgrade fails, some processes or services that are dumped will stop running. - - You can run the `uname -r` command to obtain the \ of the current version. You can find all existing versions in the /boot directory by referring to the format of the \. - -- `nvwa init` - - Clears the running information generated by NVWA and modifies the systemd configuration. This command is used to clear the running information before the NVWA is executed or after the execution fails. - -## Restrictions - -1. For services that need to be saved using NVWA, you need to set StandardOutput and StandardError in the configuration. The following uses Redis as an example: - - ```conf - [Unit] - Description=Redis persistent key-value database - After=network.target - [Service] - ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd - Type=notify - User=redis - Group=redis - RuntimeDirectory=redis - RuntimeDirectoryMode=0755 - StandardOutput=file:/root/log1.log - StandardError=file:/root/log2.log - [Install] - WantedBy=multi-user.target - ``` - -2. To use the acceleration feature, you need to modify the cmdline and allocate proper memory. For details, see [NVWA Acceleration Feature Description and Usage](#nvwa-acceleration-feature-description-and-usage). - -3. SELINUX needs to be disabled during the running process. - - Theoretically, you need to disable the NVWA service only after you run the NVWA update command and before you restart the system to restore the process. It is recommended that SELinux be disabled during the entire process. - -## NVWA Acceleration Feature Description and Usage - -1. cpu park - - The cpu park command uses the kexec process to make the CPU stay busy waiting, so as to respond to the interrupt request sent by the primary core more quickly, and reduce the status changes. - - To use cpu park, you need to add "cpuparkmem=0x200000000" to cmdline. 0x200000000 is the start address of the memory that is not used by other programs. cpuparkmem occupies the memory space whose size is about 1 MB from this address. - - Note that if the memory is sufficient, it is recommended that the address range be after 4G(0x100000000). The first 4 GB is usually reserved by each system component, which is prone to conflict. - -2. quick kexec - - quick kexec accelerates image loading using kexec. - - To use quick kexec, you need to enable related options in the configuration file. For more information, see [Configurations](./installation-and-deployment.md#configurations). - -3. pin_memory - - pin memory accelerates the storage and recovery of the CRIU. - - To use pin memory, you need to enable related options in the configuration file. For more information, see [Configurations](./installation-and-deployment.md#configurations). - -## Generated Log Information - -The logs generated by the kernel hot upgrade tool consist of two parts: - -- Logs generated during running - - Run the service `nvwa status` command to view logs. - -- Logs generated while retaining the running information - - The logs are stored in the process/service folder in the path specified by **criu_dir**. diff --git a/docs/en/docs/KernelLiveUpgrade/installation-and-deployment.md b/docs/en/docs/KernelLiveUpgrade/installation-and-deployment.md deleted file mode 100644 index c8289dab41f23d0d9047f062e442b416f1c5bbab..0000000000000000000000000000000000000000 --- a/docs/en/docs/KernelLiveUpgrade/installation-and-deployment.md +++ /dev/null @@ -1,180 +0,0 @@ -# Installation and Deployment - -This document describes how to install and deploy the kernel hot upgrade tool. - - - -- [Installation and Deployment](#installation-and-deployment) - - [Hardware and Software Requirements](#hardware-and-software-requirements) - - [Hardware Requirements](#hardware-requirements) - - [Software Requirements](#software-requirements) - - [Environment Preparation](#environment-preparation) - - [Installing the Kernel Hot Upgrade Tool](#installing-the-kernel-hot-upgrade-tool) - - [Deploying the Kernel Hot Upgrade Tool](#deploying-the-kernel-hot-upgrade-tool) - - [Configurations](#configurations) - - [Enabling the Kernel Hot Upgrade Tool](#enabling-the-kernel-hot-upgrade-tool) - - - -## Hardware and Software Requirements - -### Hardware Requirements - -- Currently, only the ARM64 architecture is supported. - -### Software Requirements - -- Operating system: openEuler 22.03 LTS - -## Environment Preparation - -- Install the openEuler system. For details, see the [_openEuler Installation Guide_](../Installation/Installation.md). - -- The root permission is required for installing the kernel hot upgrade tool. - -## Installing the Kernel Hot Upgrade Tool - -This section describes how to install the kernel live upgrade tool. - -Perform the following steps: - -1. Mount the ISO file of openEuler. - - ```shell - mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /mnt - ``` - -2. Configure the local yum repository. - - ```shell - vim /etc/yum.repos.d/local.repo - ``` - - The configurations are as follows: - - ```conf - [local] - name=local - baseurl=file:///mnt - gpgcheck=1 - enabled=1 - ``` - -3. Import the GPG public key of the RPM digital signature to the system. - - ```shell - rpm --import /mnt/RPM-GPG-KEY-openEuler - ``` - -4. Install the kernel hot upgrade tool. - - ```shell - yum install nvwa -y - ``` - -5. Check whether the installation is successful. If the command output is as follows, the installation is successful. - - ```shell - $ rpm -qa | grep nvwa - nvwa-xxx - ``` - -## Deploying the Kernel Hot Upgrade Tool - -This section describes how to configure and deploy the kernel hot upgrade tool. - -### Configurations - -The configuration files of the kernel hot upgrade tool are stored in **/etc/nvwa**. The configuration files are as follows: - -- nvwa-restore.yaml - - This configuration file is used to instruct the kernel hot upgrade tool to save and recover the process during the kernel hot upgrade. The configuration is as follows: - - - pids - - Specifies the processes that need to be retained and recovered during the NVWA hot upgrade. The processes are identified by process ID (PID). Note that the processes managed by NVWA are automatically recovered after the NVWA service is started. - - - services - - Specifies the services that need to be retained and recovered during NVWA hot upgrade. Compared to PIDs, the kernel hot upgrade tool can directly save and recover the process. For services, the kernel hot upgrade tool depends on the systemd to perform related operations. The service name must be the same as the service name used in systemd. Note that whether the service managed by NVWA needs to be automatically recovered when the NVWA is started depends on whether the service is enabled in the systemd. Currently, only the notify and oneshot service types are supported. - - - restore_net - - Specifies whether the kernel hot upgrade tool is required to save and recover the network configuration. If the network configuration is incorrect, the network may be unavailable after the recovery. This function is disabled by default. - - - enable_quick_kexec - - Used to specify whether to enable the quick kexec feature. quick kexec is a feature launched by the NVWA community to accelerate the kernel restart process. To use this feature, add "quickkexec=128M" to cmdline. 128 indicates the size of the memory allocated to the quick kexec feature. The memory is used to load the kernel and initramfs during the upgrade. Therefore, the size must be greater than the total size of the kernel and initramfs involved in the upgrade. This feature is disabled by default. - - - enable_pin_memory - - Used to specify whether to enable the pin memory feature. pin memory is a feature launched by the NVWA community to accelerate the process storage and recovery process. The pin_memory feature is not supported for multi-process recovery. To use the feature, you need to add "max_pin_pid_num=10 redirect_space_size=2M pinmemory=200M@0x640000000" to cmdline. - - max_pin_pid_num indicates the maximum number of processes that support pin memory recovery. redirect_space_size indicates the reserved memory space required for redirecting physical pages during pin memory recovery. You are advised to set redirect_space_size to 1/100 of the total reserved pin memory. pinmemory indicates the start point and size of the memory segment. The 200 MB space starting from 0x640000000 is the total memory space used by the pin memory. This space should not be used by other programs. - -- Configuration example of **nvwa-restore.yaml** - -```yaml -pids: - - 14109 -services: - - redis -restore_net: false -enable_quick_kexec: true -enable_pin_memory: true -``` - -- **nvwa-server.yaml** - - This file contains the configuration information required during the running of the kernel hot upgrade tool. The details are as follows: - - - criu_dir - - This parameter specifies the directory for storing the information generated when the kernel hot upgrade tool saves the running information. Note that the information may occupy a large amount of disk space. - - - criu_exe - - This parameter specifies the path of the CRIU executable file used by the kernel hot upgrade tool. You are advised not to change the path unless you need to debug the CRIU. - - - kexec_exe - - This parameter specifies the path of the kexec executable file used by the kernel hot upgrade tool. You are advised not to change the path unless you need to debug kexec. - - - systemd_etc - - This parameter specifies the path of the folder used to overwrite the systemd configuration file. The path is determined by the systemd. Generally, you do not need to change the path. - - - log_dir - - This parameter stores the log information generated by the kernel hot upgrade tool. The log module is not enabled currently. For details about how to view logs of the kernel hot upgrade tool, see How to Run. - -- Configuration example of **nvwa-server.yaml** - -```yaml -criu_dir: /var/nvwa/running/ -criu_exe: /usr/sbin/criu -kexec_exe: /usr/sbin/kexec -systemd_etc: /etc/systemd/system/ -log_dir: /etc/nvwa/log/ -``` - -## Enabling the Kernel Hot Upgrade Tool - -The running of the kernel hot upgrade tool depends on the configuration file. After the configuration file is modified, you need to run the kernel hot upgrade tool again. - -After the installation is successful, you can run the systemd commands to operate the kernel hot upgrade tool. - -- Enable NVWA. - - systemctl enable nvwa - -- Start nvwa. - - systemctl start nvwa - -- View the NVWA logs. - - service nvwa status - -- For more usage, see the usage of systemd. diff --git a/docs/en/docs/Kmesh/appendix.md b/docs/en/docs/Kmesh/appendix.md deleted file mode 100644 index fd5059d26b33ff142b15f73c686e347baa9d549a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kmesh/appendix.md +++ /dev/null @@ -1,3 +0,0 @@ -# Appendix - -Learn more about [Kmesh](https://gitee.com/openeuler/Kmesh#kmesh). diff --git a/docs/en/docs/Kmesh/faqs.md b/docs/en/docs/Kmesh/faqs.md deleted file mode 100644 index cd26472eef5910828ff0f59bd863b00ad6401cf4..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kmesh/faqs.md +++ /dev/null @@ -1,23 +0,0 @@ -# FAQs - -## 1. If the Kmesh Service Is Started in the Cluster Mode and Control Plane IP Address Is Not Configured, Kmesh Reports an Error and Exits - -![](./figures/not_set_cluster_ip.png) - -Possible cause: In cluster mode, the Kmesh service communicates with the control plane program to obtain configuration information. Therefore, you need to configure the correct IP address of the control plane program. - -Solution: Configure the correct IP address of the control plane program by referring to the cluster mode in [Installation and Deployment](./Installation and Deployment.md). - -## 2. Kmesh Reports "get kube config error!" When Started - -![](./figures/get_kubeconfig_error.png) - -Possible cause: In cluster mode, the Kmesh service automatically obtains the IP address of the control plane program based on Kubernetes configurations. If the **kubeconfig** path is not configured in the system, Kmesh will fail to obtain the configurations and reports "get kube config error!" (If the IP address of the control plane program has been correctly configured in the Kmesh configuration file, ignore this problem.) - -Solution: Configure **kubeconfig** as follows: - -```shell -mkdir -p $HOME/.kube -sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config -sudo chown $(id -u):$(id -g) $HOME/.kube/config -``` diff --git a/docs/en/docs/Kmesh/figures/get_kubeconfig_error.png b/docs/en/docs/Kmesh/figures/get_kubeconfig_error.png deleted file mode 100644 index 99087b68c6fafea1506e5f8bd862c371e93bdc97..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Kmesh/figures/get_kubeconfig_error.png and /dev/null differ diff --git a/docs/en/docs/Kmesh/figures/kmesh-arch.png b/docs/en/docs/Kmesh/figures/kmesh-arch.png deleted file mode 100644 index 000ec80ff35556199caef6ce78953599c1c52312..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Kmesh/figures/kmesh-arch.png and /dev/null differ diff --git a/docs/en/docs/Kmesh/figures/not_set_cluster_ip.png b/docs/en/docs/Kmesh/figures/not_set_cluster_ip.png deleted file mode 100644 index 9c879f37fa93c0f4fe0ab0f6220beff174e5f436..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Kmesh/figures/not_set_cluster_ip.png and /dev/null differ diff --git a/docs/en/docs/Kmesh/installation-and-deployment.md b/docs/en/docs/Kmesh/installation-and-deployment.md deleted file mode 100644 index b331932cae8010b38639b3ed6f057f2d5bfd5fae..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kmesh/installation-and-deployment.md +++ /dev/null @@ -1,99 +0,0 @@ -# Installation and Deployment - -## Software - -* OS: openEuler 23.09 - -## Hardware - -* x86_64 - -## Preparing the Environment - -* Install the openEuler OS by referring to the [*openEuler Installation Guide*](../Installation/Installation.md). - -* Root permissions are required for installing Kmesh. - -## Installing Kmesh - -* Install the Kmesh software package. - -```shell -yum install Kmesh -``` - -* Check whether the installation is successful. If the command output contains the name of the software package, the installation is successful. - -```shell -rpm -q Kmesh -``` - -## Deploying Kmesh - -### Cluster Mode - -Before starting Kmesh, configure the IP address of the control plane program (for example, Istiod IP address) in the cluster. - -```json - "clusters": [ - { - "name": "xds-grpc", - "type" : "STATIC", - "connect_timeout": "1s", - "lb_policy": "ROUND_ROBIN", - "load_assignment": { - "cluster_name": "xds-grpc", - "endpoints": [{ - "lb_endpoints": [{ - "endpoint": { - "address":{ - "socket_address": { - "protocol": "TCP", - "address": "192.168.0.1",# Configure the control plane IP address (for example, Istiod IP address). - "port_value": 15010 - } - } - } - }] - }] -``` - -Currently, only the traffic orchestration function is supported in the cluster mode. - -### Local Mode - -Before starting Kmesh, modify `kmesh.service` to enable or disable required functions. - -```shell -# Choose -enable-kmesh and disable ADS. -$ vim /usr/lib/systemd/system/kmesh.service -ExecStart=/usr/bin/kmesh-daemon -enable-kmesh -enable-ads=false -$ systemctl daemon-reload -``` - -To enable mesh acceleration, run the following commands: - -```shell -# Choose -enable-mda and disable ADS. -$ vim /usr/lib/systemd/system/kmesh.service -ExecStart=/usr/bin/kmesh-daemon -enable-mda -enable-ads=false -$ systemctl daemon-reload -``` - -When the Kmesh service is started, the kmesh-daemon program is invoked. For details about how to use the kmesh-daemon program, see [Using kmesh-daemon](./usage.md). - -### Starting Kmesh - -```shell -# Start the Kmesh service. -$ systemctl start kmesh.service -# Check the Kmesh running status. -$ systemctl status kmesh.service -``` - -### Stopping Kmesh - -```shell -# Stop the Kmesh service. -$ systemctl stop kmesh.service -``` diff --git a/docs/en/docs/Kmesh/introduction-to-kmesh.md b/docs/en/docs/Kmesh/introduction-to-kmesh.md deleted file mode 100644 index 196bf4a5a43a8e541c1267d3c6ebae923f05a9c7..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kmesh/introduction-to-kmesh.md +++ /dev/null @@ -1,38 +0,0 @@ -# Introduction to Kmesh - -## Introduction - -As the number of cloud-native applications surges, the scale of cloud applications and application SLAs pose high requirements on cloud infrastructure. - -The Kubernetes-based cloud infrastructure can help implement agile deployment and management of applications. However, it does not support application traffic orchestration. The emergence of service mesh makes up for the lack of traffic orchestration in Kubernetes and complements Kubernetes to implement agile cloud application development and O&M. However, with the development of service mesh applications, the current Sidecar-based mesh architecture has obvious performance defects on the data plane, which has become a consensus in the industry. - -* Long delay - Take the typical service mesh Istio as an example. After meshing, the single-hop delay of service access increases by 2.65 ms, which cannot meet the requirements of delay-sensitive applications. - -* High overhead - In Istio, each Sidecar configuration occupies more than 50 MB memory, and the CPU exclusively occupies two cores by default. For large-scale clusters, the overhead is high, reducing the deployment density of service containers. - -Based on the programmable kernel, Kmesh offloads mesh traffic governance to the OS and shortens the data path from 3 hops to 1 hop, greatly shortening the delay of the data plane and accelerating service innovation. - -## Architecture - -The following figure shows the overall architecture of Kmesh. - -![](./figures/kmesh-arch.png) - -Kmesh consists of the following components: - -* kmesh-controller - Kmesh management program, which is responsible for Kmesh lifecycle management, xDS protocol interconnection, and O&M monitoring. - -* kmesh-api - API layer provided by Kmesh for external systems, including orchestration APIs converted by xDS and O&M monitoring channels. - -* kmesh-runtime - Runtime that supports L3 to L7 traffic orchestration implemented in the kernel. - -* kmesh-orchestration - L3 to L7 traffic orchestration implemented based on eBPF, such as routing, gray release, and load balancing. - -* kmesh-probe - O&M monitoring probe, providing E2E monitoring capabilities. diff --git a/docs/en/docs/Kmesh/kmesh.md b/docs/en/docs/Kmesh/kmesh.md deleted file mode 100644 index ae920162dd78ed35285865abf55d0d2c0435c215..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kmesh/kmesh.md +++ /dev/null @@ -1,5 +0,0 @@ -# Kmesh User Guide - -This document describes how to install, deploy, and use Kmesh, a high-performance service mesh data plane software provided by openEuler. - -This document is intended for developers, open source enthusiasts, and partners who use the openEuler operating system (OS) and want to learn and use Kmesh. Users must have basic knowledge of the Linux OS. diff --git a/docs/en/docs/Kmesh/usage.md b/docs/en/docs/Kmesh/usage.md deleted file mode 100644 index 36ca45afc5eeac9ca454ab8edf6492d412c4b503..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kmesh/usage.md +++ /dev/null @@ -1,69 +0,0 @@ -# Usage - -## Using kmesh-daemon - -```shell -# Display help information. -[root@openEuler ~]# kmesh-daemon -h -Usage of kmesh-daemon: - -bpf-fs-path string - bpf fs path (default "/sys/fs/bpf") - -cgroup2-path string - cgroup2 path (default "/mnt/kmesh_cgroup2") - -config-file string - [if -enable-kmesh] deploy in kube cluster (default "/etc/kmesh/kmesh.json") - -enable-ads - [if -enable-kmesh] enable control-plane from ads (default true) - -enable-kmesh - enable bpf kmesh - -service-cluster string - [if -enable-kmesh] TODO (default "TODO") - -service-node string - [if -enable-kmesh] TODO (default "TODO") - -# Enable ADS by default. -[root@openEuler ~]# kmesh-daemon -enable-kmesh - -# Enable ADS and specify the path of the configuration file. -[root@openEuler ~]# kmesh-daemon -enable-kmesh -enable-ads=true -config-file=/examples/kmesh.json - -# Disable ADS. -[root@openEuler ~]# kmesh-daemon -enable-kmesh -enable-ads=false -``` - -## Using kmesh-cmd - -```shell -# Display help information. -[root@openEuler ~]# kmesh-cmd -h -Usage of kmesh-cmd: - -config-file string - input config-resources to bpf maps (default "./config-resources.json") - -# Manually load configurations. -[root@openEuler ~]# kmesh-cmd -config-file=/examples/config-resources.json -``` - -## Using O&M Commands - -```shell -# Display help information. -[root@openEuler ~]# curl http://localhost:15200/help - /help: print list of commands - /options: print config options - /bpf/kmesh/maps: print bpf kmesh maps in kernel - /controller/envoy: print control-plane in envoy cache - /controller/kubernetes: print control-plane in kubernetes cache - -# Read the loaded configurations. -[root@openEuler ~]# curl http://localhost:15200/bpf/kmesh/maps -[root@openEuler ~]# curl http://localhost:15200/options -``` - -## Precautions - -* If `-enable-ads=true` is configured, Kmesh automatically receives orchestration rules from the service mesh control plane. In this case, do not run the `kmesh-cmd` command to deliver rules to avoid duplicated configurations. - -* The `-bpf-fs-path` option specifies the BPF directory of the OS. Data related to the Kmesh BPF program will be stored in this directory. The default directory is `/sys/fs/bpf`. - -* The `-cgroup2-path` option specifies the cgroup directory of the OS. The default directory is `/mnt/kmesh_cgroup2`. diff --git a/docs/en/docs/KubeEdge/kubeedge-usage-guide.md b/docs/en/docs/KubeEdge/kubeedge-usage-guide.md deleted file mode 100644 index 02c244ade325e223fa41dc3542f3160d84208b24..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeEdge/kubeedge-usage-guide.md +++ /dev/null @@ -1,221 +0,0 @@ -# KubeEdge Usage Guide - -KubeEdge extends the capabilities of Kubernetes to edge scenarios and provides infrastructure support for the network, application deployment, and metadata synchronization between the cloud and the edge. The usage of KubeEdge is the same as that of Kubernetes. In addition, KubeEdge supports the management and control of edge devices. The following example describes how to use KubeEdge to implement edge-cloud synergy. - -## 1. Preparations - -Example: **KubeEdge Counter Demo** - -The counter is a pseudo device. You can run this demo without any additional physical devices. The counter runs on the edge side. You can use the web interface on the cloud side to control the counter and get the counter value. Click the link below to view the schematic diagram. - -For details, see . - -1. This demo requires the KubeEdge v1.2.1 or later. In this example, the latest KubeEdge v1.8.0 is used. - - ```shell - [root@ke-cloud ~]# kubectl get node - NAME STATUS ROLES AGE VERSION - ke-cloud Ready master 13h v1.20.2 - ke-edge1 Ready agent,edge 64s v1.19.3-kubeedge-v1.8.0 - - # Note: In this document, the edge node ke-edge1 is used for verification. If you perform verification by referring to this document, you need to change the edge node name based on your actual deployment. - ``` - -2. Ensure that the following configuration items are enabled for the Kubernetes API server: - - ```shell - --insecuret-port=8080 - --insecure-bind-address=0.0.0.0 - ``` - - You can modify the `/etc/kubernetes/manifests/kube-apiserver.yaml` file, and then restart the Pod of the Kubernetes API server component to make the modifications take effect. - -3. Download the sample code: - - ```shell - [root@ke-cloud ~]# git clone https://github.com/kubeedge/examples.git $GOPATH/src/github.com/kubeedge/examples - ``` - -## 2. Creating the Device Model and Device - -1. Create the device model. - - ```shell - [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds - [root@ke-cloud crds~]# kubectl create -f kubeedge-counter-model.yaml - ``` - -2. Create the device. - - Modify **matchExpressions** as required. - - ```shell - [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds - [root@ke-cloud crds~]# vim kubeedge-counter-instance.yaml - apiVersion: devices.kubeedge.io/v1alpha1 - kind: Device - metadata: - name: counter - labels: - description: 'counter' - manufacturer: 'test' - spec: - deviceModelRef: - name: counter-model - nodeSelector: - nodeSelectorTerms: - - matchExpressions: - - key: 'kubernetes.io/hostname' - operator: In - values: - - ke-edge1 - - status: - twins: - - propertyName: status - desired: - metadata: - type: string - value: 'OFF' - reported: - metadata: - type: string - value: '0' - - [root@ke-cloud crds~]# kubectl create -f kubeedge-counter-instance.yaml - ``` - -## 3. Deploying the Cloud Application - -1. Modify the code. - - The cloud application **web-controller-app** controls the edge application **pi-counter-app**. The default listening port of the cloud application is 80. Change the port number to 8089. - - ```shell - [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app - [root@ke-cloud web-controller-app~]# vim main.go - package main - - import ( - "github.com/astaxie/beego" - "github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app/controller" - ) - - func main() { - beego.Router("/", new(controllers.TrackController), "get:Index") - beego.Router("/track/control/:trackId", new(controllers.TrackController), "get,post:ControlTrack") - - beego.Run(":8089") - } - ``` - -2. Build the image. - - Note: When building the image, copy the source code to the path specified by **GOPATH**. Disable Go modules if they are enabled. - - ```shell - [root@ke-cloud web-controller-app~]# make all - [root@ke-cloud web-controller-app~]# make docker - ``` - -3. Deploy web-controller-app. - - ```shell - [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds - [root@ke-cloud crds~]# kubectl apply -f kubeedge-web-controller-app.yaml - ``` - -## 4. Deploying the Edge Application - -The **pi-counter-app** application on the edge is controlled by the cloud application. The edge application communicates with the MQTT server to perform simple counting. - -1. Modify the code and build the image. - - Change the value of **GOARCH** to **amd64** in `Makefile` to run the container. - - ```shell - [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/counter-mapper - [root@ke-cloud counter-mapper~]# vim Makefile - .PHONY: all pi-execute-app docker clean - all: pi-execute-app - - pi-execute-app: - GOARCH=amd64 go build -o pi-counter-app main.go - - docker: - docker build . -t kubeedge/kubeedge-pi-counter:v1.0.0 - - clean: - rm -f pi-counter-app - - [root@ke-cloud counter-mapper~]# make all - [root@ke-cloud counter-mapper~]# make docker - ``` - -2. Deploy pi-counter-app. - - ```shell - [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds - [root@ke-cloud crds~]# kubectl apply -f kubeedge-pi-counter-app.yaml - - Note: To prevent Pod deployment from being stuck at `ContainerCreating`, run the docker save, scp, and docker load commands to release the image to the edge. - - [root@ke-cloud ~]# docker save -o kubeedge-pi-counter.tar kubeedge/kubeedge-pi-counter:v1.0.0 - [root@ke-cloud ~]# scp kubeedge-pi-counter.tar root@192.168.1.56:/root - [root@ke-edge1 ~]# docker load -i kubeedge-pi-counter.tar - ``` - -## 5. Trying the Demo - -Now, the KubeEdge Demo is deployed on the cloud and edge as follows: - -```shell -[root@ke-cloud ~]# kubectl get pods -o wide -NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES -kubeedge-counter-app-758b9b4ffd-f8qjj 1/1 Running 0 26m 192.168.1.66 ke-cloud -kubeedge-pi-counter-c69698d6-rb4xz 1/1 Running 0 2m 192.168.1.56 ke-edge1 -``` - -Let's test the running effect of the Demo. - -1. Execute the ON command. - On the web page, select **ON** and click **Execute**. You can run the following command on the edge node to view the execution result: - - ```shell - [root@ke-edge1 ~]# docker logs -f counter-container-id - ``` - -2. Check the counter's STATUS. - On the web page, select **STATUS** and click **Execute**. The current counter status is displayed on the web page. - -3. Execute the OFF command. - On the web page, select **OFF** and click **Execute**. You can run the following command on the edge node to view the execution result: - - ```shell - [root@ke-edge1 ~]# docker logs -f counter-container-id - ``` - -## 6. Others - -1. For more official KubeEdge examples, visit . - - |Name | Description | - |---|---| - | [LED-RaspBerry-Pi](https://github.com/kubeedge/examples/blob/master/led-raspberrypi/README.md) |Controlling a LED light with Raspberry Pi using KubeEdge platform.| - |[Data Analysis @ Edge](https://github.com/kubeedge/examples/blob/master/apache-beam-analysis/README.md) | Analyzing data at edge by using Apache Beam and KubeEdge.| - | [Security@Edge](https://github.com/kubeedge/examples/blob/master/security-demo/README.md) | Security at edge using SPIRE for identity management.| - |[Bluetooth-CC2650-demo](https://github.com/kubeedge/examples/blob/master/bluetooth-CC2650-demo/README.md) |Controlling a CC2650 SensorTag bluetooth device using KubeEdge platform.| - | [Play Music @Edge through WeChat](https://github.com/kubeedge/examples/blob/master/wechat-demo/README.md) | Play music at edge based on WeChat and KubeEdge.| - | [Play Music @Edge through Web](https://github.com/kubeedge/examples/blob/master/web-demo/README.md) | Play music at edge based on Web and KubeEdge.| - | [Collecting temperature @Edge](https://github.com/kubeedge/examples/blob/master/temperature-demo/README.md) | Collecting temperature at edge based KubeEdge.| - | [Control pseudo device counter and collect data](https://github.com/kubeedge/examples/blob/master/kubeedge-counter-demo/README.md) | Control pseudo device counter and collect data based KubeEdge.| - |[Play Music @Edge through Twitter](https://github.com/kubeedge/examples/blob/master/ke-twitter-demo/README.md)| Play music at edge based on Twitter and KubeEdge.| - |[Control Zigbee @Edge through cloud](https://github.com/kubeedge/examples/blob/master/kubeedge-edge-ai-application/README.md) | Face detection at cloud using OpenCV and using it to control zigbee on edge using KubeEdge.| - -2. Use EdgeMesh to discover edge services. - - - -3. Customize the cloud-edge message route. - - diff --git a/docs/en/docs/KubeEdge/overview.md b/docs/en/docs/KubeEdge/overview.md deleted file mode 100644 index 8b81038bda1ac5fb4618c68678351f99e1b3c63b..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeEdge/overview.md +++ /dev/null @@ -1,3 +0,0 @@ -# KubeEdge User Guide - -This document describes how to deploy and use the KubeEdge edge computing platform for users and administrators. \ No newline at end of file diff --git a/docs/en/docs/KubeOS/about-kubeos.md b/docs/en/docs/KubeOS/about-kubeos.md deleted file mode 100644 index ec282d26ef1dfbbe0d4a4da2a62e025241b80a10..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeOS/about-kubeos.md +++ /dev/null @@ -1,39 +0,0 @@ -# About KubeOS - -## Introduction - -Containers and Kubernetes are widely used in cloud scenarios. However, a current manner of managing the containers and the OSs separately usually faces problems of function redundancy and difficult collaboration between scheduling systems. In addition, it is difficult to manage OS versions. Software packages are installed, updated, and deleted separately in OSs of the same version. After a period of time, the OS versions become inconsistent, causing version fragmentation. Besides, the OSs may be tightly coupled with services, making it difficult to upgrade major versions. To solve the preceding problems, openEuler provides KubeOS, a container OS upgrade tool based on openEuler. - -Container OSs are lightweight OSs designed for scenarios where services run in containers. KubeOS connects container OSs as components to Kubernetes, so that the container OSs are in the same position as services. With KubeOS, a Kubernetes cluster manages containers and container OSs in a unified system. - -KubeOS is a Kubernetes operator for controlling the container OS upgrade process and upgrading the container OSs as a whole to implement collaboration between the OS managers and services. Before the container OSs are upgraded, services are migrated to other nodes to reduce the impact on services during OS upgrade and configuration. In this upgrade pattern, the container OSs are upgraded atomically so that the OSs remain synchronized with the expected status. This ensures that the OS versions in the cluster are consistent, preventing version fragmentation. - -## Architecture - -### KubeOS Architecture - -**Figure 1** KubeOS architecture - -![](./figures/kubeos-architecture.png) - -As shown in the preceding figure, KubeOS consists of three components: os-operator, os-proxy, and os-agent. The os-operator and os-proxy components run in containers and are deployed in the Kubernetes cluster. os-agent is not considered a cluster component. Its instances run on worker nodes as processes. - -- os-operator: global container OS manager, which continuously checks the container OS versions of all nodes, controls the number of nodes to be upgraded concurrently based on the configured information, and marks the nodes to be upgraded. - -- os-proxy: OS manager of a single node, which continuously checks the container OS version of the node. If a node is marked as the node to be upgraded by os-operator, the node is locked, the pod is evicted, and the upgrade information is forwarded to os-agent. - -- os-agent: receives information from os-proxy, downloads the container OS image used for upgrade from the OS image server, upgrades the container OS, and restarts the node. - -### File System of a Container OS - -**Figure 2** File system layout of a container OS - -![](./figures/file-system-layout-of-a-container-os.png) - -As shown in the figure, a container OS comprises four partitions: - -- boot partition: GRUB2 file partition. -- Persist partition: stores persistent user data. When the container OS is upgraded, the data in this partition is retained. -- Two root partitions: Container OSs use the dual-partition mode with two root partitions, rootA and rootB. Assume that the container runs the OS stored in the rootA partition after initialization. When the system is upgraded, the new system is downloaded to the rootB partition. GRUB has two boot options: A and B. The default boot option of GRUB is set to B and the node is restarted. After the node is started, the container runs the upgraded OS in the rootB partition. - -The root file system of a container OS is read-only. Users' persistent data is stored in the Persist partition. diff --git a/docs/en/docs/KubeOS/figures/file-system-layout-of-a-container-os.png b/docs/en/docs/KubeOS/figures/file-system-layout-of-a-container-os.png deleted file mode 100644 index add62e72f85b103b7dd5780d2e360049f5f712df..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/KubeOS/figures/file-system-layout-of-a-container-os.png and /dev/null differ diff --git a/docs/en/docs/KubeOS/figures/kubeos-architecture.png b/docs/en/docs/KubeOS/figures/kubeos-architecture.png deleted file mode 100644 index 7834a3793b73c49ddd046502c65335a08f576c30..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/KubeOS/figures/kubeos-architecture.png and /dev/null differ diff --git a/docs/en/docs/KubeOS/installation-and-deployment.md b/docs/en/docs/KubeOS/installation-and-deployment.md deleted file mode 100644 index 6788abf9d804c7aea5a53c81ddcd778d51ada3fe..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeOS/installation-and-deployment.md +++ /dev/null @@ -1,203 +0,0 @@ -# Installation and Deployment - -This chapter describes how to install and deploy the KubeOS tool. - -- [Installation and Deployment](#installation-and-deployment) - - [Software and Hardware Requirements](#software-and-hardware-requirements) - - [Hardware Requirements](#hardware-requirements) - - [Software Requirements](#software-requirements) - - [Environment Preparation](#environment-preparation) - - [KubeOS Installation](#kubeos-installation) - - [KubeOS Deployment](#kubeos-deployment) - - [Building the os-operator and os-proxy Images](#building-the-os-operator-and-os-proxy-images) - - [Creating a KubeOS VM Image](#creating-a-kubeos-vm-image) - - [Deploying CRD, os-operator, and os-proxy](#deploying-crd-os-operator-and-os-proxy) - -## Software and Hardware Requirements - -### Hardware Requirements - -- Currently, only the x86 and AArch64 architectures are supported. - -### Software Requirements - -- OS: openEuler 22.09 - -### Environment Preparation - -- Install the openEuler system. For details, see the [*openEuler Installation Guide*](../Installation/Installation.md). -- Install qemu-img, bc, Parted, tar, Yum, Docker, and dosfstools. - -## KubeOS Installation - -To install KubeOS, perform the following steps: - -1. Configure the Yum sources openEuler 22.09 and openEuler 22.09:EPOL: - - ```conf - [openEuler22.09] # openEuler 22.09 official source - name=openEuler22.09 - baseurl=http://repo.openeuler.org/openEuler-22.09/everything/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=http://repo.openeuler.org/openEuler-22.09/everything/$basearch/RPM-GPG-KEY-openEuler - ``` - - ```conf - [Epol] # openEuler 22.09:EPOL official source - name=Epol - baseurl=http://repo.openeuler.org/openEuler-22.09/EPOL/main/$basearch/ - enabled=1 - gpgcheck=1 - gpgkey=http://repo.openeuler.org/openEuler-22.09/OS/$basearch/RPM-GPG-KEY-openEuler - ``` - -2. Install KubeOS as the **root** user. - - ```shell - # yum install KubeOS KubeOS-scripts -y - ``` - -> ![](./public_sys-resources/icon-note.gif)**NOTE**: -> -> KubeOS is installed in the **/opt/kubeOS** directory, including the os-operator, os-proxy, os-agent binary files, KubeOS image build tools, and corresponding configuration files. - -## KubeOS Deployment - -After KubeOS is installed, you need to configure and deploy it. This section describes how to configure and deploy KubeOS. - -### Building the os-operator and os-proxy Images - -#### Environment Preparation - -Before using Docker to create a container image, ensure that Docker has been installed and configured. - -#### Procedure - -1. Go to the working directory. - - ```shell - cd /opt/kubeOS - ``` - -2. Specify the image repository, name, and version for os-proxy. - - ```shell - export IMG_PROXY=your_imageRepository/os-proxy_imageName:version - ``` - -3. Specify the image repository, name, and version for os-operator. - - ```shell - export IMG_OPERATOR=your_imageRepository/os-operator_imageName:version - ``` - -4. Compile a Dockerfile to build an image. Pay attention to the following points when compiling a Dockerfile: - - - The os-operator and os-proxy images must be built based on the base image. Ensure that the base image is safe. - - Copy the os-operator and os-proxy binary files to the corresponding images. - - Ensure that the owner and owner group of the os-proxy binary file in the os-proxy image are **root**, and the file permission is **500**. - - Ensure that the owner and owner group of the os-operator binary file in the os-operator image are the user who runs the os-operator process in the container, and the file permission is **500**. - - The locations of the os-operator and os-proxy binary files in the image and the commands run during container startup must correspond to the parameters specified in the YAML file used for deployment. - - An example Dockerfile is as follows: - - ```text - FROM your_baseimage - COPY ./bin/proxy /proxy - ENTRYPOINT ["/proxy"] - ``` - - ```text - FROM your_baseimage - COPY --chown=6552:6552 ./bin/operator /operator - ENTRYPOINT ["/operator"] - ``` - - Alternatively, you can use multi-stage builds in the Dockerfile. - -5. Build the images (the os-operator and os-proxy images) to be included in the containers OS image. - - ```shell - # Specify the Dockerfile path of os-proxy. - export DOCKERFILE_PROXY=your_dockerfile_proxy - # Specify the Dockerfile path of os-operator. - export DOCKERFILE_OPERATOR=your_dockerfile_operator - # Build images. - docker build -t ${IMG_OPERATOR} -f ${DOCKERFILE_OPERATOR} . - docker build -t ${IMG_PROXY} -f ${DOCKERFILE_PROXY} . - ``` - -6. Push the images to the image repository. - - ```shell - docker push ${IMG_OPERATOR} - docker push ${IMG_PROXY} - ``` - -### Creating a KubeOS VM Image - -#### Precautions - -- The VM image is used as an example. For details about how to create a physical machine image, see **KubeOS Image Creation**. -- The root permission is required for creating a KubeOS image. -- The RPM sources of the kbimg are the **everything** and **EPOL** repositories of openEuler of a specific version. In the Repo file provided during image creation, you are advised to configure the **everything** and **EPOL** repositories of a specific openEuler version for the Yum source. -- By default, the KubeOS VM image built using the default RPM list is stored in the same path as the kbimg tool. This partition must have at least 25 GiB free drive space. -- When creating a KubeOS image, you cannot customize the file system to be mounted. - -#### Procedure - -Use the **kbimg.sh** script to create a KubeOS VM image. For details about the commands, see **KubeOS Image Creation**. - -To create a KubeOS VM image, perform the following steps: - -1. Go to the working directory. - - ```shell - cd /opt/kubeOS/scripts - ``` - -2. Run `kbming.sh` to create a KubeOS image. The following is a command example: - - ```shell - bash kbimg.sh create vm-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' - ``` - - In the command, **xx.repo** indicates the actual Yum source file used for creating the image. You are advised to configure both the **everything** and **EPOL** repositories as Yum sources. - - After the KubeOS image is created, the following files are generated in the **/opt/kubeOS/scripts** directory: - - - **system.img**: system image in raw format. The default size is 20 GB. The size of the root file system partition is less than 2,020 MiB, and the size of the Persist partition is less than 16 GiB. - - **system.qcow2**: system image in QCOW2 format. - - **update.img**: partition image of the root file system that is used for upgrade. - - The created KubeOS VM image can be used only in a VM of the x86 or AArch64 architecture. KubeOS does not support legacy boot in an x86 VM - -### Deploying CRD, os-operator, and os-proxy - -#### Precautions - -- The Kubernetes cluster must be deployed first. For details, see the *openEuler 22.09 Kubernetes Cluster Deployment Guide*. - -- The OS of the worker nodes to be upgraded in the cluster must be the KubeOS built using the method described in the previous section. If it is not, use **system.qcow2** to deploy the VM again. For details about how to deploy a VM, see the *openEuler 22.09 Virtualization User Guide*. Currently, KubeOS does not support the master nodes. Use openEuler 22.09 to deploy the upgrade on the master nodes. -- The YAML files for deploying CustomResourceDefinition (CRD), os-operator, os-proxy, and role-based access control (RBAC) of the OS need to be compiled. -- The os-operator and os-proxy components are deployed in the Kubernetes cluster. os-operator must be deployed as a Deployment, and os-proxy as a DaemonSet. -- Kubernetes security mechanisms, such as the RBAC, pod service account, and security policies, must be deployed. - -#### Procedure - -1. Prepare YAML files used for deploying CRD, RBAC, os-operator, and os-proxy of the OS. For details, see [YAML examples](https://gitee.com/openeuler/KubeOS/tree/master/docs/example/config). The following uses **crd.yaml**, **rbac.yaml**, and **manager.yaml** as examples. - -2. Deploy CRD, RBAC, os-operator, and os-proxy. Assume that the **crd.yaml**, **rbac.yaml**, and **manager.yaml** files are stored in the **config/crd**, **config/rbac**, and **config/manager** directories, respectively. Run the following commands: - - ```shell - kubectl apply -f config/crd - kubectl apply -f config/rbac - kubectl apply -f config/manager - ``` - -3. After the deployment is complete, run the following command to check whether each component is started properly. If **STATUS** of all components is **Running**, the components are started properly. - - ```shell - kubectl get pods -A - ``` diff --git a/docs/en/docs/KubeOS/kubeos-image-creation.md b/docs/en/docs/KubeOS/kubeos-image-creation.md deleted file mode 100644 index cda7c1acc2e2c5d8585f11b2e6737b86eef042ba..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeOS/kubeos-image-creation.md +++ /dev/null @@ -1,169 +0,0 @@ -# KubeOS Image Creation - -## Introduction - -kbimg is an image creation tool required for KubeOS deployment and upgrade. You can use kbimg to create KubeOS Docker, VM, and physical machine images. - -## Commands - -### Command Format - -**bash kbimg.sh** \[ --help | -h \] create \[ COMMANDS \] \[ OPTIONS \] - -### Parameter Description - -* COMMANDS - - | Parameter | Description | - | ------------- | ---------------------------------------------- | - | upgrade-image | Generates a Docker image for installation and upgrade.| - | vm-image | Generates a VM image for installation and upgrade. | - | pxe-image | Generates images and files required for physical machine installation. | - -* OPTIONS - - | Option | Description | - | ------------ | ------------------------------------------------------------ | - | -p | Path of the repo file. The Yum source required for creating an image is configured in the repo file. | - | -v | Version of the created KubeOS image. | - | -b | Path of the os-agent binary file. | - | -e | Password of the **root** user of the KubeOS image, which is an encrypted password with a salt value. You can run the OpenSSL or KIWI command to generate the password.| - | -d | Generated or used Docker image. | - | -h --help | Help Information. | - -## Usage Description - -### Precautions - -* The root permission is required for executing **kbimg.sh**. -* Currently, only the x86 and AArch64 architectures are supported. -* The RPM sources of the kbimg are the **everything** and **EPOL** repositories of openEuler of a specific version. In the Repo file provided during image creation, you are advised to configure the **everything** and **EPOL** repositories of a specific openEuler version for the Yum source. - -### Creating a KubeOS Docker Image - -#### Precautions - -* The created Docker image can be used only for subsequent VM or physical machine image creation or upgrade. It cannot be used to start containers. -* If the default RPM list is used to create a KubeOS image, at least 6 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 6 GB. - -#### Example - -* To configure the DNS, customize the `resolv.conf` file in the `scripts` directory. - -```shell - cd /opt/kubeOS/scripts - touch resolv.conf - vim resolv.conf -``` - -* Create a KubeOS image. - -``` shell -cd /opt/kubeOS/scripts -bash kbimg.sh create upgrade-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' -d your_imageRepository/imageName:version -``` - -* After the creation is complete, view the created KubeOS image. - -``` shell -docker images -``` - -### Creating a KubeOS VM Image - -#### Precautions - -* To use a Docker image to create a KubeOS VM image, pull the corresponding image or create a Docker image first and ensure the security of the Docker image. -* The created KubeOS VM image can be used only in a VM of the x86 or AArch64 architecture. -* Currently, KubeOS does not support legacy boot in an x86 VM. -* If the default RPM list is used to create a KubeOS image, at least 25 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 25 GB. - -#### Example - -* Using the Repo Source - * To configure the DNS, customize the `resolv.conf` file in the `scripts` directory. - - ```shell - cd /opt/kubeOS/scripts - touch resolv.conf - vim resolv.conf - ``` - - * Create a KubeOS VM image. - - ``` shell - cd /opt/kubeOS/scripts - bash kbimg.sh create vm-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' - ``` - -* Using a Docker Image - - ``` shell - cd /opt/kubeOS/scripts - bash kbimg.sh create vm-image -d your_imageRepository/imageName:version - ``` - -* Result Description - After the KubeOS image is created, the following files are generated in the **/opt/kubeOS/scripts** directory: - * **system.qcow2**: system image in QCOW2 format. The default size is 20 GiB. The size of the root file system partition is less than 2,020 MiB, and the size of the Persist partition is less than 16 GiB. - * **update.img**: partition image of the root file system used for upgrade. - -### Creating Images and Files Required for Installing KubeOS on Physical Machines - -#### Precautions - -* To use a Docker image to create a KubeOS VM image, pull the corresponding image or create a Docker image first and ensure the security of the Docker image. -* The created image can only be used to install KubeOS on a physical machine of the x86 or AArch64 architecture. -* The IP address specified in the **Global.cfg** file is a temporary IP address used during installation. After the system is installed and started, configure the network by referring to **openEuler 22.09 Administrator Guide** > **Configuring the Network**. -* KubeOS cannot be installed on multiple drives at the same time. Otherwise, the startup may fail or the mounting may be disordered. -* Currently, KubeOS does not support legacy boot in an x86 physical machine. -* If the default RPM list is used to create a KubeOS image, at least 5 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 5 GB. - -#### Example - -* Modify the `00bootup/Global.cfg` file. All parameters are mandatory. Currently, only IPv4 addresses are supported. The following is a configuration example: - - ```shell - # rootfs file name - rootfs_name=kubeos.tar - # select the target disk to install kubeOS - disk=/dev/sda - # pxe server ip address where stores the rootfs on the http server - server_ip=192.168.1.50 - # target machine temporary ip - local_ip=192.168.1.100 - # target machine temporary route - route_ip=192.168.1.1 - # target machine temporary netmask - netmask=255.255.255.0 - # target machine netDevice name - net_name=eth0 - ``` - -* Using the Repo Source - * To configure the DNS, customize the `resolv.conf` file in the `scripts` directory. - - ```shell - cd /opt/kubeOS/scripts - touch resolv.conf - vim resolv.conf - ``` - - * Create an image required for installing KubeOS on a physical machine. - - ```shell - cd /opt/kubeOS/scripts - bash kbimg.sh create pxe-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' - ``` - -* Using a Docker Image - - ``` shell - cd /opt/kubeOS/scripts - bash kbimg.sh create pxe-image -d your_imageRepository/imageName:version - ``` - -* Result Description - - * **initramfs.img**: initramfs image used for boot from PXE. - * **kubeos.tar**: OS used for installation from PXE. diff --git a/docs/en/docs/KubeOS/kubeos-user-guide.md b/docs/en/docs/KubeOS/kubeos-user-guide.md deleted file mode 100644 index aa615c852e059d2ea061ea75ee0cd9d5264ebfa7..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeOS/kubeos-user-guide.md +++ /dev/null @@ -1,8 +0,0 @@ -# KubeOS User Guide - -This document describes how to install, deploy, and use KubeOS in the openEuler system. KubeOS connects the container OS to the scheduling system in standard extension pattern and manages the OS upgrade of nodes in the cluster through the scheduling system. - -This document is intended for community developers, open source enthusiasts, and partners who use the openEuler system and want to learn and use the container OSs. Users must: - -* Know basic Linux operations. -* Understand Kubernetes and Docker. diff --git a/docs/en/docs/KubeOS/usage-instructions.md b/docs/en/docs/KubeOS/usage-instructions.md deleted file mode 100644 index 0f3500160214567d2b9c183e7fea6c2f380ade6a..0000000000000000000000000000000000000000 --- a/docs/en/docs/KubeOS/usage-instructions.md +++ /dev/null @@ -1,169 +0,0 @@ -# Usage Instructions - -- [Usage Instructions](#usage-instructions) - - [Precautions](#precautions) - - [Upgrade](#upgrade) - - [Rollback](#rollback) - - [Application Scenarios](#application-scenarios) - - [Manual Rollback](#manual-rollback) - - [KubeOS-based Rollback](#kubeos-based-rollback) - -## Precautions - -1. KubeOS upgrades the container OS in an atomic manner, where all software packages are upgraded at the same time. By default, single-package upgrade is not supported. -2. KubeOS supports container OSs with two partitions. Partitions more than two are not supported. -3. You can view the upgrade logs of a single node in the **/var/log/messages** file on the node. -4. Strictly follow the upgrade and rollback procedures described in this document. If the steps are performed in a wrong sequence, the system may fail to be upgraded or rolled back. -5. Upgrade using a Docker image and mTLS two-way authentication are supported only in openEuler 22.09 or later. -6. Cross-major version upgrade is not supported. - -## Upgrade - -Create a custom object of the OS type in the cluster and set the corresponding fields. The OS type comes from the CRD object created in the installation and deployment sections. The following table describes the fields. - -| Parameter |Type | Description | How to Use| Mandatory (Yes/No) | -| -------------- | ------ | ------------------------------------------------------------ | ----- | ---------------- | -| imagetype | string | Type of the upgrade image | The value must be `docker` or `disk`. Other values are invalid. This parameter is valid only in upgrade scenarios.|Yes | -| opstype | string | Operation, that is, upgrade or rollback| The value must be `upgrade` or `rollback`. Other values are invalid.|Yes | -| osversion | string | OS version of the image used for upgrade or rollback | The value must be a KubeOS version, for example, `KubeOS 1.0.0`.|Yes | -| maxunavailable | int | Number of nodes to be upgraded or rolled back at the same time| If the value of `maxunavailable` is greater than the actual number of nodes in the cluster, the deployment can be performed. The upgrade or rollback is performed based on the actual number of nodes in the cluster.|Yes | -| dockerimage | string | Docker image used for upgrade | The value must be in the *repository/name:tag* format. This parameter is valid only when the Docker image is used for upgrade.|Yes | -| imageurl | string | Address of the disk image used for the upgrade| `imageurl` contains the protocol and only HTTP or HTTPS is supported. For example, `https://192.168.122.15/update.img` is valid only when a disk image is used for upgrade.|Yes | -| checksum | string | Checksum (SHA-256) value for disk image verification during the upgrade. | This parameter is valid only when a disk image is used for upgrade.|Yes | -| flagSafe | bool | Whether `imageurl` specifies a secure HTTP address | The value must be `true` or `false`. This parameter is valid only when `imageurl` specifies an HTTP address.|Yes | -| mtls | bool | Whether HTTPS two-way authentication is used for the connection to the `imageurl` address. | The value must be `true` or `false`. This parameter is valid only when `imageurl` specifies an HTTPS address.|Yes | -| cacert | string | Root certificate file used for HTTPS or HTTPS two-way authentication | This parameter is valid only when `imageurl` specifies an HTTPS address.| This parameter is mandatory when `imageurl` specifies an HTTPS address.| -| clientcert | string | Client certificate file used for HTTPS two-way authentication | This parameter is valid only when HTTPS two-way authentication is used.|This parameter is mandatory when `mtls` is set to `true`.| -| clientkey | string | Client public key used for HTTPS two-way authentication | This parameter is valid only when HTTPS two-way authentication is used.|This parameter is mandatory when `mtls` is set to `true`.| - -The address specified by `imageurl` contains the protocol. Only the HTTP or HTTPS protocol is supported. If `imageurl` is set to an HTTPS address, secure transmission is used. If `imageurl` is set to an HTTP address, set `flagSafe` to `true`, because the image can be downloaded only when the address is secure. If `imageurl` is set to an HTTP address but `flagSafe` is not set to `true`, the address is insecure by default. The image will not be downloaded, and a message is written to the log of the node to be upgraded indicating that the address is insecure. - -You are advised to set `imageurl` to an HTTPS address. In this case, ensure that the required certificate has been installed on the node to be upgraded. If the image server is maintained by yourself, you need to sign the certificate and ensure that the certificate has been installed on the node to be upgraded. Place the certificate in the **/etc/KubeOS/certs** directory of KubeOS. The administrator specifies the address and must ensure the security of the address. An intranet address is recommended. - -The container OS image provider must check the validity of the image to ensure that the downloaded container OS image is from a reliable source. - -Compile the YAML file for deploying the OS as a custom resource (CR) instance in the cluster. The following is an example YAML file for deploying the CR instance: - -- Upgrade using a disk image - - ``` - apiVersion: upgrade.openeuler.org/v1alpha1 - kind: OS - metadata: - name: os-sample - spec: - imagetype: disk - opstype: upgrade - osversion: edit.os.version - maxunavailable: edit.node.upgrade.number - dockerimage: "" - imageurl: edit.image.url - checksum: image.checksum - flagSafe: imageurl.safety - mtls: imageurl use mtls or not - cacert: ca certificate - clientcert: client certificate - clientkey: client certificate key - ``` - -- Upgrade using a Docker image - - ``` shell - apiVersion: upgrade.openeuler.org/v1alpha1 - kind: OS - metadata: - name: os-sample - spec: - imagetype: docker - opstype: upgrade - osversion: edit.os.version - maxunavailable: edit.node.upgrade.number - dockerimage: dockerimage like repository/name:tag - imageurl: "" - checksum: "" - flagSafe: false - mtls: true - ``` - - Before using a Docker image to perform the upgrade, create the image first. For details about how to create a Docker image, see **KubeOS Image Creation**. - -Assume that the YAML file is **upgrade_v1alpha1_os.yaml**. - -Check the OS version of the node that is not upgraded. - -``` -kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage' -``` - -Run the following command to deploy the CR instance in the cluster. The node is upgraded based on the configured parameters. - -``` -kubectl apply -f upgrade_v1alpha1_os.yaml -``` - -Check the node OS version again to determine whether the node upgrade is complete. - -``` -kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage' -``` - -> ![](./public_sys-resources/icon-note.gif)**NOTE**: -> -> If you need to perform the upgrade again, modify the `imageurl`, `osversion`, `checksum`, `maxunavailable`, `flagSafe`, or `dockerimage` parameters in **upgrade_v1alpha1_os.yaml**. - -## Rollback - -### Application Scenarios - -- If a node cannot be started, you can only manually roll back the container OS to the previous version that can be properly started. -- If a node can be started and run the system, you can manually or use KubeOS (similar to the upgrade) to roll back the container OS. You are advised to use KubeOS. - -### Manual Rollback - -Manually restart the node and select the second boot option to roll back the container OS. Manual rollback can only roll back the container OS to the version before the upgrade. - -### KubeOS-based Rollback - -- Roll back to any version. - - Modify the YAML configuration file (for example, **upgrade_v1alpha1_os.yaml**) of the CR instance of the OS and set the corresponding fields to the image information of the target source version. The OS type comes from the CRD object created in the installation and deployment sections. For details about the fields and examples, see the previous section. - - - After the YAML is modified, run the update command. After the custom object is updated in the cluster, the node performs rollback based on the configured field information. - - ``` - kubectl apply -f upgrade_v1alpha1_os.yaml - ``` - -- Roll back to the previous version. - - - Modify the **upgrade_v1alpha1_os.yaml** file. Set **osversion** to the previous version and **opstype** to **rollback** to roll back to the previous version (that is, switch to the previous partition). Example YAML: - - ``` - apiVersion: upgrade.openeuler.org/v1alpha1 - kind: OS - metadata: - name: os-sample - spec: - imagetype: "" - opstype: rollback - osversion: KubeOS pervious version - maxunavailable: 2 - dockerimage: "" - imageurl: "" - checksum: "" - flagSafe: false - mtls:true - ``` - - - After the YAML is modified, run the update command. After the custom object is updated in the cluster, the node performs rollback based on the configured field information. - - ``` - kubectl apply -f upgrade_v1alpha1_os.yaml - ``` - - After the update is complete, the node rolls back the container OS based on the configuration information. - -- Check the OS version of the container on the node to determine whether the rollback is successful. - - ``` - kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage' - ``` diff --git a/docs/en/docs/Kubernetes/Kubernetes.md b/docs/en/docs/Kubernetes/Kubernetes.md deleted file mode 100644 index 4b1174b926b9a10a3de4a7ecf97757a64da0c341..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/Kubernetes.md +++ /dev/null @@ -1,12 +0,0 @@ -# Kubernetes Cluster Deployment Guide - -This document describes how to deploy a Kubernetes cluster in binary mode on openEuler. - -Note: All operations in this document are performed using the `root` permission. - -## Cluster Status - -The cluster status used in this document is as follows: - -- Cluster structure: six VMs running the `openEuler 21.09` OS, three master nodes, and three nodes. -- Physical machine: `x86/ARM` server of `openEuler 21.09`. diff --git a/docs/en/docs/Kubernetes/deploying-a-Kubernetes-cluster-manually.md b/docs/en/docs/Kubernetes/deploying-a-Kubernetes-cluster-manually.md deleted file mode 100644 index 330a9610fdaf193c0ae5ce17defe0b5de5f30cdd..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/deploying-a-Kubernetes-cluster-manually.md +++ /dev/null @@ -1,18 +0,0 @@ -# Deploying a Kubernetes Cluster Manually - -**Note: Manual deployment applies only to experimental and learning environments and is not intended for commercial environments.** - -This chapter describes how to deploy a Kubernetes cluster. - -## Environment - -Deploy based on the preceding [VM installation](./preparing-VMs.md) and obtain the following VM list: - -| HostName | MAC | IPv4 | -| ---------- | ----------------- | -------------------| -| k8smaster0 | 52:54:00:00:00:80 | 192.168.122.154/24 | -| k8smaster1 | 52:54:00:00:00:81 | 192.168.122.155/24 | -| k8smaster2 | 52:54:00:00:00:82 | 192.168.122.156/24 | -| k8snode1 | 52:54:00:00:00:83 | 192.168.122.157/24 | -| k8snode2 | 52:54:00:00:00:84 | 192.168.122.158/24 | -| k8snode3 | 52:54:00:00:00:85 | 192.168.122.159/24 | diff --git a/docs/en/docs/Kubernetes/deploying-a-node-component.md b/docs/en/docs/Kubernetes/deploying-a-node-component.md deleted file mode 100644 index 485f96d6c4880007b91ec630f380b1b379658a2f..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/deploying-a-node-component.md +++ /dev/null @@ -1 +0,0 @@ -# Deploying a Node Component This section uses the `k8snode1` node as an example. ## Environment Preparation ```bash # A proxy needs to be configured for the intranet. $ dnf install -y docker iSulad conntrack-tools socat containernetworking-plugins $ swapoff -a $ mkdir -p /etc/kubernetes/pki/ $ mkdir -p /etc/cni/net.d $ mkdir -p /opt/cni # Delete the default kubeconfig file. $ rm /etc/kubernetes/kubelet.kubeconfig ## Use iSulad as the runtime ########. # Configure the iSulad. cat /etc/isulad/daemon.json { "registry-mirrors": [ "docker.io" ], "insecure-registries": [ "k8s.gcr.io", "quay.io" ], "pod-sandbox-image": "k8s.gcr.io/pause:3.2",# pause type "network-plugin": "cni", # If this parameter is left blank, the CNI network plug-in is disabled. In this case, the following two paths become invalid. After the plug-in is installed, restart iSulad. "cni-bin-dir": "/usr/libexec/cni/", "cni-conf-dir": "/etc/cni/net.d", } # Add the proxy to the iSulad environment variable and download the image. cat /usr/lib/systemd/system/isulad.service [Service] Type=notify Environment="HTTP_PROXY=http://name:password@proxy:8080" Environment="HTTPS_PROXY=http://name:password@proxy:8080" # Restart the iSulad and set it to start automatically upon power-on. systemctl daemon-reload systemctl restart isulad ## If Docker is used as the runtime, run the following command: ######## $ dnf install -y docker # If a proxy environment is required, configure a proxy for Docker, add the configuration file http-proxy.conf, and edit the following content. Replace name, password, and proxy-addr with the actual values. $ cat /etc/systemd/system/docker.service.d/http-proxy.conf [Service] Environment="HTTP_PROXY=http://name:password@proxy-addr:8080" $ systemctl daemon-reload $ systemctl restart docker ``` ## Creating kubeconfig Configuration Files Perform the following operations on each node to create a configuration file: ```bash $ kubectl config set-cluster openeuler-k8s \ --certificate-authority=/etc/kubernetes/pki/ca.pem \ --embed-certs=true \ --server=https://192.168.122.154:6443 \ --kubeconfig=k8snode1.kubeconfig $ kubectl config set-credentials system:node:k8snode1 \ --client-certificate=/etc/kubernetes/pki/k8snode1.pem \ --client-key=/etc/kubernetes/pki/k8snode1-key.pem \ --embed-certs=true \ --kubeconfig=k8snode1.kubeconfig $ kubectl config set-context default \ --cluster=openeuler-k8s \ --user=system:node:k8snode1 \ --kubeconfig=k8snode1.kubeconfig $ kubectl config use-context default --kubeconfig=k8snode1.kubeconfig ``` **Note: Change k8snode1 to the corresponding node name.** ## Copying the Certificate Similar to the control plane, all certificates, keys, and related configurations are stored in the `/etc/kubernetes/pki/` directory. ```bash $ ls /etc/kubernetes/pki/ ca.pem k8snode1.kubeconfig kubelet_config.yaml kube-proxy-key.pem kube-proxy.pem k8snode1-key.pem k8snode1.pem kube_proxy_config.yaml kube-proxy.kubeconfig ``` ## CNI Network Configuration containernetworking-plugins is used as the CNI plug-in used by kubelet. In the future, plug-ins such as calico and flannel can be introduced to enhance the network capability of the cluster. ```bash # Bridge Network Configuration $ cat /etc/cni/net.d/10-bridge.conf { "cniVersion": "0.3.1", "name": "bridge", "type": "bridge", "bridge": "cnio0", "isGateway": true, "ipMasq": true, "ipam": { "type": "host-local", "subnet": "10.244.0.0/16", "gateway": "10.244.0.1" }, "dns": { "nameservers": [ "10.244.0.1" ] } } # Loopback Network Configuration $ cat /etc/cni/net.d/99-loopback.conf { "cniVersion": "0.3.1", "name": "lo", "type": "loopback" } ``` ## Deploying the kubelet Service ### Configuration File on Which Kubelet Depends ```bash $ cat /etc/kubernetes/pki/kubelet_config.yaml kind: KubeletConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 authentication: anonymous: enabled: false webhook: enabled: true x509: clientCAFile: /etc/kubernetes/pki/ca.pem authorization: mode: Webhook clusterDNS: - 10.32.0.10 clusterDomain: cluster.local runtimeRequestTimeout: "15m" tlsCertFile: "/etc/kubernetes/pki/k8snode1.pem" tlsPrivateKeyFile: "/etc/kubernetes/pki/k8snode1-key.pem" ``` **Note: The IP address of the cluster DNS is 10.32.0.10, which must be the same as the value of service-cluster-ip-range.** ### Compiling the systemd Configuration File ```bash $ cat /usr/lib/systemd/system/kubelet.service [Unit] Description=kubelet: The Kubernetes Node Agent Documentation=https://kubernetes.io/docs/ Wants=network-online.target After=network-online.target [Service] ExecStart=/usr/bin/kubelet \ --config=/etc/kubernetes/pki/kubelet_config.yaml \ --network-plugin=cni \ --pod-infra-container-image=k8s.gcr.io/pause:3.2 \ --kubeconfig=/etc/kubernetes/pki/k8snode1.kubeconfig \ --register-node=true \ --hostname-override=k8snode1 \ --cni-bin-dir="/usr/libexec/cni/" \ --v=2 Restart=always StartLimitInterval=0 RestartSec=10 [Install] WantedBy=multi-user.target ``` **Note: If iSulad is used as the runtime, add the following configuration:** ```bash --container-runtime=remote \ --container-runtime-endpoint=unix:///var/run/isulad.sock \ ``` ## Deploying kube-proxy ### Configuration File on Which kube-proxy Depends ```bash cat /etc/kubernetes/pki/kube_proxy_config.yaml kind: KubeProxyConfiguration apiVersion: kubeproxy.config.k8s.io/v1alpha1 clientConnection: kubeconfig: /etc/kubernetes/pki/kube-proxy.kubeconfig clusterCIDR: 10.244.0.0/16 mode: "iptables" ``` ### Compiling the systemd Configuration File ```bash $ cat /usr/lib/systemd/system/kube-proxy.service [Unit] Description=Kubernetes Kube-Proxy Server Documentation=https://kubernetes.io/docs/reference/generated/kube-proxy/ After=network.target [Service] EnvironmentFile=-/etc/kubernetes/config EnvironmentFile=-/etc/kubernetes/proxy ExecStart=/usr/bin/kube-proxy \ $KUBE_LOGTOSTDERR \ $KUBE_LOG_LEVEL \ --config=/etc/kubernetes/pki/kube_proxy_config.yaml \ --hostname-override=k8snode1 \ $KUBE_PROXY_ARGS Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target ``` ## Starting a Component Service ```bash $ systemctl enable kubelet kube-proxy $ systemctl start kubelet kube-proxy ``` Deploy other nodes in sequence. ## Verifying the Cluster Status Wait for several minutes and run the following command to check the node status: ```bash $ kubectl get nodes --kubeconfig /etc/kubernetes/pki/admin.kubeconfig NAME STATUS ROLES AGE VERSION k8snode1 Ready 17h v1.20.2 k8snode2 Ready 19m v1.20.2 k8snode3 Ready 12m v1.20.2 ``` ## Deploying coredns coredns can be deployed on a node or master node. In this document, coredns is deployed on the `k8snode1` node. ### Compiling the coredns Configuration File ```bash $ cat /etc/kubernetes/pki/dns/Corefile .:53 { errors health { lameduck 5s } ready kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure endpoint https://192.168.122.154:6443 tls /etc/kubernetes/pki/ca.pem /etc/kubernetes/pki/admin-key.pem /etc/kubernetes/pki/admin.pem kubeconfig /etc/kubernetes/pki/admin.kubeconfig default fallthrough in-addr.arpa ip6.arpa } prometheus :9153 forward . /etc/resolv.conf { max_concurrent 1000 } cache 30 loop reload loadbalance } ``` Note: - Listen to port 53. - Configure the Kubernetes plug-in, including the certificate and the URL of kube api. ### Preparing the service File of systemd ```bash cat /usr/lib/systemd/system/coredns.service [Unit] Description=Kubernetes Core DNS server Documentation=https://github.com/coredns/coredns After=network.target [Service] ExecStart=bash -c "KUBE_DNS_SERVICE_HOST=10.32.0.10 coredns -conf /etc/kubernetes/pki/dns/Corefile" Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target ``` ### Starting the Service ```bash $ systemctl enable coredns $ systemctl start coredns ``` ### Creating the Service Object of coredns ```bash $ cat coredns_server.yaml apiVersion: v1 kind: Service metadata: name: kube-dns namespace: kube-system annotations: prometheus.io/port: "9153" prometheus.io/scrape: "true" labels: k8s-app: kube-dns kubernetes.io/cluster-service: "true" kubernetes.io/name: "CoreDNS" spec: clusterIP: 10.32.0.10 ports: - name: dns port: 53 protocol: UDP - name: dns-tcp port: 53 protocol: TCP - name: metrics port: 9153 protocol: TCP ``` ### Creating the Endpoint Object of coredns ```bash $ cat coredns_ep.yaml apiVersion: v1 kind: Endpoints metadata: name: kube-dns namespace: kube-system subsets: - addresses: - ip: 192.168.122.157 ports: - name: dns-tcp port: 53 protocol: TCP - name: dns port: 53 protocol: UDP - name: metrics port: 9153 protocol: TCP ``` ### Confirming the coredns Service ```bash # View the service object. $ kubectl get service -n kube-system kube-dns NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kube-dns ClusterIP 10.32.0.10 53/UDP,53/TCP,9153/TCP 51m # View the endpoint object. $ kubectl get endpoints -n kube-system kube-dns NAME ENDPOINTS AGE kube-dns 192.168.122.157:53,192.168.122.157:53,192.168.122.157:9153 52m ``` \ No newline at end of file diff --git a/docs/en/docs/Kubernetes/deploying-control-plane-components.md b/docs/en/docs/Kubernetes/deploying-control-plane-components.md deleted file mode 100644 index a9b9bb2faff7c208fe6fb3fb1f02616d5c2f7f18..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/deploying-control-plane-components.md +++ /dev/null @@ -1,357 +0,0 @@ -# Deploying Components on the Control Plane - -## Preparing the kubeconfig File for All Components - -### kube-proxy - -```bash -kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.122.154:6443 --kubeconfig=kube-proxy.kubeconfig -kubectl config set-credentials system:kube-proxy --client-certificate=/etc/kubernetes/pki/kube-proxy.pem --client-key=/etc/kubernetes/pki/kube-proxy-key.pem --embed-certs=true --kubeconfig=kube-proxy.kubeconfig -kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-proxy --kubeconfig=kube-proxy.kubeconfig -kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig -``` - -### kube-controller-manager - -```bash -kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=kube-controller-manager.kubeconfig -kubectl config set-credentials system:kube-controller-manager --client-certificate=/etc/kubernetes/pki/kube-controller-manager.pem --client-key=/etc/kubernetes/pki/kube-controller-manager-key.pem --embed-certs=true --kubeconfig=kube-controller-manager.kubeconfig -kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig -kubectl config use-context default --kubeconfig=kube-controller-manager.kubeconfig -``` - -### kube-scheduler - -```bash -kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=kube-scheduler.kubeconfig -kubectl config set-credentials system:kube-scheduler --client-certificate=/etc/kubernetes/pki/kube-scheduler.pem --client-key=/etc/kubernetes/pki/kube-scheduler-key.pem --embed-certs=true --kubeconfig=kube-scheduler.kubeconfig -kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig -kubectl config use-context default --kubeconfig=kube-scheduler.kubeconfig -``` - -### admin - -```bash -kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=admin.kubeconfig -kubectl config set-credentials admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=admin.kubeconfig -kubectl config set-context default --cluster=openeuler-k8s --user=admin --kubeconfig=admin.kubeconfig -kubectl config use-context default --kubeconfig=admin.kubeconfig -``` - -### Obtaining the kubeconfig Configuration File - -```bash -admin.kubeconfig kube-proxy.kubeconfig kube-controller-manager.kubeconfig kube-scheduler.kubeconfig -``` - -## Configuration for Generating the Key Provider - -When api-server is started, a key pair `--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml` needs to be provided. In this document, a key pair `--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml` is generated by using urandom: - -```bash -$ cat generate.bash -#!/bin/bash - -ENCRYPTION_KEY=$(head -c 32 /dev/urandom | base64) - -cat > encryption-config.yaml < ![](./public_sys-resources/icon-note.gif)**NOTE:** -> -> - When a cluster is deleted, all data in the cluster is deleted and cannot be restored. Exercise caution when performing this operation. -> - Currently, dismantling a cluster does not delete the containers and the container images. However, if the Kubernetes cluster is configured to install a container engine during the deployment, the container engine will be deleted. As a result, the containers may run abnormally. -> - Some error information may be displayed when dismantling the cluster. Generally, this is caused by the error results returned during the delete operations. The cluster can still be properly dismantled. -> - -You can use the command line to delete the entire cluster. For example, run the following command to delete the k8s-cluster: - -```shell -$ eggo -d cleanup --id k8s-cluster -``` diff --git a/docs/en/docs/Kubernetes/eggo-tool-introduction.md b/docs/en/docs/Kubernetes/eggo-tool-introduction.md deleted file mode 100644 index 0b76260e9ffbb39b1d5e053e65d69d3a3cbe17ae..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/eggo-tool-introduction.md +++ /dev/null @@ -1,431 +0,0 @@ -# Tool Introduction - -This chapter describes the information related to the automatic deployment tool. You are advised to read this chapter before deployment. - -## Deployment Modes - -The automatic Kubernetes cluster deployment tool provided by openEuler supports one-click deployment using the CLI. The tool provides the following deployment modes: - -- Offline deployment: Prepare all required RPM packages, binary files, plugins, and container images on the local host, pack the packages into a tar.gz file in a specified format, and compile the corresponding YAML configuration file. Then, you can run commands to deploy the cluster in one-click. This deployment mode can be used when the VM cannot access the external network. -- Online deployment: Compile the YAML configuration file. The required RPM packages, binary files, plugins, and container images are automatically downloaded from the Internet during installation and deployment. In this mode, the VM must be able to access the software sources and the image repository on which the cluster depends, for example, Docker Hub. - -## Configurations - -When you use the automatic Kubernetes cluster deployment tool, use the YAML configuration file to describe the cluster deployment information. This section describes the configuration items and provides configuration examples. - -### Configuration Items - -- cluster-id: Cluster name, which must comply with the naming rules for the DNS names. Example: k8s-cluster - -- username: User name used to log in to the hosts using SSH where the Kubernetes cluster is to be deployed. The user name must be identical on all hosts. - -- private-key-path:The path of the key for password-free SSH login. You only need to configure either private-key-path or password. If both are configured, private-key-path is used preferentially. - -- masters: The master node list. It is recommended that each master node is also set as a worker node. Each master node contains the following sub-items. Each master node must be configured with a group of sub-items: - - name: The name of the master node, which is the node name displayed to the Kubernetes cluster. - - ip: The IP address of the master node. - - port: The port for SSH login of the node. The default value is 22. - - arch: CPU architecture of the master node. For example, the value for x86_64 CPUs is amd64. - -- workers: The list of the worker nodes. Each worker node contains the following sub-items. Each worker node must be configured with a group of sub-items: - - name: The name of the worker node, which is the node name displayed to the Kubernetes cluster. - - ip: The IP address of the master node. - - port: The port for SSH login of the node. The default value is 22. - - arch: CPU architecture of the worker node. For example, the value for x86_64 CPUs is amd64. - -- etcds: The list of etcd nodes. If this parameter is left empty, one etcd node is deployed for each master node. Otherwise, only the configured etcd node is deployed. Each etcd node contains the following sub-items. Each etcd node must be configured with a group of sub-items: - - name: The name of the etcd node, which is the node name displayed to the Kubernetes cluster. - - ip: The IP address of the etcd node. - - port: The port for SSH login. - - arch: CPU architecture of the etcd node. For example, the value for x86_64 CPUs is amd64. - -- loadbalance: The loadbalance node list. Each loadbalance node contains the following sub-items. Each loadbalance node must be configured with a group of sub-items: - - name: The name of the loadbalance node, which is the node name displayed to the Kubernetes cluster. - - ip: The IP address of the loadbalance node. - - port: The port for SSH login. - - arch: CPU architecture of the loadbalance node. For example, the value for x86_64 CPUs is amd64. - - bind-port: The listening port of the load balancing service. - -- external-ca: Whether to use an external CA certificate. If yes, set this parameter to true. Otherwise, set this parameter to false. - -- external-ca-path: The path of the external CA certificate file. This parameter takes affect only when external-ca is set to true. - -- service: service information created by Kubernetes. The service configuration item contains the following sub-items: - - cidr: The IP address segment of the service created by Kubernetes. - - dnsaddr: DNS address of the service created by Kubernetes - - gateway: The gateway address of the service created by Kubernetes. - - dns: The configuration item of the CoreDNS created by Kubernetes. The dns configuration item contains the following sub-items: - - corednstype: The deployment type of the CoreDNS created by Kubernetes. The value can be pod or binary. - - imageversion: The CoreDNS image version of the pod deployment type. - - replicas: The number of CoreDNS replicas of the pod deployment type. - -- network: The network configuration of the Kubernetes cluster. The network configuration item contains the following sub-items: - - podcidr: IP address segment of the Kubernetes cluster network. - - plugin: The network plugin deployed in the Kubernetes cluster - - plugin-args: The configuration file path of the network plugin of the Kubernetes cluster network. Example: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"} - -- apiserver-endpoint: The IP address or domain name of the APIServer service that can be accessed by external systems. If loadbalance is configured, set this parameter to the IP address of the loadbalance node. Otherwise, set this parameter to the IP address of the first master node. - -- apiserver-cert-sans: The IP addresses and domain names that need to be configured in the APIServer certificate. This configuration item contains the following sub-items: - - dnsnames: The array list of the domain names that need to be configured in the APIServer certificate. - - ips: The array list of IP addresses that need to be configured in the APIServer certificate. - -- apiserver-timeout: APIServer response timeout interval. - -- etcd-token: The etcd cluster name. - -- dns-vip: The virtual IP address of the DNS. - -- dns-domain: The DNS domain name suffix. - -- pause-image: The complete image name of the pause container. - -- network-plugin: The type of the network plugin. This parameter can only be set to cni. If this item is not configured, the default Kubernetes network is used. - -- cni-bin-dir: network plugin address. Use commas (,) to separate multiple addresses. For example: /usr/libexec/cni,/opt/cni/bin. - -- runtime: The type of the container runtime. Currently, docker and iSulad are supported. - -- runtime-endpoint: The endpoint of the container runtime. This parameter is optional when runtime is set to docker. - -- registry-mirrors: The mirror site address of the image repository used for downloading container images. - -- insecure-registries: The address of the image repository used for downloading container images through HTTP. - -- config-extra-args: The extra parameters for starting services of each component (such as kube-apiserver and etcd). This configuration item contains the following sub-items: - - name: The component name. The value can be etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy or kubelet. - - - extra-args: The extended parameters of the component. The format is key: value. Note that the component parameter corresponding to key must be prefixed with a hyphen (-) or two hyphens (--). - - - open-ports: Configure the ports that need to be enabled additionally. The ports required by Kubernetes do not need to be configured. Other plugin ports need to be configured additionally. - - worker | master | etcd | loadbalance: The type of the node where the ports are enabled. Each configuration item contains one or more port and protocol sub-items. - - port: The port address. - - protocol: The port type. The value can be tcp or udp. - - - install: Configure the detailed information about the installation packages or binary files to be installed on each type of nodes. Note that the corresponding files must be packaged in a tar.gz installation package. The following describes the full configuration. Select the configuration items as needed. - - package-source: The detailed information about the installation package. - - type: The compression type of the installation package. Currently, only tar.gz installation packages are supported. - - dstpath: The path where the installation package is to be decompressed on the peer host. The path must be valid absolute path. - - srcpath: The path for storing the installation packages of different architectures. The architecture must correspond to the host architecture. The path must be a valid absolute path. - - arm64: The path of the installation package of the ARM64 architecture. This parameter is required if any ARM64 node is included in the configuration. - - amd64: The path of the installation package of the AMD64 architecture. This parameter is required if any x86_64 node is included in the configuration. - - > ![](./public_sys-resources/icon-note.gif)**NOTE**: - > - > - In the install configuration item, the sub-items of etcd, kubernetes-master, kubernetes-worker, network, loadbalance, container, image, and dns are the same, that is, name, type, dst, schedule, and TimeOut. dst, schedule, and TimeOut are optional. You can determine whether to configure them based on the files to be installed. The following uses the etcd and kubernetes-master nodes as an example. - - - etcd: The list of packages or binary files to be installed on etcd nodes. - - name: The names of the software packages or binary files to be installed. If the software package is an installation package, enter only the name and do not specify the version. During the installation, `$name*` is used for identification. Example: etcd. If there are multiple software packages, use commas (,) to separate them. - - type: The type of the configuration item. The value can be pkg, repo, bin, file, dir, image, yaml, or shell. If type is set to repo, configure the repo source on the corresponding node. - - dst: The path of the destination folder. This parameter is required when type is set to bin, file, or dir. It indicates the directory where a file or folder is stored. To prevent users from incorrectly configuring a path and deleting important files during cleanup, this parameter must be set to a path in the whitelist. For details, see "Whitelist Description." - - kubernetes-master: The list of packages or binary files to be installed on the Kubernetes master nodes. - - kubernetes-worker: The list of packages or binary files to be installed on the Kubernetes worker nodes. - - network: The list of packages or binary files to be installed for the network. - - loadbalance: The list of packages or binary files to be installed on the loadbalance nodes. - - container: The list of packages or binary files to be installed for the containers. - - image: The tar package of the container image. - - dns: Kubernetes CoreDNS installation package. If corednstype is set to pod, this parameter is not required. - - addition: The list of additional installation packages or binary files. - - master: The following configurations will be installed on all master nodes. - - name: The name of the software package or binary file to be installed. - - type: The type of the configuration item. The value can be pkg, repo, bin, file, dir, image, yaml, or shell. If type is set to repo, configure the repo source on the corresponding node. - - schedule: Valid only when type is set to shell. This parameter indicates when the user wants to execute the script. The value can be prejoin (before the node is added), postjoin (after the node is added), precleanup (before the node is removed), or postcleanup (after the node is removed). - - TimeOut: The script execution timeout interval. If the execution times out, the process is forcibly stopped. The default value is 30s. - - worker: The configurations will be installed on all worker nodes. The configuration format is the same as that of master under addition. - -### Whitelist Description - -The value of dst under install must match the whitelist rules. Set it to a path in the whitelist or a subdirectory of the path. The current whitelist is as follows: - -- /usr/bin -- /usr/local/bin -- /opt/cni/bin -- /usr/libexec/cni -- /etc/kubernetes -- /usr/lib/systemd/system -- /etc/systemd/system -- /tmp - -### Configuration Example - -The following is an example of the YAML file configuration. As shown in the example, nodes of different types can be deployed on a same host, but the configurations of these nodes must be the same. For example, a master node and a worker node are deployed on test0. - -```yaml -cluster-id: k8s-cluster -username: root -private-key-path: /root/.ssh/private.key -masters: -- name: test0 - ip: 192.168.0.1 - port: 22 - arch: arm64 -workers: -- name: test0 - ip: 192.168.0.1 - port: 22 - arch: arm64 -- name: test1 - ip: 192.168.0.3 - port: 22 - arch: arm64 -etcds: -- name: etcd-0 - ip: 192.168.0.4 - port: 22 - arch: amd64 -loadbalance: - name: k8s-loadbalance - ip: 192.168.0.5 - port: 22 - arch: amd64 - bind-port: 8443 -external-ca: false -external-ca-path: /opt/externalca -service: - cidr: 10.32.0.0/16 - dnsaddr: 10.32.0.10 - gateway: 10.32.0.1 - dns: - corednstype: pod - imageversion: 1.8.4 - replicas: 2 -network: - podcidr: 10.244.0.0/16 - plugin: calico - plugin-args: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"} -apiserver-endpoint: 192.168.122.222:6443 -apiserver-cert-sans: - dnsnames: [] - ips: [] -apiserver-timeout: 120s -etcd-external: false -etcd-token: etcd-cluster -dns-vip: 10.32.0.10 -dns-domain: cluster.local -pause-image: k8s.gcr.io/pause:3.2 -network-plugin: cni -cni-bin-dir: /usr/libexec/cni,/opt/cni/bin -runtime: docker -runtime-endpoint: unix:///var/run/docker.sock -registry-mirrors: [] -insecure-registries: [] -config-extra-args: - - name: kubelet - extra-args: - "--cgroup-driver": systemd -open-ports: - worker: - - port: 111 - protocol: tcp - - port: 179 - protocol: tcp -install: - package-source: - type: tar.gz - dstpath: "" - srcpath: - arm64: /root/rpms/packages-arm64.tar.gz - amd64: /root/rpms/packages-x86.tar.gz - etcd: - - name: etcd - type: pkg - dst: "" - kubernetes-master: - - name: kubernetes-client,kubernetes-master - type: pkg - kubernetes-worker: - - name: docker-engine,kubernetes-client,kubernetes-node,kubernetes-kubelet - type: pkg - dst: "" - - name: conntrack-tools,socat - type: pkg - dst: "" - network: - - name: containernetworking-plugins - type: pkg - dst: "" - loadbalance: - - name: gd,gperftools-libs,libunwind,libwebp,libxslt - type: pkg - dst: "" - - name: nginx,nginx-all-modules,nginx-filesystem,nginx-mod-http-image-filter,nginx-mod-http-perl,nginx-mod-http-xslt-filter,nginx-mod-mail,nginx-mod-stream - type: pkg - dst: "" - container: - - name: emacs-filesystem,gflags,gpm-libs,re2,rsync,vim-filesystem,vim-common,vim-enhanced,zlib-devel - type: pkg - dst: "" - - name: libwebsockets,protobuf,protobuf-devel,grpc,libcgroup - type: pkg - dst: "" - - name: yajl,lxc,lxc-libs,lcr,clibcni,iSulad - type: pkg - dst: "" - image: - - name: pause.tar - type: image - dst: "" - dns: - - name: coredns - type: pkg - dst: "" - addition: - master: - - name: prejoin.sh - type: shell - schedule: "prejoin" - TimeOut: "30s" - - name: calico.yaml - type: yaml - dst: "" - worker: - - name: docker.service - type: file - dst: /usr/lib/systemd/system/ - - name: postjoin.sh - type: shell - schedule: "postjoin" -``` - -### Installation Package Structure - -For offline deployment, you need to prepare the Kubernetes software package and the related offline installation packages, and store the offline installation packages in a specific directory structure. The directory structure is as follows: - -```shell -package -├── bin -├── dir -├── file -├── image -├── pkg -└── packages_notes.md -``` - -The preceding directories are described as follows: - -- The directory structure of the offline deployment package corresponds to the package types in the cluster configuration file config. The package types include pkg, repo, bin, file, dir, image, yaml and shell. - -- The bin directory stores binary files, corresponding to the bin package type. - -- The dir directory stores the directory that needs to be copied to the target host. You need to configure the dst destination path, corresponding to the dir package type. - -- The file directory stores three types of files: file, yaml, and shell. The file type indicates the files to be copied to the target host, and requires the dst destination path to be configured. The yaml type indicates the user-defined YAML files, which will be applied after the cluster is deployed. The shell type indicates the scripts to be executed, and requires the schedule execution time to be configured. The execution time includes prejoin (before the node is added), postjoin (after the node is added), precleanup (before the node is removed), and postcleanup (after the node is removed). - -- The image directory stores the container images to be imported. The container images must be in a tar package format that is compatible with Docker (for example, images exported by Docker or isula-build). - -- The pkg directory stores the rpm/deb packages to be installed, corresponding to the pkg package type. You are advised to use binary files to facilitate cross-release deployment. - -### Command Reference - -To utilize the cluster deployment tool provided by openEuler, use the eggo command to deploy the cluster. - -#### Deploying the Kubernetes Cluster - -Run the following command to deploy a Kubernetes cluster using the specified YAML configuration: - -**eggo deploy** \[ **-d** ] **-f** *deploy.yaml* - -| Parameter| Mandatory (Yes/No)| Description | -| ------------- | -------- | --------------------------------- | -| --debug \| -d | No| Displays the debugging information.| -| --file \| -f | Yes| Specifies the path of the YAML file for the Kubernetes cluster deployment.| - -#### Adding a Single Node - -Run the following command to add a specified single node to the Kubernetes cluster: - -**eggo** **join** \[ **-d** ] **--id** *k8s-cluster* \[ **--type** *master,worker* ] **--arch** *arm64* **--port** *22* \[ **--name** *master1*] *IP* - -| Parameter| Mandatory (Yes/No) | Description| -| ------------- | -------- | ------------------------------------------------------------ | -| --debug \| -d | No| Displays the debugging information.| -| --id | Yes| Specifies the name of the Kubernetes cluster where the node is to be added.| -| --type \| -t | No| Specifies the type of the node to be added. The value can be master or worker. Use commas (,) to separate multiple types. The default value is worker.| -| --arch \| -a | Yes| Specifies the CPU architecture of the node to be added.| -| --port \| -p | Yes| Specifies the port number for SSH login of the node to be added.| -| --name \| -n | No| Specifies the name of the node to be added.| -| *IP* | Yes| Actual IP address of the node to be added.| - -#### Adding Multiple Nodes - -Run the following command to add specified multiple nodes to the Kubernetes cluster: - -**eggo** **join** \[ **-d** ] **--id** *k8s-cluster* **-f** *nodes.yaml* - -| Parameter| Mandatory (Yes/No) | Description | -| ------------- | -------- | -------------------------------- | -| --debug \| -d | No| Displays the debugging information.| -| --id | Yes| Specifies the name of the Kubernetes cluster where the nodes are to be added.| -| --file \| -f | Yes| Specifies the path of the YAML configuration file for adding the nodes.| - -#### Deleting Nodes - -Run the following command to delete one or more nodes from the Kubernetes cluster: - -**eggo delete** \[ **-d** ] **--id** *k8s-cluster* *node* \[*node...*] - -| Parameter| Mandatory (Yes/No) | Description | -| ------------- | -------- | -------------------------------------------- | -| --debug \| -d | No| Displays the debugging information.| -| --id | Yes| Specifies the name of the cluster where the one or more nodes to be deleted are located.| -| *node* | Yes| Specifies the IP addresses or names of the one or more nodes to be deleted.| - -#### Deleting the Cluster - -Run the following command to delete the entire Kubernetes cluster: - -**eggo cleanup** \[ **-d** ] **--id** *k8s-cluster* \[ **-f** *deploy.yaml* ] - -| Parameter| Mandatory (Yes/No) | Description| -| ------------- | -------- | ------------------------------------------------------------ | -| --debug \| -d | No| Displays the debugging information.| -| --id | Yes| Specifies the name of the Kubernetes cluster to be deleted.| -| --file \| -f | No| Specifies the path of the YAML file for the Kubernetes cluster deletion. If this parameter is not specified, the cluster configuration cached during cluster deployment is used by default. In normal cases, you are advised not to set this parameter. Set this parameter only when an exception occurs.| - -> ![](./public_sys-resources/icon-note.gif)**NOTE**: -> -> - The cluster configuration cached during cluster deployment is recommended when you delete the cluster. That is, you are advised not to set the --file | -f parameter in normal cases. Set this parameter only when the cache configuration is damaged or lost due to an exception. - -#### Querying the Cluster - -Run the following command to query all Kubernetes clusters deployed using eggo: - -**eggo list** \[ **-d** ] - -| Parameter| Mandatory (Yes/No) | Description | -| ------------- | -------- | ------------ | -| --debug \| -d | No| Displays the debugging information.| - -#### Generating the Cluster Configuration File - -Run the following command to quickly generate the required YAML configuration file for the Kubernetes cluster deployment. - -**eggo template** **-d** **-f** *template.yaml* **-n** *k8s-cluster* **-u** *username* **-p** *password* **--etcd** \[*192.168.0.1,192.168.0.2*] **--masters** \[*192.168.0.1,192.168.0.2*] **--workers** *192.168.0.3* **--loadbalance** *192.168.0.4* - -| Parameter| Mandatory (Yes/No) | Description | -| ------------------- | -------- | ------------------------------- | -| --debug \| -d | No| Displays the debugging information.| -| --file \| -f | No| Specifies the path of the generated YAML file.| -| --name \| -n | No| Specifies the name of the Kubernetes cluster.| -| --username \| -u | No| Specifies the user name for SSH login of the configured node.| -| --password \| -p | No| Specifies the password for SSH login of the configured node.| -| --etcd | No| Specifies the IP address list of the etcd nodes.| -| --masters | No| Specifies the IP address list of the master nodes.| -| --workers | No| Specifies the IP address list of the worker nodes.| -| --loadbalance \| -l | No| Specifies the IP address of the loadbalance node.| - -#### Querying the Help Information - -Run the following command to query the help information of the eggo command: - - **eggo help** - -#### Querying the Help Information of Subcommands - -Run the following command to query the help information of the eggo subcommands: - -**eggo deploy | join | delete | cleanup | list | template -h** - -| Parameter| Mandatory (Yes/No) | Description | -| ----------- | -------- | ------------ | -| --help\| -h | Yes| Displays the help information.| diff --git a/docs/en/docs/Kubernetes/figures/arch.png b/docs/en/docs/Kubernetes/figures/arch.png deleted file mode 100644 index 650e4a67959136e12e49975196a4f3af28e6170e..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Kubernetes/figures/arch.png and /dev/null differ diff --git a/docs/en/docs/Kubernetes/installing-etcd.md b/docs/en/docs/Kubernetes/installing-etcd.md deleted file mode 100644 index 9bd37d031107a5b3c3f880db5a90bf5783b2935a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/installing-etcd.md +++ /dev/null @@ -1,89 +0,0 @@ -# Installing etcd - - -## Preparing the Environment - -Run the following command to enable the port used by etcd: -```bash -firewall-cmd --zone=public --add-port=2379/tcp -firewall-cmd --zone=public --add-port=2380/tcp -``` - -## Installing the etcd Binary Package - -Currently, the RPM package is used for installation. - -``` -rpm -ivh etcd*.rpm -``` - -Prepare the directories. - -```bash -mkdir -p /etc/etcd /var/lib/etcd -cp ca.pem /etc/etcd/ -cp kubernetes-key.pem /etc/etcd/ -cp kubernetes.pem /etc/etcd/ -# Disabling SELinux -setenforce 0 -# Disabling the Default Configuration of the /etc/etcd/etcd.conf File -# Commenting Out the Line, for example, ETCD_LISTEN_CLIENT_URLS="http://localhost:2379". -``` - -## Compiling the etcd.service File - -The following uses the `k8smaster0` machine as an example: - -```bash -$ cat /usr/lib/systemd/system/etcd.service -[Unit] -Description=Etcd Server -After=network.target -After=network-online.target -Wants=network-online.target - -[Service] -Type=notify -WorkingDirectory=/var/lib/etcd/ -EnvironmentFile=-/etc/etcd/etcd.conf -# set GOMAXPROCS to number of processors -ExecStart=/bin/bash -c "ETCD_UNSUPPORTED_ARCH=arm64 /usr/bin/etcd --name=k8smaster0 --cert-file=/etc/etcd/kubernetes.pem --key-file=/etc/etcd/kubernetes-key.pem --peer-cert-file=/etc/etcd/kubernetes.pem --peer-key-file=/etc/etcd/kubernetes-key.pem --trusted-ca-file=/etc/etcd/ca.pem --peer-trusted-ca-file=/etc/etcd/ca.pem --peer-client-cert-auth --client-cert-auth --initial-advertise-peer-urls https://192.168.122.154:2380 --listen-peer-urls https://192.168.122.154:2380 --listen-client-urls https://192.168.122.154:2379,https://127.0.0.1:2379 --advertise-client-urls https://192.168.122.154:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8smaster0=https://192.168.122.154:2380,k8smaster1=https://192.168.122.155:2380,k8smaster2=https://192.168.122.156:2380 --initial-cluster-state new --data-dir /var/lib/etcd" - -Restart=always -RestartSec=10s -LimitNOFILE=65536 - -[Install] -WantedBy=multi-user.target -``` - -**注意:** - -- The boot setting `ETCD_UNSUPPORTED_ARCH=arm64` needs to be added to ARM64; -- In this document, etcd and Kubernetes control are deployed on the same machine. Therefore, the `kubernetes.pem` and `kubernetes-key.pem` certificates are used to start etcd and Kubernetes control. -- A CA certificate is used in the entire deployment process. etcd can generate its own CA certificate and use its own CA certificate to sign other certificates. However, the certificate signed by the CA certificate needs to be used when the APIServer accesses the etcd client. -- `initial-cluster` needs to be added to all configurations for deploying etcd. -- To improve the storage efficiency of etcd, you can use the directory of the SSD as `data-dir`. - -Start the etcd service. - -```bash -$ systemctl enable etcd -$ systemctl start etcd -``` - -Then, deploy other hosts in sequence. - -## Verifying Basic Functions - -```bash -$ ETCDCTL_API=3 etcdctl -w table endpoint status --endpoints=https://192.168.122.155:2379,https://192.168.122.156:2379,https://192.168.122.154:2379 --cacert=/etc/etcd/ca.pem --cert=/etc/etcd/kubernetes.pem --key=/etc/etcd/kubernetes-key.pem -+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ -| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFTAPPLIED INDEX | ERRORS | -+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ -| https://192.168.122.155:2379 | b50ec873e253ebaa | 3.4.14 | 262 kB | false | false | 819 | 21 | 21 | | -| https://192.168.122.156:2379 | e2b0d126774c6d02 | 3.4.14 | 262 kB | true | false | 819 | 21 | 21 | | -| https://192.168.122.154:2379 | f93b3808e944c379 | 3.4.14 | 328 kB | false | false | 819 | 21 | 21 | | -+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ -``` - diff --git a/docs/en/docs/Kubernetes/installing-the-Kubernetes-software-package.md b/docs/en/docs/Kubernetes/installing-the-Kubernetes-software-package.md deleted file mode 100644 index e88f1adec2524cbf79e5556ce63bee85e5d1fa7f..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/installing-the-Kubernetes-software-package.md +++ /dev/null @@ -1,14 +0,0 @@ -# Installing the Kubernetes Software Package - - -```bash -$ dnf install -y docker conntrack-tools socat -``` - -After the EPOL source is configured, you can directly install Kubernetes through DNF. - -```bash -$ rpm -ivh kubernetes*.rpm -``` - - diff --git a/docs/en/docs/Kubernetes/preparing-VMs.md b/docs/en/docs/Kubernetes/preparing-VMs.md deleted file mode 100644 index 52b21caf6eb76e8ee4219640855f4adcf538bd68..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/preparing-VMs.md +++ /dev/null @@ -1,150 +0,0 @@ -# Preparing VMs - -This document describes how to use virt manager to install a VM. Ignore if your VM is prepared. - -## Installing Dependency Tools - -VM installation depends on related tools. The following command is an example for installing the dependency and enabling the libvirtd service. (If a proxy is required, configure the proxy first.) - -```bash -dnf install virt-install virt-manager libvirt-daemon-qemu edk2-aarch64.noarch virt-viewer -systemctl start libvirtd -systemctl enable libvirtd -``` - -## Preparing VM Disk Files - -```bash -dnf install -y qemu-img -virsh pool-define-as vmPool --type dir --target /mnt/vm/images/ -virsh pool-build vmPool -virsh pool-start vmPool -virsh pool-autostart vmPool -virsh vol-create-as --pool vmPool --name master0.img --capacity 200G --allocation 1G --format qcow2 -virsh vol-create-as --pool vmPool --name master1.img --capacity 200G --allocation 1G --format qcow2 -virsh vol-create-as --pool vmPool --name master2.img --capacity 200G --allocation 1G --format qcow2 -virsh vol-create-as --pool vmPool --name node1.img --capacity 300G --allocation 1G --format qcow2 -virsh vol-create-as --pool vmPool --name node2.img --capacity 300G --allocation 1G --format qcow2 -virsh vol-create-as --pool vmPool --name node3.img --capacity 300G --allocation 1G --format qcow2 -``` - -## Enabling Firewall Ports - -**Method 1** - -1. Query a port. - - ```shell - netstat -lntup | grep qemu-kvm - ``` - -2. Enable the VNC firewall port. For example, if the port number starts from 5900, run the following commands: - - ```shell - firewall-cmd --zone=public --add-port=5900/tcp - firewall-cmd --zone=public --add-port=5901/tcp - firewall-cmd --zone=public --add-port=5902/tcp - firewall-cmd --zone=public --add-port=5903/tcp - firewall-cmd --zone=public --add-port=5904/tcp - firewall-cmd --zone=public --add-port=5905/tcp - ``` - -**Method 2** - -Disable the firewall. - -```shell -systemctl stop firewalld -``` - -## Preparing the VM Configuration File - -A VM configuration file is required for creating a VM. For example, if the configuration file is master.xml and the host name of the VM is k8smaster0, the configuration is as follows: - -```bash - cat master.xml - - - k8smaster0 - 8 - 8 - - hvm - /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw - /var/lib/libvirt/qemu/nvram/k8smaster0.fd - - - - - - - - - 1 - - destroy - restart - restart - - /usr/libexec/qemu-kvm - - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -The VM configuration must be unique. Therefore, you need to modify the following to ensure that the VM is unique: - -- name: host name of the VM. You are advised to use lowercase letters. In this example, the value is `k8smaster0`. -- nvram: handle file path of the NVRAM, which must be globally unique. In this example, the value is `/var/lib/libvirt/qemu/nvram/k8smaster0.fd`. -- disk source file: VM disk file path. In this example, the value is `/mnt/vm/images/master0.img`. -- mac address of the interface: MAC address of the interface. In this example, the value is `52:54:00:00:00:80`. - -## Installing a VM - -1. Create and start a VM. - - ```shell - virsh define master.xml - virsh start k8smaster0 - ``` - -2. Obtain the VNC port number of the VM. - - ```shell - virsh vncdisplay k8smaster0 - ``` - -3. Use a VM connection tool, such as VNC Viewer, to remotely connect to the VM and perform configurations as prompted. - -4. Set the host name of the VM, for example, k8smaster0. - - ```shell - hostnamectl set-hostname k8smaster0 - ``` diff --git a/docs/en/docs/Kubernetes/preparing-certificates.md b/docs/en/docs/Kubernetes/preparing-certificates.md deleted file mode 100644 index afe150e05387fb94dd98c089e96655e3b74f56d5..0000000000000000000000000000000000000000 --- a/docs/en/docs/Kubernetes/preparing-certificates.md +++ /dev/null @@ -1,413 +0,0 @@ - -# Preparing Certificates - -**Statement: The certificate used in this document is self-signed and cannot be used in a commercial environment.** - -Before deploying a cluster, you need to generate certificates required for communication between components in the cluster. This document uses the open-source CFSSL as the verification and deployment tool to help users understand the certificate configuration and the association between certificates of cluster components. You can select a tool based on the site requirements, for example, OpenSSL. - -## Building and Installing CFSSL - -The following commands for building and installing CFSSL are for your reference (the CFSSL website access permission is required, and the proxy must be configured first): - -```bash -wget --no-check-certificate https://github.com/cloudflare/cfssl/archive/v1.5.0.tar.gz -tar -zxf v1.5.0.tar.gz -cd cfssl-1.5.0/ -make -j6 -cp bin/* /usr/local/bin/ -``` - -## Generating a Root Certificate - -Compile the CA configuration file, for example, ca-config.json: - -```bash -$ cat ca-config.json | jq -{ - "signing": { - "default": { - "expiry": "8760h" - }, - "profiles": { - "kubernetes": { - "usages": [ - "signing", - "key encipherment", - "server auth", - "client auth" - ], - "expiry": "8760h" - } - } - } -} -``` - -Compile a CA CSR file, for example, ca-csr.json: - -```bash -$ cat ca-csr.json | jq -{ - "CN": "Kubernetes", - "key": { - "algo": "rsa", - "size": 2048 - }, - "names": [ - { - "C": "CN", - "L": "HangZhou", - "O": "openEuler", - "OU": "WWW", - "ST": "BinJiang" - } - ] -} -``` - -Generate the CA certificate and key: - -```bash -cfssl gencert -initca ca-csr.json | cfssljson -bare ca -``` - -The following certificates are obtained: - -```bash -ca.csr ca-key.pem ca.pem -``` - -## Generating the admin Account Certificate - -admin is an account used by K8S for system management. Compile the CSR configuration of the admin account, for example, admin-csr.json: - -```bash -cat admin-csr.json | jq -{ - "CN": "admin", - "key": { - "algo": "rsa", - "size": 2048 - }, - "names": [ - { - "C": "CN", - "L": "HangZhou", - "O": "system:masters", - "OU": "Containerum", - "ST": "BinJiang" - } - ] -} -``` - -Generate a certificate: - -```bash -cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin -``` - -The result is as follows: - -```bash -admin.csr admin-key.pem admin.pem -``` - -## Generating a service-account Certificate - -Compile the CSR configuration file of the service-account account, for example, service-account-csr.json: - -```bash -cat service-account-csr.json | jq -{ - "CN": "service-accounts", - "key": { - "algo": "rsa", - "size": 2048 - }, - "names": [ - { - "C": "CN", - "L": "HangZhou", - "O": "Kubernetes", - "OU": "openEuler k8s install", - "ST": "BinJiang" - } - ] -} -``` - -Generate a certificate: - -```bash -cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes service-account-csr.json | cfssljson -bare service-account -``` - -The result is as follows: - -```bash -service-account.csr service-account-key.pem service-account.pem -``` - -## Generating the kube-controller-manager Certificate - -Compile the CSR configuration of kube-controller-manager: - -```bash -{ - "CN": "system:kube-controller-manager", - "key": { - "algo": "rsa", - "size": 2048 - }, - "names": [ - { - "C": "CN", - "L": "HangZhou", - "O": "system:kube-controller-manager", - "OU": "openEuler k8s kcm", - "ST": "BinJiang" - } - ] -} -``` - -Generate a certificate: - -```bash -cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager -``` - -The result is as follows: - -```bash -kube-controller-manager.csr kube-controller-manager-key.pem kube-controller-manager.pem -``` - -## Generating the kube-proxy Certificate - -Compile the CSR configuration of kube-proxy: - -```bash -{ - "CN": "system:kube-proxy", - "key": { - "algo": "rsa", - "size": 2048 - }, - "names": [ - { - "C": "CN", - "L": "HangZhou", - "O": "system:node-proxier", - "OU": "openEuler k8s kube proxy", - "ST": "BinJiang" - } - ] -} -``` - -Generate a certificate: - -```bash -cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy -``` - -The result is as follows: - -```bash -kube-proxy.csr kube-proxy-key.pem kube-proxy.pem -``` - -## Generating the kube-scheduler Certificate - -Compile the CSR configuration of kube-scheduler: - -```bash -{ - "CN": "system:kube-scheduler", - "key": { - "algo": "rsa", - "size": 2048 - }, - "names": [ - { - "C": "CN", - "L": "HangZhou", - "O": "system:kube-scheduler", - "OU": "openEuler k8s kube scheduler", - "ST": "BinJiang" - } - ] -} -``` - -Generate a certificate: - -```bash -cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler -``` - -The result is as follows: - -```bash -kube-scheduler.csr kube-scheduler-key.pem kube-scheduler.pem -``` - -## Generating the kubelet Certificate - -The certificate involves the host name and IP address of the server where kubelet is located. Therefore, the configuration of each node is different. The script is compiled as follows: - -```bash -$ cat node_csr_gen.bash - -#!/bin/bash - -nodes=(k8snode1 k8snode2 k8snode3) -IPs=("192.168.122.157" "192.168.122.158" "192.168.122.159") - -for i in "${!nodes[@]}"; do - -cat > "${nodes[$i]}-csr.json" < transpiled_config.ign -``` - -### Installing NestOS - -Use SCP to copy the **config.ign** file generated by the host machine to NestOS that is initially started, which is not installed to the disk and runs in the memory. - -``` -sudo -i -scp root@your_ipAddress:/root/config.ign /root -``` - -Run the following command and complete the installation as prompted: - -``` -nestos-installer install /dev/sda --ignition-file config.ign -``` - -After the installation is complete, restart NestOS. - -``` -systemctl reboot -``` -Complete. diff --git a/docs/en/docs/NestOS/overview.md b/docs/en/docs/NestOS/overview.md deleted file mode 100644 index a247a64c9fabc01cc59173d42a213bd2d4181c52..0000000000000000000000000000000000000000 --- a/docs/en/docs/NestOS/overview.md +++ /dev/null @@ -1,4 +0,0 @@ -# NestOS User Guide - -This document describes the installation, deployment, features, and usage of the NestOS cloud-based operating system. NestOS runs common container engines, such as Docker, iSula, PodMan, and CRI-O, and integrates technologies such as Ignition, rpm-ostree, OCI runtime, and SELinux. NestOS adopts the design principles of dual-system partitions, container technology, and cluster architecture. It can adapt to multiple basic running environments in cloud scenarios.In addition, NestOS optimizes Kubernetes and provides support for platforms such as OpenStack and oVirt for IaaS ecosystem construction. In terms of PaaS ecosystem construction, platforms such as OKD and Rancher are supported for easy deployment of clusters and secure running of large-scale containerized workloads. To download NestOS images, visit the [NestOS Repository](https://gitee.com/openeuler/NestOS). - diff --git a/docs/en/docs/NestOS/usage.md b/docs/en/docs/NestOS/usage.md deleted file mode 100644 index 84da85fa2216f2c293a7bfdbb353787e0d75db59..0000000000000000000000000000000000000000 --- a/docs/en/docs/NestOS/usage.md +++ /dev/null @@ -1,904 +0,0 @@ -# Container-based Kubernetes Deployment Using NestOS - -​ - -## Solution Overview - -Kubernetes (K8s) is a portable container orchestration and management tool developed for container services. This guide provides a solution for quickly deploying Kubernetes containers using NestOS. In this solution, multiple NestOS nodes are created on the virtualization platform as the verification environment for the Kubernetes cluster deployment. The environment required by Kubernetes is configured in a YAML-formatted Ignition configuration file in advance. The resources required by Kubernetes are deployed and nodes are created when NestOS is installed. In the bare metal environment, you can also deploy Kubernetes clusters by referring to this document and the NestOS bare metal installation document. - -- Software versions - - - NestOS image: 22.09 - - - Kubernetes: v1.23.10 - - - isulad: 2.0.16 - -- Installation requirements - - Each machine has 2 GB or more RAM and 2 or more CPU cores. - - All machines in the cluster can communicate with each other. - - Each node has a unique host name. - - The external network can be accessed for pulling images. - - The swap partition disabled. - - SELinux is disabled. -- Deployment content - - NestOS image that integrates isulad, kubeadm, kubelet, kubectl, and other binary files - - Kubernetes master node - - Container network plugins - - Kubernetes nodes to be added to the Kubernetes cluster - -## Kubernetes Node Configuration - -NestOS uses Ignition to implement batch node configuration. This section describes how to generate an Ignition file and provides an Ignition configuration example for deploying Kubernetes containers. The system configurations of a NestOS node are as follows: - -| Item | Description | -| ------------ | -------------------------------------- | -| passwd | Configures the node login user and access authentication information | -| hostname | Configures the host name of a node | -| Time zone | Configures the default time zone of a node | -| Kernel parameters | Some kernel parameters need to be enabled for Kubernetes deployment. | -| SELinux | SELinux needs to be disabled for Kubernetes deployment. | -| Time synchronization| The chronyd service is used to synchronize the cluster time in the Kubernetes environment.| - -### Generating a Login Password - -To access a NestOS instance using a password, run the following command to generate **${PASSWORD_HASH}** for Ignition configuration: - -``` -openssl passwd -1 -salt yoursalt -``` - -### Generating an SSH Key Pair - -To access a NestOS instance using an SSH public key, run the following command to generate an SSH key pair: - -``` -ssh-keygen -N '' -f /root/.ssh/id_rsa -``` - -View the public key file **id_rsa.pub** and obtain the SSH public key information for Ignition configuration: - -``` -cat /root/.ssh/id_rsa.pub -``` - -### Compiling the Butane Configuration File - -Configure the following fields in the configuration file example below based on the actual deployment. See the sections above for how to generate values of some fields. - -- **${PASSWORD_HASH}**: password for logging in to the node -- **${SSH-RSA}**: public key of the node -- **${MASTER_NAME}**: host name of the master node -- **${MASTER_IP}**: IP address of the master node -- **${MASTER_SEGMENT}**: Subnet where the master node is located -- **${NODE_NAME}**: host name of the node -- **${NODE_IP}**: IP address of the node -- **${GATEWAY}**: gateway of the node -- **${service-cidr}**: IP address range allocated to the services -- **${pod-network-cidr}**: IP address range allocated to the pods -- **${image-repository}**: image registry address, for example, https://registry.cn-hangzhou.aliyuncs.com -- **${token}**: token information for joining the cluster, which is obtained from the master node - -Example Butane configuration file for the master node: - -```yaml -variant: fcos -version: 1.1.0 -## Password-related configurations -passwd: - users: - - name: root - ## Password - password_hash: "${PASSWORD_HASH}" - "groups": [ - "adm", - "sudo", - "systemd-journal", - "wheel" - ] - ## SSH public key - ssh_authorized_keys: - - "${SSH-RSA}" -storage: - directories: - - path: /etc/systemd/system/kubelet.service.d - overwrite: true - files: - - path: /etc/hostname - mode: 0644 - contents: - inline: ${MASTER_NAME} - - path: /etc/hosts - mode: 0644 - overwrite: true - contents: - inline: | - 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 - ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 - ${MASTER_IP} ${MASTER_NAME} - ${NODE_IP} ${NODE_NAME} - - path: /etc/NetworkManager/system-connections/ens2.nmconnection - mode: 0600 - overwrite: true - contents: - inline: | - [connection] - id=ens2 - type=ethernet - interface-name=ens2 - [ipv4] - address1=${MASTER_IP}/24,${GATEWAY} - dns=8.8.8.8 - dns-search= - method=manual - - path: /etc/sysctl.d/kubernetes.conf - mode: 0644 - overwrite: true - contents: - inline: | - net.bridge.bridge-nf-call-iptables=1 - net.bridge.bridge-nf-call-ip6tables=1 - net.ipv4.ip_forward=1 - - path: /etc/isulad/daemon.json - mode: 0644 - overwrite: true - contents: - inline: | - { - "exec-opts": ["native.cgroupdriver=systemd"], - "group": "isula", - "default-runtime": "lcr", - "graph": "/var/lib/isulad", - "state": "/var/run/isulad", - "engine": "lcr", - "log-level": "ERROR", - "pidfile": "/var/run/isulad.pid", - "log-opts": { - "log-file-mode": "0600", - "log-path": "/var/lib/isulad", - "max-file": "1", - "max-size": "30KB" - }, - "log-driver": "stdout", - "container-log": { - "driver": "json-file" - }, - "hook-spec": "/etc/default/isulad/hooks/default.json", - "start-timeout": "2m", - "storage-driver": "overlay2", - "storage-opts": [ - "overlay2.override_kernel_check=true" - ], - "registry-mirrors": [ - "docker.io" - ], - "insecure-registries": [ - "${image-repository}" - ], - "pod-sandbox-image": "k8s.gcr.io/pause:3.6", - "native.umask": "secure", - "network-plugin": "cni", - "cni-bin-dir": "/opt/cni/bin", - "cni-conf-dir": "/etc/cni/net.d", - "image-layer-check": false, - "use-decrypted-key": true, - "insecure-skip-verify-enforce": false, - "cri-runtimes": { - "kata": "io.containerd.kata.v2" - } - } - - path: /root/pull_images.sh - mode: 0644 - overwrite: true - contents: - inline: | - #!/bin/sh - KUBE_VERSION=v1.23.10 - KUBE_PAUSE_VERSION=3.6 - ETCD_VERSION=3.5.1-0 - DNS_VERSION=v1.8.6 - CALICO_VERSION=v3.19.4 - username=${image-repository} - images=( - kube-proxy:${KUBE_VERSION} - kube-scheduler:${KUBE_VERSION} - kube-controller-manager:${KUBE_VERSION} - kube-apiserver:${KUBE_VERSION} - pause:${KUBE_PAUSE_VERSION} - etcd:${ETCD_VERSION} - ) - for image in ${images[@]} - do - isula pull ${username}/${image} - isula tag ${username}/${image} k8s.gcr.io/${image} - isula rmi ${username}/${image} - done - isula pull ${username}/coredns:${DNS_VERSION} - isula tag ${username}/coredns:${DNS_VERSION} k8s.gcr.io/coredns/coredns:${DNS_VERSION} - isula rmi ${username}/coredns:${DNS_VERSION} - isula pull calico/node:${CALICO_VERSION} - isula pull calico/cni:${CALICO_VERSION} - isula pull calico/kube-controllers:${CALICO_VERSION} - isula pull calico/pod2daemon-flexvol:${CALICO_VERSION} - touch /var/log/pull-images.stamp - - path: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf - mode: 0644 - contents: - inline: | - # Note: This dropin only works with kubeadm and kubelet v1.11+ - [Service] - Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf" - Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" - # This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically - EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env - # This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use - # the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file. - EnvironmentFile=-/etc/sysconfig/kubelet - ExecStart= - ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS - - path: /root/init-config.yaml - mode: 0644 - contents: - inline: | - apiVersion: kubeadm.k8s.io/v1beta2 - kind: InitConfiguration - nodeRegistration: - criSocket: /var/run/isulad.sock - name: k8s-master01 - kubeletExtraArgs: - volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/" - --- - apiVersion: kubeadm.k8s.io/v1beta2 - kind: ClusterConfiguration - controllerManager: - extraArgs: - flex-volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/" - kubernetesVersion: v1.23.10 - imageRepository: k8s.gcr.io - controlPlaneEndpoint: "${MASTER_IP}:6443" - networking: - serviceSubnet: "${service-cidr}" - podSubnet: "${pod-network-cidr}" - dnsDomain: "cluster.local" - dns: - type: CoreDNS - imageRepository: k8s.gcr.io/coredns - imageTag: v1.8.6 - links: - - path: /etc/localtime - target: ../usr/share/zoneinfo/Asia/Shanghai - -systemd: - units: - - name: kubelet.service - enabled: true - contents: | - [Unit] - Description=kubelet: The Kubernetes Node Agent - Documentation=https://kubernetes.io/docs/ - Wants=network-online.target - After=network-online.target - - [Service] - ExecStart=/usr/bin/kubelet - Restart=always - StartLimitInterval=0 - RestartSec=10 - - [Install] - WantedBy=multi-user.target - - - name: set-kernel-para.service - enabled: true - contents: | - [Unit] - Description=set kernel para for Kubernetes - ConditionPathExists=!/var/log/set-kernel-para.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=modprobe br_netfilter - ExecStart=sysctl -p /etc/sysctl.d/kubernetes.conf - ExecStart=/bin/touch /var/log/set-kernel-para.stamp - - [Install] - WantedBy=multi-user.target - - - name: pull-images.service - enabled: true - contents: | - [Unit] - Description=pull images for kubernetes - ConditionPathExists=!/var/log/pull-images.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=systemctl start isulad - ExecStart=systemctl enable isulad - ExecStart=sh /root/pull_images.sh - - [Install] - WantedBy=multi-user.target - - - name: disable-selinux.service - enabled: true - contents: | - [Unit] - Description=disable selinux for kubernetes - ConditionPathExists=!/var/log/disable-selinux.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=bash -c "sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config" - ExecStart=setenforce 0 - ExecStart=/bin/touch /var/log/disable-selinux.stamp - - [Install] - WantedBy=multi-user.target - - - name: set-time-sync.service - enabled: true - contents: | - [Unit] - Description=set time sync for kubernetes - ConditionPathExists=!/var/log/set-time-sync.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=bash -c "sed -i '3aserver ntp1.aliyun.com iburst' /etc/chrony.conf" - ExecStart=bash -c "sed -i '24aallow ${MASTER_SEGMENT}' /etc/chrony.conf" - ExecStart=bash -c "sed -i '26alocal stratum 10' /etc/chrony.conf" - ExecStart=systemctl restart chronyd.service - ExecStart=/bin/touch /var/log/set-time-sync.stamp - - [Install] - WantedBy=multi-user.target - - - name: init-cluster.service - enabled: true - contents: | - [Unit] - Description=init kubernetes cluster - Requires=set-kernel-para.service pull-images.service disable-selinux.service set-time-sync.service - After=set-kernel-para.service pull-images.service disable-selinux.service set-time-sync.service - ConditionPathExists=/var/log/set-kernel-para.stamp - ConditionPathExists=/var/log/set-time-sync.stamp - ConditionPathExists=/var/log/disable-selinux.stamp - ConditionPathExists=/var/log/pull-images.stamp - ConditionPathExists=!/var/log/init-k8s-cluster.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=kubeadm init --config=/root/init-config.yaml --upload-certs - ExecStart=/bin/touch /var/log/init-k8s-cluster.stamp - - [Install] - WantedBy=multi-user.target - - - - name: install-cni-plugin.service - enabled: true - contents: | - [Unit] - Description=install cni network plugin for kubernetes - Requires=init-cluster.service - After=init-cluster.service - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=bash -c "curl https://docs.projectcalico.org/v3.19/manifests/calico.yaml -o /root/calico.yaml" - ExecStart=/bin/sleep 6 - ExecStart=bash -c "sed -i 's#usr/libexec/#opt/libexec/#g' /root/calico.yaml" - ExecStart=kubectl apply -f /root/calico.yaml --kubeconfig=/etc/kubernetes/admin.conf - - [Install] - WantedBy=multi-user.target - -``` - -Example Butane configuration file for a node: - -```yaml -variant: fcos -version: 1.1.0 -passwd: - users: - - name: root - password_hash: "${PASSWORD_HASH}" - "groups": [ - "adm", - "sudo", - "systemd-journal", - "wheel" - ] - ssh_authorized_keys: - - "${SSH-RSA}" -storage: - directories: - - path: /etc/systemd/system/kubelet.service.d - overwrite: true - files: - - path: /etc/hostname - mode: 0644 - contents: - inline: ${NODE_NAME} - - path: /etc/hosts - mode: 0644 - overwrite: true - contents: - inline: | - 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 - ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 - ${MASTER_IP} ${MASTER_NAME} - ${NODE_IP} ${NODE_NAME} - - path: /etc/NetworkManager/system-connections/ens2.nmconnection - mode: 0600 - overwrite: true - contents: - inline: | - [connection] - id=ens2 - type=ethernet - interface-name=ens2 - [ipv4] - address1=${NODE_IP}/24,${GATEWAY} - dns=8.8.8.8; - dns-search= - method=manual - - path: /etc/sysctl.d/kubernetes.conf - mode: 0644 - overwrite: true - contents: - inline: | - net.bridge.bridge-nf-call-iptables=1 - net.bridge.bridge-nf-call-ip6tables=1 - net.ipv4.ip_forward=1 - - path: /etc/isulad/daemon.json - mode: 0644 - overwrite: true - contents: - inline: | - { - "exec-opts": ["native.cgroupdriver=systemd"], - "group": "isula", - "default-runtime": "lcr", - "graph": "/var/lib/isulad", - "state": "/var/run/isulad", - "engine": "lcr", - "log-level": "ERROR", - "pidfile": "/var/run/isulad.pid", - "log-opts": { - "log-file-mode": "0600", - "log-path": "/var/lib/isulad", - "max-file": "1", - "max-size": "30KB" - }, - "log-driver": "stdout", - "container-log": { - "driver": "json-file" - }, - "hook-spec": "/etc/default/isulad/hooks/default.json", - "start-timeout": "2m", - "storage-driver": "overlay2", - "storage-opts": [ - "overlay2.override_kernel_check=true" - ], - "registry-mirrors": [ - "docker.io" - ], - "insecure-registries": [ - "${image-repository}" - ], - "pod-sandbox-image": "k8s.gcr.io/pause:3.6", - "native.umask": "secure", - "network-plugin": "cni", - "cni-bin-dir": "/opt/cni/bin", - "cni-conf-dir": "/etc/cni/net.d", - "image-layer-check": false, - "use-decrypted-key": true, - "insecure-skip-verify-enforce": false, - "cri-runtimes": { - "kata": "io.containerd.kata.v2" - } - } - - path: /root/pull_images.sh - mode: 0644 - overwrite: true - contents: - inline: | - #!/bin/sh - KUBE_VERSION=v1.23.10 - KUBE_PAUSE_VERSION=3.6 - ETCD_VERSION=3.5.1-0 - DNS_VERSION=v1.8.6 - CALICO_VERSION=v3.19.4 - username=${image-repository} - images=( - kube-proxy:${KUBE_VERSION} - kube-scheduler:${KUBE_VERSION} - kube-controller-manager:${KUBE_VERSION} - kube-apiserver:${KUBE_VERSION} - pause:${KUBE_PAUSE_VERSION} - etcd:${ETCD_VERSION} - ) - for image in ${images[@]} - do - isula pull ${username}/${image} - isula tag ${username}/${image} k8s.gcr.io/${image} - isula rmi ${username}/${image} - done - isula pull ${username}/coredns:${DNS_VERSION} - isula tag ${username}/coredns:${DNS_VERSION} k8s.gcr.io/coredns/coredns:${DNS_VERSION} - isula rmi ${username}/coredns:${DNS_VERSION} - touch /var/log/pull-images.stamp - - path: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf - mode: 0644 - contents: - inline: | - # Note: This dropin only works with kubeadm and kubelet v1.11+ - [Service] - Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf" - Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" - # This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically - EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env - # This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use - # the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file. - EnvironmentFile=-/etc/sysconfig/kubelet - ExecStart= - ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS - - path: /root/join-config.yaml - mode: 0644 - contents: - inline: | - apiVersion: kubeadm.k8s.io/v1beta3 - caCertPath: /etc/kubernetes/pki/ca.crt - discovery: - bootstrapToken: - apiServerEndpoint: ${MASTER_IP}:6443 - token: ${token} - unsafeSkipCAVerification: true - timeout: 5m0s - tlsBootstrapToken: ${token} - kind: JoinConfiguration - nodeRegistration: - criSocket: /var/run/isulad.sock - imagePullPolicy: IfNotPresent - name: ${NODE_NAME} - taints: null - links: - - path: /etc/localtime - target: ../usr/share/zoneinfo/Asia/Shanghai - -systemd: - units: - - name: kubelet.service - enabled: true - contents: | - [Unit] - Description=kubelet: The Kubernetes Node Agent - Documentation=https://kubernetes.io/docs/ - Wants=network-online.target - After=network-online.target - - [Service] - ExecStart=/usr/bin/kubelet - Restart=always - StartLimitInterval=0 - RestartSec=10 - - [Install] - WantedBy=multi-user.target - - - name: set-kernel-para.service - enabled: true - contents: | - [Unit] - Description=set kernel para for kubernetes - ConditionPathExists=!/var/log/set-kernel-para.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=modprobe br_netfilter - ExecStart=sysctl -p /etc/sysctl.d/kubernetes.conf - ExecStart=/bin/touch /var/log/set-kernel-para.stamp - - [Install] - WantedBy=multi-user.target - - - name: pull-images.service - enabled: true - contents: | - [Unit] - Description=pull images for kubernetes - ConditionPathExists=!/var/log/pull-images.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=systemctl start isulad - ExecStart=systemctl enable isulad - ExecStart=sh /root/pull_images.sh - - [Install] - WantedBy=multi-user.target - - - name: disable-selinux.service - enabled: true - contents: | - [Unit] - Description=disable selinux for kubernetes - ConditionPathExists=!/var/log/disable-selinux.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=bash -c "sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config" - ExecStart=setenforce 0 - ExecStart=/bin/touch /var/log/disable-selinux.stamp - - [Install] - WantedBy=multi-user.target - - - name: set-time-sync.service - enabled: true - contents: | - [Unit] - Description=set time sync for kubernetes - ConditionPathExists=!/var/log/set-time-sync.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=bash -c "sed -i '3aserver ${MASTER_IP}' /etc/chrony.conf" - ExecStart=systemctl restart chronyd.service - ExecStart=/bin/touch /var/log/set-time-sync.stamp - - [Install] - WantedBy=multi-user.target - - - name: join-cluster.service - enabled: true - contents: | - [Unit] - Description=node join kubernetes cluster - Requires=set-kernel-para.service pull-images.service disable-selinux.service set-time-sync.service - After=set-kernel-para.service pull-images.service disable-selinux.service set-time-sync.service - ConditionPathExists=/var/log/set-kernel-para.stamp - ConditionPathExists=/var/log/set-time-sync.stamp - ConditionPathExists=/var/log/disable-selinux.stamp - ConditionPathExists=/var/log/pull-images.stamp - - [Service] - Type=oneshot - RemainAfterExit=yes - ExecStart=kubeadm join --config=/root/join-config.yaml - - [Install] - WantedBy=multi-user.target - -``` - -### Generating an Ignition File - -To facilitate reading and writing of Ignition configurations for human users, Ignition provides conversion from a YAML-formatted Butane file to a JSON-formatted Ignition file, which is used to boot the NestOS image. Run the following command to convert a Butane configuration file to an Ignition configuration file: - -``` -podman run --interactive --rm quay.io/coreos/butane:release --pretty --strict < your_config.bu > transpiled_config.ign -``` - - - -## Kubernetes Cluster Setup - -Run the following command to create the master node of the Kubernetes cluster based on the Ignition file generated in the previous section. You can adjust the `vcpus`, `ram`, and `disk` parameters. For details, see the virt-install manual. - -``` -virt-install --name=${NAME} --vcpus=4 --ram=8192 --import --network=bridge=virbr0 --graphics=none --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_FILE_PATH}" --disk=size=40,backing_store=${NESTOS_RELEASE_QCOW2_PATH} --network=bridge=virbr1 --disk=size=40 -``` - -After NestOS is successfully installed on the master node, a series of environment configuration services are started in the background: set-kernel-para.service configures kernel parameters, pull-images.service pulls images required by the cluster, disable-selinux.service disables SELinux, set-time-sync.service sets time synchronization, init-cluster.service initializes the cluster, and then install-cni-plugin.service installs CNI network plugins. Wait a few minutes for the cluster to pull images. - -Run the `kubectl get pods -A` command to check whether all pods are in the running state. - - -Run the following command on the master node to view the token: - -``` -kubeadm token list -``` - -Add the queried token information to the Ignition file of the node and use the Ignition file to create the node. After the node is created, run the `kubectl get nodes` command on the master node to check whether the node is added to the cluster. - -If yes, Kubernetes is successfully deployed. - -# Using rpm-ostree - -## Installing Software Packages Using rpm-ostree - -Install wget. - -``` -rpm-ostree install wget -``` - -Restart the system. During the startup, use the up and down arrow keys on the keyboard to enter system before or after the RPM package installation. **ostree:0** indicates the version after the installation. - -``` -systemctl reboot -``` - -Check whether wget is successfully installed. - -``` -rpm -qa | grep wget -``` - -## Manually Upgrading NestOS Using rpm-ostree - -Run the following command in NestOS to view the current rpm-ostree status and version: - -``` -rpm-ostree status -``` - -Run the check command to check whether a new version is available. - -``` -rpm-ostree upgrade --check -``` - -Preview the differences between the versions. - -``` -rpm-ostree upgrade --preview -``` - -In the latest version, the nano package is imported. -Run the following command to download the latest OSTree and RPM data without performing the deployment. - -``` -rpm-ostree upgrade --download-only -``` - -Restart NestOS. After the restart, the old and new versions of the system are available. Enter the latest version. - -``` -rpm-ostree upgrade --reboot -``` - -## Comparing NestOS Versions - -Check the status. Ensure that two versions of OSTree exist: **LTS.20210927.dev.0** and **LTS.20210928.dev.0**. - -``` -rpm-ostree status -``` - -Compare the OSTree versions based on commit IDs. - -``` -rpm-ostree db diff 55eed9bfc5ec fe2408e34148 -``` - -## Rolling Back the System - -When a system upgrade is complete, the previous NestOS deployment is still stored on the disk. If the upgrade causes system problems, you can roll back to the previous deployment. - -### Temporary Rollback - -To temporarily roll back to the previous OS deployment, hold down **Shift** during system startup. When the boot load menu is displayed, select the corresponding branch from the menu. - -### Permanent Rollback - -To permanently roll back to the previous OS deployment, log in to the target node and run the `rpm-ostree rollback` command. This operation sets the previous OS deployment as the default deployment to boot into. -Run the following command to roll back to the system before the upgrade: - -``` -rpm-ostree rollback -``` - - -## Switching Versions - -NestOS is rolled back to an older version. You can run the following command to switch the rpm-ostree version used by NestOS to a newer version. - -``` -rpm-ostree deploy -r 22.03.20220325.dev.0 -``` - -After the restart, check whether NestOS uses the latest OSTree version. - - -# Using Zincati for Automatic Update - -Zincati automatically updates NestOS. Zincati uses the Cincinnati backend to check whether a new version is available. If a new version is available, Zincati downloads it using rpm-ostree. - -Currently, the Zincati automatic update service is disabled by default. You can modify the configuration file to set the automatic startup upon system startup for Zincati. - -``` -vi /etc/zincati/config.d/95-disable-on-dev.toml -``` - -Set **updates.enabled** to true. -Create a configuration file to specify the address of the Cincinnati backend. - -``` -vi /etc/zincati/config.d/update-cincinnati.toml -``` - -Add the following content: - -``` -[cincinnati] -base_url="http://nestos.org.cn:8080" -``` - -Restart the Zincati service. - -``` -systemctl restart zincati.service -``` - -When a new version is available, Zincati automatically detects the new version. Check the rpm-ostree status. If the status is **busy**, the system is being upgraded. - -After a period of time, NestOS automatically restarts. Log in to NestOS again and check the rpm-ostree status. If the status changes to **idle** and the current version is **20220325**, rpm-ostree has been upgraded. - -View the zincati service logs to check the upgrade process and system restart logs. In addition, the information "auto-updates logic enabled" in the logs indicates that the update is automatic. - -# Customizing NestOS - -You can use the nestos-installer tool to customize the original NestOS ISO file and package the Ignition file to generate a customized NestOS ISO file. The customized NestOS ISO file can be used to automatically install NestOS after the system is started for easy installation. - -Before customizing NestOS, make the following preparations: - -- Downloading the NestOS ISO. -- Preparing a **config.ign** File. - -## Generating a Customized NestOS ISO File - -### Setting Parameter Variables - -``` -$ export COREOS_ISO_ORIGIN_FILE=nestos-22.03.20220324.x86_64.iso -$ export COREOS_ISO_CUSTOMIZED_FILE=my-nestos.iso -$ export IGN_FILE=config.ign -``` - -### Checking the ISO File - -Ensure that the original NestOS ISO file does not contain the Ignition configuration. - -``` -$ nestos-installer iso ignition show $COREOS_ISO_ORIGIN_FILE - -Error: No embedded Ignition config. -``` - -### Generating a Customized NestOS ISO File - -Package the Ignition file into the original NestOS ISO file to generate a customized NestOS ISO file. - -``` -$ nestos-installer iso ignition embed $COREOS_ISO_ORIGIN_FILE --ignition-file $IGN_FILE $COREOS_ISO_ORIGIN_FILE --output $COREOS_ISO_CUSTOMIZED_FILE -``` - -### Checking the ISO File - -Ensure that the customized NestOS ISO file contains the Ignition configuration. - -``` -$ nestos-installer iso ignition show $COREOS_ISO_CUSTOMIZED_FILE -``` - -The previous command displays the Ignition configuration. - -## Installing the Customized NestOS ISO File - -The customized NestOS ISO file can be used to directly boot the installation. NestOS is automatically installed based on the Ignition configuration. After the installation is complete, you can use **nest/password** to log in to NestOS on the VM console. diff --git a/docs/en/docs/NfsMultipath/faqs.md b/docs/en/docs/NfsMultipath/faqs.md deleted file mode 100644 index a4b2b61e0051ac39b6b0d82c24158c20c46bf5e5..0000000000000000000000000000000000000000 --- a/docs/en/docs/NfsMultipath/faqs.md +++ /dev/null @@ -1,26 +0,0 @@ -# FAQs - -## 1. Failed to Mount NFS Multipathing and Query the .ko File of NFS Multipathing by Running the lsmod Command - -```shell -Error message: -mount.nfs: an incorrect mount option was specified -The .ko file of NFS multipathing cannot be queried by running the lsmod | grep nfs_multipath command. - -Possible cause: NFS multipathing is not installed. - -Solution: Install NFS multipathing by referring to section "Installation and Deployment." -``` - -## 2. Failed to Mount NFS Multipathing Links - -```shell -Error message: -mount.nfs: an incorrect mount option was specified -Error message in the /var/log/messages file: -Mar 18 23:38:22 localhost kernel: [ 635.016793] [MULTIPATH:nfs_multipath_parse_ip_range] iplist for remote reached 8, more than supported limit 8 - -Possible cause: The number of specified IP addresses exceeds 8. - -Solution: The number of specified IP addresses must be fewer than or equal to 8. -``` diff --git a/docs/en/docs/NfsMultipath/installation-and-deployment.md b/docs/en/docs/NfsMultipath/installation-and-deployment.md deleted file mode 100644 index 7202fa0adbf0788c3280936845b8765015d15c5d..0000000000000000000000000000000000000000 --- a/docs/en/docs/NfsMultipath/installation-and-deployment.md +++ /dev/null @@ -1,28 +0,0 @@ -# Installation and Deployment - -## Software - -* OS: openEuler 23.03 - -## Hardware - -* x86_64 -* Arm - -## Preparing the Environment - -Install the openEuler operating system. For details, see the [*openEuler Installation Guide*](../Installation/Installation.md). - -## Installing NFS Multipathing - -```shell -Insert the .ko file of NFS multipathing. -modprobe nfs_multipath - -View the .ko file of NFS multipathing. -lsmod | grep nfs_multipath -nfs_multipath 36864 1 -nfs 544768 3 nfs_multipath,nfsv3 -sunrpc 704512 30 nfs_multipath,lockd,nfsv3,nfs_acl,nfs -If `nfs_multipath` exists, the installation is successful. -``` diff --git a/docs/en/docs/NfsMultipath/introduction-to-nfs-multipathing.md b/docs/en/docs/NfsMultipath/introduction-to-nfs-multipathing.md deleted file mode 100644 index a10d85e99b06ea0cec5948a69eca7eeeee33dd75..0000000000000000000000000000000000000000 --- a/docs/en/docs/NfsMultipath/introduction-to-nfs-multipathing.md +++ /dev/null @@ -1,18 +0,0 @@ -# Introduction to NFS Multipathing - -## Introduction - -Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems (Sun) in 1984. It allows users on NFS clients to access files on NFS servers over computer networks. As the NFS service is widely used in industries such as finance, EDA, AI, and container, higher requirements are imposed on NFS performance and reliability. Traditional NFS has the following disadvantages: - -* A single mount point on a single host can be accessed only by using one client IP address and one server IP address. When multiple physical links exist between the client and server, the performance of these links cannot be fully utilized. -* If a single link of a single mount point is faulty, link failover cannot be performed. As a result, host services will be interrupted. - -NFS multipathing is developed to solve the preceding defects encountered during the use of traditional NFS. It is proposed that multiple links be established between the client and the server under a single mount point to support I/O transmission over multiple links, improving the performance of a single mount point. In addition, the link status is periodically checked to ensure fast I/O failover upon a link fault. - -NFS multipathing provides the following functions: - -* NFSv3 supports the Round Robin link selection algorithm to balance the performance of multiple links. -* NFSv3 and NFSv4 support fast link failover, improving NFS reliability. -* Provides an interface for registering link path selection algorithms and allows NFS server developers to customize path selection algorithms. -* Supports periodic link availability detection. -* Allows the link status to be viewed. diff --git a/docs/en/docs/NfsMultipath/nfs-multipathing-user-guide.md b/docs/en/docs/NfsMultipath/nfs-multipathing-user-guide.md deleted file mode 100644 index 7696f54025305a4ff0343922a5e0a3ec66e056e6..0000000000000000000000000000000000000000 --- a/docs/en/docs/NfsMultipath/nfs-multipathing-user-guide.md +++ /dev/null @@ -1,6 +0,0 @@ - -# NFS Multipathing User Guide - -This document describes how to install, deploy, and use the multipathing feature of Network File System (NFS) clients. - -This document is intended for community developers, open source enthusiasts, and partners who use the openEuler operating system and want to know and use the NFS multipathing feature. Users must have basic NFS knowledge. diff --git a/docs/en/docs/NfsMultipath/usage.md b/docs/en/docs/NfsMultipath/usage.md deleted file mode 100644 index e038a211c3723208bb84c0975ca0b40f92eeedda..0000000000000000000000000000000000000000 --- a/docs/en/docs/NfsMultipath/usage.md +++ /dev/null @@ -1,39 +0,0 @@ -# Usage - -## Mounting Multipath Links Using Parameters - -```shell -Command for mounting multipath links -mount -t nfs -o localaddrs=1.1.1.1~2.2.2.2,remoteaddrs=1.1.1.10~2.2.2.10 1.1.1.1:/test /mnt/fs1 - -The optional parameters localaddrs and remoteaddrs are added to specify multipathing links. -localaddrs indicates the IP address of the client. -remoteaddrs indicates the IP address of the server. -An IP range or multiple IP addresses can be configured. Use hyphens (-) to indicate an IP range and tilde (~) to indicate multiple IP addresses. -For example, remoteaddrs=1.1.1.1-1.1.1.3 specifies three server addresses: 1.1.1.1, 1.1.1.2, and 1.1.1.3. -localaddrs=1.1.1.1~1.1.1.3 specifies two client addresses: 1.1.1.1 and 1.1.1.3. - -Both localaddrs and remoteaddrs are optional. You can specify either or both of them. If neither of them is specified, the multipathing feature is disabled. -``` - -## Querying Multipathing Information - -```shell -Command for querying multipathing information -cat /proc/fs/nfs/mutipath/conn_info -The command output is similar to the following: -===============================Id:4 count 5 proto 3 start======================== -index:0, client_ip:1.1.1.1, server_ip:1.1.1.10, status:connect, load:3 -index:1, client_ip:1.1.1.1, server_ip:1.1.1.11, status:connect, load:3 -index:2, client_ip:1.1.1.1, server_ip:1.1.1.12, status:connect, load:3 -index:3, client_ip:1.1.1.1, server_ip:1.1.1.13, status:disconnect, load:0 -index:4, client_ip:1.1.1.1, server_ip:1.1.1.14, status:disconnect, load:0 -=========================================Id:4 end================================ -status indicates the link status. connect indicates that the link is available, and disconnect indicates that the link is unavailable. -load indicates the number of NFS requests sent over the link. -``` - -## Precautions - -* A maximum of eight client or server IP addresses are supported. If more than eight IP addresses are specified, the links cannot be mounted. -* A maximum of eight multipath links are supported. diff --git a/docs/en/docs/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip b/docs/en/docs/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip deleted file mode 100644 index ab74b7f0aa515f45d656bca9e018d47f7159c3d2..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip and /dev/null differ diff --git a/docs/en/docs/Pin/pin-user-guide.md b/docs/en/docs/Pin/pin-user-guide.md deleted file mode 100644 index f7bcd80f9767e06b01264ab27c9329526c004bea..0000000000000000000000000000000000000000 --- a/docs/en/docs/Pin/pin-user-guide.md +++ /dev/null @@ -1,134 +0,0 @@ -# Installation and Deployment - -## Software - -* OS: openEuler 23.03 - -## Hardware - -* x86_64 -* Arm - -## Preparing the Environment - -* Install the openEuler operating system. For details, see the [*openEuler 23.03 Installation Guide*](../Installation/Installation.md). - -### Install the dependency - -#### Installing the Software on Which the PIN GCC Client Depends - -```shell -yum install -y grpc -yum install -y grpc-devel -yum install -y grpc-plugins -yum install -y protobuf-devel -yum install -y jsoncpp -yum install -y jsoncpp-devel -yum install -y gcc-plugin-devel -yum install -y llvm-mlir -yum install -y llvm-mlir-devel -yum install -y llvm-devel -``` - -#### Installing the Software on Which the PIN Server Depends - -```shell -yum install -y grpc -yum install -y grpc-devel -yum install -y grpc-plugins -yum install -y protobuf-devel -yum install -y jsoncpp -yum install -y jsoncpp-devel -yum install -y llvm-mlir -yum install -y llvm-mlir-devel -yum install -y llvm-devel -``` - -## Installing PIN - -### rpmbuild - -#### Building the PIN GCC Client - -```shell -git clone https://gitee.com/src-openeuler/pin-gcc-client.git -cd pin-gcc-client -mkdir -p ~/rpmbuild/SOURCES -cp *.path pin-gcc-client.tar.gz ~/rpmbuild/SOURCES -rpmbuild -ba pin-gcc-client.spec -cd ~/rpmbuild/RPMS -rpm -ivh pin-gcc-client.rpm -``` - -#### Building the PIN Server - -```shell -git clone https://gitee.com/src-openeuler/pin-server.git -cd pin-server -mkdir -p ~/rpmbuild/SOURCES -cp *.path pin-server.tar.gz ~/rpmbuild/SOURCES -rpmbuild -ba pin-server.spec -cd ~/rpmbuild/RPMS -rpm -ivh pin-server.rpm -``` - -### Build - -#### Building the PIN GCC Client - -```shell -git clone https://gitee.com/openeuler/pin-gcc-client.git -cd pin-gcc-client -mkdir build -cd build -cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH} -make -``` - -#### Building the PIN Server - -```shell -git clone https://gitee.com/openeuler/pin-server.git -cd pin-server -mkdir build -cd build -cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH} -make -``` - -# Usage - -You can use `-fplugin` and `-fplugin-arg-libpin_xxx` to enable the Plug-IN (PIN) tool. -Command: - -```shell -$(TARGET): $(OBJS) - $(CXX) -fplugin=${CLIENT_PATH}/build/libpin_gcc_client.so \ - -fplugin-arg-libpin_gcc_client-server_path=${SERVER_PATH}/build/pin_server \ - -fplugin-arg-libpin_gcc_client-log_level="1" \ - -fplugin-arg-libpin_gcc_client-arg1="xxx" -``` - -You can use the `${INSTALL_PATH}/bin/pin-gcc-client.json` file to configure PIN. The configuration options are as follows: - -`path`: path of the executable file of the PIN server. - -`sha256file`: path of the PIN verification file `xxx.sha256`. - -`timeout`: timeout interval for cross-process communication, in milliseconds. - -Compile options: - -`-fplugin`: path of the .so file of the PIN client. - -`-fplugin-arg-libpin_gcc_client-server_path`: path of the executable program of the PIN server. - -`-fplugin-arg-libpin_gcc_client-log_level`: default log level. The value ranges from `0` to `3`. The default value is `1`. - -`-fplugin-arg-libpin_gcc_client-argN`: other parameters that can be specified as required. `argN` indicates the argument required by PIN. - -# Compatibility - -This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and will be fixed as soon as possible. Developers are welcome to join this project. - -* When PIN is enabled in the `-flto` phase, multi-process compilation using `make -j` is not supported. You are advised to use `make -j1` for compilation. diff --git a/docs/en/docs/Quickstart/figures/advanced-user-configuration.png b/docs/en/docs/Quickstart/figures/advanced-user-configuration.png deleted file mode 100644 index 59a188aece92ad19cc9b42f69e235d9a9d4f702a..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/advanced-user-configuration.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/creating-a-user.png b/docs/en/docs/Quickstart/figures/creating-a-user.png deleted file mode 100644 index 0e2befb0832d1167f5ffdcafdf7d9952d9ccdfbe..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/creating-a-user.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/drive-icon.png b/docs/en/docs/Quickstart/figures/drive-icon.png deleted file mode 100644 index b41fcb09dfbf805da4863142855e7c2de4bf4c7b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/drive-icon.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/image-dialog-box.png b/docs/en/docs/Quickstart/figures/image-dialog-box.png deleted file mode 100644 index caeb56bb46f766dd39d66a65e308c591954d32cf..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/image-dialog-box.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/installation-process.png b/docs/en/docs/Quickstart/figures/installation-process.png deleted file mode 100644 index 2d219c7605ee75e73dffba1e2dd7c277968d4801..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/installation-process.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/installation-summary.png b/docs/en/docs/Quickstart/figures/installation-summary.png deleted file mode 100644 index d5ca555a2b2291e139b67098a7c23d29b23b8b24..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/installation-summary.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/password-of-the-root-account.png b/docs/en/docs/Quickstart/figures/password-of-the-root-account.png deleted file mode 100644 index fe65e73a81e25e5fa90a13af707165911e7fa459..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/password-of-the-root-account.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/selecting-a-language.png b/docs/en/docs/Quickstart/figures/selecting-a-language.png deleted file mode 100644 index eaeb26ca208778822bf591782a689569339c3552..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/selecting-a-language.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/selecting-installation-software.png b/docs/en/docs/Quickstart/figures/selecting-installation-software.png deleted file mode 100644 index c246e997d787d0d6a0439dcaf8780a09a9b72ca7..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/selecting-installation-software.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/figures/setting-the-installation-destination.png b/docs/en/docs/Quickstart/figures/setting-the-installation-destination.png deleted file mode 100644 index 224f165b222598aa140187bdfa9b1e75af36c0c5..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Quickstart/figures/setting-the-installation-destination.png and /dev/null differ diff --git a/docs/en/docs/Quickstart/quick-start.md b/docs/en/docs/Quickstart/quick-start.md deleted file mode 100644 index d09fe522dd51b3160bc5fa30b010414103b8f2cc..0000000000000000000000000000000000000000 --- a/docs/en/docs/Quickstart/quick-start.md +++ /dev/null @@ -1,351 +0,0 @@ -# Quick Start - -This document uses openEuler 22.03 LTS SP2 installed on the TaiShan 200 server as an example to describe how to quickly install and use openEuler OS. For details about the installation requirements and methods, see the [Installation Guide](./../Installation/Installation.md). - -## Making Preparations - -- Hardware Compatibility - - [Table 1](#table14948632047) describes the types of supported servers. - - **Table 1** Supported servers - - - - - - - - - - - - - - - - - -

Server Type

-

Server Name

-

Server Model

-

Rack server

-

TaiShan 200

-

2280 balanced model

-

Rack server

-

FusionServer Pro

-

FusionServer Pro 2288H V5

-
NOTE:

The server must be configured with the Avago SAS3508 RAID controller card and the LOM-X722 NIC.

-
-
- -- Minimum Hardware Specifications - - [Table 2](#tff48b99c9bf24b84bb602c53229e2541) lists the minimum hardware specifications supported by openEuler. - - **Table 2** Minimum hardware specifications - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Component

Minimum Hardware Specifications

Description

Architecture

  • AArch64
  • x86_64
  • 64-bit Arm architecture
  • 64-bit Intel x86 architecture

CPU

  • Huawei Kunpeng 920 series
  • Intel ® Xeon® processor

-

Memory

≥ 4 GB (8 GB or higher recommended for better user experience)

-

Drive

≥ 120 GB (for better user experience)

  • IDE, SATA, and SAS drives are supported.
  • A driver software is required to use the NVMe drive with the DIF feature. Contact the drive manufacturer if the feature is not available.

- -## Obtaining the Installation Source - -Perform the following operations to obtain the openEuler release package: - -1. Visit the [openEuler](https://www.openeuler.org/en/) website. -2. Click **Downloads**. -3. Click **Community Editions**. The version list is displayed. -4. Click **Download** on the right of **openEuler 23.09**. -5. Download the required openEuler release package and the corresponding verification file based on the architecture and scenario. - - - If the AArch64 architecture is used: - - 1. Click **AArch64**. - 2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-aarch64-dvd.iso** to the local host. - 3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-aarch64-dvd.iso** to the local host. - - - If the x86\_64 architecture is used: - - 1. Click **x86_64**. - 2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-x86_64-dvd.iso** to the local host. - 3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-x86_64-dvd.iso** to the local host. - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> - When the network is available, install the environment on the network because the ISO release package is small. -> - The release package of the AArch64 architecture supports the UEFI mode, while the release package of the x86\_64 architecture supports both the UEFI and Legacy modes. -> - -## Checking the Release Package Integrity - -> ![](./public_sys-resources/icon-note.gif) **NOTE:** -> -> This section describes how to verify the integrity of the release package in the AArch64 architecture. The procedure for verifying the integrity of the release package in the x86\_64 architecture is the same. - -### Overview - -To prevent incomplete download of the software package due to network or storage device problems during the transmission, verify the integrity of the software package after obtaining it. Only the software package that passes the verification can be deployed. - -Compare the verification value recorded in the verification file with the verification value of the ISO file calculated manually to determine whether the software package is complete. If the two values are the same, the ISO file is complete. Otherwise, the ISO file integrity is damaged. In this case, obtain the ISO release package again. - -### Prerequisites - -The following files need to be prepared: - -ISO file: openEuler-22.03-LTS-SP2-aarch64-dvd.iso - -Verification file: Copy and save the SHA256 value corresponding to the ISO file. - -1. Calculate the SHA256 verification value of the file. Run the following command: - - ```sh - # sha256sum openEuler-22.03-LTS-SP2-aarch64-dvd.iso - ``` - -2. Check whether the values calculated in step 1 and step 2 are the same. - - If the values are consistent, the ISO file is not damaged. Otherwise, the file is damaged and you need to obtain it again. - If the values are consistent, the ISO file is not damaged. Otherwise, the file is damaged and you need to obtain it again. - -## Starting Installation - -1. Log in to the iBMC WebUI. - - For details, see [TaiShan 200 Server User Guide (Model 2280)](https://support.huawei.com/enterprise/en/doc/EDOC1100093459). - -2. Choose **Configuration** from the main menu, and select **Boot Device** from the navigation tree. The **Boot Device** page is displayed. - - Set **Effective** and **Boot Medium** to **One-time** and **DVD-ROM**, respectively, and click **Save**, as shown in [Figure 1](#fig1011938131018). - - **Figure 1** Setting the boot device -![fig](./figures/setting-the-boot-device.png "setting-the-boot-device") - -3. Choose **Remote Console** from the main menu. The **Remote Console** page is displayed. - - Select an integrated remote console as required to access the remote virtual console, for example, **Java Integrated Remote Console (Shared)**. - -4. On the toolbar, click the icon shown in the following figure. - - **Figure 2** Drive icon -![fig](./figures/drive-icon.png "drive-icon") - - An image dialog box is displayed, as shown in the following figure. - - **Figure 3** Image dialog box -![fig](./figures/image-dialog-box.png "image-dialog-box") - -5. Select **Image File** and then click **Browse**. The **Open** dialog box is displayed. - -6. Select the image file and click **Open**. In the image dialog box, click **Connect**. If **Connect** changes to **Disconnect**, the virtual CD/DVD-ROM drive is connected to the server. - -7. On the toolbar, click the restart icon shown in the following figure to restart the device. - - **Figure 4** Restart icon -![fig](./figures/restart-icon.png "restart-icon") - -8. A boot menu is displayed after the system restarts, as shown in [Figure 5](#fig1648754873314). - - > ![fig](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - If you do not perform any operations within 1 minute, the system automatically selects the default option **Test this media \& install openEuler 22.03-LTS-SP2** and enters the installation page. - > - During physical machine installation, if you cannot use the arrow keys to select boot options and the system does not respond after you press **Enter**, click ![fig](./figures/en-us_image_0229420473.png) on the BMC page and configure **Key \& Mouse Reset**. - - **Figure 5** Installation wizard - ![fig](./figures/Installation_wizard.png "Installation_wizard") - -9. On the installation wizard page, press **Enter** to select the default option **Test this media \& install openEuler 22.03-LTS-SP2** to enter the GUI installation page. - -## Performing Installation - -After entering the GUI installation page, perform the following operations to install the system: - -1. Set an installation language. The default language is English. You can change the language based on the site requirements, as shown in [Figure 6](#fig874344811484). - - **Figure 6** Selecting a language - ![fig](./figures/selecting-a-language.png "selecting-a-language") - -2. On the **INSTALLATION SUMMARY** page, set configuration items based on the site requirements. - - - A configuration item with an alarm symbol must be configured. When the alarm symbol disappears, you can perform the next operation. - - A configuration item without an alarm symbol is configured by default. - - You can click **Begin Installation** to install the system only when all alarms are cleared. - - **Figure 7** Installation summary - ![fig](./figures/installation-summary.png "installation-summary") - - 1. Select **Software Selection** to set configuration items. - - Based on the site requirements, select **Minimal Install** on the left box and select an add-on in the **Add-Ons for Selected Environment** area on the right, as shown in [Figure 8](#fig1133717611109). - - **Figure 8** Selecting installation software - ![fig](./figures/selecting-installation-software.png "selecting-installation-software") - - > ![fig](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - In **Minimal Install** mode, not all packages in the installation source are installed. If a required package is not installed, you can mount the installation source to the local host as a repo source, and use DNF to install the package. - > - If you select **Virtual Host**, the virtualization components QEMU, libvirt, and edk2 are installed by default. You can select whether to install the OVS component in the add-on area. - - After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - - 2. Select **Installation Destination** to set configuration items. - - On the **INSTALLATION DESTINATION** page, select a local storage device. - - > ![fig](./public_sys-resources/icon-notice.gif) **NOTICE:** - > The NVMe data protection feature is not supported because the NVMe drivers built in the BIOSs of many servers are of earlier versions. (Data protection: Format disk sectors to 512+N or 4096+N bytes.) Therefore, when selecting a proper storage medium, do not select an NVMe SSD with data protection enabled as the system disk. Otherwise, the OS may fail to boot. - > Users can consult the server vendor about whether the BIOS supports NVMe disks with data protection enabled as system disks. If you cannot confirm whether the BIOS supports NVMe disks with data protection enabled as system disks, you are not advised to use an NVMe disk to install the OS, or you can disable the data protection function of an NVMe disk to install the OS. - - You also need to configure the storage to partition the system. You can either manually configure partitions or select **Automatic** for automatic partitioning. Select **Automatic** if the software is installed in a new storage device or the data in the storage device is not required, as shown in [Figure 9](#fig153381468101). - - **Figure 9** Setting the installation destination - ![fig](./figures/setting-the-installation-destination.png "setting-the-installation-destination") - - > ![fig](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - During partitioning, to ensure system security and performance, you are advised to divide the device into the following partitions: **/boot**, **/var**, **/var/log**, **/var/log/audit**, **/home**, and **/tmp**. - > - If the system is configured with the **swap** partition, the **swap** partition is used when the physical memory of the system is insufficient. Although the **swap** partition can be used to expand the physical memory, if the **swap** partition is used due to insufficient memory, the system response slows and the system performance deteriorates. Therefore, you are not advised to configure the **swap** partition in a system with sufficient physical memory or a performance-sensitive system. - > - If you need to split a logical volume group, select **Custom** to manually partition the logical volume group. On the **MANUAL PARTITIONING** page, click **Modify** in the **Volume Group** area to reconfigure the logical volume group. - - After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - - 3. Select **Root Password** and set the root password. - - On the **ROOT PASSWORD** page, enter a password that meets the [Password Complexity](#password-complexity) requirements and confirm the password, as shown in [Figure 10](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1323165793018). - - > ![fig](./public_sys-resources/icon-note.gif) **NOTE:** - > - > - The **root** account is used to perform key system management tasks. You are not advised to use the **root** account for daily work or system access. - > - > - If you select **Lock root account** on the **Root Password** page, the **root** account will be disabled. - - **Password Complexity** - - The password of the **root** user or a new user must meet the password complexity requirements. Otherwise, the password setting or user creation will fail. The password must meet the following requirements: - - 1. Contains at least eight characters. - - 2. Contains at least three of the following: uppercase letters, lowercase letters, digits, and special characters. - - 3. Different from the user name. - - 4. Not allowed to contain words in the dictionary. - - > ![](./public_sys-resources/icon-note.gif) **NOTE:** - > - > In the openEuler environment, you can run the `cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt` command to export the dictionary library file **dictionary.txt**. You can check whether the password is in this dictionary. - - **Figure 10** root password - ![fig](./figures/password-of-the-root-account.png "Root password") - - After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - - 4. Select **Create a User** and set the parameters. - - [Figure 11](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319) shows the page for creating a user. Enter the user name and set the password. The password complexity requirements are the same as those of the root password. In addition, you can set the home directory and user group by clicking **Advanced**, as shown in [Figure 12](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319). - - **Figure 11** Creating a user - ![fig](./figures/creating-a-user.png "creating-a-user") - - **Figure 12** Advanced user configuration - ![fig](./figures/advanced-user-configuration.png "Advanced user configuration") - - After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. - - 5. Set other configuration items. You can use the default values for other configuration items. - -3. Click **Start the Installation** to install the system, as shown in [Figure 13](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319). - - **Figure 13** Starting the installation - ![fig](./figures/installation-process.png "installation-process") -4. After the installation is completed, restart the system. - - openEuler has been installed. Click **Reboot** to restart the system. - -## Viewing System Information - -After the system is installed and restarted, the system CLI login page is displayed. Enter the username and password set during the installation to log in to openEuler and view the following system information. For details about system management and configuration, see the [openEuler 22.03-LTS-SP2 Administrator Guide](../Administration/administration.md). - -- Run the following command to view the system information: - - ```sh - cat /etc/os-release - ``` - - ```sh - cat /etc/os-release - ``` - -- View system resource information. - - Run the following command to view the CPU information: - - ```sh - # lscpu - ``` - - Run the following command to view the memory information: - - ```sh - # free - ```` - - Run the following command to view the disk partition information: - - ```sh - # fdisk -l - ``` - - Run the following command to view the CPU information: - - ```sh - # lscpu - ``` - - Run the following command to view the memory information: - - ```sh - # free - ```` - - Run the following command to view the disk partition information: - - ```sh - # fdisk -l - ``` - -- Run the following command to view the IP address: - - ```sh - # ip addr - ``` \ No newline at end of file diff --git a/docs/en/docs/ROS/ROS.md b/docs/en/docs/ROS/ROS.md deleted file mode 100644 index 2ce5f290bf08f1380a7fb7ee2b2dc659b6c9fa21..0000000000000000000000000000000000000000 --- a/docs/en/docs/ROS/ROS.md +++ /dev/null @@ -1,5 +0,0 @@ -# ROS User Guide - -This document introduces the installation, deployment, and usage of Robot Operating System (ROS) on openEuler. - -This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use ROS. You need to have basic knowledge of the Linux OS. diff --git a/docs/en/docs/ROS/appendix.md b/docs/en/docs/ROS/appendix.md deleted file mode 100644 index b1bb73b54cf7dcbbb8c4c734ebfdbca79d4dfbf3..0000000000000000000000000000000000000000 --- a/docs/en/docs/ROS/appendix.md +++ /dev/null @@ -1,3 +0,0 @@ -# Appendix - -A more detailed introduction to ROS can be found at [ROS wiki](https://wiki.ros.org/), [ROS docs](http://docs.ros.org/). diff --git a/docs/en/docs/ROS/faqs.md b/docs/en/docs/ROS/faqs.md deleted file mode 100644 index c7e0b9afd038ecb2b91ad4cc58e3ea87ccf282e6..0000000000000000000000000000000000000000 --- a/docs/en/docs/ROS/faqs.md +++ /dev/null @@ -1,20 +0,0 @@ -# FAQs - -## Question 1 - -![](./figures/problem.png) - -Cause: The reason for this warning is that there are both ROS1 and ROS2 in the environment variable. -Solution: Modify the environment variable to avoid the conflict between the two versions. - -```shell -vim /opt/ros/foxy/share/ros_environment/environment/1.ros_distro.sh -``` - -```shell -# generated from ros_environment/env-hooks/1.ros_distro.sh.in - -#export ROS_DISTRO=foxy -``` - -Comment out everything inside. diff --git a/docs/en/docs/ROS/installation-and-deployment.md b/docs/en/docs/ROS/installation-and-deployment.md deleted file mode 100644 index 9c6225a5ebb9ea55a3fd9fba17767cf712c14b67..0000000000000000000000000000000000000000 --- a/docs/en/docs/ROS/installation-and-deployment.md +++ /dev/null @@ -1,53 +0,0 @@ -# Installation and Deployment - -## Software - -* OS: openEuler 22.03 LTS SP2 - -## Hardware - -* x86_64 - -## Preparing the Environment - -* Install the openEuler OS by referring to the [*openEuler Installation Guide*](../Installation/Installation.md). - -## 1. ROS2 - -### 1. ros-humble - -#### 1. Installing ros-humble - -1. Install ros-humble software package - - ```shell - yum install openeuler-ros - yum install ros-humble-ros-base ros-humble-xxx e.g. ros-humble-turtlesim - ``` - -2. Run the following command to check whether the installation is successful - - ```shell - rpm -q ros-humble - ``` - -#### 2. Test ros-humble - -##### Run turtlesim - -1. Run turtlesim - - ```shell - source /opt/ros/humble/setup.bash - ros2 run turtlesim turtlesim_node - ``` - -2. Open turtlesim terminal - - ```shell - source /opt/ros/humble/setup.bash - ros2 run turtlesim turtle_teleop_key - ``` - -3. Use the arrow keys to control the movement of the turtle - ![ros-humble](./figures/ros-humble.png) diff --git a/docs/en/docs/ROS/introduction-to-ROS.md b/docs/en/docs/ROS/introduction-to-ROS.md deleted file mode 100644 index 83f31fa65a3440734c2992f9d7d89b5d947d8369..0000000000000000000000000000000000000000 --- a/docs/en/docs/ROS/introduction-to-ROS.md +++ /dev/null @@ -1,33 +0,0 @@ -# Introduction to ROS - -## Introduction - -ROS is an open source meta-operating system for robotics. It provides the services that an operating system should have, including hardware abstraction, low-level device control, implementation of common functions, inter-process message passing, and package management. It also provides the tools and library functions needed to fetch, compile, write, and run code across computers. - -ROS's operating architecture is a processing architecture that uses ROS communication modules to implement loosely coupled network connections between modules [P2P](https://en.wikipedia.org/wiki/Peer-to-peer), which implements several types of communications, including: - -1. Service-based synchronous [RPC](https://en.wikipedia.org/wiki/Remote_procedure_call) (remote procedure call) communication; -2. Topic-based asynchronous data flow communication, as well as data storage on the parameter server. - -Since the beginning of ROS in 2007, along with the great development of robot technology, the core ideas and basic software packages of ROS have been gradually improved and different ROS distributions have been released. Below is a list of current and historical ROS releases, the rows marked in green in the table are the currently supported releases. - -![ROS release](./figures/ROS-release.png) - -Although ROS is still a powerful development tool in the field of robotics, due to the limitations of the initial design, many problems have gradually been exposed. For example: poor real-time performance, high system overhead, unfriendly support for Python3, no encryption mechanism and low security. Many developers and research institutions have also made improvements to address the limitations of ROS, but these local function improvements are often difficult to bring about overall performance improvements. At ROSCon 2014, the design architecture of the new generation of ROS (Next-generation ROS: Building on DDS) was officially announced. On August 31, 2015, the first alpha version of ROS2.0 was launched, and different release versions were released later. Below is a list of current and historical ROS2 releases, the rows marked in green in the table are currently supported releases. - -![ROS2 release](./figures/ROS2-release.png) - -## Architecture - -The overall architecture of ROS is shown in the figure below: - -![ROS architecture](./figures/ROS-ROS2.png) - -1. OS Layer - - ROS1 is mainly built on the Linux system, and ROS2 has brought changes. The systems supported include Linux, Windows, Mac, RTOS, and even bare metal without an operating system. -2. Middleware Layer - - One of the most important concepts in ROS is the "node" based on the publish/subscribe model, which allows developers to develop low-coupling functional modules in parallel and facilitates secondary reuse. The communication system of ROS1 is based on TCPROS/UDPROS, while the communication system of ROS2 is based on DDS. DDS is a standard solution for data publishing/subscribing in distributed real-time systems, which will be explained in detail in the next section. ROS2 provides an abstract layer implementation of DDS internally, and users do not need to pay attention to the underlying DDS provider. - - In the ROS1 architecture, Nodelet and TCPROS/UDPROS are parallel layers, providing a more optimized data transmission method for multiple nodes in the same process. This data transmission method is also retained in ROS2, but it is called "Intra-process", which is also independent of DDS. -3. Application Layer - - - ROS1 is strongly dependent on the ROS Master, and one can imagine what kind of dilemma the entire system will face once the Master goes down. But from the architecture of ROS2 on the right, we can find that the Master, which was a worry before, has finally disappeared, and nodes use a discovery mechanism called "Discovery" to obtain each other's information. diff --git a/docs/en/docs/ROS/usage.md b/docs/en/docs/ROS/usage.md deleted file mode 100644 index 61e9587ce2aa0406a8586fd7e4a978b25a606a0b..0000000000000000000000000000000000000000 --- a/docs/en/docs/ROS/usage.md +++ /dev/null @@ -1,56 +0,0 @@ -# Usage - -## Using ROS - -ROS provides some useful command line tools, which can be used to obtain various information of different nodes. Commonly used commands are as follows: - -- rosnode : operation node -- rostopic : operation topic -- rosservice : operation service -- rosmsg : operation msg -- rossrv : operation srv msg -- rosparam : operation param - -Please also refer to: - - - -## Using ROS2 - -```shell -# Help information of the command -$ ros2 --help -usage: ros2 [-h] Call `ros2 -h` for more detailed usage. ... - -ros2 is an extensible command-line tool for ROS 2. - -optional arguments: - -h, --help show this help message and exit - -Commands: - action Various action related sub-commands - bag Various rosbag related sub-commands - component Various component related sub-commands - daemon Various daemon related sub-commands - doctor Check ROS setup and other potential issues - interface Show information about ROS interfaces - launch Run a launch file - lifecycle Various lifecycle related sub-commands - multicast Various multicast related sub-commands - node Various node related sub-commands - param Various param related sub-commands - pkg Various package related sub-commands - run Run a package specific executable - security Various security related sub-commands - service Various service related sub-commands - test Run a ROS2 launch test - topic Various topic related sub-commands - trace Trace ROS nodes to get information on their execution - wtf Use `wtf` as alias to `doctor` - - Call `ros2 -h` for more detailed usage. -``` - -## Precautions - -- You need to run the `source /opt/ros/foxy/local_setup.bash` or `source /opt/ros/noetic/setup.bash` command on a newly opened terminal. diff --git a/docs/en/docs/Releasenotes/installing-the-os.md b/docs/en/docs/Releasenotes/installing-the-os.md deleted file mode 100644 index 48f672163b9c596187bff0e9e6be64de70796129..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/installing-the-os.md +++ /dev/null @@ -1,310 +0,0 @@ -# OS Installation - -## Release Files - -The openEuler release files include [ISO release package](http://repo.openeuler.org/openEuler-22.09/ISO/), [VM images](http://repo.openeuler.org/openEuler-22.09/virtual_machine_img/), [container images](http://repo.openeuler.org/openEuler-22.09/docker_img/), [embedded images](http://repo.openeuler.org/openEuler-22.09/embedded_img/), and [repo sources](http://repo.openeuler.org/openEuler-22.09/). [Table 1](#table8396719144315) describes the ISO release packages. [Table 3](#table1276911538154) describes the container images. [Table 5](#table953512211576) describes the repo sources, which are convenient for online use. - -**Table 1** ISO release packages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Name

-

Description

-

openEuler-22.09-aarch64-dvd.iso

-

Base installation ISO file of the AArch64 architecture, including the core components for running the minimum system.

-

openEuler-22.09-everything-aarch64-dvd.iso

-

Full installation ISO file of the AArch64 architecture, including all components for running the entire system.

-

openEuler-22.09-everything-debug-aarch64-dvd.iso

-

ISO file for openEuler debugging in the AArch64 architecture, including the symbol table information required for debugging.

-

openEuler-22.09-x86_64-dvd.iso

-

Base installation ISO file of the x86_64 architecture, including the core components for running the minimum system.

-

openEuler-22.09-everything-x86_64-dvd.iso

-

Full installation ISO file of the x86_64 architecture, including all components for running the entire system.

-

openEuler-22.09-everything-debuginfo-x86_64-dvd.iso

-

ISO file for openEuler debugging in the x86_64 architecture, including the symbol table information required for debugging.

-

openEuler-22.09-source-dvd.iso

-

ISO file of the openEuler source code.

-

openEuler-21.09-edge-aarch64-dvd.iso

-

Edge ISO file in the AArch64 architecture, including the core components for running the minimum system.

-

openEuler-21.09-edge-x86_64-dvd.iso

-

Edge ISO file in the x86_64 architecture, including the core components for running the minimum system.

-

openEuler-21.09-Desktop-aarch64-dvd.iso

-

Desktop ISO file in the AArch64 architecture, including the minimum software set for running the development desktop.

-

openEuler-21.09-Desktop-x86_64-dvd.iso

-

Desktop ISO file in the x86_64 architecture, including the minimum software set for running the development desktop.

-
- -**Table 2** VM images - - - - - - - - - - - - - - -

Name

-

Description

-

openEuler-22.09-aarch64.qcow2.xz

-

VM image of openEuler in the AArch64 architecture.

-

openEuler-22.09-x86_64.qcow2.xz

-

VM image of openEuler in the x86_64 architecture.

-
- ->![](./public_sys-resources/icon-note.gif) **NOTE** ->The default password of the **root** user of the VM image is **openEuler12#$**. Change the password upon the first login. - -**Table 3** Container images - - - - - - - - - - - - - - -

Name

-

Description

-

openEuler-docker.aarch64.tar.xz

-

AContainer image of openEuler in the AArch64 architecture.

-

openEuler-docker.x86_64.tar.xz

-

Container image of openEuler in the x86_64 architecture.

-
- -**Table 4** Embedded images - -| Name | Description | -| -------------------------------------- | ------------------------------------------------------------ | -| arm64/aarch64-std/zImage | Kernel image that supports QEMU in the AArch64 architecture. | -| arm64/aarch64-std/\*toolchain-22.09.sh | Development and compilation toolchain in the AArch64 architecture. | -| arm64/aarch64-std/\*rootfs.cpio.gz | File system that supports QEMU in the AArch64 architecture. | -| arm32/arm-std/zImage | Kernel image that supports QEMU in the ARM architecture. | -| arm32/arm-std/\*toolchain-22.09.sh | Development and compilation toolchain in the ARM architecture. | -| arm32/arm-std/\*rootfs.cpio.gz | File system that supports QEMU in the ARM architecture. | -| source-list/manifest.xml | Manifest of source code used for building. | - -**Table 5** Repo sources - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Name

-

Description

-

ISO

-

Stores ISO images.

-

OS

-

Stores basic software package sources.

-

debuginfo

-

Stores debugging package sources.

-

docker_img

-

Stores container images.

-

virtual_machine_img

-

Stores VM images.

-

embedded_img

-

Stores embedded images.

-

everything

-

Stores full software package sources.

-

extras

-

Stores extended software package sources.

-

source

-

Stores source code software package.

-

update

-

Stores update software package sources.

-

EPOL

-

Stores extended openEuler package sources.

-
- -## Minimum Hardware Specifications - -[Table 6](#zh-cn_topic_0182825778_tff48b99c9bf24b84bb602c53229e2541) lists the minimum hardware specifications for installing openEuler 22.09. - -**Table 6** Minimum hardware requirements - - - - - - - - - - - - - - - - - -

Component

-

Minimum Hardware Specification

-

CPU

-

Kunpeng 920 (AArch64)/ x86_64 (later than Skylake)

-

x86-64 (later than Skylake)

-

Memory

-

≥ 8 GB

-

Hard drive

-

≥ 120 GB

-
- -## Hardware Compatibility - -[Table 7](#zh-cn_topic_0227922427_table39822012) describes the typical configurations of servers and components supported by openEuler. openEuler will support more servers in the future. Partners and developers are welcome to participate in the contribution and verification. For details about the servers supported by openEuler, see [Compatibility List](https://www.openeuler.org/en/compatibility/). - -**Table 7** Supported servers and configurations - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Vendor

-

Server Name

-

Server Model

-

Component

-

Configuration

-

Huawei

-

TaiShan 200

-

2280 (balanced model)

-

CPU

-

HiSilicon Kunpeng 920

-

Memory

-

4 x 32 GB 2933MHz

-

RAID card

-

LSI SAS3508

-

Network

-

TM210

-

Huawei

-

FusionServer Pro

-

2288H V5 (rack server)

-

CPU

-

Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz

-

Memory

-

4 x 32 GB 2400MHz

-

RAID card

-

LSI SAS3508

-

Network

-

X722

-
diff --git a/docs/en/docs/Releasenotes/key-features.md b/docs/en/docs/Releasenotes/key-features.md deleted file mode 100644 index ed48af6124fc8c5781b450346052aee2dc9f05a6..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/key-features.md +++ /dev/null @@ -1,297 +0,0 @@ -# Key Features - -## GMEM - -Generalized Memory Management (GMEM) is an optimal solution for memory management in OS for AI. It provides a centralized management mechanism for heterogeneous memory interconnection. GMEM innovates the memory management architecture in the Linux kernel. Its logical mapping system masks the differences between the ways how the CPU and accelerator access memory addresses. The Remote Pager memory message interaction framework provides the device access abstraction layer. In the unified address space, GMEM automatically migrates data to the OS or accelerator when data needs to be accessed or paged. GMEM APIs are consistent with native Linux memory management APIs, and feature high usability, performance, and portability. - -- **Logical mapping system**: GMEM high-level APIs in the kernel allow the accelerator driver to directly obtain memory management functions and create logical page tables. The logical page tables decouple the high-layer logic of memory management from the hardware layer of the CPU, so as to abstract the high-layer memory management logic that can be reused by various accelerators. - -- **Remote Pager**: This framework has the message channel, process management, memory swap, and memory prefetch modules for the interaction between the host and accelerator. The remote_pager abstraction layer simplifies device adaptation by enabling third-party accelerators to easily access the GMEM system. - -- **User APIs**: Users can directly use the memory map (mmap) of the OS to allocate the unified virtual memory. GMEM adds the flag (MMAP_PEER_SHARED) for allocating the unified virtual memory to the mmap system call. The libgmem user-mode library provides the hmadvise API of memory prefetch semantics to help users optimize the accelerator memory access efficiency. - -## Native Support for Open Source Large Language Models (LLaMA and ChatGLM) - -The two model inference frameworks, llama.cpp and chatglm-cpp, are implemented based on C/C++. They allow users to deploy and use open source large language models on CPUs by means of model quantization. llama.cpp supports the deployment of multiple open source LLMs, such as LLaMa, LLaMa2, and Vicuna. It supports the deployment of multiple open source Chinese LLMs, such as ChatGLM-6B, ChatGLM2-6B, and Baichuan-13B. - -- Implemented in GGML-based C/C++. - -- They accelerate memory for efficient CPU inference through int4/int8 quantization, optimized KV cache, and parallel computing. - -## Features in the openEuler Kernel 6.4 - -openEuler 23.09 runs on Linux kernel 6.4. It inherits the competitive advantages of community versions and innovative features released in the openEuler community. - -- **Tidal affinity scheduling:** The system dynamically adjusts CPU affinity based on the service load. When the service load is light, the system uses preferred CPUs to enhance resource locality. When the service load is heavy, the system has new CPU cores added to improve the QoS. - -- **CPU QoS priority-based load balancing**: CPU QoS isolation is enhanced in online and offline hybrid deployments, and QoS load balancing across CPUs is supported to further reduce QoS interference from offline services. - -- **Simultaneous multithreading (SMT) expeller free of priority inversion**: This feature resolves the priority inversion problem in the SMT expeller feature and reduces the impact of offline tasks on the quality of service (QoS) of online tasks. - -- **Multiple priorities in a hybrid deployment**: Each cgroup can have a **cpu.qos_level** that ranges from -2 to 2. You can set **qos_level_weight** to assign a different priority to each cgroup and allocate CPU resources to each cgroup based on the CPU usage. This feature is also capable of wakeup preemption. - -- Programmable scheduling: The programmable scheduling framework based on eBPF allows the kernel scheduler to dynamically expand scheduling policies to meet performance requirements of different loads. - -- **NUMA-aware spinlock**: The lock transfer algorithm is optimized for the multi-NUMA system based on the MCS spinlock. The lock is preferentially transferred within the local NUMA node, greatly reducing cross-NUMA cache synchronization and ping-pong. As a result, the overall lock throughput is increased and service performance is improved. - -- **TCP compression**: The data of specified ports can be compressed at the TCP layer before transmission. After the data is received, it is decompressed and transferred to the user mode. TCP compression accelerates data transmission between nodes. - -- **Kernel live patches**: Kernel live patches are used to fix bugs in kernel function implementation without a system restart. They dynamically replace functions when the system is running. Live patches on openEuler work by modifying instructions. They feature a high patching efficiency because they directly jump to new functions without search and transfer, while live patches on the Linux mainline version work based on ftrace. - -- **Sharepool shared memory**: This technology shares data among multiple processes and allows multiple processes to access the same memory area for data sharing. - -- **Memcg asynchronous reclamation**: This optimized mechanism asynchronously reclaims memory in the memcg memory subsystem when the system load is low to avoid memory reclamation delay when the system load becomes heavy. - -- **filescgroup**: The filescgroup subsystem manages the number of files (that is, the number of handles) opened by a process group. This subsystem provides easy-to-call APIs for resource management. Compared with the rlimit method, the filescgroup subsystem can better control the number of file handles for resource application and release, dynamic adjustment of resource usage, and group control. - -- **Cgroup writeback for cgroup v1**: Cgroup writeback provides a flexible method to manage the writeback behavior of the file system cache. The main functions of cgroup writeback include cache writeback control, I/O priority control, and writeback policy adjustment. - -- **Core suspension detection**: If the performance monitor unit (PMU) stops counting, the hard lockup cannot detect system suspension. The core suspension detection feature enables each CPU to check whether adjacent CPUs are suspended. This ensures that the system can perform self-healing even when some CPUs are suspended due to interruption disabling. - -## Embedded - -openEuler 23.09 Embedded is equipped with an embedded virtualization base that is available in the Jailhouse virtualization solution or the OpenAMP lightweight hybrid deployment solution. You can select the most appropriate solution to suite your services. This version also supports the Robot Operating System (ROS) Humble version, which integrates core software packages such as ros-core, ros-base, and simultaneous localization and mapping (SLAM) to meet the ROS 2 runtime requirements. - -- **Southbound ecosystem**: openEuler Embedded Linux supports AArch64 and x86-64 chip architectures and related hardware such as RK3568, Hi3093, RK3399, RK3588, Raspberry Pi 4B, and x86-64 industrial computers. It preliminarily supports AArch32 and RISC-V chip architectures based on QEMU simulation. - -- **Embedded elastic virtualization base**: The converged elastic base of openEuler Embedded is a collection of technologies used to enable multiple OSs or runtimes to run on a system-on-a-chip (SoC). These technologies include bare metal, embedded virtualization, lightweight containers, LibOS, trusted execution environment (TEE), and heterogeneous deployment. - -- **Mixed criticality deployment framework**: The mixed-criticality (MICA) deployment framework is built on the converged elastic base. The unified framework masks the differences between the technologies used in the underlying converged elastic base, enabling Linux to be deployed together with other OSs. - -- **Northbound ecosystem**: More than 350 common embedded software packages can be built using openEuler. The ROS 2 Humble version is supported, which contains core software packages such as ros-core, ros-base, and SLAM. The ROS SDK is provided to simplify embedded ROS development. The soft real-time capability based on Linux kernel 5.10 allows for response to soft real-time interrupts within microseconds. DSoftBus and HiChain for point-to-point authentication of OpenHarmony have been integrated to implement interconnection between openEuler-based embedded devices and between openEuler-based embedded devices and OpenHarmony-based devices. - -- **UniProton**: This hard RTOS features ultra-low latency and flexible MICA deployments. It is suited for industrial control because it supports both microcontroller units and multi-core CPUs. - -## SysCare - -SysCare is a system-level hotfix software that provides security patches and hot fixing for OSs. It can fix system errors without restarting hosts. By combining kernel-mode and user-mode hot patching, SysCare takes over system repair, allowing users to focus on core services. In the future, OS hot upgrade will be provided to further free O&M users and improve O&M efficiency. - -**Patches built in containers**: - -- eBPF is used to monitor the compiler process. In this way, hot patch change information can be obtained in pure user mode without creating character devices, and users can compile hot patches in multiple containers concurrently. - -- Users can install different RPM packages (syscare-build-kmod or syscare-build-ebpf) to use ko or eBPF. The syscare-build process automatically adapts to the corresponding underlying implementation. - -## GCC for openEuler - -GCC for openEuler is developed based on the open source GCC 12.3 and supports features such as automatic feedback-directed optimization (FDO), software and hardware collaboration, memory optimization, SVE, and vectorized math libraries. - -- The default language is upgraded from C14/C++14 to C17/C++17, and more hardware architecture features such as the Armv9-A architecture and x86 AVX512-FP16 are supported. - -- GCC for openEuler supports structure optimization and instruction selection optimization, fully utilizing the hardware features of the Arm architecture to achieve higher operating efficiency. In the benchmark tests such as SPEC CPU 2017, GCC for openEuler delivers much better performance than GCC 10.3 of the upstream community. - -- GCC for openEuler also supports automatic FDO to greatly improve the performance of the MySQL database at the application layer. - -## A-Ops - -The amount of data generated by IT infrastructure and applications sees a 2- to 3-fold increase every year. The application of big data and machine learning technologies is maturing, driving the generation of efficient and intelligent O&M systems to help enterprises reduce costs and improve efficiency. A-Ops is an intelligent O&M framework that supports basic capabilities such as CVE management, exception detection (database scenario), and quick troubleshooting to reduce O&M costs. - -- **Intelligent patch management**: Supports the patch service, kernel hot fix, intelligent patch inspection, and Hybrid management of cold and hot patches. - -- **Exception detection**: Detects network I/O delays, packet loss, interruption, and high disk I/O loads in MySQL and openGauss service scenarios. - -- **Configuration source tracing**: Provides cluster configuration collection and baseline capabilities to implement manageable and controllable configurations. The configuration of the entire cluster is checked and compared with the baseline in real time to quickly identify unauthorized configuration changes and locate faults. - -## A-Ops gala - -The gala project will fully support fault diagnosis in Kubernetes scenarios, including application drill-down analysis, observable microservice and DB performance, cloud-native network monitoring, cloud-native performance profiling, process performance diagnosis, and minute-level diagnosis of five types of OS issues (network, drive, process, memory, and scheduling). - -- **Easy deployment of the Kubernetes environment**: gala-gopher can be deployed as a DaemonSet, and a gala-gopher instance is deployed on each worker node. gala-spider and gala-anteater are deployed as containers on the Kubernetes management node. - -- **Application drill-down analysis**: Diagnoses subhealth problems in cloud native scenarios and demarcates problems between applications and the cloud platform in minutes. - -- **Full-stack monitoring**: Provides application-oriented refined monitoring for cross-software stacks, including the language runtime (JVM), glibc, system call, and kernel (TCP, I/O, and scheduling), and allows users to view the impact of system resources on applications in real time. - -- **Full-link monitoring**: Provides network traffic topology (TCP and RPC) and software deployment topology information, and builds a system 3D topology based on the information to accurately show the resource scope on which applications depend and quickly identify the fault radius. - -- **Causal AI model**: Provides visualized root cause derivation to demarcate faults to resource nodes in minutes. - -- **Observable microservice and DB performance**: Provides non-intrusive microservice, DB, and HTTP1.*x* access performance observation, including the throughput, latency, and error rate; and supports refined API observation and HTTP TRACE to view abnormal HTTP requests. - -- **Observable PostgreSQL access performance**: Observes performance in terms of the throughput, latency, and error rate; and supports refined SQL access observation and slow SQL trace to show SQL statements of slow SQL queries. - -- **Cloud-native application performance profiling**: Provides a non-intrusive and zero-modification cross-stack profiling analysis tool and can connect to the common UI front end of Pyroscope. - -- **Cloud-native network monitoring**: Provides TCP, socket, and DNS monitoring for Kubernetes scenarios for more refined network monitoring. - -- **Process performance diagnosis**: Provides process-level performance problem diagnosis for middleware (such as MySQL and Redis) in cloud native scenarios, monitors process performance KPIs and process-related system-layer metrics (such as I/O, memory, and TCP), and detects process performance KPI exceptions and system-layer metrics that affect the KPIs. - -## sysMaster - -sysMaster is a collection of ultra-lightweight and highly reliable service management programs. It provides an innovative implementation of PID 1 to replace the conventional init process. Written in Rust, sysMaster is equipped with fault monitoring, second-level self-recovery, and quick startup capabilities, which help improve OS reliability and service availability. In version 0.5.0, sysMaster can manage system services in container and VM scenarios. - -- Added the devMaster component to manage device hot swap. - -- Added the live update and hot reboot functions to sysMaster. - -- Allows PID 1 to run on a VM. - -## utsudo - -utsudo uses Rust to reconstruct sudo to deliver a more efficient, secure, and flexible privilege escalation tool. The involved modules include the common utility, overall framework, and function plugins. - -- **Access control**: Restricts the commands that can be executed by users as required, and specifies the required authentication method. - -- **Audit log**: Records and traces the commands and tasks executed by each user using utsudo. - -- **Temporary privilege escalation**: allows common users to enter their passwords to temporarily escalate to the super user to execute specific commands or tasks. - -- **Flexible configuration**: Allows users to set arguments such as command aliases, environment variables, and execution parameters to meet complex system management requirements. - -## utshell - -utshell is a new shell that inherits the usage habits of Bash. It can interact with users through command lines, specifically, responding to user operations to execute commands and provide feedback. In addition, it can execute automated scripts to facilitate O&M. - -- **Command execution**: Runs commands deployed on the user's machine and sends return values to the user. - -- **Batch processing**: Automates task execution using scripts. - -- **Job control**: Concurrently executes multiple user commands as background jobs, and manages and controls the tasks that are executed concurrently. - -- **Historical records**: Records the commands entered by users. - -- **Command aliases**: Allows users to create aliases for commands to customize their operations. - -## migration-tools - -Developed by UnionTech Software Technology Co., Ltd., migration-tools is oriented to users who want to quickly, smoothly, stably, and securely migrate services to the openEuler OS. migration-tools consists of the following modules: - -- **Server module**: It is developed on the Python Flask Web framework. As the core of migration-tools, it receives task requests, processes execution instructions, and distributes the instructions to each Agent. - -- **Agent module**: It is installed in the OS to be migrated to receive task requests from the Server module and perform migration. - -- **Configuration module**: It reads configuration files for the Server and Agent modules. - -- **Log module**: It records logs during migration. - -- **Migration evaluation module**: It provides evaluation reports such as basic environment check, software package comparison analysis, and ABI compatibility check before migration, providing a basis for users' migration work. - -- **Migration function module**: It provides migration with a few clicks, displays the migration progress, and checks the migration result. - -## DDE - -DDE focuses on delivering polished user interaction and visual design. DDE is powered by independently developed core technologies for desktop environments and provides login, screen locking, desktop and file manager, launcher, dock, window manager, control center, and more functions. As one of the preferred desktop environments, DDE features a user-friendly interface, elegant interaction, high reliability, and privacy protection. You can use DDE to work more creatively and efficiently or enjoy media entertainment while keeping in touch with friends. - -## Kmesh - -Based on the programmable kernel, Kmesh offloads service governance to the OS, thus shortening the inter-service communication latency to only 1/5 of the industry average. - -- Kmesh can connect to a mesh control plane (such as Istio) that complies with the Dynamic Resource Discovery (xDS) protocol. - -- **Traffic orchestration**: Polling and other load balancing policies, L4 and L7 routing support, and backend service policies available in percentage mode are supported. - -- **Sockmap for mesh acceleration**: Take the typical service mesh scenario as an example. When a sockmap is used, the eBPF program takes over the communication between service containers and Envoy containers. As a result, the communication path is shortened to achieve mesh acceleration. The eBPF program can also accelerate the communication between pods on the same node. - -## RISC-V QEMU - -openEuler 23.09 is released with support for the RISC-V architecture. openEuler 23.09 aims to provide basic support for upper-layer applications and is highly customizable, flexible, and secure. It provides a stable and reliable operating environment for the computing platform of the RISC-V architecture, facilitating the installation and verification of upper-layer applications and promoting the enrichment and quality improvement of the software ecosystem in the RISC-V architecture. - -- The OS kernel is updated to version 6.4.0, which is consistent with mainstream architectures. - -- It features a stable base, including core functions such as processor management, memory management, task scheduling, and device drivers, as well as common utilities. - -## DIM - -Dynamic Integrity Measurement (DIM) measures key data (such as code segments) in memory during program running and compares the measurement result with the reference value to determine whether the data in memory has been tampered with. In this way, attacks can be detected and countermeasures can be taken. - -- Measures user-mode processes, kernel modules, and code segment in the kernel memory. - -- Extends measurements to the PCR register of the TPM 2.0 chip for remote attestation. - -- Configures measurements and verifies measurement signatures. - -- Generates and imports measurement baseline data using tools, and verifies baseline data signatures. - -- Supports the SM3 algorithm. - -## Kuasar - -Kuasar is a container runtime that supports unified management of multiple types of sandboxes. It supports multiple mainstream sandbox isolation technologies. Based on the Kuasar container runtime combined with the iSulad container engine and StratoVirt virtualization engine, openEuler builds lightweight full-stack self-developed secure containers for cloud native scenarios, delivering key competitiveness of ultra-low overhead and ultra-fast startup. - -Kuasar 0.1.0 supports the StratoVirt lightweight VM sandbox and StratoVirt secure container instances created through Kubernetes+iSulad. - -- Compatible with the Kubernetes ecosystem when the iSulad container engine interconnects with the Kuasar container. - -- Secure container sandboxes based on the StratoVirt lightweight VM sandbox. - -- StratoVirt secure containers for precise resource restriction and management. - -## sysBoost - -SsysBoost is a tool for optimizing the system microarchitecture for applications. The optimization involves assembly instructions, code layout, data layout, memory huge pages, and system calls. - -- **Binary file merging**: Only full static merging is supported. Applications and their dependent dynamic libraries are merged into one binary file, and segment-level reordering is performed. Multiple discrete code segments or data segments are merged into one to improve application performance. - -- **sysBoost daemon**: sysBoost registers with systemd to enable out-of-the-box optimization. systemd will start the sysBoost daemon after the system is started. Then, the sysBoost daemon reads the configuration file to obtain the binary files to be optimized and the corresponding optimization methods. - -- RTO binary file loading kernel module: This binary loading module is added to automatically load the optimized binary file when the kernel loads binary files. - -- **Huge page pre-loading of binary code or data segments**: sysBoost provides the huge page pre-loading function. After binary optimization is complete, sysBoost immediately loads the content to the kernel as a huge page. When an application is started, sysBoost maps the pre-loaded content to the user-mode page table in batches to reduce page faults and memory access delay of the application, thereby improving the application startup speed and running efficiency. - -## CTinspector - -CTinspector is a language VM running framework developed by China Telecom e-Cloud Technology Co., Ltd. based on the eBPF instruction set. The CTinspector running framework enables application instances to be quickly expanded to diagnose network performance bottlenecks, storage I/O hotspots, and load balancing, improving the stability and timeliness of diagnosis during system running. - -- CTinspector uses a packet VM of the eBPF instruction set. The minimum size of the packet VM is 256 bytes, covering all VM components, including registers, stack segments, code segments, data segments, and page tables. - -- The packet VM supports independent migration. That is, the code in the packet VM can invoke migrate kernel function to migrate the packet VM to a specified node. - -- The packet VM also supports resumable execution. That is, after being migrated to another node, the packet VM can continue to execute the next instruction from the position where it has been interrupted on the previous node. - -## CVE-ease - -CVE-ease is an innovative Common Vulnerabilities and Exposures (CVE) platform developed by China Telecom e-Cloud Technology Co., Ltd. It collects various CVE information released by multiple security platforms and notifies users of the information through multiple channels, such as email, WeChat, and DingTalk. The CVE-ease platform aims to help users quickly learn about and cope with vulnerabilities in the system. In addition to improving system security and stability, users can view CVE details on the CVE-ease platform, including vulnerability description, impact scope, and fixing suggestions, and select a fixing solution as required. - -CVE-ease has the following capabilities: - -- Dynamically tracks CVEs on multiple platforms in real time and integrates the information into the CVE database. - -- Extracts key information from the collected CVE information and updates changed CVE information in real time. - -- Automatically maintains and manages the CVE database. - -- Queries historical CVE information based on various conditions in interactive mode. - -- Reports historical CVE information in real time through WeCom, DingTalk, and email. - -## PilotGo - -The PilotGo O&M management platform is a plugin-based O&M management tool developed by the openEuler community. It adopts a lightweight modular design of functional modules that can be iterated and evolved independently, while ensuring stability of core functions. Plugins are used to enhance platform functions and remove barriers between different O&M components, implementing global status awareness and automation. - -PilotGo has the following core functional modules: - -- **User management**: Manages users by group based on the organizational structure, and imports existing platform accounts, facilitating migration. - -- **Permission management**: Supports RBAC-based permission management, which is flexible and reliable. - -- **Host management**: visualized front-end status, software package management, service management, and kernel parameter optimization. - -- **Batch management**: Concurrently performs O&M operation, which is stable and efficient. - -- **Log audit**: Traces and records user and plugin change operations, facilitating issue backtracking and security audit. - -- **Alarm management**: Detects platform exceptions in real time. - -- **Real-time exception detection**: Extends platform functions and associates plugins to realize automation and reduce manual intervention. - -## CPDS - -The wide application of cloud native technologies makes modern application deployment environments more and more complex. The container architecture provides flexibility and convenience, but also brings more monitoring and maintenance challenges. Container Problem Detect System (CPDS) is developed to ensure reliability and stability of containerized applications. - -- **Cluster information collection**: Node agents are implemented on host machines to monitor key container services using systemd, initv, eBPF, and other technologies. Cross-NS agents are configured on nodes and containers in non-intrusive mode to keep track of the application status, resource consumption, key system function execution status, and I/O execution status of containers. The collected information covers network, kernel, and drive LVM of the nodes. - -- **Cluster exception detection**: Raw data from each node is collected to detect exceptions based on exception rules and extract key information. Then, the detection results and raw data are uploaded online and saved permanently. - -- **Fault/Sub-Health diagnosis on nodes and service containers**: Nodes and service containers are diagnosed based on exception detection data. Diagnosis results are saved permanently and can be displayed on the UI for users to view real-time and historical diagnosis data. - -## EulerMaker Build System - -EulerMaker is a package build system. It converts source code into binary packages and allows developers to assemble and tailor scenario-specific OSs based on their requirements. It provides incremental/full build, package layer tailoring, and image tailoring capabilities. - -- Incremental/Full build: Analyzes the impact based on software changes and dependencies, obtains the list of packages to be built, and delivers parallel build tasks based on the dependency sequence. - -- **Build dependency query**: Provides a software package build dependency table in a project, and filters and collects statistics on software package dependencies and depended software packages. - -- Layer tailoring: In a build project, developers can select and configure layer models to tailor patches, build dependencies, installation dependencies, and compilation options for software packages. - -- **Image tailoring**: Developers can configure the repository source to generate ISO, QCOW2, and container OS images, and tailor the list of software packages for the images. diff --git a/docs/en/docs/Releasenotes/known-issues.md b/docs/en/docs/Releasenotes/known-issues.md deleted file mode 100644 index 790e6f011eafc512e5fc416d14f0637335d4de95..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/known-issues.md +++ /dev/null @@ -1,10 +0,0 @@ -# Known Issues - -| No. | Issue Ticket Number | Issue Description | Severity Level | Impact Analysis | Workaround Measure | History Scenario | -| ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | -------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ---------------- | -| 1 | [I5LZXD](https://gitee.com/src-openEuler/openldap/issues/I5LZXD) | openldap build problem in openEuler:22.09 | Minor | The case fails to be executed during the build. This problem is caused by the case design. The impact is controllable. Switch to the sleep mode and wait until the operation is complete. The operation occasionally fails in high-load scenarios. | Skip related case. Track the upstream community to solve the problem. | | -| 2 | [I5NLZI](https://gitee.com/src-openEuler/dde/issues/I5NLZI) | \[openEuler 22.09 rc2] Some application icons in the initiator are improperly displayed. | Minor | Only the initiator icon of the DDE desktop is improperly displayed. This issue does not affect functions. The overall impact on usability is controllable. | Switch the theme to avoid this problem. | | -| 3 | [I5P5HM](https://gitee.com/src-openEuler/afterburn/issues/I5P5HM) | \[22.09_RC3_EPOL]\[arm/x86] **Failed to stop afterburn-sshkeys@.service** is displayed when uninstalling the afterburn. | Minor | | | | -| 4 | [I5PQ3O](https://gitee.com/src-openEuler/openmpi/issues/I5PQ3O) | \[openEuler-22.09-RC3] An error is reported when the **ompi-clean -v -d** parameter is executed. | Major | This package is used by the NestOS and has a limited application scope. By default, this package is enabled by the **core** user in the NestOS. This package has little impact on the server version. | No workaround is provided by the SIG. | | -| 5 | [I5Q2FE](https://gitee.com/src-openEuler/udisks2/issues/I5Q2FE) | udisks2 build problem in openEuler:22.09 | Minor | The case fails to be executed during the build. The environment is not retained, and the problem does not recur during long-term local build. | Keep tracking the build success rate in the community. | | -| 6 | [I5SJ0R](https://gitee.com/src-openEuler/podman/issues/I5SJ0R) | \[22.09RC5 arm/x86] An error is reported when executing **podman create --blkio-weight-device /dev/loop0:123:15 fedora ls**. | Minor | blkio-weight is a kernel feature of version 4.xx. Not supported in version 5.10 | Upgrade the podman component. | | diff --git a/docs/en/docs/Releasenotes/release_notes.md b/docs/en/docs/Releasenotes/release_notes.md deleted file mode 100644 index f9329ac8f0f47ad6e69b37046af5a8bc0f57c97b..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/release_notes.md +++ /dev/null @@ -1,3 +0,0 @@ -# Release Notes - -This document is the release notes of openEuler 24.03 LTS SP1. diff --git a/docs/en/docs/Releasenotes/resolved-issues.md b/docs/en/docs/Releasenotes/resolved-issues.md deleted file mode 100644 index 9a3f7ccfb0e54e0c67438559fae8431322fd51fb..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/resolved-issues.md +++ /dev/null @@ -1,15 +0,0 @@ -# Resolved Issues - -For the complete issue list, see [Issues](https://gitee.com/organizations/src-openeuler/issues). - -For the complete list kernel related commits, see [Commits](https://gitee.com/openeuler/kernel/commits/openEuler-21.03). - -For details about resolved issues, see [Table 1](#table249714911433). - -**Table 1** Resolved issues - -| ISSUE | Description | -| :----------------------------------------------------------- | :----------------------------------------------------------- | -|[I5J302](https://gitee.com/open_euler/dashboard?issue_id=I5J302)|\[Installation conflict arm/x86_64]openEuler 22.09: fwupd conflicts with dbxtool during installation| -|[I5J36Q](https://gitee.com/open_euler/dashboard?issue_id=I5J36Q)|\[Installation conflict arm/x86_64]openEuler 22.09: python3-wrapt conflicts during installation | -|[I5J3K1](https://gitee.com/open_euler/dashboard?issue_id=I5J3K1)|\[Installation conflict arm/x86_64]openEuler 22.09 : mariadb conflicts with mysql during installation| diff --git a/docs/en/docs/Releasenotes/source-code.md b/docs/en/docs/Releasenotes/source-code.md deleted file mode 100644 index c207d103605c45658196acc76f1d1bde1e95ef04..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/source-code.md +++ /dev/null @@ -1,8 +0,0 @@ -# Source Code - -openEuler contains two code repositories: - -- Code repository: [https://gitee.com/openeuler](https://gitee.com/openeuler) -- Software package repository: [https://gitee.com/src-openeuler](https://gitee.com/src-openeuler) - -The openEuler release packages also provide the source ISO files. For details, see [OS Installation](./installing-the-os.md). diff --git a/docs/en/docs/Releasenotes/terms-of-use.md b/docs/en/docs/Releasenotes/terms-of-use.md deleted file mode 100644 index 8fab84f65144bb0c9208baf04afcc3251bccf46d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Releasenotes/terms-of-use.md +++ /dev/null @@ -1,13 +0,0 @@ -# Terms of Use - -**Copyright © 2023 openEuler Community** - -Your replication, use, modification, and distribution of this document are governed by the Creative Commons License Attribution-ShareAlike 4.0 International Public License \(CC BY-SA 4.0\). You can visit [https://creativecommons.org/licenses/by-sa/4.0/](https://creativecommons.org/licenses/by-sa/4.0/) to view a human-readable summary of \(and not a substitute for\) CC BY-SA 4.0. For the complete CC BY-SA 4.0, visit [https://creativecommons.org/licenses/by-sa/4.0/legalcode](https://creativecommons.org/licenses/by-sa/4.0/legalcode). - -**Trademarks and Permissions** - -All trademarks and registered trademarks mentioned in the documents are the property of their respective holders. The use of the openEuler trademark must comply with the [Use Specifications of the openEuler Trademark](https://www.openeuler.org/en/other/brand/). - -**Disclaimer** - -This document is used only as a guide. Unless otherwise specified by applicable laws or agreed by both parties in written form, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, including but not limited to non-infringement, timeliness, and specific purposes. diff --git a/docs/en/docs/SecHarden/account-passwords.md b/docs/en/docs/SecHarden/account-passwords.md deleted file mode 100644 index a5a584fd42ee7212f4ebbd28dcb006a2898da3e6..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/account-passwords.md +++ /dev/null @@ -1,329 +0,0 @@ -# Account Passwords - -- [Account Passwords](#account-passwords) - - [Shielding System Accounts](#shielding-system-accounts) - - [Restricting Account Permissions on the su Command](#restricting-account-permissions-on-the-su-command) - - [Setting Password Complexity](#setting-password-complexity) - - [Setting the Password Validity Period](#setting-the-password-validity-period) - - [Setting Password Encryption Algorithms](#setting-password-encryption-algorithms) - - [Locking an Account After Three Login Failures](#locking-an-account-after-three-login-failures) - - [Hardening the su Command](#hardening-the-su-command) - -## Shielding System Accounts - -### Description - -Accounts excluding user accounts are system accounts. System accounts cannot be used for logins or performing other operations. Therefore, system accounts must be shielded. - -### Implementation - -Modify the shell of a system account to **/sbin/nologin**. - -```shell -usermod -L -s /sbin/nologin $systemaccount -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->_$systemaccount_ indicates the system account. - -## Restricting Account Permissions on the su Command - -### Description - -The **su** command is used to switch user accounts. To improve system security, only the user **root** and users in the **wheel** group can use the **su** command. - -### Implementation - -Modify the **/etc/pam.d/su** file as follows: - -```text -auth required pam_wheel.so use_uid -``` - -**Table 1** Configuration item in pam\_wheel.so - - - - - - - - - - -

Item

-

Description

-

use_uid

-

UID of the current account.

-
- -## Setting Password Complexity - -### Description - -You can set the password complexity requirements by modifying the corresponding configuration file. You are advised to set the password complexity based on the site requirements. - -### Implementation - -The password complexity is implemented by the **pam\_pwquality.so** and **pam\_pwhistory.so** modules in the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files. You can modify the configuration items of the two modules to change the password complexity requirements. - -### Example - -This section provides an example for configuring password complexity. - -**Password Complexity Requirements** - -1. Contains at least eight characters. -2. Contains at least three types of the following characters: - - - At least one lowercase letter - - - At least one uppercase letter - - - At least one digit - - - At least one space or one of the following special characters: \` \~ ! @ \# $ % ^ & \* \( \) - \_ = + \\ | \[ \{ \} \] ; : ' " , < . \> / ? - -3. Cannot be the same as an account name or the account name in reverse order. -4. Cannot be the last five passwords used. - -**Implementation** - -Add the following content to the first two lines of the **password** configuration item in the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files: - -```text -password requisite pam_pwquality.so minlen=8 minclass=3 enforce_for_root try_first_pass local_users_only retry=3 dcredit=0 ucredit=0 lcredit=0 ocredit=0 -password required pam_pwhistory.so use_authtok remember=5 enforce_for_root -``` - -**Configuration Item Description** - -For details about the configuration items of **pam\_pwquality.so** and **pam\_pwhistory.so**, see [Table 1](#table201221044172117) and [Table 2](#table1212544452120), respectively. - -**Table 1** Configuration items in pam\_pwquality.so - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Item

-

Description

-

minlen=8

-

A password must contain at least eight characters.

-

minclass=3

-

A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters.

-

ucredit=0

-

A password contains any number of uppercase letters.

-

lcredit=0

-

A password contains any number of lowercase letters.

-

dcredit=0

-

A password contains any number of digits.

-

ocredit=0

-

A password contains any number of special characters.

-

retry=3

-

Each time a maximum of three password changes is allowed.

-

enforce_for_root

-

This configuration is also effective for user root.

-
- -**Table 2** Configuration items in pam\_pwhistory.so - - - - - - - - - - - - - -

Item

-

Description

-

remember=5

-

A password must be different from the last five passwords used.

-

enforce_for_root

-

This configuration is also effective for user root.

-
- -## Setting the Password Validity Period - -### Description - -To ensure system security, you are advised to set the password validity period and notify users to change passwords before the passwords expire. - -### Implementation - -The password validity period is set by modifying the **/etc/login.defs** file. [Table 1](#en-us_topic_0152100281_t77b5d0753721450c81911c18b74e82eb) describes the hardening items. All hardening items in the table are in the **/etc/login.defs** file. You can directly modify the items in the configuration file. - -**Table 1** Configuration items in login.defs - - - - - - - - - - - - - - - - - - - - - - - - -

Item

-

Description

-

Suggestion

-

Configured as Suggested

-

PASS_MAX_DAYS

-

Maximum validity period of a password.

-

90

-

No

-

PASS_MIN_DAYS

-

Minimum interval between password changes.

-

0

-

No

-

PASS_WARN_AGE

-

Number of days before the password expires.

-

7

-

No

-
- ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The **login.defs** file is used to set restrictions on user accounts, such as setting the maximum password validity period and maximum length. The configuration in this file is invalid for the user **root**. If the **/etc/shadow** file contains the same items, the **/etc/shadow** configuration takes precedence over the **/etc/login.defs** configuration. When a user attempts to log in after the password expires, the user will be informed of the password expiry and is required to change the password. If the user does not change the password, the user cannot access the system. - -## Setting Password Encryption Algorithms - -### Description - -For system security, passwords cannot be stored in plaintext in the system and must be encrypted. The passwords that do not need to be restored must be encrypted using irreversible algorithms. Set the password encryption algorithm to SHA-512. This item has been set by default in openEuler. The preceding settings can effectively prevent password disclosure and ensure password security. - -### Implementation - -To set the password encryption algorithm, add the following configuration to the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files: - -```text -password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok -``` - -**Table 1** Configuration items in pam\_unix.so - - - - - - - - - - -

Item

-

Description

-

sha512

-

The SHA-512 algorithm is used for password encryption.

-
- -## Locking an Account After Three Login Failures - -### Description - -To ensure user system security, you are advised to set the maximum number of failed login attempts \(three attempts are recommended\) and the automatic unlocking time \(300 seconds are recommended\) for a locked account. - -If an account is locked, any input is invalid but does not reset the locking countdown timer. Records of the user's invalid inputs are cleared once unlocked. The preceding settings protect passwords from being forcibly cracked and improve system security. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->By default, the maximum number of failed login attempts is 3 in openEuler. After an account is locked, the automatic unlock time is 60 seconds. - -### Implementation - -The password complexity is set by modifying the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files. The maximum number of failed login attempts is set to **3**, and the unlocking time after an account is locked is set to **300** seconds. The configuration is as follows: - -```text -auth required pam_faillock.so preauth audit deny=3 even_deny_root unlock_time=300 -auth [default=die] pam_faillock.so authfail audit deny=3 even_deny_root unlock_time=300 -auth sufficient pam_faillock.so authsucc audit deny=3 even_deny_root unlock_time=300 -``` - -**Table 1** Configuration items in pam\_faillock.so - - - - - - - - - - - - - - - - - - - -

Item

-

Description

-

authfail

-

Captures account login failure events.

-

deny=3

-

A user account will be locked after three failed login attempts.

-

unlock_time=300

-

A locked common user account is automatically unlocked after 300 seconds.

-

even_deny_root

-

This configuration is also effective for user root.

-
- -## Hardening the su Command - -### Description - -To enhance system security and prevent the environment variables of the current user from being brought into other environments when you run the **su** command to switch to another user, this item has been configured by default in openEuler. The **PATH** variable is always initialized when the **su** command is used to switch users. - -### Implementation - -Modify the **/etc/login.defs** file. The configuration is as follows: - -```text -ALWAYS_SET_PATH=yes -``` diff --git a/docs/en/docs/SecHarden/appendix.md b/docs/en/docs/SecHarden/appendix.md deleted file mode 100644 index f57674a77570540cfbbaa7b76f26772b1d9320d3..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/appendix.md +++ /dev/null @@ -1,31 +0,0 @@ -# Appendix - -This chapter describes the file permissions and **umask** values. - -- [Appendix](#appendix) - - [Permissions on Files and Directories](#permissions-on-files-and-directories) - - [umask Values](#umask-values) - -## Permissions on Files and Directories - -Permission on files and directories in Linux specifies the users who can access and perform operations on files and directories and the access and operation modes. Permissions on files and directories include read only, write only, and execute. - -The following types of users can access files and directories: - -- File creator -- Users in the same group as a file creator -- Users not in the same group as a file creator - -An example of permission on files and directories is described as follows: - -If the permission on **/usr/src** is set to **755** which is 111101101 in binary mode, permissions for each type of users are described as follows: - -- The left-most **111** indicates that the file owner can read, write, and execute the file. -- The middle **101** indicates the group users can read and execute but cannot write the file. -- The right-most **101** indicates that other users can read and execute but cannot write the file. - -## umask Values - -When a user creates a file or directory, the file or directory has a default permission. The default permission is specified by the **umask** value. - -The **umask** value is the complement of the permission value. The actual permission value is obtained by subtracting the **umask** value from the default maximum permission value. The default maximum permission of a file is readable and writable. The default maximum permission of a directory is readable, writable, and executable. The default permission of a file is 666 minus the **umask** value. The default permission of a directory is 777 minus the **umask** value. diff --git a/docs/en/docs/SecHarden/authentication-and-authorization.md b/docs/en/docs/SecHarden/authentication-and-authorization.md deleted file mode 100644 index f5b0884954ccc2ed1ec98207ac52d1aa2294305c..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/authentication-and-authorization.md +++ /dev/null @@ -1,145 +0,0 @@ -# Authentication and Authorization - -## Setting a Warning for Remote Network Access - -### Description - -A warning for remote network access is configured and displayed for users who attempt to remotely log in to the system. The warning indicates the penalty for authorized access and is used to threaten potential attackers. When the warning is displayed, system architecture and other system information are hidden to protect the system from being attacked. - -### Implementation - -This setting can be implemented by modifying the **/etc/issue.net** file. Replace the original content in the **/etc/issue.net** file with the following information \(which has been set by default in openEuler\): - -```text -Authorized users only. All activities may be monitored and reported. -``` - -## Forestalling Unauthorized System Restart by Pressing Ctrl+Alt+Delete - -### Description - -By default, you can restart the system by pressing **Ctrl**+**Alt**+**Delete**. You are advised to disable this function to prevent data loss due to misoperations. - -### Implementation - -To disable the feature of restarting the system by pressing **Ctrl**+**Alt**+**Delete**, perform the following steps: - -1. Run the following commands to delete the two **ctrl-alt-del.target** files: - - ```shell - rm -f /etc/systemd/system/ctrl-alt-del.target - rm -f /usr/lib/systemd/system/ctrl-alt-del.target - ``` - -2. Change **\#CtrlAltDelBurstAction=reboot-force** to **CtrlAltDelBurstAction=none** in the **/etc/systemd/system.conf** file. -3. Run the following command to restart systemd for the modification to take effect. Note that running the command may cause system services to be unavailable or restarted temporarily. In addition, you must be the **root** user or a user with the sudo permission to perform this operation. - - ```shell - systemctl daemon-reexec - ``` - -## Setting an Automatic Exit Interval for Shell - -### Description - -An unattended shell is prone to listening or attacks. Therefore, it is advised that a mechanism be configured to ensure that a shell can automatically exit when it does not run for a period. - -### Implementation - -At the end of file **/etc/profile**, set the **TMOUT** field \(unit: second\) that specifies the interval for automatic exit as follows: - -```shell -export TMOUT=300 -``` - -## Setting the Default umask Value for Users to 0077 - -### Description - -The **umask** value is used to set default permission on files and directories. A smaller **umask** value indicates that group users or other users have incorrect permission, which brings system security risks. Therefore, the default **umask** value must be set to **0077** for all users, that is, the default permission on user directories is **700** and the permission on user files is **600**. The **umask** value indicates the complement of a permission. For details about how to convert the **umask** value to a permission, see [umask Values](./appendix.md#umask-values). - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->By default, the **umask** value of the openEuler user is set to **0022**. - -### Implementation - -1. Add **umask 0077** to the **/etc/bashrc** file and all files in the **/etc/profile.d/** directory. - - ```shell - echo "umask 0077" >> $FILE - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >_$FILE_ indicates the file name, for example, echo "umask 0077" \>\> /etc/bashrc. - -2. Set the ownership and group of the **/etc/bashrc** file and all files in the **/etc/profile.d/** directory to **root**. - - ```shell - chown root.root $FILE - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >_$FILE_ indicates the file name, for example, **chown root.root /etc/bashrc**. - -## Setting the GRUB2 Encryption Password - -### Description - -GRand Unified Bootloader \(GRUB\) is an operating system boot manager used to boot different systems \(such as Windows and Linux\). GRUB2 is an upgraded version of GRUB. - -When starting the system, you can modify the startup parameters of the system on the GRUB2 screen. To ensure that the system startup parameters are not modified randomly, you need to encrypt the GRUB2 screen. The startup parameters can be modified only when the correct GRUB2 password is entered. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The default password of GRUB2 is **openEuler\#12**. You are advised to change the default password upon the first login and periodically update the password. If the password is leaked, startup item configurations may be modified, causing the system startup failure. - -### Implementation - -1. Run the **grub2-mkpasswd-pbkdf2** command to generate an encrypted password. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >SHA-512 is used as the GRUB2 encryption algorithm. - - ```shell - $ grub2-mkpasswd-pbkdf2 - Enter password: - Reenter password: - PBKDF2 hash of your password is - grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08 - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Enter the same password in the **Enter password** and **Reenter password** lines. - >After **openEuler\#12** is encrypted by **grub2-mkpasswd-pbkdf2**, the output is **grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08**. The ciphertext is different each time. - -2. Open **/boot/efi/EFI/openEuler/grub.cfg** in a vi editor. In different modes, the paths of the **grub.cfg** file are different. See the note below. Append the following fields to the beginning of **/boot/efi/EFI/openEuler/grub.cfg**. - - ```text - set superusers="root" - password_pbkdf2 root grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08 - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- In different modes, the paths of the **grub.cfg** file are different: In the UEFI mode of the x86 architecture, the path is **/boot/efi/EFI/openEuler/grub.cfg**. In the Legacy BIOS mode of the x86 architecture, the path is **/boot/grub2/grub.cfg**. In the aarch64 architecture, the path is **/boot/efi/EFI/openEuler/grub.cfg**. - >- The **superusers** field is used to set the account name of the super GRUB2 administrator. - >- The first parameter following the **password\_pbkdf2** field is the GRUB2 account name, and the second parameter is the encrypted password of the account. - -## Setting the Secure Single-user Mode - -### Description - -When you log in to the system as user **root** in single-user mode, if the **root** password is not set, high security risks exist. - -### Implementation - -This setting can be implemented by modifying the **/etc/sysconfig/init** file. Set **SINGLE** to **SINGLE=/sbin/sulogin**. - -## Disabling Interactive Startup - -### Description - -With interactive guidance, console users can disable audit, firewall, or other services, which compromises system security. Users can disable interactive startup to improve security. This item is disabled by default in openEuler. - -### Implementation - -This setting can be implemented by modifying the **/etc/sysconfig/init** file. Set **PROMPT** to **no**. diff --git a/docs/en/docs/SecHarden/figures/en-us_image_0221925211.png b/docs/en/docs/SecHarden/figures/en-us_image_0221925211.png deleted file mode 100644 index 62ef0decdf6f1e591059904001d712a54f727e68..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SecHarden/figures/en-us_image_0221925211.png and /dev/null differ diff --git a/docs/en/docs/SecHarden/figures/en-us_image_0221925212.png b/docs/en/docs/SecHarden/figures/en-us_image_0221925212.png deleted file mode 100644 index ad5ed3f7beeb01e6a48707c4806606b41d687e22..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SecHarden/figures/en-us_image_0221925212.png and /dev/null differ diff --git a/docs/en/docs/SecHarden/file-permissions.md b/docs/en/docs/SecHarden/file-permissions.md deleted file mode 100644 index eaee0cb7a3f59b3cb970a7bf89d015934f036e16..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/file-permissions.md +++ /dev/null @@ -1,232 +0,0 @@ - -# File Permissions - -- [File Permissions](#file-permissions) - - [Setting the Permissions on and Ownership of Files](#setting-the-permissions-on-and-ownership-of-files) - - [Deleting Unowned Files](#deleting-unowned-files) - - [Removing a Symbolic Link to /dev/null](#removing-a-symbolic-link-to-devnull) - - [Setting the umask Value for a Daemon](#setting-the-umask-value-for-a-daemon) - - [Adding a Sticky Bit Attribute to Globally Writable Directories](#adding-a-sticky-bit-attribute-to-globally-writable-directories) - - [Disabling the Globally Writable Permission on Unauthorized Files](#disabling-the-globally-writable-permission-on-unauthorized-files) - - [Restricting Permissions on the at Command](#restricting-permissions-on-the-at-command) - - [Restricting Permissions on the cron Command](#restricting-permissions-on-the-cron-command) - - [Restricting Permissions on the sudo Command](#restricting-permissions-on-the-sudo-command) - -## Setting the Permissions on and Ownership of Files - -### Description - -In Linux, all objects are processed as files. Even a directory will be processed as a large file containing many files. Therefore, the most important thing in Linux is the security of files and directories. Their security is ensured by permissions and owners. - -By default, the permissions and ownership of common directories, executable files, and configuration files in the system are set in openEuler. - -### Implementation - -The following uses the **/bin** directory as an example to describe how to change the permission and ownership of a file: - -- Modify the file permission. For example, set the permission on the **/bin** directory to **755**. - - ```shell - chmod 755 /bin - ``` - -- Change the ownership of the file. For example, set the ownership and group of the **/bin** directory to **root:root**. - - ```shell - chown root:root /bin - ``` - -## Deleting Unowned Files - -### Description - -When deleting a user or group, the system administrator may forget to delete the files of the user or group. If the name of a new user or group is the same as that of the deleted user or group, the new user or group will own files on which it has no permission. You are advised to delete these files. - -### Implementation - -Delete the file whose user ID does not exist. - -1. Search for the file whose user ID does not exist. - - ```shell - find / -nouser - ``` - -2. Delete the found file. In the preceding command, _filename_ indicates the name of the file whose user ID does not exist. - - ```shell - rm -f filename - ``` - -Delete the file whose group ID does not exist. - -1. Search for the file whose user ID does not exist. - - ```shell - find / -nogroup - ``` - -2. Delete the found file. In the preceding command, _filename_ indicates the name of the file whose group ID does not exist. - - ```shell - rm -f filename - ``` - -## Removing a Symbolic Link to /dev/null - -### Description - -A symbolic link to **/dev/null** may be used by malicious users. This affects system security. You are advised to delete these symbolic links to improve system security. - -### Special Scenario - -After openEuler is installed, symbolic links to **/dev/null** may exist. These links may have corresponding functions. \(Some of them are preconfigured and may be depended by other components.\) Rectify the fault based on the site requirements. For details, see [Implementation](#en-us_topic_0152100319_s1b24647cdd834a8eaca3032611baf072). - -For example, openEuler supports UEFI and legacy BIOS installation modes. The GRUB packages supported in the two boot scenarios are installed by default. If you select the legacy BIOS installation mode, a symbolic link **/etc/grub2-efi.cfg** is generated. If you select the UEFI installation mode, a symbolic link **/etc/grub2.cfg** is generated. You need to process these symbolic links based on the site requirements. - -### Implementation - -1. Run the following command to search for symbolic links to **/dev/null**: - - ```shell - find dirname -type l -follow 2>/dev/null - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >_dir__name_ indicates the directory to be searched. Normally, key system directories, such as **/bin**, **/boot**, **/usr**, **/lib64**, **/lib**, and **/var**, need to be searched. - -2. If these symbolic links are useless, run the following command to delete them: - - ```shell - rm -f filename - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >_filename_ indicates the file name obtained in [Step 1](#en-us_topic_0152100319_l4dc74664c4fb400aaf91fb314c4f9da6). - -## Setting the umask Value for a Daemon - -### Description - -The **umask** value is used to set default permission on files and directories. If the **umask** value is not specified, the file has the globally writable permission. This brings risks. A daemon provides a service for the system to receive user requests or network customer requests. To improve the security of files and directories created by the daemon, you are advised to set **umask** to **0027**. The **umask** value indicates the complement of a permission. For details about how to convert the **umask** value to a permission, see [umask Values](./appendix.md#umask-values). - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->By default, the **umask** value of the daemon is set to **0022** in openEuler. - -### Implementation - -In configuration file **/etc/sysconfig/init**, add **umask 0027** as a new row. - -## Adding a Sticky Bit Attribute to Globally Writable Directories - -### Description - -Any user can delete or modify a file or directory in a globally writable directory, which leads to unauthorized file or directory deletion. Therefore, the sticky bit attribute is required for globally writable directories. - -### Implementation - -1. Search for globally writable directories. - - ```shell - find / -type d -perm -0002 ! -perm -1000 -ls | grep -v proc - ``` - -2. Add the sticky bit attribute to globally writable directories. _dirname_ indicates the name of the directory that is found. - - ```shell - chmod +t dirname - ``` - -## Disabling the Globally Writable Permission on Unauthorized Files - -### Description - -Any user can modify globally writable files, which affects system integrity. - -### Implementation - -1. Search for all globally writable files. - - find / -type d ( -perm -o+w ) | grep -v proc - find / -type f ( -perm -o+w ) | grep -v proc - -2. View the settings of files \(excluding files and directories with sticky bits\) listed in step 1, and delete the files or disable the globally writable permission on them. Run the following command to remove the permission. In the command, _filename_ indicates the file name. - - ```shell - chmod o-w filename - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >You can run the following command to check whether the sticky bit is set for the file or directory. If the command output contains the **T** flag, the file or directory is with a sticky bit. In the command, _filename_ indicates the name of the file or directory to be queried. - > - >```shell - >ls -l filename - >``` - -## Restricting Permissions on the at Command - -### Description - -The **at** command is used to create a scheduled task. Users who can run the **at** command must be specified to protect the system from being attacked. - -### Implementation - -1. Delete the **/etc/at.deny** file. - - ```shell - rm -f /etc/at.deny - ``` - -2. Run the following commands to create the **/etc/at.allow** file and change its ownership to **root:root**. - - ```shell - touch /etc/at.allow - chown root:root /etc/at.allow - ``` - -3. Set that only user **root** can operate file **/etc/at.allow**. - - ```shell - chmod og-rwx /etc/at.allow - ``` - -## Restricting Permissions on the cron Command - -### Description - -The **cron** command is used to create a routine task. Users who can run the **cron** command must be specified to protect the system from being attacked. - -### Implementation - -1. Delete the **/etc/cron.deny** file. - - ```shell - rm -f /etc/at.deny - ``` - -2. Run the following commands to create the **/etc/cron.allow** file and change its ownership to **root:root**: - - ```shell - touch /etc/cron.allow - chown root:root /etc/cron.allow - ``` - -3. Set that only user **root** can operate file **/etc/cron.allow**. - - ```shell - chmod og-rwx /etc/cron.allow - ``` - -## Restricting Permissions on the sudo Command - -### Description - -A common user can use the **sudo** command to run commands as the user **root**. To harden system security, it is necessary to restrict permissions on the **sudo** command. Only user **root** can use the **sudo** command. By default, openEuler does not restrict the permission of non-root users to run the sudo command. - -### Implementation - -Modify the **/etc/sudoers** file to restrict permissions on the **sudo** command. Comment out the following configuration line: - -```text -#%wheel ALL=(ALL) ALL -``` diff --git a/docs/en/docs/SecHarden/kernel-parameters.md b/docs/en/docs/SecHarden/kernel-parameters.md deleted file mode 100644 index b193426b3b65a5e4bcad99a471cc2c82c2d46cc8..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/kernel-parameters.md +++ /dev/null @@ -1,229 +0,0 @@ -# Kernel Parameters - -- [Kernel Parameters](#kernel-parameters) - - [Hardening the Security of Kernel Parameters](#hardening-the-security-of-kernel-parameters) - - -## Hardening the Security of Kernel Parameters - -### Description - -Kernel parameters specify the status of network configurations and application privileges. The kernel provides system control which can be fine-tuned or configured by users. This function can improve the security of the OS by controlling configurable kernel parameters. For example, you can fine-tune or configure network options to improve system security. - -### Implementation - -1. Write the hardening items in [Table 1](#en-us_topic_0152100187_t69b5423c26644b26abe94d88d38878eb) to the **/etc/sysctl.conf** file. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Record security hardening items as follows: - >``` - >net.ipv4.icmp_echo_ignore_broadcasts = 1 - >net.ipv4.conf.all.rp_filter = 1 - >net.ipv4.conf.default.rp_filter = 1 - >``` - - **Table 1** Policies for hardening the security of kernel parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Item

-

Description

-

Suggestion

-

Configured as Suggested

-

net.ipv4.icmp_echo_ignore_broadcasts

-

Specifies whether ICMP broadcast packets are accepted. They are not accepted according to the hardening policy.

-

1

-

Yes

-

net.ipv4.conf.all.rp_filter

-

Specifies whether the actual source IP address used by a data packet is related to a routing table and whether the data packet receives responses through interfaces. The item is enabled according to the hardening policy.

-

1

-

Yes

-

net.ipv4.conf.default.rp_filter

-

1

-

Yes

-

net.ipv4.ip_forward

-

The IP forwarding function prevents unauthorized IP address packets from being transferred to a network. The item is disabled according to the hardening policy.

-

0

-

Yes

-

net.ipv4.conf.all.accept_source_route

-

accept_source_route indicates that a packet sender can specify a path for sending the packet and a path for receiving a response. The item is disabled according to the hardening policy.

-

0

-

Yes

-

net.ipv4.conf.default.accept_source_route

-

0

-

Yes

-

net.ipv4.conf.all.accept_redirects

-

Specifies whether a redirected ICMP packet is sent. The packet is not sent according to the hardening policy.

-

0

-

Yes

-

net.ipv4.conf.default.accept_redirects

-

0

-

Yes

-

net.ipv6.conf.all.accept_redirects

-

0

-

Yes

-

net.ipv6.conf.default.accept_redirects

-

0

-

Yes

-

net.ipv4.conf.all.send_redirects

-

Specifies whether a redirected ICMP packet is sent to another server. This item is enabled only when the host functions as a route. The item is disabled according to the hardening policy.

-

0

-

Yes

-

net.ipv4.conf.default.send_redirects

-

0

-

Yes

-

net.ipv4.icmp_ignore_bogus_error_responses

-

Fake ICMP packets are not recorded to logs, which saves disk space. The item is enabled according to the hardening policy.

-

1

-

Yes

-

net.ipv4.tcp_syncookies

-

SYN attack is a DoS attack that forces system restart by occupying system resources. TCP-SYN cookie protection is enabled according to the hardening policy.

-

1

-

Yes

-

kernel.dmesg_restrict

-

Hardens dmesg messages. Only the administrator is allowed to view the messages.

-

1

-

Yes

-

kernel.sched_autogroup_enabled

-

Determines whether the kernel automatically groups and schedules threads. After this item is enabled, scheduling groups compete for time slices, and threads in a scheduling group compete for the time slices allocated to the scheduling group. The item is disabled according to the hardening policy.

-

0

-

No

-

kernel.sysrq

-

Disables the magic key.

-
NOTE:

You are advised to disable the magic key so that commands cannot be directly passed to the kernel.

-
-

0

-

Yes

-

net.ipv4.conf.all.secure_redirects

-

Specifies whether redirected ICMP messages sent from any servers or from gateways listed in the default gateway list are accepted. Redirected ICMP messages are received from any servers according to the hardening policy.

-

0

-

Yes

-

net.ipv4.conf.default.secure_redirects

-

0

-

Yes

-
- -2. Run the following command to load the kernel parameters set in the **sysctl.conf** file: - - ``` - sysctl -p /etc/sysctl.conf - ``` - - -### Other Security Suggestions - -- **net.ipv4.icmp\_echo\_ignore\_all**: ignores ICMP requests. - - For security purposes, you are advised to enable this item. The default value is **0**. Set the value to **1** to enable this item. - - After this item is enabled, all incoming ICMP Echo request packets will be ignored, which will cause failure to ping the target host. Determine whether to enable this item based on your actual networking condition. - -- **net.ipv4.conf.all.log\_martians/net.ipv4.conf.default.log\_martians**: logs spoofed, source routed, and redirect packets. - - For security purposes, you are advised to enable this item. The default value is **0**. Set the value to **1** to enable this item. - - After this item is enabled, data from forbidden IP addresses will be logged. Too many new logs will overwrite old logs because the total number of logs allowed is fixed. Determine whether to enable this item based on your actual usage scenario. - -- **net.ipv4.tcp\_timestamps**: disables tcp\_timestamps. - - For security purposes, you are advised to disable tcp\_timestamps. The default value is **1**. Set the value to **0** to disable tcp\_timestamps. - - After this item is disabled, TCP retransmission timeout will be affected. Determine whether to disable this item based on the actual usage scenario. - -- **net.ipv4.tcp\_max\_syn\_backlog**: determines the number of queues that is in SYN\_RECV state. - - This parameter determines the number of queues that is in SYN\_RECV state. When this number is exceeded, new TCP connection requests will not be accepted. This to some extent prevents system resource exhaustion. Configure this parameter based on your actual usage scenario. diff --git a/docs/en/docs/SecHarden/os-hardening-overview.md b/docs/en/docs/SecHarden/os-hardening-overview.md deleted file mode 100644 index 9e6e8fa69978b185307b7ec9c6486c0341e3919c..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/os-hardening-overview.md +++ /dev/null @@ -1,118 +0,0 @@ -# OS Hardening Overview - -- [OS Hardening Overview](#os-hardening-overview) - - [Security Hardening Purpose](#security-hardening-purpose) - - [Security Hardening Solution](#security-hardening-solution) - - [Security Hardening Impacts](#security-hardening-impacts) - -This chapter describes the purpose and solution of openEuler system hardening. - -## security-hardening-purpose - -The OS, as the core of the information system, manages hardware and software resources and is the basis of information system security. Applications must depend on the OS to ensure the integrity, confidentiality, availability, and controllability of information. Without the OS security protection, protective methods against hackers and virus attacks at other layers cannot meet the security requirements. - -Therefore, security hardening is essential for an OS. Security hardening helps build a dynamic and complete security system, enhance product security, and improve product competitiveness. - -## security-hardening-solution - -This section describes the openEuler security hardening solution, including the hardening method and items. - -### Security Hardening Method - -You can manually modify security hardening configurations or run commands to harden the system, or use the security hardening tool to modify security hardening items in batches. The openEuler security hardening tool runs as openEuler-security.service. When the system is started for the first time, the system automatically runs the service to execute the default hardening policy, and automatically set the service not to start as the system starts. - -You can modify the **security.conf** file and use the security hardening tool to implement user-defined security hardening. - -## security hardening impacts - -Security hardening on file permissions and account passwords may change user habits, affecting system usability. For details about common hardening items that affect system usability, see [Table 1](#en-us_topic_0152100325_ta4a48f54ff2849ada7845e2380209917). - -**Table 1** Security hardening impacts - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Item

-

Suggestion

-

Impact

-

Configured By Default

-

Timeout setting on the text-based user interface (TUI)

-

When the TUI is idle for a long period of time, it automatically exits.

-
NOTE:

When a user logs in to the system using SSH, the timeout period is determined by the smaller value of the TMOUT field in the /etc/profile file and the ClientAliveInterval field in the /etc/ssh/sshd_config file. You are advised to set this parameter to 300 seconds.

-
-

If you do not perform any operation on the TUI for a long time, TUI automatically exits.

-

No

-

Password complexity

-

The password is a string containing at least eight characters chosen from three or four of the following types: uppercase letters, lowercase letters, digits, and special characters.

-

All passwords must comply with the complexity requirements.

-

No

-

Password retry limits

-

If a user fails to enter the correct password for three consecutive times when logging in to the OS, the user account will be locked for 60 seconds.

-

After the account is locked, the user can log in to the system only after 60 seconds.

-

Yes

-

Default umask value

-

The default umask value of all users is set to 077 so that the default permission on files created by users is 600 and the default permission on directories is 700.

-

Users must modify the permission on specified files or directories as required.

-

No

-

Password validity period

-

The password validity period can be modified in the /etc/login.defs file and is set to 90 days by default. It can be modified in any time. An expiration notification will be displayed seven days before a password is to expire.

-

When a user attempts to log in after the password expires, the user will be informed of the password expiry and is required to change the password. If the user does not change the password, the user cannot access the system.

-

No

-

su permission control

-

The su command is used to switch user accounts. To improve system security, only the user root and users in the wheel group can use the su command.

-

Common users can successfully run the su command only after joining in the wheel group.

-

Yes

-

Disabling user root from logging in using SSH

-

Set the value of the PermitRootLogin field in the /etc/ssh/sshd_config file to no. In this way, user root cannot directly log in to the system using SSH.

-

You need to log in to the system as a common user in SSH mode and then switch to user root.

-

No

-

Strong SSH encryption algorithm

-

The MACs and Ciphers configurations of SSH services support the CTR and SHA2 algorithms and do not support the CBC, MD5, and SHA1 algorithms.

-

Some early Xshell and PuTTY versions on Windows do not support aes128-ctr, aes192-ctr, aes256-ctr, hmac-sha2-256, and hmac-sha2-512 algorithms. Ensure that the latest PuTTY (0.63 or later) and Xshell (5.0 or later) are used.

-

Yes

-
diff --git a/docs/en/docs/SecHarden/security-hardening-guide.md b/docs/en/docs/SecHarden/security-hardening-guide.md deleted file mode 100644 index 1863f6a1230fb77085954fada3892ee4c8282f82..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/security-hardening-guide.md +++ /dev/null @@ -1,3 +0,0 @@ -# Security Hardening Guide - -You can modify the hardening policy configuration file or script to harden the system. This chapter describes the hardening items, whether the items are hardened by default, and how to perform security hardening. diff --git a/docs/en/docs/SecHarden/security-hardening-tools.md b/docs/en/docs/SecHarden/security-hardening-tools.md deleted file mode 100644 index 015249e6dd50ce0647704a698a9a53b4ca3405a8..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/security-hardening-tools.md +++ /dev/null @@ -1,122 +0,0 @@ -# Security Hardening Tools - -## Security Hardening Procedure - -### Overview - -You need to modify the **usr-security.conf** file so that the security hardening tool can set hardening policies based on the **usr-security.conf** file. This section describes rules for modifying the **usr-security.conf** file. For details about the configurable security hardening items, see [Security Hardening Guide](security-hardening-guide.md). - -### Precautions - -- After modifying the items, restart the security hardening service for the modification to take effect. For details about how to restart the service, see [Hardening Items Taking Effect](#hardening-items-taking-effect). -- When modifying security hardening items, you only need to modify the **/etc/openEuler\_security/usr-security.conf** file. You are not advised to modify the **/etc/openEuler\_security/security.conf** file. The **security.conf** file contains basic hardening items which are executed only once. -- After the security hardening service is restarted for the configuration to take effect, the configuration in effect cannot be deleted by deleting the corresponding hardening items from the **usr-security.conf** file and restarting the security hardening service. -- Security hardening operations are recorded in the **/var/log/openEuler-security.log** file. - -### Configuration Format - -Each line in the **usr-security.conf** file indicates a configuration item. The configuration format varies according to the configuration content. The following describes the format of each configuration item. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- All configuration items start with an execution ID. The execution ID is a positive integer and can be customized. ->- Contents of a configuration item are separated by an at sign \(@\). ->- If the actual configuration content contains an at sign \(@\), use two at signs \(@@\) to distinguish the content from the separator. For example, if the actual content is **xxx@yyy**, set this item to **xxx@@yyy**. Currently, an at sign \(@\) cannot be placed at the beginning or end of the configuration content. - -- **d**: comment - - Format: _Execution ID_**@d@**_Object file_**@**_Match item_ - - Function: Comment out lines starting with the match item \(the line can start with a space\) in an object file by adding a number sign \(\#\) at the beginning of the line. - - Example: If the execution ID is **401**, comment out lines starting with **%wheel** in the **/etc/sudoers** file. - - ```sh - 401@d@/etc/sudoers@%wheel - ``` - -- **m**: replacement - - Format: _Execution ID_**@m@**_Object file_**@**_Match item_**@**_Target value_ - - Function: Replace lines starting with the match item \(the line can start with a space\) in an object file with _match item_ and _target value_. If the match line starts with spaces, the spaces will be deleted after the replacement. - - Example: If the execution ID is **101**, replace lines starting with **Protocol** in the **/etc/ssh/sshd\_config** file with **Protocol 2**. The spaces after **Protocol** are matched and replaced. - - ```sh - 101@m@/etc/ssh/sshd_config@Protocol @2 - ``` - -- **sm**: accurate modification - - Format: _Execution ID_**@sm@**_Object file_**@**_Match item_**@**_Target value_ - - Function: Replace lines starting with the match item \(the line can start with a space\) in an object file with _match item_ and _target value_. If the match line starts with spaces, the spaces are retained after the replacement. This is the difference between **sm** and **m**. - - Example: If the execution ID is **201**, replace lines starting with **size** in the **/etc/audit/hzqtest** file with **size 2048**. - - ```sh - 201@sm@/etc/audit/hzqtest@size@ 2048 - ``` - -- **M**: subitem modification - - Format: _Execution ID_**@M@**_Object file_**@**_Match item_**@**_Match subitem__\[@Value of the match subitem\]_ - - Function: Match lines starting with the match item \(the line can start with a space\) in an object file and replace the content starting with the match subitem in these lines with the _match subitem_ and _value of the match subitem_. The value of the match subitem is optional. - - Example: If the execution ID is **101**, find lines starting with **key** in the file and replace the content starting with **key2** in these lines with **key2value2**. - - ```sh - 101@M@file@key@key2@value2 - ``` - -- **systemctl**: service management - - Format: _Execution ID_**@systemctl@**_Object service_**@**_Operation_ - - Function: Use **systemctl** to manage object services. The value of **Operation** can be **start**, **stop**, **restart**, or **disable**. - - Example: If the execution ID is **218**, stop the **cups.service**. This provides the same function as running the **systemctl stop cups.service** command. - - ```sh - 218@systemctl@cups.service@stop - ``` - -- Other commands - - Format: _Execution ID_**@**_Command_**@**_Object file_ - - Function: Run the corresponding command, that is, run the command line _Command_ _Object file_. - - Example 1: If the execution ID is **402**, run the **rm -f** command to delete the **/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem** file. - - ```sh - 402@rm -f @/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem - ``` - - Example 2: If the execution ID is **215**, run the **touch** command to create the **/etc/cron.allow** file. - - ```sh - 215@touch @/etc/cron.allow - ``` - - Example 3: If the execution ID is **214**, run the **chown** command to change the owner of the **/etc/at.allow** file to **root:root**. - - ```sh - 214@chown root:root @/etc/at.allow - ``` - - Example 4: If the execution ID is **214**, run the **chmod** command to remove the **rwx** permission of the group to which the owner of the**/etc/at.allow** file belongs and other non-owner users. - - ```sh - 214@chmod og-rwx @/etc/at.allow - ``` - -## Hardening Items Taking Effect - -After modifying the **usr-security.conf** file, run the following command for the new configuration items to take effect: - -```sh -systemctl restart openEuler-security.service -``` diff --git a/docs/en/docs/SecHarden/selinux-configuration.md b/docs/en/docs/SecHarden/selinux-configuration.md deleted file mode 100644 index 34f71d6969f90f4c4fb8a23cc9983c19dceaeadc..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/selinux-configuration.md +++ /dev/null @@ -1,273 +0,0 @@ -# SELinux Configuration - -## Overview - -Discretionary access control \(DAC\) determines whether a resource can be accessed based on users, groups, and other permissions. It does not allow the system administrator to create comprehensive and fine-grained security policies. SELinux \(Security-Enhanced Linux\) is a module of the Linux kernel and a security subsystem of Linux. SELinux implements mandatory access control \(MAC\). Each process and system resource has a special security label. In addition to the principles specified by the DAC, the SELinux needs to determine whether each type of process has the permission to access a type of resource. - -By default, openEuler uses SELinux to improve system security. SELinux has three modes: - -- **permissive**: The SELinux outputs alarms but does not forcibly execute the security policies. -- **enforcing**: The SELinux security policies are forcibly executed. -- **disabled**: The SELinux security policies are not loaded. - -## Configuration Description - -- Obtain the SELinux running status: - - ```shell - $ getenforce - Enforcing - ``` - -- Set the enforcing mode when SELinux is enabled: - - ```shell - $ setenforce 1 - $ getenforce - Enforcing - ``` - -- Set the permissive mode when SELinux is enabled: - - ```shell - $ setenforce 0 - $ getenforce - Permissive - ``` - -- Disable SELinux when SELinux is enabled. (A reboot is required.) - 1. Modify the SELinux configuration file **/etc/selinux/config** and set **SELINUX=disabled**. - - ```shell - $ cat /etc/selinux/config | grep "SELINUX=" - SELINUX=disabled - ``` - - 2. Reboot the system. - - ```shell - reboot - ``` - - 3. The status is changed successfully. - - ```shell - $ getenforce - Disabled - ``` - -- Set the permissive mode when SELinux is disabled: - 1. Modify the SELinux configuration file **/etc/selinux/config** and set **SELINUX=permissive**. - - ```shell - $ cat /etc/selinux/config | grep "SELINUX=" - SELINUX=permissive - ``` - - 2. Create the **.autorelabel** file in the root directory. - - ```shell - touch /.autorelabel - ``` - - 3. Reboot the system. The system will restart twice. - - ```shell - reboot - ``` - - 4. The status is changed successfully. - - ```shell - $ getenforce - Permissive - ``` - -- Set the enforcing mode when SELinux is disabled: - 1. Set SELinux to the permissive mode. - 2. Modify the SELinux configuration file **/etc/selinux/config** and set **SELINUX=enforcing**. - - ```shell - $ cat /etc/selinux/config | grep "SELINUX=" - SELINUX=enforcing - ``` - - 3. Reboot the system. - - ```shell - reboot - ``` - - 4. The status is changed successfully. - - ```shell - $ getenforce - Enforcing - ``` - -## SELinux Commands - -- Query the SELinux status. **SELinux status** indicates the SELinux status. **enabled** indicates that SELinux is enabled, and **disabled** indicates that SELinux is disabled. **Current mode** indicates the current mode of the SELinux. - - ```shell - $ sestatus - SELinux status: enabled - SELinuxfs mount: /sys/fs/selinux - SELinux root directory: /etc/selinux - Loaded policy name: targeted - Current mode: enforcing - Mode from config file: enforcing - Policy MLS status: enabled - Policy deny_unknown status: allowed - Memory protection checking: actual (secure) - Max kernel policy version: 31 - ``` - -## Adding Policies - -- Obtain and add missing policies based on the audit logs. (The audit service must be enabled and SELinux access denial logs must exist in audit logs.) - 1. Check whether the audit logs contain SELinux access denial logs. Use the actual audit log path. - - ```shell - grep avc /var/log/audit/audit.log* - ``` - - 2. Query missing rules. - - ```shell - audit2allow -a /var/log/audit/audit.log* - ``` - - 3. Generate a policy module based on the missing rule and name it **demo**. - - ```shell - $ audit2allow -a /var/log/audit/audit.log* -M demo - ******************** IMPORTANT *********************** - To make this policy package active, execute: - semodule -i demo.pp - ``` - - 4. Load the **demo** policy module. - - ```shell - semodule -i demo.pp - ``` - -- Compose and add the SELinux policy module. - 1. Compose the FC file (if the security context of file creation is involved). - - ```shell - $ cat demo.fc - /usr/bin/example -- system_u:object_r:example_exec_t:s0 - /resource -- system_u:object_r:resource_file_t:s0 - ``` - - 2. Compose the TE file (example). - - ```shell - $ cat demo.te - module demo 1.0; - require - { - role unconfined_r; - role system_r; - type user_devpts_t; - type root_t; - attribute file_type; - attribute domain; - class dir { getattr search add_name create open remove_name rmdir setattr write }; - class file { entrypoint execute getattr open read map setattr write create }; - class process { sigchld rlimitinh siginh transition setcap getcap }; - class unix_stream_socket { accept bind connect listen recvfrom sendto listen create lock read write getattr setattr getopt setopt append shutdown ioctl connectto }; - class capability { chown dac_override dac_read_search }; - class chr_file { append getattr ioctl read write }; - }; - role unconfined_r types example_t; - role system_r types example_t; - type example_exec_t, file_type; - type resource_file_t, file_type; - type example_t, domain; - allow example_t user_devpts_t : chr_file { append getattr ioctl read write }; - allow example_t file_type : dir { getattr search }; - allow example_t example_exec_t : file { entrypoint execute getattr map open read }; - allow domain example_exec_t : file { execute getattr map open read }; - allow example_t example_exec_t : process { sigchld }; - allow domain example_t : process { rlimitinh siginh transition }; - allow example_t resource_file_t : file { create getattr open read setattr write }; - allow example_t root_t : dir { add_name create getattr open remove_name rmdir search setattr write }; - allow example_t example_t : unix_stream_socket { accept append bind connect create getattr getopt ioctl listen listen lock read recvfrom sendto setattr setopt shutdown write }; - allow example_t domain : unix_stream_socket { connectto }; - allow example_t example_t : capability { chown dac_override dac_read_search }; - allow example_t example_t : process { getcap setcap }; - type_transition domain example_exec_t : process example_t; - type_transition example_t root_t : file resource_file_t "resource"; - ``` - - 3. Compile **demo.te** as **demo.mod**. - - ```shell - checkmodule -Mmo demo.mod demo.te - ``` - - 4. Package **demo.mod** and **demo.fc** as a policy module file. - - ```shell - semodule_package -m demo.mod -f demo.fc -o demo.pp - ``` - - 5. Load the policy module. - - ```shell - semodule -i demo.pp - ``` - - 6. Delete the loaded policy module file. - - ```shell - $ semodule -r demo - libsemanage.semanage_direct_remove_key: Removing last demo module (no other demo module exists at another priority). - ``` - -## Function Verification - -- SELinux adopts an whitelist mechanism. Modules that are not configured with proper policies may fail to run properly due to lack of permissions. It is necessary to verify the functions of the modules and configure reasonable rules. - 1. Check whether the audit service is enabled: - - ```shell - systemctl status auditd - ``` - - 2. Set the SELinux mode to permissive. (Alarms are printed, but SELinux polices are not enforced. For details, see [Configuration Description](#configuration-description).) - - ```shell - $ getenforce - Permissive - ``` - - 3. Execute all function cases of the test module and check the SELinux access denial logs in the audit logs. - - ```shell - grep avc /var/log/audit/audit.log* - ``` - - 4. Analyze access denial logs and filter out missing rules. - - ```text - type=AVC msg=audit(1596161643.271:1304): avc: denied { read } for pid=1782603 comm="smbd" name=".viminfo" dev="dm-0" ino=2488208 scontext=system_u:system_r:smbd_t:s0 tcontext=staff_u:object_r:user_home_t:s0 tclass=file permissive=1 - Indicates that the smbd process (security context: system_u:system_r:smbd_t:s0) is denied the permission to read the .viminfo file (security context: staff_u:object_r:user_home_t:s0). - permissive=1 indicates that the permissive mode is running. This log records only the operations that are not forbidden. - ``` - - 5. Supplement the missing rules by referring to [Adding Policies](#adding-policies). - -## Precautions - -- Before enabling SELinux, you are advised to upgrade selinux-policy to the latest version using DNF. Otherwise, applications may fail to run properly. For example: - - ```shell - dnf update selinux-policy -y - ``` - -- If the system cannot be started due to improper SELinux configuration (for example, a policy is deleted by mistake or no proper rule or security context is configured), you can add **selinux=0** to the startup parameters to disable SELinux. - -- After SELinux is enabled, permission check is performed on access behaviors, which affects the operating system performance to some extent (related to the frequency of access operations in the running environment). diff --git a/docs/en/docs/SecHarden/system-services.md b/docs/en/docs/SecHarden/system-services.md deleted file mode 100644 index 42d306874f89c881439ec7b762c39856c9a0270e..0000000000000000000000000000000000000000 --- a/docs/en/docs/SecHarden/system-services.md +++ /dev/null @@ -1,455 +0,0 @@ -# System Services - -## Hardening the SSH Service - -### Description - -The Secure Shell \(SSH\) is a reliable security protocol for remote logins and other network services. SSH prevents information disclosure during remote management. SSH encrypts transferred data to prevent domain name server \(DNS\) spoofing and IP spoofing. OpenSSH was created as an open source alternative to the proprietary SSH protocol. - -Hardening the SSH service is to modify configurations of the SSH service to set the algorithm and authentication parameters when the system uses the OpenSSH protocol, improving the system security. [Table 1](#en-us_topic_0152100390_ta2fdb8e4931b4c1a8f502b3c7d887b95) describes the hardening items, recommended hardening values, and default policies. - -### Implementation - -To harden a server, perform the following steps: - -1. Open the configuration file **/etc/ssh/sshd\_config** of the SSH service on the server, and modify or add hardening items and values in the file. -2. Save the **/etc/ssh/sshd\_config** file. -3. Run the following command to restart the SSH service: - - ```shell - systemctl restart sshd - ``` - -To harden a client, perform the following steps: - -1. Open the configuration file **/etc/ssh/ssh\_config** of the SSH service on the client, and modify or add hardening items and values in the file. -2. Save the **/etc/ssh/ssh\_config** file. -3. Run the following command to restart the SSH service: - - ```shell - systemctl restart sshd - ``` - -### Hardening Items - -- Server hardening policies - - All SSH service hardening items are stored in the **/etc/ssh/sshd\_config** configuration file. For details about the server hardening items, hardening suggestions, and whether the hardening items are configured as suggested, see [Table 1](#en-us_topic_0152100390_ta2fdb8e4931b4c1a8f502b3c7d887b95). - - **Table 1** SSH hardening items on a server - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Item

-

Description

-

Suggestion

-

Configured as Suggested

-

Protocol

-

SSH protocol version.

-

2

-

Yes

-

SyslogFacility

-

Log type of the SSH service. The item is set to AUTH, indicating authentication logs.

-

AUTH

-

Yes

-

LogLevel

-

Level for recording SSHD logs.

-

VERBOSE

-

Yes

-

X11Forwarding

-

Specifies whether a GUI can be used after login using SSH.

-

no

-

Yes

-

MaxAuthTries

-

Maximum number of authentication attempts.

-

3

-

No

-

PubkeyAuthentication

-

Specifies whether public key authentication is allowed.

-

yes

-

Yes

-

RSAAuthentication

-

Specifies whether only RSA security authentication is allowed.

-

yes

-

Yes

-

IgnoreRhosts

-

Specifies whether the rhosts and shosts files are used for authentication. The rhosts and shosts files record the names of the servers that support remote access and related login names.

-

yes

-

Yes

-

RhostsRSAAuthentication

-

Specifies whether the RSA algorithm security authentication based on the rhosts file is used. The rhosts file records the names of the servers that support remote access and related login names.

-

no

-

Yes

-

HostbasedAuthentication

-

Specifies whether host-based authentication is used. Host-based authentication indicates that any user of a trusted client can use the SSH service.

-

no

-

Yes

-

PermitRootLogin

-

Specifies whether to allow user root to log in to the system using SSH.

-
NOTE:

If you want to log in to the system using SSH as user root, set the value of the PermitRootLogin field in the /etc/ssh/sshd_config file to yes.

-
-

no

-

No

-

PermitEmptyPasswords

-

Specifies whether accounts with empty passwords can log in.

-

no

-

Yes

-

PermitUserEnvironment

-

Specifies whether to resolve the environment variables set in ~/.ssh/environment and ~/.ssh/authorized_keys.

-

no

-

Yes

-

Ciphers

-

Encryption algorithm of SSH data transmission.

-

aes128-ctr,aes192-ctr,aes256-ctr,chacha20-poly1305@openssh.com,aes128-gcm@openssh.com,aes256-gcm@openssh.com

-

Yes

-

ClientAliveCountMax

-

Timeout count. After the server sends a request, if the number of times that the client does not respond reaches a specified value, the server automatically disconnects from the client.

-

0

-

No

-

Banner

-

File of the prompt information displayed before and after SSH login.

-

/etc/issue.net

-

Yes

-

MACs

-

Hash algorithm for SSH data verification.

-

hmac-sha2-512,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-256-etm@openssh.com

-

Yes

-

StrictModes

-

Specifies whether to check the permission on and ownership of the home directory and rhosts file before SSH receives login requests.

-

yes

-

Yes

-

UsePAM

-

Specifies whether to use PAM for login authentication.

-

yes

-

Yes

-

AllowTcpForwarding

-

Specifies whether to allow TCP forwarding.

-

no

-

Yes

-

Subsystem sftp /usr/libexec/openssh/sftp-server

-

SFTP log record level, which records the INFO level and authentication logs.

-

-l INFO -f AUTH

-

Yes

-

AllowAgentForwarding

-

Specifies whether to allow SSH Agent forwarding.

-

no

-

Yes

-

GatewayPorts

-

Specifies whether SSH can connect to ports on the forwarding client.

-

no

-

Yes

-

PermitTunnel

-

Specifies whether Tunnel devices are allowed.

-

no

-

Yes

-

KexAlgorithms

-

SSH key exchange algorithms.

-

curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

-

Yes

-

LoginGraceTime

-

Time limit for users passing the authentication. 0 indicates no limit. The default value is 60 seconds.

-

60

-

No

-
- - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >By default, the messages displayed before and after SSH login are saved in the **/etc/issue.net** file. The default information in the **/etc/issue.net** file is **Authorized users only.** **All activities may be monitored and reported.** - -- Client hardening policies - - All SSH service hardening items are stored in the **/etc/ssh/ssh\_config** configuration file. For details about the client hardening items, hardening suggestions, and whether the hardening items are configured as suggested, see [Table 2](#en-us_topic_0152100390_tb289c5a6f1c7420ab4339187f9018ea4). - - **Table 2** SSH hardening items on a client - - - - - - - - - - - - - - - - - - - -

Item

-

Description

-

Suggestion

-

Configured as Suggested

-

KexAlgorithms

-

SSH key exchange algorithms.

-

ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256

-

No

-

VerifyHostKeyDNS

-

Specifies whether to verify HostKey files by using DNS or SSHFP.

-

ask

-

No

-
- - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Third-party clients and servers that use the Diffie-Hellman algorithm are required to allow at least 2048-bit connection. - -### Other Security Suggestions - -- The SSH service only listens on specified IP addresses. - - For security purposes, you are advised to only listen on required IP addresses rather than listen on 0.0.0.0 when using the SSH service. You can specify the IP addresses that SSH needs to listen on in the ListenAddress configuration item in the **/etc/ssh/sshd\_config** file. - - 1. Open and modify the **/etc/ssh/sshd\_config** file. - - ```shell - vi /etc/ssh/sshd_config - ``` - - The following information indicates that the bound listening IP address is **192.168.1.100**. You can change the listening IP address based on the site requirements. - - ```text - ... - ListenAddress 192.168.1.100 - ... - ``` - - 2. Restart the SSH service. - - ```shell - systemctl restart sshd.service - ``` - -- SFTP users are restricted from access to upper-level directories. - - SFTP is a secure FTP designed to provide secure file transfer over SSH. Users can only use dedicated accounts to access SFTP for file upload and download, instead of SSH login. In addition, directories that can be accessed over SFTP are limited to prevent directory traversal attacks. The configuration process is as follows: - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >In the following configurations, **sftpgroup** is an example user group name, and **sftpuser** is an example username. - - 1. Create an SFTP user group. - - ```shell - groupadd sftpgroup - ``` - - 2. Create an SFTP root directory. - - ```shell - mkdir /sftp - ``` - - 3. Modify the ownership of and permission on the SFTP root directory. - - ```shell - chown root:root /sftp - chmod 755 /sftp - ``` - - 4. Create an SFTP user. - - ```shell - useradd -g sftpgroup -s /sbin/nologin sftpuser - ``` - - 5. Set the password of the SFTP user. - - ```shell - passwd sftpuser - ``` - - 6. Create a directory used to store files uploaded by the SFTP user. - - ```shell - mkdir /sftp/sftpuser - ``` - - 7. Modify the ownership of and permission on the upload directory of the SFTP user. - - ```shell - chown root:root /sftp/sftpuser - chmod 777 /sftp/sftpuser - ``` - - 8. Modify the **/etc/ssh/sshd\_config** file. - - ```shell - vi /etc/ssh/sshd_config - ``` - - Modify the following information: - - ```text - #Subsystem sftp /usr/libexec/openssh/sftp-server -l INFO -f AUTH - Subsystem sftp internal-sftp -l INFO -f AUTH - ... - - Match Group sftpgroup - ChrootDirectory /sftp/%u - ForceCommand internal-sftp - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >- **%u** is a wildcard character. Enter **%u** to represent the username of the current SFTP user. - >- The following content must be added to the end of the **/etc/ssh/sshd\_config** file: - > - > ```text - > Match Group sftpgroup - > ChrootDirectory /sftp/%u - > ForceCommand internal-sftp - > ``` - - 9. Restart the SSH service. - - ```shell - systemctl restart sshd.service - ``` - -- Remotely execute commands using SSH. - - When a command is executed remotely through OpenSSH, TTY is disabled by default. If a password is required during command execution, the password is displayed in plain text. To ensure password input security, you are advised to add the **-t** option to the command. Example: - - ```shell - ssh -t testuser@192.168.1.100 su - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >**192.168.1.100** is an example IP address, and **testuser** is an example username. diff --git a/docs/en/docs/ShangMi/algorithm-library.md b/docs/en/docs/ShangMi/algorithm-library.md deleted file mode 100644 index 63eaf7bd69356da05dd740b0b27ee0fe6f7202f6..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/algorithm-library.md +++ /dev/null @@ -1,194 +0,0 @@ -# Algorithm Library - -## OpenSSL Cryptographic Interface - -OpenSSL is a common cryptographic algorithm library software that supports SM2, SM3, and SM4 algorithms. You can invoke the encryption and decryption functions of SM series cryptographic algorithms through command lines or APIs. - -### Prerequisites - -OpenSSL 1.1.1m-6 or later - -``` -$ rpm -qa openssl -openssl-1.1.1m-6.oe2209.x86_64 -``` - -### How to Use - -#### Scenario 1: Using the CLI to Call Cryptographic Algorithms - -1. SM2 public key algorithm - -Generate an SM2 private key. - -``` -$ openssl ecparam -genkey -name SM2 -out priv.key -``` - -Generate a public key based on the private key. - -``` -$ openssl ec -in priv.key -pubout -out pub.key -read EC key -writing EC key -``` - -Use the SM2 algorithm to sign the file and set the message digest algorithm to SM3. - -``` -$ openssl dgst -sm3 -sign priv.key -out data.sig data -``` - -Use the public key to verify the signature. - -``` -$ openssl dgst -sm3 -verify pub.key -signature data.sig data -Verified OK -``` - -2. SM3 message digest algorithm - -Use the SM3 algorithm for data digest. - -``` -$ openssl dgst -sm3 data -SM3(data)= a794922bb9f0a034257f6c7090a3e8429801a42d422c21f1473e83b7f7eac385 -``` - -3. SM4 symmetric cipher algorithm - -Use the SM4 algorithm to encrypt data. **-K** and **-iv** specify the key value and IV value used for encryption, respectively. Generally, the key value and IV value need to be randomly generated. - -``` -$ openssl enc -sm4 -in data -K 123456789ABCDEF0123456789ABCDEF0 -iv 123456789ABCDEF0123456789ABCDEF0 -out data.enc -``` - -Use the SM4 algorithm to decrypt data. - -``` -$ openssl enc -d -sm4 -in data.enc -K 123456789ABCDEF0123456789ABCDEF0 -iv 123456789ABCDEF0123456789ABCDEF0 -out data.raw -``` - -Compare the encrypted and decrypted data. The results are consistent. - -``` -$ diff data data.raw -``` - -#### Scenario 2: Using APIs to Call Cryptographic Algorithms - -You can directly install openssl-help and query the **man** manual. - -``` -$ yum install openssl-help -$ man sm2 -$ man EVP_sm3 -$ man EVP_sm4_cbc -``` - -## Kernel Cryptographic Interface - -### Overview - -The cryptographic algorithms of the Linux kernel is managed by the crypto framework. Different algorithm implementations can be registered and invoked in the crypto framework. Kernel 5.10 provided by openEuler supports ShangMi (SM) series cryptographic algorithms (SM2, SM3, and SM4). The SM2 and SM3 algorithms are compiled in the kernel by default, and the SM4 algorithm is provided as a kernel module. - -### Prerequisites - -Kernel 5.10.0-106 or later - -``` -# rpm -qa kernel -kernel-5.10.0-106.1.0.55.oe2209.x86_64 -``` - -### How to Use - -#### Scenario 1: Querying the Cryptographic Algorithms Supported by the Kernel - -Use **/proc/crypto** to query the registered SM series cryptographic algorithms. By default, the SM2 and SM3 algorithms are loaded. - -``` -$ cat /proc/crypto | grep sm3 -A8 -name : sm3 -driver : sm3-generic -module : kernel -priority : 100 -refcnt : 1 -selftest : passed -internal : no -type : shash -blocksize : 64 -digestsize : 32 - -$ cat /proc/crypto | grep sm2 -A6 -name : sm2 -driver : sm2-generic -module : kernel -priority : 100 -refcnt : 1 -selftest : passed -internal : no -type : akcipher -``` - -By default, the SM4 algorithm is not loaded. You need to insert the corresponding module first. - -``` -$ modprobe sm4-generic -$ cat /proc/crypto | grep sm4 -A8 -name : sm4 -driver : sm4-generic -module : sm4_generic -priority : 100 -refcnt : 1 -selftest : passed -internal : no -type : cipher -blocksize : 16 -min keysize : 16 -max keysize : 16 -``` - -#### Scenario 2: Calling Algorithm APIs - -The method of calling SM series cryptographic algorithms is the same as that of calling other algorithms of the same type. For details, see the Linux kernel document. - -``` -https://www.kernel.org/doc/html/v5.10/crypto/userspace-if.html -``` - -#### Scenario 3: Optimizing Algorithm Performance Through Instruction Sets - -The crypto framework allows registration of algorithm implementations related to the architecture. The algorithm performance can be optimized through a specific instruction set. Currently, the kernel 5.10 of openEuler supports algorithm performance optimization using the following instruction sets. - -| Driver | Instruction Set | Priority| -| -------------------------------- | ---------------------- | ------ | -| sm4-neon(ecb/cbc/cfb/ctr) | ARM64-NEON | 200 | -| sm3-avx | x86-AVX | 300 | -| sm4-aesni-avx (ecb/cbc/cfb/ctr) | x86-AVX | 400 | -| sm4-aesni-avx 2(ecb/cbc/cfb/ctr) | x86-AVX2 | 500 | - -When multiple instances of the same algorithm are registered, the default algorithm is selected based on the registered priority of each algorithm instance. A larger **priority** value indicates a higher priority. The priority of a pure software algorithm (with the suffix **-generic**) is fixed to **100**. By default, the performance optimization through instruction sets is disabled for the SM series cryptographic algorithms and is provided for users in the form of a kernel module. For example, to enable the AVX instruction set optimization of the SM3 algorithm, do as follows: - -``` -$ modprobe sm3-avx -$ cat /proc/crypto | grep sm3 -A8 -name : sm3 -driver : sm3-avx -module : sm3_avx_x86_64 -priority : 300 -refcnt : 1 -selftest : passed -internal : no -type : shash -blocksize : 64 -digestsize : 32 - -...... -``` - -#### Notes - -1. The prerequisite for enabling algorithm instruction set optimization is that the CPU supports the corresponding instruction set. You can query the instruction set supported by the CPU by calling **/proc/cpuinfo**. -2. Calling a specific instruction set has certain overhead. Therefore, it cannot be ensured that the performance optimized by the instruction set is higher than that of software implementation in all scenarios. -3. The optimization through some instruction sets has certain restrictions. For example, the NEON instruction set has optimization effect only in the encryption mode that supports parallel computing. diff --git a/docs/en/docs/ShangMi/disk-encryption.md b/docs/en/docs/ShangMi/disk-encryption.md deleted file mode 100644 index dc0fbc67d7be95049bbea115399e29d6b5563b08..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/disk-encryption.md +++ /dev/null @@ -1,89 +0,0 @@ -# Disk Encryption - -## Overview - -Disk encryption protects the storage confidentiality of important data. Data is encrypted based on a specified encryption algorithm and then written to disks. This feature mainly involves the user-mode tool cryptsetup and the kernel-mode module dm-crypt. Currently, the disk encryption feature provided by the openEuler OS supports ShangMi (SM) series cryptographic algorithms. Parameters are as follows: - -- Encryption modes: luks2 and plain; -- Key length: 256 bits; -- Message digest algorithm: SM3; -- Encryption algorithm: sm4-xts-plain64. - -## Prerequisites - -1. Kernel 5.10.0-106 or later - -``` -$ rpm -qa kernel -kernel-5.10.0-106.1.0.55.oe2209.x86_64 -``` - -2. cryptsetup 2.4.1-1 or later - -``` -$ rpm -qa cryptsetup -cryptsetup-2.4.1-1.oe2209.x86_64 -``` - -## How to Use - -A disk is formatted into a disk in a specified encryption mode and mapped to **/dev/mapper** as a dm device. Subsequent disk read and write operations are performed through the dm device. Data encryption and decryption are performed in kernel mode and are not perceived by users. The procedure is as follows: - -1. Format the disk and map the disk as a dm device. - -a. luks2 mode - -Set the encryption mode to luks2, encryption algorithm to sm4-xts-plain64, key length to 256 bits, and message digest algorithm to SM3. - -``` -# cryptsetup luksFormat /dev/sdd -c sm4-xts-plain64 --key-size 256 --hash sm3 -``` - -b. plain mode - -Set the encryption mode to plain, encryption algorithm to sm4-xts-plain64, key length to 256 bits, and message digest algorithm to SM3. - -``` -# cryptsetup plainOpen /dev/sdd crypt1 -c sm4-xts-plain64 --key-size 256 --hash sm3 -``` - -2. After the mapping is successful, run the **lsblk** command to view the device information. - -``` -# lsblk -NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS -...... -sdd 8:48 0 50G 0 disk -└─crypt1 253:3 0 50G 0 crypt -...... -``` - -3. Perform I/O read and write operations on the encrypted device. - -Deliver I/Os to raw disks. - -``` -# dd if=/dev/random of=/dev/mapper/crypt1 bs=4k count=10240 -``` - -Deliver I/Os through the file system. - -``` -# mkfs.ext4 /dev/mapper/crypt1 -# mount /dev/mapper/crypt1 /mnt/crypt/ -# dd if=/dev/random of=/mnt/crypt/tmp bs=4k count=10240 -``` - -4. Disable device mapping. - -If a file system is mounted, unmount it first. - -``` -# umount /mnt/crypt -``` - -Closes a device. - -``` -# cryptsetup luksClose crypt1 -``` diff --git a/docs/en/docs/ShangMi/file-integrity-protection.md b/docs/en/docs/ShangMi/file-integrity-protection.md deleted file mode 100644 index 0e04b149ceef71bba007c9e9ac85ced4223f41a6..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/file-integrity-protection.md +++ /dev/null @@ -1,512 +0,0 @@ -# File Integrity Protection - -## Kernel Integrity Measurement Architecture (IMA) - -IAM is a mandatory access control subsystem provided by the Linux kernel. It measures and verifies file integrity by adding check hooks to file access system calls. - -### Prerequisites - -1. The openEuler kernel compilation environment has been prepared. For details, see . -2. In openEuler kernel 5.10, the ShangMi (SM) series cryptographic algorithms are supported for kernel module signing. You are advised to select the latest kernel 5.10 source code for compilation. -3. The kernel SM2 root certificate has been generated. - - ```shell - # Generate a certificate configuration file. (Other fields in the configuration file can be defined as required.) - $ echo 'subjectKeyIdentifier=hash' > ca.cfg - # Generate a private key for SM2 signing. - $ openssl ecparam -genkey -name SM2 -out ca.key - # Generate a signing request. - $ openssl req -new -sm3 -key ca.key -out ca.csr - # Generate an SM2 certificate. - $ openssl x509 -req -days 3650 -extfile ca.cfg -signkey ca.key -in ca.csr -out ca.crt - ``` - -4. The level-2 IMA certificate has been generated. - - ```shell - # Create a certificate configuration file. - echo 'subjectKeyIdentifier=hash' > ima.cfg - echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg - # Generate a private key. - openssl ecparam -genkey -name SM2 -out ima.key - # Generate a signing request. - openssl req -new -sm3 -key ima.key -out ima.csr - # Generate a level-2 certificate based on the level-1 certificate. - openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -extfile ima.cfg -in ima.csr -out ima.crt - # Convert to the DER format. - openssl x509 -outform DER -in ima.crt -out x509_ima.der - ``` - -5. The root certificate has been placed in the kernel source code directory, and **CONFIG_SYSTEM_TRUSTED_KEYS** has been modified to compile the certificate to the kernel trusted key. - - ```shell - $ cp /path/to/ca.crt . - $ make openeuler_defconfig - $ cat .config | grep CONFIG_SYSTEM_TRUSTED_KEYS - CONFIG_SYSTEM_TRUSTED_KEYS="ca.crt" - ``` - -6. The kernel has been compiled and installed. - -```shell -make -j64 -make modules_install -make install -``` - -### How to Use - -#### Scenario 1: Native IMA - -##### IMA-Measurement - -Configure the IMA policy and message digest algorithm, disable the IMA-appraisal mode, and restart the system. - -```shell -ima_policy=tcb ima_hash=sm3 ima_appraise=off -``` - -Check the measurement log. It is found that the IMA measures all protected files and the message digest algorithm is SM3. - -```shell -cat /sys/kernel/security/ima/ascii_runtime_measurements -10 601989730f01fb4688bba92d0ec94340cd90757f ima-sig sm3:0000000000000000000000000000000000000000000000000000000000000000 boot_aggregate -10 dc0a98316b03ab15edd2b8daae75a0d64bca7c56 ima-sig sm3:3c62ee3c13ee32d7a287e04c843c03ebb428a5bb3dd83561efffe9b08444be22 /usr/lib/systemd/systemd -10 1d0a5140e3924e2542963ad887a80def0aa8acac ima-sig sm3:4d3b83e143bd9d5288ef099eff4d01444947516d680165c6dd08cd5900768032 /usr/lib64/ld-linux-x86-64.so.2 -...... -``` - -##### IMA-Appraisal (Hash) - -Configure the IMA policy and message digest algorithm, enable the IMA-appraisal fix mode, and restart the system. - -```shell -ima_policy=appraise_tcb ima_hash=sm3 ima_appraise=fix -``` - -**appraise_tcb** indicates to appraise all files owned by the **root** user. - -Perform an open operation on all files to be appraised to automatically mark the .ima extension. - -```shell -find / -fstype ext4 -type f -uid 0 -exec dd if='{}' of=/dev/null count=0 status=none \; -``` - -After the marking is complete, you can see that all files with the .ima extension of the SM3 message digest algorithm are marked. - -```shell -getfattr -m - -d -e hex /bin/bash -getfattr: Removing leading '/' from absolute path names -# file: bin/bash -security.ima=0x0411a794922bb9f0a034257f6c7090a3e8429801a42d422c21f1473e83b7f7eac385 -security.selinux=0x73797374656d5f753a6f626a6563745f723a7368656c6c5f657865635f743a733000 - -openssl dgst -sm3 /bin/bash -SM3(/bin/bash)= a794922bb9f0a034257f6c7090a3e8429801a42d422c21f1473e83b7f7eac385 -``` - -Enable the enforce mode and restart the system. The system can run properly. - -```shell -ima_policy=appraise_tcb ima_hash=sm3 ima_appraise=enforce -``` - -##### IMA-Appraisal (Signing) - -**Prerequisites** - -1. The SM root certificate has been preset in the kernel. -2. The ima-evm-utils software package whose version is later than or equal to the specified version has been installed. - -```shell -$ rpm -qa ima-evm-utils -ima-evm-utils-1.3.2-4.oe2209.x86_64 -``` - -Generate a level-2 IMA certificate. - -```shell -# Create a certificate configuration file. -echo 'subjectKeyIdentifier=hash' > ima.cfg -echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg -# Generate a private key. -openssl ecparam -genkey -name SM2 -out ima.key -# Generate a signing request. -openssl req -new -sm3 -key ima.key -out ima.csr -# Generate a level-2 certificate based on the level-1 certificate. -openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -extfile ima.cfg -in ima.csr -out ima.crt -# Convert to the DER format. -openssl x509 -outform DER -in ima.crt -out x509_ima.der -``` - -Place the IMA certificate in the **/etc/keys** directory and run the **dracut** command to create initrd again. - -```shell -mkdir -p /etc/keys -cp x509_ima.der /etc/keys -echo 'install_items+=" /etc/keys/x509_ima.der "' >> /etc/dracut.conf -dracut -f -``` - -Sign the files to be protected. For example, sign all executable files of the **root** user in the **/usr/bin** directory. - -```shell -find /usr/bin -fstype ext4 -type f -executable -uid 0 -exec evmctl -a sm3 ima_sign --key /path/to/ima.key '{}' \; -``` - -Enable the enforce mode and restart the system. The system can run properly. - -```shell -ima_policy=appraise_tcb ima_hash=sm3 ima_appraise=enforce -``` - -Check the protection effect of the signing mode. - -```shell -# getfattr -m - -d /bin/echo -getfattr: Removing leading '/' from absolute path names -# file: bin/echo -security.ima=0sAwIRNJFkBQBIMEYCIQDLBg/bYlrkBqSaXNQMyK7rhiZj+qRiKdu+0fqW8lSmPQIhAJY2qSZJ0HgSu7kygydrS4MCC0KTK59nUkvISenZAUCo -security.selinux="system_u:object_r:bin_t:s0" - -# echo 123 >> /bin/echo --bash: /bin/echo: Permission denied -``` - -**Note:** - -For each file under the IMA protection, you need to use either the hash mode or the signing mode to mark the integrity information. Generally, files that may change (such as data files and configuration files) are marked in hash mode, and files that do not change (such as executable files and dynamic link libraries) are marked in signing mode. - -#### Scenario 2: IMA Digest Lists Mode - -##### Generating SM3 Digest Lists - -Configure **-a sm3** when calling **gen_digest_lists** to generate SM3 digest lists. - -```shell -gen_digest_lists -a sm3 -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/bash -d -i i: -gen_digest_lists -a sm3 -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/bash -d -i i: -T -``` - -##### Configuring the SM3 Message Digest Algorithm - -The overall procedure is the same as that for enabling the IMA Digest Lists feature. The only difference is that the **ima_hash** startup parameter is set to **sm3**. The following gives an example: - -```shell -# Log mode -ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sm3 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest -# enforce mode -ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sm3 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest -``` - -For details about other steps, see **Administrator Guide** > **Trusted Computing** > **[Initial Deployment in the Digest Lists Scenario](../Administration/trusted-computing.md#initial-deployment-in-the-digest-lists-scenario)**. - -After the configuration is complete, restart the system and query the measurement log. The default algorithm in the measurement log is changed to SM3. - -```shell -$ cat /sys/kernel/security/ima/ascii_runtime_measurements -...... -11 9e32183b5b1da72c6ff4298a44026e3f9af510c9 ima-sig sm3:5a2d81cd135f41e73e0224b9a81c3d8608ccde8caeafd5113de959ceb7c84948 /usr/bin/upload_digest_lists -11 f3b9264761dbaeaf637d08b86cc3655e8f3380f7 ima-sig sm3:cc6faecee9976c12279dab1627a78ef36f6998c65779f3b846494ac5fe5493a1 /usr/libexec/rpm_parser -11 dc0a98316b03ab15edd2b8daae75a0d64bca7c56 ima-sig sm3:3c62ee3c13ee32d7a287e04c843c03ebb428a5bb3dd83561efffe9b08444be22 /usr/lib/systemd/systemd -...... -``` - -##### Verifying Digest Lists Using the SM2 Certificates - -**Prerequisites** - -1. The SM root certificate has been preset in the kernel. -2. The digest-list-tools and ima-evm-utils software packages of the specified versions or later have been installed. - -```shell -$ rpm -qa ima-evm-utils -ima-evm-utils-1.3.2-4.oe2209.x86_64 -$ rpm -qa digest-list-tools -digest-list-tools-0.3.95-10.oe2209.x86_64 -``` - -**Procedure** - -1. Generate level-2 IMA and EVM certificates (sub-certificates of the SM root certificate preset in the kernel). - - ```shell - # Create a certificate configuration file. - echo 'subjectKeyIdentifier=hash' > ima.cfg - echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg - # Generate a private key. - openssl ecparam -genkey -name SM2 -out ima.key - # Generate a signing request. - openssl req -new -sm3 -key ima.key -out ima.csr - # Generate a level-2 certificate based on the level-1 certificate. - openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -extfile ima.cfg -in ima.csr -out ima.crt - # Convert to the DER format. - openssl x509 -outform DER -in ima.crt -out x509_ima.der - openssl x509 -outform DER -in ima.crt -out x509_evm.der - ``` - -2. Place the IMA and EVM certificates in the **/etc/keys** directory and run the **dracut** command to create initrd again. - - ```shell - mkdir -p /etc/keys - cp x509_ima.der /etc/keys - cp x509_evm.der /etc/keys - echo 'install_items+=" /etc/keys/x509_ima.der /etc/keys/x509_evm.der "' >> /etc/dracut.conf - dracut -f -e xattr - ``` - -3. Enable the IMA Digest Lists function. After the restart, check whether the certificates are imported to the IMA and EVM key rings. - - ```shell - $ cat /proc/keys - ...... - 024dee5e I------ 1 perm 1f0f0000 0 0 keyring .evm: 1 - ...... - 3980807f I------ 1 perm 1f0f0000 0 0 keyring .ima: 1 - ...... - ``` - -4. Sign the IMA digest lists using the private keys corresponding to the IMA and EVM certificates. The signed IMA digest lists can be imported to the kernel. - - ```shell - # Use **evmctl** to sign the digest lists. - evmctl ima_sign --key /path/to/ima.key -a sm3 0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 - # Check the extension after signing. - getfattr -m - -d 0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 - # file: 0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 - security.ima=0sAwIRNJFkBQBHMEUCIQCzdKVWdxw1hoVm9lgZB6sl+sxapptUFNjqHt5XZD87hgIgBMuZqBdrcNm7fXq/reQw7rzY/RN/UXPrIOxrVvpTouw= - security.selinux="unconfined_u:object_r:admin_home_t:s0" - # Import the signed digest lists file to the kernel. - echo /root/tree/etc/ima/digest_lists/0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 > /sys/kernel/security/ima/digest_list_data - # Check the measurement log. You can view the import record of the digest lists. - cat /sys/kernel/security/ima/ascii_runtime_measurements - 11 43b6981f84ba2725d05e91f19577cedb004adffb ima-sig sm3:b9430bbde2b7f30e935d91e29ab6778b6a825a2c3e5e7255895effb8747b7c1a /root/tree/etc/ima/digest_lists/0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 0302113491640500473045022100b374a556771c35868566f6581907ab25facc5aa69b5414d8ea1ede57643f3b86022004cb99a8176b70d9bb7d7abfade430eebcd8fd137f5173eb20ec6b56fa53a2ec - ``` - -**Notes:** - -1. By default, the hash algorithm used in the digest lists provided by openEuler is SHA256. When the IMA digest lists measurement algorithm is set to SM3, you must remove the digest lists from the **/etc/ima/digest_lists** directory, generate new digest lists, and sign the digest lists. Otherwise, an error occurs during file integrity check. The procedure is as follows: - - ```shell - # Reset the SELinux tag of a disk (perform this operation when IMA extension verification and SELinux are enabled). - fixfiles -F restore - # Generate digest lists for all files (you can also specify the files). - gen_digest_lists -a sm3 -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/ -d /etc/ima/digest_lists -i i: - # (Optional) Change the name of the generated digest lists file. - mv /etc/ima/digest_lists/0-metadata_list-compact- /etc/ima/digest_lists/0-metadata_list-compact-everything-sm3 - # Sign - evmctl ima_sign --key /path/to/ima.key -a sm3 /etc/ima/digest_lists/0-metadata_list-compact-everything-sm3 - # Re-create initrd. - dracut -f -e xattr - ``` - -2. For the software packages released by openEuler, the IMA digest lists file is provided by default. The default message digest algorithm is SHA256. Therefore, when the message digest algorithm is changed to SM3, the digest lists released by openEuler cannot be imported, and the following message may be displayed during software package installation: - - ```text - Cannon parse /etc/ima/digest_lists/0-metadata_list-rpm-...... - ``` - -# Lightweight Intrusion Detection (AIDE) - -AIDE is a lightweight intrusion detection tool. It checks file integrity to detect malicious intrusions to the system in a timely manner. The AIDE database can use hash algorithms such as SHA256 and SHA512 to create a verification code or hash value for each file in ciphertext. The AIDE provided by openEuler supports the SM3 algorithm in addition to the open source software. - -## Prerequisites - -AIDE 0.17.4-1 or later - -```shell -$ rpm -qa aide -aide-0.17.4-1.oe2209.x86_64 -``` - -## How to Use - -Add the SM3 algorithm to the **/etc/aide.conf** configuration file. - -```text -...... -FIPSR = p+i+n+u+g+s+m+c+acl+selinux+xattrs+sha256+sm3 -...... -DATAONLY = p+n+u+g+s+acl+selinux+xattrs+sha256+sm3 -...... -``` - -1. Initialize the database and save the database as the benchmark. - -Initialize the database. - -```shell -aide -c /etc/aide.conf -i -``` - -The example output is as follows: - -```text -AIDE initialized database at /var/lib/aide/aide.db.new.gz - -Number of entries: 64249 - ---------------------------------------------------- -The attributes of the (uncompressed) database(s): ---------------------------------------------------- - -/var/lib/aide/aide.db.new.gz - MD5 : a7y5ErdpBAezV2iGdaVleg== - SHA1 : u7W7jxomFtZn8rwMlkIRCN0r7iQ= - SHA256 : 88Kw5b2yJ9bejwO+NqT6lyAieno+K0+W - BPVBjXcUl08= - SHA512 : WyOIgRxk9SeSoktF6BYVV0tRL7nGNDKQ - A9QyxVCgzg+PwPMV7tzxmwOZI/dB64pP - vQ/D2jqJdf3NS2PHMI4yvg== - RMD160 : qTEPs2SIxPm3iEwsCnwvp9hR4s4= - TIGER : 0HgLucmhCcB56bxOMj+j1Kuja8UIsFRg - CRC32 : VKE1TA== - WHIRLPOOL : JSA35/NmkMOkUWEpcZJf3PR1UUz5WcLG - AmBKPkao3fzQUsLMTJizCV4CwAE0G/Yc - KX0mpW5vx+gk3njya8rAvA== - GOST : yKjiytOwRr3bJcFsxnJ310t1FY6YE3HB - YNT8XP93xpc= - STRIBOG256: 9bzS+5j4ZAoU/P7v3tkKOWn4ZfggcX28 - 9dLQVhaiJtQ= - STRIBOG512: 9LLXgqsRIRiXP2WOrOJt1qhx6psfbACd - un+GTVmu441quX4zaaPIIG9lzDMBAqMg - hZx5DlxsQj3YjMezSUsXLg== - SM3 : Vwii+uw3Ge5Hh3eo1KOombxn2jWgyYRX - ZdyCRZqWZ/E= - - -End timestamp: 2022-08-12 09:01:25 +0800 (run time: 2m 43s) -``` - -Save the database as the benchmark. - -```shell -mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz -``` - -### Scenario 1: Detecting Changes of Protected Files - -```shell -$ aide -c /etc/aide.conf --check ---------------------------------------------------- -Detailed information about changes: ---------------------------------------------------- - -File: /boot/config-5.10.0-106.3.0.57.oe2209.aarch64 - Size : 182936 | 182938 - Mtime : 2022-08-04 08:00:00 +0800 | 2022-08-12 09:05:34 +0800 - Ctime : 2022-08-11 01:42:44 +0800 | 2022-08-12 09:05:34 +0800 - SHA256 : ae0fOzf7U+e/evTZKpk6JQa00kvSkc5J | gOlhcUgnZWhcyJYMEPxCYccXwFr9lERX - vMTX5Ysh+1k= | KK3O/ytfR/g= - SHA512 : zAPIxIAM7YvJPjwl/PH23leBb/HiO0Rf | p+WxVOZ6DX323rHDRF864w297yh7POk6 - PlRME7yvpzFZk/5BrNe2ofQWR/0sFu1m | 11dOzahlKTWpAKaexC/u+4REiCzjl1rm - JsDSy8m57wzCpJA9iUFq1g== | eb/kd3Xgp1LoKwn49mtqxw== - SM3 : CW0GnITxNeGeYOCAm4xfu78Vqm+wLp/Z | GWq/3nXL16tMYyxyFD/HTZbvJi2h+ttg - cOmXmIKJT4Q= | 6d8XmSHu26A= -``` - -### Scenario 2: Updating the Database - -Update the database. After the update, the database file is **/var/lib/aide/aide.db.new.gz**. - -```shell -$ aide -c /etc/aide.conf --update ---------------------------------------------------- -Detailed information about changes: ---------------------------------------------------- - -File: /boot/config-5.10.0-106.3.0.57.oe2209.aarch64 - Size : 182936 | 182938 - Mtime : 2022-08-04 08:00:00 +0800 | 2022-08-12 09:05:34 +0800 - Ctime : 2022-08-11 01:42:44 +0800 | 2022-08-12 09:05:34 +0800 - SHA256 : ae0fOzf7U+e/evTZKpk6JQa00kvSkc5J | gOlhcUgnZWhcyJYMEPxCYccXwFr9lERX - vMTX5Ysh+1k= | KK3O/ytfR/g= - SHA512 : zAPIxIAM7YvJPjwl/PH23leBb/HiO0Rf | p+WxVOZ6DX323rHDRF864w297yh7POk6 - PlRME7yvpzFZk/5BrNe2ofQWR/0sFu1m | 11dOzahlKTWpAKaexC/u+4REiCzjl1rm - JsDSy8m57wzCpJA9iUFq1g== | eb/kd3Xgp1LoKwn49mtqxw== - SM3 : CW0GnITxNeGeYOCAm4xfu78Vqm+wLp/Z | GWq/3nXL16tMYyxyFD/HTZbvJi2h+ttg - cOmXmIKJT4Q= | 6d8XmSHu26A= -``` - -### Scenario 3: Comparing Databases - -Configure two databases in **/etc/aide.conf**. - -```conf -# The location of the database to be read. -database_in=file:@@{DBDIR}/aide.db.gz -database_new=file:@@{DBDIR}/aide.db.new.gz -``` - -Compare the databases. - -```shell -$ aide -c /etc/aide.conf --compare ---------------------------------------------------- -Detailed information about changes: ---------------------------------------------------- - -File: /boot/config-5.10.0-106.3.0.57.oe2209.aarch64 - Size : 182936 | 182938 - Mtime : 2022-08-04 08:00:00 +0800 | 2022-08-12 09:05:34 +0800 - Ctime : 2022-08-11 01:42:44 +0800 | 2022-08-12 09:05:34 +0800 - SHA256 : ae0fOzf7U+e/evTZKpk6JQa00kvSkc5J | gOlhcUgnZWhcyJYMEPxCYccXwFr9lERX - vMTX5Ysh+1k= | KK3O/ytfR/g= - SHA512 : zAPIxIAM7YvJPjwl/PH23leBb/HiO0Rf | p+WxVOZ6DX323rHDRF864w297yh7POk6 - PlRME7yvpzFZk/5BrNe2ofQWR/0sFu1m | 11dOzahlKTWpAKaexC/u+4REiCzjl1rm - JsDSy8m57wzCpJA9iUFq1g== | eb/kd3Xgp1LoKwn49mtqxw== - SM3 : CW0GnITxNeGeYOCAm4xfu78Vqm+wLp/Z | GWq/3nXL16tMYyxyFD/HTZbvJi2h+ttg - cOmXmIKJT4Q= | 6d8XmSHu26A= - ---------------------------------------------------- -The attributes of the (uncompressed) database(s): ---------------------------------------------------- - -/var/lib/aide/aide.db.gz - MD5 : a7y5ErdpBAezV2iGdaVleg== - SHA1 : u7W7jxomFtZn8rwMlkIRCN0r7iQ= - SHA256 : 88Kw5b2yJ9bejwO+NqT6lyAieno+K0+W - BPVBjXcUl08= - SHA512 : WyOIgRxk9SeSoktF6BYVV0tRL7nGNDKQ - A9QyxVCgzg+PwPMV7tzxmwOZI/dB64pP - vQ/D2jqJdf3NS2PHMI4yvg== - RMD160 : qTEPs2SIxPm3iEwsCnwvp9hR4s4= - TIGER : 0HgLucmhCcB56bxOMj+j1Kuja8UIsFRg - CRC32 : VKE1TA== - WHIRLPOOL : JSA35/NmkMOkUWEpcZJf3PR1UUz5WcLG - AmBKPkao3fzQUsLMTJizCV4CwAE0G/Yc - KX0mpW5vx+gk3njya8rAvA== - GOST : yKjiytOwRr3bJcFsxnJ310t1FY6YE3HB - YNT8XP93xpc= - STRIBOG256: 9bzS+5j4ZAoU/P7v3tkKOWn4ZfggcX28 - 9dLQVhaiJtQ= - STRIBOG512: 9LLXgqsRIRiXP2WOrOJt1qhx6psfbACd - un+GTVmu441quX4zaaPIIG9lzDMBAqMg - hZx5DlxsQj3YjMezSUsXLg== - SM3 : Vwii+uw3Ge5Hh3eo1KOombxn2jWgyYRX - ZdyCRZqWZ/E= - -/var/lib/aide/aide.db.new.gz - MD5 : sKt4dVDKY/8A9EY/X4Ue2A== - SHA1 : hagLXMv7G+KbEKh861kjjFSYpRw= - SHA256 : HTHF7k5U294ECjCLneoZ3o8bH6PYgY5u - AzoIyCacZp4= - SHA512 : 5gWi7K/ztWMl7H+PK1doV/tWDHmaE2m/ - ndRXGR7b5J3v82Jv2HeJPoOt5A4Z/9FH - 5H+uCLYaHwRleyalyy5Wew== - RMD160 : uMM1HtAbfz+G3Y9Z+rVR4qjdqcQ= - TIGER : OTHdXNQOxnHnOl6C9M3czSC42+SeZAZA - CRC32 : T9G1Tw== - WHIRLPOOL : FRMnQ2wHgylsTmpKE8RwdUvkzXucHwu1 - W9ZkUrxoXeci2g7jIgoMmpoeDPhH73qz - nZ7fKj1lStSpiUGD5KPeWA== - GOST : haeO5dhT+t34C1GJf+2dc3q1GMN71FqB - kqoiODo+j2o= - STRIBOG256: lgZUZhhd9JfMOXgNzYptapqagwgmvdM+ - 7uWzJsmOxoY= - STRIBOG512: PA6jksprS37xQzHm1ZIvLR9ROa+FxoiF - /xbAe0pSi4lMXXzABrPKkjyK0WtjxFvx - 07Poj2iDwNNcUJWekbaEXA== - SM3 : R5/HXng5MNvrjoCh8/JzrWle1IO8ggsR - P5i2ePX5BpY= -``` diff --git a/docs/en/docs/ShangMi/kernel-module-signing.md b/docs/en/docs/ShangMi/kernel-module-signing.md deleted file mode 100644 index 0119a8c54e5479fb5acf8f787b5730379255157f..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/kernel-module-signing.md +++ /dev/null @@ -1,107 +0,0 @@ -# Kernel Module Signing - -## Overview - -The kernel module signing facility is an important mechanism for protecting Linux kernel security. Signature information is added to the end of the kernel module file in a certain format, and the system checks whether the signature matches the public key preset in the kernel when the kernel module is loaded. In this way, the authenticity and integrity of the kernel module file are ensured. - -## Prerequisites - -1. The openEuler kernel compilation environment has been prepared. For details, see https://gitee.com/openeuler/kernel/wikis/kernel. -2. In openEuler kernel 5.10, the ShangMi (SM) series cryptographic algorithms are supported for kernel module signing. You are advised to select the latest kernel 5.10 source code for compilation. -3. The SM2 private key and certificate used for kernel module signing have been generated. The reference commands using OpenSSL are as follows: - -``` -# Generate a certificate configuration file. (Other fields in the configuration file can be defined as required.) -$ echo 'subjectKeyIdentifier=hash' > mod.cfg -# Generate a private key for SM2 signing. -$ openssl ecparam -genkey -name SM2 -out mod.key -# Generate a signing request. -$ openssl req -new -sm3 -key mod.key -out mod.csr -# Generate an SM2 certificate. -$ openssl x509 -req -days 3650 -extfile mod.cfg -signkey mod.key -in mod.csr -out mod.crt -``` - -## How to Use - -### Scenario 1: Automatic Signing - -Write the certificate and private key to the **mod.pem** file. - -``` -$ cat /path/to/mod.key > mod.pem -$ cat /path/to/mod.crt >> mod.pem -``` - -Use the SM3 algorithm to sign the kernel module in the kernel compilation options. - -``` -$ make openeuler_defconfig -$ make menuconfig -``` - -Choose **Enable loadable module support** > **Sign modules with SM3** on the GUI. - -``` -Which hash algorithm should modules be signed with? (Sign modules with SM3) -``` - -Configure **Cryptographic API** > **Certificates for signature checking** to read the private key and certificate used for kernel signing from **mod.pem**. - -``` -(mod.pem) File name or PKCS#11 URI of module signing key -``` - -Build the kernel. - -``` -$ make -j64 -$ make modules_install -$ make install -``` - -Run the **modinfo** command to check the signature information of the kernel module. - -``` -$ modinfo /usr/lib/modules/5.10.0/kernel/crypto/sm4.ko -filename: /usr/lib/modules/5.10.0/kernel/crypto/sm4.ko -license: GPL v2 -description: Generic SM4 library -srcversion: 371050FDB8BF9878D9B5B9B -depends: -retpoline: Y -intree: Y -name: sm4 -vermagic: 5.10.0 SMP mod_unload modversions -sig_id: PKCS#7 -signer: Internet Widgits Pty Ltd -sig_key: 33:0B:96:3E:1F:C1:CA:28:98:72:F5:AE:FF:3F:A4:F3:50:5D:E1:87 -sig_hashalgo: sm3 -signature: 30:45:02:21:00:81:96:8D:40:CE:7F:7D:AE:3A:4B:CC:DC:9A:F2:B4: - 16:87:3E:C3:DC:77:ED:BC:6E:F5:D8:F3:DD:77:2B:D4:05:02:20:3B: - 39:5A:89:9D:DC:27:83:E8:D8:B4:75:86:FF:33:2B:34:33:D0:90:76: - 32:4D:36:88:84:34:31:5C:83:63:6B -``` - -### Scenario 2: Manual Signing - -Call **sign_file** in the kernel source code directory to sign the specified kernel module. - -``` -$ ./scripts/sign-file sm3 /path/to/mod.key /path/to/mod.crt -``` - -Other steps are the same as those in scenario 1. - -### Scenario 3: Module Loading Verification - -Add **module.sig_enforce** to the kernel startup parameters to enable forcible signature verification for the kernel module. - -``` -linux /vmlinuz-5.10.0-106.1.0.55.oe2209.x86_64 root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap crashkernel=512M module.sig_enforce -``` - -After the system is restarted, only the kernel modules that pass the certificate verification can be loaded. - -``` -# insmod /usr/lib/modules/5.10.0/kernel/crypto/sm4.ko -``` diff --git a/docs/en/docs/ShangMi/overview.md b/docs/en/docs/ShangMi/overview.md deleted file mode 100644 index 5a398bd72e24cb75ef0a7b15d9ebd410e6cb4b54..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/overview.md +++ /dev/null @@ -1,18 +0,0 @@ -# Overview - -The ShangMi (SM) features for the openEuler OS aims to enable SM series cryptographic algorithms for key security features of the OS and provide cryptographic services such as the SM series cryptographic algorithm library, certificates, and secure transmission protocols for upper-layer applications. - -Currently, the following SM features are supported: - -1. SM2, SM3, and SM4 algorithms are supported in the user-mode algorithm libraries, such as OpenSSL and libgcrypt. -2. SM2, SM3, and SM4 cipher suites are supported in OpenSSH. -3. The SM Transport Layer Cryptography Protocol (TLCP) stack is supported in OpenSSL. -4. SM3 and SM4 algorithms are supported for disk encryption (dm-crypt/cryptsetup). -5. The SM3 algorithm is supported for password encryption in user identity authentication (pam/libuser/shadow). -6. The SM3 algorithm is supported for data digest in intrusion detection (AIDE). -7. SM2, SM3, and SM4 algorithms are supported in the kernel cryptographic framework (crypto) and algorithm performance optimization using instruction sets such as AVX/CE/NEON is allowed. -8. The SM3 message digest algorithm and SM2 certificate are supported in Integrity Measurement Architecture and Extended Verification Module (IMA/EVM) of the kernel. -9. The SM2 certificate is supported in kernel module signing and module signature verification. -10. SM4-CBC and SM4-GCM algorithms are supported in Kernel Transport Layer Security (KTLS). -11. SM3 and SM4 algorithms are supported in Kunpeng Accelerator Engine (KAE). -12. The SM3 algorithm and SM2 certificated signature are supported for UEFI secure boot. \ No newline at end of file diff --git a/docs/en/docs/ShangMi/secure-boot.md b/docs/en/docs/ShangMi/secure-boot.md deleted file mode 100644 index 092a05510ad2ba1fff0a99e005f92e95183ce359..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/secure-boot.md +++ /dev/null @@ -1,185 +0,0 @@ -# Secure Boot - -Secure Boot is a standard feature defined in the UEFI specification. It verifies the signature of each component level by level during system startup to ensure the integrity and authenticity of the boot sequence. The UEFI Secure Boot of the Linux system includes the following three procedures: - -1. The BIOS uses its built-in certificate to verify the signature of the shim component. -2. The shim component uses its built-in certificate to verify the signature of the grub component. -3. The grub component verifies the signature of the kernel component through the interface provided by the shim component. - -openEuler adds support for ShangMi (SM) algorithms to the pesign EFI signature tool and its nss algorithm library. That is, openEuler supports SM3 for hash calculation, SM2 for signing and verifying EFI files. In this way, the secure boot of openEuler can be implemented using SM algorithms. - -## Constraints - -- The openEuler shim component supports SM-based Secure Boot, including verification of the GRUB signature by shim, and verification of the kernel signature by GRUB. The verification of the shim signature depends on the BIOS. -- An ARM64/x86 physical machine that supports UEFI Secure Boot is required. -- The pesign tool can sign signatures with a maximum level of 2. -- Currently, the pesign tool can only generate signatures but cannot verify signatures. - -## Preparations - -1. The following software packages (or their later versions) must be installed: - -``` -openssl-1.1.1m-15.oe2203.aarch64 -nss-3.72.0-4.oe2203.aarch64 -pesign-115-2.oe2203.aarch64 -shim-15.6-7.oe2203.aarch64 -crypto-policies-20200619-3.git781bbd4.oe2203.noarch -``` - -2. Download the source code of the openEuler shim component. Ensure that the version in the spec file is later than 15.6-7. - -``` -git clone https://gitee.com/src-openeuler/shim.git -b openEuler-22.03-LTS-SP1 --depth 1 -``` - -3. Install software packages required for building the shim component: - -``` -yum install elfutils-libelf-devel gcc gnu-efi gnu-efi-devel openssl-devel make git rpm-build -``` - -4. Check whether the SM3 algorithm is enabled for nss. If not, modify the file content as follows: - -``` -cat /usr/share/crypto-policies/DEFAULT/nss.txt | grep SM3 -config="disallow=ALL allow=HMAC-SHA256:HMAC-SHA1:HMAC-SHA384:HMAC-SHA512:CURVE25519:SECP256R1:SECP384R1:SECP521R1:aes256-gcm:chacha20-poly1305:aes256-cbc:aes128-gcm:aes128-cbc:SHA256:SHA384:SHA512:SHA224:SHA1:ECDHE-RSA:ECDHE-ECDSA:RSA:DHE-RSA:ECDSA:RSA-PSS:RSA-PKCS:tls-version-min=tls1.0:dtls-version-min=dtls1.0:DH-MIN=1023:DSA-MIN=2048:RSA-MIN=2048:SM3" -``` - -## Generation of Keys and Certificates - -1. Generate the key and certificate for signing the shim component. The shim signature is verified by the BIOS. As most BIOSs do not support SM algorithms, the RSA algorithm is used. For BIOSs that support SM algorithms you can generate the SM2 key and certificate by referring to the next step. - -``` -openssl genrsa -out rsa.key 4096 -openssl req -new -key rsa.key -out rsa.csr -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=secure boot BIOS' -openssl x509 -req -days 365 -in rsa.csr -signkey rsa.key -out rsa.crt -openssl x509 -in rsa.crt -out rsa.der -outform der -``` - -2. Generate the SM2 key and certificate for signing the GRUB and kernel components. - -``` -openssl ecparam -genkey -name SM2 -out sm2.key -openssl req -new -sm3 -key sm2.key -out sm2.csr -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=secure boot shim' -openssl x509 -req -days 3650 -signkey sm2.key -in sm2.csr -out sm2.crt -openssl x509 -in sm2.crt -out sm2.der -outform der -``` - -3. Create an NSS database and import the keys and certificates generated in the preceding two steps to the NSS database. - -``` -# The NSS database is organized in the form of directories. The storage location can be customized. -mkdir certdb -certutil -N -d certdb -# Import the SM2 and RSA certificates to the NSS database and name them sm2 and rsa respectively. -certutil -A -n sm2 -d certdb -t CT,CT,CT -i sm2.crt -certutil -A -n rsa -d certdb -t CT,CT,CT -i rsa.crt -# To import the SM2 and RSA keys to the NSS database, compress them into PKCS 12 files. -openssl pkcs12 -export -out rsa.p12 -inkey rsa.key -in rsa.crt -openssl pkcs12 -export -out sm2.p12 -inkey sm2.key -in sm2.crt -pk12util -d certdb -i rsa.p12 -pk12util -d certdb -i sm2.p12 -``` - -## shim Component Building - -1. Go to the shim source code directory, modify the configuration variables in shim.spec to enable the support for SM algorithms, and specify the built-in SM2 certificate. - -``` -%global enable_sm 1 -%global vendor_cert /path/to/sm2.der -``` - -2. Build the shim software package. - -``` -rpmbuild -ba shim.spec --define "_sourcedir $PWD" -``` - -3. Install the built shim software package. - -``` -rpm -Uvh ~/rpmbuild/RPMS/aarch64/shim-xxx.rpm -``` - -## SM Signature for UEFI Files - -1. Sign the shim component with the RSA key and certificate and replace the original one. - -``` -# ARM64 -pesign -n certdb -c rsa -s -i /boot/efi/EFI/openEuler/shimaa64.efi -o shimaa64.efi.signed -cp shimaa64.efi.signed /boot/efi/EFI/openEuler/shimaa64.efi -# x86 -pesign -n certdb -c rsa -s -i /boot/efi/EFI/openEuler/shimx64.efi -o shimx64.efi.signed -cp shimx64.efi.signed /boot/efi/EFI/openEuler/shimx64.efi -``` - -2. Sign the GRUB component with the SM2 key and certificate and replace the original one. - -``` -# ARM64 -pesign -n certdb -c sm2 -s -i /boot/efi/EFI/openEuler/grubaa64.efi -o grubaa64.efi.signed -d sm3 -cp grubaa64.efi.signed /boot/efi/EFI/openEuler/grubaa64.efi -# x86 -pesign -n certdb -c sm2 -s -i /boot/efi/EFI/openEuler/grubx64.efi -o grubx64.efi.signed -d sm3 -cp grubx64.efi.signed /boot/efi/EFI/openEuler/grubx64.efi -``` - -3. Sign the kernel component with the SM2 key and certificate and replace the original one. (Note that the file name contains the actual version number.) - -``` -# For the ARM64 architecture,you need to decompress and sign the component, and compress it again. -cp /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64 vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.gz -gzip -d vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.gz -pesign -n certdb -c sm2 -s -i vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64 -o vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.signed -d sm3 -gzip vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.signed -cp vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.signed.gz /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64 -# x86 -pesign -n certdb -c sm2 -s -i /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64 -o vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64.signed -d sm3 -cp vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64.signed /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64 -``` - -4. Check the signature information. The following uses shim and GRUB as examples: - -``` -pesign -S -i /boot/efi/EFI/openEuler/grubaa64.efi -pesign -S -i /boot/efi/EFI/openEuler/shimaa64.efi -``` - -## Secure Boot - -Enter the BIOS, import the certificate for signing the shim component, and enable the Secure Boot option. The operation method varies depending on the BIOS. The following uses the Kunpeng 2280 v2 server as an example: - -1. Place the RSA certificate for signing the shim component in the **/boot/efi/EFI/openEuler** directory. - -``` -cp rsa.der /boot/efi/EFI/openEuler -``` - -2. Restart the system. -3. Enter BIOS to enable Secure Boot: - -``` -Setup > Security > Secure Boot > Enable -``` - -4. Set the Secure Boot mode to custom: - -``` -Setup > Security > Secure Boot Certificate Configuration > Secure Boot Mode > Custom -``` - -5. Import the Secure Boot certificate: - -``` -Setup > Security > Secure Boot Certificate Configuration > Options Related to Secure Boot Custom Mode > DB Options > Import Signature > Add Signature by File > Select rsa.der > Save and exit. -``` - -6. Save the configuration and restart the system. The system is started successfully. Secure Boot is enabled. - -``` -mokutil --sb-state -SecureBoot enabled -``` diff --git a/docs/en/docs/ShangMi/ssh-stack.md b/docs/en/docs/ShangMi/ssh-stack.md deleted file mode 100644 index d2ad40da79467da371d2a9c6646ccc25394c6022..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/ssh-stack.md +++ /dev/null @@ -1,52 +0,0 @@ -# SSH Stack - -## Overview - -The OpenSSH component is a Secure Shell Protocol (SSH) component implemented based on libcrypto of OpenSSL in C language. The main function is remote login to ensure the integrity and reliability of encrypted information over an unsecured network. The SSH server and client configuration items provided by openEuler involve key exchange, public key authentication, symmetric encryption, and integrity authentication. The values of these configuration items can be ShangMi (SM) Cipher Suites (including SM2, SM3, and SM4 algorithms). - -## Prerequisites - -OpenSSH 8.8p1-5 or later - -``` -$ rpm -qa | grep openssh -openssh-8.8p1-5.oe2209.x86_64 -openssh-server-8.8p1-5.oe2209.x86_64 -openssh-clients-8.8p1-5.oe2209.x86_64 -``` - -## How to Use - -### Scenario 1: Remote Login - -1. On the client, call **ssh-keygen** to generate a user key, which is saved as **\~/.ssh/id_sm2** and **\~/.ssh/id_sm2.pub** by default. Then, send **\~/.ssh/id_sm2.pub** from the client to the server. (You can also run the **ssh-copy-id** command to send the file.) - -``` -$ ssh-keygen -t sm2 -m PEM -``` - -2. On the server, call **ssh-keygen** to generate a host key and add the public key sent by the client to the authorized key file list. (If you run the **ssh-copy-id** command, the public key is automatically written.) - -``` -$ ssh-keygen -t sm2 -m PEM -f /etc/ssh/ssh_host_sm2_key -$ cat /path/to/id_sm2.pub >> ~/.ssh/authorized_keys -``` - -3. On the server, modify the **/etc/ssh/sshd_config** file to support login using SM series cryptographic algorithms. The following table lists the SM configuration items. - -| Description | Configuration Item | SM Value | -|---------------------|------------------------|---------------| -| Authentication key for the host key and public key (configurable only on the server)| HostKeyAlgorithms | /etc/ssh/ssh_host_sm2_key | -| Host key and public key authentication algorithm | HostKeyAlgorithms | sm2 | -| Key exchange algorithm | KexAlgorithms | sm2-sm3 | -| Symmetric cryptographic algorithm | Ciphers | sm4-ctr | -| Integrity check algorithm | MACs | hmac-sm3 | -| User public key authentication algorithm | PubkeyAcceptedKeyTypes | sm2 | -| Authentication key for the user public key (configurable only on the client) | IdentityFile | ~/.ssh/id_sm2 | -| Hash algorithm used for printing key fingerprints | FingerprintHash | sm3 | - -4. On the client, configure the SM series cryptographic algorithms to complete the login. You can enable the SM Cipher Suites on the client by running commands or modifying the configuration file. The following shows how to log in using the CLI: - -``` -ssh -o PreferredAuthentications=publickey -o HostKeyAlgorithms=sm2 -o PubkeyAcceptedKeyTypes=sm2 -o Ciphers=sm4-ctr -o MACs=hmac-sm3 -o KexAlgorithms=sm2-sm3 -i ~/.ssh/id_sm2 [remote-ip] -``` diff --git a/docs/en/docs/ShangMi/tlcp-stack.md b/docs/en/docs/ShangMi/tlcp-stack.md deleted file mode 100644 index 30e6b9f92ae6e93050e1e6a03ce5cf5e3f4b21be..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/tlcp-stack.md +++ /dev/null @@ -1,518 +0,0 @@ -# TLCP Stack - -## Overview - -Transport Layer Cryptography Protocol (TLCP) is a secure communication protocol that complies with the *GB/T 38636-2020 Information security technology-Transport layer cryptography protocol (TLCP)*. TLCP requires an encryption certificate and a signature certificate with their corresponding private keys in communication. Based on the open source version, the OpenSSL software released after openEuler 22.09 supports the TLCP and provides the following functions: - -- Double certificates for SM series cryptographic algorithms based on TLCP -- ECC_SM4_CBC_SM3 and ECDHE_SM4_CBC_SM3 cipher suites -- SM2 certificates - -## Prerequisites - -The version of the OpenSSL software installed in the openEuler operating system is later than 1.1.1m-4. - -```shell -$ rpm -qa openssl -openssl-1.1.1m-6.oe2209.x86_64 -``` - -## How to Use - -### Scenario 1: Generating Two SM2 Certificates - -According to the TLCP, two certificates are required for communication: signature certificate and encryption certificate. The signature certificate is used for identity authentication, and the encryption certificate is used for data encryption during key negotiation. CA is short for Certificate Authority. A certificate is generated only after the CSR certificate request file is issued by the CA. The following is a reference case that describes how to use a self-signed CA certificate to issue an entity certificate: - -1. Prepare the configuration file **openssl.cnf** required for generating the certificates. The reference content is as follows (modified based on the OpenSSL source code **apps/openssl.cnf**): - - ```conf - # This definition stops the following lines choking if HOME isn't - # defined. - HOME = . - - # Extra OBJECT IDENTIFIER info: - #oid_file = $ENV::HOME/.oid - oid_section = new_oids - - # To use this configuration file with the "-extfile" option of the - # "openssl x509" utility, name here the section containing the - # X.509v3 extensions to use: - # extensions = - # (Alternatively, use a configuration file that has only - # X.509v3 extensions in its main [= default] section.) - - [ new_oids ] - - # We can add new OIDs in here for use by 'ca', 'req' and 'ts'. - # Add a simple OID like this: - # testoid1=1.2.3.4 - # Or use config file substitution like this: - # testoid2=${testoid1}.5.6 - - # Policies used by the TSA examples. - tsa_policy1 = 1.2.3.4.1 - tsa_policy2 = 1.2.3.4.5.6 - tsa_policy3 = 1.2.3.4.5.7 - - #################################################################### - [ ca ] - default_ca = CA_default # The default ca section - - #################################################################### - [ CA_default ] - - dir = ./demoCA # Where everything is kept - certs = $dir/certs # Where the issued certs are kept - crl_dir = $dir/crl # Where the issued crl are kept - database = $dir/index.txt # database index file. - #unique_subject = no # Set to 'no' to allow creation of - # several certs with same subject. - new_certs_dir = $dir/newcerts # default place for new certs. - - certificate = $dir/cacert.pem # The CA certificate - serial = $dir/serial # The current serial number - crlnumber = $dir/crlnumber # the current crl number - # must be commented out to leave a V1 CRL - crl = $dir/crl.pem # The current CRL - private_key = $dir/private/cakey.pem# The private key - - x509_extensions = usr_cert # The extensions to add to the cert - - # Comment out the following two lines for the "traditional" - # (and highly broken) format. - name_opt = ca_default # Subject Name options - cert_opt = ca_default # Certificate field options - - # Extension copying option: use with caution. - # copy_extensions = copy - - # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs - # so this is commented out by default to leave a V1 CRL. - # crlnumber must also be commented out to leave a V1 CRL. - # crl_extensions = crl_ext - - default_days = 365 # how long to certify for - default_crl_days= 30 # how long before next CRL - default_md = default # use public key default MD - preserve = no # keep passed DN ordering - - # A few difference way of specifying how similar the request should look - # For type CA, the listed attributes must be the same, and the optional - # and supplied fields are just that :-) - policy = policy_match - - # For the CA policy - [ policy_match ] - countryName = match - stateOrProvinceName = match - organizationName = match - organizationalUnitName = optional - commonName = supplied - emailAddress = optional - - # For the 'anything' policy - # At this point in time, you must list all acceptable 'object' - # types. - [ policy_anything ] - countryName = optional - stateOrProvinceName = optional - localityName = optional - organizationName = optional - organizationalUnitName = optional - commonName = supplied - emailAddress = optional - - #################################################################### - [ req ] - default_bits = 2048 - default_keyfile = privkey.pem - distinguished_name = req_distinguished_name - attributes = req_attributes - x509_extensions = v3_ca # The extensions to add to the self signed cert - - # Passwords for private keys if not present they will be prompted for - # input_password = secret - # output_password = secret - - # This sets a mask for permitted string types. There are several options. - # default: PrintableString, T61String, BMPString. - # pkix : PrintableString, BMPString (PKIX recommendation before 2004) - # utf8only: only UTF8Strings (PKIX recommendation after 2004). - # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). - # MASK:XXXX a literal mask value. - # WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings. - string_mask = utf8only - - # req_extensions = v3_req # The extensions to add to a certificate request - - [ req_distinguished_name ] - countryName = Country Name (2 letter code) - countryName_default = AU - countryName_min = 2 - countryName_max = 2 - - stateOrProvinceName = State or Province Name (full name) - stateOrProvinceName_default = Some-State - - localityName = Locality Name (eg, city) - - 0.organizationName = Organization Name (eg, company) - 0.organizationName_default = Internet Widgits Pty Ltd - - # we can do this but it is not needed normally :-) - #1.organizationName = Second Organization Name (eg, company) - #1.organizationName_default = World Wide Web Pty Ltd - - organizationalUnitName = Organizational Unit Name (eg, section) - #organizationalUnitName_default = - - commonName = Common Name (e.g. server FQDN or YOUR name) - commonName_max = 64 - - emailAddress = Email Address - emailAddress_max = 64 - - # SET-ex3 = SET extension number 3 - - [ req_attributes ] - challengePassword = A challenge password - challengePassword_min = 4 - challengePassword_max = 20 - - unstructuredName = An optional company name - - [ usr_cert ] - - # These extensions are added when 'ca' signs a request. - - # This goes against PKIX guidelines but some CAs do it and some software - # requires this to avoid interpreting an end user certificate as a CA. - - basicConstraints=CA:FALSE - - # Here are some examples of the usage of nsCertType. If it is omitted - # the certificate can be used for anything *except* object signing. - - # This is OK for an SSL server. - # nsCertType = server - - # For an object signing certificate this would be used. - # nsCertType = objsign - - # For normal client use this is typical - # nsCertType = client, email - - # and for everything including object signing: - # nsCertType = client, email, objsign - - # This is typical in keyUsage for a client certificate. - # keyUsage = nonRepudiation, digitalSignature, keyEncipherment - - # This will be displayed in Netscape's comment listbox. - nsComment = "OpenSSL Generated Certificate" - - # PKIX recommendations harmless if included in all certificates. - subjectKeyIdentifier=hash - authorityKeyIdentifier=keyid,issuer - - # This stuff is for subjectAltName and issuerAltname. - # Import the email address. - # subjectAltName=email:copy - # An alternative to produce certificates that aren't - # deprecated according to PKIX. - # subjectAltName=email:move - - # Copy subject details - # issuerAltName=issuer:copy - - #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem - #nsBaseUrl - #nsRevocationUrl - #nsRenewalUrl - #nsCaPolicyUrl - #nsSslServerName - - # This is required for TSA certificates. - # extendedKeyUsage = critical,timeStamping - - [ v3_req ] - - # Extensions to add to a certificate request - - basicConstraints = CA:FALSE - keyUsage = nonRepudiation, digitalSignature - - [ v3enc_req ] - - # Extensions to add to a certificate request - - basicConstraints = CA:FALSE - keyUsage = keyAgreement, keyEncipherment, dataEncipherment - - [ v3_ca ] - - # Extensions for a typical CA - - - # PKIX recommendation. - - subjectKeyIdentifier=hash - - authorityKeyIdentifier=keyid:always,issuer - - basicConstraints = critical,CA:true - - # Key usage: this is typical for a CA certificate. However since it will - # prevent it being used as an test self-signed certificate it is best - # left out by default. - keyUsage = cRLSign, keyCertSign - - # Some might want this also - # nsCertType = sslCA, emailCA - - # Include email address in subject alt name: another PKIX recommendation - # subjectAltName=email:copy - # Copy issuer details - # issuerAltName=issuer:copy - - # DER hex encoding of an extension: beware experts only! - # obj=DER:02:03 - # Where 'obj' is a standard or added object - # You can even override a supported extension: - # basicConstraints= critical, DER:30:03:01:01:FF - - [ crl_ext ] - - # CRL extensions. - # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. - - # issuerAltName=issuer:copy - authorityKeyIdentifier=keyid:always - - [ proxy_cert_ext ] - # These extensions should be added when creating a proxy certificate - - # This goes against PKIX guidelines but some CAs do it and some software - # requires this to avoid interpreting an end user certificate as a CA. - - basicConstraints=CA:FALSE - - # Here are some examples of the usage of nsCertType. If it is omitted - # the certificate can be used for anything *except* object signing. - - # This is OK for an SSL server. - # nsCertType = server - - # For an object signing certificate this would be used. - # nsCertType = objsign - - # For normal client use this is typical - # nsCertType = client, email - - # and for everything including object signing: - # nsCertType = client, email, objsign - - # This is typical in keyUsage for a client certificate. - # keyUsage = nonRepudiation, digitalSignature, keyEncipherment - - # This will be displayed in Netscape's comment listbox. - nsComment = "OpenSSL Generated Certificate" - - # PKIX recommendations harmless if included in all certificates. - subjectKeyIdentifier=hash - authorityKeyIdentifier=keyid,issuer - - # This stuff is for subjectAltName and issuerAltname. - # Import the email address. - # subjectAltName=email:copy - # An alternative to produce certificates that aren't - # deprecated according to PKIX. - # subjectAltName=email:move - - # Copy subject details - # issuerAltName=issuer:copy - - #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem - #nsBaseUrl - #nsRevocationUrl - #nsRenewalUrl - #nsCaPolicyUrl - #nsSslServerName - - # This really needs to be in place for it to be a proxy certificate. - proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo - - #################################################################### - [ tsa ] - - default_tsa = tsa_config1 # the default TSA section - - [ tsa_config1 ] - - # These are used by the TSA reply generation only. - dir = ./demoCA # TSA root directory - serial = $dir/tsaserial # The current serial number (mandatory) - crypto_device = builtin # OpenSSL engine to use for signing - signer_cert = $dir/tsacert.pem # The TSA signing certificate - # (optional) - certs = $dir/cacert.pem # Certificate chain to include in reply - # (optional) - signer_key = $dir/private/tsakey.pem # The TSA private key (optional) - signer_digest = sha256 # Signing digest to use. (Optional) - default_policy = tsa_policy1 # Policy if request did not specify it - # (optional) - other_policies = tsa_policy2, tsa_policy3 # acceptable policies (optional) - digests = sha1, sha256, sha384, sha512 # Acceptable message digests (mandatory) - accuracy = secs:1, millisecs:500, microsecs:100 # (optional) - clock_precision_digits = 0 # number of digits after dot. (optional) - ordering = yes # Is ordering defined for timestamps? - # (optional, default: no) - tsa_name = yes # Must the TSA name be included in the reply? - # (optional, default: no) - ess_cert_id_chain = no # Must the ESS cert id chain be included? - # (optional, default: no) - ess_cert_id_alg = sha1 # algorithm to compute certificate - # identifier (optional, default: sha1) - ``` - -2. Generate a self-signed CA certificate. - - ```shell - openssl ecparam -name SM2 -out SM2.pem - openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=root ca' -keyout CA.key -newkey ec:SM2.pem -new -out CA.csr - openssl x509 -sm3 -req -days 30 -in CA.csr -extfile ./openssl.cnf -extensions v3_ca -signkey CA.key -out CA.crt - ``` - -3. Generate the server signature certificate and encryption certificate. - - ```shell - openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=server sign' -keyout SS.key -newkey ec:SM2.pem -new -out SS.csr - openssl x509 -sm3 -req -days 30 -in SS.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3_req -out SS.crt -CAcreateserial - openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=server enc' -keyout SE.key -newkey ec:SM2.pem -new -out SE.csr - openssl x509 -sm3 -req -days 30 -in SE.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3enc_req -out SE.crt -CAcreateserial - ``` - -4. Generate the client signature certificate and encryption certificate. - - ```shell - openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=client sign' -keyout CS.key -newkey ec:SM2.pem -new -out CS.csr - openssl x509 -sm3 -req -days 30 -in CS.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3_req -out CS.crt -CAcreateserial - openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=client enc' -keyout CE.key -newkey ec:SM2.pem -new -out CE.csr - openssl x509 -sm3 -req -days 30 -in CE.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3enc_req -out CE.crt -CAcreateserial - ``` - -### Scenario 2: Using the OpenSSL CLI to Verify the TLCP Stack - -The **s_server/s_client** tool provided by OpenSSL can be used to test the TLCP. - -```shell -# Start the server. -openssl s_server -verify 5 -accept 4433 \ - -cert SS.crt \ - -key SS.key \ - -dcert SE.crt \ - -dkey SE.key \ - -CAfile CA.crt - -# Start the client. -openssl s_client -verify 5 -connect 127.0.0.1:4433 \ - -cert CS.crt \ - -key CS.key \ - -dcert CE.crt \ - -dkey CE.key \ - -CAfile CA.crt -tlcp -``` - -### Scenario 3: Using OpenSSL APIs - -Reference code on the server: - -```cpp -int main() { - // Define the variables. - SSL_CTX *ctx = NULL; - const char *sign_cert_file = "SS.crt"; - const char *sign_key_file = "SS.key"; - const char *enc_cert_file = "SE.crt"; - const char *enc_key_file = "SE.key"; - - // Generate the context. - ctx = SSL_CTX_new(TLS_server_method()); - - // Load the signature certificate and encryption certificate and their private keys. - if (!SSL_CTX_use_gm_certificate_file(ctx, sign_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) - goto err; - - if (!SSL_CTX_use_gm_PrivateKey_file(ctx, sign_key_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) - goto err; - - if (!SSL_CTX_use_gm_certificate_file(ctx, enc_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) - goto err; - - if (!SSL_CTX_use_gm_PrivateKey_file(ctx, enc_key_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) - goto err; - - SSL_CTX_set_options(ctx, SSL_OP_ENCCERT_SECOND_POSITION); - - // The subsequent procedure is the same as that of the standard TLS. - SSL *ssl = SSL_new(ctx); -} -``` - -Client reference code: - -```cpp -int main() { - // Define the variables. - SSL_CTX *ctx = NULL; - const char *sign_cert_file = "CS.crt"; - const char *sign_key_file = "CS.key"; - const char *enc_cert_file = "CE.crt"; - const char *enc_key_file = "CE.key"; - - // Generate the context. - ctx = SSL_CTX_new(TLCP_client_method()); - - // Load the signature certificate and encryption certificate and their private keys. - if (!SSL_CTX_use_gm_certificate_file(ctx, sign_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) - goto err; - - if (!SSL_CTX_use_gm_PrivateKey_file(ctx, sign_key_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) - goto err; - - if (!SSL_CTX_use_gm_certificate_file(ctx, enc_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) - goto err; - - if (!SSL_CTX_use_gm_PrivateKey_file(ctx, enc_key_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) - goto err; - - // Set the cipher suite to ECC-SM4-CBC-SM3 or ECDHE-SM4-CBC-SM3. - // This step is not mandatory. By default, ECC-SM4-CBC-SM3 is preferred. - if(SSL_CTX_set_cipher_list(ctx, "ECC-SM4-CBC-SM3") <= 0) - goto err; - - // The subsequent procedure is the same as that of the standard TLS. - SSL *ssl = SSL_new(ctx); -} -``` - -# KTLS Uninstallation - -## Overview - -The Linux kernel protocol stack implements only the TCP/IP model and does not support the SSL/TLS session layer protocol. Currently, TLS encryption and decryption are implemented in user mode. However, in some scenarios, for example, when the kernel sends a file, multiple cross-state copies are generated, causing performance overhead. Therefore, the kernel implements KTLS. That is, the encryption context can be configured for the socket to offload data encryption to the kernel mode or lower-layer hardware. - -The KTLS feature of the openEuler kernel 5.10 supports SM series cryptographic algorithms. Currently, the SM4-GCM and SM4-CCM algorithms are supported. - -## Prerequisites - -Kernel 5.10.0-106 or later - -```shell -$ rpm -qa kernel -kernel-5.10.0-106.1.0.55.oe2209.x86_64 -``` - -## How to Use - -The method of calling SM series cryptographic algorithms is the same as that of calling other algorithms of the same type. For details, see the [Linux kernel document](https://www.kernel.org/doc/html/v5.10/networking/tls.html). diff --git a/docs/en/docs/ShangMi/user-identity-authentication.md b/docs/en/docs/ShangMi/user-identity-authentication.md deleted file mode 100644 index 4ce8db4c75e3124861e31d44fb4963a08b61ff09..0000000000000000000000000000000000000000 --- a/docs/en/docs/ShangMi/user-identity-authentication.md +++ /dev/null @@ -1,132 +0,0 @@ -# User Identity Authentication - -Operating systems usually use the password mechanism to authenticate users. openEuler provides user password management components, such as PAM, passwd, shadow, and libuser. After a user password is set, it needs to be encrypted for storage. Generally, the hash algorithm is used for encryption. The user password management component provided by openEuler supports the ShangMi 3 (SM3) cryptographic hash algorithm. - -## Using PAM to Encrypt User Passwords - -### Overview - -PAM is a pluggable authentication module of the system that provides an authentication mechanism for upper-layer applications. PAM released by openEuler supports user password encryption using the SM3 algorithm. - -### Prerequisites - -1. PAM 1.5.2-2 or later - -``` -$ rpm -qa pam -pam-1.5.2-2.oe2209.x86_64 -``` - -2. libxcrypt 4.4.26-2 or later - -``` -$ rpm -qa libxcrypt -pam-4.4.26-2.oe2209.x86_64 -``` - -### How to Use - -1. Open the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files, locate the line starting with **password sufficient pam_unix.so**, and change the algorithm field in the line to **sm3**. - -``` -$ cat /etc/pam.d/password-auth -...... -password sufficient pam_unix.so sm3 shadow nullok try_first_pass use_authtok -...... - -$ cat /etc/pam.d/system-auth -...... -password sufficient pam_unix.so sm3 shadow nullok try_first_pass use_authtok -...... -``` - -2. After the configuration is modified, the password changed by running the **passwd** command or the password created by a new user is encrypted using the SM3 algorithm. The encryption result starts with **sm3** and is stored in **/etc/shadow**. - -``` -$ passwd testuser -Changing password for user testuser. -New password: -Retype new password: -passwd: all authentication tokens updated successfully. -$ cat /etc/shadow | grep testuser -testuser:$sm3$wnY86eyUlB5946gU$99LlMr0ddeZNDqnB2KRxn9f30SFCCvMv1WN1cFdsKJ2:19219:0:90:7:35:: -``` - -### Notes - -1. By default, the SHA512 algorithm is used. After the SM3 algorithm is used, the existing user passwords are not affected. The cryptographic algorithm takes effect only after the passwords are changed. -2. If PAM and libxcrypt need to be downgraded to non-SM versions and existing user passwords are encrypted using the SM3 algorithm, modify the configuration first. - Set the algorithm to a non-SM algorithm, change the user passwords, and downgrade the software to a non-SM version. Otherwise, these users cannot log in to the system. - - -## Using Shadow to Encrypt User Passwords - -### Overview - -Shadow is a common user management component in Linux. It provides commands such as **chpasswd**, **chgpasswd**, and **newusers**. The shadow component provided by openEuler supports the SM3 algorithm in user management. By default, Shadow uses the PAM security authentication mechanism. Therefore, the SM3 algorithm supported by Shadow affects only the **chpasswd** and **chgpasswd** commands. - -### Prerequisites - -Shadow 4.9-4 or later - -``` -$ rpm -qa shadow -shadow-4.9-4.oe2209.x86_64 -``` - -### How to Use - -1. By default, **chpasswd** uses the PAM configuration. Use **-c** to specify the SM3 algorithm. The encryption result starts with **sm3** and is stored in **/etc/shadow**. - -``` -$ echo testuser:testPassword |chpasswd -c SM3 -$ cat /etc/shadow | grep testuser -testuser:$sm3$moojQQeBfdGOrL14$NqjckLHlk3ICs1cx.0rKZwRHafjVlqksdSJqfx9eYh6:19220:0:99999:7::: - -``` -2. By default, **chgpasswd** uses the PAM configuration. Use **-c** to specify the SM3 algorithm. The encryption result starts with **sm3** and is stored in **/etc/shadow**. - -``` -$ echo testGroup:testPassword |chpasswd -c SM3 -$ cat /etc/gshadow | grep testGroup -testGroup:$sm3$S3h3X6U6KsXg2Gkc$LFCAnKbi6JItarQz4Y/Aq9/hEbEMQXq9nQ4rY1j9BY9:: -``` - -### Notes - -By default, Shadow uses the PAM security authentication mechanism. When **-c** is used to specify the encryption algorithm in related commands, the PAM mechanism is not used. - -## Using libuser to Encrypt User Passwords - -### Overview - -The libuser library implements a standardized interface for operating and managing users and group accounts. This library is encapsulated and provides the command line interface (CLI) and Python interface for managing users and groups. User passwords can be encrypted using algorithms such as DES, MD5, Blowfish, SHA256, and SHA512. libuser released by openEuler supports the SM3 algorithm. - -### Prerequisites - -libuser 0.63-3 or later - -``` -$ rpm -qa libuser -libuser-0.63-3.oe2209.x86_64 -``` - -### How to Use - -1. Configure **crypt_style=sm3** in the **[defaults]** section in **/etc/libuser.conf**. - -``` -$ cat /etc/libuser.conf -...... -[defaults] -crypt_style = sm3 -...... -``` - -2. When you run the **lusermod**, **lpasswd**, or **luseradd** command to set a user password, the default password encryption algorithm is SM3. The encryption result starts with **sm3** and is stored in **/etc/shadow**. - -``` -# luseradd testuser -P Test@123 -# cat /etc/shadow | grep testuser -testuser:$sm3$1IJtoN6zlBDCiPKC$5oxscBTgiquPAEmZWGNDVqTPrboHJw3fFSohjF6sONB:18862:0:90:7:35:: -``` diff --git a/docs/en/docs/StratoVirt/Install_StratoVirt.md b/docs/en/docs/StratoVirt/Install_StratoVirt.md deleted file mode 100644 index 4751e1131703fe67f0b865e685a5e202193e85d8..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/Install_StratoVirt.md +++ /dev/null @@ -1,37 +0,0 @@ -# Installing StratoVirt - -## Software and Hardware Requirements - -### Minimum Hardware Requirements - -- Processor architecture: Only the AArch64 and x86_64 processor architectures are supported. AArch64 requires ARMv8 or a later version that supports virtualization extension. x86_64 requires VT-x support. - -- 2-core CPU -- 4 GiB memory -- 16 GiB available disk space - -### Software Requirements - -Operating system: openEuler 21.03 - - - -## Component Installation - -To use StratoVirt virtualization, it is necessary to install StratoVirt. Before the installation, ensure that the openEuler Yum source has been configured. - -1. Run the following command as user **root** to install the StratoVirt component: - - ``` - # yum install stratovirt - ``` - - -2. Check whether the installation is successful. - - ``` - $ stratovirt -version - StratoVirt 2.1.0 - ``` - - diff --git a/docs/en/docs/StratoVirt/Interconnect_libvirt.md b/docs/en/docs/StratoVirt/Interconnect_libvirt.md deleted file mode 100644 index 6f36f9aedea6d51dedba84c4be01f3c6af047977..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/Interconnect_libvirt.md +++ /dev/null @@ -1,683 +0,0 @@ -# Interconnecting with libvirt - -## Overview - -libvirt is an upper-layer software that manages different types of Hypervisors using different drivers and provides unified and stable APIs. - -In cloud scenarios, libvirt is widely used to manage large numbers of VMs. To facilitate the deployment, orchestration, and management of large-scale StratoVirt VMs, StratoVirt interconnects with libvirt through the libvirt northbound interface. In this case, you can use an XML file of libvirt to describe a VM, including the VM name, CPU, and disks. - -This chapter describes the XML configurations supported by the StratoVirt platform and how to use the `virsh` command to manage VMs. - -## Prerequisites - -To interconnect StratoVirt with libvirt, the host must meet the following requirements: - -- The Yum source has been correctly configured. -- libvirt has been correctly installed and started. -- StratoVirt has been correctly installed. - -## VM Configuration - -The libvirt tool uses an XML file to describe features about a VM, including the VM name, CPUs, memory, disks, and NICs. You can manage the VM by modifying the XML configuration file. - -Before interconnecting StratoVirt with libvirt, configure the XML file first. This section describes the supported XML configuration items and configuration methods during interconnection between StratoVirt and libvirt. - -> ![](./public_sys-resources/icon-note.gif)**NOTE** -> -> Before using libvirt to manage StratoVirt VMs, pay attention to the features supported by StratoVirt, including mutually exclusive relationships between features, and feature prerequisites and specifications. For details, see [Configuring VMs](VM_configuration.md) in CLI mode. - -### VM Description - -A VM XML file must contain the two basic elements that describe the VM: **domain** and **name**. - -#### Elements - -- **domain**: root element of the VM configuration, which is used to configure the Hypervisor type that runs the StratoVirt VM. - - Attribute **type**: type of **domain**. In StratoVirt, the value is **kvm**. - -- **name**: VM name. - - A VM name contains a maximum of 255 characters, consisting of digits, letters, underscores, hyphens, and colons. Names of VMs on the same host must be unique. - -#### Configuration Example - -Assume that the VM name is StratoVirt. The following is the example: - -```shell - - StratoVirt - ... - -``` - -### Virtual CPU and Memory - -This section describes how to configure virtual CPUs and memory. - -#### Elements - -- **vcpu**: number of virtual processors. - -- **memory**: size of the virtual memory. - - Attribute **unit**: memory unit. The value can be **KiB** (210 bytes), **MiB** (220 bytes), **GiB** (230 bytes), or **TiB** (240 bytes). - - > ![](./public_sys-resources/icon-note.gif)**NOTE** - > - > StratoVirt does not support the CPU topology. Do not set this item. - -#### Configuration Example - -The following is an example of configuring 8 GiB memory and four virtual CPUs: - -```xml - - ... - 4 - 8 - ... - -``` - -### VM Devices - -This section describes how to use the XML file to configure VM devices, including disk, NIC, RNG (random number generator), balloon, console, and vsock devices. - -#### Disks - -#### Elements - -- Attribute **type**: type of the backend storage medium. In StraroVirt, the value is **file**. - - Attribute **device**: type of the storage medium presented to the VM. In StraroVirt, the value is **disk**. - -- **driver**: details about the backend driver. - - Attribute **type**: disk format type. In StraroVirt, the value is **raw**. Currently, StratoVirt supports only **raw** disks. - - Attribute **iothread**: iothread configured for the disk. The value is the iothread ID. Before configuring the disk iothread, use the **iothread** element to configure the iothread quantity. - -- **source**: backend storage medium. - - Attribute **file**: disk path. - -- **target**: details about the backend driver. - - Attribute **dev**: disk name. - - Attribute **bus**: disk device type. In StraroVirt, the value is **virtio**. - -- **iotune**: disk I/O feature. - - Attribute **total_iops_sec**: disk IOPS. - -- **address**: attribute of the bus to which the device is to be mounted. - - Attribute **type**: bus type. In StratoVirt, the value is **pci**. - - Attribute **domain**: domain of the VM. - - Attribute **bus**: ID of the bus to which the device is to be mounted. - - Attribute **slot**: ID of the slot to which the device is to be mounted. The value range is [0, 31]. - - Attribute **function**: ID of the function to which the device is to be mounted. The value range is [0, 7]. - -#### Configuration Example - -Set the disk path to **/home/openEuler-21.09-stratovirt.img**, iothread quantity to **1**, disk iothread to **iothread1**, and IOPS to **10000**, and mount the disk to the PCI bus whose bus ID is 1, slot ID is 0, and function ID is 0. The following is the example: - -```xml - - ... - 1 - - - - - - - 10000 - -
- - ... - - -``` - -#### Network Devices - -#### Elements - -- **interface**: network interface. - - Attribute **type**: network device type. - -- **mac**: virtual NIC address. - - Attribute **address**: virtual NIC address. - -- **source**: backend network bridge. - - Attribute **bridge**: network bridge. - -- **target**: backend NIC. - - Attribute **dev**: backend tap device. - -- **model**: virtual NIC type. - - Attribute **type**: virtual NIC type. In StratoVirt, the value is **virtio**. - -- **driver**: whether to enable the vhost. - - Attribute **name**: If **name** is set to **qemu**, the virtio-net device is used. If **driver** is not configured or **name** is set to **vhost**, the vhost-net device is used. - -#### Configuration Example - -Before configuring the network, [configure the Linux bridge](https://gitee.com/yanhuiling/docs/blob/stable2-20.03_LTS_SP2/docs/en/docs/Virtualization/environment-preparation.md#preparing-the-vm-network) first. Set the MAC address to **de:ad:be:ef:00:01** and network bridge to **br0**. Use the virtio-net device, and mount it to the PCI bus whose bus ID is 2, slot ID is 0, and function ID is 0. The following is the example: - -```xml - - ... - - - - - - -
- - ... - - -``` - -#### Balloon Devices - -#### Elements - -- **memballoon**: balloon device type. - - Attribute **model**: type of the balloon device. In StratoVirt, the value is **virtio**. - -- **alias**: alias of the balloon device. - - Attribute **name**: ID of the balloon device. - - Attribute **autodeflate**: auto deflate feature. The options are **on** and **off**. - -#### Configuration Example - -Configure the balloon device, enable the auto deflate feature, and mount it to the PCI bus whose bus ID is 3, slot ID is 0, and function ID is 0. The following is the example: - -```xml - - ... - - - - -
- - ... - - -``` - -#### Console Devices - -The console device is mounted to the virtio-serial bus. Therefore, you need to create a virtio-serial device when creating a console device. - -> ![](./public_sys-resources/icon-note.gif)**NOTE** -> -> The console device of StratoVirt does not support the multi-port feature. Each VM can be configured with only one console device. - -#### Elements - -- **controller**: controller. - - Attribute **type**: controller type. The value is **virtio-serial**. - -- **alias**: alias. - - Attribute **name**: device ID. - -- **console**: console device. - - Attribute **type**: redirection mode of the console device. The following redirection modes are supported: **pty**, **file**, and **unix**. - -- **target**: configuration of the console device. - - Attribute **type**: console device type. In StratoVirt, the value is **virtio**. - -#### Configuration Example - -Set the redirection mode to **pty** and mount the console device to the PCI bus whose bus ID is 4, slot ID is 0, and function ID is 0. The following is the example: - -```xml - - ... - - - -
- - - - - - ... - - -``` - -#### RNG Devices - -#### Elements - -- **rng**: RNG device. - - Attribute **model**: type of the RNG device. In StratoVirt, the value is **virtio**. - -- **rate**: rate at which the RNG device generates random numbers. - - Attribute **period**: period of random number generation, in milliseconds. Currently, the StratoVirt does not allow you to set the period value. The default value (1000 milliseconds) is used. - - Attribute **bytes**: maximum number of bytes generated in a period. - -- **backend**: RNG device backend. The value is the path of the RNG device on the host. - - Attribute **model**: type of the backend device. In StratoVirt, the value is **random**. - -#### Configuration Example - -Configure that a maximum of 1234 bytes are generated within 1000 ms. The path of the RNG device on the host is **/dev/random**, and the device is mounted to the PCI bus whose bus ID is 5, slot ID is 0, and function ID is 0. The following is the example: - -```xml - - ... - - - - /dev/random -
- - ... - - -``` - -#### vsock Devices - -#### Elements - -- **vsock**: vsock device. - - Attribute **model**: type of the vsock device. In StratoVirt, the value is **virtio**. - -- **cid**: CID of the vsock device. - - Attribute **address**: sets the CID value. - -#### Configuration Example - -Set **cid** to **8** and mount the device to the PCI bus whose bus ID is 6, slot ID is 0, and function ID is 0. The following is the example: - -```xml - - ... - - - -
- - ... - - -``` - -### System Architecture Configuration - -The XML file also contains some architecture-related configurations, such as the pflash and mainboard. - -#### Elements - -- **os**: defines VM startup parameters. - - Child element **type**: VM type. Attribute **arch** indicates the architecture and **machine** indicates the mainboard type. In StratoVirt, the AArch64 architecture supports only the virt mainboard, and the x86_64 architecture supports only the Q35 mainboard. - - Child element **kernel**: kernel path. - - Child element **cmdline**: command line startup parameters. - - Child element **loader**: loading firmware. Attribute **readonly** indicates that the firmware is read-only and **type** indicates the firmware type. In StratoVirt, the type value is **pflash**. - -- **features**: features supported by Hypervisors. - - Child element **acpi**: whether to support ACPI. The ACPI feature is used in StratoVirt, so it must be configured. - - Child element **gic**: interrupt processor specified for ARM processors. Attribute **version** indicates the GIC version. In StratoVirt, the value is **3**. - -#### Configuration Example - -Set the CPU architecture of the VM to ARM and the mainboard to **virt**. The startup command is `console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw`. The path of pflash is **/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw**, which is read-only. The kernel path is **/home/std-vmlinuxz**. The following is the example: - -```xml - - ... - - hvm - /home/std-vmlinuxz - console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw - `/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw` - - ... - -``` - -### Huge Page Memory - -##### Elements - -- **memoryBacking**: configures the memory information. - -- **hugepages**: configures memory huge pages. - -- **page**: configures huge pages. - - Attribute **size**: size of huge memory pages. - - Attribute **unit**: unit of the huge page size. - -#### Configuration Example - -The following is an example of configuring 2 MiB huge pages: - -```xml - - ... - - - - - - ... - -``` - -### Configuration Examples - -#### x86 Configuration Example - -Configure a server named StratoVirt with 8 GiB memory, 1 GiB huge pages, and four vCPUs. Its architecture is x86_64 and the mainboard type is Q35. The following is a configuration example of the corresponding XML file: - -```xml - - StratoVirt - 8 - - - - - - - 4 - - 1 - - hvm - /path/to/standard_vm_kernel - console=hvc0 root=/dev/vda reboot=k panic=1 rw - /path/to/pflash - /path/to/OVMF_VARS - - - - - - /path/to/StratoVirt_binary_file - - - - - - - - - - - - - - - - 1000 - -
- - - - - - -
- - - - -
- - - - - - - - -
- - - - - /path/to/random_file -
- - - - -
- - - -``` - -#### ARM Configuration Example - -Configure a server named StratoVirt with 8 GiB memory, 1 GiB huge pages, four vCPUs. Its architecture is AArch64 and the mainboard type is virt. The configuration example of the corresponding XML file is as follows: - -```xml - - StratoVirt - 8 - - - - - - - 4 - - 1 - - hvm - /path/to/standard_vm_kernel - console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw - /path/to/pflash - - - - - - - /path/to/StratoVirt_binary_file - - - - - - -
- - 1000 - - - - - - - -
- - - - -
- - - - - - - - -
- - - - - /path/to/random_file -
- - - - -
- - - -``` - -## VM Management - -libvirt uses `virsh` commands to manage VMs. After the StratoVirt platform is interconnected with libvirt, only the following commands for interaction with StratoVirt are supported: - -- `create`: creates a VM. - -- `suspend`: suspends a VM. - -- `resume`: resumes a VM. - -- `destroy`: destroys a VM. - -- `console`: logs in to a VM through the console. - -- `define`: creates a VM base on the XML configuration. - -- `undefine`: deletes a defined VM. - -- `start`: starts a stopped VM. - -> ![](./public_sys-resources/icon-note.gif)**NOTE** -> -> The interface for interconnecting with StratoVirt has been implemented in libvirt. However, the commands for stopping and restarting the VM is not supported. Use `virsh -c stratovirt:///system 'command'` to call the stratovirt_driver interface of libvirt, and `virsh -c stratovirt:///system` to access the virsh CLI. In this case, stratovirt_driver is connected. - -### VM Lifecycle Management - -If you have created a VM configuration file named **StratoVirt** in st.xml format, you can use the following commands for VM lifecycle management: - -- Create a VM. - - ```shell - virsh create st.xml - /// - virsh -c stratovirt:///system create st.xml - ``` - - After the VM is created, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether a VM named **StratoVirt** in the **running** status exists. - -- Create a VM base on the XML configuration. - - ```shell - virsh define st.xml - /// - virsh -c stratovirt:///system create st.xml - ``` - - After the VM is created, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether a VM named **StratoVirt** in the **shut off** status exists. - -- Start a stopped VM. - - ```shell - virsh start StratoVirt - /// - virsh -c stratovirt:///system start StratoVirt - ``` - - After the VM is started, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether the status of VM **StratoVirt** is changed from **shut off** to **running**. - -- Suspend a VM. - - ```shell - virsh suspend StratoVirt - /// - virsh -c stratovirt:///system suspend StratoVirt - ``` - - After the VM is suspended, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether the status of VM **StratoVirt** is **paused**. - -- Resume a VM. - - ```shell - virsh resume StratoVirt - /// - virsh -c stratovirt:///system resume StratoVirt - ``` - - After the VM is resumed, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether the status of VM **StratoVirt** is **running**. - -- Destroy a VM. - - ```shell - virsh destroy StratoVirt - /// - virsh -c stratovirt:///system destroy StratoVirt - ``` - - After the VM is destroyed, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether VM **StratoVirt** exists. - -- Delete a defined VM - - ```shell - virsh undefine StratoVirt - /// - virsh -c stratovirt:///system undefine StratoVirt - ``` - - After the defined VM is deleted, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether VM **StratoVirt** in the **shut off** status exists. If VM **StratoVirt** is in the **running** status, after it is destroyed, it no longer enters the **shut off** status. - -### VM Login - -After the VM is created, you can run the `virsh console` or `virsh -c stratovirt:///system console` command to log in to it. If the VM name is **StratoVirt**, run the following command: - -``` -virsh console StratoVirt -/// -virsh -c stratovirt:///system console StratoVirt -``` - -> ![](./public_sys-resources/icon-note.gif)**NOTE** -> -> To use the `virsh console` command, set the redirection type of the console device to **pty** in the XML file. diff --git a/docs/en/docs/StratoVirt/Prepare_env.md b/docs/en/docs/StratoVirt/Prepare_env.md deleted file mode 100644 index 5b827dd29f00b1cef4fdb94d957bc1a975fca21f..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/Prepare_env.md +++ /dev/null @@ -1,172 +0,0 @@ -# Preparing the Environment - - -## Usage - -- StratoVirt can run on VMs with the x86_64 or AArch64 processor architecture. -- You are advised to compile, debug, and deploy StratoVirt on openEuler 22.03 LTS. -- StratoVirt can run with non-root permissions. - -## Environment Requirements - -The following are required in the environment for running StratoVirt: - -- /dev/vhost-vsock device (for implementing MMIO) -- nmap tool -- Kernel and rootfs images - - - -## Preparing Devices and Tools - -- To run StratoVirt, the MMIO device must be implemented. Therefore, before running StratoVirt, ensure that the **/dev/vhost-vsock** device exists. - - Check whether the device exists. - - ``` - $ ls /dev/vhost-vsock - /dev/vhost-vsock - ``` - - If the device does not exist, run the following command to generate it: - - ``` - $ modprobe vhost_vsock - ``` - - -- To use QMP commands, install the nmap tool first. After configuring the Yum source, run the following command to install the tool: - - ``` - # yum install nmap - ``` - -## Preparing Images - -### Creating the Kernel Image - -StratoVirt of the current version supports only the PE kernel image of the x86_64 and AArch64 platforms. The kernel image in PE format can be generated by using the following method: - -1. Run the following commands to obtain the kernel source code of openEuler: - - ``` - $ git clone https://gitee.com/openeuler/kernel.git - $ cd kernel - ``` - -2. Run the following command to check and switch to the kernel version openEuler-22.03-LTS: - - ``` - $ git checkout openEuler-22.03-LTS - ``` - -3. Configure and compile the Linux kernel. You are advised to use the [recommended configuration file](https://gitee.com/openeuler/stratovirt/tree/master/docs/kernel_config)). Copy the file to the kernel directory, rename it to **.config**, and run the `make olddefconfig` command to update to the latest default configuration (otherwise, you may need to manually select options for subsequent compilation). Alternatively, you can run the following command to configure the kernel as prompted. The system may display a message indicating that specific dependencies are missing. Run the `yum install` command to install the dependencies as prompted. - - ``` - $ make menuconfig - ``` - -4. Run the following command to create and convert the kernel image to the PE format. The converted image is **vmlinux.bin**. - - ``` - $ make -j vmlinux && objcopy -O binary vmlinux vmlinux.bin - ``` - -5. If you want to use the kernel in bzImzge format on the x86 platform, run the following command: - - ``` - $ make -j bzImage - ``` - - -## Creating the Rootfs Image - -The rootfs image is a file system image. When StratoVirt is started, the ext4 image with **init** can be loaded. To create an ext4 rootfs image, perform the following steps: - -1. Prepare a file with a proper size (for example, create a file with the size of 10 GB in **/home**). - - ``` - $ cd /home - $ dd if=/dev/zero of=./rootfs.ext4 bs=1G count=10 - ``` - -2. Create an empty ext4 file system on this file. - - ``` - $ mkfs.ext4 ./rootfs.ext4 - ``` - -3. Mount the file image. Create the **/mnt/rootfs** directory and mount **rootfs.ext4** to the directory as user **root**. - - ``` - $ mkdir /mnt/rootfs - # Return to the directory where the file system is created, for example, **/home**. - $ cd /home - $ sudo mount ./rootfs.ext4 /mnt/rootfs && cd /mnt/rootfs - ``` - -4. Obtain the latest alpine-mini rootfs of the corresponding processor architecture. - - - If the AArch64 processor architecture is used, you can get the latest rootfs from the [alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/). For example, alpine-minirootfs-3.16.0-aarch64.tar.gz, the reference commands are as follows: - - ``` - $ wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/aarch64/alpine-minirootfs-3.16.0-aarch64.tar.gz - $ tar -zxvf alpine-minirootfs-3.16.0-aarch64.tar.gz - $ rm alpine-minirootfs-3.16.0-aarch64.tar.gz - ``` - - - - If the x86_64 processor architecture is used, you can get the latest rootfs from the [alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/). For example, alpine-minirootfs-3.16.0-x86_64.tar.gz, the reference commands are as follows: - - ``` - $ wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/x86_64/alpine-minirootfs-3.16.0-x86_64.tar.gz - $ tar -zxvf alpine-minirootfs-3.16.0-x86_64.tar.gz - $ rm alpine-minirootfs-3.16.0-x86_64.tar.gz - ``` - - -5. Run the following commands to create a simple **/sbin/init** for the ext4 file image: - - ``` - $ rm sbin/init; touch sbin/init && cat > sbin/init < /sys/bus/pci/devices/0000:03:00.0/driver/unbind - ``` - - Finally bind the PCI device to the vfio-pci driver. - - ```shell - lspci -ns 0000:03:00.0 |awk -F':| ' '{print 5" "6}' > /sys/bus/pci/drivers/vfio-pci/new_id - ``` - - After the NIC is bound to the vfio-pci driver, the NIC information cannot be queried on the host. Only the PCI device information can be queried. - -### VFIO Device Passthrough - -#### Introduction - -The VFIO is a user-mode device driver solution provided by the kernel. The VFIO driver can securely present capabilities such as device I/O, interrupt, and DMA to user space. After StratoVirt uses the VFIO device passthrough solution, the I/O performance of VMs is greatly improved. - -#### Using VFIO Passthrough - -StratoVirt interconnects with libvirt to enable you to manage and configure VMs by modifying corresponding XML files. The following describes how to enable VFIO passthrough by modifying the XML file of a VM. - -**Step 1** Modify the XML file. - -(1) Run the following command on the host to query the CPU architecture information: - - ```shell - # uname -m - ``` - -(2) For the AArch64 and x86_64 architectures, [download](https://gitee.com/openeuler/stratovirt/tree/master/docs) the StratoVirt XML file **stratovirt_aarch64.xml** or **stratovirtvirt_x86.xml** and save it to any directory, for example, **/home**. - - ```shell - # cp stratovirt/docs/stratovirt_$arch.xml /home - ``` - -(3) Modify the VFIO configuration in the XML file based on the site requirements. **bus**, **slot**, and **function** specify the PCI device bound to the vfio-pci driver. The related configurations are as follows: - -```shell - - - - -
- - -``` - -In the preceding example, the device type is PCI, and **managed='yes'** indicates that libvirt unbinds the PCI device from the host and rebinds it to the vfio-pci driver. In the**source** item, the **domain**, **bus**, **slot**, and **function** of the VFIO passthrough device are configured. - -**Step 2** Create and log in to a VM using the libvirt command line. - -```shell -# virsh create stratovirt_$arch.xml -# virsh list --all -Id Name State --------------------- -1 StratoVirt running -# virsh console 1 -``` - -**Step 3** View and use the VFIO passthrough NIC on the VM. - -(1) Check the NIC information before configuration. - - ```shell - # ip a - 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - 2: enp1s0: mtu 1500 qdisc noop state DOWN group default qlen 1000 - link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff - ``` - - -(2) Dynamically configure the IP address of the NIC. - - ```shell - # dhclient - ``` - - -(3) Check whether the IP address is configured successfully. - - ```shell - # ip a - 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - 2: enp1s0: mtu 1500 qdisc mq state UP group default qlen 1000 - link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff - inet 192.168.1.3/16 brd 192.168.255.255 scope global dynamic enp1s0 - valid_lft 86453sec preferred_lft 86453sec - ``` - - The preceding command output indicates that the IP address 192.168.1.3 is successfully assigned and the VM can directly use the configured NIC. - - Note: If the passthrough NIC is not connected to a physical network, network information cannot be obtained. - -#### Unbinding the VFIO Driver - -To unbind a passthrough NIC from a VM, log in to the host and run the following command to bind the NIC to the host again.**hinic** indicates the NIC driver type. - -```shell -# echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind -# echo 0000:03:00.0 > /sys/bus/pci/drivers/hinic/bind -``` - -Note: Before binding a VFIO driver, you can run the **ethtool -i enp0** command on the host to obtain the NIC driver type.**enp0** indicates the name of the corresponding NIC. - -### SR-IOV Passthrough - -#### Introduction - -When VFIO passthrough is enabled, VMs can directly access hardware, but each device can be exclusively used by only one VM. The SR-IOV passthrough technology can virtualize a physical function (PF) into multiple virtual functions (VFs) and directly pass the VFs to different VMs. This technology increases the number of available devices. - -#### Procedure - -**Step 1** Create multiple VFs. - -The **sriov_numvfs** file is used to describe the count of VFs provided by SR-IOV and is stored in **/sys/bus/pci/devices/domain\:bus\:slot.function/**. For example, for the device whose bus ID is 03, slot ID is 00, and function ID is 0 in the preceding example, you can run the following command to create four VFs: - -```shell -# echo 4 > /sys/bus/pci/devices/0000\:03\:00.0/sriov_numvfs -``` - -**Step 2** Verify that the VFs are successfully created. - -```shell -# lspci -v | grep "Eth" | grep 1822 -``` - -If the following information is displayed, four VFs 03:00.1, 03:00.2, 03:00.3, and 03:00.4 are successfully created: - -```shell -03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) -03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) -03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) -03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) -03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) -``` - -**Step 3** All the created VFs can be passed to VMs. The method for using an SR-IOV device is the same as that for using a common PCI device. - diff --git a/docs/en/docs/StratoVirt/StratoVirt_guidence.md b/docs/en/docs/StratoVirt/StratoVirt_guidence.md deleted file mode 100644 index 461f0bf0490f0a18176972f10c4ea8f7edee1491..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/StratoVirt_guidence.md +++ /dev/null @@ -1,4 +0,0 @@ -# StratoVirt Virtualization User Guide - -This document describes Stratovirt virtualization, providing instructions on how to install Stratovirt based on openEuler and how to use Stratovirt virtualization. The purpose is to help users learn about Stratovirt and guide users and administrators to install and use StratoVirt. - diff --git a/docs/en/docs/StratoVirt/StratoVirt_introduction.md b/docs/en/docs/StratoVirt/StratoVirt_introduction.md deleted file mode 100644 index f7a5afcb498593eb133d09ef34b0813b606c0cdb..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/StratoVirt_introduction.md +++ /dev/null @@ -1,51 +0,0 @@ -# Introduction to StratoVirt - - -## Overview - -StratoVirt is an enterprise-class Virtual Machine Monitor (VMM) oriented to cloud data centers in the computing industry. It uses a unified architecture to support VM, container, and serverless scenarios. StratoVirt has competitive advantages in key technologies such as lightweight low noise, software and hardware synergy, and Rust language-level security. -StratoVirt reserves component-based assembling capabilities and APIs in the architecture design. Advanced features can be flexibly assembled as required until they evolve to support standard virtualization. In this way, StratoVirt can strike a balance between feature requirements, application scenarios, and flexibility. - - - -## Architecture Description - -The StratoVirt core architecture consists of three layers from top to bottom: - -- External API: compatible with the QEMU Monitor Protocol (QMP), has complete OCI compatibility capabilities, and supports interconnection with libvirt. -- BootLoader: discards the traditional BIOS+GRUB boot mode to achieve fast boot in lightweight scenarios, and provides UEFI boot support for standard VMs. -- Emulated mainboard: - - MicroVM: Fully utilizes software and hardware collaboration capabilities, simplifies device models, and provides low-latency resource scaling capabilities. - - Standard VM: implements UEFI boot with constructed ACPI tables. Virtio-pci and VFIO devices can be attached to greatly improve the VM I/O performance. - -Figure 1 shows the overall architecture. - -**Figure 1** Overall architecture of StratoVirt - -![](./figures/StratoVirt_architecture.jpg) - -## Features - -- Highly isolated based on hardware. -- Fast cold boot: Benefiting from the minimalist design, a microVM can be started within 50ms. -- Low memory overhead: StratoVirt works with a memory footprint of less than 4 MB. -- I/O enhancement: StratoVirt offers normal I/O capabilities and minimalist I/O device emulation. -- OCI compatibility: StratoVirt works with iSula and Kata containers, and can be integrated into the Kubernetes ecosystem perfectly. -- Multi-platform support: StratoVirt fully supports Intel and Arm platforms. -- Extensibility: StratoVirt has interface and design for importing more features, and can be extended to support standard virtualization. -- Security: fewer than 46 syscalls while running. - -## Implementation - -#### Running Architecture - -- A StratoVirt VM is an independent process in Linux. The process has three types of threads: main thread, vCPU thread, and I/O thread: - - The main thread is a cycle for asynchronously collecting and processing events from external modules, such as a vCPU thread. - - Each vCPU has a thread to handle trap events of this vCPU. - - I/O threads can be configured for I/O devices to improve I/O performance. - -## Restrictions - -- Only the Linux operating system is supported, and the recommended kernel versions are 4.19 and 5.10. -- Only Linux is supported as the operating system of the VM, and the recommended kernel versions are 4.19 and 5.10. -- A maximum of 254 CPUs are supported. diff --git a/docs/en/docs/StratoVirt/VM_configuration.md b/docs/en/docs/StratoVirt/VM_configuration.md deleted file mode 100644 index 72170ecfb81206f635507287db1746cbcdf6e040..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/VM_configuration.md +++ /dev/null @@ -1,688 +0,0 @@ -# Configuring VMs - -## Overview - -With StratoVirt, you can use command line parameters to specify VM configurations. Alternatively, you can interconnect StratoVirt with libvirt and use XML files to configure VMs. This chapter describes the command-line configuration mode. - -> ![](./public_sys-resources/icon-note.gif)**NOTE** -> -> In this document, **/path/to/socket** indicates the socket file in the user-defined path. -> -> In openEuler 21.09 and later versions, JSON files are not supported. - -## Specifications - -StratoVirt supports lightweight and standard VMs. - -- Lightweight VMs use the lightweight microVM mainboard and the MMIO bus. -- Standard VMs support standard startup. They use the Q35 mainboard on x86 platforms, and the virt mainboard and PCI bus on AArch64 platforms. - -### Lightweight VMs - -- Number of VM CPUs: [1, 254] -- VM memory size: [128 MiB, 512 GiB]. The default memory size is 256 MiB. -- Number of VM disks (including hot plugged-in disks): [0, 6] -- Number of VM NICs (including hot plugged-in NICs): [0, 2] -- The VM console device supports only single way connection. -- If the host CPU architecture is x86_64, a maximum of 11 MMIO devices can be configured. However, you are advised to configure a maximum of two other devices except disks and NICs. On the AArch64 platform, a maximum of 160 MMIO devices can be configured. You are advised to configure a maximum of 12 other devices except disks and NICs. - -### Standard VMs - -- Number of VM CPUs: [1, 254] -- VM memory size: [128 MiB, 512 GiB]. The default memory size is 256 MiB. -- The VM console device supports only single way connection. -- Only one console device is supported. -- A maximum of 32 PCI devices are supported. -- PCI bus to which the PCI device is mounted: slot ID [0, 32); function ID [0, 8). - -## Minimal Configuration - -The minimum configuration for running StratoVirt is as follows: - -- Use the Linux kernel image in PE or bzImage format (x86_64 only). -- Set the rootfs image as the virtio-blk device and add it to kernel parameters. -- Use QMP to control StratoVirt. -- To use a serial port for login, add one to the kernel startup command line. The standard model on the AArch64 platform is ttyAMA0, and the model used in other scenarios is ttyS0. - -## Configuration Description - -### **Command Format** - -The format of the command configured by running cmdline is as follows: - -**$ /path/to/stratovirt** *- [Parameter 1] [Option]-[Parameter 2] [Option]...* - -### **Usage Instructions** - -1. To ensure that the socket required by QMP can be created, run the following command to clear the environment: - - ``` - $rm [parameter] *[user-defined socket file path]* - ``` - -2. Run the cmdline command. - - ``` - $/path/to/stratovirt - *[Parameter 1] [Parameter option] - [Parameter 2] [Parameter option]*... - ``` - -### Basic Parameters - -The following table lists the basic configuration information. - -| Parameter| Option| Description| -| ---------------- | ----------------------------------------------- | ------------------------------------------------------------ | -| -name | *VMname* | Configures the VM name (a string of 1 to 255 characters).| -| -kernel | /path/to/vmlinux.bin| Configures the kernel image.| -| -append | console=ttyS0 root=/dev/vda reboot=k panic=1 rw | Configures the kernel command line parameter. For lightweight VMs, **console** is fixed at **ttyS0** (irrelevant to the platform architecture). For the standard x86_64 virtualization platform, **console** is default to **ttyS0**. For the AArch64 platform, **console** is default to **ttyAMA0**. If the virtio-console device is configured but the serial port device is not configured, set **console** to **hvc0** (irrelevant to the architecture).| -| -initrd | /path/to/initrd.img | Configures the initrd file.| -| -smp | [cpus=]n[,maxcpus=,sockets=,dies=,clusters=,cores=,threads=]| **cpus** specifies the number of CPUs with the value range of [1,254]. **maxcpus** specifies the maximum number of CPUs with the value range of [1,254]. **sockets**, **dies**, **clusters**, **cores**, and **threads** specifies the number of sockets, dies, clusters, cores, and threads respectively. The values of **sockets**, **cores**, and **threads**, if not specified, depend on the value of **maxcpus**. The values satisfy the following relationship: **maxcpus**=**sockets** x **dies** x **clusters** x **cores** x **threads**.| -| -m | Memory size (MiB/GiB). The default unit is MiB.| Configures the memory size. The value range is [128 MiB, 512 GiB]. The default memory size is 256 MiB.| -| -qmp | unix:/path/to/socket,server,nowait | Configures QMP. Before running QMP, ensure that the socket file does not exist.| -| -D | /path/to/logfile | Configures the log file.| -| -pidfile | /path/to/pidfile | Configures the pid file. This parameter must be used together with **-daemonize**. Ensure that the pid file does not exist before running the script.| -| -disable-seccomp | N/A| Disables Seccomp. Seccomp is enabled by default.| -| -daemonize | N/A| Enables daemon processes.| - -### VM Types - -You can run the **-machine** parameter to specify the type of the VM to be started. - -Parameters: - -- **type**: VM startup type. The value is **MicroVm** for lightweight virtualization, **q35** for standard virtualization on the x86_64 platform, and **virt** for standard virtualization on the AArch64 platform. -- **dump-guest-core** (optional): whether to dump the VM memory when a process panics. -- **mem-share** (optional): whether to share memory with other processes. - -### Disk Configuration - -VM disk configuration includes the following configuration items: - -- **drive_id**: disk ID. -- **path_on_host**: disk path. -- **serial_num** (optional): serial number of the disk. -- **read_only** (optional): whether the disk is read-only. -- **direct** (optional): whether to open the disk in O_DIRECT mode. -- **iothread** (optional): iothread attribute. -- **throttling.iops-total** (optional): disk QoS for limiting disk I/O operations. -- **if** (optional): driver type. The default value is **none**. The block device is **none**. -- **bus**: bus to which the device is to be mounted. -- **addr**: IDs of the slot and function to which the device is to be mounted. -- **multifunction** (optional): whether to enable PCI multi-function. -- **bootindex** (optional, only for standard machine): boot priority of the block device. If this parameter is not set, the priority is default to the lowest. The value ranges from 0 to 255. A smaller value indicates a higher priority. - -#### Disk Configuration Modes - -Disk configuration consists of two steps: driver configuration and block device configuration. - -The lightweight VM configuration format is as follows: - -``` --drive id=drive_id,file=path_on_host[,readonly=off][,direct=off][,throttling.iops-total=200][,if=none] --device virtio-blk-device,drive=drive_id[,iothread=iothread1][,serial=serial_num] -``` - -The standard VM configuration format is as follows: - -``` --drive id=drive_id,file=path_on_host[,readonly=off][,direct=off][,throttling.iops-total=200][,if=none] --device virtio-blk-pci,drive=drive_id,bus=pcie.0,addr=0x3.0x0[,iothread=iothread1,][serial=serial_num][,multifunction=on][,bootindex=1] -``` - -The following describes the **throttling.iops-total** and **iothread** configuration items: - -#### Disk QoS - -##### Introduction - -QoS is short for quality of service. In cloud scenarios, multiple VMs are started on a single host. Because the total disk access bandwidth of the host is limited, when a VM has heavy disk access pressure, it will occupy the access bandwidth of other VMs. As a result, the I/O performance of other VMs will be affected. To reduce the impact between VMs, you can configure QoS to limit the disk access rate of the VMs. - -##### Precautions - -- Currently, QoS supports the configuration of disk IOPS. -- The value range of IOPS is [0, 1000000]. The value **0** indicates that the IOPS is not limited. The actual IOPS does not exceed the preset value or the upper limit of the actual backend disk performance. -- Only the average IOPS can be limited. Instantaneous burst traffic cannot be limited. - -##### Configuration Methods - -Usage: - -**CLI** - -``` --drive xxx,throttling.iops-total=200 -``` - -Parameters: - -- **throttling.iops-total**: I/O delivery speed of the disk on a VM after IOPS is configured. It does not exceed the value of this parameter. -- *xxx*: other settings of the disk. - -#### **iothread** - -For details about the iothread configuration, see [iothread Configuration](#iothread-configuration). - -### NIC Configuration - -VM NIC configuration includes the following configuration items: - -- **idv**: unique device ID. -- **tap**: tap device. -- **ifname**: name of the tap device on the host. -- **mac** (optional): MAC address of the VM. -- **iothread** (optional): iothread attribute of the disk. For details about the iothread configuration of the NIC, see [iothread Configuration](#iothread-configuration). - -#### Configuration Methods - -> ![](./public_sys-resources/icon-note.gif)**NOTE**: -> -> Before using the network, run the following commands to configure the host bridge and tap device: -> -> ``` -> $ brctl addbr qbr0 -> $ ip tuntap add tap0 mode tap -> $ brctl addif qbr0 tap0 -> $ ifconfig qbr0 up; ifconfig tap0 up -> $ ifconfig qbr0 192.168.0.1 -> ``` - -1. Configure virtio-net. ([] indicates an optional parameter.) - -Lightweight VMs: - -``` --netdev tap,id=netdevid,ifname=host_dev_name[,vhostfd=2] --device virtio-net-device,netdev=netdevid,id=netid[,iothread=iothread1,mac=12:34:56:78:9A:BC] -``` - -Standard VMs: - -``` --netdev tap,id=netdevid,ifname=host_dev_name[,vhostfd=2] --device virtio-net-pci,netdev=netdevid,id=netid,bus=pcie.0,addr=0x2.0x0[,multifunction=on,iothread=iothread1,mac=12:34:56:78:9A:BC] -``` - -2. Configure vhost-net. - -Lightweight VMs: - -``` --netdev tap,id=netdevid,ifname=host_dev_name,vhost=on[,vhostfd=2] --device virtio-net-device,netdev=netdevid,id=netid[,iothread=iothread1,mac=12:34:56:78:9A:BC] -``` - -Standard VMs: - -``` --netdev tap,id=netdevid,ifname=host_dev_name,vhost=on[,vhostfd=2] --device virtio-net-pci,netdev=netdevid,id=netid,bus=pcie.0,addr=0x2.0x0[,multifunction=on,iothread=iothread1,mac=12:34:56:78:9A:BC] -``` - -### chardev Configuration - -Redirect I/Os from the Guest to chardev on the host. The chardev backend type can be **stdio**, **pty**, **socket**, or **file**.**file** can be set only during output. The configuration items are as follows: - -- **id**: unique device ID. -- **backend**: redirection type. -- **path**: path of the device redirection file. This parameter is required only for **socket** and **file** devices. -- **server**: uses chardev as a server. This parameter is required only for **socket** devices. -- **nowait**: The expected status is disconnected. This parameter is required only for **socket** devices. - -When chardev is used, a console file is created and used. Therefore, ensure that the console file does not exist before starting StratoVirt. - -#### Configuration Methods - -``` --chardev backend,id=chardev_id[,path=path,server,nowait] -``` - -### Serial Port Configuration - -A serial port is a VM device used to transmit data between hosts and VMs. To use a serial port, configure **console** to **ttyS0** in the kernel command line, and to **ttyAMA0** for standard startup on the AArch64 platform. The configuration items are as follows: - -- **chardev**: redirected chardev device. -- **backend**, **path**, **server**, and **nowait**: The meanings of these parameters are the same as those in **chardev**. - -#### Configuration Methods - -``` --serial chardev:chardev_id -``` - -Or: - -``` --chardev backend[,path=path,server,nowait] -``` - -### Console Device Configuration - -virtio-console is a universal serial port device used for data transmission between hosts and VMs. If only the console device is configured and I/O operations are performed through the console device, set **console** to **hvc0** in the kernel startup parameters. The console device has the following configuration items: - -- **id**: device ID. -- **path**: path of virtio console files. -- **socket**: redirection in socket mode. -- **chardev**: redirected chardev device. - -#### Configuration Methods - -The console configuration consists of three steps: specify virtio-serial, create a character device, and then create a virtconsole device. - -Lightweight VMs: - -``` --device virtio-serial-device[,id=virtio-serial0] --chardev socket,path=socket_path,id=virtioconsole1,server,nowait --device virtconsole,chardev=virtioconsole1,id=console_id -``` - -Standard VMs: - -``` --device virtio-serial-pci,bus=pcie.0,addr=0x1.0x0[,multifunction=on,id=virtio-serial0] --chardev socket,path=socket_path,id=virtioconsole1,server,nowait --device virtconsole,chardev=virtioconsole1,id=console_id -``` - -### vsock Device Configuration - -The vsock is also a device for communication between hosts and VMs. It is similar to the console but has better performance. The configuration items are as follows: - -- **id**: unique device ID. -- **guest_cid**: unique context ID. - -#### Configuration Methods - -Lightweight VMs: - -``` --device vhost-vsock-device,id=vsock_id,guest-cid=3 -``` - -Standard VMs: - -``` --device vhost-vsock-pci,id=vsock_id,guest-cid=3,bus=pcie.0,addr=0x1.0x0[,multifunction=on] -``` - -### Memory Huge Page Configuration - -#### Introduction - -StratoVirt supports the configuration of huge pages for VMs. Compared with the traditional 4 KB memory page mode, huge page memory can effectively reduce the number of TLB misses and page fault interrupts, significantly improving the performance of memory-intensive services. - -#### Precautions - -- The directory to which the huge pages are mounted must be an absolute path. -- Memory huge pages can be configured only during startup. -- Only static huge pages are supported. -- Configure huge pages on the host before use. -- To use the huge page feature, ensure that the VM memory size is an integer multiple of *huge page size*. - -#### Mutually Exclusive Features - -- If the huge page feature is configured, the balloon feature does not take effect. - -#### Configuration Methods - -##### Configuring Huge Pages on the Host - -###### Mounting - -Mount the huge page file system to a specified directory. `/path/to/hugepages` is the user-defined empty directory. - -``` -mount -t hugetlbfs hugetlbfs /path/to/hugepages -``` - -###### Setting the Number of Huge Pages - -- Set the number of static huge pages. `num` indicates the specified number. - - ``` - sysctl vm.nr_hugepages=num - ``` - -* Query huge page statistics. - - ``` - cat /proc/meminfo | grep Hugepages - ``` - - To view statistics about huge pages of other sizes, view the related information in the `/sys/kernel/mm/hugepages/hugepages-*/` directory. - -> ![](./public_sys-resources/icon-note.gif)**NOTE**: -> -> Configure the StratoVirt memory specifications and huge pages based on the huge page usage. If the huge page resources are insufficient, the VM fails to be started. - -#### Adding Huge Page Configuration When Starting StratoVirt - -- CLI - - ``` - -mem-path /page/to/hugepages - ``` - - In the preceding command, `/page/to/hugepages` indicates the directory to which the huge page file system is mounted. Only absolute paths are supported. - -
- -> ![](./public_sys-resources/icon-note.gif)**NOTE**: -> -> **Typical configuration**: Set **mem-path** in the StratoVirt command line to the *huge page file system mount directory*. The StratoVirt huge page feature is recommended for the typical configuration. - -### iothread Configuration - -#### Introduction - -After a VM with the iothread configuration is started on StratoVirt, threads independent of the main thread are started on the host. These independent threads can be used to process I/O requests of devices, improving the device I/O performance and reducing the impact on message processing on the management plane. - -#### Precautions - -- A maximum of eight iothreads can be configured. -- The iothread attribute can be configured for disks and NICs. -- iothreads occupy CPU resources of the host. When the I/O pressure is high in a VM, the CPU resources occupied by a single iothread depend on the disk access speed. For example, a common SATA disk occupies less than 20% CPU resources. - -#### Creating an iothread - -**CLI** - -```shell --object iothread,id=iothread1 -object iothread,id=iothread2 -``` - -Parameters: - -- **id**: identifies an iothread. This ID can be set to the iothread attribute of the disk or NIC. If iothread is configured in the startup parameter, the thread with the specified ID is started on the host after the VM is started. - -#### Configuring the iothread Attribute for a Disk or NIC - -**CLI-based configurations** - -Lightweight VMs: - -Disks - -``` --device virtio-blk-device xxx,iothread=iothread1 -``` - -NICs - -``` --device virtio-net-device xxx,iothread=iothread2 -``` - -Standard VMs: - -Disks - -``` --device virtio-blk-pci xxx,iothread=iothread1 -``` - -NICs - -``` --device virtio-net-pci xxx,iothread=iothread2 -``` - -``` -Parameters: -``` - -1. **iothread**: Set this parameter to the iothread ID, indicating the thread that processes the I/O of the device. -2. *xxx*: other configurations of the disk or NIC. - -### Balloon Device Configuration - -#### Introduction - -During running of a VM, the balloon driver in it occupies or releases memory to dynamically adjust the VM's available memory, achieving memory elasticity. - -#### Precautions - -- Before enabling balloon, ensure that the page size of the guest is the same as that of the host. -- The balloon feature must be enabled for the guest kernel. -- When memory elastic scaling is enabled, slight frame freezing may occur in the VM and the memory performance may deteriorate. - -#### Mutually Exclusive Features - -- This feature is mutually exclusive with huge page memory. -- In the x86 architecture, the number of interrupts is limited. Therefore, the total number of balloon devices and other virtio devices cannot exceed 11. By default, six block devices, two net devices, and one serial port device are used. - -#### Specifications - -- Each VM can be configured with only one balloon device. - -#### Configuration Methods - -Lightweight VMs: - -``` --device virtio-balloon-device[,deflate-on-oom=true|false][,free-page-reporting=true|false] -``` - -Standard VMs: - -``` --device virtio-balloon-pci,bus=pcie.0,addr=0x4.0x0[,deflate-on-oom=true|false][,free-page-reporting=true|false][,multifunction=on|off] -``` - -![](./public_sys-resources/icon-note.gif)**NOTE** - -1. The value of **deflate-on-oom** is of the Boolean type, indicating whether to enable the auto deflate feature. When this feature is enabled, if the balloon device has reclaimed some memory, it automatically releases the memory to the guest when the guest requires the memory. If this feature is disabled, the memory is not automatically returned. -2. The value of **free-page-reporting** is of the Boolean type, indicating whether to release free pages from kernel reporting. This feature can be used to reuse memory. -3. When running the QMP command to reclaim the VM memory, ensure that the VM has sufficient memory to keep basic running. Otherwise, some operations may time out and the VM cannot apply for idle memory. -4. If the huge page feature is enabled in the VM, the balloon device cannot reclaim the memory occupied by the huge pages. - -> If **deflate-on-oom** is set to **false**, when the guest memory is insufficient, the balloon device does not automatically release the memory. As a result, the guest OOM may occur, the processes may be killed, and even the VM cannot run properly. - -### RNG Configuration - -#### Introduction - -Virtio RNG is a paravirtualized random number generator that generates hardware random numbers for the guest. - -#### Configuration Methods - -Virtio RNG can be configured as the Virtio MMIO device or Virtio PCI device. To configure the Virtio RNG device as a Virtio MMIO device, run the following command: - -``` --object rng-random,id=objrng0,filename=/path/to/random_file --device virtio-rng-device,rng=objrng0,max-bytes=1234,period=1000 -``` - -To configure the Virtio RNG device as a Virtio PCI device, run the following command: - -``` --object rng-random,id=objrng0,filename=/path/to/random_file --device virtio-rng-pci,rng=objrng0,max-bytes=1234,period=1000,bus=pcie.0,addr=0x1.0x0,id=rng-id[,multifunction=on] -``` - -Parameters: - -- **filename**: path of the character device used to generate random numbers on the host, for example, **/dev/random**. -- **period**: period for limiting the read rate of random number characters, in milliseconds. -- **max-bytes**: maximum number of bytes of a random number generated by a character device within a period. -- **bus**: name of the bus to which the Virtio RNG device is mounted. -- **addr**: address of the Virtio RNG device. The parameter format is **addr=***[slot].[function]*, where *slot* and *function* indicate the slot number and function number of the device respectively. The slot number and function number are hexadecimal numbers. The function number of the Virtio RNG device is **0x0**. - -#### Precautions - -- If **period** and **max-bytes** are not configured, the read rate of random number characters is not limited. -- Otherwise, the value range of **max-bytes/period\*1000** is [64, 1000000000]. It is recommended that the value be not too small to prevent the rate of obtaining random number characters from being too slow. -- Only the average number of random number characters can be limited, and the burst traffic cannot be limited. -- If the guest needs to use the Virtio RNG device, the guest kernel requires the following configurations: **CONFIG_HW_RANDOM=y**, **CONFIG_HW_RANDOM_VIA=y**, and **CONFIG_HW_RANDOM_VIRTIO=y**. -- When configuring the Virtio RNG device, check whether the entropy pool is sufficient to avoid VM freezing. For example, if the character device path is **/dev/random**, you can check **/proc/sys/kernel/random/entropy_avail** to view the current entropy pool size. When the entropy pool is full, the entropy pool size is **4096**. Generally, the value is greater than 1000. - -### VNC Configuration - -#### Introduction - -You can log in to the VM through the VNC client and interact with the remote VM system through mouse and keyboard operations on the on VNC desktop. - -#### Precautions - -- Only standard VMs support VNC. -- Only clients that use RFB of version 3.3 to 3.8 can connect to VNC. -- Only one client can be connected at the same time. Follow-up clients connections will result in failure. -- The VNC feature supports only the ARM architecture. - -#### Mutually Exclusive Features - -- The VNC feature and live migration are mutually exclusive. - -#### Specifications - -- Only one VNC server can be configured on each VM. - -#### Configuration Methods - -Standard VMs: - -```shell --vnc 0.0.0.0:11 -``` - -![](./public_sys-resources/icon-note.gif)**NOTE** - -1. The Pixman library is required for rendering graphics. Install the **pixman.rpm** and **pixman-devel.rpm** packages of the Pixman library on the VM. -2. A USB controller, a mouse and a keyboard are required to perform mouse and keyboard operations. -3. A display device needs to be configured, such as virtio-gpu and ramfb. - -### USB Keyboard and Mouse Configuration - -#### Introduction - -StratoVirt supports USB keyboards and mice. You can remotely connect to the VM through VNC and operate in the VM GUI using a USB keyboard and mouse. Because USB devices are mounted to the USB controller, you need to configure a USB controller in the CLI in advance. - -#### Precautions - -- Only standard VMs support USB keyboards and mice. - -#### Mutually Exclusive Features - -- USB keyboards and mice and live migration are mutually exclusive. - -#### Specifications - -- Only one USB controller, one USB keyboard, and one USB mouse can be configured for each VM. - -#### Configuration Methods - -Add the following option to the StratoVirt startup command to configure the USB controller: - -``` --device nec-usb-xhci,id=xhci,bus=pcie.0,addr=0xa.0x0 -``` - -Parameters: - -- `id`: unique ID of the device -- `bus`: bus to mount the device -- `addr`: slot and function numbers for mounting the device - -The configured `bus` and `addr` values cannot conflict with other PCI devices. Otherwise, the VM will fail to start. - -Add the following option to the StratoVirt startup command to configure the USB keyboard: - -``` --device usb-bkd,id=kbd -``` - -Parameters: - -- `id`: unique ID of the device - -Add the following option to the StratoVirt startup command to configure the USB mouse: - -``` --device usb-tablet,id=tablet -``` - -Parameters: - -- `id`: unique ID of the device - -### virtio-gpu Device Configuration - -#### Introduction - -virtio-gpu devices can be configured for standard VMs for graphics display. - -#### Precautions - -- Only 2D graphics can be displayed. -- Recommended `max_hostmem` (usable memory on the host) is equal to or larger than 256MiB to ensure enough resolution. -- `max_outputs` (number of screens) cannot exceed 16. -- Live migration is not supported. - -#### Specifications - -- Only one virtio-gpu device can be configured for each VM. - -#### Configuration Methods - -Standard VMs: - -``` --device virtio-gpu-pci,id=XX,bus=pcie.0,addr=0x2.0x0[,max_outputs=XX][,edid=true|false][,xres=XX][,yres=XX][,max_hostmem=XX] -``` - -Parameters: - -- `max_outputs`: number of screens to be supported by the device. The recommended value is **1**, the maximum value is 16. -- `edid`: whether EDID is supported by the device. The recommended value is **true**. The VM kernel will verify whether the device supports EDID. -- `xres/yres`: horizontal/vertical size of the login window. -- `max_hostmem`: usable memory for the device on the host in bytes - -## Configuration Examples - -### Lightweight VMs - -This section provides an example of the minimum configuration for creating a lightweight VM. - -1. Log in to the host and delete the socket file to ensure that the QMP can be created. - - ``` - rm -f /tmp/stratovirt.socket - ``` - -2. Run StratoVirt. - - ``` - $ /path/to/stratovirt \ - -kernel /path/to/vmlinux.bin \ - -append console=ttyS0 root=/dev/vda rw reboot=k panic=1 \ - -drive file=/home/rootfs.ext4,id=rootfs,readonly=false \ - -device virtio-blk-device,drive=rootfs \ - -qmp unix:/tmp/stratovirt.socket,server,nowait \ - -serial stdio - ``` - - After the running is successful, the VM is created and started based on the specified configuration parameters. - -### Standard VMs - -This section provides an example of the minimum configuration for creating a standard VM on the ARM platform. - -1. Delete the socket file to ensure that QMP can be created. - - ``` - rm -f /tmp/stratovirt.socket - ``` - -2. Run StratoVirt. - - ``` - $ /path/to/stratovirt \ - -kernel /path/to/vmlinux.bin \ - -append console=ttyAMA0 root=/dev/vda rw reboot=k panic=1 \ - -drive file=/path/to/edk2/code_storage_file,if=pflash,unit=0[,readonly=true] \ - -drive file=/path/to/edk2/data_storage_file,if=pflash,unit=1, \ - -drive file=/home/rootfs.ext4,id=rootfs,readonly=false \ - -device virtio-blk-device,drive=rootfs,bus=pcie.0,addr=0x1 \ - -qmp unix:/tmp/stratovirt.socket,server,nowait \ - -serial stdio - ``` diff --git a/docs/en/docs/StratoVirt/VM_management.md b/docs/en/docs/StratoVirt/VM_management.md deleted file mode 100644 index 55bcdb7893202a949605b9fc38399b7cc59a6a94..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/VM_management.md +++ /dev/null @@ -1,738 +0,0 @@ -# Managing VMs - - -## Overview - -StratoVirt allows you to query VM information and manage VM resources and lifecycle with QMP. To query the information about a VM, connect to the VM first. - -## Querying VM Information - -### Introduction - -StratoVirt can be used to query the VM status, vCPU topology, and vCPU online status. - -### Querying VM Status - -Run the **query-status** command to query the running status of a VM. - -- Usage: - - **{ "execute": "query-status" }** - -- Example: - -``` -<- { "execute": "query-status" } --> { "return": { "running": true,"singlestep": false,"status": "running" } -``` - - - -### Querying Topology Information - -Run the **query-cpus** command to query the topologies of all CPUs. - -- Usage: - - **{ "execute": "query-cpus" }** - -- Example: - -``` -<- { "execute": "query-cpus" } --> {"return":[{"CPU":0,"arch":"x86","current":true,"halted":false,"props":{"core-id":0,"socket-id":0,"thread-id":0},"qom_path":"/machine/unattached/device[0]","thread_id":8439},{"CPU":1,"arch":"x86","current":true,"halted":false,"props":{"core-id":0,"socket-id":1,"thread-id":0},"qom_path":"/machine/unattached/device[1]","thread_id":8440}]} -``` - -### Querying vCPU Online Status - -Run the **query-hotpluggable-cpus** command to query the online/offline statuses of all vCPUs. - -- Usage: -**{ "execute": "query-hotpluggable-cpus" }** - -- Example: - -``` -<- { "execute": "query-hotpluggable-cpus" } --> {"return":[{"props":{"core-id":0,"socket-id":0,"thread-id":0},"qom-path":"/machine/unattached/device[0]","type":"host-x86-cpu","vcpus-count":1},{"props":{"core-id":0,"socket-id":1,"thread-id":0},"qom-path":"/machine/unattached/device[1]","type":"host-x86-cpu","vcpus-count":1}]} -``` - -Online vCPUs have the `qom-path` item, while offline vCPUs do not. - -## Managing VM Lifecycle - -### Introduction - -StratoVirt can manage the lifecycle of a VM, including starting, stopping, resuming, and exiting the VM. - -### Creating and Starting a VM - -Use the command line parameters to specify the VM configuration, and create and start a VM. - -- When using the command line parameters to specify the VM configuration, run the following command to create and start the VM: - -``` -$/path/to/stratovirt - *[Parameter 1] [Parameter option] - [Parameter 2] [Parameter option]*... -``` - - -> ![](./public_sys-resources/icon-note.gif) -> -> After the lightweight VM is started, there are two NICs: eth0 and eth1. The two NICs are reserved for hot plugging: eth0 first and then eth1. Currently, only two virtio-net NICs can be hot plugged. - -### Connecting to a VM - -StratoVirt uses QMP to manage VMs. To stop, resume, or exit a VM, connect it the StratoVirt through QMP first. - -Open a new CLI (CLI B) on the host and run the following command to connect to the api-channel as the **root** user: - -``` -# ncat -U /path/to/socket -``` - -After the connection is set up, you will receive a greeting message from StratoVirt, as shown in the following: - -``` -{"QMP":{"version":{"qemu":{"micro":1,"minor":0,"major":4},"package":""},"capabilities":[]}} -``` - -You can now manage the VM by entering the QMP commands in CLI B. - -> ![](./public_sys-resources/icon-note.gif) -> -> QMP provides **stop**, **cont**, **quit**, and **query-status** commands to manage and query VM statuses. -> -> All QMP commands for managing VMs are entered in CLI B. `<-` indicates the command input, and `->` indicates the QMP returned result. - -### Stopping a VM - -QMP provides the **stop** command to stop a VM, that is, to stop all vCPUs of the VM. The command syntax is as follows: - -**{"execute":"stop"}** - -**Example:** - -The **stop** command and the command output are as follows: - -``` -<- {"execute":"stop"} --> {"event":"STOP","data":{},"timestamp":{"seconds":1583908726,"microseconds":162739}} --> {"return":{}} -``` - -### Resuming a VM - -QMP provides the **cont** command to resume a stopped VM, that is, to resume all vCPUs of the VM. The command syntax is as follows: - -**{"execute":"cont"}** - -**Example:** - -The **cont** command and the command output are as follows: - -``` -<- {"execute":"cont"} --> {"event":"RESUME","data":{},"timestamp":{"seconds":1583908853,"microseconds":411394}} --> {"return":{}} -``` - -### Exiting a VM - -QMP provides the **quit** command to exit a VM, that is, to exit the StratoVirt process. The command syntax is as follows: - -**{"execute":"quit"}** - -**Example:** - -``` -<- {"execute":"quit"} --> {"return":{}} --> {"event":"SHUTDOWN","data":{"guest":false,"reason":"host-qmp-quit"},"timestamp":{"ds":1590563776,"microseconds":519808}} -``` - -## Managing VM Resources - -### Hot-Pluggable Disks - -StratoVirt allows you to adjust the number of disks when a VM is running. That is, you can add or delete VM disks without interrupting services. - -**Note** - -* For a standard VM, the **CONFIG_HOTPLUG_PCI_PCIE=y** configuration must be enabled for the VM kernel. - -* For a standard VM, devices can be hot added to the root port. The root port device must be configured before the VM is started. - -* You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner. - -#### Hot Adding Disks - -**Usage:** - -Lightweight VM: - -``` -{"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} -{"execute": "device_add", "arguments": {"id": "drive-0", "driver": "virtio-blk-mmio", "addr": "0x1"}} -``` - -Standard VM: - -``` -{"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} -{"execute":"device_add", "arguments":{"id":"drive-0", "driver":"virtio-blk-pci", "drive": "drive-0", "addr":"0x0", "bus": "pcie.1"}} -``` - -**Parameters:** - - -- For a lightweight VM, the value of **node-name** in **blockdev-add** must be the same as that of **id** in **device_add**. For example, the values of **node-name** and **id** are both **drive-0** as shown above. - -- For a standard VM, the value of **drive** must be the same as that of **node-name** in **blockdev-add**. - -- **/path/to/block** is the image path of the hot added disks. It cannot be the path of the disk image that boots the rootfs. - -- For a lightweight VM, the value of **addr**, starting from **0x0**, is mapped to a virtio device on the VM. **0x0** is mapped to **vda**, **0x1** is mapped to **vdb**, and so on. To be compatible with the QMP protocol, **addr** can be replaced by **lun**, but **lun=0** is mapped to the **vdb** of the guest machine. For a standard VM, the value of **addr** must be **0x0**. - -- For a standard VM, **bus** indicates the name of the bus to mount the device. Currently, the device can be hot added only to the root port device. The value of **bus** must be the ID of the root port device. - -- For a lightweight VM, StratoVirt supports a maximum of six virtio-blk disks. Note this when hot adding disks. For a standard VM, the maximum number of hot added disks depends on the number of root port devices. - -**Example:** - -Lightweight VM: - -``` -<- {"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} --> {"return": {}} -<- {"execute": "device_add", "arguments": {"id": "drive-0", "driver": "virtio-blk-mmio", "addr": "0x1"}} --> {"return": {}} -``` - -Standard VM: - -``` -<- {"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} --> {"return": {}} -<- {"execute":"device_add", "arguments":{"id":"drive-0", "driver":"virtio-blk-pci", "drive": "drive-0", "addr":"0x0", "bus": "pcie.1"}} --> {"return": {}} -``` -#### Hot Removing Disks - -**Usage:** - -Lightweight VM: - -``` -{"execute": "device_del", "arguments": {"id":"drive-0"}} -``` - -Standard VM: - -``` -{"execute": "device_del", "arguments": {"id":"drive-0"}} -{"execute": "blockdev-del", "arguments": {"node-name": "drive-0"}} -``` - -**Parameters:** - -**id** indicates the ID of the disk to be hot removed. -- **node-name** indicates the backend name of the disk. - -**Example:** - -Lightweight VM: - -``` -<- {"execute": "device_del", "arguments": {"id": "drive-0"}} --> {"event":"DEVICE_DELETED","data":{"device":"drive-0","path":"drive-0"},"timestamp":{"seconds":1598513162,"microseconds":367129}} --> {"return": {}} -``` - -Standard VM: - -``` -<- {"execute": "device_del", "arguments": {"id":"drive-0"}} --> {"return": {}} --> {"event":"DEVICE_DELETED","data":{"device":"drive-0","path":"drive-0"},"timestamp":{"seconds":1598513162,"microseconds":367129}} -<- {"execute": "blockdev-del", "arguments": {"node-name": "drive-0"}} --> {"return": {}} -``` - -A **DEVICE_DELETED** event indicates that the device is removed from StratoVirt. -### Hot-Pluggable NICs - -StratoVirt allows you to adjust the number of NICs when a VM is running. That is, you can add or delete VM NICs without interrupting services. - -**Note** - -* For a standard VM, the **CONFIG_HOTPLUG_PCI_PCIE=y** configuration must be enabled for the VM kernel. - -* For a standard VM, devices can be hot added to the root port. The root port device must be configured before the VM is started. - -* You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner. - -#### Hot Adding NICs - -**Preparations (Requiring the root Permission)** - -1. Create and enable a Linux bridge. For example, if the bridge name is **qbr0**, run the following command: - -```shell -# brctl addbr qbr0 -# ifconfig qbr0 up -``` - -2. Create and enable a tap device. For example, if the tap device name is **tap0**, run the following command: - -```shell -# ip tuntap add tap0 mode tap -# ifconfig tap0 up -``` - -3. Add the tap device to the bridge. - -```shell -# brctl addif qbr0 tap0 -``` - -**Usage:** - -Lightweight VM: - -``` -{"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} -{"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-mmio", "addr":"0x0"}} -``` - -Standard VM: - -``` -{"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} -{"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-pci", "addr":"0x0", "netdev": "net-0", "bus": "pcie.1"}} -``` - -**Parameters:** - -- For a lightweight VM, **id** in **netdev_add** must be the same as that in **device_add**. **ifname** is the name of the backend tap device. - -- For a standard VM, the value of **netdev** must be the value of **id** in **netdev_add**. - -- For a lightweight VM, the value of **addr**, starting from **0x0**, is mapped to an NIC on the VM. **0x0** is mapped to **eth0 **, **0x1** is mapped to **eth1**. For a standard VM, the value of **addr** must be **0x0**. - -- For a standard VM, **bus** indicates the name of the bus to mount the device. Currently, the device can be hot added only to the root port device. The value of **bus** must be the ID of the root port device. - -- For a lightweight VM, StratoVirt supports a maximum of two virtio-net NICs. Therefore, pay attention to the specification restrictions when hot adding in NICs. For a standard VM, the maximum number of hot added disks depends on the number of root port devices. - -**Example:** - -Lightweight VM: - -``` -<- {"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} --> {"return": {}} -<- {"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-mmio", "addr":"0x0"}} --> {"return": {}} -``` - -**addr:0x0** corresponds to **eth0** in the VM. - -Standard VM: - -``` -<- {"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} --> {"return": {}} -<- {"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-pci", "addr":"0x0", "netdev": "net-0", "bus": "pcie.1"}} --> {"return": {}} -``` - -#### Hot Removing NICs - -**Usage:** - -Lightweight VM: - -``` -{"execute": "device_del", "arguments": {"id": "net-0"}} -``` - -Standard VM: - -``` -{"execute": "device_del", "arguments": {"id":"net-0"}} -{"execute": "netdev_del", "arguments": {"id": "net-0"}} -``` - - -**Parameters:** - -**id**: NIC ID, for example, **net-0**. - -- **id** in **netdev_del** indicates the backend name of the NIC. - -**Example:** - -Lightweight VM: - -``` -<- {"execute": "device_del", "arguments": {"id": "net-0"}} --> {"event":"DEVICE_DELETED","data":{"device":"net-0","path":"net-0"},"timestamp":{"seconds":1598513339,"microseconds":97310}} --> {"return": {}} -``` - -Standard VM: - -``` -<- {"execute": "device_del", "arguments": {"id":"net-0"}} --> {"return": {}} --> {"event":"DEVICE_DELETED","data":{"device":"net-0","path":"net-0"},"timestamp":{"seconds":1598513339,"microseconds":97310}} -<- {"execute": "netdev_del", "arguments": {"id": "net-0"}} --> {"return": {}} -``` - -A **DEVICE_DELETED** event indicates that the device is removed from StratoVirt. - -### Hot-swappable Pass-through Devices - -You can add or delete the passthrough devices of a StratoVirt standard VM when it is running. - -**Note** - -* The **CONFIG_HOTPLUG_PCI_PCIE=y** configuration must be enabled for the VM kernel. - -* Devices can be hot added to the root port. The root port device must be configured before the VM is started. - -* You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner. - -#### Hot Adding Pass-through Devices - -**Usage:** - -``` -{"execute":"device_add", "arguments":{"id":"vfio-0", "driver":"vfio-pci", "bus": "pcie.1", "addr":"0x0", "host": "0000:1a:00.3"}} -``` - -**Parameters:** - -- **id** indicates the ID of the hot added device. - -- **bus** indicates the name of the bus to mount the device. - -- **addr** indicates the slot and function numbers to mount the device. Currently, **addr** must be set to **0x0**. - -- **host** indicates the domain number, bus number, slot number, and function number of the passthrough device on the host machine. - -**Example:** - -``` -<- {"execute":"device_add", "arguments":{"id":"vfio-0", "driver":"vfio-pci", "bus": "pcie.1", "addr":"0x0", "host": "0000:1a:00.3"}} --> {"return": {}} -``` - -#### Hot Removing Pass-through Devices - -**Usage:** - -``` -{"execute": "device_del", "arguments": {"id": "vfio-0"}} -``` - -**Parameters:** - -- **id** indicates the ID of the device to be hot removed, which is specified when the device is hot added. - -**Example:** - -``` -<- {"execute": "device_del", "arguments": {"id": "vfio-0"}} --> {"return": {}} --> {"event":"DEVICE_DELETED","data":{"device":"vfio-0","path":"vfio-0"},"timestamp":{"seconds":1614310541,"microseconds":554250}} -``` - -A **DEVICE_DELETED** event indicates that the device is removed from StratoVirt. - -## Using Ballon Devices - -The balloon device is used to reclaim idle memory from a VM. It called by running the QMP command. - -**Usage:** - -``` -{"execute": "balloon", "arguments": {"value": 2147483648‬}} -``` - -**Parameters:** - -- **value**: size of the guest memory to be set. The unit is byte. If the value is greater than the memory value configured during VM startup, the latter is used. - -**Example:** - -The memory size configured during VM startup is 4 GiB. If the idle memory of the VM queried by running the free command is greater than 2 GiB, you can run the QMP command to set the guest memory size to 2147483648 bytes. - -``` -<- {"execute": "balloon", "arguments": {"value": 2147483648‬}} --> {"return": {}} -``` - -Query the actual memory of the VM: - -``` -<- {"execute": "query-balloon"} --> {"return":{"actual":2147483648}} -``` - -## Using VM Memory Snapshots - -### Introduction - -A VM memory snapshot stores the device status and memory information of a VM in a snapshot file. If the VM is damaged, you can use the snapshot to restore it to the time when the snapshot was created, improving system reliability. - -StratoVirt allows you to create snapshots for stopped VMs and create VMs in batches with a snapshot file as the VM template. As long as a snapshot is created after a VM is started and enters the user mode, the quick startup can skip the kernel startup and user-mode service initialization phases and complete the VM startup in milliseconds. - -### Mutually Exclusive Features - -Memory snapshots cannot be created or used for VMs that are configured with the following devices or use the following features: - -- vhost-net device -- VFIO passthrough device -- Balloon device -- Huge page memory feature -- mem-shared feature -- memory backend file **mem-path** - -### Creating a Snapshot - -For StratoVirt VMs, perform the following steps to create a storage snapshot: - -1. Create and start a VM. - -2. Run the QMP command on the host to stop the VM. - - ```shell - <- {"execute":"stop"} - -> {"event":"STOP","data":{},"timestamp":{"seconds":1583908726,"microseconds":162739}} - -> {"return":{}} - - ``` - -3. Confirm that the VM is stopped. - - ```shell - <- {"execute":"query-status"} - -> {"return":{"running":true,"singlestep":false,"status":"paused"}} - - ``` - -4. Run the following QMP command to create a VM snapshot in a specified absolute path, for example, **/path/to/template**: - - ```shell - <- {"execute":"migrate", "arguments":{"uri":"file:/path/to/template"}} - -> {"return":{}} - - ``` - -5. Check whether the snapshot is successfully created. - - ```shell - <- {"execute":"query-migrate"} - - ``` - - If "{"return":{"status":"completed"}}" is displayed, the snapshot is successfully created. - - If the snapshot is created successfully, the `memory` and `state` directories are generated in the specified path **/path/to/template**. The `state` file contains VM device status information, and the `memory` file contains VM memory data. The size of the `memory` file is close to the configured VM memory size. - -### Querying Snapshot Status - -There are five statuses in the snapshot process. - -- `None`: The snapshot resource is not ready. -- `Setup`: The snapshot resource is ready. You can create a snapshot. -- `Active`: The snapshot is being created. -- `Completed`: The snapshot is created successfully. -- `Failed`: The snapshot fails to be created. - -You can run the `query-migrate` QMP command on the host to query the status of the current snapshot. For example, if the VM snapshot is created successfully, the following output is displayed: - -```shell -<- {"execute":"query-migrate"} --> {"return":{"status":"completed"}} - -``` - -### Restoring a VM - -#### Precautions - -- The following models support the snapshot and boot from snapshot features: - - microvm - - Q35 (x86_64) - - virt (AArch64) -- When a snapshot is used for restoration, the configured devices must be the same as those used when the snapshot is created. -- If a microVM is used and the disk/NIC hot plugging-in feature is enabled before the snapshot is taken, you need to configure the hot plugged-in disks or NICs in the startup command line during restoration. - -#### Restoring a VM from a Snapshot File - -**Command Format** - -```shell -stratovirt -incoming URI -``` - -**Parameters** - -**URI**: snapshot path. The current version supports only the `file` type, followed by the absolute path of the snapshot file. - -**Example** - -Assume that the VM used for creating a snapshot is created by running the following command: - -```shell -$ stratovirt \ - -machine microvm \ - -kernel /path/to/kernel \ - -smp 1 -m 1024 \ - -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ - -drive file=/path/to/rootfs,id=rootfs,readonly=off,direct=off \ - -device virtio-blk-device,drive=rootfs \ - -qmp unix:/path/to/socket,server,nowait \ - -serial stdio - -``` - -Then, the command for restoring the VM from the snapshot (assume that the snapshot storage path is **/path/to/template**) is as follows: - -```shell -$ stratovirt \ - -machine microvm \ - -kernel /path/to/kernel \ - -smp 1 -m 1024 \ - -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ - -drive file=/path/to/rootfs,id=rootfs,readonly=off,direct=off \ - -device virtio-blk-device,drive=rootfs \ - -qmp unix:/path/to/another_socket,server,nowait \ - -serial stdio \ - -incoming file:/path/to/template - -``` - -## VM Live Migration - -### Introduction - -StratoVirt provides the VM live migration capability, that is, migrating a VM from one server to another without interrupting VM services. - -VM live migration can be used in the following scenarios: -- When a server is overloaded, the VM live migration technology can be used to migrate VMs to another physical server for load balancing. -- When a server needs maintenance, VMs on the server can be migrated to another physical server without interrupting services. -- When a server is faulty and hardware needs to be replaced or the networking needs to be adjusted, VMs on the server can be migrated to another physical machine to prevent VM service interruption. - - -### Live Migration Operations - -This section describes how to live migrate a VM. - -#### Preparing for Live Migration - -1. Log in to the host where the source VM is located as the **root** user and run the following command to start the source VM. Modify the parameters as required: - -```shell -./stratovirt \ - -machine q35 \ - -kernel ./vmlinux.bin \ - -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ - -drive file=path/to/rootfs,id=rootfs,readonly=off,direct=off \ - -device virtio-blk-pci,drive=rootfs,id=rootfs,bus=pcie.0,addr=0 \ - -qmp unix:path/to/socket1,server,nowait \ - -serial stdio \ -``` - -2. Log in to the host where the target VM is located as the **root** user and run the following command to start the target VM. The parameters must be consistent with those of the source VM: - -```shell -./stratovirt \ - -machine q35 \ - -kernel ./vmlinux.bin \ - -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ - -drive file=path/to/rootfs,id=rootfs,readonly=off,direct=off \ - -device virtio-blk-pci,drive=rootfs,id=rootfs,bus=pcie.0,addr=0 \ - -qmp unix:path/to/socket2,server,nowait \ - -serial stdio \ - -incoming tcp:192.168.0.1:4446 \ -``` - -> ![](public_sys-resources/icon-note.gif) **NOTE:** -> -> - The parameters for starting the target VM must be consistent with those for starting the source VM: -> - To change the data transmission mode for live migration from TCP to the UNIX socket protocol, change the `-incoming tcp:192.168.0.1:4446` parameter for starting the target VM to `-incoming unix:/tmp/stratovirt-migrate.socket`. However, the UNIX socket protocol supports only live migration between different VMs on the same physical host. - -#### Starting Live Migration - -On the host where the source VM is located, run the following command to start the VM live migration task: - -```shell -$ ncat -U path/to/socket1 --> {"QMP":{"version":{"StratoVirt":{"micro":1,"minor":0,"major":0},"package":""},"capabilities":[]}} -<- {"execute":"migrate", "arguments":{"uri":"tcp:192.168.0.1:4446"}} --> {"return":{}} -``` -> ![](public_sys-resources/icon-note.gif) **NOTE:** -> -> If the UNIX socket protocol is used for live migration transmission, change `"uri":"tcp:192.168.0.1:4446"` in the command to `"uri":"unix:/tmp/stratovirt-migrate.socket"`. - -#### Finishing Live Migration - -After the `QMP` command is executed, the VM live migration task starts. If no live migration error log is generated, the source VM is migrated to the target VM, and the source VM is automatically destroyed. - -#### Canceling Live Migration - -During live migration, the migration may take a long time or the load of the host where the target VM is located changes. In this case, you need to adjust the migration policy. StratoVirt provides the feature of canceling live migration. - -To cancel live migration, log in to the host where the source VM is located and run the following `QMP` command: - -```shell -$ ncat -U path/to/socket1 --> {"QMP":{"version":{"StratoVirt":{"micro":1,"minor":0,"major":0},"package":""},"capabilities":[]}} -<- {"execute":"migrate_cancel"} --> {"return":{}} -``` - -If the live migration task on the target VM stops and the log indicates that the live migration is canceled, the live migration task is canceled successfully. - -#### Querying the Live Migration Status - -A live migration task can be in the following statuses: - -- **None**: Resources, such as vCPUs, memory, and devices, are not ready for live migration. -- **Setup**: Live migration resources have been prepared and live migration can be performed. -- **Active**: The VM is being live migrated. -- **Completed**: Live migration is complete. -- **Failed**: Live migration fails. - -The following `QMP` command queries completed live migration tasks: - -```shell -$ ncat -U path/to/socket --> {"QMP":{"version":{"StratoVirt":{"micro":1,"minor":0,"major":0},"package":""},"capabilities":[]}} -<- {"execute":"query-migrate"} --> {"return":{"status":"completed"}} -``` - -### Constraints - -StratoVirt supports live migration of the following standard VM boards: -- Q35 (x86_64) -- virt (AArch64) - -The following devices and features do not support live migration: -- vhost-net device -- vhost-user-net device -- virtio balloon device -- vfio device -- Shared backend storage -- Shared memory (back-end memory feature) - -The following command parameters for starting the source and target VMs must be the same: -- virtio-net: MAC address -- device: BDF number -- smp -- m \ No newline at end of file diff --git a/docs/en/docs/StratoVirt/figures/StratoVirt_architecture.jpg b/docs/en/docs/StratoVirt/figures/StratoVirt_architecture.jpg deleted file mode 100644 index 93f1697131dd2a6f8c010def9f42ad067b9b0bd9..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/StratoVirt/figures/StratoVirt_architecture.jpg and /dev/null differ diff --git a/docs/en/docs/StratoVirt/interconnect_isula.md b/docs/en/docs/StratoVirt/interconnect_isula.md deleted file mode 100644 index baa77be73278f76cbd80117a3228f012bf3fdb39..0000000000000000000000000000000000000000 --- a/docs/en/docs/StratoVirt/interconnect_isula.md +++ /dev/null @@ -1,223 +0,0 @@ -# Connecting to the iSula Secure Container - -## Overview - -To provide a better isolation environment for containers and improve system security, you can interconnect StratoVirt with iSula secure containers. - -## Connecting to the iSula Secure Container - -### Prerequisites - -iSulad and kata-containers have been installed, and iSulad supports the containerd-kata-shim-v2 container runtime and devicemapper storage driver. - -The following describes how to install and configure iSulad and kata-containers. - -1. Configure the Yum source and install iSulad and kata-containers as the **root** user. - - ```shell - # yum install iSulad - # yum install kata-containers - ``` - -2. Create and configure a storage device. - - You need to plan the drive, for example, **/dev/sdxx**, which will be formatted. - - ```shell - # pvcreate /dev/sdxx - # vgcreate isulaVG0 /dev/sdxx - # lvcreate --wipesignatures y -n thinpool isulaVG0 -l 95%VG - # lvcreate --wipesignatures y -n thinpoolmeta isulaVG0 -l 1%VG - # lvconvert -y --zero n -c 512K --thinpool isulaVG0/thinpool --poolmetadata isulaVG0/thinpoolmeta - ``` - - Add the following information to the **/etc/lvm/profile/isulaVG0-thinpool.profile** configuration file: - - ``` - activation { - thin_pool_autoextend_threshold=80 - thin_pool_autoextend_percent=20 - } - ``` - - Modify **storage-driver** and **storage-opts** in the **/etc/isulad/daemon.json** configuration file as follows. Set the default storage driver type **overlay** to **devicemapper**. - - ``` - "storage-driver": "devicemapper", - "storage-opts": [ - "dm.thinpooldev=/dev/mapper/isulaVG0-thinpool", - "dm.fs=ext4", - "dm.min_free_space=10%" - ], - ``` - -3. Restart **isulad**. - - ```shell - # systemctl daemon-reload - # systemctl restart isulad - ``` - -4. Check whether the iSula storage driver is successfully configured. - - ```shell - # isula info - ``` - - If the following information is displayed, the configuration is successful: - - ``` - Storage Driver: devicemapper - ``` - - - -### Interconnection Guide - -This section describes how to interconnect StratoVirt with kata-containers to access the iSula container ecosystem. - -#### Connecting to a Lightweight VM - -1. Modify the kata configuration file. Its default path is **/usr/share/defaults/kata-containers/configuration.toml**. You can also configure the file by referring to **configuration-stratovirt.toml** in the same directory. Modify the **hypervisor** type of the secure container to **stratovirt**, **kernel** to the absolute path of the kernel image of kata-containers, and **initrd** to the **initrd** image file of kata-containers. (If you use Yum to install kata-containers, the two image files are downloaded and stored in the **/var/lib/kata/** directory by default. You can also use other images during the configuration.) - - The modified configurations are as follows: - - ```shell - [hypervisor.stratovirt] - path = "/usr/bin/stratovirt" - kernel = "/var/lib/kata/kernel" - initrd = "/var/lib/kata/kata-containers-initrd.img" - machine_type = "microvm" - block_device_driver = "virtio-mmio" - use_vsock = true - enable_netmon = true - internetworking_model="tcfilter" - sandbox_cgroup_with_emulator = false - disable_new_netns = false - disable_block_device_use = false - disable_vhost_net = true - ``` - -2. Run the `isula` command with **root** permissions to start the BusyBox secure container and interconnect StratoVirt with it. - - ```shell - # isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh - ``` - -3. Run the `isula ps` command to check whether the secure container **test** is running properly. Then run the following command to access the container: - - ```shell - # isula exec –ti test sh - ``` - -4. Use a VM snapshot to accelerate startup of the secure container and reduce the VM memory overhead. - - Modify the kata configuration file **configuration.toml** and set **enable_template** to **true** to allow the VM to start by creating a snapshot. - - ```shell - [factory] - # VM templating support. Once enabled, new VMs are created from template - # using vm cloning. They will share the same initial kernel, initramfs and - # agent memory by mapping it readonly. It helps speeding up new container - # creation and saves a lot of memory if there are many kata containers running - # on the same host. - # - # When disabled, new VMs are created from scratch. - # - # Note: Requires "initrd=" to be set ("image=" is not supported). - # - # Default false - enable_template = true - ``` - - After the **enable_template** configuration item is set to **true**, kata-containers checks whether a snapshot file exists in the default path (**/run/vc/vm/template**) during secure container creation. If yes, kata-containers starts the VM using the snapshot file. If no, kata-containers creates a VM snapshot and start the VM using the snapshot file. - -5. Use the security component Ozone to further enhance the isolation of secure containers. - - Modify the kata configuration file **configuration.toml** and set the configuration item **ozone_path** to the path of the Ozone executable file. (If StratoVirt is installed using Yum, the Ozone executable file is stored in the **/usr/bin** directory by default.) After this item is configured, the Ozone security sandbox function is enabled to protect the VM against attacks after the virtualization layer isolation is broken and further enhance the isolation of StratoVirt secure containers. - - ```shell - # Path for the ozone specific to stratovirt - # If the ozone path is set, stratovirt will be launched in - # ozone secure environment. It is disabled by default. - ozone_path = "/usr/bin/ozone" - ``` - - You can now run container commands in the **test** container. - -#### Connecting to a Standard VM - -To use a StratoVirt standard VM as the sandbox of a secure container, you need to modify some other configurations. - -1. The configurations are as follows: - - ```shell - [hypervisor.stratovirt] - path = "/usr/bin/stratovirt" - kernel = "/var/lib/kata/kernel" - initrd = "/var/lib/kata/kata-containers-initrd.img" - # x86_64 architecture - machine_type = "q35" - # AArch64 architecture - machine_type = "virt" - block_device_driver = "virtio-blk" - pcie_root_port = 2 - use_vsock = true - enable_netmon = true - internetworking_model = "tcfilter" - sandbox_cgroup_with_emulator = false - disable_new_netns = false - disable_block_device_use = false - disable_vhost_net = true - ``` - - In the configurations above, modify the VM type according to the architecture of the host machine. Change the value of **block_device_driver** to **virtio-blk**. StratoVirt supports only devices hot-plugged to the root port. Set a proper value of **pcie_root_port** based on the number of devices to be hot-plugged. - -2. Install the firmware required for starting a standard VM. - - x86_64 architecture: - - ```shell - # yum install -y edk2-ovmf - ``` - - AArch64 architecture: - - ```shell - # yum install -y edk2-aarch64 - ``` - -3. Build and replace the binary file of kata-containers 2.x. - - Currently, a StratoVirt standard VMs can only be used as the sandbox of a kata-containers 2.x container (corresponding to the openEuler-21.09 branch in the kata-containers repository). You need to download and compile the kata-containers source code and replace the **containerd-shim-kata-v2** binary file in the **/usr/bin** directory. - - ```shell - # mkdir -p /root/go/src/github.com/ - # cd /root/go/src/github.com/ - # git clone https://gitee.com/src-openeuler/kata-containers.git - # cd kata-containers - # git checkout openEuler-21.09 - # ./apply-patches - # cd src/runtime - # make - ``` - - Back up the kata binary file in the **/usr/bin/** directory and replace it with the compiled binary file **containerd-shim-kata-v2**. - - ```shell - # cp /usr/bin/containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2.bk - # cp containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2 - ``` - -4. Run the `isula` command with **root** permissions to start the BusyBox secure container and interconnect StratoVirt with it. - - ```shell - # isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh - ``` - -5. Run the `isula ps` command to check whether the secure container **test** is running properly. Then run the following command to access the container: - - ```shell - # isula exec -ti test sh - ``` - diff --git a/docs/en/docs/StratoVirt/public_sys-resources/note.png b/docs/en/docs/StratoVirt/public_sys-resources/note.png deleted file mode 100644 index ad5ed3f7beeb01e6a48707c4806606b41d687e22..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/StratoVirt/public_sys-resources/note.png and /dev/null differ diff --git a/docs/en/docs/SysCare/SysCare_introduction.md b/docs/en/docs/SysCare/SysCare_introduction.md deleted file mode 100644 index 2613ebe8a0da6a28dc616a6fd657000a785600f5..0000000000000000000000000000000000000000 --- a/docs/en/docs/SysCare/SysCare_introduction.md +++ /dev/null @@ -1,22 +0,0 @@ -# Introduction to SysCare - -## Overview - -SysCare is an online live patching tool that automatically fixes bugs and vulnerabilities in OS components, such as kernels, user-mode services, and dynamic libraries. - -![img](./figures/syscare_arch.png) - -## SysCare Functions - -SysCare supports live patching for kernels and user-mode services: - -1. One-click creation -SysCare is a unified environment for both kernel- and user-mode live patches that ignores differences between patches, ensuring they can be created with just one click. -2. Patch lifecycle operations -SysCare provides a unified patch management interface for users to install, activate, uninstall, and query patches. - -## SysCare Technologies - -1. Unified patches: SysCare masks differences in detail when creating patches, providing a unified management tool to improve O&M efficiency. -2. User-mode live patching: SysCare supports live patching of multi-process and multi-thread services in user mode, which takes effect when a process or thread is started or restarted. -3. Lazy mechanism: SysCare fixes the ptrace defect (all kernel calls are ended) and improves the fix success rate. diff --git a/docs/en/docs/SysCare/SysCare_user_guide.md b/docs/en/docs/SysCare/SysCare_user_guide.md deleted file mode 100644 index ee022874c6c1a67fa3557df60ca150dcf1d9f69f..0000000000000000000000000000000000000000 --- a/docs/en/docs/SysCare/SysCare_user_guide.md +++ /dev/null @@ -1,3 +0,0 @@ -# SysCare User Guide - -This document describes how to install and use SysCare on openEuler. diff --git a/docs/en/docs/SysCare/faqs.md b/docs/en/docs/SysCare/faqs.md deleted file mode 100644 index 979e131eded61c76adf93a15b336d7f0d921e242..0000000000000000000000000000000000000000 --- a/docs/en/docs/SysCare/faqs.md +++ /dev/null @@ -1,19 +0,0 @@ -# FAQs - -## Issue 1: "alloc upatch module memory failed" - -Possible cause: The SELinux constraint is triggered. - -Solution: Manually add policies according to the error. The policies to be added vary according to the actual situation. For details, see . - -## Issue 2: "patch file error 2" - -Possible cause: The patch cannot be detected. - -Solution: Use another patch. - -## Issue 3: "build project error 11" - -Possible cause: The source package fails to be compiled. - -Solution: Run `rpmbuild -ra *.src.rpm` to check if the source package can be compiled and the compilation dependencies are satisfied. diff --git a/docs/en/docs/SysCare/installing_SysCare.md b/docs/en/docs/SysCare/installing_SysCare.md deleted file mode 100644 index 9f80648d6b64c45069b40be475c896d28c48141a..0000000000000000000000000000000000000000 --- a/docs/en/docs/SysCare/installing_SysCare.md +++ /dev/null @@ -1,43 +0,0 @@ -# Installing SysCare - -This chapter describes how to install SysCare on openEuler. - -## Installing SysCare Core Components - -### Minimum Hardware Requirements - -* 2 CPUs (x86_64 or AArch64) -* 4 GB memory -* 100 GB drive - -### Prerequisites - -1. openEuler 23.09 has been installed. - -### Installing from Source - -Clone the SysCare source code and then compile and install SysCare as follows: - -```shell -git clone https://gitee.com/openeuler/syscare.git -cd syscare -mkdir build -cd build -cmake -DCMAKE_INSTALL_PREFIX=/usr -DKERNEL_VERSION=$(uname -r) .. -make -make install -``` - -### Installing SysCare from a Repository - -If the repository source contains SysCare packages, you can use the `dnf` or `yum` command to download and install them. - -```shell -dnf install syscare syscare-kmod syscare-build syscare-build-ebpf -``` - -### Uninstalling SysCare - -```shell -dnf erase syscare syscare-kmod syscare-build syscare-build-ebpf -``` diff --git a/docs/en/docs/SysCare/using_SysCare.md b/docs/en/docs/SysCare/using_SysCare.md deleted file mode 100644 index 080de365ec5a2fee57a81b2166bb9dd5b5a10433..0000000000000000000000000000000000000000 --- a/docs/en/docs/SysCare/using_SysCare.md +++ /dev/null @@ -1,301 +0,0 @@ -# Using SysCare - -This chapter describes how to use SysCare on openEuler. - -## Prerequisites - -openEuler 22.03 LTS SP2 has been installed. - -## Using SysCare CLI Tools - -You can use `syscare build` to create patches and use `syscare patch` to manage patches, including installing, activating, deactivating, confirming, and uninstalling patches. - -### Creating Patches - -`syscare-build` is used to create patches, for example: - -```shell -syscare build \ - --patch-name "HP001" \ - --source ./redis-6.2.5-1.src.rpm \ - --debuginfo ./redis-debuginfo-6.2.5-1.x86_64.rpm \ - --output ./output \ - ./0001-Prevent-unauthenticated-client-from-easily-consuming.patch -``` - -### Managing Patches - -The pattern for matching a patch name is **TARGET_PACKAGE_NAME/PATCH_NAME**. If **PATCH_NAME** is unique, **TARGET_PACKAGE_NAME/** can be omitted. UUIDs can also be used to identify packages. - -1. Installing a patch: - - ```shell - syscare apply PATCH_NAME - ``` - -2. Activating a patch: - - ```shell - syscare active PATCH_NAME - ``` - -3. Deactivating a patch: - - ```shell - syscare deactive PATCH_NAME - ``` - -4. Uninstalling/removing a patch: - - ```shell - syscare remove PATCH_NAME - ``` - -5. Confirming a patch: - - ```shell - syscare accept patch-name - ``` - -6. Querying the status of a patch: - - ```shell - syscare status PATCH_NAME - ``` - -7. Querying all SysCare patches: - - ```shell - syscare list - ``` - -## Patch Making Module - -### SysCare Patch Making Tool - -`syscare-build` is a CLI tool that creates kernel- and user-mode live patches from RPM packages. Patches are encapsulated into RPM packages. - -### Command Parameters - -```text -Usage: syscare-build [OPTIONS] --patch-name --source --debuginfo ... - -Arguments: - ... Patch file(s) - -Options: - -n, --patch-name Patch name - --patch-arch Patch architecture [default: x86_64] - --patch-version Patch version [default: 1] - --patch-release Patch release [default: 1] - --patch-description Patch description [default: (none)] - --target-name Patch target name - -t, --target-elfname Patch target executable name - --target-arch parch target architecture - --target-epoch Patch target epoch - --target-version Patch target version - --target-release Patch target release - --target-license Patch target license - -s, --source Source package - -d, --debuginfo Debuginfo package - --workdir Working directory [default: .] - -o, --output Generated patch output directory [default: .] - -j, --jobs Parallel build jobs [default: 96] - --skip-compiler-check Skip compiler version check (not recommended) - --skip-cleanup Skip post-build cleanup - -v, --verbose Provide more detailed info - -h, --help Print help information - -V, --version Print version information -``` - -### Parameters - -|Name|Description|Type|Note| -| ---- | ---- | ---- | ---- | -| *\* |Patch file path|String|Mandatory. The value can be multiple valid paths.| - -### Options - -|Name|Description|Type|Note| -| ---- | ---- | ---- | ---- | -|-n, --patch-name *\*|Patch name|String|Mandatory. The value must comply with the RPM package naming convention.| -|--patch-arch *\*|Patch architecture|String|The default value is the current architectures. The value must comply with the RPM package naming convention.| -|--patch-version *\*|Patch version|String|The default value is **1**. The value must comply with the RPM package naming convention.| -|--patch-release *\*|Patch release|Integer|The default value is **1**. The value must comply with the RPM package naming convention.| -|--patch-description *\*|Patch description|String|The default value is **none**.| -|--target-name *\*|Target software RPM package name|String|The default value is determined by the **src.rpm** package specified by `--source`.| -|--target-arch *\*|Target software RPM package architecture|String|The default value is determined by the **src.rpm** package specified by `--source`.| -|--target-epoch *\*|Target software RPM package epoch|String|The default value is determined by the **src.rpm** package specified by `--source`.| -|--target-version *\*|Target software RPM package version|String|The default value is determined by the **src.rpm** package specified by `--source`.| -|--target-release *\*|Target software RPM package release|String|The default value is determined by the **src.rpm** package specified by `--source`.| -|--target-license *\*|Target software RPM package license|String|The default value is determined by the **src.rpm** package specified by `--source`.| -|-s, --source *\*|Target software **src.rpm** package path|String|Mandatory. The value must be a valid path.| -|-d, --debuginfo *\*|Target software **debuginfo** package path|String|Mandatory. The value must be a valid path.| -|--workdir *\*|Temporary directory|String|The default value is the current directory. The value must be a valid path.| -|-o, --output *\*|Patch output directory|String|The default value is the current directory. The value must be a valid path.| -|-j, --jobs *\*|Number of parallel compilation jobs|Integer|The default value is the number of CPU threads| -|--skip-compiler-check|Skip compiler check|Flag|-| -|--skip-cleanup|Skip temporary file cleanup|Flag|-| -|-v, --verbose|Print detail information|Flag|-| -|-h, --help|Print help information|Flag|-| -|-V, --version|Print version information|Flag|-| - -An example command is as follows: - -```shell -syscare build \ - --patch-name "HP001" \ - --patch-description "CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users." \ - --source ./redis-6.2.5-1.src.rpm \ - --debuginfo ./redis-debuginfo-6.2.5-1.x86_64.rpm \ - --output ./output \ - ./0001-Prevent-unauthenticated-client-from-easily-consuming.patch -``` - -### Patch Output - -- A patch package that contains the binary file of SysCare and meta information. This package is used to install the live patch. -- A patch source package that contains the target software source code and the new patch. This package is used to create live patches for new versions. - -Naming rules: - -- Patch package: patch-*TARGET_SOFTWARE_FULL_NAME*-*PATCH_NAME*-*PATCH_VERSION*-*PATCH_RELEASE*.*PATCH_ARCHITECTURE*.rpm -- Patch source code package: *TARGET_SOFTWARE_FULL_NAME*-*PATCH_NAME*-*PATCH_VERSION*-*PATCH_RELEASE*.*PATCH_ARCHITECTURE*.src.rpm - -### Patch Information - -The patch meta information contains the following fields: - -| Field | Description | -| ----------- | ---------------------- | -| uuid | Patch ID | -| name | Patch name | -| version | Patch version | -| release | Patch release | -| arch | Patch architecture | -| type | Patch type | -| target | Target software name | -| target_elf | Name of the executable file of the target software | -| digest | Patch fingerprint | -| license | Target software license | -| description | Patch description | -| patch| Patch file list | - -Example: - -```text -syscare info redis-6.2.5-1/HP001 -uuid: ec503257-aa75-4abc-9045-c4afdd7ae0f2 -name: HP001 -version: 1 -release: 1 -arch: x86_64 -type: UserPatch -target: redis-6.2.5-1 -target_elf: redis-cli, redis-server, redis-benchmark -digest: 31fc7544 -license: BSD and MIT -description: CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users. -patch: -31fc7544 0001-Prevent-unauthenticated-client-from-easily-consuming.patch -``` - -### Patch Making Process - -1. Prepare the source package (source RPM) and debugging information package (debuginfo RPM) of the target software. - - Example: - - ```shell - - yumdownloader kernel --source - - yumdownloader kernel --debuginfo - ``` - -2. Ensure that the related software build dependencies are installed. - - Example: - - ```shell - dnf install make gcc bison flex openssl-devel dwarves python3-devel elfutils-libelf-devel - ``` - -3. Run the `syscare-build` command. - - Example: - - ```shell - syscare build \ - --patch-name HP001 \ - --source kernel-5.10.0-60.66.0.91.oe2203.src.rpm \ - --debuginfo kernel-debuginfo-5.10.0-60.66.0.91.oe2203.x86_64.rpm \ - --output output \ - 001-kernel-patch-test.patch - ``` - - During patch making, a temporary folder whose name starts with **syscare-build** is created in the directory specified by `--workdir` (the current directory by default) to store temporary files and build logs. - - Example: - - ```shell - $ ls -l syscare-build.111602/ - total 100 - -rw-r--r--. 1 dev dev 92303 Nov 12 00:00 build.log - drwxr-xr-x. 6 dev dev 4096 Nov 12 00:00 package - drwxr-xr-x. 4 dev dev 4096 Nov 12 00:00 patch - ``` - - Build logs (**build.log**) are generated in the temporary folder. - - ```shell - $ cat syscare-build.111602/build.log | less - ... - ``` - - If the patch is created successfully and `--skip-compiler-check` is not specified, the temporary folder will be deleted after patch making. - -4. Check the build result. - - Example: - - ```shell - $ ls -l - total 189680 - -rw-r--r--. 1 dev dev 194218767 Nov 12 00:00 kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm - -rw-r--r--. 1 dev dev 10937 Nov 12 00:00 patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm - ``` - - In the output: - - **patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm** is the patch package. - - **kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm** is the patch source package. - -5. Install the patch. - - ```shell - dnf install patch-xxx.rpm - ``` - - After the patch is installed, files in the patch are stored in the **/usr/lib/syscare/patches/target_software_package_name/patch_name** directory - -6. Uninstall the patch. - - ```shell - dnf remove patch-xxx - ``` - - The patch package will be uninstalled when the patch is beyond the **ACTIVED** state. - -### Error Handling - -If an error occurs, see the build logs: - -Error output example: - -```text -... -Building patch, this may take a while -ERROR: Process '/usr/libexec/syscare/upatch-build' exited unsuccessfully, exit_code=255 -``` diff --git a/docs/en/docs/SystemOptimization/big-data-tuning.md b/docs/en/docs/SystemOptimization/big-data-tuning.md deleted file mode 100644 index cb958c1503dbfe5cfc37c34a0df91c175c971393..0000000000000000000000000000000000000000 --- a/docs/en/docs/SystemOptimization/big-data-tuning.md +++ /dev/null @@ -1,809 +0,0 @@ -# Big Data Tuning Guide - -## Hardware Tuning - -### Configuring BIOS - -#### Purpose - -You can set some advanced configuration options in the BIOS for different server hardware devices to improve server performance. - -#### Method - -This method applies to Kunpeng servers. For x86 servers, such as Intel servers, you can retain the default BIOS configurations. - -1. Disable the SMMU (only for the Kunpeng server). -2. During the server restart, press **Delete** to access the BIOS, choose **Advanced** \> **MISC Config**, and press **Enter**. -3. Set **Support Smmu** to **Disable**. - - Note: Disable the SMMU feature only in non-virtualization scenarios. In virtualization scenarios, enable the SMMU. - -4. Disable prefetch. - - - On the BIOS, choose **Advanced** \> **MISC Config** and press **Enter**. - - - Set **CPU Prefetching Configuration** to **Disabled** and press **F10**. - -### Creating RAID 0 - -#### Purpose - -A RAID array can improve the overall storage and access performance of drives. If an LSI SAS3108, LSI SAS3408, or LSI SAS3508 RAID controller card is used, you are advised to create a single-drive RAID 0 array or a RAID 5 array to fully use the RAID controller card cache (which is 2 GB for the LSI SAS3508 RAID controller) and increase the read/write speed. - -#### Method - -1. Use the **storcli64_arm** file to check the RAID group creation. - - ```shell - ./storcli64_arm /c0 show - ``` - - Note: You can download the **storcli64_arm** file from the following URL and run it in any directory: . - -2. Create a RAID 0 array. In the following example, the command is used to create a RAID 0 array for the second 1.2 TB drive. **c0** indicates the ID of the RAID controller card, and **r0** indicates RAID 0. Perform this operation on all drives except the system drive. - - ```shell - ./storcli64_arm /c0 add vd r0 drives=65:1 - ``` - - - -### Enabling the RAID Controller Card Cache - -#### Purpose - -Use RAWBC (without a supercapacitor) or RWBC (with a supercapacitor) with a RAID controller card to provide better performance. - -Use the storcli tool to modify the cache settings of the RAID array. Set **Read Policy** to **Read ahead** to use the cache of the RAID controller card for prefetch. Set **Write Policy** to **Write back** to use the cache of the RAID controller card for write back instead of write through. Set **IO policy** to **Cached IO** to use the cache of the RAID controller card to cache I/Os. - -#### Method - -1. Configure the cache of the RAID controller card. In the command, **vx** indicates the volume group number (v0, v1, and v2, etc.). Set **vx** based on the environment requirements. - - ```shell - ./storcli64_arm /c0/vx set rdcache=RA - ./storcli64_arm /c0/vx set wrcache=WB/AWB - ./storcli64_arm /c0/vx set iopolicy=Cached - ``` - -2. Check whether the configuration has taken effect. - - ```shell - ./storcli64_arm /c0 show - ``` - -### Adjusting rx_buff - -#### Purpose - -Take the 1822 NIC as an example. The default value of **rx_buff** is **2** KB. When 64 KB packets are aggregated, multiple discontinuous memory fragments are required, resulting in low utilization. This parameter can be set to **2**, **4**, **8**, or **16** KB. After the value is changed, discontinuous memory can be reduced and the memory utilization is improved. - -#### Method - -1. Check the value of **rx_buff**. The default value is **2**. - - ```shell - cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff - ``` - -2. Add the **hinic.conf** file to the **/etc/modprobe.d/** directory and change the value of **rx_buff** to **8**. - - ```shell - options hinic rx_buff=8 - ``` - -3. Mount the HiNIC driver again for the new parameters to take effect. - - ```shell - rmmod hinic - modprobe hinic - ``` - -4. Check whether the **rx_buff** parameter is updated successfully. - - ```shell - cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff - ``` - -### Adjusting the Ring Buffer - -#### Purpose - -Take the 1822 NIC as an example. The maximum ring buffer size of the NIC is **4096**. The default value is **1024**. You can increase the size of the ring buffer. - -#### Method - -1. Check the default size of the ring buffer. Assume that the current NIC is named **enp131s0**. - - ```shell - ethtool -g enp131s0 - ``` - -2. Change the value of **Ring Buffer** to **4096**. - - ```shell - ethtool -G enp131s0 rx 4096 tx 4096 - ``` - -3. Confirm that the ring buffer value has been updated. - - ```shell - ethtool -g enp131s0 - ``` - - - -4. Reduce the quantity of queues. - - ```shell - ethtool -L enp131s0 combined 4 - ethtool -l enp131s0 - ``` - -### Enabling LRO - -#### Purpose - -Take the 1822 NIC as an example. The NIC supports large receive offload (LRO). You can enable this function and set the LRO parameter of the 1822 properly. - -#### Method - -1. Check whether the value of the LRO parameter is set to **on**. The default value is **off**. Assume that the current NIC is named **enp131s0**. - - ```shell - ethtool -k enp131s0 - ``` - -2. Enable LRO. - - ```shell - ethtool -K enp131s0 lro on - ``` - -3. Check whether the value of the LRO parameter is set to **on**. - - ```shell - ethtool -k enp131s0 - ``` - - - -### Binding NIC Interrupts to Cores - -#### Purpose - -To help the service network improve the capability of receiving and sending packets, binding cores manually to fix interrupts is a better option than using irqbalance of the kernel to schedule NIC interrupts on all cores. - -#### Method - -1. Disable irqbalance. - - Before binding cores to NICs, disable irqbalance. - - Run the following command: - - ```shell - systemctl stop irqbalance.service # (Stop irqbalance. The setting will be invalid after the system restarts.) - systemctl disable irqbalance.service # (Disable irqbalance. The setting takes effect permanently.) - systemctl status irqbalance.service # (Check whether the irqbalance service is disabled.) - ``` - -2. Check the PCI device number of the NIC. Assume that the current NIC is named **enp131s0**. - - ```shell - ethtool -i enp131s0 - ``` - - - -3. Check the NUMA node to which the PCIe NIC connects. - - ```shell - lspci -vvvs - ``` - - - -4. Check the core range corresponding to the NUMA node. For example, for the Kunpeng 920 5250 processor, the core range can be 48 to 63. - - - -5. Bind interrupts to cores. The Hi1822 NIC has 16 queues. Bind the interrupts to the 16 cores of the NUMA node to which the NIC connect (for example, cores 48 to 63 corresponding to NUMA node 1). - - ```shell - bash smartIrq.sh - ``` - - The script content is as follows: - - ```shell - #!/bin/bash - irq_list=(`cat /proc/interrupts | grep enp131s0 | awk -F: '{print $1}'`) - cpunum=48 # Change the value to the first core of the node. - for irq in ${irq_list[@]} - do - echo $cpunum > /proc/irq/$irq/smp_affinity_list - echo `cat /proc/irq/$irq/smp_affinity_list` - (( cpunum+=1 )) - done - ``` - -6. Check whether the core binding is successful. - - ```shell - sh irqCheck.sh enp131s0 - ``` - - - - The script content is as follows: - - ```shell - #!/bin/bash - # NIC name - intf=$1 - log=irqSet-`date "+%Y%m%d-%H%M%S"`.log - # Number of available CPUs - cpuNum=$(cat /proc/cpuinfo |grep processor -c) - # RX and TX interrupt lists - irqListRx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}') - irqListTx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}') - # Bind the RX interrupt requests (IRQs). - for irqRX in ${irqListRx[@]} - do - cat /proc/irq/${irqRX}/smp_affinity_list - done - # Bind the TX IRQs. - for irqTX in ${irqListTx[@]} - do - cat /proc/irq/${irqTX}/smp_affinity_list - done - ``` - -## OS Tuning - -### Disabling the Transparent Huge Page - -#### Purpose - -Disable the transparent huge page (THP) to prevent memory leak and reduce lag. - -#### Method - -Disable the THP. - -```shell -echo never > /sys/kernel/mm/transparent_hugepage/enabled -echo never > /sys/kernel/mm/transparent_hugepage/defrag -``` - -### Disabling the Swap Partition - -#### Purpose - -The virtual memory of the Linux OS is automatically adjusted based on the system load. Swapping memory pages to the swap partition of drives affects the test performance. - -#### Method - -```shell -swapoff -a -``` - -Comparison before and after the modification - - - -## Spark Tuning - -### Tuning I/O Configuration Items - -- I/O queue adjustment - - - Purpose - - Use the single-queue (soft queue) mode for better performance during Spark tests. - - - Method - - Add **scsi_mod.use_blk_mq=0** to the kernel startup command line in **/etc/grub2-efi.cfg** and restart the system for the modification to take effect. - - - -- Kernel I/O parameter configuration - - ```shell - #! /bin/bash - - echo 3000 > /proc/sys/vm/dirty_expire_centisecs - echo 500 > /proc/sys/vm/dirty_writeback_centisecs - - echo 15000000 > /proc/sys/kernel/sched_wakeup_granularity_ns - echo 10000000 > /proc/sys/kernel/sched_min_granularity_ns - - systemctl start tuned - sysctl -w kernel.sched_autogroup_enabled=0 - sysctl -w kernel.numa_balancing=0 - - echo 11264 > /proc/sys/vm/min_free_kbytes - echo 60 > /proc/sys/vm/dirty_ratio - echo 5 > /proc/sys/vm/dirty_background_ratio - - list="b c d e f g h i j k l m" # Modify as required. - for i in $list - do - echo 1024 > /sys/block/sd$i/queue/max_sectors_kb - echo 32 > /sys/block/sd$i/device/queue_depth - echo 256 > /sys/block/sd$i/queue/nr_requests - echo mq-deadline > /sys/block/sd$i/queue/scheduler - echo 2048 > /sys/block/sd$i/queue/read_ahead_kb - echo 2 > /sys/block/sd$i/queue/rq_affinity - echo 0 > /sys/block/sd$i/queue/nomerges - done - ``` - -### Adapting JVM Parameters and Version - -#### Purpose - -The latest JDK version optimizes the Spark performance. - -#### Method - -Add the following configurations to the **spark-defaults.conf** file to use the new JDK parameters: - -```shell -spark.executorEnv.JAVA_HOME /usr/local/jdk8u222-b10 -spark.yarn.appMasterEnv.JAVA_HOME /usr/local/jdk8u222-b10 -spark.executor.extraJavaOptions -XX:+UseNUMA -XX:BoxTypeCachedMax=100000 -spark.yarn.am.extraJavaOptions -XX:+UseNUMA -XX:BoxTypeCachedMax=100000 -``` - -Note: In the x86 architecture, the JVM parameter **-XX:BoxTypeCachedMax** is not applicable. - -### Tuning Spark Application Parameters - -#### Purpose - -Based on the basic Spark configuration values, obtain a group of executor parameters using the theoretical formula to boost the performance of Kunpeng-powered servers. - -#### Method - -- If you use Spark-Test-Tool to test SQL 1 to SQL 10, open the **script/spark-default.conf** file in the tool directory and add the following configuration items: - - ```shell - yarn.executor.num 15 - yarn.executor.cores 19 - spark.executor.memory 44G - spark.driver.memory 36G - ``` - -- If you use HiBench to test the WordCount, TeraSort, Bayesian, and k-means scenarios, open the **conf/spark.conf** file in the tool directory and adjust the number of running cores and memory size based on the actual environment. - - ```shell - yarn.executor.num 15 - yarn.executor.cores 19 - spark.executor.memory 44G - spark.driver.memory 36G - ``` - -### Tuning Items for Dedicated Scenarios - -#### SQL Scenario - -##### I/O-Intensive Scenario: SQL 1 - -- Purpose - - SQL 1 is an I/O-intensive scenario. You can tune I/O parameters for the optimal performance. - -- Method - - - Set the following I/O parameters. **sd$i** indicates the names of all drives that are tested. - - ```shell - echo 128 > /sys/block/sd$i/queue/nr_requests - echo 512 > /sys/block/sd$i/queue/read_ahead_kb - ``` - - - Set the parameters of dirty pages in the memory. - - ```shell - /proc/sys/vm/vm.dirty_expire_centisecs 500 - /proc/sys/vm/vm.dirty_writeback_centisecs 100 - ``` - - - Set the degree of parallelism is in **Spark-Test-Tool/script/spark-default.conf**. - - ```shell - spark.sql.shuffle.partitions 350 - spark.default.parallelism 580 - ``` - - - In this scenario, set other parameters to the general tuning values in [Tuning Spark Application Parameters](#tuning-spark-application-parameters). - -##### CPU-Intensive Scenarios: SQL 2 & SQL 7 - -- Purpose - - ​SQL 2 and SQL 7 are CPU-intensive scenarios. You can tune Spark executor parameters for the optimal performance. - -- Method - - - Based on the actual environment, adjust the number of running cores and memory size specified by Spark-Test-Tool in the **script/spark-default.conf** file to achieve the optimal performance. For example, for the Kunpeng 920 5220 processor, the following executor parameters are recommended for SQL 2 and SQL 7. - - ```shell - yarn.executor.num 42 - yarn.executor.cores 6 - spark.executor.memory 15G - spark.driver.memory 36G - ``` - - - Set the degree of parallelism is in **Spark-Test-Tool/script/spark-default.conf**. - - For SQL 2, set the parameters as follows: - - ```shell - spark.sql.shuffle.partitions 150 - spark.default.parallelism 504 - ``` - - For SQL 7, set the parameters as follows: - - ```shell - spark.sql.shuffle.partitions 300 - spark.default.parallelism 504 - ``` - -##### I/O- and CPU-Intensive Scenario: SQL 3 - -- Purpose - - SQL 3 is an I/O- and CPU-intensive scenario. You can tune Spark executor parameters and adjust I/O parameters for the optimal performance. - -- Method - - - Based on the actual environment, adjust the number of running cores and memory size specified by Spark-Test-Tool in the **script/spark-default.conf** file to achieve the optimal performance. For example, for the Kunpeng 920 5220 processor, the following executor parameters are recommended for SQL 3. - - ```shell - yarn.executor.num 30 - yarn.executor.cores 6 - spark.executor.memory 24G - spark.driver.memory 36G - ``` - - - Adjust the I/O prefetch value. **sd$i** indicates the names of all Spark drives. - - ```shell - echo 4096 > /sys/block/sd$i/queue/read_ahead_kb - ``` - - - Set the degree of parallelism is in **Spark-Test-Tool/script/spark-default.conf**. - - ```shell - spark.sql.shuffle.partitions 150 - spark.default.parallelism 360 - ``` - -##### CPU-Intensive Scenario: SQL 4 - -- Purpose - - SQL 4 is a CPU-intensive scenario. You can tune Spark executor parameters and adjust I/O parameters for the optimal performance. - -- Method - - Based on the actual environment, adjust the number of running cores and memory size specified by Spark-Test-Tool in the **script/spark-default.conf** file to achieve the optimal performance. For example, for the Kunpeng 920 5220 processor, the following executor parameters are recommended for SQL 4. - - - Open the **script/spark-default.conf** file in the tool directory and add the following configuration items: - - ```shell - yarn.executor.num 42 - yarn.executor.cores 6 - spark.executor.memory 15G - spark.driver.memory 36G - ``` - - - Adjust the I/O prefetch value. **sd$i** indicates the names of all Spark drives. - - ```shell - echo 4096 > /sys/block/sd$i/queue/read_ahead_kb - ``` - - - Set the degree of parallelism is in **Spark-Test-Tool/script/spark-default.conf**. - - ```shell - spark.sql.shuffle.partitions 150 - spark.default.parallelism 504 - ``` - -##### SQL 5, SQL 6, SQL 8, SQL 9, SQL 10 - -- Set the degree of parallelism is in **Spark-Test-Tool/script/spark-default.conf**. - - ```shell - spark.sql.shuffle.partitions 300 - spark.default.parallelism 504 - ``` - -- In this scenario, set other parameters to the general tuning values in [Tuning Spark Application Parameters](#tuning-spark-application-parameters). - -#### HiBench Scenario - -##### I/O- and CPU-Intensive Scenario: WordCount - -- Purpose - - WordCount is an I/O- and CPU-intensive scenario, where the mq-deadline algorithm and I/O parameter adjustment can deliver higher performance than that of the single-queue deadline scheduling algorithm. - -- Method - - - Modify the following configurations. **sd$i** indicates the names of all drives that are tested. - - ```shell - echo mq-deadline > /sys/block/sd$i/queue/scheduler - echo 512 > /sys/block/sd$i/queue/nr_requests - echo 8192 > /sys/block/sd$i/queue/read_ahead_kb - echo 500 > /proc/sys/vm/dirty_expire_centisecs - echo 100 > /proc/sys/vm/dirty_writeback_centisecs - echo 5 > /proc/sys/vm/dirty_background_ratio - ``` - - - In this scenario, you can set the quantity of partitions and parallelism to three to five times of the total cluster core quantity for data sharding. This reduces the size of a single task file for better performance. You can use the following shard settings: - - ```shell - spark.sql.shuffle.partitions 300 - spark.default.parallelism 600 - ``` - - - Based on the actual environment, adjust the number of running cores and memory size specified by HiBench in the configuration file to achieve the optimal performance. For example, for the Kunpeng 920 5220 processor, the following executor parameters are recommended for WordCount. - - ```shell - yarn.executor.num 51 - yarn.executor.cores 6 - spark.executor.memory 13G - spark.driver.memory 36G - ``` - -##### I/O- and CPU-Intensive Scenario: TeraSort - -- Purpose - - ​TeraSort is an I/O- and CPU-intensive scenario. You can adjust I/O parameters and Spark executor parameters for the optimal performance. In addition, TeraSort requires high network bandwidth. You can tune network parameters to improve system performance. - -- Method - - - Modify the following configurations. **sd$i** indicates the names of all drives that are tested. - - ```shell - echo bfq > /sys/block/sd$i/queue/scheduler - echo 512 > /sys/block/sd$i/queue/nr_requests - echo 8192 > /sys/block/sd$i/queue/read_ahead_kb - echo 4 > /sys/block/sd$i/queue/iosched/slice_idle - echo 500 > /proc/sys/vm/dirty_expire_centisecs - echo 100 > /proc/sys/vm/dirty_writeback_centisecs - ``` - - - In this scenario, you can set the quantity of partitions and parallelism to three to five times of the total cluster core quantity for data sharding. This reduces the size of a single task file for better performance. Open the **conf/spark.conf** file of HiBench and use the following shard settings: - - ```shell - spark.sql.shuffle.partitions 1000 - spark.default.parallelism 2000 - ``` - - - Open the **conf/spark.conf** file of HiBench and add the following executor parameters: - - ```shell - yarn.executor.num 27 - yarn.executor.cores 7 - spark.executor.memory 25G - spark.driver.memory 36G - ``` - - - Tune network parameters. - - ```shell - ethtool -K enp131s0 gro on - ethtool -K enp131s0 tso on - ethtool -K enp131s0 gso on - ethtool -G enp131s0 rx 4096 tx 4096 - ethtool -G enp131s0 rx 4096 tx 4096 - # The TM 280 NIC supports a maximum of 9,000 MTUs. - ifconfig enp131s0 mtu 9000 up - ``` - -##### CPU-Intensive Scenario: Bayesian - -- Purpose - - ​Bayesian is a CPU-intensive scenario. You can adjust I/O parameters and Spark executor parameters for the optimal performance. - -- Method - - - You can use the following shard settings: - - ```shell - spark.sql.shuffle.partitions 1000 - spark.default.parallelism 2500 - ``` - - - Open the **conf/spark.conf** file of HiBench and add the following executor parameters: - - ```shell - yarn.executor.num 9 - yarn.executor.cores 25 - spark.executor.memory 73G - spark.driver.memory 36G - ``` - - - Use the following kernel parameters: - - ```shell - echo mq-deadline > /sys/block/sd$i/queue/scheduler - echo 0 > /sys/module/scsi_mod/parameters/use_blk_mq - echo 50 > /proc/sys/vm/dirty_background_ratio - echo 80 > /proc/sys/vm/dirty_ratio - echo 500 > /proc/sys/vm/dirty_expire_centisecs - echo 100 > /proc/sys/vm/dirty_writeback_centisecs - ``` - -##### Compute-Intensive Scenario: k-means - -- Purpose - - ​k-means is a CPU-intensive scenario. You can adjust I/O parameters and Spark executor parameters for the optimal performance. - -- Method - - - Adjust the Spark executor parameters to proper values. In this scenario, you can use the following shard settings: - - ```shell - spark.sql.shuffle.partitions 1000 - spark.default.parallelism 2500 - ``` - - - Adjust the I/O prefetch value. **sd$i** indicates the names of all Spark drives. - - ```shell - echo 4096 > /sys/block/sd$i/queue/read_ahead_kb - ``` - - - Open the **conf/spark.conf** file of HiBench and add the following executor parameters: - - ```shell - yarn.executor.num 42 - yarn.executor.cores 6 - spark.executor.memory 15G - spark.driver.memory 36G - spark.locality.wait 10s - ``` - -## Tuning Hive - -### Tuning I/O Configuration Items - -The **scheduler**, **read_ahead_kb**, and **sector_size** configurations are involved. - -- You are advised to use mq-deadline as the scheduler to achieve higher I/O efficiency and improve performance. - -- It is recommended that the prefetch size of a block device be set to 4 MB to achieve better read performance. The default value is 128 KB. - -- The **sector_size** of a block device must match the sector size of the physical drive. The **hw_sector_size**, **max_hw_sectors_kb**, and **max_sectors_kb** parameters can be used for matching. The values of the first two parameters are read from the hardware, and the value of the third parameter is the maximum aggregation block size of the kernel block device. It is recommended that the value of the third parameter be the same as that of the hardware. That is, the values of the latter two parameters should be the same. - -Set the parameters for all involved data drives as follows: - -```shell -list="b c d e f g h i j k l m" # Modify as required. -for i in $list -do - echo mq-deadline > /sys/block/sd$i/queue/scheduler - echo 4096 > /sys/block/sd$i/queue/read_ahead_kb - echo 512 > sys/block/sd$i/queue/hw_sector_size - echo 1024 > /sys/block/sd$i/queue/max_hw_sectors_kb - echo 256 > /sys/block/sd$i/queue/max_sectors_kb -done -``` - -### Tuning Dirty Page Parameters - -```shell -echo 500 > /proc/sys/vm/dirty_expire_centisecs -echo 100 > /proc/sys/vm/dirty_writeback_centisecs -``` - -### Hive Tuning - -#### Configuring Component Parameters - -| Component | Parameter | Recommended Value | Reason for Change | -| ----------------------------------------- | --------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------ | -| Yarn-\>NodeManager Yarn-\>ResourceManager | ResourceManager Java heap size | 1024 | Increase the JVM memory size to reduce the GC frequency.
Remarks: The values are not fixed. You need to increase or decrease the values of **Xms** and **Xmx** based on the GC release status.| -|Yarn-\>NodeManager Yarn-\>ResourceManager | NodeManager Java heap size | 1024 |Increase the JVM memory size to reduce the GC frequency.
Remarks: The values are not fixed. You need to increase or decrease the values of **Xms** and **Xmx** based on the GC release status.| -| Yarn-\>NodeManager | yarn.nodemanager.resource.cpu-vcores | Same as the actual total number of physical cores of the DataNode | Number of CPU cores that can be allocated to a container. | -|Yarn->NodeManager | yarn.nodemanager.resource.memory-mb | Same as the actual physical memory capacity of the DataNode | Memory that can be allocated to a container. | -| Yarn->NodeManager | yarn.nodemanager.numa-awareness.enabled | true | NUMA awareness when NodeManager starts a container, which needs to be manually added. | -|Yarn->NodeManager | yarn.nodemanager.numa-awareness.read-topology | true | Automatic NUMA topology awareness of NodeManager, which needs to be manually added. | -| MapReduce2 | mapreduce.map.memory.mb | 7168 | Maximum memory that can be used by a map task. | -|MapReduce2 | mapreduce.reduce.memory.mb | 14336 | Maximum memory that can be used by a reduce task. | -| MapReduce2 | mapreduce.job.reduce.slowstart.completedmaps | 0.35 | Only if the map completion ratio reaches the value of this parameter, the system allocates resources to reduce. | -| HDFS-\>NameNode | NameNode Java heap size | 3072 | Increase the JVM memory size to reduce the GC frequency. | -| HDFS->NameNode | NameNode new generation size | 384 |Increase the JVM memory size to reduce the GC frequency.| -|HDFS->NameNode | NameNode maximum new generation size | 384 |Increase the JVM memory size to reduce the GC frequency.| -|HDFS->NameNode | dfs.namenode.service.handler.count | 32 | Number of threads used by the NameNode RPC server to monitor DataNode requests and other requests. You can increase the value as required.| -|HDFS->NameNode | dfs.namenode.handler.count | 1200 | Number of threads used by the NameNode RPC server to monitor client requests. You can increase the value as required. | -| HDFS-\>DataNode | dfs.datanode.handler.count | 512 | Number of DataNode service threads. You can increase the value as required. | -| TEZ | tez.am.resource.memory.mb | 7168 | This parameter is equivalent to **yarn.scheduler.minimum-allocation-mb**. The default value is **7168**. | -| TEZ | tez.runtime.io.sort.mb | SQL1: 32 SQL2: 256 SQL3: 256 SQL4: 128 SQL5: 64 | Adjust the value of this parameter based on different scenarios. | -|TEZ | tez.am.container.reuse.enabled | true | Container reuse switch. | -|TEZ | tez.runtime.unordered.output.buffer.size-mb | 537 | 10% x Value of **hive.tez.container.size** | -|TEZ | tez.am.resource.cpu.vcores | 10 | Number of used virtual CPUs. The default value is **1**. You need to manually add this parameter. | -|TEZ | tez.container.max.java.heap.fraction | 0.85 | Percentage of memory allocated to the Java process against the memory provided by Yarn. The default value is **0.8**. You need to manually add this parameter.| - -#### Enabling the NUMA Feature - -Yarn 3.1.0 allows NUMA awareness to be enabled when the container is started. The principle is to read the CPU core and memory capacity of each NUMA node on the physical nodes. You can use the **numactl** command to specify the CPU range and membind range for starting the container and reduce cross-chip access. - -1. Install numactl. - - ```shell - yum install numactl.aarch64 - ``` - -2. Enable NUMA awareness. - - ```text - yarn.nodemanager.numa-awareness.enabled true - yarn.nodemanager.numa-awareness.read-topology true - ``` - -The parameters are listed in the preceding table. - -## HBase Tuning - -### Tuning HBase - -#### Configuring Component Parameters - -The following parameters are configured in this test. Only some Yarn parameters are different between the x86 computing platform and Kunpeng 920 computing platform (the differences are described in the table). HBase and HDFS use the same set of parameters for the test. - -| Component | Parameter | Recommended Value | Reason for Change | -| ----------------------------------------- | --------------------------------------------- | -------------------------------- | ------------------------------------------------------------ | -| Yarn-\>NodeManager Yarn-\>ResourceManager | ResourceManager Java heap size | 1024 | Increase the JVM memory size to reduce the GC frequency. | -| Yarn-\>NodeManager Yarn-\>ResourceManager | NodeManager Java heap size | 1024 | | -| Yarn-\>NodeManager | yarn.nodemanager.resource.cpu-vcores | Same as the actual total number of physical cores of the DataNode| Number of CPU cores that can be allocated to a container. | -|Yarn->NodeManager | yarn.nodemanager.resource.memory-mb | Same as the actual physical memory capacity of the DataNode| Memory that can be allocated to a container. | -|Yarn->NodeManager | yarn.nodemanager.numa-awareness.enabled | true | NUMA awareness when NodeManager starts a container, which needs to be manually added. | -|Yarn->NodeManager | yarn.nodemanager.numa-awareness.read-topology | true | Automatic NUMA topology awareness of NodeManager, which needs to be manually added. | -| MapReduce2 | mapreduce.map.memory.mb | 7168 | Maximum memory that can be used by a map task. | -|MapReduce2 | mapreduce.reduce.memory.mb | 14336 | Maximum memory that can be used by a reduce task. | -|MapReduce2 | mapreduce.job.reduce.slowstart.completedmaps | 0.35 | Only if the map completion ratio reaches the value of this parameter, the system allocates resources to reduce. | -| HDFS-\>NameNode | NameNode Java heap size | 3072 | Increase the JVM memory size to reduce the GC frequency. | -| HDFS-\>NameNode | NameNode new generation size | 384 |Increase the JVM memory size to reduce the GC frequency.| -| HDFS-\>NameNode | NameNode maximum new generation size | 384 |Increase the JVM memory size to reduce the GC frequency.| -| HDFS-\>NameNode | dfs.namenode.service.handler.count | 128 | Number of threads used by the NameNode RPC server to monitor DataNode requests and other requests. You can increase the value as required.| -| HDFS-\>NameNode | dfs.namenode.handler.count | 1200 | Number of threads used by the NameNode RPC server to monitor client requests. You can increase the value as required. | -| HDFS-\>DataNode | dfs.datanode.handler.count | 512 | Number of DataNode service threads. You can increase the value as required. | -| HBase-\>RegionServer | HBase RegionServer Maximum Memory | 31744 | Increase the JVM memory size to reduce the GC frequency. | -|HBase-\>RegionServer | hbase.regionserver.handler.count | 150 | Number of RPC server instances on RegionServer. | -|HBase-\>RegionServer | hbase.regionserver.metahandler.count | 150 | Number of program instances that process high-priority requests in RegionServer. | -| HBase-\>RegionServer | hbase.regionserver.global.memstore.size | 0.4 | Percentage of the maximum JVM heap size (specified by **Java -Xmx**) allocated to MemStore. | -|HBase-\>RegionServer | hfile.block.cache.size | 0.4 | Percentage of RegionServer GC **-Xmx** occupied by the data cache. | -|HBase-\>RegionServer | hbase.hregion.memstore.flush.size | 267386880 | Size of RegionServer MemStore. Increasing the value can reduce the block. | - -#### Enabling the NUMA Feature - -Yarn 3.1.0 allows NUMA awareness to be enabled when the container is started. The principle is to read the CPU core and memory capacity of each NUMA node on the physical nodes. You can use the **numactl** command to specify the CPU range and membind range for starting the container and reduce cross-chip access. - -1. Install numactl. - - ```shell - yum install numactl.aarch64 - ``` - -2. Enable NUMA awareness. - - ```text - yarn.nodemanager.numa-awareness.enabled true - yarn.nodemanager.numa-awareness.read-topology true - ``` - - The parameters are listed in the preceding table. diff --git a/docs/en/docs/SystemOptimization/figures/add-kernel-startup-item.png b/docs/en/docs/SystemOptimization/figures/add-kernel-startup-item.png deleted file mode 100644 index b674ab4a93e3fb2abd3f30749d96e724fd77019c..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/add-kernel-startup-item.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/check-whether-the-lro-parameter-is-enabled.png b/docs/en/docs/SystemOptimization/figures/check-whether-the-lro-parameter-is-enabled.png deleted file mode 100644 index 351e0d41ec47d790d4f3556d840e9c951e480680..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/check-whether-the-lro-parameter-is-enabled.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/core-binding-success-verification.png b/docs/en/docs/SystemOptimization/figures/core-binding-success-verification.png deleted file mode 100644 index 342e691a50fc63ea8a71fdf752a6df46daafe14c..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/core-binding-success-verification.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/core-binding.png b/docs/en/docs/SystemOptimization/figures/core-binding.png deleted file mode 100644 index 627d0ba137a169c37afa1cc6dd81a2fffd9a0085..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/core-binding.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/create-raid0.png b/docs/en/docs/SystemOptimization/figures/create-raid0.png deleted file mode 100644 index 31fc68e727aa3e1f3e9e29851e13ee2e05568735..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/create-raid0.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/kernel-boot-option-parameters.png b/docs/en/docs/SystemOptimization/figures/kernel-boot-option-parameters.png deleted file mode 100644 index 30bafb334c64617d4963b6781e8976a08de5b553..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/kernel-boot-option-parameters.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/mysql-tuning-flow.png b/docs/en/docs/SystemOptimization/figures/mysql-tuning-flow.png deleted file mode 100644 index 93e825f089964955735cfde5254090273785a10e..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/mysql-tuning-flow.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/pci-device-number.png b/docs/en/docs/SystemOptimization/figures/pci-device-number.png deleted file mode 100644 index 02dab7ffc45389886a5a7aec7222b1a53b62d509..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/pci-device-number.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/pci-nic-numa-node.png b/docs/en/docs/SystemOptimization/figures/pci-nic-numa-node.png deleted file mode 100644 index 401028d9f88ea936c4e08bc572aeee573ce84b92..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/pci-nic-numa-node.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/ring_buffer.png b/docs/en/docs/SystemOptimization/figures/ring_buffer.png deleted file mode 100644 index 4b4a608150554bf677f503213d0a0227310b0a17..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/ring_buffer.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/figures/swapoff-before-and-after-modification.png b/docs/en/docs/SystemOptimization/figures/swapoff-before-and-after-modification.png deleted file mode 100644 index 080c9f9bd79a0090d0ed962358e9da2457afdc77..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/figures/swapoff-before-and-after-modification.png and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/mysql-performance-tuning.md b/docs/en/docs/SystemOptimization/mysql-performance-tuning.md deleted file mode 100644 index b396d3adcfb94ad8177a290e771c7c612b399470..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/SystemOptimization/mysql-performance-tuning.md and /dev/null differ diff --git a/docs/en/docs/SystemOptimization/overview.md b/docs/en/docs/SystemOptimization/overview.md deleted file mode 100644 index 29e89cb96d468f50db7e3a2a90a32ffeb78676c5..0000000000000000000000000000000000000000 --- a/docs/en/docs/SystemOptimization/overview.md +++ /dev/null @@ -1,9 +0,0 @@ -# System Analysis and Tuning - -System analysis and tuning includes the MySQL performance tuning and big data tuning. The big data tuning includes: - -- Spark tuning -- Hive tuning -- HBase tuning - -It acts as a guide for you to perform tuning on openEuler, including tuning methods, suggestions, and directions. diff --git a/docs/en/docs/TailorCustom/figures/flowchart.png b/docs/en/docs/TailorCustom/figures/flowchart.png deleted file mode 100644 index d3a71e8bfdb886222151cea3b2a3c0e8d8eae64a..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/TailorCustom/figures/flowchart.png and /dev/null differ diff --git a/docs/en/docs/TailorCustom/imageTailor-user-guide.md b/docs/en/docs/TailorCustom/imageTailor-user-guide.md deleted file mode 100644 index 2ad4ae70147104cf945e4eeeedfa07e587a552e0..0000000000000000000000000000000000000000 --- a/docs/en/docs/TailorCustom/imageTailor-user-guide.md +++ /dev/null @@ -1,928 +0,0 @@ -# ImageTailor User Guide - - - [Introduction](#introduction) - - [Installation](#installation) - - [Software and Hardware Requirements](#software-and-hardware-requirements) - - [Obtaining the Installation Package](#obtaining-the-installation-package) - - [Installing imageTailor](#installing-imagetailor) - - [Directory Description](#directory-description) - - [Image Customization](#image-customization) - - [Overall Process](#overall-process) - - [Customizing Service Packages](#customizing-service-packages) - - [Setting a Local Repo Source](#setting-a-local-repo-source) - - [Adding Files](#adding-files) - - [Adding RPM Packages](#adding-rpm-packages) - - [Adding Hook Scripts](#adding-hook-scripts) - - [Configuring System Parameters](#configuring-system-parameters) - - [Configuring Host Parameters](#configuring-host-parameters) - - [Configuring Initial Passwords](#configuring-initial-passwords) - - [Configuring Partitions](#configuring-partitions) - - [Configuring the Network](#configuring-the-network) - - [Configuring Kernel Parameters](#configuring-kernel-parameters) - - [Creating an Image](#creating-an-image) - - [Command Description](#command-description) - - [Image Creation Guide](#image-creation-guide) - - [Tailoring Time Zones](#tailoring-time-zones) - - [Customization Example](#customization-example) - - - -## Introduction - -In addition to the kernel, an operating system contains various peripheral packages. These peripheral packages provide functions of a general-purpose operating system but also cause the following problems: - -- A large number of resources (such as memory, disks, and CPUs) are occupied, resulting in low system performance. -- Unnecessary functions increase the development and maintenance costs. - -To address these problems, openEuler provides the imageTailor tool for tailoring and customization images. You can tailor unnecessary peripheral packages in the OS image or add service packages or files as required. This tool includes the following functions: - -- System package tailoring: Tailors system commands, libraries, and drivers based on the list of RPM packages to be installed. -- System configuration modification: Configures the host name, startup services, time zone, network, partitions, drivers to be loaded, and kernel version. -- Software package addition: Adds custom RPM packages or files to the system. - - - -## Installation - -This section uses openEuler 22.03 LTS in the AArch64 architecture as an example to describe the installation method. - -### Software and Hardware Requirements - -The software and hardware requirements of imageTailor are as follows: - -- The architecture is x86_64 or AArch64. - -- The OS is openEuler 22.03 LTS (the kernel version is 5.10 and the Python version is 3.9, which meet the tool requirements). - -- The root directory **/** of the host to run the tool have at least 40 GB space. - -- The Python version is 3.9 or later. - -- The kernel version is 5.10 or later. - -- The SElinux service is disabled. - - ```shell - $ sudo setenforce 0 - $ getenforce - Permissive - ``` - - - -### Obtaining the Installation Package - -Download the openEuler release package to install and use imageTailor. - -1. Obtain the ISO image file and the corresponding verification file. - - The image must be an everything image. Assume that the image is to be stored in the **root** directory. Run the following commands: - - ```shell - $ cd /root/temp - $ wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso - $ wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso.sha256sum - ``` - -3. Obtain the verification value in the sha256sum verification file. - - ```shell - $ cat openEuler-22.03-LTS-everything-aarch64-dvd.iso.sha256sum - ``` - -4. Calculate the verification value of the ISO image file. - - ```shell - $ sha256sum openEuler-22.03-LTS-everything-aarch64-dvd.iso - ``` - -5. Compare the verification value in the sha256sum file with that of the ISO image. If they are the same, the file integrity is verified. Otherwise, the file integrity is damaged. You need to obtain the file again. - -### Installing imageTailor - -The following uses openEuler 22.03 LTS in AArch64 architecture as an example to describe how to install imageTailor. - -1. Ensure that openEuler 22.03 LTS (or a running environment that meets the requirements of imageTailor) has been installed on the host. - - ```shell - $ cat /etc/openEuler-release - openEuler release 22.03 LTS - ``` - -2. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum source. The following is an example of the configuration file. **baseurl** indicates the directory for mounting the ISO image. - - ```shell - [local] - name=local - baseurl=file:///root/imageTailor_mount - gpgcheck=0 - enabled=1 - ``` - -3. Run the following commands as the **root** user to mount the image to the **/root/imageTailor_mount** directory as the Yum source (ensure that the value of **baseurl** is the same as that configured in the repo file and the disk space of the directory is greater than 20 GB): - - ```shell - $ mkdir /root/imageTailor_mount - $ sudo mount -o loop /root/temp/openEuler-22.03-LTS-everything-aarch64-dvd.iso /root/imageTailor_mount/ - ``` - -4. Make the Yum source take effect. - - ```shell - $ yum clean all - $ yum makecache - ``` - -5. Install the imageTailor tool as the **root** user. - - ```shell - $ sudo yum install -y imageTailor - ``` - -6. Run the following command as the **root** user to verify that the tool has been installed successfully: - - ```shell - $ cd /opt/imageTailor/ - $ sudo ./mkdliso -h - ------------------------------------------------------------------------------------------------------------- - Usage: mkdliso -p product_name -c configpath [--minios yes|no|force] [-h] [--sec] - Options: - -p,--product Specify the product to make, check custom/cfg_yourProduct. - -c,--cfg-path Specify the configuration file path, the form should be consistent with custom/cfg_xxx - --minios Make minios: yes|no|force - --sec Perform security hardening - -h,--help Display help information - - Example: - command: - ./mkdliso -p openEuler -c custom/cfg_openEuler --sec - - help: - ./mkdliso -h - ------------------------------------------------------------------------------------------------------------- - ``` - -### Directory Description - -After imageTailor is installed, the directory structure of the tool package is as follows: - -```shell -[imageTailor] - |-[custom] - |-[cfg_openEuler] - |-[usr_file] // Stores files to be added. - |-[usr_install] //Stores hook scripts to be added. - |-[all] - |-[conf] - |-[hook] - |-[cmd.conf] // Configures the default commands and libraries used by an ISO image. - |-[rpm.conf] // Configures the list of RPM packages and drivers installed by default for an ISO image. - |-[security_s.conf] // Configures security hardening policies. - |-[sys.conf] // Configures ISO image system parameters. - |-[kiwi] // Basic configurations of imageTailor. - |-[repos] //RPM sources for obtaining the RPM packages required for creating an ISO image. - |-[security-tool] // Security hardening tool. - |-mkdliso // Executable script for creating an ISO image. -``` - -## Image Customization - -This section describes how to use the imageTailor tool to package the service RPM packages, custom files, drivers, commands, and libraries to the target ISO image. - -### Overall Process - -The following figure shows the process of using imageTailor to customize an image. - -![](./figures/flowchart.png) - -The steps are described as follows: - -- Check software and hardware environment: Ensure that the host for creating the ISO image meets the software and hardware requirements. - -- Customize service packages: Add RPM packages (including service RPM packages, commands, drivers, and library files) and files (including custom files, commands, drivers, and library files). - - - Adding service RPM packages: Add RPM packages to the ISO image as required. For details, see [Installation](#installation). - - Adding custom files: If you want to perform custom operations such as hardware check, system configuration check, and driver installation when the target ISO system is installed or started, you can compile custom files and package them to the ISO image. - - Adding drivers, commands, and library files: If the RPM package source of openEuler does not contain the required drivers, commands, or library files, you can use imageTailor to package the corresponding drivers, commands, or library files into the ISO image. - -- Configure system parameters: - - - Configuring host parameters: To ensure that the ISO image is successfully installed and started, you need to configure host parameters. - - Configuring partitions: You can configure service partitions based on the service plan and adjust system partitions. - - Configuring the network: You can set system network parameters as required, such as the NIC name, IP address, and subnet mask. - - Configuring the initial password: To ensure that the ISO image is successfully installed and started, you need to configure the initial passwords of the **root** user and GRUB. - - Configuring kernel parameters: You can configure the command line parameters of the kernel as required. - -- Configure security hardening policies. - - ImageTailor provides default security hardening policies. You can modify **security_s.conf** (in the ISO image customization phase) to perform secondary security hardening on the system based on service requirements. For details, see the [Security Hardening Guide](https://docs.openeuler.org/en/docs/22.03_LTS/docs/SecHarden/secHarden.html). - -- Create an ISO image: - - Use the imageTailor tool to create an ISO image. - -### Customizing Service Packages - -You can pack service RPM packages, custom files, drivers, commands, and library files into the target ISO image as required. - -#### Setting a Local Repo Source - -To customize an ISO image, you must set a repo source in the **/opt/imageTailor/repos/euler_base/** directory. This section describes how to set a local repo source. - -1. Download the ISO file released by openEuler. (The RPM package of the everything image released by the openEuler must be used.) - ```shell - $ cd /opt - $ wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso - ``` - -2. Create a mount directory **/opt/openEuler_repo** and mount the ISO file to the directory. - ```shell - $ sudo mkdir -p /opt/openEuler_repo - $ sudo mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /opt/openEuler_repo - mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only. - ``` - -3. Copy the RPM packages in the ISO file to the **/opt/imageTailor/repos/euler_base/** directory. - ```shell - $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base - $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base - $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base - $ sudo ls /opt/imageTailor/repos/euler_base|wc -l - 2577 - $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo - $ cd /opt/imageTailor - ``` - -#### Adding Files - -You can add files to an ISO image as required. The file types include custom files, drivers, commands, or library file. Store the files to the **/opt/imageTailor/custom/cfg_openEuler/usr_file** directory. - -##### Precautions - -- The commands to be packed must be executable. Otherwise, imageTailor will fail to pack the commands into the ISO. - -- The file stored in the **/opt/imageTailor/custom/cfg_openEuler/usr_file** directory will be generated in the root directory of the ISO. Therefore, the directory structure of the file must be a complete path starting from the root directory so that imageTailor can place the file in the correct directory. - - For example, if you want **file1** to be in the **/opt** directory of the ISO, create an **opt** directory in the **usr_file** directory and copy **file1** to the **opt** directory. For example: - - ```shell - $ pwd - /opt/imageTailor/custom/cfg_openEuler/usr_file - - $ tree - . - ├── etc - │   ├── default - │   │   └── grub - │   └── profile.d - │   └── csh.precmd - └── opt - └── file1 - - 4 directories, 3 files - ``` - -- The paths in **/opt/imageTailor/custom/cfg_openEuler/usr_file** must be real paths. For example, the paths do not contain soft links. You can run the `realpath` or `readlink -f` command to query the real path. - -- If you need to invoke a custom script in the system startup or installation phase, that is, a hook script, store the file in the **hook** directory. - -#### Adding RPM Packages - -##### Procedure - -To add RPM packages (drivers, commands, or library files) to an ISO image, perform the following steps: - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- The **rpm.conf** and **cmd.conf** files are stored in the **/opt/imageTailor/custom/cfg_openEuler/** directory. ->- The RPM package tailoring granularity below indicates **sys_cut='no'**. For details about the cutout granularity, see [Configuring Host Parameters](#configuring-host-parameters). ->- If no local repo source is configured, configure a local repo source by referring to [Setting a Local Repo Source](#setting-a-local-repo-source). -> - -1. Check whether the **/opt/imageTailor/repos/euler_base/** directory contains the RPM package to be added. - - - If yes, go to step 2. - - If no, go to step 3. -2. Configure the RPM package information in the **\** section in the **rpm.conf** file. - - For the RPM package tailoring granularity, no further action is required. - - For other tailoring granularities, go to step 4. -3. Obtain the RPM package and store it in the **/opt/imageTailor/custom/cfg_openEuler/usr_rpm** directory. If the RPM package depends on other RPM packages, store the dependency packages to this directory because the added RPM package and its dependent RPM packages must be packed into the ISO image at the same time. - - For the RPM package tailoring granularity, go to step 4. - - For other tailoring granularities, no further action is required. -4. Configure the drivers, commands, and library files to be retained in the RPM package in the **rpm.conf** and **cmd.conf** files. If there are common files to be tailored, configure them in the **\\** section in the **cmd.conf** file. - - -##### Configuration File Description - -| Operation | Configuration File| Section | -| :----------- | :----------- | :----------------------------------------------------------- | -| Adding drivers | rpm.conf | \
\
\
Note: The **driver_name** is the relative path of **/lib/modules/{kernel_version_number}/kernel/**.| -| Adding commands | cmd.conf | \
\
\ | -| Adding library files | cmd.conf | \
\
\ | -| Deleting other files| cmd.conf | \
\
\
Note: The file name must be an absolute path.| - -**Example** - -- Adding drivers - - ```shell - - - - - ...... - - ``` - -- Adding commands - - ```shell - - - - - ...... - - ``` - -- Adding library files - - ```shell - - - - - - ``` - -- Deleting other files - - ```shell - - - - - - ``` - -#### Adding Hook Scripts - -A hook script is invoked by the OS during startup and installation to execute the actions defined in the script. The directory for storing hook scripts of imageTailor is **custom/cfg_openEuler/usr_install/hook directory**, which has different subdirectories. Each subdirectory represents an OS startup or installation phase. Store the scripts based on the phases in which the scripts are invoked. - -##### Script Naming Rule - -The script name must start with **S+number** (the number must be at least two digits). The number indicates the execution sequence of the hook script. Example: **S01xxx.sh** - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->The scripts in the **hook** directory are executed using the `source` command. Therefore, exercise caution when using the `exit` command in the scripts because the entire installation script exits after the `exit` command is executed. - - - -##### Description of hook Subdirectories - -| Subdirectory | Script Example | Time for Execution | Description | -| :-------------------- | :---------------------| :------------------------------- | :----------------------------------------------------------- | -| insmod_drv_hook | N/A | After OS drivers are loaded | N/A | -| custom_install_hook | S01custom_install.sh | After the drivers are loaded, that is, after **insmod_drv_hook** is executed| You can customize the OS installation process by using a custom script.| -| env_check_hook | S01check_hw.sh | Before the OS installation initialization | The script is used to check hardware specifications and types before initialization.| -| set_install_ip_hook | S01set_install_ip.sh | When network configuration is being performed during OS installation initialization. | You can customize the network configuration by using a custom script.| -| before_partition_hook | S01checkpart.sh | Before partitioning | You can check correctness of the partition configuration file by using a custom script.| -| before_setup_os_hook | N/A | Before the repo file is decompressed | You can customize partition mounting.
If the decompression path of the installation package is not the root partition specified in the partition configuration, customize partition mounting and assign the decompression path to the input global variable.| -| before_mkinitrd_hook | S01install_drv.sh | Before the `mkinitrd` command is run | The hook script executed before running the `mkinitrd` command when **initrd** is saved to the disk. You can add and update driver files in **initrd**.| -| after_setup_os_hook | N/A | After OS installation | After the installation is complete, you can perform custom operations on the system files, such as modifying **grub.cfg**.| -| install_succ_hook | N/A | When the OS is successfully installed | The scripts in this subdirectory are used to parse the installation information and send information of whether the installation succeeds.**install_succ_hook** cannot be set to **install_break**.| -| install_fail_hook | N/A | When the OS installation fails | The scripts in this subdirectory are used to parse the installation information and send information of whether the installation succeeds.**install_fail_hook** cannot be set to **install_break**.| - -### Configuring System Parameters - -Before creating an ISO image, you need to configure system parameters, including host parameters, initial passwords, partitions, network, compilation parameters, and system command line parameters. - -#### Configuring Host Parameters - - The **\ \** section in the **/opt/imageTailor/custom/cfg_openEuler/sys.conf** file is used to configure common system parameters, such as the host name and kernel boot parameters. - -The default configuration provided by openEuler is as follows. You can modify the configuration as required. - -```shell - - sys_service_enable='ipcc' - sys_service_disable='cloud-config cloud-final cloud-init-local cloud-init' - sys_utc='yes' - sys_timezone='' - sys_cut='no' - sys_usrrpm_cut='no' - sys_hostname='Euler' - sys_usermodules_autoload='' - sys_gconv='GBK' - -``` - -The parameters are described as follows: - -- sys_service_enable - - This parameter is optional. Services enabled by the OS by default. Separate multiple services with spaces. If you do not need to add a system service, use the default value **ipcc**. Pay attention to the following during the configuration: - - - Default system services cannot be deleted. - - You can configure service-related services, but the repo source must contain the service RPM package. - - By default, only the services configured in this parameter are enabled. If a service depends on other services, you need to configure the depended services in this parameter. - -- sys_service_disable - - This parameter is optional. Services that are not allowed to automatically start upon system startup. Separate multiple services with spaces. If no system service needs to be disabled, leave this parameter blank. - -- sys_utc - - (Mandatory) Indicates whether to use coordinated universal time (UTC) time. The value can be **yes** or **no**. The default value is **yes**. - -- sys_timezone - - This parameter is optional. Sets the time zone. The value can be a time zone supported by openEuler, which can be queried in the **/usr/share/zoneinfo/zone.tab** file. - -- sys_cut - - (Mandatory) Indicates whether to tailor the RPM packages. The value can be **yes**, **no**, or **debug**.**yes** indicates that the RPM packages are tailored. **no** indicates that the RPM packages are not tailored (only the RPM packages in the **rpm.conf** file is installed). **debug** indicates that the RPM packages are tailored but the `rpm` command is retained for customization after installation. The default value is **no**. - - >![](./public_sys-resources/icon-note.gif) NOTE: - > - > - imageTailor installs the RPM package added by the user, deletes the files configured in the **\** section of the **cmd.conf** file, and then deletes the commands, libraries, and drivers that are not configured in **cmd.conf** or **rpm.conf**. - > - When **sys_cut='yes'** is configured, imageTailor does not support the installation of the `rpm` command. Even if the `rpm` command is configured in the **rpm.conf** file, the configuration does not take effect. - -- sys_usrrpm_cut - - (Mandatory) Indicates whether to tailor the RPM packages added by users to the **/opt/imageTailor/custom/cfg_openEuler/usr_rpm** directory. The value can be **yes** or **no**. The default value is **no**. - - - **sys_usrrpm_cut='yes'**: imageTailor installs the RPM packages added by the user, deletes the file configured in the **\** section in the **cmd.conf** file, and then deletes the commands, libraries, and drivers that are not configured in **cmd.conf** or **rpm.conf**. - - - **sys_usrrpm_cut='no'**: imageTailor installs the RPM packages added by the user but does not delete the files in the RPM packages. - -- sys_hostname - - (Mandatory) Host name. After the OS is deployed in batches, you are advised to change the host name of each node to ensure that the host name of each node is unique. - - The host name must be a combination of letters, digits, and hyphens (-) and must start with a letter or digit. Letters are case sensitive. The value contains a maximum of 63 characters. The default value is **Euler**. - -- sys_usermodules_autoload - - (Optional) Driver loaded during system startup. When configuring this parameter, you do not need to enter the file extension **.ko**. If there are multiple drivers, separate them by space. By default, this parameter is left blank, indicating that no additional driver is loaded. - -- sys_gconv - - (Optional) This parameter is used to tailor **/usr/lib/gconv** and **/usr/lib64/gconv**. The options are as follows: - - - **null**/**NULL**: indicates that this parameter is not configured. If **sys_cut='yes'** is configured, **/usr/lib/gconv** and **/usr/lib64/gconv** will be deleted. - - **all**/**ALL**: keeps **/usr/lib/gconv** and **/usr/lib64/gconv**. - - **xxx,xxx**: keeps the corresponding files in the **/usr/lib/gconv** and **/usr/lib64/gconv** directories. If multiple files need to be kept, use commas (,) to separate them. - -- sys_man_cut - - (Optional) Indicates whether to tailor the man pages. The value can be **yes** or **no**. The default value is **yes**. - - - ->![](./public_sys-resources/icon-note.gif) NOTE: -> -> If both **sys_cut** and **sys_usrrpm_cut** are configured, **sys_cut** is used. The following rules apply: -> -> - sys_cut='no' -> -> No matter whether **sys_usrrpm_cut** is set to **yes** or **no**, the system RPM package tailoring granularity is used. That is, imageTailor installs the RPM packages in the repo source and the RPM packages in the **usr_rpm** directory, however, the files in the RPM package are not deleted. Even if some files in the RPM packages are not required, imageTailor will delete them. -> -> - sys_cut='yes' -> -> - sys_usrrpm_cut='no' -> -> System RPM package tailoring granularity: imageTailor deletes files in the RPM packages in the repo sources as configured. -> -> - sys_usrrpm_cut='yes' -> -> System and user RPM package tailoring granularity: imageTailor deletes files in the RPM packages in the repo sources and the **usr_rpm** directory as configured. -> - - - -#### Configuring Initial Passwords - -The **root** and GRUB passwords must be configured during OS installation. Otherwise, you cannot log in to the OS as the **root** user after the OS is installed using the tailored ISO image. This section describes how to configure the initial passwords. - -> ![](./public_sys-resources/icon-note.gif) NOTE: -> -> You must configure the initial **root** and GRUB passwords manually. - -##### Configuring the Initial Password of the root User - -###### Introduction - -The initial password of the **root** user is stored in the **/opt/imageTailor/custom/cfg_openEuler/rpm.conf** file. You can modify this file to set the initial password of the **root** user. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- If the `--minios yes/force` parameter is required when you run the `mkdliso` command to create an ISO image, you need to enter the corresponding information in the **/opt/imageTailor/kiwi/minios/cfg_minios/rpm.conf** file. - -The default configuration of the initial password of the **root** user in the **/opt/imageTailor/custom/cfg_openEuler/rpm.conf** file is as follows. Add a password of your choice. - -``` - - - -``` - -The parameters are described as follows: - -- **group**: group to which the user belongs. -- **pwd**: ciphertext of the initial password. The encryption algorithm is SHA-512. Replace **${pwd}** with the actual ciphertext. -- **home**: home directory of the user. -- **name**: name of the user to be configured. - -###### Modification Method - -Before creating an ISO image, you need to change the initial password of the **root** user. The following describes how to set the initial password of the **root** user (**root** permissions are required): - -1. Add a user for generating a password, for example, **testUser**. - - ```shell - $ sudo useradd testUser - ``` - -2. Set the password of **testUser**. Run the following command and set the password as prompted: - - ```shell - $ sudo passwd testUser - Changing password for user testUser. - New password: - Retype new password: - passwd: all authentication tokens updated successfully. - ``` - -3. View the **/etc/shadow** file. The content following **testUser** (string between two colons) is the ciphertext of the password. - - ``` shell script - $ sudo cat /etc/shadow | grep testUser - testUser:$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0:19052:0:90:7:35:: - ``` - -4. Copy and paste the ciphertext to the **pwd** field in the **/opt/imageTailor/custom/cfg_openEuler/rpm.conf** file. - ``` shell script - - - - ``` - -5. If the `--minios yes/force` parameter is required when you run the `mkdliso` command to create an ISO image, configure the **pwd** field of the corresponding user in **/opt/imageTailor/kiwi/minios/cfg_minios/rpm.conf**. - - ``` shell script - - - - ``` - -##### Configuring the Initial GRUB Password - -The initial GRUB password is stored in the **/opt/imageTailor/custom/cfg_openEuler/usr_file/etc/default/grub** file. Modify this file to configure the initial GRUB password. If the initial GRUB password is not configured, the ISO image will fail to be created. - -> ![](./public_sys-resources/icon-note.gif) NOTE: -> -> - The **root** permissions are required for configuring the initial GRUB password. -> - The default user corresponding to the GRUB password is **root**. -> -> - The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance. - -1. Run the following command and set the GRUB password as prompted: - - ```shell - $ sudo grub2-set-password -o ./ - Enter password: - Confirm password: - grep: .//grub.cfg: No such file or directory - WARNING: The current configuration lacks password support! - Update your configuration with grub2-mkconfig to support this feature. - ``` - -2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB password. - - ```shell - $ sudo cat user.cfg - GRUB2_PASSWORD=grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1 - B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02 - 4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615 - ``` - -3. Copy the preceding ciphertext and add the following configuration to the **/opt/imageTailor/custom/cfg_openEuler/usr_file/etc/default/grub** file: - - ```shell - GRUB_PASSWORD="grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1 - B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02 - 4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615" - ``` - - -#### Configuring Partitions - -If you want to adjust system partitions or service partitions, modify the **\** section in the **/opt/imageTailor/custom/cfg_openEuler/sys.conf** file. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- System partition: partition for storing the OS. ->- Service partition: partition for service data. ->- The type of a partition is determined by the content it stores, not the size, mount path, or file system. ->- Partition configuration is optional. You can manually configure partitions after OS installation. - - The format of **\** is as follows: - -disk_ID mount _path partition _size partition_type file_system [Secondary formatting flag] - -The default configuration is as follows: - -``` shell script - -hd0 /boot 512M primary ext4 yes -hd0 /boot/efi 200M primary vfat yes -hd0 / 30G primary ext4 -hd0 - - extended - -hd0 /var 1536M logical ext4 -hd0 /home max logical ext4 - -``` - -The parameters are described as follows: - -- disk_ID: - ID of a disk. Set this parameter in the format of **hd***x*, where *x* indicates the *x*th disk. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >Partition configuration takes effect only when the disk can be recognized. - -- mount_path: - Mount path to a specified partition. You can configure service partitions and adjust the default system partition. If you do not mount partitions, set this parameter to **-**. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- You must configure the mount path to **/**. You can adjust mount paths to other partitions according to your needs. - >- When the UEFI boot mode is used, the partition configuration in the x86_64 architecture must contain the mount path **/boot**, and the partition configuration in the AArch64 architecture must contain the mount path **/boot/efi**. - -- partition_size: - The value types are as follows: - - - G/g: The unit of a partition size is GB, for example, 2G. - - M/m: The unit of a partition size is MB, for example, 300M. - - T/t: The unit of a partition size is TB, for example, 1T. - - MAX/max: The rest space of a hard disk is used to create a partition. This value can only be assigned to the last partition. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** -> - >- A partition size value cannot contain decimal numbers. If there are decimal numbers, change the unit of the value to make the value an integer. For example, 1.5 GB should be changed to 1536 MB. - >- When the partition size is set to **MAX**/**max**, the size of the remaining partition cannot exceed the limit of the supported file system type (the default file system type is **ext4**, and the maximum size is **16T**). - -- partition_type: - The values of partition types are as follows: - - - primary: primary partitions - - extended: extended partition (configure only *disk_ID* for this partition) - - logical: logical partitions - -- file_system: - Currently, **ext4** and **vfat** file systems are supported. - -- [Secondary formatting flag]: - Indicates whether to format the disk during secondary installation. This parameter is optional. - - - The value can be **yes** or **no**. The default value is **no**. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >Secondary formatting indicates that openEuler has been installed on the disk before this installation. If the partition table configuration (partition size, mount point, and file type) used in the previous installation is the same as that used in the current installation, this flag can be used to configure whether to format the previous partitions, except the **/boot** and **/** partitions. If the target host is installed for the first time, this flag does not take effect, and all partitions with specified file systems are formatted. - -#### Configuring the Network - -The system network parameters are stored in **/opt/imageTailor/custom/cfg_openEuler/sys.conf**. You can modify the network parameters of the target ISO image, such as the NIC name, IP address, and subnet mask, by configuring **\\** in this file. - -The default network configuration in the **sys.conf** file is as follows. **netconfig-0** indicates the **eth0** NIC. If you need to configure an additional NIC, for example, **eth1**, add **\\** to the configuration file and set the parameters of **eth1**. - -```shell - -BOOTPROTO="dhcp" -DEVICE="eth0" -IPADDR="" -NETMASK="" -STARTMODE="auto" - -``` - -The following table describes the parameters. - -- | Parameter | Mandatory or Not| Value | Description | - | :-------- | -------- | :------------------------------------------------ | :----------------------------------------------------------- | - | BOOTPROTO | Yes | none / static / dhcp | **none**: No protocol is used for boot, and no IP address is assigned.
**static**: An IP address is statically assigned.
**dhcp**: An IP address is dynamically obtained using the dynamic host configuration protocol (DHCP).| - | DEVICE | Yes | Example: **eth1** | NIC name. | - | IPADDR | Yes | Example: **192.168.11.100** | IP address.
This parameter must be configured only when the value of **BOOTPROTO** is **static**.| - | NETMASK | Yes | - | Subnet mask.
This parameter must be configured only when the value of **BOOTPROTO** is **static**.| - | STARTMODE | Yes | manual / auto / hotplug / ifplugd / nfsroot / off | NIC start mode.
**manual**: A user runs the `ifup` command on a terminal to start an NIC.
**auto**/**hotplug**/**ifplug**/**nfsroot**: An NIC is started when the OS identifies it.
**off**: An NIC cannot be started in any situations.
For details about the parameters, run the `man ifcfg` command on the host that is used to create the ISO image.| - - -#### Configuring Kernel Parameters - -To ensure stable and efficient running of the system, you can modify kernel command line parameters as required. For an OS image created by imageTailor, you can modify the **GRUB_CMDLINE_LINUX** configuration in the **/opt/imageTailor/custom/cfg_openEuler/usr_file/etc/default/grub** file to modify the kernel command line parameters. The default settings of the kernel command line parameters in **GRUB_CMDLINE_LINUX** are as follows: - -```shell -GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0 crashkernel=512M oops=panic softlockup_panic=1 reserve_kbox_mem=16M crash_kexec_post_notifiers panic=3 console=tty0" -``` - -The meanings of the configurations are as follows (for details about other common kernel command line parameters, see related kernel documents): - -- net.ifnames=0 biosdevname=0 - - Name the NIC in traditional mode. - -- crashkernel=512M - - The memory space reserved for kdump is 512 MB. - -- oops=panic panic=3 - - The kernel panics when an oops error occurs, and the system restarts 3 seconds later. - -- softlockup_panic=1 - - The kernel panics when a soft-lockup is detected. - -- reserve_kbox_mem=16M - - The memory space reserved for Kbox is 16 MB. - -- console=tty0 - - Specifies **tty0** as the output device of the first virtual console. - -- crash_kexec_post_notifiers - - After the system crashes, the function registered with the panic notification chain is called first, and then kdump is executed. - -### Creating an Image - -After customizing the operating system, you can use the `mkdliso` script to create the OS image file. The OSimage created using imageTailor is an ISO image file. - -#### Command Description - -##### Syntax - -**mkdliso -p openEuler -c custom/cfg_openEuler [--minios yes|no|force] [--sec] [-h]** - -##### Parameter Description - -| Parameter| Mandatory| Description | Value Range | -| -------- | -------- | ------------------------------------------------------------ | ------------------------------------------------------------ | -| -p | Yes | Specifies the product name. | **openEuler** | -| c | Yes | Specifies the relative path of the configuration file. | **custom/cfg_openEuler** | -| --minios | No | Specifies whether to create the **initrd** file that is used to boot the system during system installation. | The default value is **yes**.
**yes**: The **initrd** file will be created when the command is executed for the first time. When a subsequent `mkdliso` is executed, the system checks whether the **initrd** file exists in the **usr_install/boot** directory using sha256 verification. If the **initrd** file exists, it is not created again. Otherwise, it is created.
**no**: The **initrd** file is not created. The **initrd** file used for system boot and running is the same.
**force**: The **initrd** file will be created forcibly, regardless of whether it exists in the **usr_install/boot** directory or not.| -| --sec | No | Specifies whether to perform security hardening on the generated ISO file.
If this parameter is not specified, the user should undertake the resultant security risks| N/A | -| -h | No | Obtains help information. | N/A | - -#### Image Creation Guide - -To create an ISO image using`mkdliso`, perform the following steps: - ->![](./public_sys-resources/icon-note.gif) NOTE: -> -> - The absolute path to `mkdliso` must not contain spaces. Otherwise, the ISO image creation will fail. -> - In the environment for creating the ISO image, the value of **umask** must be set to **0022**. - -1. Run the `mkdliso` command as the **root** user to generate the ISO image file. The following command is used for reference: - - ```shell - # sudo /opt/imageTailor/mkdliso -p openEuler -c custom/cfg_openEuler --sec - ``` - - After the command is executed, the created files are stored in the **/opt/imageTailor/result/{date}** directory, including **openEuler-aarch64.iso** and **openEuler-aarch64.iso.sha256**. - -2. Verify the integrity of the ISO image file. Assume that the date and time is **2022-03-21-14-48**. - - ```shell - $ cd /opt/imageTailor/result/2022-03-21-14-48/ - $ sha256sum -c openEuler-aarch64.iso.sha256 - ``` - - If the following information is displayed, the ISO image creation is complete. - - ``` - openEuler-aarch64.iso: OK - ``` - - If the following information is displayed, the image is incomplete. The ISO image file is damaged and needs to be created again. - - ```shell - openEuler-aarch64.iso: FAILED - sha256sum: WARNING: 1 computed checksum did NOT match - ``` - -3. View the logs. - - After an image is created, you can view logs as required (for example, when an error occurs during image creation). When an image is created for the first time, the corresponding log file and security hardening log file are compressed into a TAR package (the log file is named in the format of **sys_custom_log_{Date}.tar.gz**) and stored in the **result/log directory**. Only the latest 50 compressed log packages are stored in this directory. If the number of compressed log packages exceeds 50, the earliest files will be overwritten. - - - -### Tailoring Time Zones - -After the customized ISO image is installed, you can tailor the time zones supported by the openEuler system as required. This section describes how to tailor the time zones. - -The information about time zones supported by openEuler is stored in the time zone folder **/usr/shre/zoninfo**. You can run the following command to view the time zone information: - -```shell -$ ls /usr/share/zoneinfo/ -Africa/ America/ Asia/ Atlantic/ Australia/ Etc/ Europe/ -Pacific/ zone.tab -``` - -Each subfolder represents an area. The current areas include continents, oceans, and **Etc**. Each area folder contains the locations that belong to it. Generally, a location is a city or an island. - -All time zones are in the format of *area/location*. For example, if China Standard Time is used in southern China, the time zone is Asia/Shanghai (location may not be the capital). The corresponding time zone file is: - -``` -/usr/share/zoneinfo/Asia/Shanghai -``` - -If you want to tailor some time zones, delete the corresponding time zone files. - -### Customization Example - -This section describes how to use imageTailor to create an ISO image. - -1. Check whether the environment used to create the ISO meets the requirements. - - ``` shell - $ cat /etc/openEuler-release - openEuler release 22.03 LTS - ``` - -2. Ensure that the root directory has at least 40 GB free space. - - ```shell - $ df -h - Filesystem Size Used Avail Use% Mounted on - ...... - /dev/vdb 196G 28K 186G 1% / - ``` - -3. Install the imageTailor tailoring tool. For details, see [Installation](#installation). - - ```shell - $ sudo yum install -y imageTailor - $ ll /opt/imageTailor/ - total 88K - drwxr-xr-x. 3 root root 4.0K Mar 3 08:00 custom - drwxr-xr-x. 10 root root 4.0K Mar 3 08:00 kiwi - -r-x------. 1 root root 69K Mar 3 08:00 mkdliso - drwxr-xr-x. 2 root root 4.0K Mar 9 14:48 repos - drwxr-xr-x. 2 root root 4.0K Mar 9 14:48 security-tool - ``` - -4. Configure a local repo source. - - ```shell - $ wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso - $ sudo mkdir -p /opt/openEuler_repo - $ sudo mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /opt/openEuler_repo - mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only. - $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base - $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base - $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base - $ sudo ls /opt/imageTailor/repos/euler_base|wc -l - 2577 - $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo - $ cd /opt/imageTailor - ``` - -5. Change the **root** and GRUB passwords. - - Replace **\${pwd}** with the encrypted password by referring to [Configuring Initial Passwords](#configuring-initial-passwords). - - ```shell - $ cd /opt/imageTailor/ - $ sudo vi custom/cfg_openEuler/usr_file/etc/default/grub - GRUB_PASSWORD="${pwd1}" - $ - $ sudo vi kiwi/minios/cfg_minios/rpm.conf - - - - $ - $ sudo vi custom/cfg_openEuler/rpm.conf - - - - ``` - -6. Run the tailoring command. - - ```shell - $ sudo rm -rf /opt/imageTailor/result - $ sudo ./mkdliso -p openEuler -c custom/cfg_openEuler --minios force - ...... - Complete release iso file at: result/2022-03-09-15-31/openEuler-aarch64.iso - move all mkdliso log file to result/log/sys_custom_log_20220309153231.tar.gz - $ ll result/2022-03-09-15-31/ - total 889M - -rw-r--r--. 1 root root 889M Mar 9 15:32 openEuler-aarch64.iso - -rw-r--r--. 1 root root 87 Mar 9 15:32 openEuler-aarch64.iso.sha256 - ``` diff --git a/docs/en/docs/TailorCustom/isocut-user-guide.md b/docs/en/docs/TailorCustom/isocut-user-guide.md deleted file mode 100644 index dbafbe87057fcc05f8e6eb95b7f28466f38fcf0f..0000000000000000000000000000000000000000 --- a/docs/en/docs/TailorCustom/isocut-user-guide.md +++ /dev/null @@ -1,760 +0,0 @@ -# isocut User Guide - -- [Introduction](#introduction) -- [Software and Hardware Requirements](#software-and-hardware-requirements) -- [Installation](#Installation) -- [Tailoring and Customizing an Image](#tailoring-and-customizing-an-image) - - [Command Description](#command-description) - - [Software Package Source](#software-package-source) - - [Operation Guide](#operation-guide) - - -## Introduction -The size of an openEuler image is large, and the process of downloading or transferring an image is time-consuming. In addition, when an openEuler image is used to install the OS, all RPM packages contained in the image are installed. You cannot choose to install only the required software packages. - -In some scenarios, you do not need to install the full software package provided by the image, or you need to install additional software packages. Therefore, openEuler provides an image tailoring and customization tool. You can use this tool to customize an ISO image that contains only the required RPM packages based on an openEuler image. The software packages can be the ones contained in an official ISO image or specified in addition to meet custom requirements. - -This document describes how to install and use the openEuler image tailoring and customization tool. - -## Software and Hardware Requirements - -The hardware and software requirements of the computer to make an ISO file using the openEuler tailoring and customization tool are as follows: - -- The CPU architecture is AArch64 or x86_64. -- The operating system is openEuler 20.03 LTS SP3. -- You are advised to reserve at least 30 GB drive space for running the tailoring and customization tool and storing the ISO image. - -## Installation - -The following uses openEuler 20.03 LTS SP3 on the AArch64 architecture as an example to describe how to install the ISO image tailoring and customization tool. - -1. Ensure that openEuler 20.03 LTS SP3 has been installed on the computer. - - ``` shell script - $ cat /etc/openEuler-release - openEuler release 20.03 (LTS-SP3) - ``` - -2. Download the ISO image (must be an **everything** image) of the corresponding architecture and save it to any directory (it is recommended that the available space of the directory be greater than 20 GB). In this example, the ISO image is saved to the **/home/isocut_iso** directory. - - The download address of the AArch64 image is as follows: - - https://repo.openeuler.org/openEuler-20.03-LTS-SP3/ISO/aarch64/openEuler-20.03-LTS-SP3-everything-aarch64-dvd.iso - - > **Note:** - > The download address of the x86_64 image is as follows: - > - > https://repo.openeuler.org/openEuler-20.03-LTS-SP3/ISO/x86_64/openEuler-20.03-LTS-SP3-everything-x86_64-dvd.iso - -3. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum source. The following is an example of the configuration file. **baseurl** is the directory for mounting the ISO image. - - ``` shell script - [local] - name=local - baseurl=file:///home/isocut_mount - gpgcheck=0 - enabled=1 - ``` - -4. Run the following command as the **root** user to mount the image to the **/home/isocut_mount** directory (ensure that the mount directory is the same as **baseurl** configured in the **repo** file) as the Yum source: - - ```shell - sudo mount -o loop /home/isocut_iso/openEuler-20.03-LTS-SP3-everything-aarch64-dvd.iso /home/isocut_mount - ``` - -5. Make the Yum source take effect. - - ```shell - yum clean all - yum makecache - ``` - -6. Install the image tailoring and customization tool as the **root** user. - - ```shell - sudo yum install -y isocut - ``` - -7. Run the following command as the **root** user to verify that the tool has been installed successfully: - - ```shell - $ sudo isocut -h - Checking input ... - usage: isocut [-h] [-t temporary_path] [-r rpm_path] [-k file_path] source_iso dest_iso - - Cut openEuler iso to small one - - positional arguments: - source_iso source iso image - dest_iso destination iso image - - optional arguments: - -h, --help show this help message and exit - -t temporary_path temporary path - -r rpm_path extern rpm packages path - -k file_path kickstart file - ``` - - - -## Tailoring and Customizing an Image - -This section describes how to use the image tailoring and customization tool to create an image by tailoring or adding RPM packages to an openEuler image. - -### Command Description - -#### Format - -Run the `isocut` command to use the image tailoring and customization tool. The command format is as follows: - -**isocut** [ --help | -h ] [ -t <*temp_path*> ] [ -r <*rpm_path*> ] [ -k <*file_path*> ] < *source_iso* > < *dest_iso* > - -#### Parameter Description - -| Parameter| Mandatory| Description| -| ------------ | -------- | -------------------------------------------------------- | -| --help \| -h | No| Queries the help information about the command.| -| -t <*temp_path*> | No| Specifies the temporary directory *temp_path* for running the tool, which is an absolute path. The default value is **/tmp**.| -| -r <*rpm_path*> | No| Specifies the path of the RPM packages to be added to the ISO image.| -| -k <*file_path*> | No | Specifies the kickstart template path if kickstart is used for automatic installation. | -| *source_iso* | Yes| Path and name of the ISO source image to be tailored. If no path is specified, the current path is used by default.| -| *dest_iso* | Yes| Specifies the path and name of the new ISO image created by the tool. If no path is specified, the current path is used by default.| - - - -### Software Package Source - -The RPM packages of the new image can be: - -- Packages contained in an official ISO image. In this case, the RPM packages to be installed are specified in the configuration file **/etc/isocut/rpmlist**. The configuration format is *software_package_name.architecture*. For example, **kernel.aarch64**. - -- Specified in addition. In this case, use the `-r` parameter to specify the path in which the RPM packages are stored when running the `isocut` command and add the RPM package names to the **/etc/isocut/rpmlist** configuration file. (See the name format above.) - - - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- When customizing an image, if an RPM package specified in the configuration file cannot be found, the RPM package will not be added to the image. - >- If the dependency of the RPM package is incorrect, an error may be reported when running the tailoring and customization tool. - -### kickstart Functions - -You can use kickstart to install images automatically by using the `-k` parameter to specify a kickstart file when running the **isocut** command. - -The isocut tool provides a kickstart template (**/etc/isocut/anaconda-ks.cfg**). You can modify the template as required. - -#### Modifying the kickstart Template - -If you need to use the kickstart template provided by the isocut tool, perform the following modifications: - -- Configure the root user password and the GRUB2 password in the **/etc/isocut/anaconda-ks.cfg** file. Otherwise, the automatic image installation will pause during the password setting process, waiting for you to manually enter the passwords. -- If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file. - -See the next section for details about how to modify the kickstart file. - -##### Configuring Initial Passwords - -###### Setting the Initial Password of the **root** User - -Set the initial password of the **root** user as follows in the **/etc/isocut/anaconda-ks.cfg** file. Replace **${pwd}** with the encrypted password. - -```shell -rootpw --iscrypted ${pwd} -``` - -Obtain the initial password of the **root** user as follows (**root** permissions are required): - -1. Add a user for generating the password, for example, **testUser**. - - ``` shell script - $ sudo useradd testUser - ``` - -2. Set the password for the **testUser** user. Run the following command to set the password as prompted: - - ``` shell script - $ sudo passwd testUser - Changing password for user testUser. - New password: - Retype new password: - passwd: all authentication tokens updated successfully. - ``` - -3. View the **/etc/shadow** file to obtain the encrypted password. The encrypted password is the string between the two colons (:) following the **testUser** user name. (******* is used as an example.) - - ``` shell script - $ sudo cat /etc/shadow | grep testUser - testUser:***:19052:0:90:7:35:: - ``` - -4. Run the following command to replace the **pwd** field in the **/etc/isocut/anaconda-ks.cfg** file with the encrypted password (replace __***__ with the actual password): - ``` shell script - rootpw --iscrypted *** - ``` - -###### Configuring the Initial GRUB2 Password - -Add the following configuration to the **/etc/isocut/anaconda-ks.cfg** file to set the initial GRUB2 password: Replace **${pwd}** with the encrypted password. - -```shell -%addon com_huawei_grub_safe --iscrypted --password='${pwd}' -%end -``` - -> ![](./public_sys-resources/icon-note.gif) NOTE: -> -> - The **root** permissions are required for configuring the initial GRUB password. -> - The default user corresponding to the GRUB password is **root**. -> -> - The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance. - -1. Run the following command and set the GRUB2 password as prompted: - - ```shell - $ sudo grub2-set-password -o ./ - Enter password: - Confirm password: - grep: .//grub.cfg: No such file or directory - WARNING: The current configuration lacks password support! - Update your configuration with grub2-mkconfig to support this feature. - ``` - -2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB2 password. - - ```shell - $ sudo cat user.cfg - GRUB2_PASSWORD=grub.pbkdf2.sha512.*** - ``` - -3. Add the following information to the **/etc/isocut/anaconda-ks.cfg** file. Replace ******* with the encrypted GRUB2 password. - - ```shell - %addon com_huawei_grub_safe --iscrypted --password='grub.pbkdf2.sha512.***' - %end - ``` - -##### Configuring the %packages Field - -If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file. - -This section describes how to specify RPM packages in the **/etc/isocut/anaconda-ks.cfg** file. - -The default configurations of **%packages** in the **/etc/isocut/anaconda-ks.cfg** file are as follows: - -```shell -%packages --multilib --ignoremissing -acl.aarch64 -aide.aarch64 -...... -NetworkManager.aarch64 -%end -``` - -Add specified RPM packages to the **%packages** configurations in the following format: - -*software_package_name.architecture*. For example, **kernel.aarch64**. - -```shell -%packages --multilib --ignoremissing -acl.aarch64 -aide.aarch64 -...... -NetworkManager.aarch64 -kernel.aarch64 -%end -``` - -### Operation Guide - - - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- Do not modify or delete the default configuration items in the **/etc/isocut/rpmlist** file. ->- All `isocut` operations require **root** permissions. ->- The source image to be tailored can be a basic image or **everything** image. In this example, the basic image **openEuler-20.03-LTS-SP3-aarch64-dvd.iso** is used. ->- In this example, assume that the new image is named **new.iso** and stored in the **/home/result** directory, the temporary directory for running the tool is **/home/temp**, and the additional RPM packages are stored in the **/home/rpms** directory. - - - -1. Open the configuration file **/etc/isocut/rpmlist** and specify the RPM packages to be installed (from the official ISO image). - - ``` shell script - sudo vi /etc/isocut/rpmlist - ``` - -2. Ensure that the space of the temporary directory for running the image tailoring and customization tool is greater than 8 GB. The default temporary directory is** /tmp**. You can also use the `-t` parameter to specify another directory as the temporary directory. The path of the directory must be an absolute path. In this example, the **/home/temp** directory is used. The following command output indicates that the available drive space of the **/home** directory is 38 GB, which meets the requirements. - - ```shell - $ df -h - Filesystem Size Used Avail Use% Mounted on - devtmpfs 1.2G 0 1.2G 0% /dev - tmpfs 1.5G 0 1.5G 0% /dev/shm - tmpfs 1.5G 23M 1.5G 2% /run - tmpfs 1.5G 0 1.5G 0% /sys/fs/cgroup - /dev/mapper/openeuler_openeuler-root 69G 2.8G 63G 5% / - /dev/sda2 976M 114M 796M 13% /boot - /dev/mapper/openeuler_openeuler-home 61G 21G 38G 35% /home - ``` - -3. Tailor and customize the image. - - **Scenario 1**: All RPM packages of the new image are from the official ISO image. - - ``` shell script - $ sudo isocut -t /home/temp /home/isocut_iso/openEuler-20.03-LTS-SP3-aarch64-dvd.iso /home/result/new.iso - Checking input ... - Checking user ... - Checking necessary tools ... - Initing workspace ... - Copying basic part of iso image ... - Downloading rpms ... - Finish create yum conf - finished - Regenerating repodata ... - Checking rpm deps ... - Getting the description of iso image ... - Remaking iso ... - Adding checksum for iso ... - Adding sha256sum for iso ... - ISO cutout succeeded, enjoy your new image "/home/result/new.iso" - isocut.lock unlocked ... - ``` - If the preceding information is displayed, the custom image **new.iso** is successfully created. - - **Scenario 2**: The RPM packages of the new image are from the official ISO image and additional packages in **/home/rpms**. - - ```shell - sudo isocut -t /home/temp -r /home/rpms /home/isocut_iso/openEuler-20.03-LTS-SP3-aarch64-dvd.iso /home/result/new.iso - ``` - **Scenario 3**: The kickstart file is used for automatic installation. You need to modify the **/etc/isocut/anaconda-ks.cfg** file. - ```shell - sudo isocut -t /home/temp -k /etc/isocut/anaconda-ks.cfg /home/isocut_iso/openEuler-20.03-LTS-SP3-aarch64-dvd.iso /home/result/new.iso - ``` - - -## FAQs - -### The System Fails to Be Installed Using an Image Tailored Based on the Default RPM Package List - -#### Context - -When isocut is used to tailor an image, the **/etc/isocut/rpmlist** configuration file is used to specify the software packages to be installed. - -Images of different OS versions contain different software packages. As a result, some packages may be missing during image tailoring. -Therefore, the **/etc/isocut/rpmlist** file contains only the kernel software package by default, -ensuring that the image can be successfully tailored. - -#### Symptom - -The ISO image is successfully tailored using the default configuration, but fails to be installed. - -An error message is displayed during the installation, indicating that packages are missing: - -![](./figures/lack_pack.png) - -#### Possible Cause - -The ISO image tailored based on the default RPM package list lacks necessary RPM packages during installation. -The missing RPM packages are displayed in the error message, and may vary depending on the version. - -#### Solution - -1. Add the missing packages. - - 1. Find the missing RPM packages based on the error message. - 2. Add the missing RPM packages to the **/etc/isocut/rpmlist** configuration file. - 3. Tailor and install the ISO image again. - - For example, if the missing packages are those in the example error message, modify the **rpmlist** configuration file as follows: - ```shell - $ cat /etc/isocut/rpmlist - kernel.aarch64 - lvm2.aarch64 - chrony.aarch64 - authselect.aarch64 - shim.aarch64 - efibootmgr.aarch64 - grub2-efi-aa64.aarch64 - dosfstools.aarch64 - ``` -# isocut Usage Guide - -- [Introduction](#introduction) -- [Software and Hardware Requirements](#software-and-hardware-requirements) -- [Installation](#Installation) -- [Tailoring and Customizing an Image](#tailoring-and-customizing-an-image) - - [Command Description](#command-description) - - [Software Package Source](#software-package-source) - - [Operation Guide](#operation-guide) - - -## Introduction -The size of an openEuler image is large, and the process of downloading or transferring an image is time-consuming. In addition, when an openEuler image is used to install the OS, all RPM packages contained in the image are installed. You cannot choose to install only the required software packages. - -In some scenarios, you do not need to install the full software package provided by the image, or you need to install additional software packages. Therefore, openEuler provides an image tailoring and customization tool. You can use this tool to customize an ISO image that contains only the required RPM packages based on an openEuler image. The software packages can be the ones contained in an official ISO image or specified in addition to meet custom requirements. - -This document describes how to install and use the openEuler image tailoring and customization tool. - -## Software and Hardware Requirements - -The hardware and software requirements of the computer to make an ISO file using the openEuler tailoring and customization tool are as follows: - -- The CPU architecture is AArch64 or x86_64. -- The operating system is openEuler 22.03 LTS. -- You are advised to reserve at least 30 GB drive space for running the tailoring and customization tool and storing the ISO image. - -## Installation - -The following uses openEuler 22.03 LTS on the AArch64 architecture as an example to describe how to install the ISO image tailoring and customization tool. - -1. Ensure that openEuler 22.03 LTS has been installed on the computer. - - ``` shell script - $ cat /etc/openEuler-release - openEuler release 22.03 LTS - ``` - -2. Download the ISO image (must be an **everything** image) of the corresponding architecture and save it to any directory (it is recommended that the available space of the directory be greater than 20 GB). In this example, the ISO image is saved to the **/home/isocut_iso** directory. - - The download address of the AArch64 image is as follows: - - https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso - - > **Note:** - > The download address of the x86_64 image is as follows: - > - > https://repo.openeuler.org/openEuler-22.03-LTS/ISO/x86_64/openEuler-22.03-LTS-everything-x86_64-dvd.iso - -3. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum source. The following is an example of the configuration file. **baseurl** is the directory for mounting the ISO image. - - ``` shell script - [local] - name=local - baseurl=file:///home/isocut_mount - gpgcheck=0 - enabled=1 - ``` - -4. Run the following command as the **root** user to mount the image to the **/home/isocut_mount** directory (ensure that the mount directory is the same as **baseurl** configured in the **repo** file) as the Yum source: - - ```shell - sudo mount -o loop /home/isocut_iso/openEuler-22.03-LTS-everything-aarch64-dvd.iso /home/isocut_mount - ``` - -5. Make the Yum source take effect. - - ```shell - yum clean all - yum makecache - ``` - -6. Install the image tailoring and customization tool as the **root** user. - - ```shell - sudo yum install -y isocut - ``` - -7. Run the following command as the **root** user to check whether the tool has been installed successfully: - - ```shell - $ sudo isocut -h - Checking input ... - usage: isocut [-h] [-t temporary_path] [-r rpm_path] [-k file_path] source_iso dest_iso - - Cut EulerOS iso to small one - - positional arguments: - source_iso source iso image - dest_iso destination iso image - - optional arguments: - -h, --help show this help message and exit - -t temporary_path temporary path - -r rpm_path extern rpm packages path - -k file_path kickstart file - ``` - - - -## Tailoring and Customizing an Image - -This section describes how to use the image tailoring and customization tool to create an image by tailoring or adding RPM packages to an openEuler image. - -### Command Description - -#### Format - -Run the `isocut` command to use the image tailoring and customization tool. The command format is as follows: - -**isocut** [ --help | -h ] [ -t <*temp_path*> ] [ -r <*rpm_path*> ] [ -k <*file_path*> ] < *source_iso* > < *dest_iso* > - -#### Parameter Description - -| Parameter| Mandatory| Description| -| ------------ | -------- | -------------------------------------------------------- | -| --help \| -h | No| Queries the help information about the command.| -| -t <*temp_path*> | No| Specifies the temporary directory *temp_path* for running the tool, which is an absolute path. The default value is **/tmp**.| -| -r <*rpm_path*> | No| Specifies the path of the RPM packages to be added to the ISO image.| -| -k <*file_path*> | No | Specifies the kickstart template path if kickstart is used for automatic installation. | -| *source_iso* | Yes| Path and name of the ISO source image to be tailored. If no path is specified, the current path is used by default.| -| *dest_iso* | Yes| Specifies the path and name of the new ISO image created by the tool. If no path is specified, the current path is used by default.| - - - -### Software Package Source - -The RPM packages of the new image can be: - -- Packages contained in an official ISO image. In this case, the RPM packages to be installed are specified in the configuration file **/etc/isocut/rpmlist**. The configuration format is *software_package_name.architecture*. For example, **kernel.aarch64**. - -- Specified in addition. In this case, use the `-r` parameter to specify the path in which the RPM packages are stored when running the `isocut` command and add the RPM package names to the **/etc/isocut/rpmlist** configuration file. (See the name format above.) - - - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- When customizing an image, if an RPM package specified in the configuration file cannot be found, the RPM package will not be added to the image. - >- If the dependency of the RPM package is incorrect, an error may be reported when running the tailoring and customization tool. - -### kickstart Functions - -You can use kickstart to install images automatically by using the `-k` parameter to specify a kickstart file when running the **isocut** command. - -The isocut tool provides a kickstart template (**/etc/isocut/anaconda-ks.cfg**). You can modify the template as required. - -#### Modifying the kickstart Template - -If you need to use the kickstart template provided by the isocut tool, perform the following modifications: - -- Configure the root user password and the GRUB2 password in the **/etc/isocut/anaconda-ks.cfg** file. Otherwise, the automatic image installation will pause during the password setting process, waiting for you to manually enter the passwords. -- If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file. - -See the next section for details about how to modify the kickstart file. - -##### Configuring Initial Passwords - -###### Setting the Initial Password of the **root** User - -Set the initial password of the **root** user as follows in the **/etc/isocut/anaconda-ks.cfg** file. Replace **${pwd}** with the encrypted password. - -```shell -rootpw --iscrypted ${pwd} -``` - -Obtain the initial password of the **root** user as follows (**root** permissions are required): - -1. Add a user for generating the password, for example, **testUser**. - - ``` shell script - $ sudo useradd testUser - ``` - -2. Set the password for the **testUser** user. Run the following command to set the password as prompted: - - ``` shell script - $ sudo passwd testUser - Changing password for user testUser. - New password: - Retype new password: - passwd: all authentication tokens updated successfully. - ``` - -3. View the **/etc/shadow** file to obtain the encrypted password. The encrypted password is the string between the two colons (:) following the **testUser** user name. (******* is used as an example.) - - ``` shell script - $ sudo cat /etc/shadow | grep testUser - testUser:***:19052:0:90:7:35:: - ``` - -4. Run the following command to replace the **pwd** field in the **/etc/isocut/anaconda-ks.cfg** file with the encrypted password (replace __***__ with the actual password): - ``` shell script - rootpw --iscrypted *** - ``` - -###### Configuring the Initial GRUB2 Password - -Add the following configuration to the **/etc/isocut/anaconda-ks.cfg** file to set the initial GRUB2 password: Replace **${pwd}** with the encrypted password. - -```shell -%addon com_huawei_grub_safe --iscrypted --password='${pwd}' -%end -``` - -> ![](./public_sys-resources/icon-note.gif) NOTE: -> -> - The **root** permissions are required for configuring the initial GRUB password. -> - The default user corresponding to the GRUB password is **root**. -> -> - The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance. - -1. Run the following command and set the GRUB2 password as prompted: - - ```shell - $ sudo grub2-set-password -o ./ - Enter password: - Confirm password: - grep: .//grub.cfg: No such file or directory - WARNING: The current configuration lacks password support! - Update your configuration with grub2-mkconfig to support this feature. - ``` - -2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB2 password. - - ```shell - $ sudo cat user.cfg - GRUB2_PASSWORD=grub.pbkdf2.sha512.*** - ``` - -3. Add the following information to the **/etc/isocut/anaconda-ks.cfg** file. Replace ******* with the encrypted GRUB2 password. - - ```shell - %addon com_huawei_grub_safe --iscrypted --password='grub.pbkdf2.sha512.***' - %end - ``` - -##### Configuring the %packages Field - -If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file. - -This section describes how to specify RPM packages in the **/etc/isocut/anaconda-ks.cfg** file. - -The default configurations of **%packages** in the **/etc/isocut/anaconda-ks.cfg** file are as follows: - -```shell -%packages --multilib --ignoremissing -acl.aarch64 -aide.aarch64 -...... -NetworkManager.aarch64 -%end -``` - -Add specified RPM packages to the **%packages** configurations in the following format: - -*software_package_name.architecture*. For example, **kernel.aarch64**. - -```shell -%packages --multilib --ignoremissing -acl.aarch64 -aide.aarch64 -...... -NetworkManager.aarch64 -kernel.aarch64 -%end -``` - -### Operation Guide - - - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- Do not modify or delete the default configuration items in the **/etc/isocut/rpmlist** file. ->- All `isocut` operations require **root** permissions. ->- The source image to be tailored can be a basic image or **everything** image. In this example, the basic image **openEuler-22.03-LTS-aarch64-dvd.iso** is used. ->- In this example, assume that the new image is named **new.iso** and stored in the **/home/result** directory, the temporary directory for running the tool is **/home/temp**, and the additional RPM packages are stored in the **/home/rpms** directory. - - - -1. Open the configuration file **/etc/isocut/rpmlist** and specify the RPM packages to be installed (from the official ISO image). - - ``` shell script - sudo vi /etc/isocut/rpmlist - ``` - -2. Ensure that the space of the temporary directory for running the image tailoring and customization tool is greater than 8 GB. The default temporary directory is** /tmp**. You can also use the `-t` parameter to specify another directory as the temporary directory. The path of the directory must be an absolute path. In this example, the **/home/temp** directory is used. The following command output indicates that the available drive space of the **/home** directory is 38 GB, which meets the requirements. - - ```shell - $ df -h - Filesystem Size Used Avail Use% Mounted on - devtmpfs 1.2G 0 1.2G 0% /dev - tmpfs 1.5G 0 1.5G 0% /dev/shm - tmpfs 1.5G 23M 1.5G 2% /run - tmpfs 1.5G 0 1.5G 0% /sys/fs/cgroup - /dev/mapper/openeuler_openeuler-root 69G 2.8G 63G 5% / - /dev/sda2 976M 114M 796M 13% /boot - /dev/mapper/openeuler_openeuler-home 61G 21G 38G 35% /home - ``` - -3. Tailor and customize the image. - - **Scenario 1**: All RPM packages of the new image are from the official ISO image. - - ``` shell script - $ sudo isocut -t /home/temp /home/isocut_iso/openEuler-22.03-LTS-aarch64-dvd.iso /home/result/new.iso - Checking input ... - Checking user ... - Checking necessary tools ... - Initing workspace ... - Copying basic part of iso image ... - Downloading rpms ... - Finish create yum conf - finished - Regenerating repodata ... - Checking rpm deps ... - Getting the description of iso image ... - Remaking iso ... - Adding checksum for iso ... - Adding sha256sum for iso ... - ISO cutout succeeded, enjoy your new image "/home/result/new.iso" - isocut.lock unlocked ... - ``` - If the preceding information is displayed, the custom image **new.iso** is successfully created. - - **Scenario 2**: The RPM packages of the new image are from the official ISO image and additional packages in **/home/rpms**. - - ```shell - sudo isocut -t /home/temp -r /home/rpms /home/isocut_iso/openEuler-22.03-LTS-aarch64-dvd.iso /home/result/new.iso - ``` - **Scenario 3**: The kickstart file is used for automatic installation. You need to modify the **/etc/isocut/anaconda-ks.cfg** file. - ```shell - sudo isocut -t /home/temp -k /etc/isocut/anaconda-ks.cfg /home/isocut_iso/openEuler-22.03-LTS-aarch64-dvd.iso /home/result/new.iso - ``` - - -## FAQs - -### The System Fails to Be Installed Using an Image Tailored Based on the Default RPM Package List - -#### Context - -When isocut is used to tailor an image, the **/etc/isocut/rpmlist** configuration file is used to specify the software packages to be installed. - -Images of different OS versions contain different software packages. As a result, some packages may be missing during image tailoring. -Therefore, the **/etc/isocut/rpmlist** file contains only the kernel software package by default, -ensuring that the image can be successfully tailored. - -#### Symptom - -The ISO image is successfully tailored using the default configuration, but fails to be installed. - -An error message is displayed during the installation, indicating that packages are missing: - -![](./figures/lack_pack.png) - -#### Possible Cause - -The ISO image tailored based on the default RPM package list lacks necessary RPM packages during installation. -The missing RPM packages are displayed in the error message, and may vary depending on the version. - -#### Solution - -1. Add the missing packages. - - 1. Find the missing RPM packages based on the error message. - 2. Add the missing RPM packages to the **/etc/isocut/rpmlist** configuration file. - 3. Tailor and install the ISO image again. - - For example, if the missing packages are those in the example error message, modify the **rpmlist** configuration file as follows: - ```shell - $ cat /etc/isocut/rpmlist - kernel.aarch64 - lvm2.aarch64 - chrony.aarch64 - authselect.aarch64 - shim.aarch64 - efibootmgr.aarch64 - grub2-efi-aa64.aarch64 - dosfstools.aarch64 - ``` diff --git a/docs/en/docs/Virtualization/LibcarePlus.md b/docs/en/docs/Virtualization/LibcarePlus.md deleted file mode 100644 index 1a6fe540b61c631256a9612b1d451887a398dc7a..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/LibcarePlus.md +++ /dev/null @@ -1,410 +0,0 @@ -# LibcarePlus - - - -- [LibcarePlus](#libcareplus) - - [Overview](#overview) - - [Hardware and Software Requirements](#hardware-and-software-requirements) - - [Precautions and Constraints](#precautions-and-constraints) - - [Installing LibcarePlus](#installing-libcareplus) - - [Software Installation Dependencies](#software-installation-dependencies) - - [Installing LibcarePlus](#installing-libcareplus-1) - - [Creating LibcarePlus Hot Patches](#creating-libcareplus-hot-patches) - - [Introduction](#introduction) - - [Manual Creation](#manual-creation) - - [Creation Through a Script](#creation-through-a-script) - - [Applying the LibcarePlus Hot Patch](#applying-the-libcareplus-hot-patch) - - [Preparation](#preparation) - - [Loading the Hot Patch](#loading-the-hot-patch) - - [Querying a Hot Patch](#querying-a-hot-patch) - - [Uninstalling the Hot Patch](#uninstalling-the-hot-patch) - - - -## Overview - -LibcarePlus is a hot patch framework for user-mode processes. It can perform hot patch operations on target processes running on the Linux system without restarting the processes. Hot patches can be used to fix CVEs and urgent bugs that do not interrupt application services. - -## Hardware and Software Requirements - -The following software and hardware requirements must be met to use LibcarePlus on openEuler: - -- Currently, the x86 and ARM64 architectures are supported. - -- LibcarePlus can run on any Linux distribution that supports **libunwind**, **elfutils**, and **binutils**. -- LibcarePlus uses the **ptrace()** system call, which requires the kernel configuration option enabled for the corresponding Linux distribution. -- LibcarePlus needs the symbol table of the original executable file when creating a hot patch. Do not strip the symbol table too early. -- On the Linux OS where SELinux is enabled, manually adapt the SELinux policies. - -## Precautions and Constraints - -When using LibcarePlus, comply with the following hot patch specifications and constraints: - -- Only the code written in the C language is supported. The assembly language is not supported. -- Only user-mode programs are supported. Dynamic library patches are not supported. -- The code file name must comply with the C language identifier naming specifications. That is, the code file name consists of letters (A-Z and a-z), digits (0-9), and underscores (_) but the first character cannot be a digit. Special characters such as hyphens (-) and dollar signs ($) are not allowed. -- Incremental patches are supported. Multiple patches can be installed on a process. However, you need to design the patch installation and uninstallation management. Generally, the installation and uninstallation comply with the first-in, last-out (FILO) rule. -- Automatic patch loading is not natively supported. You can design an automatic patch loading method for a specific process. -- Patch query is supported. -- The static function patch is restricted by the symbol table that can find the function in the system. -- Hot patches are process-specific. That is, a hot patch of a dynamic library can be applied only to process that invoke the dynamic library. -- The number of patches that can be applied to a process is limited by the range of the jump instruction and the size of the hole in the virtual memory address space. Generally, up to 512 patches can be applied to a process. -- Thread local storage (TLS) variables of the initial executable (IE) model can be modified. -- Symbols defined in a patch cannot be used in subsequent patches. -- Hot patches are not supported in the following scenarios: - - Infinite loop function, non-exit function, inline function, initialization function, and non-maskable interrupt (NMI) function - - Replacing global variables - - Functions less than 5 bytes - - Modifying the header file - - Adding or deleting the input and output parameters of the target function - - Changing (adding, deleting, or modifying) data structure members - - Modifying the C files that contain GCC macros such as __LINE__ and __FILE__ - - Modifying the Intel vector assembly instruction - -## Installing LibcarePlus - -### Software Installation Dependencies - -The LibcarePlus running depends on **libunwind**, **elfutils**, and **binutils**. On the openEuler system configured with the Yum repo, you can run the following commands to install the software on which LibcarePlus depends: - -``` shell -# yum install -y binutils elfutils elfutils-libelf-devel libunwind-devel -``` - -#### Installing LibcarePlus - -```shell -# yum install libcareplus libcareplus-devel -y -``` - -Check whether LibcarePlus is installed. - -``` shell -# libcare-ctl -h -usage: libcare-ctl [options] [args] - -Options: - -v - verbose mode - -h - this message - -Commands: - patch - apply patch to a user-space process - unpatch- unapply patch from a user-space process - info - show info on applied patches -``` - -## Creating LibcarePlus Hot Patches - -### Introduction - -LibcarePlus hot patch creation methods: - -- Manual creation -- Creation through a script - -The process of manually creating a hot patch is complex. For a project with a large amount of code, for example, QEMU, it is extremely difficult to manually create a hot patch. You are advised to use the script provided by LibcarePlus to generate a hot patch file with one click. - -#### Manual Creation - -The following takes the original file **foo.c** and the patch file **bar.c** as examples to describe how to manually create a hot patch. - -1. Prepare the original file and patch file written in the C language. For example, **foo.c** and **bar.c**. - -
- Expand foo.c -

- - ``` c - // foo.c - #include - #include - - void print_hello(void) - { - printf("Hello world!\n"); - } - - int main(void) - { - while (1) { - print_hello(); - sleep(1); - } - } - ``` - -

-
- -
- Expand bar.c -

- - ``` c - // bar.c - #include - #include - - void print_hello(void) - { - printf("Hello world %s!\n", "being patched"); - } - - int main(void) - { - while (1) { - print_hello(); - sleep(1); - } - } - ``` - -

-
- -2. Build the original file and patch file to obtain the assembly files **foo.s** and **bar.s**. - - ``` shell - # gcc -S foo.c - # gcc -S bar.c - # ls - bar.c bar.s foo.c foo.s - ``` - -3. Run `kpatch_gensrc` to compare **foo.s** and **bar.s** and generate the **foobar.s** file that contains the assembly content of the original file and the differences. - - ``` shell - # sed -i 's/bar.c/foo.c/' bar.s - # kpatch_gensrc --os=rhel6 -i foo.s -i bar.s -o foobar.s --force-global - ``` - - By default, `kpatch_gensrc` compares the original files in the same C language. Therefore, before the comparison, you need to run the `sed` command to change the file name **bar.c** in the patch assembly file **bar.s** to the original file name **foo.c**. Call `kpatch_gensrc` to specify the input files as **foo.s** and **bar.s** and the output file as **foobar.s**. - -4. Build the assembly file **foo.s** in the original file and the generated assembly file **foobar.s** to obtain the executable files **foo** and **foobar**. - - ``` shell - # gcc -o foo foo.s - # gcc -o foobar foobar.s -Wl,-q - ``` - - The **-Wl, -q** linker options reserve the relocation sections in **foobar**. - -5. Use `kpatch_strip` to remove the duplicate content from the executables **foo** and **foobar** and reserve the content required for creating hot patches. - - ``` shell - # kpatch_strip --strip foobar foobar.stripped - # kpatch_strip --rel-fixup foo foobar.stripped - # strip --strip-unneeded foobar.stripped - # kpatch_strip --undo-link foo foobar.stripped - ``` - - The options in the preceding command are described as follows: - - - **--strip** removes useless sections for patch creation from **foobar**. - - **--rel-fixup** repairs the address of the variables and functions accessed in the patch. - - **strip --strip-unneeded** removes the useless symbol information for hot patch relocation. - - **--undo-link** changes the symbol address in a patch from absolute to relative. - -6. Create a hot patch file. - - After the preceding operations, the contents required for creating the hot patch are obtained. Run the `kpatch_make` command to input parameters Build ID of the original executable file and **foobar.stripped** (output file of `kpatch_strip`) to `kpatch_make` to generate a hot patch file. - - ``` shell - # str=$(readelf -n foo | grep 'Build ID') - # substr=${str##* } - # kpatch_make -b $substr -i 0001 foobar.stripped -o foo.kpatch - # ls - bar.c bar.s foo foobar foobar.s foobar.stripped foo.c foo.kpatch foo.s - ``` - - The final hot patch file **foo.kpatch** whose patch ID is **0001** is obtained. - -#### Creation Through a Script - -This section describes how to use LibcarePlus built-in **libcare-patch-make** script to create a hot patch file. The original file **foo.c** and patch file **bar.c** are used as an example. - -1. Run the `diff` command to generate the comparison file of **foo.c** and **bar.c**. - - ``` shell - # diff -up foo.c bar.c > foo.patch - ``` - - The content of the **foo.patch** file is as follows: - -
- Expand foo.patch -

- - - ``` diff - --- foo.c 2020-12-09 15:39:51.159632075 +0800 - +++ bar.c 2020-12-09 15:40:03.818632220 +0800 - @@ -1,10 +1,10 @@ - -// foo.c - +// bar.c - #include - #include - - void print_hello(void) - { - - printf("Hello world!\n"); - + printf("Hello world %s!\n", "being patched"); - } - - int main(void) - ``` - -

-
- - -2. Write the **makefile** for building **foo.c** as follows: - -
- Expand makefile -

- - ``` makefile - all: foo - - foo: foo.c - $(CC) -o $@ $< - - clean: - rm -f foo - - install: foo - mkdir $$DESTDIR || : - cp foo $$DESTDIR - ``` - -

-
- - -3. After the **makefile** is done, directly call `libcare-patch-make`. If `libcare-patch-make` asks you which file to install the patch, enter the original file name, as shown in the following: - - ``` shell - # libcare-patch-make --clean -i 0001 foo.patch - rm -f foo - BUILDING ORIGINAL CODE - /usr/local/bin/libcare-cc -o foo foo.c - INSTALLING ORIGINAL OBJECTS INTO /libcareplus/test/lpmake - mkdir $DESTDIR || : - cp foo $DESTDIR - applying foo.patch... - can't find file to patch at input line 3 - Perhaps you used the wrong -p or --strip option? - The text leading up to this was: - -------------------------- - |--- foo.c 2020-12-10 09:43:04.445375845 +0800 - |+++ bar.c 2020-12-10 09:48:36.778379648 +0800 - -------------------------- - File to patch: foo.c - patching file foo.c - BUILDING PATCHED CODE - /usr/local/bin/libcare-cc -o foo foo.c - INSTALLING PATCHED OBJECTS INTO /libcareplus/test/.lpmaketmp/patched - mkdir $DESTDIR || : - cp foo $DESTDIR - MAKING PATCHES - Fixing up relocation printf@@GLIBC_2.2.5+fffffffffffffffc - Fixing up relocation print_hello+0 - patch for /libcareplus/test/lpmake/foo is in /libcareplus/test/patchroot/700297b7bc56a11e1d5a6fb564c2a5bc5b282082.kpatch - ``` - - After the command is executed, the output indicates that the hot patch file is in the **patchroot** directory of the current directory, and the executable file is in the **lpmake** directory. By default, the Build ID is used to name a hot patch file generated by a script. - - - -## Applying the LibcarePlus Hot Patch - -This following uses the original file **foo.c** and patch file **bar.c** as an example to describe how to use the LibcarePlus hot patch. - -### Preparation - -Before using the LibcarePlus hot patch, prepare the original executable file **foo** and hot patch file **foo.kpatch**. - -### Loading the Hot Patch - -The procedure for applying the LibcarePlus hot patch is as follows: - -1. In the first shell window, run the executable file to be patched: - - ``` shell - # ./lpmake/foo - Hello world! - Hello world! - Hello world! - ``` - -2. In the second shell window, run the `libcare-ctl` command to apply the hot patch: - - ``` shell - # libcare-ctl -v patch -p $(pidof foo) ./patchroot/BuildID.kpatch - ``` - - If the hot patch is applied successfully, the following information is displayed in the second shell window: - - ``` shell - 1 patch hunk(s) have been successfully applied to PID '10999' - ``` - - The following information is displayed for the target process running in the first shell window: - - ``` shell - Hello world! - Hello world! - Hello world being patched! - Hello world being patched! - ``` - - -### Querying a Hot Patch - -The procedure for querying a LibcarePlus hot patch is as follows: - -1. Run the following command in the second shell window: - - ```shell - # libcare-ctl info -p $(pidof foo) - - ``` - - If a hot patch is installed, the following information is displayed in the second shell window: - - ```shell - Pid: 551763 - Target: foo - Build id: df05a25bdadd282812d3ee5f0a460e69038575de - Applied patch number: 1 - Patch id: 0001 - ``` - - - - -### Uninstalling the Hot Patch - -The procedure for uninstalling the LibcarePlus hot patch is as follows: - -1. Run the following command in the second shell window: - - ``` shell - # libcare-ctl unpatch -p $(pidof foo) -i 0001 - ``` - - If the hot patch is uninstalled successfully, the following information is displayed in the second shell window: - - ``` shell - 1 patch hunk(s) were successfully cancelled from PID '10999' - ``` - -2. The following information is displayed for the target process running in the first shell window: - - ``` shell - Hello world being patched! - Hello world being patched! - Hello world! - Hello world! - ``` diff --git a/docs/en/docs/Virtualization/Skylark.md b/docs/en/docs/Virtualization/Skylark.md deleted file mode 100644 index b844683707094f67ee57b094595aa48cb9612b3d..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/Skylark.md +++ /dev/null @@ -1,195 +0,0 @@ -# Skylark - -- [Skylark](#skylark) - - [Skylark Introduction](#skylark-introduction) - - [Architecture and Features](#architecture-and-features) - - [Skylark Installation](#skylark-installation) - - [Skylark Configuration](#skylark-configuration) - - [Skylark Usage](#skylark-usage) - - [Best Practices](#best-practices) - -## Skylark Introduction - -### Scenario - -With the rapid growth of the cloud computing market, cloud vendors are increasing their investment in cloud infrastructure. However, the industry still faces the problem of low resource utilization. Improving resource utilization has become an important technical subject. This document describes openEuler Skylark, as well as how to install and use it. - -### Overview - -Hybrid deployment of services of different priorities is a typical and effective method to improve resource utilization. Services can be classified into high-priority and low-priority services based on latency sensitivity. When high-priority services compete with low-priority services for resources, resources are preferentially provided for high-priority services. Therefore, the core technology of service hybrid deployment is resource isolation control, which involves kernel-mode basic resource isolation and user-mode QoS control. - -This document describes the user-mode QoS control technology provided by Skylark of openEuler 22.09. In Skylark, the priority granularity is VMs. That is, a priority attribute is added to each VM. Resources are isolated and controlled based on VMs. Skylark is a QoS-aware resource scheduler in hybrid deployment scenarios. It improves physical machine resource utilization while ensuring the QoS of high-priority VMs. - -For details about how to better use the priority feature of Skylark in actual application scenarios, see [Best Practices](#best-practices). - -## Architecture and Features - -### Overall Architecture - -The core class of Skylark is `QoSManager`. Class members include data collection class instances, QoS analysis class instances, QoS control class instances, and task scheduling class instances. -- `DataCollector`: data collection class. It has the `HostInfo` and `GuestInfo` members, which collect host information and VM information, respectively. -- `PowerAnalyzer`: power consumption analysis class, which analyzes power consumption interference and low-priority VMs to be restricted. -- `CpuController`: CPU bandwidth control class, which limits the CPU bandwidth of low-priority VMs. -- `CacheMBWController`: last-level cache (LLC) and memory bandwidth control class, which limits the LLC and memory bandwidth of low-priority VMs. -- `BackgroundScheduler`: task scheduling class, which periodically drives the preceding modules to continuously manage QoS. - -After checking the host environment, Skylark creates a daemon process. The daemon has a main scheduling thread and one or more job threads. -- The main scheduling thread is unique. It connects to libvirt, creates and initializes the `QosManager` class instance, and then starts to drive the Job threads. -- Each Job thread periodically executes a QoS management task. - -### Power Consumption Interference Control - -Compared with non-hybrid deployment, host resource utilization is higher in hybrid deployment scenarios. High utilization means high power consumption. When the power consumption exceeds the thermal design power (TDP) of the server, CPU frequency reduction is triggered. When the power consumption exceeds the preset TDP (that is, a TDP hotspot occurs), Skylark limits the CPU bandwidth of low-priority VMs to reduce the power consumption of the entire system and ensure the QoS of high-priority VMs. - -During initialization, Skylark sets the power consumption interference control attributes based on the related configuration values in [Skylark Configuration](#skylark-configuration). In each control period, host information and control attributes are comprehensively analyzed to determine whether TDP hotspots occur. If a hotspot occurs, Skylark analyzes the low-priority VMs whose CPU bandwidth needs to be limited based on the VM information. - -### LLC/MB Interference Control - -Skylark can limit the LLC and memory bandwidth of low-priority VMs. Currently, only static allocation is supported. Skylark uses the **/sys/fs/resctrl** interface provided by the OS to implement the limitation. - -1. Skylark creates the **low_prio_machine** folder in the **/sys/fs/resctrl** directory and writes the PID of the low-priority VM to the **/sys/fs/resctrl/low_prio_machine/tasks** file. -2. Skylark allocates LLC ways and memory bandwidth for low-priority VMs based on the LLC/MB configuration items in [Skylark Configuration](#skylark-configuration). The configuration items are written into the **/sys/fs/resctrl/low_prio_machine/schemata** file. - -### CPU Interference Control - -In hybrid deployment scenarios, low-priority VMs generate CPU time slice interference and hardware hyper-threading (SMT) interference on high-priority VMs. - -- When threads of high- and low-priority VMs are running on the same minimum CPU topology unit (core or SMT execution unit), they compete for CPU time slices. -- When threads of high- and low-priority VMs are running on different SMT execution units of the same CPU core at the same time, they compete for resources in the core shared by the SMT execution units. - -CPU interference control includes CPU time slice interference control and SMT interference control, which are implemented based on the **QOS_SCHED** and **SMT_EXPELLER** features provided by the kernel, respectively. - -- The **QOS_SCHED** feature enables high-priority VM threads on a single CPU core or SMT execution unit to suppress low-priority VM threads, eliminating CPU time slice interference. -- The **SMT_EXPELLER** feature enables high-priority VM threads to suppress low-priority VM threads on different SMT execution units of the same CPU core, eliminating SMT interference. - -During initialization, Skylark sets the **cpu.qos_level** field of the slice level corresponding to the low-priority VM under the cgroup CPU subcontroller to -1 to enable the preceding kernel features. By doing this, the kernel controls CPU-related interference without the intervention of Skylark. - -## Skylark Installation - -### Hardware Requirements - -Processor architecture: AArch64 or x86_64 -- For Intel processors, the RDT function must be supported. -- For the AArch64 architecture, only Kunpeng 920 processor is supported, and the BIOS must be upgraded to 1.79 or later to support the MPAM function. - -### Software Requirements - -- python3, python3-APScheduler, and python3-libvirt -- systemd 249-32 or later -- libvirt 1.0.5 or later -- openEuler kernel 5.10.0 or later. - -### Installation Procedure - -You are advised to install the Skylark component using Yum for automatic processing of the software dependencies: - -```shell -# yum install -y skylark -``` - -Check whether the Skylark is successfully installed. If the installation is successful, the skylarkd background service status is displayed: - -```shell -# systemctl status skylarkd -``` - -(Optional) Enable the Skylark service to automatically start upon system startup: - -```shell -# systemctl enable skylarkd -``` - -## Skylark Configuration - -After the Skylark component is installed, you can modify the configuration file if the default configuration does not meet your requirements. The Skylark configuration file is stored in **/etc/sysconfig/skylarkd**. The following describes the configuration items in the configuration file. - -### Logs - -- The **LOG_LEVEL** parameter is a character string used to set the minimum log level. The supported log levels are **critical > error > warning > info > debug**. Logs whose levels are lower than **LOG_LEVEL** are not recorded in the log file **/var/log/skylark.log**. Skylark backs up logs every seven days for a maximum of four times. (When the number of backup times reaches the limit, the oldest logs are deleted.) The backup log is saved as **/var/log/skylark.log. %Y- %m- %d**. - -### Power Consumption Interference Control - -- **POWER_QOS_MANAGEMENT** is a boolean value used to control whether to enable power consumption QoS management. Only x86 processors support this function. This function is useful if the CPU usage of VMs on the host can be properly limited. - -- **TDP_THRESHOLD** is a floating point number used to control the maximum power consumption of a VM. When the power consumption of the host exceeds **TDP * TDP_THRESHOLD**, a TDP hotspot occurs, and a power consumption control operation is triggered. The value ranges from 0.8 to 1, with the default value being 0.98. - -- **FREQ_THRESHOLD** is a floating point number used to control the minimum CPU frequency when a TDP hotspot occurs on the host. The value ranges from 0.8 to 1, with the default value being 0.98. - 1. When the frequency of some CPUs is lower than **max_freq * FREQ_THRESHOLD**, Skylark limits the CPU bandwidth of low-priority VMs running on these CPUs. - 2. If such a CPU does not exist, Skylark limits the CPU bandwidth of some low-priority VMs based on the CPU usage of low-priority VMs. - -- **QUOTA_THRESHOLD** is a floating point number used to control the CPU bandwidth that a restricted low-priority VM can obtain (CPU bandwidth before restriction x **QUOTA_THRESHOLD**). The value ranges from 0.8 to 1, with the default value being 0.9. - -- **ABNORMAL_THRESHOLD** is an integer used to control the number of low-priority VM restriction periods. The value ranges from 1 to 5, with the default value being 3. - 1. In each power consumption control period, if a low-priority VM is restricted, its number of remaining restriction periods is updated to **ABNORMAL_THRESHOLD**. - 2. Otherwise, its number of remaining restriction periods decreases by 1. When the number of remaining restriction periods of the VM is 0, the CPU bandwidth of the VM is restored to the value before the restriction. - -### LLC/MB Interference Control - -Skylark's interference control on LLC/MB depends on the RDT/MPAM function provided by hardware. For Intel x86_64 processors, **rdt=cmt,mbmtotal,mbmlocal,l3cat,mba** needs to be added to kernel command line parameters. For Kunpeng920 processors, **mpam=acpi** needs to be added to kernel command line parameters. - -- **MIN_LLC_WAYS_LOW_VMS** is an integer used to control the number of LLC ways that can be accessed by low-priority VMs. The value ranges from 1 to 3, with the default value being 2. During initialization, Skylark limits the numfer of accessible LLC ways for low-priority VMs to this value. - -- **MIN_MBW_LOW_VMS** is a floating point number used to control the memory bandwidth ratio available to low-priority VMs. The value ranges from 0.1 to 0.2, with the default value being 0.1. Skylark limits the memory bandwidth of low-priority VMs based on this value during initialization. - -## Skylark Usage - -### Starting the Service - -Start Skylark for the first time: - -```shell -# systemctl start skylarkd -``` - -Restart Skylark (a service restart is required after modifying the configuration file): - -```shell -# systemctl restart skylarkd -``` - -### Creating VMs - -Skylark uses the **partition** tag in the XML configuration file of a VM to identify the VM priority. - -To create a low-priority VM, configure the XML file as follows: - -```xml - - ... - - /low_prio_machine - - ... - -``` - -To create a high-priority VM, configure the XML file as follows: - -```xml - - ... - - /high_prio_machine - - ... - -``` - -The subsequent VM creation process is the same as the normal process. - -### Running VMs - -Skylark detects VM creation events, manages VMs of different priorities, and performs automatic QoS management based on CPU, power consumption, and LLC/MB resources. - -## Best Practices - -### VM Service Recommendation - -- High-priority VMs are suitable for latency-sensitive services, such as web services, high-performance databases, real-time rendering, and AI inference. -- Low-priority VMs are suitable for non-latency-sensitive services, such as video encoding, big data processing, offline rendering, and AI training. - -### CPU Binding Configuration - -To ensure optimal performance of high-priority VMs, you are advised to bind each vCPU of high-priority VMs to a physical CPU. To enable low-priority VMs to make full use of idle physical resources, you are advised to bind vCPUs of low-priority VMs to CPUs that are bound to high-priority VMs. - -To ensure that low-priority VMs are scheduled when high-priority VMs occupy CPU resources for a long time, you are advised to reserve a small number of for low-priority VMs. \ No newline at end of file diff --git a/docs/en/docs/Virtualization/appendix.md b/docs/en/docs/Virtualization/appendix.md deleted file mode 100644 index 4277aa8ac9e0afb6a7e8bfa89764e7a8762708f6..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/appendix.md +++ /dev/null @@ -1,145 +0,0 @@ -# Appendix - -- [Appendix](#appendix.md) - - [Terminology & Acronyms and Abbreviations](#terminology-acronyms-and-abbreviations) - - -## Terminology & Acronyms and Abbreviations - -For the terminology & acronyms and abbreviation used in this document, see [Table 1](#table201236162279) and [Table 2](#table1423422319271). - -**Table 1** Terminology - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Term

-

Description

-

AArch64

-

AArch64 is an execution state of the ARMv8 architecture. AArch64 is not only an extension of the 32-bit ARM architecture, but also a brand new architecture in ARMv8 that uses the brand new A64 instruction set.

-

Domain

-

A collection of configurable resources, including memory, vCPUs, network devices, and disk devices. Run the VM in the domain. A domain is allocated with virtual resources and can be independently started, stopped, and restarted.

-

Libvirt

-

A set of tools used to manage virtualization platforms, including KVM, QEMU, Xen, and other virtualization platforms.

-

Guest OS

-

The OS running on the VM.

-

Host OS

-

The OS of the virtual physical machine.

-

Hypervisor

-

Virtual machine monitor (VMM), is an intermediate software layer that runs between a basic physical server and an OS. It allows multiple OSs and applications to share hardware.

-

VM

-

A complete virtual computer system that is constructed by using the virtualization technology and simulating the functions of a complete computer hardware system through software.

-
- -**Table 2** Acronyms and abbreviations - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Acronyms and abbreviations

-

Full spelling

-

Full name

-

Description

-

NUMA

-

Non-Uniform Memory Access Architecture

-

Non Uniform Memory Access Architecture

-

NUMA is a memory architecture designed for multi-processor computers. Under NUMA, a processor accesses its own local memory faster than accessing non-local memory (the memory is located on another processor, or the memory shared between processors).

-

KVM

-

Kernel-based Virtual Machine

-

Kernel-based VM

-

KVM is a kernel-based VM. It is a kernel module of Linux and makes Linux a hypervisor.

-

OVS

-

Open vSwitch

-

Open vSwitch

-

OVS is a high-quality multi-layer vSwitch that uses the open-source Apache 2.0 license protocol.

-

QEMU

-

Quick Emulator

-

Quick Emulator

-

QEMU is a general-purpose, open-source emulator that implements hardware virtualization.

-

SMP

-

Symmetric Multi-Processor

-

Symmetric Multi-Processor

-

SMP is a multi-processor computer hardware architecture. Currently, most processor systems use a symmetric multi-processor architecture. The architecture system has multiple processors, each processor shares the memory subsystem and bus structure.

-

UEFI

-

Unified Extensible Firmware Interface

-

Unified Extensible Firmware Interface

-

A standard that describes new interfaces in detail. This interface is used by the OS to automatically load the prestart operation environment to an OS.

-

VM

-

Virtual Machine

-

VM

-

A complete virtual computer system that is constructed by using the virtualization technology and simulating the functions of a complete computer hardware system through software.

-

VMM

-

Virtual Machine Monitor

-

VM Monitor

-

An intermediate software layer that runs between a basic physical server and an OS. It allows multiple OSs and applications to share hardware.

-
- diff --git a/docs/en/docs/Virtualization/best-practices.md b/docs/en/docs/Virtualization/best-practices.md deleted file mode 100644 index de57ecfd54fbb2c16c15a9440fa882356d1451bb..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/best-practices.md +++ /dev/null @@ -1,640 +0,0 @@ -# Best Practices - - -## Performance Best Practices - -### Halt-Polling - -#### Overview - -If compute resources are sufficient, the halt-polling feature can be used to enable VMs to obtain performance similar to that of physical machines. If the halt-polling feature is not enabled, the host allocates CPU resources to other processes when the vCPU exits due to idle timeout. When the halt-polling feature is enabled on the host, the vCPU of the VM performs polling when it is idle. The polling duration depends on the actual configuration. If the vCPU is woken up during the polling, the vCPU can continue to run without being scheduled from the host. This reduces the scheduling overhead and improves the VM system performance. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The halt-polling mechanism ensures that the vCPU thread of the VM responds in a timely manner. However, when the VM has no load, the host also performs polling. As a result, the host detects that the CPU usage of the vCPU is high, but the actual CPU usage of the VM is not high. - -#### Instructions - -The halt-polling feature is enabled by default. You can dynamically change the halt-polling time of vCPU by modifying the **halt\_poll\_ns** file. The default value is **500000**, in ns. - -For example, to set the polling duration to 400,000 ns, run the following command: - -``` -# echo 400000 > /sys/module/kvm/parameters/halt_poll_ns -``` - -### I/O Thread Configuration - -#### Overview - -By default, QEMU main threads handle backend VM read and write operations on the KVM. This causes the following issues: - -- VM I/O requests are processed by a QEMU main thread. Therefore, the single-thread CPU usage becomes the bottleneck of VM I/O performance. -- The QEMU global lock \(qemu\_global\_mutex\) is used when VM I/O requests are processed by the QEMU main thread. If the I/O processing takes a long time, the QEMU main thread will occupy the global lock for a long time. As a result, the VM vCPU cannot be scheduled properly, affecting the overall VM performance and user experience. - -You can configure the I/O thread attribute for the virtio-blk disk or virtio-scsi controller. At the QEMU backend, an I/O thread is used to process read and write requests of a virtual disk. The mapping relationship between the I/O thread and the virtio-blk disk or virtio-scsi controller can be a one-to-one relationship to minimize the impact on the QEMU main thread, enhance the overall I/O performance of the VM, and improve user experience. - -#### Instructions - -To use I/O threads to process VM disk read and write requests, you need to modify VM configurations as follows: - -- Configure the total number of high-performance virtual disks on the VM. For example, set **** to **4** to control the total number of I/O threads. - - ``` - - VMName - 4194304 - 4194304 - 4 - 4 - ``` - -- Configure the I/O thread attribute for the virtio-blk disk. **** indicates I/O thread IDs. The IDs start from 1 and each ID must be unique. The maximum ID is the value of ****. For example, to allocate I/O thread 2 to the virtio-blk disk, set parameters as follows: - - ``` - - - - -
- - ``` - -- Configure the I/O thread attribute for the virtio-scsi controller. For example, to allocate I/O thread 2 to the virtio-scsi controller, set parameters as follows: - - ``` - - - -
- - ``` - -- Bind I/O threads to a physical CPU. - - Binding I/O threads to specified physical CPUs does not affect the resource usage of vCPU threads. **** indicates I/O thread IDs, and **** indicates IDs of the bound physical CPUs. - - ``` - - - - - ``` - - -### Raw Device Mapping - -#### Overview - -When configuring VM storage devices, you can use configuration files to configure virtual disks for VMs, or connect block devices \(such as physical LUNs and LVs\) to VMs for use to improve storage performance. The latter configuration method is called raw device mapping \(RDM\). Through RDM, a virtual disk is presented as a small computer system interface \(SCSI\) device to the VM and supports most SCSI commands. - -RDM can be classified into virtual RDM and physical RDM based on backend implementation features. Compared with virtual RDM, physical RDM provides better performance and more SCSI commands. However, for physical RDM, the entire SCSI disk needs to be mounted to a VM for use. If partitions or logical volumes are used for configuration, the VM cannot identify the disk. - -#### Instructions - -VM configuration files need to be modified for RDM. The following is a configuration example. - -- Virtual RDM - - The following is an example of mounting the SCSI disk **/dev/sdc** on the host to the VM as a virtual raw device: - - ``` - - - ... - - - - - -
- - ... - - - ``` - - -- Physical RDM - - The following is an example of mounting the SCSI disk **/dev/sdc** on the host to the VM as a physical raw device: - - ``` - - - ... - - - - - -
- - ... - - - ``` - - -### kworker Isolation and Binding - -#### Overview - -kworker is a per-CPU thread implemented by the Linux kernel. It is used to execute workqueue requests in the system. kworker threads will compete for physical core resources with vCPU threads, resulting in virtualization service performance jitter. To ensure that the VM can run stably and reduce the interference of kworker threads on the VM, you can bind kworker threads on the host to a specific CPU. - -#### Instructions - -You can modify the **/sys/devices/virtual/workqueue/cpumask** file to bind tasks in the workqueue to the CPU specified by **cpumasks**. Masks in **cpumask** are in hexadecimal format. For example, if you need to bind kworker to CPU0 to CPU7, run the following command to change the mask to **ff**: - -``` -# echo ff > /sys/devices/virtual/workqueue/cpumask -``` - -### HugePage Memory - -#### Overview - -Compared with traditional 4 KB memory paging, openEuler also supports 2 MB/1 GB memory paging. HugePage memory can effectively reduce TLB misses and significantly improve the performance of memory-intensive services. openEuler uses two technologies to implement HugePage memory. - -- Static HugePages - - The static HugePage requires that a static HugePage pool be reserved before the host OS is loaded. When creating a VM, you can modify the XML configuration file to specify that the VM memory is allocated from the static HugePage pool. The static HugePage ensures that all memory of a VM exists on the host as the HugePage to ensure physical continuity. However, the deployment difficulty is increased. After the page size of the static HugePage pool is changed, the host needs to be restarted for the change to take effect. The size of a static HugePage can be 2 MB or 1 GB. - - -- THP - - If the transparent HugePage \(THP\) mode is enabled, the VM automatically selects available 2 MB consecutive pages and automatically splits and combines HugePages when allocating memory. When no 2 MB consecutive pages are available, the VM selects available 64 KB \(AArch64 architecture\) or 4 KB \(x86\_64 architecture\) pages for allocation. By using THP, users do not need to be aware of it and 2 MB HugePages can be used to improve memory access performance. - - -If VMs use static HugePages, you can disable THP to reduce the overhead of the host OS and ensure stable VM performance. - -#### Instructions - -- Configure static HugePages. - - Before creating a VM, modify the XML file to configure a static HugePage for the VM. - - ``` - - - - - - ``` - - The preceding XML segment indicates that a 1 GB static HugePage is configured for the VM. - - ``` - - - - - - ``` - - The preceding XML segment indicates that a 2 MB static HugePage is configured for the VM. - -- Configure the THP. - - Dynamically enable the THP through sysfs. - - ``` - # echo always > /sys/kernel/mm/transparent_hugepage/enabled - ``` - - Dynamically disable the THP. - - ``` - # echo never > /sys/kernel/mm/transparent_hugepage/enabled - ``` - - -### PV-qspinlock - -#### Overview - -PV-qspinlock optimizes the spin lock in the virtual scenario of CPU overcommitment. It allows the hypervisor to set the vCPU in the lock context to the block state and wake up the corresponding vCPU after the lock is released. In this way, pCPU resources can be better used in the overcommitment scenario, and the compilation application scenario is optimized to reduce the compilation duration. - -#### Instructions - -Modify the **/boot/efi/EFI/openEuler/grub.cfg** configuration file of the VM, add **arm_pvspin** to the startup parameter in the command line, and restart the VM for the modification to take effect. After PV-qspinlock takes effect, run the **dmesg** command on the VM. The following information is displayed: - -``` -[ 0.000000] arm-pv: PV qspinlocks enabled -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->PV-qspinlock is supported only when the operating systems of the host machine and VM are both openEuler 20.09 or later and the VM kernel compilation option **CONFIG_PARAVIRT_SPINLOCKS** is set to **y** (default value for openEuler). - -### Guest-Idle-Haltpoll - -#### Overview - -To ensure fairness and reduce power consumption, when the vCPU of the VM is idle, the VM executes the WFx/HLT instruction to exit to the host machine and triggers context switchover. The host machine determines whether to schedule other processes or vCPUs on the physical CPU or enter the energy saving mode. However, overheads of switching between a virtual machine and a host machine, additional context switching, and IPI wakeup are relatively high, and this problem is particularly prominent in services where sleep and wakeup are frequently performed. The Guest-Idle-Haltpoll technology indicates that when the vCPU of a VM is idle, the WFx/HLT is not executed immediately and VM-exit occurs. Instead, polling is performed on the VM for a period of time. During this period, the tasks of other vCPUs that share the LLC on the vCPU are woken up without sending IPI interrupts. This reduces the overhead of sending and receiving IPI interrupts and the overhead of VM-exit, thereby reducing the task wakeup latency. - ->![](public_sys-resources/icon-note.gif) **NOTE:** - The execution of the **idle-haltpoll** command by the vCPU on the VM increases the CPU overhead of the vCPU on the host machine. Therefore, it is recommended that the vCPU exclusively occupy physical cores on the host machine when this feature is enabled. - -#### Procedure - -The Guest-Idle-Haltpoll feature is disabled by default. The following describes how to enable this feature. - -1. Enable the Guest-Idle-Haltpoll feature. - - If the processor architecture of the host machine is x86, you can configure hint-dedicated in the XML file of the VM on the host machine to enable this feature. In this way, the status that the vCPU exclusively occupies the physical core can be transferred to the VM through the VM XML configuration. The host machine ensures the status of the physical core exclusively occupied by the vCPU. - - ``` - - ... - - - ... - - - - ... - - ``` - - Alternatively, set **cpuidle\_haltpoll.force** to **Y** in the kernel startup parameters of the VM to forcibly enable the function. This method does not require the host machine to configure the vCPU to exclusively occupy the physical core. - ``` - cpuidle_haltpoll.force=Y - ``` - - - If the processor architecture of the host machine is AArch64, this feature can be enabled only by configuring **cpuidle\_haltpoll.force=Y haltpoll.enable=Y** in the VM kernel startup parameters. - - ``` - cpuidle_haltpoll.force=Y haltpoll.enable=Y - ``` - -2. Check whether the Guest-Idle-Haltpoll feature takes effect. Run the following command on the VM. If **haltpoll** is returned, the feature has taken effect. - - ``` - # cat /sys/devices/system/cpu/cpuidle/current_driver - ``` - -3. (Optional) Set the Guest-Idle-Haltpoll parameter. - - The following configuration files are provided in the **/sys/module/haltpoll/parameters/** directory of the VM. You can adjust the configuration parameters based on service characteristics. - - - **guest\_halt\_poll\_ns**: a global parameter that specifies the maximum polling duration after the vCPU is idle. The default value is **200000** (unit: ns). - - **guest\_halt\_poll\_shrink**: a divisor that is used to shrink the current vCPU **guest\_halt\_poll\_ns** when the wakeup event occurs after the **global guest\_halt\_poll\_ns** time. The default value is **2**. - - **guest\_halt\_poll\_grow**: a multiplier that is used to extend the current vCPU **guest\_halt\_poll\_ns** when the wakeup event occurs after the current vCPU **guest\_halt\_poll\_ns** and before the global **guest\_halt\_poll\_ns**. The default value is **2**. - - **guest\_halt\_poll\_grow\_start**: When the system is idle, the **guest\_halt\_poll\_ns** of each vCPU reaches 0. This parameter is used to set the initial value of the current vCPU **guest\_halt\_poll\_ns** to facilitate scaling in and scaling out of the vCPU polling duration. The default value is **50000** (unit: ns). - - **guest\_halt\_poll\_allow\_shrink**: a switch that is used to enable vCPU **guest\_halt\_poll\_ns** scale-in. The default value is **Y**. (**Y** indicates enabling the scale-in; **N** indicates disabling the scale-in.) - - You can run the following command as the **root** user to change the parameter values. In the preceding command, _value_ indicates the parameter value to be set, and _configFile_ indicates the corresponding configuration file. - - ``` - # echo value > /sys/module/haltpoll/parameters/configFile - ``` - - For example, to set the global guest\_halt\_poll\_ns to 200000 ns, run the following command: - - ``` - # echo 200000 > /sys/module/haltpoll/parameters/guest_halt_poll_ns - ``` - - -## Security Best Practices - -### Libvirt Authentication - -#### Overview - -When a user uses libvirt remote invocation but no authentication is performed, any third-party program that connects to the host's network can operate VMs through the libvirt remote invocation mechanism. This poses security risks. To improve system security, openEuler provides the libvirt authentication function. That is, users can remotely invoke a VM through libvirt only after identity authentication. Only specified users can access the VM, thereby protecting VMs on the network. - -#### Enabling Libvirt Authentication - -By default, the libvirt remote invocation function is disabled on openEuler. This following describes how to enable the libvirt remote invocation and libvirt authentication functions. - -1. Log in to the host. -2. Modify the libvirt service configuration file **/etc/libvirt/libvirtd.conf** to enable the libvirt remote invocation and libvirt authentication functions. For example, to enable the TCP remote invocation that is based on the Simple Authentication and Security Layer \(SASL\) framework, configure parameters by referring to the following: - - ``` - #Transport layer security protocol. The value 0 indicates that the protocol is disabled, and the value 1 indicates that the protocol is enabled. You can set the value as needed. - listen_tls = 0 - #Enable the TCP remote invocation. To enable the libvirt remote invocation and libvirt authentication functions, set the value to 1. - listen_tcp = 1 - #User-defined protocol configuration for TCP remote invocation. The following uses sasl as an example. - auth_tcp = "sasl" - ``` - -3. Modify the **/etc/sasl2/libvirt.conf** configuration file to set the SASL mechanism and SASLDB. - - ``` - #Authentication mechanism of the SASL framework. - mech_list: digest-md5 - #Database for storing usernames and passwords - sasldb_path: /etc/libvirt/passwd.db - ``` - -4. Add the user for SASL authentication and set the password. Take the user **userName** as an example. The command is as follows: - - ``` - # saslpasswd2 -a libvirt userName - Password: - Again (for verification): - ``` - -5. Modify the **/etc/sysconfig/libvirtd** configuration file to enable the libvirt listening option. - - ``` - LIBVIRTD_ARGS="--listen" - ``` - -6. Restart the libvirtd service to make the modification to take effect. - - ``` - # systemctl restart libvirtd - ``` - -7. Check whether the authentication function for libvirt remote invocation takes effect. Enter the username and password as prompted. If the libvirt service is successfully connected, the function is successfully enabled. - - ``` - # virsh -c qemu+tcp://192.168.0.1/system - Please enter your authentication name: openeuler - Please enter your password: - Welcome to virsh, the virtualization interactive terminal. - - Type: 'help' for help with commands - 'quit' to quit - - virsh # - ``` - - -#### Managing SASL - -The following describes how to manage SASL users. - -- Query an existing user in the database. - - ``` - # sasldblistusers2 -f /etc/libvirt/passwd.db - user@localhost.localdomain: userPassword - ``` - -- Delete a user from the database. - - ``` - # saslpasswd2 -a libvirt -d user - ``` - - -### qemu-ga - -#### Overview - -QEMU guest agent \(qemu-ga\) is a daemon running within VMs. It allows users on a host OS to perform various management operations on the guest OS through outband channels provided by QEMU. The operations include file operations \(open, read, write, close, seek, and flush\), internal shutdown, VM suspend \(suspend-disk, suspend-ram, and suspend-hybrid\), and obtaining of VM internal information \(including the memory, CPU, NIC, and OS information\). - -In some scenarios with high security requirements, qemu-ga provides the blacklist function to prevent internal information leakage of VMs. You can use a blacklist to selectively shield some functions provided by qemu-ga. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The qemu-ga installation package is **qemu-guest-agent-**_xx_**.rpm**. It is not installed on openEuler by default. _xx_ indicates the actual version number. - -#### Procedure - -To add a qemu-ga blacklist, perform the following steps: - -1. Log in to the VM and ensure that the qemu-guest-agent service exists and is running. - - ``` - # systemctl status qemu-guest-agent |grep Active - Active: active (running) since Wed 2018-03-28 08:17:33 CST; 9h ago - ``` - -2. Query which **qemu-ga** commands can be added to the blacklist: - - ``` - # qemu-ga --blacklist ? - guest-sync-delimited - guest-sync - guest-ping - guest-get-time - guest-set-time - guest-info - ... - ``` - - -3. Set the blacklist. Add the commands to be shielded to **--blacklist** in the **/usr/lib/systemd/system/qemu-guest-agent.service** file. Use spaces to separate different commands. For example, to add the **guest-file-open** and **guest-file-close** commands to the blacklist, configure the file by referring to the following: - - ``` - [Service] - ExecStart=-/usr/bin/qemu-ga \ - --blacklist=guest-file-open guest-file-close - ``` - - -4. Restart the qemu-guest-agent service. - - ``` - # systemctl daemon-reload - # systemctl restart qemu-guest-agent - ``` - -5. Check whether the qemu-ga blacklist function takes effect on the VM, that is, whether the **--blacklist** parameter configured for the qemu-ga process is correct. - - ``` - # ps -ef|grep qemu-ga|grep -E "blacklist=|b=" - root 727 1 0 08:17 ? 00:00:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open guest-file-close guest-file-read guest-file-write guest-file-seek guest-file-flush -F/etc/qemu-ga/fsfreeze-hook - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >For more information about qemu-ga, visit [https://wiki.qemu.org/Features/GuestAgent](https://wiki.qemu.org/Features/GuestAgent). - - -### sVirt Protection - -#### Overview - -In a virtualization environment that uses the discretionary access control \(DAC\) policy only, malicious VMs running on hosts may attack the hypervisor or other VMs. To improve security in virtualization scenarios, openEuler uses sVirt for protection. sVirt is a security protection technology based on SELinux. It is applicable to KVM virtualization scenarios. A VM is a common process on the host OS. In the hypervisor, the sVirt mechanism labels QEMU processes corresponding to VMs with SELinux labels. In addition to types which are used to label virtualization processes and files, different categories are used to label different VMs. Each VM can access only file devices of the same category. This prevents VMs from accessing files and devices on unauthorized hosts or other VMs, thereby preventing VM escape and improving host and VM security. - -#### Enabling sVirt Protection - -I. Enable SELinux on the host. - 1. Log in to the host. - 2. Enable the SELinux function on the host. - a. Modify the system startup parameter file **grub.cfg** to set **selinux** to **1**. - - ``` - selinux=1 - ``` - - b. Modify **/etc/selinux/config** to set the **SELINUX** to **enforcing**. - - ``` - SELINUX=enforcing - ``` - - 3. Restart the host. - - ``` - # reboot - ``` - - - -II. Create a VM where the sVirt function is enabled. - 1. Add the following information to the VM configuration file: - - ``` - - ``` - - Or check whether the following configuration exists in the file: - - ``` - - ``` - - 2. Create a VM. - - ``` - # virsh define openEulerVM.xml - ``` - -III. Check whether sVirt is enabled. - Run the following command to check whether sVirt protection has been enabled for the QEMU process of the running VM. If **svirt\_t:s0:c** exists, sVirt protection has been enabled. - - ``` - # ps -eZ|grep qemu |grep "svirt_t:s0:c" - system_u:system_r:svirt_t:s0:c200,c947 11359 ? 00:03:59 qemu-kvm - system_u:system_r:svirt_t:s0:c427,c670 13790 ? 19:02:07 qemu-kvm - ``` - - -### VM Trusted Boot - -#### Overview - -Trusted boot includes measure boot and remote attestation. The measure boot function is mainly provided by virtualization component. The remote attestation function is enabled by users who install related software (RA client) on VMs and set up the RA server. - -The two basic elements for measure boot are the root of trust (RoT) and chain of trust. The basic idea is to establish a RoT in the computer system. The trustworthiness of the RoT is ensured by physical security, technical security, and management security, that is, CRTM (Core Root of Trust for Measurement). A chain of trust is established, starting from the RoT to the BIOS/BootLoader, operating system, and then to the application. The measure boot and trust is performed by one level to the previous level. Finally, the trust is extended to the entire system. The preceding process looks like a chain, so it is called a chain of trust. - -The CRTM is the root of the measure boot and the first component of the system startup. No other code is used to check the integrity of the CRTM. Therefore, as the starting point of the chain of trust, it must be an absolutely trusted source of trust. The CRTM needs to be technically designed as a segment of read-only or strictly restricted code to defend against BIOS attacks and prevent remote injection of malicious code or modification of startup code at the upper layer of the operating system. In a physical host, the CPU microcode is used as the CRTM. In a virtualization environment, the sec part of the vBIOS is generally the CRTM. - -During startup, the previous component measures (calculates the hash value) the next component, and then extends the measurement value to the trusted storage area, for example, the PCR of the TPM. The CRTM measurement BootLoader extends the measurement value to the PCR, and the BootLoader measurement OS extends the measurement value to the PCR. - - - -#### Configuring the vTPM Device to Enable Measurement Startup - -**Installing the swtpm and libtpms Software** - -swtpm provides a TPM emulator (TPM 1.2 and TPM 2.0) that can be integrated into a virtualization environment. So far, it has been integrated into QEMU and serves as a prototype system in RunC. swtpm uses libtpms to provide TPM1.2 and TPM2.0 simulation functions. -Currently, openEuler 21.03 provides the libtpms and swtpm sources. You can run the yum command to install them. - -``` -# yum install libtpms swtpm swtpm-devel swtpm-tools - -``` - - -**Configuring the vTPM Device for the VM** - -1. Add the following configuration to the VM configuration file: - - ``` - - ... - - ... - - - - ... - - ... - - ``` - - >![](public_sys-resources/icon-note.gif) **NOTE:** - >Currently, trusted boot of VMs on the AArch64 architecture of openEuler 20.09 does not support the ACPI feature. Therefore, do not configure the ACPI feature for VMs. Otherwise, vTPM devices cannot be identified after VMs are started. If the AArch64 architecture is used in versions earlier than openEuler 22.09, set **tpm model** to **<tpm model='tpm-tis-device'>**. - -2. Create a VM. - - ``` - # virsh define MeasuredBoot.xml - ``` -3. Start the VM. - - Before starting the VM, run the **chmod** command to grant the following permissions to the **/var/lib/swtpm-localca/** directory. Otherwise, libvirt cannot start swtpm. - - ``` - # chmod -R 777 /var/lib/swtpm-localca/ - # - # virsh start MeasuredbootVM - `` - - -**Confirming that the Measure Boot Is Successfully Enabled** - -The vBIOS determines whether to enable the measure boot function. Currently, the vBIOS in openEuler 20.09 has the measure boot capability. If the host machine uses the edk2 component of another version, check whether the edk2 component supports the measure boot function. - -Log in to the VM as the **root** user and check whether the TPM driver, tpm2-tss protocol stack, and tpm2-tools are installed on the VM. -By default, the tpm driver (tpm_tis.ko), tpm2-tss protocol stack, and tpm2-tools are installed in openEuler 21.03. If another OS is used, run the following command to check whether the driver and related tools are installed: - - -``` -# lsmod |grep tpm -# tpm_tis 16384 0 -# -# yum list installed | grep -E 'tpm2-tss|tpm2-tools' -# -# yum install tpm2-tss tpm2-tools -``` -You can run the **tpm2_pcrread** (**tpm2_pcrlist** in tpm2_tools of earlier versions) command to list all PCR values. - -``` -# tpm2_pcrread -sha1 : - 0 : fffdcae7cef57d93c5f64d1f9b7f1879275cff55 - 1 : 5387ba1d17bba5fdadb77621376250c2396c5413 - 2 : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236 - 3 : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236 - 4 : e5d40ace8bb38eb170c61682eb36a3020226d2c0 - 5 : 367f6ea79688062a6df5f4737ac17b69cd37fd61 - 6 : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236 - 7 : 518bd167271fbb64589c61e43d8c0165861431d8 - 8 : af65222affd33ff779780c51fa8077485aca46d9 - 9 : 5905ec9fb508b0f30b2abf8787093f16ca608a5a - 10 : 0000000000000000000000000000000000000000 - 11 : 0000000000000000000000000000000000000000 - 12 : 0000000000000000000000000000000000000000 - 13 : 0000000000000000000000000000000000000000 - 14 : 0000000000000000000000000000000000000000 - 15 : 0000000000000000000000000000000000000000 - 16 : 0000000000000000000000000000000000000000 - 17 : ffffffffffffffffffffffffffffffffffffffff - 18 : ffffffffffffffffffffffffffffffffffffffff - 19 : ffffffffffffffffffffffffffffffffffffffff - 20 : ffffffffffffffffffffffffffffffffffffffff - 21 : ffffffffffffffffffffffffffffffffffffffff - 22 : ffffffffffffffffffffffffffffffffffffffff - 23 : 0000000000000000000000000000000000000000 -sha256 : - 0 : d020873038268904688cfe5b8ccf8b8d84c1a2892fc866847355f86f8066ea2d - 1 : 13cebccdb194dd916f2c0c41ec6832dfb15b41a9eb5229d33a25acb5ebc3f016 - 2 : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969 - 3 : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969 - 4 : 07f9074ccd4513ef1cafd7660f9afede422b679fd8ad99d25c0659eba07cc045 - 5 : ba34c80668f84407cd7f498e310cc4ac12ec6ec43ea8c93cebb2a688cf226aff - 6 : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969 - 7 : 65caf8dd1e0ea7a6347b635d2b379c93b9a1351edc2afc3ecda700e534eb3068 - 8 : f440af381b644231e7322babfd393808e8ebb3a692af57c0b3a5d162a6e2c118 - 9 : 54c08c8ba4706273f53f90085592f7b2e4eaafb8d433295b66b78d9754145cfc - 10 : 0000000000000000000000000000000000000000000000000000000000000000 - 11 : 0000000000000000000000000000000000000000000000000000000000000000 - 12 : 0000000000000000000000000000000000000000000000000000000000000000 - 13 : 0000000000000000000000000000000000000000000000000000000000000000 - 14 : 0000000000000000000000000000000000000000000000000000000000000000 - 15 : 0000000000000000000000000000000000000000000000000000000000000000 - 16 : 0000000000000000000000000000000000000000000000000000000000000000 - 17 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff - 18 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff - 19 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff - 20 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff - 21 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff - 22 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff - 23 : 0000000000000000000000000000000000000000000000000000000000000000 -``` diff --git a/docs/en/docs/Virtualization/environment-preparation.md b/docs/en/docs/Virtualization/environment-preparation.md deleted file mode 100644 index b9c39a0897156e257aca2b00c1e628b2fb393ce5..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/environment-preparation.md +++ /dev/null @@ -1,367 +0,0 @@ -# Environment Preparation - -- [Environment Preparation](#environment-preparation) - - [Preparing a VM Image](#preparing-a-vm-image) - - [Preparing the VM Network](#preparing-the-vm-network) - - [Preparing Boot Firmware](#preparing-boot-firmware) - -## Preparing a VM Image - -### Overview - -A VM image is a file that contains a virtual disk that has been installed and can be used to start the OS. VM images are in different formats, such as raw and qcow2. Compared with the raw format, the qcow2 format occupies less space and supports features such as snapshot, copy-on-write, AES encryption, and zlib compression. However, the performance of the qcow2 format is slightly lower than that of the raw format. The qemu-img tool is used to create image files. This section uses the qcow2 image file as an example to describe how to create a VM image. - -### Creating an Image - -To create a qcow2 image file, perform the following steps: - -1. Install the **qemu-img** software package. - - ```shell - yum install -y qemu-img - ``` - -2. Run the **create** command of the qemu-img tool to create an image file. The command format is as follows: - - ```shell - qemu-img create -f -o - ``` - - The parameters are described as follows: - - - _imgFormat_: Image format. The value can be **raw** or **qcow2**. - - _fileOption_: File option, which is used to set features of an image file, such as specifying a backend image file, compression, and encryption. - - _fileName_: File name. - - _diskSize_: Disk size, which specifies the size of a block disk. The unit can be K, M, G, or T, indicating KiB, MiB, GiB, or TiB. - - For example, to create an image file openEuler-image.qcow2 whose disk size is 4 GB and format is qcow2, the command and output are as follows: - - ```shell - $ qemu-img create -f qcow2 openEuler-image.qcow2 4G - Formatting 'openEuler-image.qcow2', fmt=qcow2 size=4294967296 cluster_size=65536 lazy_refcounts=off refcount_bits=16 - ``` - -### Changing the Image Disk Space - -If a VM requires larger disk space, you can use the qemu-img tool to change the disk space of the VM image. The method is as follows: - -1. Run the following command to query the disk space of the VM image: - - ```shell - qemu-img info - ``` - - For example, if the command and output for querying the disk space of the openEuler-image.qcow2 image are as follows, the disk space of the image is 4 GiB. - - ```shell - $ qemu-img info openEuler-image.qcow2 - image: openEuler-image.qcow2 - file format: qcow2 - virtual size: 4.0G (4294967296 bytes) - disk size: 196K - cluster_size: 65536 - Format specific information: - compat: 1.1 - lazy refcounts: false - refcount bits: 16 - corrupt: false - ``` - -2. Run the following command to change the image disk space. In the command, _imgFileName_ indicates the image name, and **+** and **-** indicate the image disk space to be increased and decreased, respectively. The unit is KB, MB, GB, and T, indicating KiB, MiB, GiB, and TiB, respectively. - - ```shell - qemu-img resize [+|-] - ``` - - For example, to expand the disk space of the openEuler-image.qcow2 image to 24 GiB, that is, to add 20 GiB to the original 4 GiB, the command and output are as follows: - - ```shell - $ qemu-img resize openEuler-image.qcow2 +20G - Image resized. - ``` - -3. Run the following command to check whether the image disk space is changed successfully: - - ```shell - qemu-img info - ``` - - For example, if the openEuler-image.qcow2 image disk space has been expanded to 24 GiB, the command and output are as follows: - - ```shell - $ qemu-img info openEuler-image.qcow2 - image: openEuler-image.qcow2 - file format: qcow2 - virtual size: 24G (25769803776 bytes) - disk size: 200K - cluster_size: 65536 - Format specific information: - compat: 1.1 - lazy refcounts: false - refcount bits: 16 - corrupt: false - ``` - -## Preparing the VM Network - -### Overview - -To enable the VM to communicate with external networks, you need to configure the network environment for the VM. KVM virtualization supports multiple types of bridges, such as Linux bridge and Open vSwitch bridge. As shown in [Figure 1](#fig1785384714917), the data transmission path is **VM \> virtual NIC device \> Linux bridge or Open vSwitch bridge \> physical NIC**. In addition to configuring virtual NICs \(vNICs\) for VMs, creating a bridge for a host is the key to connecting to a virtualized network. - -This section describes how to set up a Linux bridge and an Open vSwitch bridge to connect a VM to the network. You can select a bridge type based on the site requirements. - -**Figure 1** Virtual network structure -![](./figures/virtual-network-structure.png "virtual-network-structure") - -### Setting Up a Linux Bridge - -The following describes how to bind the physical NIC eth0 to the Linux bridge br0. - -1. Install the **bridge-utils** software package. - - The Linux bridge is managed by the brctl tool. The corresponding installation package is bridge-utils. The installation command is as follows: - - ```shell - yum install -y bridge-utils - ``` - -2. Create bridge br0. - - ```shell - brctl addbr br0 - ``` - -3. Bind the physical NIC eth0 to the Linux bridge. - - ```shell - brctl addif br0 eth0 - ``` - -4. After eth0 is connected to the bridge, the IP address of eth0 is set to 0.0.0.0. - - ```shell - ifconfig eth0 0.0.0.0 - ``` - -5. Set the IP address of br0. - - If a DHCP server is available, set a dynamic IP address through the dhclient. - - ```shell - dhclient br0 - ``` - - - If no DHCP server is available, configure a static IP address for br0. For example, set the static IP address to 192.168.1.2 and subnet mask to 255.255.255.0. - - ```shell - ifconfig br0 192.168.1.2 netmask 255.255.255.0 - ``` - -### Setting Up an Open vSwitch Bridge - -The Open vSwitch bridge provides more convenient automatic orchestration capabilities. This section describes how to install network virtualization components to set up an Open vSwitch bridge. - -**1. Install the Open vSwitch component.** - -If the Open vSwitch is used to provide virtual network, you need to install the Open vSwitch network virtualization component. - -1. Install the Open vSwitch component. - - ```shell - yum install -y openvswitch - ``` - -2. Start the Open vSwitch service. - - ```shell - systemctl start openvswitch - ``` - -**2. Check whether the installation is successful.** - -1. Check whether the openvswitch component is successfully installed. If the installation is successful, the software package information is displayed. The command and output are as follows: - - ```shell - $ rpm -qi openvswitch - Name : openvswitch - Version : 2.11.1 - Release : 1 - Architecture: aarch64 - Install Date: Thu 15 Aug 2019 05:08:35 PM CST - Group : System Environment/Daemons - Size : 6051185 - License : ASL 2.0 - Signature : (none) - Source RPM : openvswitch-2.11.1-1.src.rpm - Build Date : Thu 08 Aug 2019 05:24:46 PM CST - Build Host : armbuild10b247b121b105 - Relocations : (not relocatable) - Vendor : Nicira, Inc. - URL : http://www.openvswitch.org/ - Summary : Open vSwitch daemon/database/utilities - Description : - Open vSwitch provides standard network bridging functions and - support for the OpenFlow protocol for remote per-flow control of - traffic. - ``` - -2. Check whether the Open vSwitch service is started successfully. If the service is in the **Active** state, the service is started successfully. You can use the command line tool provided by the Open vSwitch. The command and output are as follows: - - ```shell - $ systemctl status openvswitch - ● openvswitch.service - LSB: Open vSwitch switch - Loaded: loaded (/etc/rc.d/init.d/openvswitch; generated) - Active: active (running) since Sat 2019-08-17 09:47:14 CST; 4min 39s ago - Docs: man:systemd-sysv-generator(8) - Process: 54554 ExecStart=/etc/rc.d/init.d/openvswitch start (code=exited, status=0/SUCCESS) - Tasks: 4 (limit: 9830) - Memory: 22.0M - CGroup: /system.slice/openvswitch.service - ├─54580 ovsdb-server: monitoring pid 54581 (healthy) - ├─54581 ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate> - ├─54602 ovs-vswitchd: monitoring pid 54603 (healthy) - └─54603 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/open> - ``` - -**3. Set up an Open vSwitch bridge** - -The following describes how to set up an Open vSwitch layer-1 bridge br0. - -1. Create the Open vSwitch bridge br0. - - ```shell - ovs-vsctl add-br br0 - ``` - -2. Add the physical NIC eth0 to br0. - - ```shell - ovs-vsctl add-port br0 eth0 - ``` - -3. After eth0 is connected to the bridge, the IP address of eth0 is set to 0.0.0.0. - - ```shell - ifconfig eth0 0.0.0.0 - ``` - -4. Assign an IP address to OVS bridge br0. - - If a DHCP server is available, set a dynamic IP address through the dhclient. - - ```shell - dhclient br0 - ``` - - - If no DHCP server is available, configure a static IP address for br0, for example, 192.168.1.2. - - ```shell - ifconfig br0 192.168.1.2 - ``` - -## Preparing Boot Firmware - -### Overview - -The boot mode varies depending on the architecture. x86 servers support the Unified Extensible Firmware Interface \(UEFI\) and BIOS boot modes, and AArch64 servers support only the UEFI boot mode. By default, boot files corresponding to the BIOS mode have been installed on openEuler. No additional operations are required. This section describes how to install boot files corresponding to the UEFI mode. - -The Unified Extensible Firmware Interface \(UEFI\) is a new interface standard used for power-on auto check and OS boot. It is an alternative to the traditional BIOS. EDK II is a set of open source code that implements the UEFI standard. In virtualization scenarios, the EDK II tool set is used to start a VM in UEFI mode. Before using the EDK II tool, you need to install the corresponding software package before starting a VM. This section describes how to install the EDK II tool. - -### Installation Methods - -If the UEFI mode is used, the tool set EDK II needs to be installed. The installation package for the AArch64 architecture is **edk2-aarch64**, and that for the x86 architecture is **edk2-ovmf**. This section uses the AArch64 architecture as an example to describe the installation method. For the x86 architecture, you only need to replace **edk2-aarch64** with **edk2-ovmf**. - -1. Run the following command to install the **edk** software package: - - In the AArch64 architecture, the **edk2** package name is **edk2-aarch64**. - - ```shell - yum install -y edk2-aarch64 - ``` - - In the x86\_64 architecture, the **edk2** package name is **edk2-ovmf**. - - ```shell - yum install -y edk2-ovmf - ``` - -2. Run the following command to check whether the **edk** software package is successfully installed: - - In the AArch64 architecture, the command is as follows: - - ```shell - rpm -qi edk2-aarch64 - ``` - - If information similar to the following is displayed, the **edk** software package is successfully installed: - - ```text - Name : edk2-aarch64 - Version : 20180815gitcb5f4f45ce - Release : 1.oe3 - Architecture: noarch - Install Date: Mon 22 Jul 2019 04:52:33 PM CST - Group : Applications/Emulators - ``` - - In the x86\_64 architecture, the command is as follows: - - ```shell - rpm -qi edk2-ovmf - ``` - - If information similar to the following is displayed, the **edk** software package is successfully installed: - - ```text - Name : edk2-ovmf - Version : 201908 - Release : 6.oe1 - Architecture: noarch - Install Date: Thu 19 Mar 2020 09:09:06 AM CST - ``` - -## Configuring a Non-root User - -### Overview - -openEuler virtualization uses virsh to manage VMs. If you want to use the virsh commands to manage VMs as a non-root user, you need to perform related configurations before using the commands. This section provides the configuration guide. - -### Operation Guide - -To allow a non-root user to run the virsh commands to manage VMs, perform the following operations (replace **userName** in the following commands with the actual non-root user name): - -1. Log in to the host as the **root** user. - -2. Add the non-root user to the libvirt user group. - - ```shell - usermod -a -G libvirt userName - ``` - -3. Switch to the non-root user. - - ```shell - su userName - ``` - -4. Configure environment variables for the non-root user. Run the **vim** command to open the ~/.bashrc file: - - ```shell - vim ~/.bashrc - ``` - - Add the following content to the end of the file and save the file: - - ```shell - export LIBVIRT_DEFAULT_URI="qemu:///system" - ``` - - Run the following command for the configuration to take effect: - - ```shell - source ~/.bashrc - ``` - -5. Add the following content to the domain root element in the XML configuration file of the VM so that the qemu-kvm process can access the disk image files. - - ```xml - - ``` diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP1.png b/docs/en/docs/Virtualization/figures/CertEnrollP1.png deleted file mode 100644 index 536e0618a3ab5b70937292205242a08237e34712..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP1.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP2.png b/docs/en/docs/Virtualization/figures/CertEnrollP2.png deleted file mode 100644 index 0557c8782960188dbe9d84a1d0e66c9b45d2b303..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP2.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP3.png b/docs/en/docs/Virtualization/figures/CertEnrollP3.png deleted file mode 100644 index 326fcf1e8d5e3c795ebcde286d8e0fef14bec7d1..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP3.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP4.png b/docs/en/docs/Virtualization/figures/CertEnrollP4.png deleted file mode 100644 index bc77c038e1e3a5ec30d7ba4f805ca937792e9327..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP4.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP5.png b/docs/en/docs/Virtualization/figures/CertEnrollP5.png deleted file mode 100644 index 0f22b3cbd84f7c93f74898a926bc3e32f231667f..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP5.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP6.png b/docs/en/docs/Virtualization/figures/CertEnrollP6.png deleted file mode 100644 index 08235013ca71f1ec51e9af2f143629d1a6132fe9..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP6.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP7.png b/docs/en/docs/Virtualization/figures/CertEnrollP7.png deleted file mode 100644 index f934521d59dd4a75449fcb2ca8abc54045b9102b..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP7.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/CertEnrollP8.png b/docs/en/docs/Virtualization/figures/CertEnrollP8.png deleted file mode 100644 index 9a8158e3378bf25dee05b892cc60f424542455d7..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/CertEnrollP8.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/OSBootFlow.png b/docs/en/docs/Virtualization/figures/OSBootFlow.png deleted file mode 100644 index f496c5675c72359e5160384c766a11399b04bfa6..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/OSBootFlow.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/SecureBootFlow.png b/docs/en/docs/Virtualization/figures/SecureBootFlow.png deleted file mode 100644 index d639975800752c6eca6765a416c256a4752fb590..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/SecureBootFlow.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/en-us_image_0218587435.png b/docs/en/docs/Virtualization/figures/en-us_image_0218587435.png deleted file mode 100644 index a6107f2308d194c92ebe75b58e9125819e7fe9eb..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/en-us_image_0218587435.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/en-us_image_0218587436.png b/docs/en/docs/Virtualization/figures/en-us_image_0218587436.png deleted file mode 100644 index 28a8d25b19c5a5ed043a8f4701b8f920de365ea2..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/en-us_image_0218587436.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/kvm-architecture.png b/docs/en/docs/Virtualization/figures/kvm-architecture.png deleted file mode 100644 index 74cc91f2944b4ed5404edf036b1d71cd84df7e29..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/kvm-architecture.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/status-transition-diagram.png b/docs/en/docs/Virtualization/figures/status-transition-diagram.png deleted file mode 100644 index f0e9bb814b155e4abde372c93677a97f207b5475..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/status-transition-diagram.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/virtual-network-structure.png b/docs/en/docs/Virtualization/figures/virtual-network-structure.png deleted file mode 100644 index 4d57a184352c1a3558eeac56499888b2b98b31f1..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/virtual-network-structure.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/figures/virtualized-architecture.png b/docs/en/docs/Virtualization/figures/virtualized-architecture.png deleted file mode 100644 index 2e8b01628fb51bb6cc1162d6158259192506bc3a..0000000000000000000000000000000000000000 Binary files a/docs/en/docs/Virtualization/figures/virtualized-architecture.png and /dev/null differ diff --git a/docs/en/docs/Virtualization/introduction-to-virtualization.md b/docs/en/docs/Virtualization/introduction-to-virtualization.md deleted file mode 100644 index 04370eebbcc81ea3485835c0affe811478a77523..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/introduction-to-virtualization.md +++ /dev/null @@ -1,90 +0,0 @@ -# Introduction to Virtualization - -## Overview - -In computer technologies, virtualization is a resource management technology. It abstracts various physical resources \(such as processors, memory, disks, and network adapters\) of a computer, converts the resources, and presents the resources for segmentation and combination into one or more computer configuration environments. This resource management technology breaks the inseparable barrier of the physical structure, and makes these resources not restricted by the architecture or geographical or physical configuration of the existing resources after virtualization.. In this way, users can better leverage the computer hardware resources and maximize the resource utilization. - -Virtualization enables multiple virtual machines \(VMs\) to run on a physical server. VMs share resources such as processors, memory, and I/O devices of physical machines, but are logically isolated from each other. In the virtualization technology, the physical server is called a host machine, the VM running on the host machine is called a guest, and the operating system \(OS\) running on the VM is called a guest OS. A layer of software, called the virtualization layer, exists between a host machine and a VM to simulate virtual hardware. This virtualization layer is called a VM monitor, as shown in the following figure. - -**Figure 1** Virtualized architecture -![](./figures/virtualized-architecture.png "virtualized-architecture") - -## Virtualized Architecture - -Currently, mainstream virtualization technologies are classified into two types based on the implementation structure of the Virtual Machine Monitor \(VMM\): - -- Hypervisor model - - In this model, VMM is considered as a complete operating system \(OS\) and has the virtualization function. VMM directly manages all physical resources, including processors, memory, and I/O devices. - -- Host model - - In this model, physical resources are managed by a host OS, which is a traditional OS, such as Linux and Windows. The host OS does not provide the virtualization capability. The VMM that provides the virtualization capability runs on the host OS as a driver or software of the system. The VMM invokes the host OS service to obtain resources and simulate the processor, memory, and I/O devices. The virtualization implementation of this model includes KVM and Virtual Box. - - -Kernel-based Virtual Machine \(KVM\) is a kernel module of Linux. It makes Linux a hypervisor. [Figure 2](#fig310953013541) shows the KVM architecture. KVM does not simulate any hardware device. It is used to enable virtualization capabilities provided by the hardware, such as Intel VT-x, AMD-V, ARM virtualization extensions. The user-mode QEMU simulates the mainboard, memory, and I/O devices. The user-mode QEMU works with the KVM module to simulate VM hardware. The guest OS runs on the hardware simulated by the QEMU and KVM. - -**Figure 2** KVM architecture -![](./figures/kvm-architecture.png "kvm-architecture") - -## Virtualization Components - -Virtualization components provided in the openEuler software package: - -- KVM: provides the core virtualization infrastructure to make the Linux system a hypervisor. Multiple VMs can run on the same host at the same time. -- QEMU: simulates a processor and provides a set of device models to work with KVM to implement hardware-based virtualization simulation acceleration. -- Libvirt: provides a tool set for managing VMs, including unified, stable, and open application programming interfaces \(APIs\), daemon process \(libvirtd\), and default command line management tool \(virsh\). -- Open vSwitch: provides a virtual network tool set for VMs, supports programming extension and standard management interfaces and protocols \(such as NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, and 802.1ag\). - -## Virtualization Characteristics - -Virtualization has the following characteristics: - -- Partition - - Virtualization can logically divide software on a physical server to run multiple VMs \(virtual servers\) with different specifications. - - -- Isolation - - Virtualization can simulate virtual hardware and provide hardware conditions for VMs to run complete OSs. The OSs of each VM are independent and isolated from each other. For example, if the OS of a VM breaks down due to a fault or malicious damage, the OSs and applications of other VMs are not affected. - - -- Encapsulation - - Encapsulation is performed on a per VM basis. The excellent encapsulation capability makes VMs more flexible than physical machines. Functions such as live migration, snapshot, and cloning of VMs can be realized, implementing quick deployment and automatic O&M of data centers. - - -- Hardware-irrelevant - - After being abstracted by the virtualization layer, VMs are not directly bound to underlying hardware and can run on other servers without being modified. - - -## Virtualization Advantages - -Virtualization brings the following benefits to infrastructure of the data center: - -- Flexibility and scalability - - Users can dynamically allocate and reclaim resources based to meet dynamic service requirements. In addition, users can plan different VM specifications based on product requirements and adjust the scale without changing the physical resource configuration. - - -- Higher availability and better O&M methods - - Virtualization provides O&M methods such as live migration, snapshot, live upgrade, and automatic DR. Physical resources can be deleted, upgraded, or changed without affecting users, improving service continuity and implementing automatic O&M. - - -- Security hardening - - Virtualization provides OS-level isolation and hardware-based processor operation privilege-level control. Compared with simple sharing mechanisms, virtualization provides higher security and implements controllable and secure access to data and services. - - -- High resource utilization - - Virtualization supports dynamic sharing of physical resources and resource pools, improving resource utilization. - - -## openEuler Virtualization - -openEuler provides KVM virtualization components that support the AArch64 and x86\_64 processor architectures. - diff --git a/docs/en/docs/Virtualization/managing-devices.md b/docs/en/docs/Virtualization/managing-devices.md deleted file mode 100644 index b8ee22dc217551d458ee5d0c341394d1fc0c509e..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/managing-devices.md +++ /dev/null @@ -1,796 +0,0 @@ -# Managing Devices - -- [Managing Devices](#managing-devices) - - [Configuring a PCIe Controller for a VM](#configuring-a-pcie-controller-for-a-vm) - - [Overview](#overview) - - [Configuring the PCIe Root, PCIe Root Port, and PCIe-PCI-Bridge](#configuring-the-pcie-root-pcie-root-port-and-pcie-pci-bridge) - - [Managing Virtual Disks](#managing-virtual-disks) - - [Managing vNICs](#managing-vnics) - - [Configuring a Virtual Serial Port](#configuring-a-virtual-serial-port) - - [Managing Device Passthrough](#managing-device-passthrough) - - [PCI Passthrough](#pci-passthrough) - - [SR-IOV Passthrough](#sr-iov-passthrough) - - [Managing VM USB](#managing-vm-usb) - - [Configuring USB Controllers](#configuring-usb-controllers) - - [Configuring a USB Passthrough Device](#configuring-a-usb-passthrough-device) - - [Storing Snapshots](#storing-snapshots) - - [Overview](#overview-7) - - [Procedure](#procedure-4) - - [Configuring Disk I/O Suspension](#configuring-disk-io-suspension) - - [Introduction](#introduction) - - [Overview](#overview-8) - - [Application Scenarios](#application-scenarios) - - [Precautions and Restrictions](#precautions-and-restrictions) - - [Disk I/O Suspension Configuration](#disk-io-suspension-configuration) - - [Using the QEMU CLI](#using-the-qemu-cli) - - [Using an XML Configuration File](#using-an-xml-configuration-file) - - -## Configuring a PCIe Controller for a VM - -### Overview - -The NIC, disk controller, and PCIe pass-through devices in a VM must be mounted to a PCIe root port. Each root port corresponds to a PCIe slot. The devices mounted to the root port support hot swap, but the root port does not support hot swap. Therefore, users need to consider the hot swap requirements and plan the maximum number of PCIe root ports reserved for the VM. Before the VM is started, the root port is statically configured. - -### Configuring the PCIe Root, PCIe Root Port, and PCIe-PCI-Bridge - -The VM PCIe controller is configured using the XML file. The **model** corresponding to PCIe root, PCIe root port, and PCIe-PCI-bridge in the XML file are **pcie-root**, **pcie-root-port**, and **pcie-to-pci-bridge**, respectively. - -- Simplified configuration method - - Add the following contents to the XML file of the VM. Other attributes of the controller are automatically filled by libvirt. - - ``` - - - - - - - ``` - - The **pcie-root** and **pcie-to-pci-bridge** occupy one **index** respectively. Therefore, the final **index** is the number of required **root ports + 1**. - -- Complete configuration method - - Add the following contents to the XML file of the VM: - - ``` - - - - -
- - - -
- - - - -
- - - ``` - - In the preceding contents: - - - The **chassis** and **port** attributes of the root port must be in ascending order. Because a PCIe-PCI-bridge is inserted in the middle, the **chassis** number skips **2**, but the **port** numbers are still consecutive. - - The **address function** of the root port ranges from **0\*0** to **0\*7**. - - A maximum of eight functions can be mounted to each slot. When the slot is full, the slot number increases. - - The complete configuration method is complex. Therefore, the simplified one is recommended. - - -## Managing Virtual Disks - -### Overview - -Virtual disk types include virtio-blk, virtio-scsi, and vhost-scsi. virtio-blk simulates a block device, and virtio-scsi and vhost-scsi simulate SCSI devices. - -- virtio-blk: It can be used for common system disk and data disk. In this configuration, the virtual disk is presented as **vd\[a-z\]** or **vd\[a-z\]\[a-z\]** in the VM. -- virtio-scsi: It is recommended for common system disk and data disk. In this configuration, the virtual disk is presented as **sd\[a-z\]** or **sd\[a-z\]\[a-z\]** in the VM. -- vhost-scsi: It is recommended for the virtual disk that has high performance requirements. In this configuration, the virtual disk is presented as **sd\[a-z\]** or **sd\[a-z\]\[a-z\]** on the VM. - -### Procedure - -For details about how to configure a virtual disk, see **VM Configuration** > **Network Devices**. This section uses the virtio-scsi disk as an example to describe how to attach and detach a virtual disk. - -- Attach a virtio-scsi disk. - - Run the **virsh attach-device** command to attach the virtio-scsi virtual disk. - - ``` - # virsh attach-device - ``` - - The preceding command can be used to attach a disk to a VM online. The disk information is specified in the **attach-device.xml** file. The following is an example of the **attach-device.xml** file: - - ``` - ### attach-device.xml ### - - - - - -
- - ``` - - The disk attached by running the preceding commands becomes invalid after the VM is shut down and restarted. If you need to permanently attach a virtual disk to a VM, run the **virsh attach-device** command with the **--config** parameter. - -- Detach a virtio-scsi disk. - - If a disk attached online is no longer used, run the **virsh detach** command to dynamically detach it. - - ``` - # virsh detach-device - ``` - - **detach-device.xml** specifies the XML information of the disk to be detached, which must be the same as the XML information during dynamic attachment. - - -## Managing vNICs - -### Overview - -The vNIC types include virtio-net, vhost-net, and vhost-user. After creating a VM, you may need to attach or detach a vNIC. openEuler supports NIC hot swap, which can change the network throughput and improve system flexibility and scalability. - -### Procedure - -For details about how to configure a virtual NIC, see [3.2.4.2 Network Devices](#network-device). This section uses the vhost-net NIC as an example to describe how to attach and detach a vNIC. - -- Attach the vhost-net NIC. - - Run the **virsh attach-device** command to attach the vhost-net vNIC. - - ``` - # virsh attach-device - ``` - - The preceding command can be used to attach a vhost-net NIC to a running VM. The NIC information is specified in the **attach-device.xml** file. The following is an example of the **attach-device.xml** file: - - ``` - ### attach-device.xml ### - - - - - - - - ``` - - The vhost-net NIC attached using the preceding commands becomes invalid after the VM is shut down and restarted. If you need to permanently attach a vNIC to a VM, run the **virsh attach-device** command with the **--config** parameter. - -- Detach the vhost-net NIC. - - If a NIC attached online is no longer used, run the **virsh detach** command to dynamically detach it. - - ``` - # virsh detach-device - ``` - - **detach-device.xml** specifies the XML information of the vNIC to be detached, which must be the same as the XML information during dynamic attachment. - - -## Configuring a Virtual Serial Port - -### Overview - -In a virtualization environment, VMs and host machines need to communicate with each other to meet management and service requirements. However, in the complex network architecture of the cloud management system, services running on the management plane and VMs running on the service plane cannot communicate with each other at layer 3. As a result, service deployment and information collection are not fast enough. Therefore, a virtual serial port is required for communication between VMs and host machines. You can add serial port configuration items to the XML configuration file of a VM to implement communication between VMs and host machines. - -### Procedure - -The Linux VM serial port console is a pseudo terminal device connected to the host machine through the serial port of the VM. It implements interactive operations on the VM through the host machine. In this scenario, the serial port needs to be configured in the pty type. This section describes how to configure a pty serial port. - -- Add the following virtual serial port configuration items under the **devices** node in the XML configuration file of the VM: - - ``` - - - - - - ``` - -- Run the **virsh console** command to connect to the pty serial port of the running VM. - - ``` - # virsh console - ``` - -- To ensure that no serial port message is missed, use the **--console** option to connect to the serial port when starting the VM. - - ``` - # virsh start --console - ``` - - -## Managing Device Passthrough - -The device passthrough technology enables VMs to directly access physical devices. The I/O performance of VMs can be improved in this way. - -Currently, the VFIO passthrough is used. It can be classified into PCI passthrough and SR-IOV passthrough based on device type. - -### PCI Passthrough - -PCI passthrough directly assigns a physical PCI device on the host to a VM. The VM can directly access the device. PCI passthrough uses the VFIO device passthrough mode. The PCI passthrough configuration file in XML format for a VM is as follows: - -``` - - - -
- - -
- -``` - -**Table 1** Device configuration items for PCI passthrough - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value

-

hostdev.source.address.domain

-

Domain ID of the PCI device on the host OS.

-

≥ 0

-

hostdev.source.address.bus

-

Bus ID of the PCI device on the host OS.

-

≥ 1

-

hostdev.source.address.slot

-

Device ID of the PCI device on the host OS.

-

≥ 0

-

hostdev.source.address.function

-

Function ID of the PCI device on the host OS.

-

≥ 0

-

hostdev.driver.name

-

Backend driver of PCI passthrough. This parameter is optional.

-

vfio (default value)

-

hostdev.rom

-

Whether the VM can access the ROM of the passthrough device.

-

This parameter can be set to on or off. The default value is on.

-
  • on: indicates that the VM can access the ROM of the passthrough device. For example, if a VM with a passthrough NIC needs to boot from the preboot execution environment (PXE), or a VM with a passthrough Host Bus Adapter (HBA) card needs to boot from the ROM, you can set this parameter to on.
  • off: indicates that the VM cannot access the ROM of the passthrough device.
-

hostdev.address.type

-

Device type displayed on the guest, which must be the same as the actual device type.

-

**pci** (default configuration)

-

hostdev.address.domain

-

Domain number of the device displayed on the guest.

-

0x0000

-

hostdev.address.bus

-

Bus number of the device displayed on the guest.

-

**0x00** (default configuration). This parameter can only be set to the bus number configured in section "Configuring a PCIe Controller for a VM."

-

hostdev.address.slot

-

Slot number of the device displayed on the guest.

-

The slot number range is [0x03,0x1e]

-

Note:

-
  • The first slot number 0x00 is occupied by the system, the second slot number 0x01 is occupied by the IDE controller and USB controller, and the third slot number 0x02 is occupied by the video.
  • The last slot number 0x1f is occupied by the pvchannel.
-

hostdev.address.function

-

Function number of the device displayed on the guest.

-

**0x0** (default configuration): The function number range is [0x0,0x7]

-
- ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->VFIO passthrough is implemented by IOMMU group. Devices are divided to IOMMU groups based on access control services (ACS) on hardware. Devices in the same IOMMU group can be assigned to only one VM. If multiple functions on a PCI device belong to the same IOMMU group, they can be directly assigned to only one VM as well. - -### SR-IOV Passthrough - -#### Overview - -Single Root I/O Virtualization (SR-IOV) is a hardware-based virtualization solution. With the SR-IOV technology, a physical function (PF) can provide multiple virtual functions (VFs), and each VF can be directly assigned to a VM. This greatly improves hardware resource utilization and I/O performance of VMs. A typical application scenario is SR-IOV passthrough for NICs. With the SR-IOV technology, a physical NIC (PF) can function as multiple VF NICs, and then the VFs can be directly assigned to VMs. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->- SR-IOV requires the support of physical hardware. Before using SR-IOV, ensure that the hardware device to be directly assigned supports SR-IOV and the device driver on the host OS works in SR-IOV mode. ->- The following describes how to query the NIC model: ->In the following command output, values in the first column indicate the PCI numbers of NICs, and **19e5:1822** indicates the vendor ID and device ID of the NIC. ->``` -># lspci | grep Ether ->05:00.0 Ethernet controller: Device 19e5:1822 (rev 45) ->07:00.0 Ethernet controller: Device 19e5:1822 (rev 45) ->09:00.0 Ethernet controller: Device 19e5:1822 (rev 45) ->0b:00.0 Ethernet controller: Device 19e5:1822 (rev 45) ->81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) ->81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) ->``` - - -#### Procedure - -To configure SR-IOV passthrough for a NIC, perform the following steps: - -1. Enable the SR-IOV mode for the NIC. - 1. Ensure that VF driver support provided by the NIC supplier exists on the guest OS. Otherwise, VFs in the guest OS cannot work properly. - 2. Enable the SMMU/IOMMU support in the BIOS of the host OS. The enabling method varies depending on the servers of different vendors. For details, see the help documents of the servers. - 3. Configure the host driver to enable the SR-IOV VF mode. The following uses the Hi1822 NIC as an example to describe how to enable 16 VFs. - - ``` - echo 16 > /sys/class/net/ethX/device/sriov_numvfs - ``` - -2. Obtain the PCI BDF information of PFs and VFs. - 1. Run the following command to obtain the NIC resource list on the current board: - - ``` - # lspci | grep Eth - 03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) - 04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) - 05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) - 06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) - 7d:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Device a222 (rev 20) - 7d:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Device a222 (rev 20) - 7d:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Device a221 (rev 20) - 7d:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Device a221 (rev 20) - ``` - - 2. Run the following command to view the PCI BDF information of VFs: - - ``` - # lspci | grep "Virtual Function" - 03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:00.5 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:00.6 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:00.7 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:01.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:01.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - 03:01.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) - ``` - - 3. Select an available VF and write its configuration to the VM configuration file based on its BDF information. For example, the bus ID of the device **03:00.1** is **03**, its slot ID is **00**, and its function ID is **1**. - -3. Identify and manage the mapping between PFs and VFs. - 1. Identify VFs corresponding to a PF. The following uses PF 03.00.0 as an example: - - ``` - # ls -l /sys/bus/pci/devices/0000\:03\:00.0/ - ``` - - The following symbolic link information is displayed. You can obtain the VF IDs (virtfnX) and PCI BDF IDs based on the information. - - 2. Identify the PF corresponding to a VF. The following uses VF 03:00.1 as an example: - - ``` - # ls -l /sys/bus/pci/devices/0000\:03\:00.1/ - ``` - - The following symbolic link information is displayed. You can obtain PCI BDF IDs of the PF based on the information. - - ``` - lrwxrwxrwx 1 root root 0 Mar 28 22:44 physfn -> ../0000:03:00.0 - ``` - - 3. Obtain names of NICs corresponding to the PFs or VFs. For example: - - ``` - # ls /sys/bus/pci/devices/0000:03:00.0/net - eth0 - ``` - - 4. Set the MAC address, VLAN, and QoS information of VFs to ensure that the VFs are in the **Up** state before passthrough. The following uses VF 03:00.1 as an example. The PF is eth0 and the VF ID is **0**. - - ``` - # ip link set eth0 vf 0 mac 90:E2:BA:21:XX:XX #Sets the MAC address. - # ifconfig eth0 up - # ip link set eth0 vf 0 rate 100 #Sets the VF outbound rate, in Mbit/s. - # ip link show eth0 #Views the MAC address, VLAN ID, and QoS information to check whether the configuration is successful. - ``` - -4. Mount the SR-IOV NIC to the VM. - - When creating a VM, add the SR-IOV passthrough configuration item to the VM configuration file. - - ``` - - - -
- - - - - - ``` - - **Table 1** SR-IOV configuration options - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter

-

Description

-

Value

-

hostdev.managed

-

Two modes for libvirt to process PCI devices.

-

no: default value. The passthrough device is managed by the user.

-

yes: The passthrough device is managed by libvirt. Set this parameter to yes in the SR-IOV passthrough scenario.

-

hostdev.source.address.bus

-

Bus ID of the PCI device on the host OS.

-

≥ 1

-

hostdev.source.address.slot

-

Device ID of the PCI device on the host OS.

-

≥ 0

-

hostdev.source.address.function

-

Function ID of the PCI device on the host OS.

-

≥ 0

-
- - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >Disabling the SR-IOV function: - >To disable the SR-IOV function after the VM is stopped and no VF is in use, run the following command: - >The following uses the Hi1822 NIC corresponding network interface name: eth0) as an example: - >``` - >echo 0 > /sys/class/net/eth0/device/sriov_numvfs - >``` - -#### Configuring SR-IOV Passthrough for the HPRE Accelerator - -The accelerator engine is a hardware acceleration solution provided by TaiShan 200 servers based on the Kunpeng 920 processors. The HPRE accelerator is used to accelerate SSL/TLS applications. It significantly reduces processor consumption and improves processor efficiency. -On the Kunpeng server, you need to pass through the VFs of the HPRE accelerator on the host to the VM for internal services of the VM. - -**Table 1** HPRE accelerator description - -| | Description | -|-------------|-----------------------------------------------------------------------------------------------------| -| Device name | Hi1620 on-chip RSA/DH security algorithm accelerator (HPRE engine) | -| Description | Modular exponentiation, RSA key pair operation, DH calculation, and some large-number auxiliary operations (modular exponentiation, modular multiplication, modulo operation, multiplication, modular inversion, prime number test, and mutual prime test)| -| VendorID | 0x19E5 | -| PF DeviceID | 0xA258 | -| VF DeviceID | 0xA259 | -| Maximum number of VFs | An HPRE PF supports a maximum of 63 VFs. | - - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->When a VM is using a VF device, the driver on the host cannot be uninstalled, and the accelerator does not support hot swap. ->VF operation (If **VFNUMS** is **0**, the VF is disabled, and **hpre_num** is used to identify a specific accelerator device): ->``` ->echo $VFNUMS > /sys/class/uacce/hisi_hpre-$hpre_num/device/sriov_numvfs ->``` - - -## Managing VM USB - -To facilitate the use of USB devices such as USB key devices and USB mass storage devices on VMs, openEuler provides the USB device passthrough function. Through USB passthrough and hot-swappable interfaces, you can configure USB passthrough devices for VMs, or hot swap USB devices when VMs are running. - -### Configuring USB Controllers - -#### Overview - -A USB controller is a virtual controller that provides specific USB functions for USB devices on VMs. To use USB devices on a VM, you must configure USB controllers for the VM. Currently, openEuler supports the following types of USB controllers: - -- Universal host controller interface (UHCI): also called the USB 1.1 host controller specification. -- Enhanced host controller interface (EHCI): also called the USB 2.0 host controller specification. -- Extensible host controller interface (xHCI): also called the USB 3.0 host controller specification. - -#### Precautions - -- The host server must have USB controller hardware and modules that support USB 1.1, USB 2.0, and USB 3.0 specifications. -- You need to configure USB controllers for the VM by following the order of USB 1.1, USB 2.0, and USB 3.0. -- An xHCI controller has eight ports and can be mounted with a maximum of four USB 3.0 devices and four USB 2.0 devices. An EHCI controller has six ports and can be mounted with a maximum of six USB 2.0 devices. A UHCI controller has two ports and can be mounted with a maximum of two USB 1.1 devices. -- On each VM, only one USB controller of the same type can be configured. -- USB controllers cannot be hot swapped. -- If the USB 3.0 driver is not installed on a VM, the xHCI controller may not be identified. For details about how to download and install the USB 3.0 driver, refer to the official description provided by the corresponding OS distributor. -- To ensure the compatibility of the OS, set the bus ID of the USB controller to **0** when configuring a USB tablet for the VM. The tablet is mounted to the USB 1.1 controller by default. - -#### Configuration Methods - -The following describes the configuration items of USB controllers for a VM. You are advised to configure USB 1.1, USB 2.0, and USB 3.0 to ensure the VM is compatible with three types of devices. - -The configuration item of the USB 1.1 controller (UHCI) in the XML configuration file is as follows: - -``` - - -``` - -The configuration item of the USB 2.0 controller (EHCI) in the XML configuration file is as follows: - -``` - - -``` - -The configuration item of the USB 3.0 controller (xHCI) in the XML configuration file is as follows: - -``` - - -``` - -### Configuring a USB Passthrough Device - -#### Overview - -After USB controllers are configured for a VM, a physical USB device on the host can be mounted to the VM through device passthrough for the VM to use. In the virtualization scenario, in addition to static configuration, hot swapping the USB device is supported. That is, the USB device can be mounted or unmounted when the VM is running. - -#### Precautions - -- A USB device can be assigned to only one VM. -- A VM with a USB passthrough device does not support live migration. -- VM creation fails if no USB passthrough devices exist in the VM configuration file. -- Forcibly hot removing a USB storage device that is performing read or write operation may damage files in the USB storage device. - -#### Configuration Description - -The following describes the configuration items of a USB device for a VM. - -Description of the USB device in the XML configuration file: - -``` - - -
- -
- -``` - -- **
**: *m_ indicates the USB bus address on the host, and _n* indicates the device ID. -- **
**: indicates that the USB device is to be mounted to the USB controller specified on the VM. *x_ indicates the controller ID, which corresponds to the index number of the USB controller configured on the VM. _y* indicates the port address. When configuring a USB passthrough device, you need to set this parameter to ensure that the controller to which the device is mounted is as expected. - -#### Configuration Methods - -To configure USB passthrough, perform the following steps: - -1. Configure USB controllers for the VM. For details, see [Configuring USB Controllers](#configuring-usb-controllers). -2. Query information about the USB device on the host. - - Run the **lsusb** command (the **usbutils** software package needs to be installed) to query the USB device information on the host, including the bus address, device address, device vendor ID, device ID, and product description. For example: - - ``` - # lsusb - ``` - - ``` - Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub - Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub - Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub - Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub - Bus 006 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. - Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub - Bus 005 Device 003: ID 136b:0003 STEC - Bus 005 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. - Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub - Bus 001 Device 003: ID 12d1:0003 Huawei Technologies Co., Ltd. - Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. - Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub - Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub - ``` - -3. Prepare the XML description file of the USB device. Before hot removing the device, ensure that the USB device is not in use. Otherwise, data may be lost. -4. Run the hot swapping commands. - - Take a VM whose name is **openEulerVM** as an example. The corresponding configuration file is **usb.xml**. - - - Hot adding of the USB device takes effect only for the current running VM. After the VM is restarted, hot add the USB device again. - - ``` - # virsh attach-device openEulerVM usb.xml --live - ``` - - - Complete persistency configurations for hot adding of the USB device. After the VM is restarted, the USB device is automatically assigned to the VM. - - ``` - # virsh attach-device openEulerVM usb.xml --config - ``` - - - Hot removing of the USB device takes effect only for the current running VM. After the VM is restarted, the USB device with persistency configurations is automatically assigned to the VM. - - ``` - # virsh detach-device openEulerVM usb.xml --live - ``` - - - Complete persistency configurations for hot removing of the USB device. - - ``` - # virsh detach-device openEulerVM usb.xml --config - ``` - - - -## Storing Snapshots - -### Overview - -The VM system may be damaged due to virus damage, system file deletion by mistake, or incorrect formatting. As a result, the system cannot be started. To quickly restore a damaged system, openEuler provides the storage snapshot function. openEuler can create a snapshot that records the VM status at specific time points without informing users (usually within a few seconds). The snapshot can be used to restore the VM to the status when the snapshots were taken. For example, a damaged system can be quickly restored with the help of snapshots, which improves system reliability. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Currently, storage snapshots can be QCOW2 and RAW images only. Block devices are not supported. - -### Procedure - -To create VM storage snapshots, perform the following steps: - -1. Log in to the host and run the **virsh domblklist** command to query the disk used by the VM. - - ``` - # virsh domblklist openEulerVM - Target Source - --------------------------------------------- - vda /mnt/openEuler-image.qcow2 - ``` - - -1. Run the following command to create the VM disk snapshot **openEuler-snapshot1.qcow2**: - - ``` - # virsh snapshot-create-as --domain openEulerVM --disk-only --diskspec vda,snapshot=external,file=/mnt/openEuler-snapshot1.qcow2 --atomic - Domain snapshot 1582605802 created - ``` - - -1. Run the following command to query disk snapshots: - - ``` - # virsh snapshot-list openEulerVM - Name Creation Time State - --------------------------------------------------------- - 1582605802 2020-02-25 12:43:22 +0800 disk-snapshot - ``` - -## Configuring Disk I/O Suspension - -### Introduction - -#### Overview - -When a storage fault occurs (for example, the storage link is disconnected), the I/O error of the physical disk is sent to the VM front end through the virtualization layer. The VM receives the I/O error. As a result, the user file system in the VM may change to the read-only state. In this case, the VM needs to be restarted or the user needs to manually restore the file system, which brings extra workload to users. - -In this case, the virtualization platform provides the disk I/O suspension capability. That is, when the storage device is faulty, the VM I/Os are suspended when being delivered to the host. During the suspension period, no I/O error is returned to the VM. In this way, the file system in the VM does not change to the read-only state due to I/O errors. Instead, the file system is suspended. At the same time, the VM backend retries I/Os based on the specified suspension interval. If the storage fault is rectified within the suspension period, the suspended I/Os can be flushed to disks, and the internal file system of the VM automatically recovers without restarting the VM. If the storage fault is not rectified within the suspension period, an error is reported to the VM and the user is notified. - -#### Application Scenarios - -A cloud disk that may cause storage plane link disconnection is used as the backend of the virtual disk. - -#### Precautions and Restrictions - -- Only virtio-blk or virtio-scsi virtual disks support disk I/O suspension. - -- Generally, the backend of the virtual disk where the disk I/Os are suspended is the cloud disk whose storage plane link may be disconnected. - -- Disk I/O suspension can be enabled for read and write I/O errors separately. The retry interval and timeout interval for read and write I/O errors of the same disk are the same. - -- The disk I/O suspension retry interval does not include the actual read/write I/O overhead on the host. That is, the actual interval between two I/O retries is greater than the configured I/O error retry interval. - -- Disk I/O suspension cannot distinguish the specific type of I/O errors (such as storage link disconnection, bad sector, or reservation conflict). As long as the hardware returns an I/O error, suspension is performed. - -- When disk I/O suspension occurs, the internal I/Os of the VM are not returned, the system commands for accessing the disk, such as fdisk, are suspended, and the services that depend on the returned commands are suspended. - -- When disk I/O suspension occurs, the I/Os cannot be flushed to the disk. As a result, the VM may not be gracefully shut down. In this case, you need to forcibly shut down the VM. - -- When disk I/O suspension occurs, the disk data cannot be read. As a result, the VM cannot be restarted. In this case, you need to forcibly stop the VM, wait until the storage fault is rectified, and then restart the VM. - -- After a storage fault occurs, the following problems cannot be solved although disk I/O suspension occurs: - - 1. Failed to execute advanced storage features. - - Advanced features include: virtual disk hot swap, virtual disk creation, VM startup, VM shutdown, VM forcible shutdown, VM hibernation, VM wakeup, VM storage live migration, VM storage live migration cancellation, VM storage snapshot creation, VM storage snapshot combination, VM disk capacity query, online disk capacity expansion, virtual CD-ROM drive insertion, and CD-ROM drive ejection from the VM. - - 2. Failed to execute the VM lifecycle. - -- When a live migration is initiated for a VM configured with disk I/O suspension, the disk I/O suspension configuration must be the same as that of the source host in the XML configuration of the destination disk. - -### Disk I/O Suspension Configuration - -#### Using the QEMU CLI - -To enable disk I/O suspension, configure `werror=retry` `rerror=retry` on the virtual disk device. To configure the retry policy, configure `retry_interval` and `retry_timeout`. `retry_interval` indicates the I/O retry interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. If this parameter is not set, the default value 1000 ms is used. `retry_timeout` indicates the I/O retry timeout interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. The value **0** indicates that no timeout occurs. If this parameter is not set, the default value **0** is used. - -The disk I/O suspension configuration of the virtio-blk disk is as follows: - -```shell --drive file=/path/to/your/storage,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native \ --device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,\ -drive=drive-virtio-disk0,id=virtio-disk0,write-cache=on,\ -werror=retry,rerror=retry,retry_interval=2000,retry_timeout=10000 -``` - -The disk I/O suspension configuration of the virtio-scsi disk is as follows: - -```shell --drive file=/path/to/your/storage,format=raw,if=none,id=drive-scsi0-0-0-0,cache=none,aio=native \ --device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ -device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,write-cache=on,\ -werror=retry,rerror=retry,retry_interval=2000,retry_timeout=10000 -``` - -#### Using an XML Configuration File - -To enable disk I/O suspension, configure `error_policy='retry'` `rerror_policy='retry'` in the disk XML configuration. Configure `retry_interval` and `retry_timeout`. retry_interval indicates the I/O retry interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. If this parameter is not set, the default value 1000 ms is used. retry_timeout indicates the I/O retry timeout interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. The value **0** indicates that no timeout occurs. If this parameter is not set, the default value **0** is used. - -The disk I/O suspension XML configuration of the virtio-blk disk is as follows: - -```xml - - - - - - -``` - -The disk I/O suspension XML configuration of the virtio-scsi disk is as follows: - -```xml - - - - - -
- -``` diff --git a/docs/en/docs/Virtualization/managing-vms.md b/docs/en/docs/Virtualization/managing-vms.md deleted file mode 100644 index 5e8fca489006ec121c3c42b1b8e4529813b304f7..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/managing-vms.md +++ /dev/null @@ -1,801 +0,0 @@ -# Managing VMs - -- [Managing VMs](#managing-vms) - - [VM Life Cycle](#vm-life-cycle) - - [Introduction](#introduction) - - [Management Commands](#management-commands) - - [Example](#example) - - [Modifying VM Configurations Online](#modifying-vm-configurations-online) - - [Querying VM Information](#querying-vm-information) - - [Logging In to a VM](#logging-in-to-a-vm) - - [Logging In Using VNC Passwords](#logging-in-using-vnc-passwords) - - [Configuring VNC TLS Login](#configuring-vnc-tls-login) - - [VM Secure Boot](#vm-secure-boot) - - [General Introduction](#general-introduction) - - [Secure Boot Practice](#secure-boot-practice) - -## VM Life Cycle - -### Introduction - -#### Overview - -To leverage hardware resources and reduce costs, users need to properly manage VMs. This section describes basic operations during the VM lifecycle, such as creating, using, and deleting VMs. - -#### VM Status - -A VM can be in one of the following status: - -- **undefined**: The VM is not defined or created. That is, libvirt considers that the VM does not exist. -- **shut off**: The VM has been defined but is not running, or the VM is terminated. -- **running**: The VM is running. -- **paused**: The VM is suspended and its running status is temporarily stored in the memory. The VM can be restored to the running status. -- **saved**: Similar to the **paused** status, the running state is stored in a persistent storage medium and can be restored to the running status. -- **crashed**: The VM crashes due to an internal error and cannot be restored to the running status. - -#### Status Transition - -VMs in different status can be converted, but certain rules must be met. [Figure 1](#fig671014583483) describes the common rules for transiting the VM status. - -**Figure 1** Status transition diagram -![](./figures/status-transition-diagram.png "status-transition-diagram") - -#### VM ID - -In libvirt, a created VM instance is called a **domain**, which describes the configuration information of resources such as the CPU, memory, network device, and storage device of the VM. On a host, each domain has a unique ID, which is represented by the VM **Name**, **UUID**, and **Id**. For details, see [Table 1](#table84397266483). During the VM lifecycle, an operation can be performed on a specific VM by using a VM ID. - -**Table 1** Domain ID description - - - - - - - - - - - - - - - -

ID

-

Description

-

Name

-

VM name

-

UUID

-

Universally unique identifier

-

Id

-

VM running ID

-
NOTE:

The ID is not displayed for a powered off VM.

-
-
- ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->Run the **virsh** command to query the VM ID and UUID. For details, see [Querying VM Information](#querying-vm-information). - -### Management Commands - -#### Overview - -You can use the **virsh** command tool to manage the VM lifecycle. This section describes the commands related to the lifecycle. - -#### Prerequisites - -- Before performing operations on a VM, you need to query the VM status to ensure that the operations can be performed. For details about the conversion between status, see [Status Transition](#status-transition). -- You have administrator rights. -- The VM XML configuration files are prepared. - -#### Command Usage - -You can run the **virsh** command to manage the VM lifecycle. The command format is as follows: - -```shell -virsh -``` - -The parameters are described as follows: - -- _operate_: manages VM lifecycle operations, such as creating, deleting, and starting VMs. -- _obj_: specifies the operation object, for example, the VM to be operated. -- _options_: command option. This parameter is optional. - -[Table 2](#table389518422611) describes the commands used for VM lifecycle management. _VMInstance_ indicates the VM name, VM ID, or VM UUID, _XMLFile_ indicates the XML configuration file of the VM, and _DumpFile_ indicates the dump file. Change them based on the site requirements. - -**Table 2** VM Lifecycle Management Commands - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Command

-

Description

-

virsh define <XMLFile>

-

Define a persistent VM. After the definition is complete, the VM is shut down and is considered as a domain instance.

-

virsh create <XMLFile>

-

Create a temporary VM. After the VM is created, it is in the running status.

-

virsh start <VMInstance>

-

Start the VM.

-

virsh shutdown <VMInstance>

-

Shut down the VM. Start the VM shutdown process. If the VM fails to be shut down, forcibly stop it.

-

virsh destroy <VMInstance>

-

Forcibly stop the VM.

-

virsh reboot <VMInstance>

-

Reboot the VM.

-

virsh save <VMInstance> <DumpFile>

-

Dump the VM running status to a file.

-

virsh restore <DumpFile>

-

Restore the VM from the VM status dump file.

-

virsh suspend <VMInstance>

-

Suspend the VM to make the VM in the paused status.

-

virsh resume <VMInstance>

-

Resume the VM and restore the VM in the paused status to the running status.

-

virsh undefine <VMInstance>

-

After a persistent VM is destroyed, the VM lifecycle ends and no more operations can be performed on the VM.

-
- -### Example - -This section provides examples of commands related to VM life cycle management. - -- Create a VM. - - The VM XML configuration file is **openEulerVM.xml**. The command and output are as follows: - - ```shell - $ virsh define openEulerVM.xml - Domain openEulerVM defined from openEulerVM.xml - ``` - -- Start a VM. - - Run the following command to start the _openEulerVM_: - - ```shell - $ virsh start openEulerVM - Domain openEulerVM started - ``` - -- Reboot a VM. - - Run the following command to reboot the _openEulerVM_: - - ```shell - $ virsh reboot openEulerVM - Domain openEulerVM is being rebooted - ``` - -- Shut down a VM. - - Run the following command to shut down the _openEulerVM_: - - ```shell - $ virsh shutdown openEulerVM - Domain openEulerVM is being shutdown - ``` - -- Destroy a VM. - - If the **nvram** file is not used during the VM startup, run the following command to destroy the VM: - - ```shell - virsh undefine - ``` - - - If the **nvram** file is used during the VM startup, run the following command to specify the **nvram** processing policy when destroying the VM: - - ```shell - virsh undefine - ``` - - _strategy_ indicates the policy for destroying a VM. The values can be: - - --**nvram**: delete the corresponding **nvram** file when destroying a VM. - - --**keep-nvram**: destroy a VM but retain the corresponding **nvram** file. - - For example, to delete the _openEulerVM_ and its **nvram** file, run the following command: - - ```shell - $ virsh undefine openEulerVM --nvram - Domain openEulerVM has been undefined - ``` - -## Modifying VM Configurations Online - -### Overview - -After a VM is created, users can modify VM configurations. This process is called online modification of VM configuration. After the configuration is modified online, the new VM configuration file is persistent and takes effect after the VM is shut down and restarted. - -The format of the command for modifying VM configuration is as follows: - -```shell -virsh edit -``` - -The **virsh edit** command is used to edit the XML configuration file corresponding to **domain** to update VM configuration. **virsh edit** uses the **vi** program as the default editor. You can specify the editor type by modifying the environment variable _EDITOR_ or _VISUAL_. By default, **virsh edit** preferentially uses the text editor specified by the environment variable _VISUAL_. - -### Procedure - -1. \(Optional\) Set the editor of the **virsh edit** command to **vim**. - - ```shell - export VISUAL=vim - ``` - -2. Run the **virsh edit** command to open the XML configuration file of the _openEulerVM_. - - ```shell - virsh edit openEulerVM - ``` - -3. Modify the VM configuration file. -4. Save the VM configuration file and exit. -5. Shut down the VM. - - ```shell - virsh shutdown openEulerVM - ``` - -6. Start the VM for the modification to take effect. - - ```shell - virsh start openEulerVM - ``` - -## Querying VM Information - -### Overview - -The libvirt provides a set of command line tools to query VM information. This section describes how to use commands to obtain VM information. - -### Prerequisites - -To query VM information, the following requirements must be met: - -- The libvirtd service is running. - -- Only the administrator has the permission to execute command line. - -### Querying VM Information on a Host - -- Query the list of running and paused VMs on a host. - - ```shell - virsh list - ``` - - For example, the following command output indicates that three VMs exist on the host. **openEulerVM01** and **openEulerVM02** are running, and **openEulerVM03** is paused. - - ```text - Id Name State - ---------------------------------------------------- - 39 openEulerVM01 running - 40 openEulerVM02 running - 69 openEulerVM03 paused - ``` - -- Query the list of VM information defined on a host. - - ```shell - virsh list --all - ``` - - For example, the following command output indicates that four VMs are defined on the current host. **openEulerVM01** is running, **openEulerVM02** is paused, and **openEulerVM03** and **openEulerVM04** are shut down. - - ```text - Id Name State - ---------------------------------------------------- - 39 openEulerVM01 running - 69 openEulerVM02 paused - - openEulerVM03 shut off - - openEulerVM04 shut off - ``` - -### Querying Basic VM Information - -Libvirt component provides a group of commands for querying the VM status, including the VM running status, device information, and scheduling attributes. For details, see [Table 3](#table10582103963816). - -**Table 3** Querying basic VM information - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Information to be queried

-

Command line

-

Description

-

Basic information

-

virsh dominfo <VMInstance>

-

The information includes the VM ID, UUID, and VM specifications.

-

Current status

-

virsh domstate <VMInstance>

-

You can use the --reason option to query the reason why the VM changes to the current status.

-

Scheduling information

-

virsh schedinfo <VMInstance>

-

The information includes the vCPU share.

-

Number of vCPUs

-

virsh vcpucount <VMInstance>

-

Number of vCPUs of the VM.

-

Virtual block device status

-

virsh domblkstat <VMInstance>

-

To query the name of a block device, run the virsh domblklist command.

-

vNIC status

-

virsh domifstat <VMInstance>

-

To query the NIC name, run the virsh domiflist command.

-

I/O thread

-

virsh iothreadinfo <VMInstance>

-

VM I/O thread and CPU affinity.

-
- -### Example - -- Run the **virsh dominfo** command to query the basic information about a created VM. The query result shows that the VM ID is **5**, UUID is **ab472210-db8c-4018-9b3e-fc5319a769f7**, memory size is 8 GiB, and the number of vCPUs is 4. - - ```shell - $ virsh dominfo openEulerVM - Id: 5 - Name: openEulerVM - UUID: ab472210-db8c-4018-9b3e-fc5319a769f7 - OS Type: hvm - State: running - CPU(s): 4 - CPU time: 6.8s - Max memory: 8388608 KiB - Used memory: 8388608 KiB - Persistent: no - Autostart: disable - Managed save: no - Security model: none - Security DOI: 0 - ``` - -- Run the **virsh domstate** command to query the VM status. The query result shows that VM **openEulerVM** is running. - - ```shell - $ virsh domstate openEulerVM - running - ``` - -- Run **virsh schedinfo** to query the VM scheduling information. The query result shows that the CPU reservation share of the VM is 1024. - - ```shell - $ virsh schedinfo openEulerVM - Scheduler : posix - cpu_shares : 1024 - vcpu_period : 100000 - vcpu_quota : -1 - emulator_period: 100000 - emulator_quota : -1 - global_period : 100000 - global_quota : -1 - iothread_period: 100000 - iothread_quota : -1 - ``` - -- Run the **virsh vcpucount** command to query the number of vCPUs. The query result shows that the VM has four CPUs. - - ```shell - $ virsh vcpucount openEulerVM - maximum live 4 - current live 4 - ``` - -- Run the **virsh domblklist** command to query the VM disk information. The query result shows that the VM has two disks. sda is a virtual disk in qcow2 format, and sdb is a cdrom device. - - ```shell - $ virsh domblklist openEulerVM - Target Source - --------------------------------------------------------------------- - sda /home/openeuler/vm/openEuler_aarch64.qcow2 - sdb /home/openeuler/vm/openEuler-23.09-aarch64-dvd.iso - ``` - -- Run the **virsh domiflist** command to query the VM NIC information. The query result shows that the VM has one NIC, the backend is vnet0, which is on the br0 bridge of the host. The MAC address is 00:05:fe:d4:f1:cc. - - ```shell - $ virsh domiflist openEulerVM - Interface Type Source Model MAC - ------------------------------------------------------- - vnet0 bridge br0 virtio 00:05:fe:d4:f1:cc - ``` - -- Run the **virsh iothreadinfo** command to query the VM I/O thread information. The query result shows that the VM has five I/O threads, which are scheduled on physical CPUs 7-10. - - ```shell - $ virsh iothreadinfo openEulerVM - IOThread ID CPU Affinity - --------------------------------------------------- - 3 7-10 - 4 7-10 - 5 7-10 - 1 7-10 - 2 7-10 - ``` - -## Logging In to a VM - -This section describes how to log in to a VM using VNC. - -### Logging In Using VNC Passwords - -#### Overview - -After the OS is installed on a VM, you can remotely log in to the VM using VNC to manage the VM. - -#### Prerequisites - -Before logging in to a VM using a client, such as RealVNC or TightVNC, ensure that: - -- You have obtained the IP address of the host where the VM resides. -- The environment where the client resides can access the network of the host. -- You have obtained the VNC listening port of the VM. This port is automatically allocated when the client is started. Generally, the port number is **5900 + x** \(_x_ is a positive integer and increases in ascending order based on the VM startup sequence. **5900** is invisible to users.\) -- If a password has been set for the VNC, you also need to obtain the VNC password of the VM. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >To set a password for the VM VNC, edit the XML configuration file of the VM. That is, add the **passwd** attribute to the **graphics** element and set the attribute value to the password to be configured. For example, to set the VNC password of the VM to **n8VfjbFK**, configure the XML file as follows: -> - >``` - > - > - > - >``` - -#### Procedure - -#### Procedure - -1. Query the VNC port number used by the VM. For example, if the VM name is _openEulerVM_, run the following command: - - ```shell - $ virsh vncdisplay openEulerVM - :3 - ``` - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >To log in to the VNC, you need to configure firewall rules to allow the connection of the VNC port. The reference command is as follows, where _X_ is **5900 + Port number**, for example, **5903**. - > - >```shell - >firewall-cmd --zone=public --add-port=X/tcp - >``` - -2. Start the VncViewer software and enter the IP address and port number of the host. The format is **host IP address:port number**, for example, **10.133.205.53:3**. -3. Click **OK** and enter the VNC password \(optional\) to log in to the VM VNC. - -### Configuring VNC TLS Login - -#### Overview - -By default, the VNC server and client transmit data in plaintext. Therefore, the communication content may be intercepted by a third party. To improve security, openEuler allows the VNC server to configure the Transport Layer Security \(TLS\) mode for encryption and authentication. TLS implements encrypted communication between the VNC server and client to prevent communication content from being intercepted by third parties. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- To use the TLS encryption authentication mode, the VNC client must support the TLS mode \(for example, TigerVNC\). Otherwise, the VNC client cannot be connected. ->- The TLS encryption authentication mode is configured at the host level. After this feature is enabled, the TLS encryption authentication mode is enabled for the VNC clients of all VMs running on the host. - -#### Procedure - -To enable the TLS encryption authentication mode for the VNC, perform the following steps: - -1. Log in to the host where the VNC server resides, and edit the corresponding configuration items in the **/etc/libvirt/qemu.conf** configuration file of the server. The configuration is as follows: - - ```conf - vnc_listen = "x.x.x.x" # "x.x.x.x" indicates the listening IP address of the VNC. Set this parameter based on the site requirements. The VNC server allows only the connection requests from clients whose IP addresses are in this range. - vnc_tls = 1 # If this parameter is set to 1, VNC TLS is enabled. - vnc_tls_x509_cert_dir = "/etc/pki/libvirt-vnc" # Specify /etc/pki/libvirt-vnc as the path for storing the certificate. - vnc_tls_x509_verify = 1 #If this parameter is set to 1, the X509 certificate is used for TLS authentication. - ``` - -2. Create a certificate and a private key file for the VNC. The following uses GNU TLS as an example. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >To use GNU TLS, install the gnu-utils software package in advance. - - 1. Create a certificate file issued by the Certificate Authority \(CA\). - - ```shell - certtool --generate-privkey > ca-key.pem - ``` - - 2. Create a self-signed public and private key for the CA certificate. _Your organization name_ indicates the organization name, which is specified by the user. - - ```shell - $ cat > ca.info< server.info< server-key.pem - certtool --generate-certificate \ - --load-ca-certificate ca-cert.pem \ - --load-ca-privkey ca-key.pem \ - --load-privkey server-key.pem \ - --template server.info \ - --outfile server-cert.pem - ``` - - In the preceding generated file, **server-key.pem** is the private key of the VNC server, and **server-cert.pem** is the public key of the VNC server. - - 4. Issue a certificate to the VNC client. - - ```shell - $ cat > client.info< client-key.pem - certtool --generate-certificate \ - --load-ca-certificate ca-cert.pem \ - --load-ca-privkey ca-key.pem \ - --load-privkey client-key.pem \ - --template client.info \ - --outfile client-cert.pem - ``` - - In the preceding generated file, **client-key.pem** is the private key of the VNC client, and **client-cert.pem** is the public key of the VNC client. The generated public and private key pairs need to be copied to the VNC client. - -3. Shut down the VM to be logged in to and restart the libvirtd service on the host where the VNC server resides. - - ```shell - systemctl restart libvirtd - ``` - -4. Save the generated server certificate to the specified directory on the VNC server and grant the read and write permissions on the certificate only to the current user. - - ```shell - sudo mkdir -m 750 /etc/pki/libvirt-vnc - cp ca-cert.pem /etc/pki/libvirt-vnc/ca-cert.pem - cp server-cert.pem /etc/pki/libvirt-vnc/server-cert.pem - cp server-key.pem /etc/pki/libvirt-vnc/server-key.pem - chmod 0600 /etc/pki/libvirt-vnc/* - ``` - -5. Copy the generated client certificates **ca-cert.pem**, **client-cert.pem**, and **client-key.pem** to the VNC client. After the TLS certificate of the VNC client is configured, you can use VNC TLS to log in to the VM. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - >- For details about how to configure the VNC client certificate, see the usage description of each client. - >- For details about how to log in to the VM, see [Logging In Using VNC Passwords](#logging-in-using-vnc-passwords). - -## VM Secure Boot - -### General Introduction - -#### Overview - -Secure boot uses public and private key pairs to sign and validate boot components. During the startup, the previous component validates the digital signature of the next component. If the validation is successful, the next component starts. If the validation fails, the startup fails. Secure boot is used to detect whether the firmware and software during startup of the device are tampered with to prevent malware from intrusion and modification. Secure boot ensures the integrity of each component during system startup and prevents unauthorized components from being loaded and running, thereby preventing security threats to the system and user data. Secure boot is implemented based on the UEFI boot mode. It is not supported by the legacy boot mode. According to UEFI specifications, some reliable public keys can be built in the mainboard before delivery. Any operating system or hardware drivers that you want to load on this mainboard must be authenticated by these public keys. The secure boot of a physical machine is implemented by the physical BIOS, while the secure boot of a VM is simulated by software. The process of the VM secure boot is the same as that of the host secure boot, both complying with the open-source UEFI specifications. The UEFI on the virtualization platform is provided by the edk component. When a VM starts, QEMU maps the UEFI image to the memory to simulate the firmware startup process for the VM. Secure boot is a security protection capability provided by edk during the VM startup to protect the OS kernel of the VM from being tampered with. The sequence of signature validation for the secure boot is as follows: UEFI BIOS->shim->GRUB->vmlinuz (signature validation is passed and loaded in sequence). - -| English | Acronyms and Abbreviations | Description | -| :----- | :----- | :----- | -| Secure boot | - | Secure boot indicates that a component validates the digital signature of the next component during startup. If the validation is successful, the component runs. If the validation fails, the component stops running. It ensures the integrity of each component during system startup. | -| Platform key | PK | Platform key is owned by the OEM vendor and must be RSA2048 or stronger. The PK establishes a trusted relationship between the platform owner and the platform firmware. The platform owner registers the PKpub, public key of the PK, with the platform firmware. The platform owner can use the PKpriv, private part of the PK, to change the ownership of the platform or register the KEK key. | -| Key exchange key | KEK | Key exchange key creates a trusted relationship between the platform firmware and the OS. Each OS and third-party application that communicates with the platform firmware register the KEKpub, public part of the KEK key, in the platform firmware. | -| Database trustlist | DB | Database trustlist stores and validates the keys of components such as shim, GRUB, and vmlinuz. | -| Database blocklist | DBx | Database blocklist stores revoked keys. | - -#### Function Description - -The VM secure boot feature is implemented based on the edk open-source project. In non-secure boot mode, the basic Linux process is as follows: - -**Figure 2** System startup process - -![](./figures/OSBootFlow.png) - -In secure boot mode, the first component loaded after UEFI BIOS starts is shim in the system image. By interacting with UEFI BIOS, shim obtains the key stored in the variable DB of UEFI BIOS to validate GRUB. After GRUB is loaded, the key and the authentication API are also called to validate the kernel. The Linux boot process is as follows: - -**Figure 3** Secure boot process - -![](./figures/SecureBootFlow.png) - -The secure boot feature involves multiple key scenarios. Based on the scenario analysis and system breakdown, the secure boot feature involves the following subsystems: UEFI BIOS validating shim, shim validating GRUB, and GRUB validating kernel. When UEFI BIOS validates shim, if the validation is successful, shim is started. If the validation fails, an error message is displayed and shim fails to start. Shim needs to use the private key for signature during image compilation and creation, and the public key certificate needs to be imported to the variable area DB of UEFI BIOS. After shim is started, validate the startup of GRUB. If the validation is successful, GRUB is started. If the validation fails, an error message is displayed and GRUB fails to start. GRUB needs to be signed during image compilation and creation. The public and private key pairs are the same as those of shim. After GRUB is started, it calls the key and the authentication API key registered in UEFI BIOS to validate the kernel. If the validation is successful, GRUB starts the kernel. If the validation fails, an error message is displayed. GRUB needs to sign the image during compilation and creation and uses the public and private key pair that is the same as that of shim. - -#### Constraints - -- Running on the UEFI BIOS that does not support secure boot does not affect existing functions and services. -- The secure boot feature depends on the UEFI BIOS and takes effect only when the UEFI supports this feature. -- When secure boot is enabled in the UEFI BIOS, the system cannot be started if the related components have no signature or the signature is incorrect. -- If secure boot is disabled in the UEFI BIOS, the validation function during the boot process is disabled. -- The second half of the secure boot validation chain, that is, shim->GRUB->kernel, guides the kernel to start. This part of the validation chain is implemented by the OS image. If the OS does not support guiding the kernel for secure boot, the VM secure boot fails. -- Currently, the x86 architecture do not provide nvram file configuration to configure VM secure boot. - -### Secure Boot Practice - -VM secure boot depends on UEFI BIOS. The UEFI BIOS image is installed using the edk rpm package. This section uses AArch64 as an example to describe how to configure VM secure boot. - -#### Configuring VM - -The components in the edk rpm package are installed in the /usr/share/edk2/aarch64 directory, including `QEMU_EFI-pflash.raw` and `vars-template-pflash.raw`. The following describes the XML configuration of the UEFI BIOS during VM startup. - -```xml - - hvm - /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw - /path/to/QEMU-VARS.fd - -``` - -In the preceding configuration, /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw indicates the path of the UEFI BIOS image. /usr/share/edk2/aarch64/vars-template-pflash.raw is the path of the NVRAM image template, and /path/to/QEMU-VARS.fd is the path of the NVRAM image file of the current VM, which is used to store environment variables in the UEFI BIOS. - -#### Importing Certificate - -The certificate for VM secure boot is imported from the BIOS page. Before importing the certificate, you need to import the certificate file to the VM. You can mount the directory where the certificate file is located to the VM by mounting a disk. For example, you can create an image that contains the certificate and mount the image in the XML configuration file of the VM. - -Create a certificate file image. - -```shell -dd of='/path/to/data.img' if='/dev/zero' bs=1M count=64 -mkfs.vfat -I /path/to/data.img -mkdir /path/to/mnt -mount path/to/data.img /path/to/mnt/ -cp -a /path/to/certificates/* /path/to/mnt/ -umount /path/to/mnt/ -``` - -In the preceding command, /path/to/certificates/ indicates the path where the certificate file is located, /path/to/data.img indicates the path where the certificate file image is located, and /path/to/mnt/ indicates the image mounting path. - -Mount the image in the XML file of the VM. - -```xml - - - - - - - - - -``` - -Start the VM and import the PK certificate. The procedure is as follows (the procedure for importing the KEK certificate is the same as that for importing the DB certificate): - -After the VM is started, press F2 to go to the BIOS screen. - -**Figure 4** BIOS screen - -![](./figures/CertEnrollP1.png) - -**Figure 5** Device Manager - -![](./figures/CertEnrollP2.png) - -**Figure 6** Custom Secure Boot Options - -![](./figures/CertEnrollP3.png) - -**Figure 7** PK Options - -![](./figures/CertEnrollP4.png) - -**Figure 8** Enrolling PK - -![](./figures/CertEnrollP5.png) - -In the File Explorer window, many disk directories are displayed, including the certificate file directory mounted through the disk. - -**Figure 9** File Explorer - -![](./figures/CertEnrollP6.png) - -Select the PK certificate to be imported in the disk directory. - -**Figure 10** Disk where the certificate is stored - -![](./figures/CertEnrollP7.png) - -**Figure 11** Selecting **Commit Changes and Exit** to save the imported certificate - -![](./figures/CertEnrollP8.png) - -After the certificate is imported, the UEFI BIOS writes the certificate information and secure boot attributes to the NVRAM configuration file /path/to/QEMU-VARS.fd. Upon the next startup, the VM reads related configurations from the /path/to/QEMU-VARS.fd file, initializes certificate information and secure boot attributes, automatically imports the certificate, and enables secure boot. Similarly, you can use /path/to/QEMU-VARS.fd as the UEFI BIOS boot configuration template file of other VMs with the same configuration. Modify the nvram template field so that the certificate is automatically imported and the secure boot option is enabled when other VMs are started. The VM XML configuration is modified as follows: - -```xml - - hvm - /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw - - -``` - -#### Secure Boot Observation - -After the VM is correctly configured and the PK, KEK, and DB certificates are imported, the VM runs in secure boot mode. You can configure the serial port log file in the VM configuration file in XML format to check whether the VM is in the secure boot mode. The following figure shows how to configure the serial port log file. - -```xml - - - -``` - -After the OS image is successfully loaded to the VM, if "UEFI Secure Boot is enabled" is displayed in the serial port log file, the VM is in the secure boot state. diff --git a/docs/en/docs/Virtualization/system-resource-management.md b/docs/en/docs/Virtualization/system-resource-management.md deleted file mode 100644 index 18030aa275e58f1c0e6650f54c39538bdf45d957..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/system-resource-management.md +++ /dev/null @@ -1,480 +0,0 @@ -# system Resource Management - -The `libvirt` command manages VM system resources, such as vCPU and virtual memory resources. - -Before you start: - -- Ensure that the libvirtd daemon is running on the host. -- Run the `virsh list --all` command to check that the VM has been defined. - -## Managing vCPU - -### CPU Shares - -#### Overview - -In a virtualization environment, multiple VMs on the same host compete for physical CPUs. To prevent some VMs from occupying too many physical CPU resources and affecting the performance of other VMs on the same host, you need to balance the vCPU scheduling of VMs to prevent excessive competition for physical CPUs. - -The CPU share indicates the total capability of a VM to compete for physical CPU computing resources. You can set **cpu\_shares** to specify the VM capacity to preempt physical CPU resources. The value of **cpu\_shares** is a relative value without a unit. The CPU computing resources obtained by a VM are the available computing resources of physical CPUs \(excluding reserved CPUs\) allocated to VMs based on the CPU shares. Adjust the CPU shares to ensure the service quality of VM CPU computing resources. - -#### Procedure - -Change the value of **cpu\_shares** allocated to the VM to balance the scheduling between vCPUs. - -- Check the current CPU share of the VM. - - ```shell - $ virsh schedinfo - Scheduler : posix - cpu_shares : 1024 - vcpu_period : 100000 - vcpu_quota : -1 - emulator_period: 100000 - emulator_quota : -1 - global_period : 100000 - global_quota : -1 - iothread_period: 100000 - iothread_quota : -1 - ``` - -- Online modification: Run the `virsh schedinfo` command with the `--live` parameter to modify the CPU share of a running VM. - - ```shell - virsh schedinfo --live cpu_shares= - ``` - - For example, to change the CPU share of the running _openEulerVM_ from **1024** to **2048**, run the following commands: - - ```shell - $ virsh schedinfo openEulerVM --live cpu_shares=2048 - Scheduler : posix - cpu_shares : 2048 - vcpu_period : 100000 - vcpu_quota : -1 - emulator_period: 100000 - emulator_quota : -1 - global_period : 100000 - global_quota : -1 - iothread_period: 100000 - iothread_quota : -1 - ``` - - The modification of the **cpu\_shares** value takes effect immediately. The running time of the _openEulerVM_ is twice the original running time. However, the modification will become invalid after the VM is shut down and restarted. - -- Permanent modification: Run the `virsh schedinfo` command with the `--config` parameter to change the CPU share of the VM in the libvirt internal configuration. - - ```shell - virsh schedinfo --config cpu_shares= - ``` - - For example, run the following command to change the CPU share of _openEulerVM_ from **1024** to **2048**: - - ```shell - $ virsh schedinfo openEulerVM --config cpu_shares=2048 - Scheduler : posix - cpu_shares : 2048 - vcpu_period : 0 - vcpu_quota : 0 - emulator_period: 0 - emulator_quota : 0 - global_period : 0 - global_quota : 0 - iothread_period: 0 - iothread_quota : 0 - ``` - - The modification on **cpu\_shares** does not take effect immediately. Instead, the modification takes effect after the _openEulerVM_ is started next time and takes effect permanently. The running time of the _openEulerVM_ is twice that of the original VM. - -### Binding the QEMU Process to a Physical CPU - -#### Overview - -You can bind the QEMU main process to a specific physical CPU range, ensuring that VMs running different services do not interfere with adjacent VMs. For example, in a typical cloud computing scenario, multiple VMs run on one physical machine, and they carry diversified services, causing different degrees of resource occupation. To avoid interference of a VM with dense-storage I/O to an adjacent VM, storage processes that process I/O of different VMs need to be completely isolated. The QEMU main process handles frontend and backend services. Therefore, isolation needs to be implemented. - -#### Procedure - -Run the `virsh emulatorpin` command to bind the QEMU main process to a physical CPU. - -- Check the range of the physical CPU bound to the QEMU process: - - ```shell - $ virsh emulatorpin openEulerVM - emulator: CPU Affinity - ---------------------------------- - *: 0-63 - ``` - - This indicates that the QEMU main process corresponding to VM **openEulerVM** can be scheduled on all physical CPUs of the host. - -- Online binding: Run the `virsh emulatorpin` command with the `--live` parameter to modify the binding relationship between the QEMU process and the running VM. - - ```shell - $ virsh emulatorpin openEulerVM --live 2-3 - - $ virsh emulatorpin openEulerVM - emulator: CPU Affinity - ---------------------------------- - *: 2-3 - ``` - - The preceding commands bind the QEMU process corresponding to VM **openEulerVM** to physical CPUs **2** and **3**. That is, the QEMU process is scheduled only on the two physical CPUs. The binding relationship takes effect immediately but becomes invalid after the VM is shut down and restarted. - -- Permanent binding: Run the `virsh emulatorpin` command with the `--config` parameter to modify the binding relationship between the VM and the QEMU process in the libvirt internal configuration. - - ```shell - $ virsh emulatorpin openEulerVM --config 0-3,^1 - - $ virsh emulatorpin euler - emulator: CPU Affinity - ---------------------------------- - *: 0,2-3 - ``` - - The preceding commands bind the QEMU process corresponding to VM **openEulerVM** to physical CPUs **0**, **2** and **3**. That is, the QEMU process is scheduled only on the three physical CPUs. The modification of the binding relationship does not take effect immediately. Instead, the modification takes effect after the next startup of the VM and takes effect permanently. - -### Adjusting the vCPU Binding Relationship - -#### Overview - -The vCPU of a VM is bound to a physical CPU. That is, the vCPU is scheduled only on the bound physical CPU to improve VM performance in specific scenarios. For example, in a NUMA system, vCPUs are bound to the same NUMA node to prevent cross-node memory access and VM performance deterioration. If the vCPU is not bound, by default, the vCPU can be scheduled on any physical CPU. The specific binding policy is determined by the user. - -#### Procedure - -Run the `virsh vcpupin` command to adjust the binding relationship between vCPUs and physical CPUs. - -- View the vCPU binding information of the VM. - - ```shell - $ virsh vcpupin openEulerVM - VCPU CPU Affinity - ---------------------- - 0 0-63 - 1 0-63 - 2 0-63 - 3 0-63 - ``` - - This indicates that all vCPUs of VM **openEulerVM** can be scheduled on all physical CPUs of the host. - -- Online adjustment: Run the `vcpu vcpupin` command with the `--live` parameter to modify the vCPU binding relationship of a running VM. - - ```shell - $ virsh vcpupin openEulerVM --live 0 2-3 - - $ virsh vcpupin euler - VCPU CPU Affinity - ---------------------- - 0 2-3 - 1 0-63 - 2 0-63 - 3 0-63 - ``` - - The preceding commands bind vCPU **0** of VM **openEulerVM** to pCPU **2** and pCPU **3**. That is, vCPU **0** is scheduled only on the two physical CPUs. The binding relationship takes effect immediately but becomes invalid after the VM is shut down and restarted. - -- Permanent adjustment: Run the `virsh vcpupin` command with the `--config` parameter to modify the vCPU binding relationship of the VM in the libvirt internal configuration. - - ```shell - $ virsh vcpupin openEulerVM --config 0 0-3,^1 - - $ virsh vcpupin openEulerVM - VCPU CPU Affinity - ---------------------- - 0 0,2-3 - 1 0-63 - 2 0-63 - 3 0-63 - ``` - - The preceding commands bind vCPU **0** of VM **openEulerVM** to physical CPUs **0**, **2**, and **3**. That is, vCPU **0** is scheduled only on the three physical CPUs. The modification of the binding relationship does not take effect immediately. Instead, the modification takes effect after the next startup of the VM and takes effect permanently. - -### CPU Hotplug - -#### Overview - -CPU hotplug allows you to increase or decrease the number of CPUs for a running VM without affecting services on it. When the internal service pressure rises to a level where existing CPUs become saturated, CPU hotplug can dynamically boost the computing power of a VM, guaranteeing stable service throughput. CPU hotplug also enables the removal of unused computing resources during low service load, minimizing computing costs. - -Note: CPU hotplug is added for the AArch64 architecture in openEuler 24.03 LTS. However, the new implementation of the mainline community is not compatible with that of earlier openEuler versions. Therefore, the guest OS must match the host OS. That is, the guest and host machines must both run openEuler 24.03 LTS or later versions, or versions earlier than openEuler 24.03 LTS. - -#### Constraints - -- For processors using the AArch64 architecture, the specified VM chipset type \(machine\) needs to be virt-4.1 or a later version when a VM is created. For processors using the x86\_64 architecture, the specified VM chipset type \(machine\) needs to be pc-i440fx-1.5 or a later version when a VM is created. -- The initial CPU of an AArch64 VM cannot be hot removed. -- When configuring Guest NUMA, you need to configure the vCPUs that belong to the same socket in the same vNode. Otherwise, the VM may be soft locked up after the CPU is hot added or removed, which may cause the VM panic. -- VMs do not support CPU hotplug during migration, hibernation, wake-up, or snapshot. -- Whether the hot added CPU can automatically go online depends on the VM OS logic rather than the virtualization layer. -- CPU hot add is restricted by the maximum number of CPUs supported by the Hypervisor and GuestOS. -- When a VM is being started, stopped, or restarted, the hot added CPU may become invalid. However, the hot added CPU takes effect after the VM is restarted. -- CPU hotplug may time out when a VM is starting, shutting down, or restarting. Retry when the VM is in the normal running state. -- During VM CPU hotplug, if the number of added or removed CPUs is not an integer multiple of the number of cores in the VM CPU topology configuration item, the CPU topology displayed in the VM may be disordered. You are advised to add or remove CPUs whose number is an integer multiple of the number of cores each time. -- If the hot added or removed CPU needs to take effect online and is still valid after the VM is restarted, the `--config` and `--live` options need to be transferred to the `virsh setvcpus` interface to persist the hot added or removed CPU. - -#### Procedure - -**VM XML Configuration** - -1. To use the CPU hot add function, configure the number of CPUs, the maximum number of CPUs supported by the VM, and the VM chipset type when creating the VM. (For the AArch64 architecture, the virt-4.2 or a later version is required. For the x86\_64 architecture, the pc-i440fx-1.5 or later version is required. The AArch64 VM is used as an example. The configuration template is as follows: - - ```xml - - ... - n - - hvm - - ... - - ``` - - >![](./public_sys-resources/icon-note.gif) **Note** - > - >- The value of placement must be static. - >- m indicates the current number of CPUs on the VM, that is, the default number of CPUs after the VM is started. n indicates the maximum number of CPUs that can be hot added to a VM. The value cannot exceed the maximum CPU specifications supported by the Hypervisor or GuestOS. n is greater than or equal to m. - - For example, if the current number of CPUs of a VM is 4 and the maximum number of hot added CPUs is 64, the XML configuration is as follows: - - ```xml - - ... - 64 - - hvm - - ... - ``` - -**Hot Adding and Bringing CPUs Online** - -1. If the hot added CPU needs to be automatically brought online, create the udev rules file /etc/udev/rules.d/99-hotplug-cpu.rules in the VM as user root and define the udev rules in the file. The following is an example: - - ```text - ### automatically online hot-plugged cpu - ACTION=="add", SUBSYSTEM=="cpu", ATTR{online}="1" - ``` - - >![](./public_sys-resources/icon-note.gif) **Note** - >If you do not use the udev rules, you can use the root permission to manually bring the hot added CPU online by running the following commands: - > - >```shell - >for i in `grep -l 0 /sys/devices/system/cpu/cpu*/online` - >do - > echo 1 > $i - >done - >``` - -2. Use the virsh tool to hot add CPUs to the VM. For example, to set the number of CPUs after hot adding to 6 on the VM named openEulerVM and make the hot add take effect online, run the following command: - - ```shell - virsh setvcpus openEulerVM 6 --live - ``` - - >![](./public_sys-resources/icon-note.gif) **Note** - >The format for running the `virsh setvcpus` command to hot add VM CPUs is as follows: - > - >```shell - >virsh setvcpus [--config] [--live] - >``` - > - >- `domain`: Parameter, which is mandatory. Specifies the name of a VM. - >- `count`: Parameter, which is mandatory. Specifies the number of target CPUs, that is, the number of CPUs after hot adding. - >- `--config`: Option, which is optional. This parameter is still valid when the VM is restarted. - >- `--live`: Option, which is optional. The configuration takes effect online. - -**Hot Removing CPUs** - -Use the virsh tool to hot remove CPUs from the VM. For example, to set the number of CPUs after hot removal to 4 on the VM named openEulerVM, run the following command: - -```shell -virsh setvcpus openEulerVM 4 --live -``` - ->![](./public_sys-resources/icon-note.gif) **Note** ->The format for running the `virsh setvcpus` command to hot remove VM CPUs is as follows: -> ->```shell ->virsh setvcpus [--config] [--live] ->``` -> ->- `domain`: Parameter, which is mandatory. Specifies the name of a VM. ->- `count`: Parameter, which is mandatory. Specifies the number of target CPUs, that is, the number of CPUs after hot removal. ->- `--config`: Option, which is optional. This parameter is still valid when the VM is restarted. ->- `--live`: Option, which is optional. The configuration takes effect online. - -## Managing Virtual Memory - -### Introduction to NUMA - -Traditional multi-core computing uses the symmetric multi-processor \(SMP\) mode. Multiple processors are connected to a centralized memory and I/O bus. All processors can access only the same physical memory. Therefore, the SMP system is also referred to as a uniform memory access \(UMA\) system. Uniformity means that a processor can only maintain or share a unique value for each data record in memory at any time. Obviously, the disadvantage of SMP is its limited scalability, because when the memory and the I/O interface are saturated, adding a processor cannot obtain higher performance. - -The non-uniform memory access architecture \(NUMA\) is a distributed memory access mode. In this mode, a processor can access different memory addresses at the same time, which greatly improves concurrency. With this feature, a processor is divided into multiple nodes, each of which is allocated a piece of local memory space. The processors of all nodes can access all physical memories, but the time required for accessing the memory on the local node is much shorter than that on a remote node. - -### Configuring Host NUMA - -To improve VM performance, you can specify NUMA nodes for a VM using the VM XML configuration file before the VM is started so that the VM memory is allocated to the specified NUMA nodes. This feature is usually used together with the vCPU to prevent the vCPU from remotely accessing the memory. - -#### Procedure - -- Check the NUMA topology of the host. - - ```shell - $ numactl -H - available: 4 nodes (0-3) - node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 - node 0 size: 31571 MB - node 0 free: 17095 MB - node 1 cpus: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 - node 1 size: 32190 MB - node 1 free: 28057 MB - node 2 cpus: 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 - node 2 size: 32190 MB - node 2 free: 10562 MB - node 3 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 - node 3 size: 32188 MB - node 3 free: 272 MB - node distances: - node 0 1 2 3 - 0: 10 15 20 20 - 1: 15 10 20 20 - 2: 20 20 10 15 - 3: 20 20 15 10 - ``` - -- Add the **numatune** field to the VM XML configuration file to create and start the VM. For example, to allocate NUMA node 0 on the host to the VM, configure parameters as follows: - - ```xml - - - - ``` - - If the vCPU of the VM is bound to the physical CPU of **node 0**, the performance deterioration caused by the vCPU accessing the remote memory can be avoided. - - >![](./public_sys-resources/icon-note.gif) **NOTE:** - > - >- The sum of memory allocated to the VM cannot exceed the remaining available memory of the NUMA node. Otherwise, the VM may fail to start. - >- You are advised to bind the VM memory and vCPU to the same NUMA node to avoid the performance deterioration caused by vCPU access to the remote memory. For example, bind the vCPU to NUMA node 0 as well. - -### Configuring Guest NUMA - -Many service software running on VMs is optimized for the NUMA architecture, especially for large-scale VMs. openEuler provides the Guest NUMA feature to display the NUMA topology in VMs. You can identify the structure to optimize the performance of service software and ensure better service running. - -When configuring guest NUMA, you can specify the location of vNode memory on the host to implement memory block binding and vCPU binding so that the vCPU and memory on the vNode are on the same physical NUMA node. - -#### Procedure - -After Guest NUMA is configured in the VM XML configuration file, you can view the NUMA topology on the VM. **** is mandatory for Guest NUMA. - -```xml - - - - - - - - - - - [...] - - - - - - -``` - ->![](./public_sys-resources/icon-note.gif) **NOTE:** -> ->- **** provides the NUMA topology function for VMs. **cell id** indicates the vNode ID, **cpus** indicates the vCPU ID, and **memory** indicates the memory size on the vNode. ->- If you want to use Guest NUMA to provide better performance, configure <**numatune\>** and **** so that the vCPU and memory are distributed on the same physical NUMA node. ->- **cellid** in **** corresponds to **cell id** in ****. **mode** can be set to **strict** \(apply for memory from a specified node strictly. If the memory is insufficient, the application fails.\), **preferred** \(apply for memory from a node first. If the memory is insufficient, apply for memory from another node\), or **interleave** \(apply for memory from a specified node in cross mode\).; **nodeset** indicates the specified physical NUMA node. ->- In ****, you need to bind the vCPU in the same **cell id** to the physical NUMA node that is the same as the **memnode**. - -### Memory Hot Add - -#### Overview - -In virtualization scenarios, the memory, CPU, and external devices of VMs are simulated by software. Therefore, the memory can be adjusted online for VMs at the virtualization bottom layer. In the current openEuler version, memory can be added to a VM online. If the physical memory of a VM is insufficient and the VM cannot be shut down, you can use this feature to add physical memory resources to the VM. - -#### Constraints - -- For processors using the AArch64 architecture, the specified VM chipset type \(machine\) needs to be virt-4.2 or a later version when a VM is created.For processors using the x86 architecture, the specified VM chipset type \(machine\) needs to be a later version than pc-i440fx-1.5 when a VM is created. -- Guest NUMA on which the memory hot add feature depends needs to be configured on the VM. Otherwise, the memory hot add process cannot be completed. -- When hot adding memory, you need to specify the ID of Guest NUMA node to which the new memory belongs. Otherwise, the memory hot add fails. -- The VM kernel should support memory hot add. Otherwise, the VM cannot identify the newly added memory or the memory cannot be brought online. -- For a VM that uses hugepages, the capacity of the hot added memory should be an integral multiple of hugepagesz. Otherwise, the hot add fails. -- The hot added memory size should be an integral multiple of the Guest physical memory block size (block\_size\_bytes). Otherwise, the VM cannot go online. The value of block\_size\_bytes can be obtained using the lsmem command in Guest. -- After n pieces of virtio-net NICs are configured, the maximum number of hot add times is set to min\{max\_slot, 64 - n\} to reserve slots for NICs. -- The vhost-user device and the memory hot add feature are mutually exclusive. A VM configured with the vhost-user device does not support memory hot add. After the memory is hot added to a VM, the vhost-user device cannot be hot added. -- If the VM OS is Linux, ensure that the initial memory is greater than or equal to 4 GB. -- If the VM OS is Windows, the first hot added memory needs to be specified to Guest NUMA node0. Otherwise, the hot added memory cannot be identified by the VM. -- In passthrough scenarios, memory needs to be allocated in advance. Therefore, it is normal that the startup and hot add of memory are slower than those of common VMs (especially large-specification VMs). -- It is recommended that the ratio of the available memory to the hot added memory be at least 1:32. That is, at least 1 GB available memory is required for the VM with 32 GB hot added memory. If the ratio is less than 1:32, the VM may be suspended. -- Whether the hot added memory can automatically go online depends on the VM OS logic. You can manually bring the memory online or configure the udev rules to automatically bring the memory online. - -#### Procedure - -**VM XML Configuration** - -1. To use the memory hot add function, configure the maximum hot add memory size and reserved slot number, and configure the Guest NUMA topology when creating a VM. - - For example, run the following command to configure 32 GB initial memory for a VM, reserve 256 slots, set the memory upper limit to 1 TB, and configure two NUMA nodes: - - ```xml - - 32 - 1024 - - - - - - - - .... - ``` - ->![](./public_sys-resources/icon-note.gif) **Note** ->In the preceding information, ->the value of slots in the maxMemory field indicates the reserved memory slots. The maximum value is 256. ->maxMemory indicates the maximum physical memory supported by the VM. ->For details about how to configure Guest NUMA, see "Configuring Guest NUMA." - -**Hot Adding and Bringing Memory Online** - -1. If the hot added memory needs to be automatically brought online, create the udev rules file /etc/udev/rules.d/99-hotplug-memory.rules in the VM as user root and define the udev rules in the file. The following is an example: - - ```text - ### automatically online hot-plugged memory - ACTION=="add", SUBSYSTEM=="memory", ATTR{state}="online" - ``` - -2. Create a memory description XML file based on the size of the memory to be hot added and the Guest NUMA node of the VM. - - For example, to hot add 1 GB memory to NUMA node0, run the following command: - - ```xml - - - 1024 - 0 - - - ``` - -3. Run the virsh attach-device command to hot add memory to the VM. In the command, openEulerVM indicates the VM name, memory.xml indicates the description file of the hot added memory, and --live indicates that the hot added memory takes effect online. You can also run the --config command to persist the hot added memory to the VM XML file. - - ```shell - virsh attach-device openEulerVM memory.xml --live - ``` - - >![](./public_sys-resources/icon-note.gif) **Note** - >If you do not use the udev rules, you can use the root permission to manually bring the hot added memory online by running the following command: - > - >```text - >for i in `grep -l offline /sys/devices/system/memory/memory*/state` - >do - > echo online > $i - >done - >``` diff --git a/docs/en/docs/Virtualization/tool-guide.md b/docs/en/docs/Virtualization/tool-guide.md deleted file mode 100644 index 565ebecf35494d5cc45e3a4f8f2a4973841fa0dd..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/tool-guide.md +++ /dev/null @@ -1 +0,0 @@ -To help users better use virtualization, openEuler provides a set of tools, including vmtop and LibcarePlus. This section describes how to install and use these tools. \ No newline at end of file diff --git a/docs/en/docs/Virtualization/virtualization-installation.md b/docs/en/docs/Virtualization/virtualization-installation.md deleted file mode 100644 index 81e0efb790d84d6a9e1723c54bd2f017e682c7a0..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/virtualization-installation.md +++ /dev/null @@ -1,154 +0,0 @@ -# Installing Virtualization Components - -This chapter describes how to install virtualization components in openEuler. - -- [Installing Virtualization Components](#installing-virtualization-components) - - [Minimum Hardware Requirements](#minimum-hardware-requirements) - - [Installing Core Virtualization Components](#installing-core-virtualization-components) - - [Installation Methods](#installation-methods) - - [Prerequisites](#prerequisites) - - [Procedure](#procedure) - - [Verifying the Installation](#verifying-the-installation) - - -## Minimum Hardware Requirements - -The minimum hardware requirements for installing virtualization components on openEuler are as follows: - -- AArch64 processor architecture: ARMv8 or later, supporting virtualization expansion -- x86\_64 processor architecture, supporting VT-x -- 2-core CPU -- 4 GB memory -- 16 GB available disk space - -## Installing Core Virtualization Components - -### Installation Methods - -#### Prerequisites - -- The yum source has been configured. For details, see the _openEuler 21.03 Administrator Guide_. -- Only the administrator has permission to perform the installation. - -#### Procedure - -1. Install the QEMU component. - - ```shell - # yum install -y qemu - ``` - - >![](./public_sys-resources/icon-caution.gif) Notice: - >By default, the QEMU component runs as user qemu and user group qemu. If you are not familiar with Linux user group and user permission management, you may encounter insufficient permission when creating and starting VMs. You can use either of the following methods to solve this problem: - >Method 1: Modify the QEMU configuration file. Run the `sudo vim /etc/libvirt/qemu.conf` command to open the QEMU configuration file, find `user = "root"` and `group = "root"`, uncomment them (delete `#`), save the file, and exit. - >Method 2: Change the owner of the VM files. Ensure that user qemu has the permission to access the folder where VM files are stored. Run the `sudo chown qemu:qemu xxx.qcow2` command to change the owner of the VM files that need to be read and written. - -2. Install the libvirt component. - - ```shell - # yum install -y libvirt - ``` - -3. Start the libvirtd service. - - ```shell - # systemctl start libvirtd - ``` - - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The KVM module is integrated in the openEuler kernel and does not need to be installed separately. - -### Verifying the Installation - -1. Check whether the kernel supports KVM virtualization, that is, check whether the **/dev/kvm** and **/sys/module/kvm** files exist. The command and output are as follows: - - ```shell - # ls /dev/kvm - /dev/kvm - ``` - - ``` - # ls /sys/module/kvm - parameters uevent - ``` - - If the preceding files exist, the kernel supports KVM virtualization. If the preceding files do not exist, KVM virtualization is not enabled during kernel compilation. In this case, you need to use the Linux kernel that supports KVM virtualization. - -2. Check whether QEMU is successfully installed. If the installation is successful, the QEMU software package information is displayed. The command and output are as follows: - - ```shell - # rpm -qi qemu - Name : qemu - Epoch : 2 - Version : 4.0.1 - Release : 10 - Architecture: aarch64 - Install Date: Wed 24 Jul 2019 04:04:47 PM CST - Group : Unspecified - Size : 16869484 - License : GPLv2 and BSD and MIT and CC-BY - Signature : (none) - Source RPM : qemu-4.0.0-1.src.rpm - Build Date : Wed 24 Jul 2019 04:03:52 PM CST - Build Host : localhost - Relocations : (not relocatable) - URL : http://www.qemu.org - Summary : QEMU is a generic and open source machine emulator and virtualizer - Description : - QEMU is a generic and open source processor emulator which achieves a good - emulation speed by using dynamic translation. QEMU has two operating modes: - - * Full system emulation. In this mode, QEMU emulates a full system (for - example a PC), including a processor and various peripherals. It can be - used to launch different Operating Systems without rebooting the PC or - to debug system code. - * User mode emulation. In this mode, QEMU can launch Linux processes compiled - for one CPU on another CPU. - - As QEMU requires no host kernel patches to run, it is safe and easy to use. - ``` - -3. Check whether libvirt is successfully installed. If the installation is successful, the libvirt software package information is displayed. The command and output are as follows: - - ```shell - # rpm -qi libvirt - Name : libvirt - Version : 5.5.0 - Release : 1 - Architecture: aarch64 - Install Date: Tue 30 Jul 2019 04:56:21 PM CST - Group : Unspecified - Size : 0 - License : LGPLv2+ - Signature : (none) - Source RPM : libvirt-5.5.0-1.src.rpm - Build Date : Mon 29 Jul 2019 08:14:57 PM CST - Build Host : 71e8c1ce149f - Relocations : (not relocatable) - URL : https://libvirt.org/ - Summary : Library providing a simple virtualization API - Description : - Libvirt is a C toolkit to interact with the virtualization capabilities - of recent versions of Linux (and other OSes). The main package includes - the libvirtd server exporting the virtualization support. - ``` - -4. Check whether the libvirt service is started successfully. If the service is in the **Active** state, the service is started successfully. You can use the virsh command line tool provided by the libvirt. The command and output are as follows: - - ```shell - # systemctl status libvirtd - ● libvirtd.service - Virtualization daemon - Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) - Active: active (running) since Tue 2019-08-06 09:36:01 CST; 5h 12min ago - Docs: man:libvirtd(8) - https://libvirt.org - Main PID: 40754 (libvirtd) - Tasks: 20 (limit: 32768) - Memory: 198.6M - CGroup: /system.slice/libvirtd.service - ─40754 /usr/sbin/libvirtd - - ``` - - diff --git a/docs/en/docs/Virtualization/virtualization.md b/docs/en/docs/Virtualization/virtualization.md deleted file mode 100644 index ff1c7cb60c45b2cae0c90a48e75b5ea9f7fa5ef4..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/virtualization.md +++ /dev/null @@ -1,3 +0,0 @@ -# Virtualization User Guide - -This document describes virtualization, how to install virtualization based on openEuler, and how to use virtualization, helping administrators and common users install and use virtualization. diff --git a/docs/en/docs/Virtualization/vm-configuration.md b/docs/en/docs/Virtualization/vm-configuration.md deleted file mode 100644 index de74da00de45abed40d38fd4d1e362f7dabb34b9..0000000000000000000000000000000000000000 --- a/docs/en/docs/Virtualization/vm-configuration.md +++ /dev/null @@ -1,821 +0,0 @@ -# VM Configuration - -- [VM Configuration](#vm-configuration) - - [Introduction](#introduction) - - [VM Description](#vm-description) - - [vCPU and Virtual Memory](#vcpu-and-virtual-memory) - - [Virtual Device Configuration](#virtual-device-configuration) - - [Storage Devices](#storage-devices) - - [Network Devices](#network-devices) - - [Bus Configuration](#bus-configuration) - - [Other Common Devices](#other-common-devices) - - [Configurations Related to the System Architecture](#configurations-related-to-the-system-architecture) - - [Other Common Configuration Items](#other-common-configuration-items) - - [XML Configuration File Example](#xml-configuration-file-example) - -## Introduction - -### Overview - -Libvirt tool uses XML files to describe a VM feature, including the VM name, CPU, memory, disk, NIC, mouse, and keyboard. You can manage a VM by modifying configuration files. This section describes the elements in the XML configuration file to help users configure VMs. - -### Format - -The VM XML configuration file uses domain as the root element, which contains multiple other elements. Some elements in the XML configuration file can contain corresponding attributes and attribute values to describe VM information in detail. Different attributes of the same element are separated by spaces. - -The basic format of the XML configuration file is as follows. In the format, **label** indicates the label name, **attribute** indicates the attribute, and **value** indicates the attribute value. Change them based on the site requirements. - -```xml - - VMName - 8 - 4 - - - - - -``` - -### Process - -1. Create an XML configuration file with domain root element. -2. Use the name tag to specify a unique VM name based on the naming rule. -3. Configure system resources such as the virtual CPU \(vCPU\) and virtual memory. -4. Configure virtual devices. - 1. Configure storage devices. - 2. Configure network devices. - 3. Configure the external bus structure. - 4. Configure external devices such as the mouse. - -5. Save the XML configuration file. - -## VM Description - -### Overview - -This section describes how to configure the VM **domain** root element and VM name. - -### Elements - -- **domain**: Root element of a VM XML configuration file, which is used to configure the type of the hypervisor that runs the VM. - - **type**: Type of a domain in virtualization. In the openEuler virtualization, the attribute value is **kvm**. - -- **name**: VM name. - - The VM name is a unique character string on the same host. The VM name can contain only digits, letters, underscores \(\_\), hyphens \(-\), and colons \(:\), but cannot contain only digits. The VM name can contain a maximum of 64 characters. - -### Configuration Example - -For example, if the VM name is **openEuler**, the configuration is as follows: - -```xml - - openEuler - ... - -``` - -## vCPU and Virtual Memory - -### Overview - -This section describes how to configure the vCPU and virtual memory. - -### Elements - -- **vcpu**: The number of virtual processors. -- **memory**: The size of the virtual memory. - - **unit**: The memory unit. The value can be KiB \(210 bytes\), MiB \(220 bytes\), GiB \(230 bytes\), or TiB \(240 bytes\). - -- **cpu**: The mode of the virtual processor. - - **mode**: The mode of the vCPU. - - - **host-passthrough**: indicates that the architecture and features of the virtual CPU are the same as those of the host. - - - **custom**: indicates that the architecture and features of the virtual CPU are configured by the **cpu** element. - - Sub-element **topology**: A sub-element of the element cpu, used to describe the topology structure of a vCPU mode. - - - The attributes **socket**, **cores**, and **threads** of the sub-element topology describe the number of CPU sockets of a VM, the number of processor cores included in each CPU socket, and the number of threads included in each processor core, respectively. The attribute value is a positive integer, and the product of the three values equals the number of vCPUs. - - The ARM architecture supports the virtual hyper-threading function. The virtual CPU hot add and the virtual hyper-threading function are mutually exclusive. - Sub-element **model**: A sub-element of the element cpu, used to describe the CPU model when **mode** is custom. - - Sub-element **feature**: A sub-element of the element cpu, used to enable/disable a CPU feature when **mode** is custom. The attribute **name** describes the name of the CPU feature. And whether enable the CPU feature is controlled by the attribute **policy**: - - - **force**: force enable the CPU feature regardless of it being supported by host CPU. - - - **require**: enable the CPU feature. - - - **optional**: the CPU feature will be enabled if and only if it is supported by host CPU. - - - **disable**: disable the CPU feature. - - - **forbid**: disable the CPU feature and guest creation will fail if the feature is supported by host CPU. - -### Configuration Example - -For example, if the number of vCPUs is 4, the processing mode is host-passthrough, the virtual memory is 8 GiB, the four CPUs are distributed in two CPU sockets, and hyperthreading is not supported, the configuration is as follows: - -```xml - - ... - 4 - 8 - - - -... - -``` - -If the virtual memory is 8 GiB, the number of vCPUs is 4, the processing mode is custom, the CPU model is Kunpeng-920, and pmull is disabled, the configuration is as follows: - -```xml - - ... - 4 - 8 - - Kunpeng-920 - - - ... - -``` - -## Virtual Device Configuration - -The VM XML configuration file uses the **devices** elements to configure virtual devices, including storage devices, network devices, buses, and mouse devices. This section describes how to configure common virtual devices. - -### Storage Devices - -#### Overview - -This section describes how to configure virtual storage devices, including floppy disks, disks, and CD-ROMs and their storage types. - -#### Elements - -The XML configuration file uses the **disk** element to configure storage devices. [Table 1](#table14200183410353) describes common **disk** attributes. [Table 2](#table4866134925114) describes common subelements and their attributes. - -**Table 1** Common attributes of the **disk** element - - - - - - - - - - - - - - - - - - - -
ElementAttributeDescriptionAttribute Value and Description
disktypeSpecifies the type of the backend storage medium.block: block device
file: file device
dir: directory path
network: network disk
deviceSpecifies the storage medium to be presented to the VM.disk: disk (default)
floppy: floppy disk
cdrom: CD-ROM
- -**Table 2** Common subelements and attributes of the **disk** element - -| Subelement | Subelement Description | Attribute Description | -| ---------- | ------------------------------------------------------------ | ------------------------------------------------------------ | -| source | Specifies the backend storage medium, which corresponds to the type specified by the **type** attribute of the **disk** element. | **file**: file type. The value is the fully qualified path of the corresponding file.
**dev**: block type. The value is the fully qualified path of the corresponding host device.
**dir**: directory type. The value is the fully qualified path of the disk directory.
**protocol**: protocol in use.
**name**: RBD disk name. The format is as follows: $pool/$volume .**host name**: mon address.
**port**: port of the mon address.
| -| driver | Details about the specified backend driver | **type**: disk format type. The value can be **raw** or **qcow2**, which must be the same as that of source.
**io**: disk I/O mode. The options are **native** and **threads**.
**cache**: disk cache mode. The value can be **none**, **writethrough**, **writeback**, or **directsync**.
**iothread**: I/O thread allocated to the disk.
**error_policy**: processing policy when an I/O write error occurs. The value can be stop, report, ignore, enospace, or retry.
**rerror_policy**: processing policy when an I/O read error occurs. The value can be stop, report, ignore, enospac, or retry.
**retry_interval**: I/O retry interval. The value ranges from 0 to MAX_INT, in milliseconds. This parameter can be set only when error_policy or rerror_policy is set to retry.
**retry_timeout**: I/O retry timeout interval. The value ranges from 0 to MAX_INT, in milliseconds. This parameter can be set only when error_policy or rerror_policy is set to retry. | -| target | The bus and device that a disk presents to a VM. | **dev**: specifies the logical device name of a disk, for example, sd\[a-p] for SCSI, SATA, and USB buses and hd\[a-d] for IDE disks.

**bus**: specifies the type of a disk. Common types include scsi, usb, sata, and virtio. | -| boot | The disk can be used as the boot disk. | **order**: specifies the disk startup sequence. | -| readonly | The disk is read-only and cannot be modified by the VM. Generally, it is used together with the CD-ROM drive. | - | - -#### Configuration Example - -After the VM image is prepared according to [Preparing a VM Image](./environment-preparation.md#preparing-a-vm-image), you can use the following XML configuration file to configure the virtual disk for the VM. - -For example, this example configures two I/O threads for the virtual machine, one for a block disk device, one for an optical disc device, and one for an RBD disk, and the first I/O thread is allocated to the block disk device for use. The backend medium of the disk device is in qcow2 format and is used as the preferred boot disk. -Before using an RBD disk, ensure that the qemu-block-rbd driver is installed. Run the following command as the **root** user to install the driver: - -```sh -yum install qemu-block-rbd -``` - -Configuration example: - -```xml - - ... - 2 - - - - - - - - - - - - - - - - - - - - - - ... - - -``` - -### Network Devices - -#### Overview - -The XML configuration file can be used to configure virtual network devices, including the ethernet mode, bridge mode, and vhostuser mode. This section describes how to configure vNICs. - -#### Elements - -In the XML configuration file, the element **interface** is used, and its attribute **type** indicates the mode of the vNIC. The options are **ethernet**, **bridge**, and **vhostuser**. The following uses the vNIC in bridge mode as an example to describe its subelements and attributes. - -**Table 1** Common subelements of a vNIC in bridge mode - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Subelement

-

Subelement Description

-

Attribute and Description

-

mac

-

The mac address of the vNIC.

-

address: specifies the mac address. If this parameter is not set, the system automatically generates a mac address.

-

target

-

Name of the backend vNIC.

-

dev: name of the created backend tap device.

-

source

-

Specify the backend of the vNIC.

-

bridge: used together with the bridge mode. The value is the bridge name.

-

boot

-

The NIC can be used for remote startup.

-

order: specifies the startup sequence of NICs.

-

model

-

Indicates the type of a vNIC.

-

type: virtio is usually used for the NIC in bridge mode.

-

virtualport

-

Port type

-

type: If an OVS bridge is used, set this parameter to openvswitch.

-

driver

-

Backend driver type

-

name: driver name. The value is vhost.

-

queues: the number of NIC queues.

-
- -#### Configuration Example - -- After creating the Linux bridge br0 by referring to [Preparing a VM Image](./environment-preparation.md#preparing-a-vm-image), configure a vNIC of the VirtIO type bridged on the br0 bridge. The corresponding XML configuration is as follows: - - ```xml - - ... - - - - - - ... - - - ``` - -- If an OVS network bridge is created according to [Preparing a VM Image](./environment-preparation.md#preparing-a-vm-image), configure a VirtIO vNIC device that uses the vhost driver and has four queues. - - ```xml - - ... - - - - - - - - ... - - - ``` - -### Bus Configuration - -#### Overview - -The bus is a channel for information communication between components of a computer. An external device needs to be mounted to a corresponding bus, and each device is assigned a unique address \(specified by the subelement **address**\). Information exchange with another device or a central processing unit \(CPU\) is completed through the bus network. Common device buses include the ISA bus, PCI bus, USB bus, SCSI bus, and PCIe bus. - -The PCIe bus is a typical tree structure and has good scalability. The buses are associated with each other by using a controller. The following uses the PCIe bus as an example to describe how to configure a bus topology for a VM. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->The bus configuration is complex. If the device topology does not need to be precisely controlled, the default bus configuration automatically generated by libvirt can be used. - -#### Elements - -In the XML configuration of libvirt, each controller element \(**controller**\) represents a bus, and one or more controllers or devices can be mounted to one controller depending on the VM architecture. This topic describes common attributes and subelements. - -**controller**: controller element, which indicates a bus. - -- Attribute **type**: bus type, which is mandatory for the controller. The common values are **pci**, **usb**, **scsi**, **virtio-serial**, **fdc**, and **ccid**. -- Attribute **index**: bus number of the controller \(the number starts from 0\), which is mandatory for the controller. This attribute can be used in the **address** element. -- Attribute **model**: specific model of the controller, which is mandatory for the controller. The available values are related to the value of **type**. For details about the mapping and description, see [Table 1](#table191911761111). -- Subelement **address**: mount location of a device or controller on the bus network. - - Attribute **type**: device address type. The common values are **pci**, **usb**, or **drive**. The attribute varies according to the **type** of the **address**. For details about the common **type** attribute value and the corresponding **address** attribute, see [Table 2](#table1200165711314). - -- Subelement **model**: name of a controller model. - - Attribute **name**: name of a controller model, which corresponds to the **model** attribute in the parent element controller. - -**Table 1** Mapping between the common values of **type** and **model** for the controller. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Value of Type

-

Value of Model

-

Introduction

-

pci

-

pcie-root

-

PCIe root node, which can be used to mount PCIe devices or controllers.

-

pcie-root-port

-

Only one slot can be used to mount a PCIe device or controller.

-

pcie-to-pci-bridge

-

PCIe-to-PCI bridge controller, which can be used to mount PCI devices.

-

usb

-

ehci

-

USB 2.0 controller, which can be used to mount USB 2.0 devices.

-

nec-xhci

-

USB 3.0 controller, which can be used to mount USB 3.0 devices.

-

scsi

-

virtio-scsi

-

VirtIO SCSI controller, which can be used to mount block devices, such as disks and CD-ROMs.

-

virtio-serial

-

virtio-serial

-

VirtIO serial port controller, which can be used to mount serial port devices, such as a pty serial port.

-
- -**Table 2** Attributes of the **address** element in different devices. - - - - - - - - - - - - - - - - - - - -

Value of Type

-

Description

-

Address

-

pci

-

The address type is PCI address, indicating the mount location of the device on the PCI bus network.

-

domain: domain ID of the PCI device.

-

bus: bus number of the PCI device.

-

slot: device number of the PCI device.

-

function: function number of the PCI device.

-

multifunction: (optional) specifies whether to enable the multifunction function.

-

usb

-

The address type is USB address, indicating the location of the device on the USB bus.

-

bus: bus number of the USB device.

-

port: port number of the USB device.

-

drive

-

The address type is storage device address, indicating the owning disk controller and its position on the bus.

-

controller: the number of the owning controller.

-

bus: channel number of the device output.

-

target: target number of the storage device.

-

unit: lun number of the storage device.

-
- -#### Configuration Example - -This example shows the topology of a PCIe bus. Three PCIe-Root-Port controllers are mounted to the PCIe root node \(BUS 0\). The multifunction function is enabled for the first PCIe-Root-Port controller \(BUS 1\). A PCIe-to-PCI-bridge controller is mounted to the first PCIe-Root-Port controller to form a PCI bus \(BUS 3\). A virtio-serial device and a USB 2.0 controller are mounted to the PCI bus. A SCSI controller is mounted to the second PCIe-Root-Port controller \(BUS 2\). No device is mounted to the third PCIe-Root-Port controller \(BUS 0\). The configuration details are as follows: - -```xml - - ... - - - -
- - -
- - - -
- - -
- - -
- - -
- - -
- - ... - - -``` - -### Other Common Devices - -#### Overview - -In addition to storage devices and network devices, some external devices need to be specified in the XML configuration file. This section describes how to configure these elements. - -#### Elements - -- **serial**: serial port device - - Attribute **type**: specifies the serial port type. The common attribute values are **pty**, **tcp**, **pipe**, and **file**. - -- **video**: media device - - Attribute **type**: media device type. The common attribute value of the AArch64 architecture is **virtio**, and that of the x86\_64 architecture is **vga** or **cirrus**. - - Subelement **model**: subelement of **video**, which is used to specify the media device type. - - In the subelement **model**, if **type** is set to **vga**, a Video Graphics Array \(VGA\) video card is configured. **vram** indicates the size of the video RAM, in KB by default. - - For example, if a 16 MB VGA video card is configured for an x86\_64 VM, configuration in the XML file is as follows. In the example, the value of **vram** is the size of video RAM, in KB by default. - - ```xml - - ``` - -- **input**: output device - - **type** attribute: specifies the type of the output device. The common attribute values are **tabe** and **keyboard**, indicating that the output device is the tablet and keyboard respectively. - - **bus**: specifies the bus to be mounted. The common attribute value is **USB**. - -- **emulator**: emulator application path -- **graphics**: graphics device - - **type** attribute: specifies the type of a graphics device. The common attribute value is **vnc**. - - **listen** attribute: specifies the IP address to be listened to. - -#### Configuration Example - -For example, in the following example, the VM emulator path, pty serial port, VirtIO media device, USB tablet, USB keyboard, and VNC graphics device are configured. - ->![](./public_sys-resources/icon-note.gif) **NOTE:** ->When **type** of **graphics** is set to **VNC**, you are advised to set the **passwd** attribute, that is, the password for logging in to the VM using VNC. - -```xml - - ... - - /usr/libexec/qemu-kvm - - - - - - ... - - -``` - -## Configurations Related to the System Architecture - -### Overview - -The XML configuration file contains configurations related to the system architecture, which cover the mainboard, CPU, and some features related to the architecture. This section describes meanings of these configurations. - -### Elements - -- **os**: defines VM startup parameters. - - Subelement **type**: specifies the VM type. The attribute **arch** indicates the architecture type, for example, AArch64. The attribute **machine** indicates the type of VM chipset. Supported chipset type can be queried by running the **qemu-kvm -machine ?** command. For example, the AArch64 architecture supports the **virt** type. - - Subelement **loader**: specifies the firmware to be loaded, for example, the UEFI file provided by the EDK. The **readonly** attribute indicates whether the file is read-only. The value can be **yes** or **no**. The **type** attribute indicates the **loader** type. The common values are **rom** and **pflash**. - - Subelement **nvram**: specifies the path of the **nvram** file, which is used to store the UEFI startup configuration. - -- **features**: Hypervisor controls some VM CPU/machine features, such as the advanced configuration and power interface \(ACPI\) and the GICv3 interrupt controller specified by the ARM processor. - -### Example for AArch64 Architecture - -The VM is of the **aarch64** type and uses **virt** chipset. The VM configuration started using UEFI is as follows: - -```xml - - ... - - hvm - /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw - /var/lib/libvirt/qemu/nvram/openEulerVM.fd - - ... - -``` - -Configure ACPI and GIC V3 interrupt controller features for the VM. - -```xml - - - - -``` - -### Example for x86\_64 Architecture - -The x86\_64 architecture supports both BIOS and UEFI boot modes. If **loader** is not configured, the default BIOS boot mode is used. The following is a configuration example in which the UEFI boot mode and Q35 chipsets are used. - -```xml - - ... - - hvm - /usr/share/edk2/ovmf/OVMF.fd - - ... - -``` - -## Other Common Configuration Items - -### Overview - -In addition to system resources and virtual devices, other elements need to be configured in the XML configuration file. This section describes how to configure these elements. - -### Elements - -- **iothreads**: specifies the number of **iothread**, which can be used to accelerate storage device performance. - -- **on\_poweroff**: action taken when a VM is powered off. -- **on\_reboot**: action taken when a VM is rebooted. -- **on\_crash**: action taken when a VM is on crash. -- **clock**: indicates the clock type. - - **offset** attribute: specifies the VM clock synchronization type. The value can be **localtime**, **utc**, **timezone**, or **variable**. - -### Configuration Example - -Configure two **iothread** for the VM to accelerate storage device performance. - -```xml -2 -``` - -Destroy the VM when it is powered off. - -```xml -destroy -``` - -Restart the VM. - -```xml -restart -``` - -Restart the VM when it is crashed. - -```xml -restart -``` - -The clock uses the **utc** synchronization mode. - -```xml - -``` - -## XML Configuration File Example - -### Overview - -This section provides XML configuration files of a basic AArch64 VM and a x86\_64 VM as two examples for reference. - -### Example 1 - -An XML configuration file of AArch64 VM, which contains basic elements. The following is a configuration example: - -```xml - - openEulerVM - 8 - 4 - - hvm - /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw - /var/lib/libvirt/qemu/nvram/openEulerVM.fd - - - - - - - - - 1 - - destroy - restart - restart - - /usr/libexec/qemu-kvm - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -### Example 2 - -An XML configuration file of x86\_64 VM, which contains basic elements and bus elements. The following is a configuration example: - -```xml - - openEulerVM - 8388608 - 8388608 - 4 - 1 - - hvm - - - - - - - - - destroy - restart - restart - - /usr/libexec/qemu-kvm - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - -
- - -
- - - - - - -