代码拉取完成，页面将自动刷新
同步操作将从 openEuler/A-Tune 强制同步，此操作会覆盖自 Fork 仓库以来所做的任何修改，且无法恢复！！！
确定后同步将在后台操作，完成时将刷新页面，请耐心等待。
English | 简体中文
Copyright © Huawei Technologies Co., Ltd. 2020. All rights reserved.
Your replication, use, modification, and distribution of this document are governed by the Creative Commons License Attribution-ShareAlike 4.0 International Public License (CC BY-SA 4.0). You can visit https://creativecommons.org/licenses/by-sa/4.0/ to view a human-readable summary of (and not a substitute for) CC BY-SA 4.0. For the complete CC BY-SA 4.0, visit https://creativecommons.org/licenses/by-sa/4.0/legalcode.
Trademarks and Permissions
A-Tune and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders.
Disclaimer
This document is used only as a guide. Unless otherwise specified by applicable laws or agreed by both parties in written form, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, including but not limited to non-infringement, timeliness, and specific purposes.
This document describes how to install and use A-Tune, which is a performance self-optimization software for openEuler.
This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use A-Tune. You need to have basic knowledge of the Linux OS.
An operating system (OS) is basic software that connects applications and hardware. It is critical for users to adjust OS and application configurations and make full use of software and hardware capabilities to achieve optimal service performance. However, numerous workload types and varied applications run on the OS, and the requirements on resources are different. Currently, the application environment composed of hardware and software involves more than 7000 configuration objects. As the service complexity and optimization objects increase, the time cost for optimization increases exponentially. As a result, optimization efficiency decreases sharply. Optimization becomes complex and brings great challenges to users.
Second, as infrastructure software, the OS provides a large number of software and hardware management capabilities. Each capability applies to different scenarios. Therefore, different capabilities need to be enabled or disabled in different scenarios to combine various capabilities provided by the system, maximizing the optimal performance of applications.
In addition, thousands of actual service scenarios exist, and hardware configurations for computing, network, and storage emerge. The lab cannot list all applications, service scenarios, and different hardware combinations.
To address the preceding challenges, openEuler launches A-Tune.
A-Tune is AI-based software that optimizes system performance. It uses AI technologies to create precise system profiles for service scenarios, aware and infer service characteristics, make intelligent decisions, and match and recommend the optimal system parameter configuration combination, ensuring the optimal running status of services.
The following figure shows the A-Tune core technical architecture, which consists of intelligent decision-making, system profile, and interaction system.
Table 1 describes the main features supported by A-Tune, feature maturity, and usage suggestions.
Table 1 Feature maturity
|
Auto optimization of 11 applications in seven workload types |
||
Based on the workload characteristics of applications, A-Tune classifies services into seven types. For details about the workload characteristics of each type and the applications supported by A-Tune, see Table 2.
Table 2 Supported workload types and applications
This chapter describes how to install and deploy A-Tune.
Install an openEuler OS. For details, see openEuler 1.0 Installation Guide.
This chapter describes the installation modes and methods of the A-Tune.
A-Tune can be installed in single-node or distributed mode.
Single-node mode
The client and server are installed on the same system.
Distributed mode
The client and server are installed on different systems.
To install the A-Tune, perform the following steps:
Mount an openEuler ISO file.
# mount openEuler-1.0-aarch64-dvd.iso /mnt
Configure the local yum source.
# vim /etc/yum.repos.d/local.repo
The configured contents are as follows:
[local]
name=local
baseurl=file:///mnt
gpgcheck=0
enabled=1
Install an A-Tune server.
NOTE:
In this step, both the server and client software packages are installed. For the single-node deployment, skip Step 4.
# yum install atune -y
Install an A-Tune client.
# yum install atune-client -y
Check whether the installation is successful.
# rpm -qa | grep atune
atune-client-xxx
atune-db-xxx
atune-xxx
If the preceding information is displayed, the installation is successful.
This chapter describes how to deploy A-Tune.
The configuration items in the A-Tune configuration file /etc/atuned/atuned.cnf are described as follows:
A-Tune service startup configuration
You can modify the parameter value as required.
System information
System is the parameter information required for system optimization. You must modify the parameter information according to the actual situation.
disk: Disk information to be collected during the analysis process or specified disk during disk optimization.
network: NIC information to be collected during the analysis process or specified NIC during NIC optimization.
user: User name used for ulimit optimization. Currently, only the user root is supported.
tls: SSL/TLS certificate verification for the gRPC and HTTP services of A-Tune. This is disabled by default. After TLS is enabled, you need to set the following environment variables before running the atune-adm command to communicate with the server:
tlsservercertfile: path of the gPRC server certificate.
tlsserverkeyfile: gPRC server key path.
tlshttpcertfile: HTTP server certificate path.
tlshttpkeyfile: HTTP server key path.
tlshttpcacertfile: CA certificate path of the HTTP server.
Log information
Change the log path and level based on the site requirements. By default, the log information is stored in /var/log/message.
Monitor information
The hardware information that is collected by default when the system is started.
#################################### server ###############################
# atuned config
[server]
# the address that the grpc server to bind to, default is 127.0.0.1
address = 127.0.0.1
# the atuned grpc listening port, default is 60001
# the port can be set between 0 to 65535 which not be used
port = 60001
# the rest service listening port, default is 8383
# the port can be set between 0 to 65535 which not be used
rest_port = 8383
# when run analysis command, the numbers of collected data.
# default is 20
sample_num = 20
# enable gRPC and http server authentication SSL/TLS
# default is false
# tls = true
# tlsservercertfile = /etc/atuned/server.pem
# tlsserverkeyfile = /etc/atuned/server.key
# tlshttpcertfile = /etc/atuned/http/server.pem
# tlshttpkeyfile = /etc/atuned/http/server.key
# tlshttpcacertfile = /etc/atuned/http/cacert.pem
#################################### log ###############################
# either "debug", "info", "warn", "error", "critical", default is "info"
level = info
#################################### monitor ###############################
[monitor]
# with the module and format of the MPI, the format is {module}_{purpose}
# the module is Either "mem", "net", "cpu", "storage"
# the purpose is "topo"
module = mem_topo, cpu_topo
#################################### system ###############################
# you can add arbitrary key-value here, just like key = value
# you can use the key in the profile
[system]
# the disk to be analysis
disk = sda
# the network to be analysis
network = enp189s0f0
user = root
After the A-Tune is installed, you need to start the A-Tune service.
Start the atuned service.
$ systemctl start atuned
To query the status of the atuned service, run the following command:
$ systemctl status atuned
If the following information is displayed, the service is started successfully:
You can invoke the command line interface (CLI) provided by A-Tune to use A-Tune functions. This chapter describes the functions and usage of the A-Tune CLI.
Query the supported workload types, profiles, and the values of Active.
atune-adm list
$ atune-adm list
Support WorkloadTypes:
+-----------------------------------+------------------------+-----------+
| WorkloadType | ProfileName | Active |
+===================================+========================+===========+
| default | default | true |
+-----------------------------------+------------------------+-----------+
| webserver | ssl_webserver | false |
+-----------------------------------+------------------------+-----------+
| big_database | database | false |
+-----------------------------------+------------------------+-----------+
| big_data | big_data | false |
+-----------------------------------+------------------------+-----------+
| in-memory_computing | in-memory_computing | false |
+-----------------------------------+------------------------+-----------+
| in-memory_database | in-memory_database | false |
+-----------------------------------+------------------------+-----------+
| single_computer_intensive_jobs | compute-intensive | false |
+-----------------------------------+------------------------+-----------+
| communication | rpc_communication | false |
+-----------------------------------+------------------------+-----------+
| idle | default | false |
+-----------------------------------+------------------------+-----------+
If the value of Active is true, the profile is activated. In the example, the profile of the default type is activated.
In addition to the workload types defined in the system, A-Tune also supports user-defined workload types and corresponding profiles, and you can update or delete these workload types.
You can also add the user-defined model to A-Tune. For details about how to train a model, see User-defined Model.
Add a user-defined workload type and the corresponding profile optimization item.
atune-adm define <WORKLOAD_TYPE> <PROFILE_NAME> <PROFILE_PATH>
Add a workload type. Set workload type to test_type, profile name to test_name, and configuration file of an optimization item to example.conf.
$ atune-adm define test_type test_name ./example.conf
The example.conf file can be written as follows (the following optimization items are optional and are for reference only). You can also run the atune-adm info command to view how the existing profile is written.
[main]
# list it's parent profile
[tip]
# the recommended optimization, which should be performed manunaly
[check]
# check the environment
[affinity.irq]
# to change the affinity of irqs
[affinity.task]
# to change the affinity of tasks
[bios]
# to change the bios config
[bootloader.grub2]
# to change the grub2 config
[kernel_config]
# to change the kernel config
[script]
# the script extention of cpi
[sysctl]
# to change the /proc/sys/* config
[sysfs]
# to change the /sys/* config
[systemctl]
# to change the system service config
[ulimit]
# to change the resources limit of user
Update an optimization item of a workload type to the content in the new.conf file.
atune-adm update <WORKLOAD_TYPE> <PROFILE_NAME> <PROFILE_FILE>
Update the workload type to test_type and the optimization item of test_name to new.conf.
$ atune-adm update test_type test_name ./new.conf
Delete a user-defined workload type.
atune-adm undefine <WORKLOAD_TYPE>
Delete the test_type workload type.
$ atune-adm undefine test_type
You can train a new workload type model by running the collection and train commands.
Collect the global resource usage and OS status information during service running, and save the collected information to a CSV output file as the input dataset for model training.
This command depends on the sampling tools such as perf, mpstat, vmstat, iostat, and sar. Currently, only the Kunpeng 920 CPU is supported. You can run the dmidecode -t processor command to check the CPU model.
atune-adm collection <OPTIONS>
OPTIONS
$ atune-adm collection --filename name --interval 5 --duration 1200 --output_path /home --disk sda --network eth0 --workload_type test_type
Use the collected data to train the model. Collect data of at least two workload types during training. Otherwise, an error is reported.
atune-adm train <OPTIONS>
OPTIONS
Use the CSV file in the data directory as the training input. The generated model new-model.m is stored in the model directory.
$ atune-adm train --data_path ./data --output_file ./model/new-model.m
Collect real-time statistics from the system to identify and automatically optimize workload types.
atune-adm analysis [OPTIONS]
Use the default model for classification and identification.
$ atune-adm analysis
Use the user-defined training model for recognition.
$ atune-adm analysis --model ./model/new-model.m
View the profile content of a workload type.
atune-adm info <WORKLOAD_TYPE>
View the profile content of webserver.
$ atune-adm info webserver
*** ssl_webserver:
#
# webserver tuned configuration
#
[main]
#TODO CONFIG
[kernel_config]
#TODO CONFIG
[bios]
#TODO CONFIG
[sysfs]
#TODO CONFIG
[sysctl]
fs.file-max=6553600
fs.suid_dumpable = 1
fs.aio-max-nr = 1048576
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_local_port_range = 1024 65500
net.ipv4.tcp_max_tw_buckets = 5000
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_mem = 362619 483495 725238
net.ipv4.tcp_rmem = 4096 87380 6291456
net.ipv4.tcp_wmem = 4096 16384 4194304
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
[systemctl]
sysmonitor=stop
irqbalance=stop
[bootloader.grub2]
selinux=0
iommu.passthrough=1
[tip]
bind your master process to the CPU near the network = affinity
bind your network interrupt to the CPU that has this network = affinity
relogin into the system to enable limits setting = OS
[script]
openssl_hpre = 0
prefetch = off
[ulimit]
{user}.hard.nofile = 102400
{user}.soft.nofile = 102400
[affinity.task]
#TODO CONFIG
[affinity.irq]
#TODO CONFIG
[check]
#TODO CONFIG
Manually activate a profile of a workload type.
atune-adm profile <WORKLOAD_TYPE>
You can run the list command to query the supported workload types.
Activate the profile configuration of webserver.
$ atune-adm profile webserver
Roll back the current configuration to the initial configuration of the system.
atune-adm rollback
$ atune-adm rollback
Update the system database.
atune-adm upgrade <DB_FILE>
The database is updated to new_sqlite.db.
$ atune-adm upgrade ./new_sqlite.db
Check the CPU, BIOS, OS, and NIC information.
atune-adm check
$ atune-adm check
cpu information:
cpu:0 version: Kunpeng 920-6426 speed: 2600000000 HZ cores: 64
cpu:1 version: Kunpeng 920-6426 speed: 2600000000 HZ cores: 64
system information:
DMIBIOSVersion: 0.59
OSRelease: 4.19.36-vhulk1906.3.0.h356.eulerosv2r8.aarch64
network information:
name: eth0 product: HNS GE/10GE/25GE RDMA Network Controller
name: eth1 product: HNS GE/10GE/25GE Network Controller
name: eth2 product: HNS GE/10GE/25GE RDMA Network Controller
name: eth3 product: HNS GE/10GE/25GE Network Controller
name: eth4 product: HNS GE/10GE/25GE RDMA Network Controller
name: eth5 product: HNS GE/10GE/25GE Network Controller
name: eth6 product: HNS GE/10GE/25GE RDMA Network Controller
name: eth7 product: HNS GE/10GE/25GE Network Controller
name: docker0 product:
A-Tune provides the automatic search capability for optimal configurations, eliminating the need for repeated manual parameter adjustment and performance evaluation. This greatly improves the search efficiency of optimal configurations.
Use the specified project file to search the dynamic space for parameters to find the optimal solution under the current environment configuration.
atune-adm tuning [OPTIONS] <PROJECT_YAML>
Before running the command, ensure that the following conditions are met:
- The YAML configuration file of the server has been edited and placed in the /etc/atuned/tuning/ directory on the server by the server administrator.
- The YAML configuration file of the client has been edited and placed in any directory on the client.
YAML configuration file of the client.
The configuration items of a YAML file on a server are as follows:
|
Maximum number of optimization iterations, which is used to limit the number of iterations on the client. |
|||
Table 1 Description of object configuration item
The configuration items of a YAML file on a client are as follows:
|
Project name, which must be the same as that in the configuration file on the server. |
|||
Table 2 Description of evaluations configuration item
The following is an example of the YAML file configuration on a server:
project: "example"
maxiterations: 10
startworkload: ""
stopworkload: ""
object :
-
name : "vm.swappiness"
info :
desc : "the vm.swappiness"
get : "sysctl -a | grep vm.swappiness"
set : "sysctl -w vm.swappiness=$value"
needrestart: "false"
type : "continuous"
scope :
- 0
- 10
ref : 1
-
name : "irqbalance"
info :
desc : "system irqbalance"
get : "systemctl status irqbalance"
set : "systemctl $value sysmonitor;systemctl $value irqbalance"
needrestart: "false"
type : "discrete"
options:
- "start"
- "stop"
dtype : "string"
ref : "start"
-
name : "net.tcp_min_tso_segs"
info :
desc : "the minimum tso number"
get : "cat /proc/sys/net/ipv4/tcp_min_tso_segs"
set : "echo $value > /proc/sys/net/ipv4/tcp_min_tso_segs"
needrestart: "false"
type : "continuous"
scope:
- 1
- 16
ref : 2
-
name : "prefetcher"
info :
desc : ""
get : "cat /sys/class/misc/prefetch/policy"
set : "echo $value > /sys/class/misc/prefetch/policy"
needrestart: "false"
type : "discrete"
options:
- "0"
- "15"
dtype : "string"
ref : "15"
-
name : "kernel.sched_min_granularity_ns"
info :
desc : "Minimal preemption granularity for CPU-bound tasks"
get : "sysctl kernel.sched_min_granularity_ns"
set : "sysctl -w kernel.sched_min_granularity_ns=$value"
needrestart: "false"
type : "continuous"
scope:
- 5000000
- 50000000
ref : 10000000
-
name : "kernel.sched_latency_ns"
info :
desc : ""
get : "sysctl kernel.sched_latency_ns"
set : "sysctl -w kernel.sched_latency_ns=$value"
needrestart: "false"
type : "continuous"
scope:
- 10000000
- 100000000
ref : 16000000
The following is an example of the YAML file configuration on a client:
project: "example"
iterations : 10
benchmark : "sh /home/Benchmarks/mysql/tunning_mysql.sh"
evaluations :
-
name: "tps"
info:
get: "echo -e '$out' |grep 'transactions:' |awk '{print $3}' | cut -c 2-"
type: "negative"
weight: 100
threshold: 100
$ atune-adm tuning example-client.yaml
$ atune-adm tuning --restore --project example
Table 1 Terminology
|
Workload type, which is used to identify a type of service with the same characteristics. |
|
|
Set of optimization items and optimal parameter configuration. |
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。