# performance-common **Repository Path**: mirrors_wso2/performance-common ## Basic Information - **Project Name**: performance-common - **Description**: No description available - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2020-08-19 - **Last Updated**: 2026-05-30 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Common Artifacts for Performance Tests --- | Branch | Build Status | | :------ |:------------ | | master | [![Build Status](https://wso2.org/jenkins/buildStatus/icon?job=platform-builds/performance-common)](https://wso2.org/jenkins/job/platform-builds/job/performance-common/) | --- This repository has common artifacts to be used for Performance Tests. In [components](components), there are several Java projects and each project builds an executable JAR file. The [distribution](distribution) directory has the scripts and the Maven project to build the final distribution package including all scripts and components to be used for performance tests. The package (**performance-common-distribution-${version}.tar.gz**) built by the distribution maven module can be used as a dependency to write performance testing scripts. This package only provides helper scripts and applications. You must extend the functionality of these scripts to run performance tests. It's recommended to include the contents of this package with any scripts written to extend the functionality. ## Package contents Following is the tree view of the contents inside distribution package. ```console ubuntu@server:~$ tree --charset=ascii | sed -e 's/[0-9]\.[0-9]\.[0-9].*\.jar/${version}.jar/g' . |-- cloudformation | |-- cloudformation-common.sh | |-- create-template.py | |-- download-logs.sh | |-- get-wum-updated-wso2-product.sh | |-- python-requirements.txt | `-- templates | `-- common_perf_test_cfn.yaml |-- common | `-- common.sh |-- java | `-- install-java.sh |-- jmeter | |-- create-summary-csv.sh | |-- create-summary-markdown.py | |-- csv-to-markdown-converter.py | |-- install-jmeter.sh | |-- jmeter-server-start.sh | |-- perf-test-common.sh | `-- user.properties |-- jtl-splitter | |-- jtl-splitter-${version}.jar | `-- jtl-splitter.sh |-- netty-service | |-- netty-http-echo-service-${version}.jar | `-- netty-start.sh |-- payloads | |-- generate-payloads.sh | `-- payload-generator-${version}.jar |-- sar | `-- install-sar.sh `-- setup |-- setup-common.sh |-- setup-jmeter-client.sh |-- setup-jmeter.sh `-- setup-netty.sh 10 directories, 26 files ``` Each directory has one or more executable scripts. All scripts support `-h` (help) option. **Note:** Most of these scripts will work only on Debian based systems like Ubuntu. See following sections for more details. ### AWS CloudFormation The performance test scripts can create AWS resources to run standard server performance tests on AWS. Standard server performance tests include a Back-end service and the parameters include "Application heap memory sizes", "Concurrent users", "Message sizes", and "Back-end Sleep Times". The `create-template.py` script can create AWS CloudFormation template based on a `Jinja2` template. The `common_perf_test_cfn.yaml` file is a common `Jinja2` template, which has necessary configurations to create a performance test environment on AWS. You should extend `cloudformation-common.sh` to run standard server performance tests. By extending, you can will get all the functionality of creating the AWS stack, run the tests, delete stack and summarize results. You can even run several stacks to run performance tests parallely. The `cloudformation-common.sh` script is not directly executable, but it supports following options. ```console ubuntu@server:~$ ./cloudformation/cloudformation-common.sh -h Usage: ./cloudformation/cloudformation-common.sh -f [-d ] -k -n -j -o -g -s -b -r -J -S -N [-t ] [-p ] [-w ] [-h] -- [run_performance_tests_options] -f: Distribution containing the scripts to run performance tests. -d: The results directory. Default value is a directory with current time. For example, results-20190124094422. -k: Amazon EC2 Key File. Amazon EC2 Key Name must match with this file name. -n: Amazon EC2 Key Name. -j: Apache JMeter (tgz) distribution. -o: Oracle JDK distribution. -g: Path of GCViewer Jar file, which will be used to analyze GC logs. -s: The Amazon CloudFormation Stack Name Prefix. -b: Amazon S3 Bucket Name. -r: Amazon S3 Bucket Region. -J: Amazon EC2 Instance Type for JMeter Client. -S: Amazon EC2 Instance Type for JMeter Server. -N: Amazon EC2 Instance Type for Netty (Back-end) Service. -t: Number of stacks to create. Default: 1. -p: Parameter option of the test script, which will be used to run tests in parallel. Default: u. Allowed option characters: ubsm. -w: The minimum time to wait in minutes before polling for cloudformation stack's CREATE_COMPLETE status. Default: 5. -h: Display this help and exit. ``` When running the performance tests, you can use `get-wum-updated-wso2-product.sh` script to get a WSO2 product with all WUM updates. ### Java Use the `install-java.sh` script to install Oracle Java Development Kit (JDK) on 64bit Linux. The `install-java.sh` script in this directory will not be useful when OpenJDK is used. It's recommended to use the default package repositories to install OpenJDK. Currently `install-java.sh` script supports installing Oracle JDK 8. You must download latest [JDK 8](http://www.oracle.com/technetwork/java/javase/downloads/index.html). This script can also install Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy files. You need to copy JCE Policy zip file to the same location as the downloaded JDK file (tar.gz) The script needs to be run as root. The JDK will be extracted to `/usr/lib/jvm` directory. ```console ubuntu@server:~$ ./java/install-java.sh -h Usage: ./java/install-java.sh -f [-p ] [-u ] [-h] -f: The jdk tar.gz file. -p: Java installation directory. -u: Target user. Default: ubuntu. -h: Display this help and exit. ``` ### JMeter #### Installing JMeter The `install-jmeter.sh` script will extract JMeter, install Plugin Manager and copy the `user.properties` file. The `user.properties` file has recommended configurations for performance tests. There is an option to download latest [Apache JMeter](http://jmeter.apache.org/download_jmeter.cgi). ```console ubuntu@server:~$ ./jmeter/install-jmeter.sh -h Usage: ./jmeter/install-jmeter.sh -i [-f ] [-d] [-p ] [-h] -i: Apache JMeter installation directory. -f: Apache JMeter tgz distribution. -d: Download Apache JMeter from web. -p: The name of the JMeter Plugin to install. You can provide multiple names. -h: Display this help and exit. ``` #### Running standard server performance tests. You shoud extend `perf-test-common.sh` to run standard server performance tests. It supports testing with multiple concurrent users, different message sizes, different Back-end service delays and different heap memory sizes of the target server. The script also supports running remote (distributed) JMeter tests and it will also summarize the results for warmup and measurement periods. A script can extend this `perf-test-common.sh` script by simply sourcing the `perf-test-common.sh` script. For example: ```bash # Execute common script . $script_dir/perf-test-common.sh ``` Any script depending on this script must define test scenarios as follows: ```bash declare -A test_scenario0=( [name]="test_scenario_name1" [display_name]="Test Scenario 1" [description]="Description of Test Scenario 1" [jmx]="test_scenario_name1.jmx" [use_backend]=true [skip]=false ) declare -A test_scenario1=( [name]="test_scenario_name2" [display_name]="Test Scenario 2" [description]="Description of Test Scenario 2" [jmx]="test_scenario_name2.jmx" [use_backend]=true [skip]=false ) ``` Then define following functions in the script. 1. `initialize` 2. `before_execute_test_scenario` 3. `after_execute_test_scenario` In above functions, following variables may be used 1. `scenario_name` 2. `heap` 3. `users` 4. `msize` 5. `sleep_time` 6. `report_location` Use `jmeter_params` array in `before_execute_test_scenario` to provide JMeter parameters. In before_execute_test_scenario `JMETER_JVM_ARGS` variable can be set to provide additional JVM arguments to JMeter. Finally, execute test scenarios using the function `test_scenarios`. ```console ubuntu@server:~$ ./jmeter/perf-test-common.sh -h Usage: ./jmeter/perf-test-common.sh -m -u -b -s [-d ] [-w ] [-n ] [-j ] [-k ] [-l ] [-i ] [-e ] [-t] [-p ] [-h] -m: Application heap memory sizes. You can give multiple options to specify multiple heap memory sizes. Allowed suffixes: M, G. -u: Concurrent Users to test. You can give multiple options to specify multiple users. -b: Message sizes in bytes. You can give multiple options to specify multiple message sizes. -s: Back-end Sleep Times in milliseconds. You can give multiple options to specify multiple sleep times. -d: Test Duration in seconds. Default 900. -w: Warm-up time in seconds. Default 300. -n: Number of JMeter servers. If n=1, only client will be used. If n > 1, remote JMeter servers will be used. Default 1. -j: Heap Size of JMeter Server. Allowed suffixes: M, G. Default 4G. -k: Heap Size of JMeter Client. Allowed suffixes: M, G. Default 2G. -l: Heap Size of Netty Service. Allowed suffixes: M, G. Default 4G. -i: Scenario name to to be included. You can give multiple options to filter scenarios. -e: Scenario name to to be excluded. You can give multiple options to filter scenarios. -t: Estimate time without executing tests. -p: Estimated processing time in between tests in seconds. Default 60. -h: Display this help and exit. ``` #### Creating a summary Use `create-summary-csv.sh` to create a summary CSV file. ```console ubuntu@server:~$ ./jmeter/create-summary-csv.sh -h Usage: ./jmeter/create-summary-csv.sh -n [-c ] [-r ] [-x] [-p ] [-g ] [-d ] [-j ] [-k ] [-w] [-i] [-l] [-h] -n: Name of the application to be used in column headers. -c: Column header name for each parameter. You should give multiple header names in order for each directory in the results directory structure. Default: Heap Size,Concurrent Users,Message Size (Bytes),Back-end Service Delay (ms) -r: Regular expression with a single group to extract parameter value from directory name. You should give multiple regular expressions in order for each directory in the results directory structure. Default: ([0-9]+[a-zA-Z])_heap,([0-9]+)_users,([0-9]+)B,([0-9]+)ms_sleep -x: Print column names and exit. -p: Prefix of the files to get metrics (Load Average, GC, etc). -g: Path of GCViewer Jar file, which will be used to analyze GC logs. -d: Results directory. Default ./jmeter/results. -j: Number of JMeter servers. If n=1, only client was used. If n > 1, remote JMeter servers were used. Default 1. -k: Number of Application instances. Default 1. -w: Use warmup results instead of measurement results. -i: Include GC statistics and load averages for other servers. -l: Exclude Netty Back-end Service statistics. Works with -i. -h: Display this help and exit. ``` Use `csv-to-markdown-converter.py` to convert CSV results into Markdown format. ```console ubuntu@server:~$ ./jmeter/csv-to-markdown-converter.py Usage: {Input File(.csv)} {Output File (.md)} ``` Use `create-summary-markdown.py` to create a markdown file, which will be a summary of performance test results. This scripts looks for a `Jinja2` template with the name `summary.md` in `templates` directory. ```console ubuntu@server:~$ ./jmeter/create-summary-markdown.py -h usage: create-summary-markdown.py [-h] --json-files JSON_FILES [JSON_FILES ...] --column-names COLUMN_NAMES [COLUMN_NAMES ...] Create summary report optional arguments: -h, --help show this help message and exit --json-files JSON_FILES [JSON_FILES ...] JSON files with parameters. --column-names COLUMN_NAMES [COLUMN_NAMES ...] Columns to include in the report. ``` ### JTL Splitter The "jtl-splitter" directory has a Java program to split a single JTL file into warmup and measurement based on the number of minutes given as the warmup time. When reporting the results for the performance tests, some specified number of minutes from the beginning of the test are considered as the "Java Warm-up Time" and the from the final results, the warm-up duration is excluded. By doing this, the results reported from the test will only consider the steady-state of the server. This program should be invoked by the performance testing script after completing the JMeter performance test. For example if you specify 5 minutes warmup-time, the JTL splitter splits the `results.jtl` file and the `results-warmup.jtl` file will have the test results for first 5 minutes. The results after 5 minutes will be in `results-measurement.jtl`. ```console ubuntu@server:~$ ./jtl-splitter/jtl-splitter.sh -h Usage: ./jtl-splitter/jtl-splitter.sh [-m ] [-h] -- [jtl_splitter_flags] -m: The heap memory size. Default: 1g -h: Display this help and exit. ``` JTL Splitter usage: ```console ubuntu@server:~$ ./jtl-splitter/jtl-splitter.sh -- -h Usage: JTLSplitter [options] Options: -d, --delete-jtl-file-on-exit Delete JTL File on exit Default: false -h, --help Display Help * -f, --jtlfile JTL File -n, --precision Precision to use in statistics Default: 2 -p, --progress Show progress Default: false -s, --summarize Summarize results Default: false -u, --time-unit Time Unit Default: MINUTES Possible Values: [NANOSECONDS, MICROSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS, DAYS] * -t, --warmup-time Warmup Time Default: 0 ``` ### Netty Service The "netty-service" directory has a simple Netty HTTP Echo Service, which will echo back the body data in the HTTP request. The Netty HTTP Echo Service should be started by the performance testing script. ```console ubuntu@server:~$ ./netty-service/netty-start.sh -h Usage: ./netty-service/netty-start.sh [-m ] [-h] -- [netty_service_flags] -m: The heap memory size of Netty Service. Default: 4g -h: Display this help and exit. ``` The script also accepts an argument to specify the number of milliseconds to sleep before sending response. This is useful to test the performance with delays. ```console ubuntu@server:~$ ./netty-service/netty-start.sh -- -h Starting Netty Usage: EchoHttpServer [options] Options: --boss-threads Boss Threads Default: 4 --delay Response delay in milliseconds Default: 0 -h, --help Display Help --http2 Use HTTP/2 protocol instead of HTTP/1.1 Default: false --key-store-file Keystore file --key-store-password Keystore password Default: --port Server Port Default: 8688 --ssl Enable SSL Default: false --worker-threads Worker Threads Default: 8 --max-content-length Max HTTP content length in bytes Default: 1048576 (i.e. 1MiB) ``` ### Payloads The "payloads" directory has a Java program to generate JSON payloads with different sizes. By default, the script generates 50B, 1KiB, 10KiB, and 100KiB JSON files. If you want to generate different payload sizes, pass the payload sizes as parameters. The performance testing script can call this script to generate payloads required for the performance test. ```console ubuntu@server:~$ ./payloads/generate-payloads.sh -h Usage: ./payloads/generate-payloads.sh [-p ] [-s ] -p: The Payload Type. -s: The Payload Size. You can give multiple payload sizes. -h: Display this help and exit. ``` ### SAR The "sar" directory has a simple script to install System Activity Report (SAR) in Linux and configure it to run every one minute. The script needs to be run as root. ```console ubuntu@server:~$ sudo ./sar/install-sar.sh -h Usage: ./sar/install-sar.sh [-h] -h: Display this help and exit. ``` ### Setup scripts The "setup" directory has the scripts to setup instances (for example, JMeter Client, JMeter Server, Netty Server, etc.) #### Common Setup Script The `setup-common.sh` script is used by all other setup scripts to do some common operations. ```console ubuntu@server:~$ sudo ./setup/setup-common.sh -h Usage: ./setup/setup-common.sh [-g] [-p ] [-w ] [-o ] -g: Upgrade distribution -p: Package to install. You can give multiple -p options. -w: Download URLs. You can give multiple URLs to download. -o: Output name of the downloaded file. You can give multiple names for a given set of URLs. -h: Display this help and exit. ``` #### Setup JMeter The `setup-jmeter.sh` installs JMeter and JMeter plugins. This script uses the `install-jmeter.sh` script in "jmeter" directory. ```console ubuntu@server:~$ sudo ./setup/setup-jmeter.sh -h Usage: ./setup/setup-jmeter.sh -i [-j ] [-g] [-p ] [-w ] [-o ] -g: Upgrade distribution -p: Package to install. You can give multiple -p options. -w: Download URLs. You can give multiple URLs to download. -o: Output name of the downloaded file. You can give multiple names for a given set of URLs. -i: The JMeter installation directory. -j: The JMeter plugin name. You can give multiple JMeter plugins to install. -h: Display this help and exit. ``` #### Setup JMeter Client The `setup-jmeter-client.sh` uses `setup-jmeter.sh` internally. It also creates the SSH configurations to execute commands in other instances. ```console ubuntu@server:~$ sudo ./setup/setup-jmeter-client.sh -h Usage: ./setup/setup-jmeter-client.sh -k -i -c -a -n [-j ] [-g] [-p ] [-w ] [-o ] -g: Upgrade distribution -p: Package to install. You can give multiple -p options. -w: Download URLs. You can give multiple URLs to download. -o: Output name of the downloaded file. You can give multiple names for a given set of URLs. -k: The key file location. -i: The JMeter installation directory. -c: The SSH config location. -a: SSH Alias. You can give multiple ssh aliases. -n: SSH Hostname. You can give multiple ssh hostnames for a given set of ssh aliases. -j: The JMeter plugin name. You can give multiple JMeter plugins to install. -h: Display this help and exit. ``` #### Setup Netty Server The `setup-netty.sh` installs Netty Server and OpenJDK ```console ubuntu@server:~$ sudo ./setup/setup-netty.sh -h Usage: ./setup/setup-netty.sh [-g] [-p ] [-w ] [-o ] -g: Upgrade distribution -p: Package to install. You can give multiple -p options. -w: Download URLs. You can give multiple URLs to download. -o: Output name of the downloaded file. You can give multiple names for a given set of URLs. -h: Display this help and exit. ``` ## License Copyright 2017 WSO2 Inc. (http://wso2.com) Licensed under the Apache License, Version 2.0