Skip to content

yandex-cloud/yc-github-loadtesting-ci

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GitHub Actions for Yandex Cloud Load Testing service

USAGE EXAMPLE

USAGE EXAMPLE WORKFLOW

NOTE: All actions require auth-key-json to authorize requests to Yandex Cloud services. Please check this documentation page for instructions how to create one.

Short reference

  • test-suite - execute multiple load tests sequentially
  • agents-create - create and start load testing agents using Yandex Cloud Compute VMs as a hosting (Compute is billed separately)
  • agents-delete - delete load testing agents
  • test-single-run - execute a single load test
  • test-single-check - check results of a single load test

Action: test-suite

uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main

Execute and assess a bunch of tests in a single action step.

Each test is specified by a separate directory containing:

  • test-config.yaml test configuration file
  • (optional) meta.json json file specifying test properties
  • (optional) additional test data files which will be transferred to the agent via data-bucket

A list of directories containing test-config.yaml test configuration files should be provided via test-directories input parameter.

NOTE: Why data-bucket is needed? Test data files often contain sensitive information such as access tokens and customer data. Using data-bucket as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

Inputs

Input Description Default Required
folder-id ID of a folder in Yandex Cloud. n/a yes
auth-key-json-base64

BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.

The action will perform all operations on behalf of a service account for which this authorized key was generated.

Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.

n/a no
auth-key-json

Use 'auth-key-json-base64'.

An authorized key string in JSON format.

The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).

`` no
action-log-level Action log level. NOTICE no
data-bucket

Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent.

WARNING:
CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket

WARNING2:
agent service account must be able to download data from this bucket.

`` no
artifacts-bucket

Bucket to store artifacts generated by agent during test execution.

WARNING:
agent service account must be able to upload data to this bucket.

`` no
test-directories A whitespace-separated list of directory paths containing test configuration files. n/a yes
agent-filter

A filter expression to select agents to execute tests.

Example:
- 'name contains github' - agents with 'github' substring in name.
- 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID

labels.workflow=${{ github.run_id }} no
add-description Specified string will be added to description of created tests. Run from GitHub Actions no
add-labels Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'. ci=github no

Outputs

Output Description
test-ids IDs of performed tests.
test-infos-file File containing a JSON array of objects with information about performed tests.
test-infos JSON array of objects with information about performed tests.
execution-report-file Path to generated .md execution report file.
checks-report-file Path to generated .md checks report file.
artifacts-dir Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

  • loadtesting.loadTester - to create and run the test
  • storage.uploader - to upload test data to Object Storage (required, if data-bucket is specified)

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directories: |-
      _impl/testdata/http-const-rps
      _impl/testdata/https-const-rps

Action: agents-create

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

Create agents alongside with Cloud Compute VMs to host them.

This action is usually used used in pair with 'agents-delete' action.

NOTE: For security reasons you should create new service account with 'loadtesting.generatorClient' role to assign to loadtesting agent (see service-account-id input below)

Inputs

Input Description Default Required
folder-id ID of a folder in Yandex Cloud. n/a yes
auth-key-json-base64

BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.

The action will perform all operations on behalf of a service account for which this authorized key was generated.

Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.

n/a no
auth-key-json

Use 'auth-key-json-base64'.

An authorized key string in JSON format.

The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).

`` no
action-log-level Action log level. NOTICE no
count Number of agents to be created. 1 no
service-account-id

ID of a service account to create agent VM with.

The service account must have 'loadtesting.generatorClient' role.

`` no
vm-zone Compute zone to create agent VM in. ru-central1-d no
name-prefix

If count=0, a name of created agent.
if count>0, a name prefix of created agents.

onetime-ci-agent no
description Description of created agents. Create from Github Actions no
labels Labels of created agents. Format: 'key1=value1,key2=value2'. workflow=${{ github.run_id }} no
cli-args

Additional command line arguments to be passed to yc loadtesting agent create
(see yc loadtesting agent create -h for more details).

Override rules:
* '--zone ARG' overrides 'vm-zone' input parameter
* '--service-account-id ARG' overrides 'service-account-id' input parameter
* '--network-interface' overrides default network interface settings (which is - 1-to-1 NAT (dynamic public IP) in automatically chosen subnet)
* '--name ARG' overrides 'name-prefix' input parameter
* '--description ARG' overrides 'description' input parameter
* '--labels' overrides 'labels' input parameter

`` no
timeout

Time to wait for agents to become READY_FOR_TEST.

Usually, reaching this timeout means either missing permissions or invalid agent network settings.

600 no

Outputs

Output Description
agent-ids IDs of created agents.
artifacts-dir Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

  • loadtesting.loadTester - to create an agent
  • compute.editor - to create and start agent VM
  • iam.serviceAccounts.user - to assign service account to agent VM
  • vpc.user - to configure network interface on agent VM
  • vpc.publicAdmin - to configure public IP on agent VM (the default way, may be omitted if you know what you are doing)

Examples

Simple:

# - 2 CPU, 2 GB RAM
# - one-to-one NAT in an automatically selected subnet

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
  labels: "workflow=${{ github.run_id }}"
  # count: 1
  # vm-zone: ru-central1-a
Custom CPU and RAM settings:
uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
  labels: "workflow=${{ github.run_id }}"
  cli-args: |-
    --cores 2
    --memory 2G
  # count: 1
  # vm-zone: ru-central1-a
Custom network settings:

This version is essentially identical to yc loadtesting agent create ${cli-args}.

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  cli-args: |-
    --service-account-id "${{ vars.YC_LOADTESTING_SA_ID }}"
    --labels "workflow=${{ github.run_id }}"
    --cores 2
    --memory 2G
    --zone 'ru-central1-a'
    --network-settings "subnet-name=default-a,security-group-ids=${{ vars.YC_LOADTESTING_AGENT_SECURITY_GROUP_ID }}"
  # count: 1

Action: agents-delete

uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main

Delete agents (and the VMs).

This action is usually used used in pair with 'agents-create' action.

Inputs

Input Description Default Required
folder-id ID of a folder in Yandex Cloud. n/a yes
auth-key-json-base64

BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.

The action will perform all operations on behalf of a service account for which this authorized key was generated.

Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.

n/a no
auth-key-json

Use 'auth-key-json-base64'.

An authorized key string in JSON format.

The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).

`` no
action-log-level Action log level. NOTICE no
agent-ids

A string containing whitespace-separated list of agents to be deleted.

Normally, should be used with conjunction with agents-create.outputs.agent-ids.

n/a yes

Outputs

Output Description
artifacts-dir Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

  • loadtesting.loadTester - to delete an agent
  • compute.editor - to stop and delete agent VM

Example

This action is usually used in pair with agents-create action:

loadtesting:
  name: loadtesting job
  runs-on: ubuntu-latest
  steps:
    - id: create-agents
      uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
      with:
        folder-id: ${{ vars.YC_FOLDER_ID }}
        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
        service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
    #
    # here we run some tests on created agents
    # - id: run-test
    #   ...
    #
    - id: delete-agents
      if: always() # make sure delete is called even if previous steps fail
      uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main
      with:
        folder-id: ${{ vars.YC_FOLDER_ID }}
        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

        # pass created agent ids to make sure they are deleted
        agent-ids: ${{ steps.agent-create.outputs.agent-ids }}

Action: test-single-run

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-run@main

Execute a single test.

Each test should be specified by a separate directory containing:

  • test-config.yaml test configuration file
  • (optional) meta.json json file specifying test properties
  • (optional) additional test data files which will be transferred to the agent via data-bucket

A directory containing test-config.yaml test configuration file should be provided via test-directory input parameter.

NOTE: Why data-bucket is needed? Test data files often contain sensitive information such as access tokens and customer data. Using data-bucket as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

Inputs

Input Description Default Required
folder-id ID of a folder in Yandex Cloud. n/a yes
auth-key-json-base64

BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.

The action will perform all operations on behalf of a service account for which this authorized key was generated.

Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.

n/a no
auth-key-json

Use 'auth-key-json-base64'.

An authorized key string in JSON format.

The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).

`` no
action-log-level Action log level. NOTICE no
data-bucket

Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent.

WARNING:
CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket

WARNING2:
agent service account must be able to download data from this bucket.

`` no
artifacts-bucket

Bucket to store artifacts generated by agent during test execution.

WARNING:
agent service account must be able to upload data to this bucket.

`` no
test-directory A path to a directory containing test configuration files. n/a yes
agent-filter

A filter expression to select agents to execute tests.

Example:
- 'name contains github' - agents with 'github' substring in name.
- 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID

labels.workflow=${{ github.run_id }} no
add-description Specified string will be added to description of created tests. Run from GitHub Actions no
add-labels Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'. ci=github no

Outputs

Output Description
test-id ID of performed test.
test-info-file File containing a JSON object with information about performed test.
test-info JSON object with information about performed test.
report-file Path to generated .md report file.
artifacts-dir Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

  • loadtesting.loadTester - to create and run the test
  • storage.uploader - to upload test data to Object Storage (required, if data-bucket is specified)

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directory: _impl/testdata/https-const-rps

Action: test-single-check

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-check@main

Assess results measured in a single test run.

The assessment is performed via custom scripts (check_summary.sh and check_report.sh) which can be added to test directory:

TEST_ID="bb4asdfjljbhh" # some test id
TEST_DIR="_impl/testdata/http-const-rps" # test-directory

yc --format json loadtesting test get $TEST_ID > summary.json
yc --format json loadtesting test get-report-tables $TEST_ID > report.json

set -e
$DIR/check_summary.sh summary.json
$DIR/check_report.sh report.json

If check_report.sh or check_summary.sh are not found in test directory, some basic checks will be performed instead.

Inputs

Input Description Default Required
folder-id ID of a folder in Yandex Cloud. n/a yes
auth-key-json-base64

BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.

The action will perform all operations on behalf of a service account for which this authorized key was generated.

Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.

n/a no
auth-key-json

Use 'auth-key-json-base64'.

An authorized key string in JSON format.

The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).

`` no
action-log-level Action log level. NOTICE no
test-id ID of a test to be checked. n/a yes
test-directory Directory with test configuration files. n/a yes

Outputs

Output Description
raw-output Raw stdout of a check script.
report-file Path to generated .md report file.
artifacts-dir Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

  • loadtesting.loadTester - to retrieve information about the test

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main
  id: run-test
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directory: _impl/testdata/https-const-rps
- uses: yandex-cloud/yc-github-loadtesting-ci/test-check-single@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-id: ${{ steps.run-test.outputs.test-id }}
    test-directory: _impl/testdata/https-const-rps