Skip to content

Latest commit

 

History

History
181 lines (147 loc) · 7.15 KB

README.md

File metadata and controls

181 lines (147 loc) · 7.15 KB
Raw

SetUp

iLogtail is a configuration-driven collector. Below we will introduce how to configure and start iLogtail.

Quick Start

The iLogtail plugin is implemented based on the Go language, so you need to install the basic Go 1.16+ before proceeding with development Language development environment, how to install it can be found in official document. After the installation is complete, please follow this document Set up your development directory and environment variables such as GOPATH correctly. The rest of this article will assume that you have installed the Go development environment and set GOPATH. Currently, the Logtail plugin supports running on Linux/Windows/macOS. Some plugins may have conditional compilation and only run under Linux or Windows. Please debug in the corresponding environment. More details please see here.

Local start

Execute the make build command in the root directory, and you will get the bin/ilogtail executable file. Use the following command to quickly start the iLogtail program.

# The default startup behavior is to use the metric_mock plugin to mock data and print the data to the logs.
 ./bin/ilogtail --logger-console=true --logger-retain=false

Docker start

Execute the make docker command in the root directory, and you will get the aliyun/ilogtail:latest image. Use the following command to quickly start the docker program. The behavior of the image is the same as the above program. The log output is in /aliyun/logtail_plugin.LOG File.

make docker && docker run aliyun/ilogtail:latest

Configuration

iLogtail currently provides the following 3 modes for configuration settings:

  • Specify the configuration file to start.
  • iLogtail exposes the Http endpoint, allowing configuration changes.
  • The iLogtail-C (open source soon) program uses the program API to make configuration changes.

Specify the configuration file mode to start

After compiling the executable program iLogtail in standalone mode, you can start it by specifying a configuration file (if not specified, it will default use plugin.json in the current directory).

{
  "inputs": [
    {
      "type": "plugin name"
      "detail": {
        "Parameter1": "value"
        "Parameter2": 10
      }
    },
    {
      ...
    }
  ],
  "processors": [
    ...
  ],
  "aggregators": [
    ...
  ],
  "flushers": [
    ...
  ]
}

First, as shown above, the configuration file is a standard JSON file. The four top-level keys correspond to the four types of plugins mentioned in the article:

  • inputs: Input plugin to get data.
  • processors: Processing plugins, to process the data obtained.
  • Aggregators: Aggregation plugin, aggregate data.
  • flushers: output plugin, output data to the specified sink.

It can be seen that their types are array in a configuration, each type of plugin can specify multiple items at the same time, but you need to pay attention to the relationship between multiple plugins of the same type specified at the same time. Except for the synchronous relationship between processors, the others are parallel relationship, as shown in the following figure:

png

For each object in the array, they specify a specific plugin, the type field specifies the name of the plugin, and the object of the detail field specifies the configuration of the plugin.

The following is a very simple example configuration file (plugin.quickstart.json):

  • inputs: Use the metric_mock plugin to generate mock data. By setting the detail, the mock data generated by the two plugins will be different.
  • processors: Use the processor_default plugin. The plugin does not perform any operation on the data, but simply returns.
  • flushers: Use flusher_stdout plugin to output data to a local file. Similarly, we also use detail to output data to two files at the same time.
{
  "inputs": [
    {
      "type": "metric_mock",
      "detail": {
        "Index": 0,
        "Fields": {
          "Content": "quickstart_input_1"
        }
      }
    },
    {
      "type": "metric_mock",
      "detail": {
        "Index": 100000000,
        "Fields": {
          "Content": "quickstart_input_2"
        }
      }
    }
  ],
  "processors": [
    {
      "type": "processor_default"
    }
  ],
  "flushers": [
    {
      "type": "flusher_stdout",
      "detail": {
        "FileName": "quickstart_1.stdout"
      }
    },
    {
      "type": "flusher_stdout",
      "detail": {
        "FileName": "quickstart_2.stdout"
      }
    }
  ]
}

Execute ./bin/ilogtail --plugin=plugin.quickstart.json, after a period of time, use ctrl+c to interrupt the operation. By checking the directory, you will find that two files, quickstart_1.stdout and quickstart_2.stdout, are generated, and their contents are the same.

HTTP API configuration Reload

When iLogtail runs in standalone mode, you can use HTTP API to make configuration file changes. -Port: When iLogtail is running independently, port 18689 is enabled by default for monitoring configuration input. -Interface: /loadconfig

Next, we will use the HTTP mode to re-load the static configuration example in the section Specified configuration file mode startup.

  1. First we start the iLogtail program: ./bin/ilogtail
  2. Use the following command to reload the configuration.
curl 127.0.0.1:18689/loadconfig -X POST -d '[{"project":"e2e-test-project","logstore":"e2e-test-logstore","config_name":"test-case_0","logstore_key":1,"json_str":"{\"inputs\":[{\"type\":\"metric_mock\",\"detail\":{\"Index\":0,\"
Fields\":{\"Content\":\"quickstart_input_1\"}}},{\"type\":\"metric_mock\",\"detail\":{\"Index\":100000000,\"Fields\":{\"Content\":\"quickstart_input_2\"}}}],\"processors\":[{\"type\":\"processor_default\"}],\"flushers\":[{\"type\":\"flusher_stdout\",\"detail\":{\"FileName\":\"quickstart_1.stdout\"}},{\"type\":\"flusher_stdout\",\"detail\":{\"FileName\":\"quickstart_2.stdout\"}}]}\n"}]'
  1. Check the log to observe the configuration change.
2021-11-15 14:38:42 [INF] [logstore_config.go:113] [Start] [logtail_alarm,logtail_alarm]        config start:begin      
2021-11-15 14:38:42 [INF] [logstore_config.go:151] [Start] [logtail_alarm,logtail_alarm]        config start:success    
2021-11-15 14:38:42 [INF] [logstore_config.go:113] [Start] [test-case_0,e2e-test-logstore]      config start:begin      
2021-11-15 14:38:42 [INF] [logstore_config.go:151] [Start] [test-case_0,e2e-test-logstore]      config start:success
  1. By looking at the directory, you will find that the behavior is consistent with the above static configuration mode. Two files, quickstart_1.stdout and quickstart_2.stdout, are generated, and their contents are consistent.

C API Configuration Reload

The part of iLogtail-C will be open source soon, providing the function of compiling the iLogtail GO program in C-shared mode and using it in combination with the C program. API reference plugin_export.go.