Skip to content

Latest commit

 

History

History
73 lines (57 loc) · 2.99 KB

README.md

File metadata and controls

73 lines (57 loc) · 2.99 KB
Raw

bunyan-elasticsearch-bulk

  • A Bunyan stream for saving logs into Elasticsearch.
  • Saves logs in memory instead of sending them to the Elasticsearch server one by one.
  • The logs are accumulated in memory until a certain number of logs reached or a certain amount of time passed without sending any logs.
  • The goal is to save resources on both your application and Elastic Search server (ex: lowering costs in AWS).

Installation

npm i bunyan-elasticsearch-bulk

Usage with Node

const bunyan = require('bunyan')
const createESStream = require('bunyan-elasticsearch-bulk')

const config = {
  name: 'Application Name',
  streams: [{
    level: 'debug',
    stream: createESStream({
      indexPattern: '[my-app-]YYYY.MM.DD',
      node: 'http://localhost:9200'
    })
  }]
}

const log = bunyan.createLogger(config)

// From here, you can log things according to the best practices of Bunyan.
// Please familiarize yourself with it here: https://github.com/trentm/node-bunyan

log.info('Log this message!')
log.info({ otherInfo: 'What else do you need to log here?' }, 'Log this message!')

List of Configuration Parameters

Parameters Specific to Elasticsearch Official Client

These Parameters are passed to the client without any change.

You need to provide the following one. The rest are up to you.

Field Required? Example Description
node or nodes Yes http://localhost:9200 The Elasticsearch endpoint to use. No trailing slash!

Parameters Specific to this Module

Field Default Description
indexPattern [my-app-]YYYY[-]MM[-]DD Pattern for the daily segmentation of log indices. This will be passed to moment.utc().format(indexPattern) from moment.js
type logs Type of the logs for Elasticsearch servers prior to v7.x
limit 100 How many logs to collect before submitting them to ES
interval 5000 Time in milliseconds before submitting logs even if their count has not reached the limit.
client Client(options) If you don't want to use the included version of the client v7.x, you can configure the one you want and pass it here.

Emitted Events

Chech server.js for example usages:

Event Fires when ..
log_received we receive a log
log_submitted we bulk-submit logs to ES server. With a reason limit_exceeded or timeout
error an error happens in the whole bulk submission or when saving an individual message fails

Changes since 1.0.x

For you as a client, you no longer need to use the keyword new with this module. There is a factory function now. Example above is updated.

Development

  • A docker-compose.yaml file has been added to make it easier to develop and test locally.

Author(s)

  • Milad Kawas Cale

License

MIT