- [fix] Upgrades Log4J to avoid zero day attack
- [new] Uses Eclipse Temurin for a Docker base image. (no code had changed)
- [fix] Reduces log level for bad stream name (warn -> info).
- [new] Ignores incoming streams that have invalid name.
- [new] Ignores incoming columns that have invalid name.
- [new] Drops too long JSON rows (>4MB) not to stop Redshift loading (COPY).
- Detects Unix times in milliseconds and converts it as milliseconds, automatically.
- [fix] Configure Sentry stacktrace.app.packaages to supress warning.
- [new] Migrates to new Sentry client: from raven-logback to sentry-logback.
- [new] New column strload_columns.time_unit to support microseconds for unix times (default: seconds)
- [fix] strload_tables record may not exist for the disabled streams or blackhole streams
- [CHANGE] Moves columns strload_stream_bundles.dest_bucket, .dest_prefix to strload_tables (Only one dest prefix per stream).
- Upgrade Java (8 -> 11)
- Upgrade Spring Boot (1.5 -> 2.1)
- Upgrade Spring (4.3 -> 5.1)
cancelled
- [fix] Register metadata op
- [new] New operator "metadata".
- [fix] sequence op: Do not generate overwrapped sequence. (rev.2)
- [fix] fix null pointer exception for bad S3 events
- [fix] Reduces lazy loading.
- [fix] sequence op: Do not generate overwrapped sequence.
- [fix] Reject invalid S3 objects whose object size equals 0.
- [fix] Update chunks records only when it is really changed.
- [fix] preflight: Allows partition_source property for timestamp domains.
- [fix] Load PreprocMessage records first to reduce duplicated rows error.
- [fix] Use Long object type instead of primitive long type because table_id may be null.
- [CHANGE] Splits big preproc_log table into 4 small tables: strload_preproc_messages, strload_preproc_jobs, strload_packets, strload_chunks.
- Records unknown, incoming columns to strload_columns as "unknown" column type. Preprocessor just ignores "unknown" columns. In next version, preprocessor disables a stream which includes "unknown" columns.
- Preflight now handles only preprocessing, it no longer generates .ct or .job files.
- [new]
date
column processor accepts timestamp strings.
- [new]
timezone
column processor accepts more timestamp formats: "2018-03-05T12:34:56+0700" (ISO instant variant), "2018-03-05 12:34:56+0700" (ruby date time variant)
- [new]
timezone
column processor allows to keep source offset as-is
- [new] Introduces stream columns and column-type-based cleansing. This function is enabled by strload_streams.column_initialized flag. This version requires DB schema change.
- [fix] timezone op: support new timestamp format like "2018-02-07 12:34:56"
- [fix] sequence op: Allocates the next sequence block when
sequence
op run out the first sequence block.
- [fix] Catches more exceptions.
- [fix] Catches and reports all exceptions thrown from event handlers.
- [fix] Temporary file name was wrong.
- refactoring only
- [fix] Fixes critial configuration error.
- [new] Adds "blackhole" routing.
- Enhances preflight.