Skip to content
This repository has been archived by the owner on Apr 2, 2024. It is now read-only.

Releases: timescale/promscale

0.8.0

18 Jan 15:58
Compare
Choose a tag to compare

At a high level, this release contains:

  • Headline feature: Tracing data management
  • Renamed CLI flags
  • Add ability to configure chunk interval on startup
  • Promscale PostgreSQL extension packages for various distributions
  • Additional instrumentation and usage data

Headline feature: Trace data management

We've added some features to make the management of tracing data much simpler.

Promscale now supports configurable data retention for trace data. There are two new SQL functions: set_trace_retention_period(INTERVAL) and get_trace_retention_interval(). The default retention is set to 30 days.

We’ve also added a function to easily delete all trace data stored in the database: delete_all_traces().

Read more about how to use these new capabilities in the tracing documentation.

Improvements

Renamed CLI flags

We've worked on reorganizing and regrouping the CLI flags to make them easier to use. Almost all of the existing CLI flags have been changed. The old CLI flags can still be used, but will be removed in a future Promscale release. An overview of the current CLI flags is available in the Promscale repo at docs/cli.md. To get a hint for how to translate old flags to new flags, refer to flags.go which contains a translation map.

Add ability to configure default chunk interval on startup

Promscale stores all metric data points with the same metric name in a TimescaleDB hypertable. Hypertables are broken down into individual chunks for performance. Previously, configuring the chunk interval was only possible via an SQL interface, e.g. SELECT prom_api.set_default_chunk_interval(INTERVAL '1 hour');. This release allows configuring default_chunk_interval using the new Promscale dataset configuration. This is useful to ensure that even if Promscale starts receiving data immediately after it has been started (common in scenarios where the deployment is automated) that the chunk interval is also applied to the first chunk in each hypertable.

Promscale extension packages for various distributions

Promscale provides a PostgreSQL extension that improves the performance of PromQL and SQL queries. For that reason, we strongly recommend that you install the extension even though it is not strictly required.

Installation of the extension on non-Docker systems used to require downloading and installing the Promscale PostgreSQL extension manually. We now have packages available for the most popular Linux distributions making it easier to deploy the Promscale extension directly on hosts.
You can find the installation instructions in the promscale extension repository's README.

Additional instrumentation

We have added new metrics to the /metrics endpoint that provide counters and performance histograms for PromQL queries and queries from the Jaeger and Grafana distributed tracing UIs to retrieve traces:

  • promscale_executed_queries_total
  • promscale_queries_timed_out_total
  • promscale_trace_query_requests_executed_total
  • promscale_trace_fetch_traces_api_execution_duration_seconds
  • promscale_trace_dependency_requests_executed_total

Other

Additional usage data

We have added additional telemetry to the anonymous usage data we collect. We use this data to better understand how the community uses the product and as a result make our products better for our users. You can read more details in our documentation.

Bug fixes

  • helm-charts: use fixed target port on svc-promscale [#1009]
  • Fix passing of async flag [#1008]
  • Remove the event_name_check constraint [#979]
  • Fix _prom_catalog.metric_view() function on non-tsdb installs [#958]

Deprecations

The -migrate cli flag was deprecated, use -startup.only instead.

Breaking Changes

We changed the names of existing metrics which Promscale emits about itself:

  • promscale_sent_samples_total was renamed to promscale_ingested_samples_total
  • promscale_failed_queries_total was renamed to promscale_metrics_queries_failed_total
  • promscale_query_query_duration_seconds was renamed to promscale_metrics_query_duration_seconds
  • promscale_query_batch_duration_seconds was renamed to promscale_metrics_query_remote_read_batch_duration_seconds

Data retention for tracing is added and enabled by default in this release. The retention period is set to 30 days. Trace data older than 30 days will be automatically dropped. If you wish to change this retention period, use the ps_trace.set_span_retention_period(interval) function.

The -promql-enable-feature flag, which was deprecated in 0.7.0 has been removed. Please use -enable-feature instead (see docs/cli.md).

The long-deprecated advisory-lock based leader election feature has been removed. Please use label-based leader election instead.

Thanks

We'd like to thank @aveuiller, @ptou, and @nmaludy for their pull requests, and tilpner for their bug report.

0.7.1

03 Dec 16:12
493efa2
Compare
Choose a tag to compare

This is a bug fix release which fixes issues for users upgrading from 0.6.1 or 0.6.2. If you are on those versions, please skip 0.7.0 and upgrade directly to 0.7.1.

Changelog

  • 578a9ac Fix upgrade scripts
  • e91ad78 Prepare for release 0.7.1
  • f6060a4 Skip running goreleaser tests when releasing a tag

Docker images

  • docker pull timescale/promscale:0.7.1
  • docker pull timescale/promscale:0.7

0.7.0

02 Dec 13:58
36cfee3
Compare
Choose a tag to compare

Note: Users of Promscale 0.6.1 and 0.6.2 should not upgrade to this version. There is an issue with the underlying schema migration which affects those versions. We are working on a fix and will release version 0.7.1 as soon as possible.

At a high level, this release contains:

  • Headline feature: Support for OpenTelemetry tracing data (beta)
  • Support for Postgres 14
  • Helm chart improvements
  • New --enable-feature flag to make exposition of beta features explicit

Headline feature: Support for OpenTelemetry tracing data (beta)

We previously announced the beta release of tracing support, which we made available in the 0.7.0-beta.1 release. While 0.7.0 introduces a number of performance and stability improvements to our tracing support, the feature is still in beta. Therefore we may introduce non-backwards compatible changes in future versions that may require dropping existing trace data in order to upgrade to a new Promscale version. Note that you may also experience stability and performance issues with tracing data.

In this release we make tracing support opt-in: it must be activated with -enable-feature=tracing (more details on -enable-feature below). We are still actively working on improving the feature and its documentation. If you have any suggestions or encounter any issues you can share your feedback in this Github discussion.

Minor changes

-enable-feature

In order to make it clear to our users when they’re using beta functionality, we’ve added the -enable-feature option. When we release new beta functionality, our users will have to explicitly opt-in by enabling the feature. These features are subject to change, and do not have any API stability guarantees. Most importantly, we do not guarantee that there will be no data loss when using these features, or that we will be able to provide a painless pathway from beta to generally-available functionality.

Promscale's beta tracing support is one of the first major promscale features which can be enabled using -enable-feature=tracing. Additionally, Promscale exposes two pieces of beta PromQL functionality, promql-at-modifier, and promql-negative-offset, for more details on those, please consult the Prometheus docs on feature flags.

Note: To retain backwards compatibility, this change does not affect two existing pieces of beta functionality: Continuous Aggregates, and Exemplar Storage. Those features are both still beta, but do not require an explicit opt-in.

Helm chart improvements

We’ve improved the ergonomics of our helm charts, making it easier to get started with Promscale!

  • The helm chart now automatically configures itself, adding the required ServiceAccount, and optionally a ServiceMonitor object to monitor your Promscale instance (which is great if you’re using tobs).
  • We’ve made the deployment simpler by moving all connection configuration parameters into one Secret object.
  • The Promscale pod is now automatically recreated when the connection configuration is updated in values.yaml.
  • It’s now no longer necessary to create a Secret object with database connection details. Instead these parameters can be set directly in values.yaml. We do still provide the option of configuring connection details via a secret, see connectionSecretName.
  • All of these improvements required a break in backwards compatibility. We have restructured the connection parameter in values.yaml, and in doing so we have removed compatibility of the helm charts with TimescaleDB 1.x. If you are updating from a previous version of the helm chart please consult the upgrade notes below.

Support for Postgres 14

Promscale 0.7.0 adds support for Postgres 14.

Note: installations using the promscale extension must have version 0.3.0 of the extension installed for PG14 support. Promscale will automatically run the necessary UPDATE EXTENSION commands when it connects to the database.

Bug Fixes

We fixed a memory leak which arises when using the series endpoint (GET/POST /api/v1/series).

Deprecations

The -promql-enable-feature CLI flag has been deprecated. Its functionality has been merged with -enable-feature. The option is still supported, but will be removed in a future release.

Breaking Changes

Support for environment variables beginning with TS_PROM has been removed. This was previously deprecated in the 0.1.4 release.

The -db-connect-retries cli flag has been removed. Configuring it had no effect, so it can be removed without replacement. If this flag is set, Promscale will abort with the following error: Fatal error: cannot parse flags: configuration error: error parsing commandline args: flag provided but not defined: -db-connect-retries.

In the Promscale helm chart we replaced the service configuration in values.yaml with separate configuration sections for openTelemetry and prometheus. This is to accommodate that Promscale now has both a Prometheus API, and a OTLP gRPC API.

Upgrade Notes

Upgrading helm chart

If you are using TimescaleDB 1.x, we do not recommend upgrading your helm charts to the latest release. In the most recent release we removed the configuration of cron jobs which are necessary for users of TimescaleDB 1.x. TimescaleDB 2.x onwards does not require these maintenance cron jobs. If you are using TimescaleDB 1.x, this is a great opportunity to upgrade to TimescaleDB 2.x :)

When you upgrade to the most recent helm chart be sure to:

  • Rename service.port to prometheus.port in your values.yaml.
  • Rename args to extraArgs in your values.yaml.
  • If you were using the 0.7.0-beta.1 helm charts with tracing.enabled: true, you now need to enable tracing through openTelemetry.enabled: true.
  • If you were either connection.dbURI.secretTemplate, or connection.password.secretTemplate in your values.yaml, you will need to use connectionSecretName and put all connection details into the secret object.
  • Templating the hostname via connection.host.nameTemplate was removed. The host can still be set via connection.host.
  • If your connection.password or connection.uri was a numeric value, be sure to convert it to a string.

Upgrading the Promscale connector binary

Replace the cli flag -promql-enable-feature with -enable-feature.

Replace any environment variables of the form TS_PROM_<CONFIG_ITEM> with PROMSCALE_<CONFIG_ITEM>.

Remove the cli flag -db-connect-retries. It didn’t do anything anyway.

Thanks

We’d like to thank @knrafto for their excellent bug report, allowing us to quickly identify a memory leak in the series endpoints.

Changelog

d856e5d Add insert batching when ingesting trace data
da6f736 Add PromQL view functions
5b6205f Add README for e2e tests
4c8fbe5 Add Trace comparators for HTTP JSON responses.
8db9a38 Add -enable-feature cli flag
a749eb2 Add a bigger trace dataset and some util functions
9586c1d Add benefits
881bd51 Add binary builds to .gitignore
542e255 Add caching to trace ingest path
db7dc75 Add comment to helm chart about -enable-feature=tracing
b4d544d Add downsampling documentation
e03ea5b Add info about docker
21babe1 Add label sorting to JSON metric ingest parser
5754dfb Add link to GH discussion
df00948 Add logging of errors to GRPC server
b305a9c Add note for TimescaleDB 1.x usage to helm chart README
0320441 Add requirements for the continuous aggregate view
d36c12b Add support for PG14
77d0ec1 Add tests for tracing put/get db functions
57f1d72 Add tests to validate the tag operators
524fa0d Add troubleshooting instructions for OTel Collector
6da5f24 Address review feedback
d8b45e9 Avoid ID lookups for operations and tags
b729e0e Batch by the entire trace structure
1d378fb Centralize management of CI env variables
637bd5f Change PromQL metric view to metric view
f7b8c43 Change to cover only alerting
616948e Correctly emit warning log on use of deprecated flag
7ed4d26 Cosmetics
b712e79 Couple of small improvements
183ea56 Create CODEOWNERS
7869378 Enable compression on tracing hypertables
450ccaf Encapsulate Jaeger-container output.
0b01f64 Ensure consistent sequencing of remote commands
453d265 First commit of updates to Downsampling doc
b1bf6c4 Fix 0.5.1 to 0.6.x SQL migration script
22fbd6d Fix GOMAXPROCS for K8s
90ff454 Fix application_name set to be too long
f16b862 Fix broken link
1e77e06 Fix broken link in helm chart README
dde2b56 Fix cache population when initializing metric batcher
555784e Fix clashing function names
95708d1 Fix code nits with trace ingest batching
8f19114 Fix issue when ingesting instrumentation libs with empty schema URLs
c29495c Fix memory leak with SamplesQuerier
0ae88e6 Fix partial upgrades to 0.6.0
b3ec90c Fix setting retention for 2-step aggregation
22c3b6a Fix silly bug in tracegen.py
fa9a529 Fix the import of vfsgen
be1c380 Fix verbose output bug
8ce54b2 Format code according to golang 1.16+ requirements
ec65445 Get rid of deprecated advisory log flags in helm.
adb8334 Handle insert conflicts for schema URL and instrumentation libs
6d3ae69 Handle zero value span IDs as null value in DB
2f5e8e7 Implement closers for Jaeger container.
b90120d Implement storing & reading of Jaeger query response.
2bf33ce Implement the GetDependencies Jaeger API
db9354b Impr...

Read more

0.6.2

12 Oct 20:07
Compare
Choose a tag to compare

This is a bug fix release. This fixes a bug with caching and upgrades from 0.6.0 and with json streaming ingest. It is recommended that users upgrade to avoid problems with caching.

Changelog

072ec1f Fix 0.5.1 to 0.6.x SQL migration script
f26bbfe Fix cache population when initializing metric batcher
ce939ac Fix partial upgrades to 0.6.0
b648bcd Prepare for the 0.6.2 release

Docker images

  • docker pull timescale/promscale:0.6.2
  • docker pull timescale/promscale:0.6

0.7.0-beta.1

08 Oct 06:28
94a511f
Compare
Choose a tag to compare
0.7.0-beta.1 Pre-release
Pre-release

Changelog

5e9f407 Add OTLP GRPC server with trace ingest support
ede8fc5 Add dependabot to sync dependencies daily
42d9f39 Add initial database support for OpenTelemetry Tracing model
f657c5f Add jaeger-proxy in goreleaser
6780d66 Add tests that versioned update scripts are applied transactionally
4c167a5 Adjust queries to match new schema
23ed199 Bump actions/setup-go from 1 to 2.1.4
9ac8a3b Bump reviewdog/action-golangci-lint from 1 to 2.0.1
e415a12 Change test to downgrade image before install
b69122c Configure goreleaser to not publish latest for prerelease
d20db8a Enhance helm-chart to support Promscale trace support
5dc3135 Fix default schema in retention functions
4cb82b3 Fix missing function bodies for date-ish operators
bb24d1e Fix upgrade scripts for 0.6.0
e42aa10 Fix: json-streaming push: do not reuse previous label pairs
8c02658 Ignore distributed chunk ordering in test snapshot comparison
656f38b Ignore duplicate span conflicts on ingest
43dca3e Implement jaeger plugin interface with a proxy
b21eade Implement jaeger query interfaces
335c056 Just a small docs correction about number of slack users
e14c9ec Mark Exemplar API as experimental in docs.
c219ea4 Prepare for the 0.7.0-beta.1 release
9807563 Prepare for the next development cycle
bfc2d9e Refine tracing schema
c0d9809 Temporarily disable problematic tests
07e5dbf Undo jaeger encoding codec
31894ed Unify tag/label matching operators for tracing and metrics
d9dd5aa Update dependencies to fix build and allow dependabot upgrades
8fffd56 Update faq.md
ef79cfe Use TLS cred files for GRPC servers run by Promscale
9d156c0 cmd/jaeger-proxy: Sanitize filepath input
33bf612 fix HA docs with bare metal config details
63a3f39 pkg/jaeger/proxy: Error out on failed connection closure

Docker images

  • docker pull timescale/promscale:0.7.0-beta.1
  • docker pull timescale/promscale:0.7.0-beta.latest

0.6.1

08 Oct 01:42
Compare
Choose a tag to compare

This is a bug fix release. This fixes a bug with upgrade scripts between 0.5.1 and 0.6.0 and with json streaming ingest. It is recommended that anyone who had trouble upgrading to 0.6.0 or anyone using the json ingest endpoint upgrade to this version.

Changelog

a80bd5b Enhance helm-chart to support Promscale trace support
b19a495 Fix default schema in retention functions
e0d749c Fix upgrade scripts for 0.6.0
5e8e56c Fix: json-streaming push: do not reuse previous label pairs
c1dca01 Prepare for the 0.6.1 release
d694fd0 Prepare for the next development cycle
584e95c Temporarily disable problematic tests

Docker images

  • docker pull timescale/promscale:0.6.1
  • docker pull timescale/promscale:0.6

0.5.1

19 Jul 20:48
3a10f7a
Compare
Choose a tag to compare

This is a bug fix release. This fixes a bug with Promscale maintenance tasks and when working with the Apache 2 version of TimescaleDB. It is recommended that anyone experiencing long maintenance task runs upgrade to this version.

Changelog

dfc95b9 Add goreleaser config
110beb8 Add text format to writing to Promscale docs
04ddd3f Fix Promscale on TimescaleDB-OSS.
1eb0e68 Fix bug in execute_data_retention_policy
08261e9 Fix poor performance in maintenance tasks
bc0dd96 Prepare for the 0.5.1 release
ec9a7b5 Prepare for the next development cycle
2a38fa5 Remove promscale extension building from CI process
ee1cde6 Update cli.md
e298ffd Update install locations for binary and systemd service files
06bd9d3 add option to configure namespace in values.yaml
f8deccb fix the wrong metric table name in metric deletion
32f3e6b typo error

Docker images

  • docker pull timescale/promscale:0.5.1
  • docker pull timescale/promscale:0.5
  • docker pull timescale/promscale:latest

0.5.0

06 Jul 20:51
7abf6bd
Compare
Choose a tag to compare

At a high level, this release contains:

  • Performance improvements during ingest and queries
  • Add support for saving and retrieving metric metadata information coming from Prometheus
  • Improve system observability
  • Bug fixes

Performance improvements

This release contains significant improvements for both ingest and queries. During ingest, we improve memory management to reduce pressure on the garbage collector as well as reduce the amount of work that happens when a key is added to a labelset. For queries, we add more efficient fetching of labels, especially in high latency environments, add pushdowns of vector selectors and the rate and increase functions, reduce some pushdown limitations so that more queries can use pushdowns, and improve memory usage.

Observability improvements

  • We added better logging of the achieved throughput of Promscale.
  • New warnings when the system can't keep up with the ingest load
  • Debug level logging of the SQL queries produced by PromQL evaluation.
  • Better visibility into maintenance tasks, including a new verbose_logging mode and updates to the application name as the jobs move along.

Bug fixes

  • Password masking during logging
  • Deadlocks during maintenance tasks

Prom-migrator

This is a beta release for Prom-migrator. It contains a bunch of features that were requested by users, highlights of which includes:

  1. The ability to retry migrating a slab in case of a timeout or an error. The behavior of reader and/or writer component on a timeout or error can be controlled by [retry, skip, abort].
  2. The ability to set custom timeouts.
  3. The ability to increase the time-range of successive slabs being fetched. This can help speed up the migration.
  4. Increase the maximum duration of a slab. Non-Prometheus-based storages, which do not have 2 hours limit on blocks, can benefit from this in case of pushing of slabs.
  5. Better memory control by adding the option to run the garbage collector after each slab is pushed.
  6. Human readable time (RFC3339) for start or end of migration.

Other than these, the release also contains performance improvements related to memory usage.

Breaking CLI changes in prom-migrator

  • -mint is now renamed to -start
  • -maxt is now renamed to -end
  • -concurrent-pulls is now renamed to -concurrent-pull
  • -read-url is now renamed to -reader-url
  • -write-url is now renamed to -writer-url

For more usage/CLI flags details, please see the README.md.

Changelog

e5791cd Add debian packaging to goreleaser
91d90b8 Add debugging data in metrics and logs, refactor throughput watcher.
c39ea1d Add ewma package for rate calculation.
6a10e63 Add integration-tests to verify retrying behaviour on error.
4d26951 Add optional logging for maintenance jobs
07c815d Add support for promscale ext 0.2.x
7bcb492 Add tests for metadata ingestion/querying.
8838af6 Add warning if jobs take too long
1a78024 Adds support for TimescaleDB nighty smoke tests.
ccdc0b1 Adds support for metric-metadata in Promscale.
0aa7f12 Allow auto_explain to be turned on in maintenance jobs
2fe9ab3 Allow pushdown of multiple clauses in a single query
af9cddd Buffer allocation pool for parsing protobuf
1219879 Bunch up view creation
e3e5084 Docs for multi-tenancy.
29a04f8 Finishing refinements for metric-metadata.
ebe7ad8 Fix DB URI issues for URIs with missing ? parameter settings separator
89df3c2 Fix bad check for error in MetricTableName
9f8a5eb Fix label fetching during querying
461312a Fix possible deadlock in delete_expired_series().
c52aa84 Fix typo of README.md
c3d10f5 Implement APIs for metadata support.
66adb11 Improve delete_expired_series
b52954f Log time consumed by SQL queries in debug level & drop HA debug logs noise
1dd2b93 Make prom-migrator runtime more customizable.
aca638f Optimize SQL for pushdowns
189f275 Prepare for 0.5.0 release
6dc3fb1 Prepare for the next development cycle
e2e0d6c Pushdown rate and increase optimizations
081ecb7 Recovery actions and refactoring in prom-migrator.
23360f4 Refactor pushdowns
dd8b1ed Set application_name in maintenance jobs
4bbb867 Stop fetching timestamps for regular series from the DB
1fc1a8e Support for warning user on slow data ingestion.
13e6825 Support vector selector pushhdown
9e4f128 Test fixes
dd46a90 Test fixes
4bab710 Update PromQL with upstream upto May 11.
4483827 Update Prompb with upstream upto May 7.
bc4846c Update docs in prom-migrator.
3217a93 Upgrade to pre-release versions in tests
e59a363 Use more reliable methods of password masking
87b7f2b Use pools for pgx scan objects
cc49108 add fgprof support
41c4ad7 add static deploy.yaml that includes Kubernetes manifests for Promscale
f50c9be filter out indent characters from SQL query log
5c5eb73 move querier methods to NewQuerierPgxConn(), so that these methods will be only invoked for querying calls.
135f3e2 nits + comments + test fixes
9d18464 review iteration 2
ad00081 typo error in spelling

Docker images

  • docker pull timescale/promscale:0.5.0
  • docker pull timescale/promscale:0.5
  • docker pull timescale/promscale:latest

0.4.1

27 May 01:14
Compare
Choose a tag to compare

This is a bug fix release. This fixes a bug with Promscale on TimescaleDB 2.3. It also adds some bug and security fixes. We deem this upgrade low priority: you should only update if you are switching to TimescaleDB 2.3.

Changelog

e8f20f5 Add link to tutorial
ca5c597 Add support to reject samples if they fall on compressed chunks.
636cf4e Fixes to support timescale 2.3.0
2d008fa Improve password masking function
0f7b95b Prepare for the 0.4.1 release
8be47a0 dont panic/segv on no memory cgroup

Docker images

  • docker pull timescale/promscale:0.4.1
  • docker pull timescale/promscale:0.4
  • docker pull timescale/promscale:latest

0.4.0

18 May 22:48
ebdf5ad
Compare
Choose a tag to compare

At a high level, this release contains:

  • Stable version of new HA system (was previously experimental)
  • Support for multi-tenancy
  • Improved PostgreSQL permissions
  • Add text format ingest
  • Improvements to locking behavior

Stable version of new HA system

This release stabilizes our new system for supporting Prometheus running in high-availability mode (the previous release had an experimental version). This new HA system allows for easier deployments by allowing Prometheus and Promscale to scale independently and communicate through a load-balancer.

The new system works as follows: Prometheus is configured to attach external labels named cluster and __replica__ to the samples it sends. Each Prometheus replica of an HA Pair would send the same cluster label but a unique __replica__ label. Promscale would then elect a single Prometheus replica per cluster to write data at any point in time and would automatically failover if that Prometheus replica dies.

When deploying this new HA system, any Prometheus instance can send data to any Promscale instance via a load-balancer or similar mechanism. The correctness of the "replica election" does not depend on any mapping between Prometheus and Promscale nodes and is correct no matter the number of Prometheus or Promscale instances. This new system is enabled on the Promscale side with the CLI flag --high-availability and relies on the Prometheus instance being configured with cluster and __replica__ external labels.

The stabilization process included additional support for late and out-of-order data.

Support for multi-tenancy

Promscale supports multi-tenancy by having different Prometheus servers corresponding to different tenants all write data to the same set of Promscale servers. Each Prometheus server will indicate the tenant it is writing using either custom headers or external labels. Tenant= information will be saved in a special __tenant__ label.

When executing PromQL queries, a Promscale server can be configured to either return data for all tenants or to restrict its answers to a particular tenant or set or tenants. In this way, a tenant can be given access to a particular Promscale PromQL endpoint for
querying its own data while being sure that it could not access other tenant's data.

Improved PostgreSQL permissions

Promscale adopts a Role Based Access Control (RBAC) philosophy to managing permissions at the SQL layer. Many users will choose to simply use a single PostgreSQL user for all Promscale operations. But we also allow users more granular control over permissions management. This release includes significant improvements to this system to allow better control of which permissions to grant different
Promscale instances.

Please see the documentation for more info.

Add text format ingest

Add Prometheus and open-metrics text format support to ingest pipeline. This allows users to push data in this format directly to Promscale.

Improvements to locking behavior

This is a performance enhancement to improve the speed with which new metrics, series, and labels are created.

CLI flag changes

  • Renamed -enable-ha experimental flag to -high-availiability to keep consistent with other modes (e.g. -read-only).
  • Added promql-max-points-per-ts to allow configuration of max points per timeseries when executing PromQL queries.
  • Added -promql-lookback-delta and -promql-max-samples (int64) to allow PromQL evaluation as per the query. For more info, see PromQL evaluation flags.

Prom-migrator

  • Add support for TLS certificates
  • Add support for reading auth info from file
  • Ensure time-ordering of samples when using concurrency (necessary for writing to long-term stores that require time-ordered writes)
  • First non-beta release, reflecting the increased stability of prom-migrator.

Thanks

@niconosenzo for PR #551
@Pizzahd88 for issue #576
@Jontte for issue #582
@andrejpodzimek for issue #572
@mprochowski for PR #602
@stappersg for PR #611

Changelog

1df71c9 -version flag returns exit code 1
a83c617 Add Bors Integration
4f6eb08 Add DB connection handling end-to-end test
c2bca96 Add E2E tests for plain multi-tenancy type.
0bcd7a9 Add Prometheus and open-metrics text format support to ingest pipeline
3f95f75 Add SQL Update script
3769a7f Add backfilling support for HA mode
c5a55f6 Add header in multi-tenant files.
178b935 Add more privilege testing in upgrade snapshot
c1c3173 Add new HA method to docs
319a669 Add pprof mutex profile
218f52e Add unit tests for multi-tenancy.
943e794 Adds support for multi-tenancy in Promscale.
3db9961 Adjust read-only role tests
cb2e867 Change the add-key lock to the series instead of the data table
6eb2a39 Cleanup of SQL permissions
891f40c Compose multi-tenancy types to interfaces.
d1efcb8 Documentation for roles and permissions
47f9dd6 Documents ways to delete data in Promscale.
9433428 E2E tests for token-based multi-tenancy type.
c56267f Extend E2E tests to ensure time-orderedness.
9c6e713 Fix HA bug with last write interval
c763342 Fix cross platform docker
26a5303 Fix delete end-to-end tests
ff4af80 Fix duplicate constant
68f40f2 Fix gaps in graphs when migrating on time-based storages.
4f0d9e9 Fix golden file end-to-end test for latest TSDB version
da39c1d Fix permissions system
c8bd679 Fix prom-migrator version flag
9c2b397 Fix regex anchor
c640aa1 Fix upgrade tests on Mac
0cf42bb Fixes for multinode support
b04dbc5 Further simplify HA Lease and LeaseClient implementation and interface
40cc63a Give prom_admin all permissions from prom_modifier and prom_maintenance.
5e8966f HA service/lease refactor
4bfcbab Implement flag to optionally accept non-mt writes.
7e19b4d Improve locking on metric creation.
bc3ab1b Improve password masking utility
4fa4edb Improve password masking utility
03ea762 In new HA setup read path fails when the read query comes from prometheus as promethues appends replica labelset from external labelset for readRead requests. So dropping the replica labelSet in labelmatchers for read requests, when Promscale is in HA mode.
9a6336d Integrate multi-tenancy with ingestor and querier.
5f2908b Lockdown execute_everywhere
bd4d97f Maintenance also inside docker container
82c5809 Make PromQL engine max-sample and look-back delta customizable.
2425c04 Make per timeseries points limit configurable.
9ca4998 Prepare for the 0.4.0 release
8992b93 Prepare for the next development cycle
ffb68d0 Prom-migrator: Support for TLS certificates & auth from file.
4302d6f Refactor auth in prom-migrator to be passed as config.
338d2e2 Refactor data parsing to use an HA data filter
1843a5a Refactor ingest path
8135579 Refactor priviliges to avoid default privileges
ef10ff2 Refactor snapshot to give better diff output
9b0e6c8 Refactor tenancy to implement preprocessing stage.
a99b028 Refactor write pipeline to support multiple formats and preprocessors
3af97e9 Replace timescale extension schema with a variable
0fb7353 Small fixup for release
769fe6e Split label fetching into txns of < 1000 labels
577b62a Store test logs as GA artifacts
af6453b Switch to session locks for maintenance tasks
8a0b5b1 Tighten function permissions.
9b49a86 Turn off ingestor in read-only mode
ba0b61b Update PromQL upto 27th april.
77815a3 Use small datasets for non-extended unit-tests.
c876f5f add docker-compose file without dependency to build promscale image on setting up Promscale.
80a53a8 change the default log level to info from debug
a96f306 fix ha write logging
4956753 provide option for image pull policy in helm chart

Docker images

  • docker pull timescale/promscale:0.4.0
  • docker pull timescale/promscale:0.4
  • docker pull timescale/promscale:latest