Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

usage-based reporting & billing v2 #1123

Merged
merged 4 commits into from
Jul 25, 2023
Merged

usage-based reporting & billing v2 #1123

merged 4 commits into from
Jul 25, 2023

Conversation

jgraettinger
Copy link
Member

@jgraettinger jgraettinger commented Jul 21, 2023

Introduce interval stats for measuring fine-grain task usage. Update our reporting to roll them up, and introduce a new billing report which implements our updated pricing logic.

This PR implements all idealized changes for how reporting and billing will work.
I still need to work up a production DB migration, but I'd like to review first.

Workflow steps:

This PR adds a new billing_report_202308 (post-fix encodes the effective date-month for the report).
We'll need to transition the UI over to calling it.

Checkout out billing.test.sql for elaborate examples of what the output report looks like.
Here's an example from my local stack:

curl -s -H "Authorization: Bearer ..." "http://localhost:5431/rest/v1/rpc/billing_report_202308?billed_prefix=johnnyCo/&billed_month=2023-07-01T00:00:00Z" | jq

{
  "subtotal": 50,
  "line_items": [
    {
      "rate": 50,
      "count": 0.7067732894793153,
      "subtotal": 35,
      "description": "Data processing (first 1024GB at $0.50/GB)"
    },
    {
      "rate": 20,
      "count": 0,
      "subtotal": 0,
      "description": "Data processing (at $0.20/GB)"
    },
    {
      "rate": 14,
      "count": 1.0458333333333334,
      "subtotal": 15,
      "description": "Task usage (at $0.14/hour)"
    }
  ],
  "billed_month": "2023-07-01T00:00:00+00:00",
  "billed_prefix": "johnnyCo/",
  "recurring_fee": 0,
  "task_usage_hours": 1.0458333333333334,
  "processed_data_gb": 0.7067732894793153
}

Documentation links affected:

(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)

Notes for reviewers:

Please review commit-by-commit.

Issue #1122


This change is Reviewable

Copy link
Member

@williamhbaker williamhbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Add a new ops.Stats.Interval sub-message, which is written out at fixed
wall-time intervals (every 3 minutes, currently) for each task.

Interval models task uptime and a "usage rate" concept.
In the future it could also model current disk usage and other sampled metrics.

Issue #1122
The L1 reporting derivation calculates a `usageSeconds` statsSummary metric
as the product of `uptimeSeconds` and `usageRate`.

`uptimeSeconds` and `usageSeconds` are aggregated at each of our
reporting grains, and `usage_seconds` is a materialized column of our
`catalog_stats` table.

Issue #1122
* Add `usage_seconds` as a `catalog_stats` column, and partition `catalog_stats` on grain.

* Add `data_tiers`, `usage_tiers`, and `recurring_usd_cents` to
  `tenants` table.

* Add an internal `billing_adjustments` table.

* Add a `billing_report_202308` RPC which implements the new billing
  logic, with accompanying tests.

Issue #1122
@jgraettinger
Copy link
Member Author

jgraettinger commented Jul 24, 2023

After a conversation with @psFried on the future of control-plane DB migrations, I've re-worked the supabase changes into a more traditional DB migration (alter existing tables, rather than derive a separate diff). That's particularly important here because I've got it re-building the catalog_stats table to make it partitioned (on grain).

I've locally tested the migration and release of this feature, starting with running tasks and a populated catalog_stats table, and developed the following release run-book:

  • Run DB migration first (e.g. psql postgresql://postgres:postgres@localhost:5432/postgres --file ./supabase/migrations/21_billing_v2.sql).
    • The migration is wrapped in a transaction to fully roll back if it doesn't apply.
    • Tested that post-migration tables are still compatible with the prior agent, runtime, UI, and ops-catalog. Everything continues to function 👍
  • Publish updated ops catalog (through Tilt when local, or flowctl into prod).
    • This will aggregate and materialize the new usage-based metrics.
  • Release flow runtime.
    • This causes usage-based stat events to be produced and reported on.

Rebuilding the catalog_stats table could incur a bit of time when applied.
Otherwise things continue to work as this is played out.

@jgraettinger jgraettinger merged commit c4635db into master Jul 25, 2023
3 checks passed
@jgraettinger jgraettinger deleted the johnny/billing-v2 branch July 25, 2023 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants