Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(logstorage): create batch writer #457

Merged
merged 18 commits into from
Jul 19, 2024
Merged

refactor(logstorage): create batch writer #457

merged 18 commits into from
Jul 19, 2024

Commits on Jul 18, 2024

  1. Configuration menu
    Copy the full SHA
    a0e7de8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f5652b1 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Configuration menu
    Copy the full SHA
    b65379e View commit details
    Browse the repository at this point in the history
  2. refactor(logstorage): create batch writer

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                    │   old.txt    │              new.txt               │
                    │    sec/op    │   sec/op     vs base               │
    InserterLogs-32   2.915m ± 12%   3.112m ± 6%  +6.76% (p=0.001 n=15)
    
                    │   old.txt    │               new.txt                │
                    │     B/op     │     B/op      vs base                │
    InserterLogs-32   1.709Mi ± 0%   1.163Mi ± 0%  -31.98% (p=0.000 n=15)
    
                    │   old.txt   │               new.txt               │
                    │  allocs/op  │  allocs/op   vs base                │
    InserterLogs-32   23.17k ± 0%   16.40k ± 0%  -29.23% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    102fd87 View commit details
    Browse the repository at this point in the history
  3. perf(chstorage): re-use log inserter columns

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                    │   old.txt   │               new.txt               │
                    │   sec/op    │   sec/op     vs base                │
    InserterLogs-32   2.055m ± 1%   1.614m ± 1%  -21.47% (p=0.000 n=15)
    
                    │    old.txt    │               new.txt                │
                    │     B/op      │     B/op      vs base                │
    InserterLogs-32   1189.9Ki ± 0%   753.6Ki ± 0%  -36.67% (p=0.000 n=15)
    
                    │   old.txt    │               new.txt               │
                    │  allocs/op   │  allocs/op   vs base                │
    InserterLogs-32   16.393k ± 0%   7.897k ± 0%  -51.83% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    6eecac1 View commit details
    Browse the repository at this point in the history
  4. perf(chstorage): compute column list only once

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                    │   old.txt   │               new.txt               │
                    │   sec/op    │   sec/op     vs base                │
    InserterLogs-32   16.30m ± 1%   14.32m ± 1%  -12.16% (p=0.000 n=15)
    
                    │   old.txt    │               new.txt                │
                    │     B/op     │     B/op      vs base                │
    InserterLogs-32   7.333Mi ± 0%   4.301Mi ± 0%  -41.34% (p=0.000 n=15)
    
                    │   old.txt   │              new.txt               │
                    │  allocs/op  │  allocs/op   vs base               │
    InserterLogs-32   78.96k ± 0%   73.94k ± 0%  -6.36% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    753e84d View commit details
    Browse the repository at this point in the history
  5. perf(chstorage): re-use log inserter columns proto.Input

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                    │   old.txt   │               new.txt               │
                    │   sec/op    │   sec/op     vs base                │
    InserterLogs-32   14.48m ± 0%   12.17m ± 1%  -15.97% (p=0.000 n=15)
    
                    │   old.txt    │               new.txt                │
                    │     B/op     │     B/op      vs base                │
    InserterLogs-32   4.301Mi ± 0%   3.013Mi ± 0%  -29.95% (p=0.000 n=15)
    
                    │   old.txt   │               new.txt               │
                    │  allocs/op  │  allocs/op   vs base                │
    InserterLogs-32   73.94k ± 0%   65.29k ± 0%  -11.70% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    b635ad8 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    1cb63d8 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    c311fe4 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    00f4c17 View commit details
    Browse the repository at this point in the history
  9. perf(otelstorage): optimize replacer

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                  │   old.txt   │               new.txt               │
                  │   sec/op    │   sec/op     vs base                │
    KeyToLabel-32   91.50n ± 0%   59.52n ± 1%  -34.95% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    db78d4b View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    41a1d54 View commit details
    Browse the repository at this point in the history
  11. perf(chstorage): generate insert body only once

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                    │   old.txt    │               new.txt               │
                    │    sec/op    │   sec/op     vs base                │
    InserterLogs-32   11.644m ± 1%   7.732m ± 1%  -33.60% (p=0.000 n=15)
    
                    │   old.txt    │               new.txt                │
                    │     B/op     │     B/op      vs base                │
    InserterLogs-32   2.917Mi ± 0%   1.501Mi ± 0%  -48.55% (p=0.000 n=15)
    
                    │   old.txt   │               new.txt               │
                    │  allocs/op  │  allocs/op   vs base                │
    InserterLogs-32   55.44k ± 0%   33.24k ± 0%  -40.05% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    154ac12 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    5ff9560 View commit details
    Browse the repository at this point in the history
  13. refactor(tracestorage): create batch writer

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                      │   old.txt    │               new.txt               │
                      │    sec/op    │   sec/op     vs base                │
    InserterTraces-32   1386.1µ ± 1%   557.9µ ± 2%  -59.75% (p=0.000 n=15)
    
                      │    old.txt     │               new.txt                │
                      │      B/op      │     B/op      vs base                │
    InserterTraces-32   1145.85Ki ± 0%   56.02Ki ± 0%  -95.11% (p=0.000 n=15)
    
                      │   old.txt   │               new.txt               │
                      │  allocs/op  │  allocs/op   vs base                │
    InserterTraces-32   6.983k ± 0%   1.009k ± 0%  -85.55% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    a18c67c View commit details
    Browse the repository at this point in the history
  14. perf(otelstorage): increase Attrs on-stack sorting threshold

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                      │   old.txt   │            new.txt            │
                      │   sec/op    │   sec/op     vs base          │
    InserterTraces-32   557.1µ ± 1%   553.5µ ± 1%  ~ (p=0.081 n=15)
    
                      │   old.txt    │               new.txt                │
                      │     B/op     │     B/op      vs base                │
    InserterTraces-32   56.02Ki ± 0%   28.29Ki ± 0%  -49.50% (p=0.000 n=15)
    
                      │   old.txt   │              new.txt              │
                      │  allocs/op  │ allocs/op   vs base               │
    InserterTraces-32   1009.0 ± 0%   969.0 ± 0%  -3.96% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    a249e32 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    1364027 View commit details
    Browse the repository at this point in the history
  16. perf(otelstorage): write converted label right to column

    ```
    cpu: AMD Ryzen 9 5950X 16-Core Processor
                    │   old.txt   │              new.txt               │
                    │   sec/op    │   sec/op     vs base               │
    InserterLogs-32   7.852m ± 1%   7.479m ± 1%  -4.75% (p=0.000 n=15)
    
                    │   old.txt    │               new.txt               │
                    │     B/op     │     B/op      vs base               │
    InserterLogs-32   1.501Mi ± 0%   1.359Mi ± 0%  -9.44% (p=0.000 n=15)
    
                    │   old.txt   │               new.txt               │
                    │  allocs/op  │  allocs/op   vs base                │
    InserterLogs-32   33.24k ± 0%   24.63k ± 0%  -25.91% (p=0.000 n=15)
    ```
    tdakkota committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    34b4988 View commit details
    Browse the repository at this point in the history