Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: improve filterAttrs #345547

Merged
merged 3 commits into from
Nov 1, 2024
Merged

Conversation

h7x4
Copy link
Member

@h7x4 h7x4 commented Sep 30, 2024

Description of changes

Small optimization. Instead of having filterAttrs collect values and merge them, it only collects the names of the unwanted attributes and passes them to removeAttrs. This leads to slightly less memory copied around. Also added a test

NIX_SHOW_STATS=1 nix flake check before
{
  "cpuTime": 10.154743194580078,
  "envs": {
    "bytes": 693836664,
    "elements": 35023599,
    "number": 25852992
  },
  "gc": {
    "heapSize": 789245952,
    "totalBytes": 3407736128
  },
  "list": {
    "bytes": 60538296,
    "concats": 866078,
    "elements": 7567287
  },
  "nrAvoided": 28680092,
  "nrFunctionCalls": 23458684,
  "nrLookups": 11705269,
  "nrOpUpdateValuesCopied": 32773039,
  "nrOpUpdates": 1231329,
  "nrPrimOpCalls": 11339176,
  "nrThunks": 27686197,
  "sets": {
    "bytes": 913395872,
    "elements": 51894915,
    "number": 5192327
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 1220932,
    "number": 99185
  },
  "values": {
    "bytes": 1055012784,
    "number": 43958866
  }
}
NIX_SHOW_STATS=1 nix flake check after
{
  "cpuTime": 10.344867706298828,
  "envs": {
    "bytes": 691711896,
    "elements": 34935067,
    "number": 25764460
  },
  "gc": {
    "heapSize": 738914304,
    "totalBytes": 3402504560
  },
  "list": {
    "bytes": 60176696,
    "concats": 866078,
    "elements": 7522087
  },
  "nrAvoided": 28543943,
  "nrFunctionCalls": 23401274,
  "nrLookups": 11712564,
  "nrOpUpdateValuesCopied": 32773039,
  "nrOpUpdates": 1231329,
  "nrPrimOpCalls": 11339176,
  "nrThunks": 27657492,
  "sets": {
    "bytes": 912128368,
    "elements": 51843610,
    "number": 5164413
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 1220945,
    "number": 99186
  },
  "values": {
    "bytes": 1054323864,
    "number": 43930161
  }
}

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: lib The Nixpkgs function library label Sep 30, 2024
@h7x4 h7x4 requested a review from roberth September 30, 2024 17:02
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 1-10 labels Sep 30, 2024
Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

lib/attrsets.nix Outdated Show resolved Hide resolved
@h7x4 h7x4 force-pushed the lib-attrsets-improve-filterattrs branch from cd68034 to 83e1488 Compare October 25, 2024 13:08
@h7x4 h7x4 requested a review from infinisil October 25, 2024 13:08
`filter` is a primop which is designed for this task, and it saves
the allocation of some singleton lists here.
@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Oct 31, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild labels Oct 31, 2024
@infinisil infinisil merged commit 5690833 into NixOS:master Nov 1, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: lib The Nixpkgs function library 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 12.approvals: 1 This PR was reviewed and approved by one reputable person
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants