Code

feat: implement virtual arrays#3364

Code

Draft

ikrommyd wants to merge 29 commits intoscikit-hep:mainscikit-hep/awkward:mainfrom ikrommyd:virtual-arraysikrommyd/awkward:virtual-arraysCopy head branch name to clipboard

+491-35

Commits

Commits on Jan 23, 2025

• first try to implement virtual arrays at (least close to) the buffer level

  

  pfackeldey

  authored and

  ikrommyd

  committed
  8407680

  Copy full SHA for 8407680

• lint

  

  pfackeldey

  authored and

  ikrommyd

  committed
  938b883

  Copy full SHA for 938b883

• prepare virtual arrays for Iason (now really at the buffer level)

  

  pfackeldey

  authored and

  ikrommyd

  committed
  eccb9a8

  Copy full SHA for eccb9a8

• fix .T for virtual arrays

  

  pfackeldey

  authored and

  ikrommyd

  committed
  960f7c3

  Copy full SHA for 960f7c3

• minimal changes to get an example of a listoffsetarray with virtual buffers working in a notebook

  ikrommyd

  committed
  ecb883b

  Copy full SHA for ecb883b

• what if we start like this?

  ikrommyd

  committed
  40824c7

  Copy full SHA for 40824c7

• add repr

  ikrommyd

  committed
  7b2809c

  Copy full SHA for 7b2809c

• restore _data <-> data to original

  ikrommyd

  committed
  85d8a83

  Copy full SHA for 85d8a83

• an attempt at implementing some methods

  ikrommyd

  committed
  3b5ac6a

  Copy full SHA for 3b5ac6a

• style: pre-commit fixes

  

  pre-commit-ci[bot]

  authored and

  ikrommyd

  committed
  6b8cbc4

  Copy full SHA for 6b8cbc4

• tiny changes

  ikrommyd

  committed
  d0e4947

  Copy full SHA for d0e4947

• add ones_like and full_like to play around a bit

  ikrommyd

  committed
  a91dad0

  Copy full SHA for a91dad0

• no need to implement ufuncs separately. They are covered by apply_ufunc

  ikrommyd

  committed
  40841a2

  Copy full SHA for 40841a2

• add materialization in array_module.py

  ikrommyd

  committed
  72babd0

  Copy full SHA for 72babd0

• same repr for placeholders

  ikrommyd

  committed
  95e9e3c

  Copy full SHA for 95e9e3c

• adhere to ArrayLike interface

  ikrommyd

  committed
  6a25b1b

  Copy full SHA for 6a25b1b

• add TODO comment

  ikrommyd

  committed
  c1ff9ea

  Copy full SHA for c1ff9ea

• try kernels

  ikrommyd

  committed
  166fcb0

  Copy full SHA for 166fcb0

• never create new virtual arrays for now, let them materialize

  ikrommyd

  committed
  c5020d0

  Copy full SHA for c5020d0

• ruff fix

  ikrommyd

  committed
  7a8a074

  Copy full SHA for 7a8a074

Commits on Jan 25, 2025

• bring back creation of virtual arrays with a local cache

  ikrommyd

  committed
  b2bb788

  Copy full SHA for b2bb788

• forgot this

  ikrommyd

  committed
  6e41bda

  Copy full SHA for 6e41bda

• Merge branch 'main' into virtual-arrays

  ikrommyd

  committed
  a84d84e

  Copy full SHA for a84d84e

• maybe rapidjson submodule fix

  ikrommyd

  committed
  4137bf9

  Copy full SHA for 4137bf9

• adjust _get_item_at_placeholder for virtual arrays as well

  ikrommyd

  committed
  c773329

  Copy full SHA for c773329

Commits on Jan 26, 2025

• new _get_item_at_virtual function and different VirtualValue vs PlaceholderValue

  ikrommyd

  committed
  391fe17

  Copy full SHA for 391fe17

• remove some comments and docstrings that we decided to use in a different manner

  ikrommyd

  committed
  80f40ae

  Copy full SHA for 80f40ae

• let iterators of virtual buffers materialize

  ikrommyd

  committed
  93db3a4

  Copy full SHA for 93db3a4

Commits on Jan 27, 2025

• inherit from NDArrayOperatorsMixin as well

  ikrommyd

  committed
  4c36c7d

  Copy full SHA for 4c36c7d