-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ntuple] Remove RFieldBase::GetNElements() #16768
Merged
jblomer
merged 9 commits into
root-project:master
from
jblomer:ntuple-remove-rfield-getnelements
Oct 31, 2024
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
2b6fb82
[ntuple] replace pointer by reference in view creation
jblomer 200dd54
[ntuple] add -> operator to RColumnDescriptorIterable
jblomer d8decdf
[ntuple] add Internal::GetFieldRange() helper
jblomer 06a3ffe
[ntuple] pass iteration range to view on construction
jblomer 485ad6a
[ntuple] remove RFieldBase::GetNElements()
jblomer 7ff2985
[ntuple] add tests for field iteration in views
jblomer 56b0aa3
[ntuple] fix typo in error message
jblomer 4561707
[NFC][ntuple] fix typos in code comment
jblomer 2d4cc9e
[ntuple] rename RNTupleGlobalRange::count() --> size()
jblomer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
/// \file RNTupleView.cxx | ||
/// \ingroup NTuple ROOT7 | ||
/// \author Jakob Blomer <[email protected]> | ||
/// \date 2024-10-28 | ||
/// \warning This is part of the ROOT 7 prototype! It will change without notice. It might trigger earthquakes. Feedback | ||
/// is welcome! | ||
|
||
/************************************************************************* | ||
* Copyright (C) 1995-2024, Rene Brun and Fons Rademakers. * | ||
* All rights reserved. * | ||
* * | ||
* For the licensing terms see $ROOTSYS/LICENSE. * | ||
* For the list of contributors see $ROOTSYS/README/CREDITS. * | ||
*************************************************************************/ | ||
|
||
#include <ROOT/RError.hxx> | ||
#include <ROOT/RFieldBase.hxx> | ||
#include <ROOT/RNTupleDescriptor.hxx> | ||
#include <ROOT/RNTupleView.hxx> | ||
#include <ROOT/RPageStorage.hxx> | ||
|
||
ROOT::Experimental::RNTupleGlobalRange | ||
ROOT::Experimental::Internal::GetFieldRange(const RFieldBase &field, const RPageSource &pageSource) | ||
{ | ||
const auto &desc = pageSource.GetSharedDescriptorGuard().GetRef(); | ||
|
||
auto fnGetPrincipalColumnId = [&desc](DescriptorId_t fieldId) -> DescriptorId_t { | ||
auto columnIterable = desc.GetColumnIterable(fieldId); | ||
return (columnIterable.size() > 0) ? columnIterable.begin()->GetPhysicalId() : kInvalidDescriptorId; | ||
}; | ||
|
||
auto columnId = fnGetPrincipalColumnId(field.GetOnDiskId()); | ||
if (columnId == kInvalidDescriptorId) { | ||
for (const auto &f : field) { | ||
columnId = fnGetPrincipalColumnId(f.GetOnDiskId()); | ||
if (columnId != kInvalidDescriptorId) | ||
break; | ||
} | ||
} | ||
|
||
if (columnId == kInvalidDescriptorId) { | ||
throw RException(R__FAIL("field iteration over empty fields is unsupported: " + | ||
desc.GetQualifiedFieldName(field.GetOnDiskId()))); | ||
} | ||
|
||
auto arraySize = std::max(std::uint64_t(1), desc.GetFieldDescriptor(field.GetOnDiskId()).GetNRepetitions()); | ||
return RNTupleGlobalRange(0, desc.GetNElements(columnId) / arraySize); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it strictly necessary to throw in this case? I would imagine iterating over an empty field behaves the same as iterating over an empty vector or other collection, where a for-each loops just directly terminates and any direct access will throw an index-out-of-bounds error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's more tricky: you can have an empty struct with a read rule. In this case, you'd still like to call the read rule in every iteration, but it is tricky to figure out how often one should call it... So this is why I thought it's easiest for now to forbid this special case. We may, for the special case of top-level fields, pass the entry range to the created view.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would agree that it make sense to at least delay the implementation until we have a clearer picture of the use cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to note that this seems to break ATLAS' production workflow tests where we're reading RNTuple inputs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the failures? Have the file been regenerated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is in a mutli-chain workflow. The first step creates an RNTuple that is read by the second step. We throw on the very first event of the second step, seemingly related to reading an empty top-level field, e.g. (the messages need to be improved but that's independent):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's continue the conversation at #16826