-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split MockContext into Standalone and multi-group #42
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Preliminary review. Had not yet time to check the tests.
Co-authored-by: Torsten Kilias <[email protected]>
Co-authored-by: Torsten Kilias <[email protected]>
Co-authored-by: Torsten Kilias <[email protected]>
Co-authored-by: Torsten Kilias <[email protected]>
return MockContext(iter(groups), meta_set_emits) | ||
|
||
|
||
def test_scroll(context_set_emits): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Nicoretti suggested that we implement a generator which returns the column values and we collect these values in a list and compare this list at the end with an assert. Furthermore, we shouldn't test internal stuff like _current.
And, I thought about _next_group and _output_groups, I think, we can make them public, because the UDF will use the Interface UDFContext and not the MockContext.
def generator(ctx):
while True:
if not ctx._next_group():
break
while True:
yield ctx.t2
if not ctx.next():
break
result=list(generator(context_set_emits))
assert result==['cat', 'dog', ...]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want to put both groups in one stream. We could create a jagged array instead. But I am not in favour of having an extra piece of code in a test. I think testing should be straightforward if possible.
I should perhaps remove the checks of "private" variables, e.g. _current_context
. On that note, I think we should make the _next_group
and the _output_groups
public. That would require some changes upstream, e.g. in UDFMockExecutor. Would you agree?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agree with making _next_group and _output_groups public, because the class is anyway usually used through the Executor
and yes please remove _current_context and so on
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding the remaining issue, let me think. about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That with the jagged array sounds ok to me.
self._output.extend(tuples) | ||
|
||
@staticmethod | ||
def _validate_tuples(row: Tuple, columns: List[Column]): | ||
def validate_emit(row: Tuple, columns: List[Column]): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think, we should make this a independent function and not a static method
Fixes #33
Created two classes MockContext and Standalone mock context instead of one MockContext. The interface of the MockContext is preserved. The current emit output of either of the classes can be accessed through the
output
property.