Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for non-transactional write. #35

Conversation

Bhupender-Y
Copy link

@Bhupender-Y Bhupender-Y commented Dec 21, 2023

Use non-transactional write processing raw files. In the current scenario for raw files PSC is using transactional write. But in transactional write, there is a limit of event size i.e 8mb. To resolve it out, for raw files processing instead of using transactional writer, it will use non-transactional writer.

@Bhupender-Y Bhupender-Y changed the base branch from stability-improvements to stability-improvements-v2 December 26, 2023 07:41
@Bhupender-Y Bhupender-Y marked this pull request as ready for review December 26, 2023 10:57
@Bhupender-Y Bhupender-Y changed the base branch from stability-improvements-v2 to stability-improvements December 27, 2023 07:11
Signed-off-by: Bhupender Y <[email protected]>
Copy link
Contributor

@ShwethaSNayak ShwethaSNayak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes looks good to me. Elaborated description might help to understand the context.

I assume post changes below scenarios are validated.

  1. Raw file with size more than 8mb
  2. Raw file with size less than 8mb

if(writer.getTransactionStatus() == Transaction.Status.OPEN){
writer.abort();
if (config.exactlyOnce)
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong indentation

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Signed-off-by: Bhupender Y <[email protected]>
Signed-off-by: Bhupender Y <[email protected]>
Copy link
Contributor

@sachin-j-joshi sachin-j-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.
Let's test it thoroughly before merging

@Bhupender-Y Bhupender-Y changed the base branch from stability-improvements to stability-improvements-v2 January 17, 2024 11:14
@Bhupender-Y Bhupender-Y changed the base branch from stability-improvements-v2 to stability-improvements January 17, 2024 11:15
@Bhupender-Y Bhupender-Y changed the base branch from stability-improvements to stability-improvements-v2 January 17, 2024 12:06
Copy link
Contributor

@sachin-j-joshi sachin-j-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Bhupender Y <[email protected]>
@sachin-j-joshi sachin-j-joshi merged commit 13d830f into pravega:stability-improvements-v2 Jan 25, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants