Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[humble] Make snapshot writing into a new file each time it is triggered (backport #1842) #1850

Open
wants to merge 1 commit into
base: humble
Choose a base branch
from

Conversation

mergify[bot]
Copy link

@mergify mergify bot commented Nov 1, 2024

This PR will make snapshot writing into a new file each time it is triggered.

Note. Snapshot now becomes a blocking call and mutually exclusive with the writer::write(message) method to avoid race conditions, i.e., blocking the same writer_mutex_.
The set_data_ready() method from #1839 is not needed since "snapshot" has now become a blocking call; therefore, there are no race conditions, and we will not dump data from the cyclic buffer twice because data_ready will be settled false after the first dump with message_cache_->notify_data_ready();.

void CircularMessageCache::swap_buffers()
{
std::lock_guard<std::mutex> producer_lock(producer_buffer_mutex_);
// Swap buffers only if data is ready. Data not ready when we are calling flushing on exit and
// we should not dump buffer on exit if snapshot has not been triggered.
if (data_ready_) {
std::lock_guard<std::mutex> consumer_lock(consumer_buffer_mutex_);
consumer_buffer_->clear();
std::swap(producer_buffer_, consumer_buffer_);
data_ready_ = false;
}
}

However, we still need to call split_bagfile_local(true) to trigger callbacks and open a new storage file.

This PR could be backported. There are no API/ABI breaking changes in it.


This is an automatic backport of pull request #1842 done by Mergify.

* Make snapshot writing into a new file each time when it is triggered

- Note. Snapshot now became a blocking call and mutually exclusive with
writer::write(message) method to avoid race conditions.
i.e. blocking the same writer_mutex_

Signed-off-by: Michael Orlov <[email protected]>

* Add unit test to make sure that snapshot writing in the new file

Co-authored-by: Clemens Mühlbacher <[email protected]>
Signed-off-by: Michael Orlov <[email protected]>

* Add support for snapshot with file compression

Signed-off-by: Michael Orlov <[email protected]>

* Rename newly added tests to avoid misunderstanding

Signed-off-by: Michael Orlov <[email protected]>

* Address review comments in tests

Signed-off-by: Michael Orlov <[email protected]>

* Change order of includes in the test_sequential_compression_writer.cpp

Signed-off-by: Michael Orlov <[email protected]>

* Update metadata_.message_count unconditionally in write_messages(..)

Signed-off-by: Michael Orlov <[email protected]>

---------

Signed-off-by: Michael Orlov <[email protected]>
Co-authored-by: Clemens Mühlbacher <[email protected]>
(cherry picked from commit 3f2281f)
@mergify mergify bot requested a review from a team as a code owner November 1, 2024 23:33
@mergify mergify bot requested review from emersonknapp and jhdcs and removed request for a team November 1, 2024 23:33
@MichaelOrlov MichaelOrlov changed the title Make snapshot writing into a new file each time it is triggered (backport #1842) [humble] Make snapshot writing into a new file each time it is triggered (backport #1842) Nov 1, 2024
@MichaelOrlov MichaelOrlov requested review from MichaelOrlov and removed request for emersonknapp and jhdcs November 1, 2024 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant