Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add user encode/decode #10

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

add user encode/decode #10

wants to merge 1 commit into from

Conversation

mdavidsaver
Copy link
Member

Attempt to address #9 with a user API for (de)serialization. Three pairs of functions are added for the 3 contexts in which PVD are found in the PVA protocol: Type, Full structure, and Partial structure. The xcode::encode*() functions append to an std::vector<uint8_t>. The xcode::decode*() functions operate on a pair of pointers (like iterators). The first pointer is passed by reference, and updated based on the number of bytes consumed by each operation.

@mdavidsaver mdavidsaver self-assigned this Jul 6, 2020
@mdavidsaver mdavidsaver added enhancement New feature or request question Further information is requested labels Jul 6, 2020
@AppVeyorBot
Copy link

Build pvxs 1.0.269 failed (commit aa4b3b1941 by @mdavidsaver)

@mdavidsaver
Copy link
Member Author

@karlosp Can you comment on whether this change would be flexible enough to support your use case?

@karlosp
Copy link
Contributor

karlosp commented Jul 14, 2020

I think that we have a bit of special use cases for serialization.

Proposed changes would be great if we would need to save/send std::vector<uint8_t> buf.

Because we use the EPICS transport layer in a real-time framework for custom structures, we need to (de)serialize every field separately and not the whole Value.

In the example below I will try to describe how we solve it for now.
Do not know if this is the best approach but for now, somehow works.

In initialization time we must dynamically construct Value, because we must support custom user types.
For example, let say that we have a struct below we would like to send over EPICS.
e.g.

struct TestStructure {
  std::vector<size_t> values;
  size_t index;
  size_t hash;
}

Currently, we dynamically create Value and reserve buffer as follows:

pvxs::TypeDef pvxs_field_builder_{pvxs::TypeCode::Struct,"simple_t",{}};
std::vector<pvxs::Member>& pvxs_members;
std::vector<uint8_t> pvxs_buffer_;

// pseudocode
size_t buffer_size{0};
for member in TestStructure:
 pvxs_members.push_back(pvxs::Member(member_type, member_name));
 buffer_size += sizeof(member_type);

pvxs_field_builder_ += pvxs_members_;
Value value_ = pvxs_field_builder_.create();
pvxs_buffer_ = std::vector<uint8_t>(buffer_size, '\0');

Example of putting data from TestStructure to PutBuilder.

// Sending data
// pseudocode
copy data from TestStructure to pvxs_buffer_;
unsigned char* storage_location = nullptr;
auto put = pvxs::client::PutBuilder(context_.put(topic_name));
size_t position{0};
// pseudocode
for (pvxs::Value fld : value_.iall()) {
  if(fld.type().scalarOf().code == pvxs::TypeCode::Struct)
     continue;
  storage_location = pvxs_buffer_.data() + member_position[position].offset;
  put.set(value_.nameOf(fld), pvxs::shared_array<const int8_t>((int8_t*)(storage_location), sizeof(member) / sizeof(int8_t)));
++position;
}
put.exec()->wait(5.0);

receiving data

//pvxs::client::Subscription> monitor;
value_.assign(monitor->pop());
// pseudocod
//create empty TestStructure (take special care for initializing vectors and arrays to be correct size)
TestStructure TS;
for (auto fld : value_.iall()) {
  if (fld.type() == pvxs::TypeCode::Struct) {
    continue;
  }
 read and copy data from fld to TS // type end size taking into account
}

@mdavidsaver
Copy link
Member Author

Proposed changes would be great if we would need to save/send std::vector<uint8_t> buf.

What interface would work?

FYI. If your libraries are able to interface with a shared_ptr, then the aliasing constructor of std::shared_ptr may be helpful here. eg.

auto buf = std::make_shared<std::vector<uint8_t>>({1,2,3});
std::shared_ptr<uint8_t> aliased(buf, buf.data());
buf.reset();
aliased.reset(); // ~vector happens here

The aliased shared_ptr shares the same ref. counter, and the same (virtual) destructor.

Also, I'm having difficulty in parsing through your example. A couple of things I pick up:

 buffer_size += sizeof(member_type);

You would like a way to find the serialized size of a field based solely on the type (TypeCode)? The PVD encoding has a number of variable sized elements, which makes this difficult for all but the simplest definitions.

Is the intent to avoid resize()ing the vector during serialization?

put.set(value_.nameOf(fld), pvxs::shared_array<const int8_t>((int8_t*)(storage_location), sizeof(member) / sizeof(int8_t)));

Do I understand this correctly? You would like to send serialized data as a byte array over PVA?

@mdavidsaver
Copy link
Member Author

For lack of feedback, I have dropped this change from my development area.

@mdavidsaver mdavidsaver marked this pull request as draft June 14, 2023 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants