Examples of string log entries.

[natilivni]

Hello, all the examples I have seen have the log entries containing long values. How does one create/consume log entries which contain string data?

[sakno]

Log entry is actually a binary object that can represent any data: primitive data types, strings of various encodings, ASN.1 serialized objects or even you own custom format. The log entry implements IDataTransferObject low-level interface. DTO here is the carrier of the payload. The structure of such payload is described by DTO itself. The key method of IDataTransferObject interface is WriteAsync which is responsible for producing serialized form of the structured content. GetObjectDataAsync allows to apply transformation to DTO.

Here is trivial implementation of string DTO:

using System;
using System.Threading;
using System.Threading.Tasks;
using DotNext.IO;
using DotNext.Net.Cluster.Consensus.Raft;

public readonly struct TextLogEntry : IRaftLogEntry
{
       private readonly string? content;

        public TextLogEntry(string value, long term)
        {
          Term = term;
          Timestamp = DateTimeOffset.Now;
          content = value;
        }

        public long Term { get; }

        public DateTimeOffset Timestamp { get; }

        public int Length => Encoding.UTF8.GetByteCount(Content);

        /// <inheritdoc/>
        bool IDataTransferObject.IsReusable => true;

        /// <inheritdoc/>
        long? IDataTransferObject.Length => Length;

        /// <summary>
        /// The message content.
        /// </summary>
        public string Content => content ?? string.Empty;

        /// <inheritdoc/>
        ValueTask IDataTransferObject.WriteToAsync<TWriter>(TWriter writer, CancellationToken token)
            => writer.WriteAsync(Content.AsMemory(), Encoding.UTF8, null, token);
}

The concept of DTO is tightly coupled with IAsyncBinaryReader and IAsyncBinaryWriter interfaces. These interfaces allow to deserialize or serialize the structured content of DTO. DotNext.IO namespace contains everything you need to create custom formats of log entries.

However, I would not recommend you to use the string as a format of the custom log entry in production due to performance reasons. The string cannot be stored as-is. It should be encoded as UTF-8, UTF-16LE, UTF-16BE, UTF-32BE, UTF-32LE or whatever you need. In the interpreter, you need to decode the string first. Here is the first loose. After that you need to parse the string into some syntax tree and then apply semantic analysis to such tree. Here is the second loose. It's much better to encode log entry efficiently in binary format and then decode it directly to the syntax tree.

[natilivni]

Thank you! You have developed an amazing pattern and you are amazingly responsive!

[natilivni]

I am taking your advice and trying to replace my string with a struct that will then be used by IAsyncBinaryReader and IAsyncBinaryWriter. However, I am receiving the following error on my writer.WriteAsync(Content, token) call:

"CS8377: The type 'MyMessage' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'IAsyncBinaryWriter.WriteAsync(T, CancellationToken)'"

In order to turn my string into a value type, I defined my struct as follows:

public struct MyMessage { public ReadOnlyMemory<char> MyString { get; set; } }

Any ideas what may be wrong?

[natilivni]

I thought it might be the use of a property instead of a field but changing the message to the following but the error is still there...

public struct MyMessage { public ReadOnlyMemory<char> MyString; }

[sakno]

It's a correct error. Generic parameter T of WriteAsync<T>(in T value, CancellationToken) method has unmanaged restriction. It means that target type must be a blittable value type. Otherwise, there is no way to represent value type in the form of the sequence of bytes. ReadOnlyMemory<T> is not a blittable value type.

[sakno]

@natilivni , I'll add interpreter infrastructure in the next major version of .NEXT to simplify handling custom log entries.