-
Notifications
You must be signed in to change notification settings - Fork 1k
Extension
English | 中文
json-iterator provides some options for serialization/deserialization, but they cannot cover complicated use case. For example, there's not any option json-iterator has provided can help if you want your boolean values to be encoded as integer. However, json-iterator provide an Extension
mechanism for us to customize the serialization/deserialization
Before introducing the use of Extension
, we need to know what is ValEncoder
and ValDecoder
in json-iterator. When we use Extension
, we are actually registering different ValEncoder
and ValDecoder
for different types. Please note that ValEncoder
/ValDecoder
is different from json.Encoder
/json.Decoder
, don't confuse them.
-
ValEncoder
type ValEncoder interface { IsEmpty(ptr unsafe.Pointer) bool Encode(ptr unsafe.Pointer, stream *Stream) }
ValEncoder
is an encoder used by json-iterator to encode a certain type of data. Its two member methods are introduced as bellow:-
Encode
Encode
function is used to encode certain types of data.ptr
points to the value to be encoded.stream
provides different methods for users to write various types of data to the output stream. When you implementValEncoder
, what you need to do in this function is to convertptr
into a pointer of the data type corresponding to thisValEncoder
, and then call thestream
's methods to encode and output the value pointed to byptr
-
IsEmpty
IsEmpty
is a function related to the optionomitempty
(check godoc of encoding/json for more information) ofjson
struct field tag. If you're an encoding/json user, you may know that if a struct field'somitempty
option ofjson
tag is enabled, this field will be omitted when encoding into JSON if the value of this filed "is empty".IsEmpty
is a function for you to define what means "is empty" with your value. Therefore, what you need to do in this function is to convertptr
into a pointer of the data type corresponding to thisValEncoder
and check the value if it's empty
Let's look into an example to help us to understand
ValEncoder
. json-iterator provides a built-inTimeAsInt64Codec
, here's its implementation:func (codec *timeAsInt64Codec) IsEmpty(ptr unsafe.Pointer) bool { ts := *((*time.Time)(ptr)) return ts.UnixNano() == 0 } func (codec *timeAsInt64Codec) Encode(ptr unsafe.Pointer, stream *jsoniter.Stream) { ts := *((*time.Time)(ptr)) stream.WriteInt64(ts.UnixNano() / codec.precision.Nanoseconds()) }
In the
Encode
function,ptr
is converted into a pointer pointing totime.Time
type, and then dereferenced to get thetime.Time
object pointed to by it. Next, call its member function to calculate its corresponding unix time, and finally call theWriteInt64
function provided bystream
to encode and output theint64
type unix time.IsEmpty
gets thetime.Time
object pointed to byptr
in the same way, and then regard the value of the object as empty if its converted unix time is 0 -
-
ValDecoder
type ValDecoder interface { Decode(ptr unsafe.Pointer, iter *Iterator) }
ValDecoder
is a decoder used by json-iterator to decode a certain type of data. Its member method is introduced as bellow:-
Decode
Decode
function is used to decode a certain type of data.ptr
points to the value to be set after decoding. iter provides different methods for users to read various types of data from the input stream. When you implementValDecoder
, what you need to do in this function is to calliter
's method to read the data from input stream, and then convertptr
into a pointer of the data type corresponding to thisValDecoder
, and set the value pointed byptr
with the data we read from input stream byiter
Again look into the example of
TimeAsInt64Codec
func (codec *timeAsInt64Codec) Decode(ptr unsafe.Pointer, iter *jsoniter.Iterator) { nanoseconds := iter.ReadInt64() * codec.precision.Nanoseconds() *((*time.Time)(ptr)) = time.Unix(0, nanoseconds) }
Decode
function calliter
'sReadInt64
method to read aint64
type value from the input json stream. Next, because the data type corresponding to ourValDecoder
istime.Time
,ptr
is converted to a pointer pointed totime.Time
type, and the value it pointed to is set to atime.Time
object created with theint64
value read from the input json stream byiter
as the Unix Time. In this way, we have completed the function that read an int64 type unix time from the json string and deserialize it into atime.Time
object -
To customize the serialization/deserialization extension, you need to implement the Extension
interface and register it by calling RegisterExtension
, which contains the following methods:
type Extension interface {
UpdateStructDescriptor(structDescriptor *StructDescriptor)
CreateMapKeyDecoder(typ reflect2.Type) ValDecoder
CreateMapKeyEncoder(typ reflect2.Type) ValEncoder
CreateDecoder(typ reflect2.Type) ValDecoder
CreateEncoder(typ reflect2.Type) ValEncoder
DecorateDecoder(typ reflect2.Type, decoder ValDecoder) ValDecoder
DecorateEncoder(typ reflect2.Type, encoder ValEncoder) ValEncoder
}
There is a DummyExtension
in json-iterator, which is a basic implementation of Extension
(basically do nothing or return empty). When you are defining your own Extension
, you can anonymously embed DummyExtension
, so you don’t need to implement all the Extension
members, just focus on the functions you need. Below we use some examples to illustrate what each member function of Extension
can be used for
-
UpdateStructDescriptor
In the
UpdateStructDescriptor
function, we can customize the encoder/decoder of a field of the structure, or control which strings are bound when the field is serialized/deserializedtype testCodec struct{ } func (codec *testCodec) Encode(ptr unsafe.Pointer, stream *jsoniter.Stream){ str := *((*string)(ptr)) stream.WriteString("TestPrefix_" + str) } func (codec *testCodec) IsEmpty(ptr unsafe.Pointer) bool { str := *((*string)(ptr)) return str == "" } type sampleExtension struct { jsoniter.DummyExtension } func (e *sampleExtension) UpdateStructDescriptor(structDescriptor *jsoniter.StructDescriptor) { if structDescriptor.Type.String() != "main.testStruct" { return } binding := structDescriptor.GetField("TestField") binding.Encoder = &testCodec{} binding.FromNames = []string{"TestField", "Test_Field", "Test-Field"} } func extensionTest(){ type testStruct struct { TestField string } t := testStruct{"fieldValue"} jsoniter.RegisterExtension(&sampleExtension{}) s, _ := jsoniter.MarshalToString(t) fmt.Println(s) // Output: // {"TestField":"TestPrefix_fieldValue"} jsoniter.UnmarshalFromString(`{"Test-Field":"bbb"}`, &t) fmt.Println(t.TestField) // Output: // bbb }
In the above example, first we implemented a
ValEncoder
withtestCodec
, which added a "TestPrefix_" prefix in front of the string when encoding it. Then we registered asampleExtension
. In theUpdateStructDescriptor
function, we set the encoder ofTestField
field oftestStruct
to thetestCodec
, and then bound it with several alias strings. The effect obtained is that when this structure is serialized, the contents ofTestField
will be prefixed with "TestPrefix_ ". When deserialized, the alias ofTestField
will be mapped to this field -
CreateDecoder
-
CreateEncoder
CreateDecoder
andCreateEncoder
are used to create a decoder/encoder corresponding to a certain data typetype wrapCodec struct{ encodeFunc func(ptr unsafe.Pointer, stream *jsoniter.Stream) isEmptyFunc func(ptr unsafe.Pointer) bool decodeFunc func(ptr unsafe.Pointer, iter *jsoniter.Iterator) } func (codec *wrapCodec) Encode(ptr unsafe.Pointer, stream *jsoniter.Stream) { codec.encodeFunc(ptr, stream) } func (codec *wrapCodec) IsEmpty(ptr unsafe.Pointer) bool { if codec.isEmptyFunc == nil { return false } return codec.isEmptyFunc(ptr) } func (codec *wrapCodec) Decode(ptr unsafe.Pointer, iter *jsoniter.Iterator) { codec.decodeFunc(ptr, iter) } type sampleExtension struct { jsoniter.DummyExtension } func (e *sampleExtension) CreateDecoder(typ reflect2.Type) jsoniter.ValDecoder { if typ.Kind() == reflect.Int { return &wrapCodec{ decodeFunc:func(ptr unsafe.Pointer, iter *jsoniter.Iterator) { i := iter.ReadInt() *(*int)(ptr) = i - 1000 }, } } return nil } func (e *sampleExtension) CreateEncoder(typ reflect2.Type) jsoniter.ValEncoder { if typ.Kind() == reflect.Int { return &wrapCodec{ encodeFunc:func(ptr unsafe.Pointer, stream *jsoniter.Stream) { stream.WriteInt(*(*int)(ptr) + 1000) }, isEmptyFunc:nil, } } return nil } func extensionTest(){ i := 20000 jsoniter.RegisterExtension(&sampleExtension{}) s, _ := jsoniter.MarshalToString(i) fmt.Println(s) // Output: // 21000 jsoniter.UnmarshalFromString(`30000`, &i) fmt.Println(i) // Output: // 29000 }
In the above example, we define
wrapCodec
who implementsValEncoder
andValDecoder
, and then we registered anExtension
. In theCreateEncoder
function of thisExtension
, we return awrapCodec
whoseEncode
function is to add an integer to 1000 before encoding it and flush to the output stream. In theCreateDecoder
function of thisExtension
, we return awrapCodec
whoseDecode
function is to read an integer from the input stream, then subtract 1000 from it before setting it to the value pointed byptr
-
CreateMapKeyDecoder
-
CreateMapKeyEncoder
The usage of
CreateMapKeyDecoder
andCreateMapKeyEncoder
is similar to the usage ofCreateDecoder
andCreateEncoder
above, except that they are use for thekey
of typemap
. -
DecorateDecoder
-
DecorateEncoder
DecorateDecoder
andDecorateEncoder
can be used to decorate existingValEncoder
andValEncoder
. Considering such an example, on the basis of the examples given in the descriptions ofCreateDecoder
andCreateEncoder
above, we would like to do something more. When we meet a numeric string, we hope that it can also be parsed into integer numbers, and we want to reuse the decoder in the basic example, then we need to use a decorator.type decorateExtension struct{ jsoniter.DummyExtension } type decorateCodec struct{ originDecoder jsoniter.ValDecoder } func (codec *decorateCodec) Decode(ptr unsafe.Pointer, iter *jsoniter.Iterator) { if iter.WhatIsNext() == jsoniter.StringValue { str := iter.ReadString() if _, err := strconv.Atoi(str); err == nil{ newIter := iter.Pool().BorrowIterator([]byte(str)) defer iter.Pool().ReturnIterator(newIter) codec.originDecoder.Decode(ptr, newIter) }else{ codec.originDecoder.Decode(ptr, iter) } } else { codec.originDecoder.Decode(ptr, iter) } } func (e *decorateExtension) DecorateDecoder(typ reflect2.Type, decoder jsoniter.ValDecoder) jsoniter.ValDecoder{ if typ.Kind() == reflect.Int { return &decorateCodec{decoder} } return nil } func extensionTest(){ var i int jsoniter.RegisterExtension(&sampleExtension{}) jsoniter.RegisterExtension(&decorateExtension{}) jsoniter.UnmarshalFromString(`30000`, &i) fmt.Println(i) // Output: // 29000 jsoniter.UnmarshalFromString(`"40000"`, &i) fmt.Println(i) // Output: // 39000 }
Based on the examples of
CreateDecoder
andCreateEncoder
, we are registering anExtension
. ThisExtension
only implements the decorator function, it read numeric string and convert it to integer and it will be subtracted 1000 before being set to the value pointed byptr
json-iterator provide two RegisterExtension
functions that can be called, one is package level jsoniter.RegisterExtension
, and the other is API
(see Config chapter) level API.RegisterExtension
. Both of them can be used to register extensions, but the scope of extensions registered by the two registration methods is slightly different. The extension registered by jsoniter.RegisterExtension
is effective globally, while API.RegisterExtension
is only effective for the API
interface generated by its corresponding Config
.