Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON and msgpack encoding fails to skip UNSET values in array_like Struct #723

Open
chuckwondo opened this issue Aug 22, 2024 · 1 comment · May be fixed by #724
Open

JSON and msgpack encoding fails to skip UNSET values in array_like Struct #723

chuckwondo opened this issue Aug 22, 2024 · 1 comment · May be fixed by #724

Comments

@chuckwondo
Copy link

Description

Per the documentation of UNSET:

During encoding, any field containing UNSET is omitted from the message.

Unfortunately, this fails for Structs with array_like=True, resulting in the following error:

TypeError: Encoding objects of type msgspec.UnsetType is unsupported

This appears to be an oversight in the function json_encode_struct_array, which does not check for (and ignore) UNSET values, whereas json_encode_struct_object does ignore UNSET values, as expected.

I suspect this might be intentional since there can be cases where decoding in the face of UNSET values can present ambiguity, as least in the face of roundtripping, when attributes that can be UNSET are interspersed with those that cannot be UNSET.

However, I would argue that the developer should make judicious use of array_like=True to avoid such ambiguity. In other words, it seems reasonable to me that msgspec should indeed skip UNSET values in this case anyway, documenting that this is sort of a "use at your own risk of ambiguity" for the developer, particularly since decoding works without issue.

For example, consider this struct:

class Position(msgspec.Struct, array_like=True, frozen=True, forbid_unknown_fields=True):
    longitude: float
    latitude: float
    altitude: Union[float, msgspec.UnsetType] = msgspec.UNSET

Decoding works perfectly well:

>>> pos = msgspec.json.decode("[-3.703790, 40.416775]", type=Position)
>>> pos
Position(longitude=-3.70379, latitude=40.416775, altitude=UNSET)

Unfortunately, encoding does not:

>>> msgspec.json.encode(pos)
Traceback (most recent call last):
...
TypeError: Encoding objects of type msgspec.UnsetType is unsupported

Instead of that TypeError, I expect encoding to result in b'[-3.70379, 40.416775]' since altitude is UNSET.

When I remove array_like=True (or set it explicitly to False), decoding (a JSON object in place of a JSON array, without a value for altitude) and encoding work as expected.

@chuckwondo
Copy link
Author

Further, adding omit_defaults=True doesn't make a difference.

chuckwondo added a commit to chuckwondo/msgspec that referenced this issue Aug 22, 2024
@chuckwondo chuckwondo linked a pull request Aug 22, 2024 that will close this issue
@chuckwondo chuckwondo changed the title JSON encoding fails to skip UNSET values in array_like Struct JSON and msgpack encoding fails to skip UNSET values in array_like Struct Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant