Description

mischnic

opened on Dec 20, 2020

Issue body actions

const msgpack = require('@msgpack/msgpack');

let original = {x: undefined};
let serialized = msgpack.encode(original, {extensionCodec});
let deserialized = msgpack.decode(serialized, {extensionCodec});
console.log(original, deserialized, original.x === deserialized.x);

prints "{ x: undefined } { x: null } false". This is a problem because now code checking for strict equality fails.

I was also not able to handle this in an extension codec because that is only called for non-primitive values.

ignoreUndefined: true doesn't work in all cases:

let original = undefined;
let serialized = msgpack.encode(original, {ignoreUndefined: true});
let deserialized = msgpack.decode(serialized, {ignoreUndefined: true});
console.log(original, deserialized, original === deserialized); // undefined null false

This is stated in the readme, but is there no way to retain this with an extension codec?

Reacted by Ryan Lester, Domi and Nico Mürdter

Activity

1. gfx commented on Dec 23, 2020

   

   gfx

   on Dec 23, 2020

   Member

   More actions

   Can you elaborate on your use case? In general, it's not a good idea to distinguish null and undefined in JavaScript, and supporting it with extension types could lead to extra overheads, so I'd like to know your use cases are worth adding more overheads.
2. mischnic commented on Feb 8, 2021

   

   mischnic

   on Feb 8, 2021

   Author

   More actions

   Once case that I found was where we serialize/deserialize an entry of process.env (so {key: "foo", value: process.env["foo"]}) and then check if deserialized.value !== process.env.foo (and this breaks if the value is undefined). It's easy to workaround, but I fear there might be more situations like this in the codebase that haven't surfaced yet.

   (I'm using msgpack as a browser replacement for require("v8").(de)serialize)

   Reacted by ShikChen and Bjarke Felbo
3. grantila commented on Oct 23, 2021

   

   grantila

   on Oct 23, 2021

   Contributor

   More actions

   May I suggest allowing for overriding this in a codec? Currently undefined is handled internally by this package, and you can configure whether to a) ignore it (not include it in the encoded buffer) or b) coerce it into null. I understand that this makes sense when communication between different languages, where there is no distinction between undefined and null, which is why it's nil in the spec.

   If this could instead be allowed to be overridden though, users can encode it if they want.

   My use case is to transfer an object from one place to another and maintain it exactly as is. Turning undefined into null alters the datatype, and allowing undefined to be there is different from not having it. Object.keys(obj) e.g.
4. genki commented on Jan 23, 2024

   

   genki

   on Jan 23, 2024

   Contributor

   More actions

   @gfx
   Suppose to encode the entire arguments of functions that have optional ones.
   The values of the optional arguments are not null but undefined.
   If the msgpack coearses them into null, the information that they were not supplied is lost.
   We have to provide the additional bitmask that informs which parameters were really supplied.
   The undefined means the existence of the key in the object.
   The null means the absence of the value. They can't be treated as same things.

   Reacted by Nico Mürdter
5. WebReflection commented on Feb 21, 2025

   

   WebReflection

   on Feb 21, 2025

   More actions

   deserialized.value != process.env.foo would solve this in JS land as == with null and undefined it's the only case where it's true, everything else would be false, even false itself so this was trivial to solve to start with but I wonder how comes MessagePack decided to encode undefined at all ... I can see the == used here and there, but undefined usually means not defined and even JSON got it right, it's entirely pointless to carry non defined values, especially when every other PL might have just nil or explicit None around, but never undefined.
6. gyzerok commented on Jun 26, 2026

   

   gyzerok

   on Jun 26, 2026

   More actions

   Can you elaborate on your use case? In general, it's not a good idea to distinguish null and undefined in JavaScript, and supporting it with extension types could lead to extra overheads, so I'd like to know your use cases are worth adding more overheads.

   @gfx can you elaborate on "not a good idea to distinguish"? It's 2 separate types in JavaScript. Any place that checks value === null won't work for undefined and vice-versa. Personally I think that in general people expect serialization/deserialization cycle to be loss-less. I was definitely caught off-guard with this one.

   It's okay to be opinionated about it by default. But I think at least it'd be nice to allow people to extends it with extension codec instead of not providing them any path forward.

Issue actions