GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add support for jackson field ids by brenbar · Pull Request #868 · msgpack/msgpack-java

Conversation

@brenbar

@brenbar brenbar commented Jan 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Background

Jackson core has interfaces for field ids. This is a great opportunity for msgpack, since the protocol allows for integer keys, enabling more advanced binary serialization strategies for further reduced message size.

Current implementation of msgpack-jackson only allows for coercing strings to integers. Implementing the formal interfaces will enable end-to-end map serialization with mixed string/integer keys. Other msgpack implementations already support this.

Summary of Changes

  1. Implement the formal jackson core interface for writing integer keys as field ids instead of strings.
  2. Add a helper function on the msgpack parser that can be used in a jackson KeyDeserializer to deserialize field ids.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@brenbar
Add test demonstrating field id case
534fa03
@github-actions github-actions Bot added the feature label Jan 18, 2025
brenbar
brenbar commented Jan 18, 2025
View reviewed changes
Comment thread msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
@brenbar

brenbar commented Jan 18, 2025

Copy link
Copy Markdown
Contributor Author

@komamitsu @xerial Can CI be run on this PR to show the failing test? Then I'll follow-up with the implementation changes.
komamitsu reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
brenbar added 3 commits January 19, 2025 11:01
@brenbar
Fix java 8 error
481c223
@brenbar
Add missing import
d8a3716
@brenbar
Add missing throws
757b6f9
@brenbar

brenbar commented Jan 19, 2025

Copy link
Copy Markdown
Contributor Author

@komamitsu @xerial Another CI run please 🙏
komamitsu reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
brenbar added 2 commits January 20, 2025 09:31
@brenbar
Cleanup unused imports
546d48b
@brenbar
Cleanup test
3c08b90
@brenbar

brenbar commented Jan 20, 2025

Copy link
Copy Markdown
Contributor Author

🔨 Build failed with test showcasing new edge case.

[info] Test org.msgpack.jackson.dataformat.MessagePackDataformatForFieldIdTest.testMixedKeys started
[error] Test org.msgpack.jackson.dataformat.MessagePackDataformatForFieldIdTest.testMixedKeys failed: java.lang.AssertionError: expected: java.util.HashMap<{1=one, 2=two}> but was: java.util.HashMap<{1=one, 2=two}>, took 0.258 sec
[error]     at org.msgpack.jackson.dataformat.MessagePackDataformatForFieldIdTest.testMixedKeys(MessagePackDataformatForFieldIdTest.java:105)
[error]     ...
[info] Test run org.msgpack.jackson.dataformat.MessagePackDataformatForFieldIdTest finished: 1 failed, 0 ignored, 1 total, 0.264s

Will now demonstrate passing test with implementation changes.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brenbar
Implement jackson field ids
84cbc79
@brenbar

brenbar commented Jan 20, 2025

Copy link
Copy Markdown
Contributor Author

@komamitsu @xerial This is ready for another CI run to showcase the new test passing.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
brenbar
brenbar commented Jan 20, 2025
View reviewed changes
Comment thread msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java Outdated
@Override
public void writeFieldId(long id) throws IOException
{
addKeyNode(id);

@brenbar brenbar Jan 20, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@komamitsu @xerial For backwards compatibility, I imagine we need some sort of feature flag that defaults to the prior implementation. Do you guys have any suggestions for how that flag should be defined? Should I just use the MessagePack.PackerConfig, or should I pass it directly from the MessagePackFactory to the MessagePackGenerator?

Suggested change
addKeyNode(id);
if (writeIntegerMapKeysAsStringKeys) {
super.writeFieldId(id);
} else {
addKeyNode(id);
}

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@komamitsu komamitsu Feb 2, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, for the delayed reply. PackerConfig is for the core configuration and shouldn't be used for jackson-dataformat-msgpack. I think it's okay to pass the flag like reuseResource.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@brenbar
Address feedback from @komamitsu
8247f24
@brenbar
brenbar marked this pull request as ready for review February 3, 2025 06:27
@brenbar

brenbar commented Feb 3, 2025

Copy link
Copy Markdown
Contributor Author

@komamitsu @xerial This is ready for a final draft review. I added a test to show backwards compatibility with new feature flag. ./sbt jcheckStyle and ./sbt test both pass locally for me.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
komamitsu
komamitsu reviewed Feb 4, 2025
View reviewed changes
Comment thread msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
@brenbar
Address feedback from @komamitsu
a66fc2e
@brenbar

brenbar commented Feb 9, 2025

Copy link
Copy Markdown
Contributor Author

@komamitsu Thanks for the review. All comments addressed.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@komamitsu

komamitsu commented Feb 9, 2025

Copy link
Copy Markdown
Member

@brenbar Thanks! LGTM 👍

After reviewing all the changes, writeIntegerKeysAsStringKeys feels a bit verbose? supportIntegerKeys (false by default) or something might be simpler. What do you think?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brenbar
Address feedback from @komamitsu
9d0c9a8
@brenbar

brenbar commented Feb 10, 2025

Copy link
Copy Markdown
Contributor Author

@komamitsu I think your feedback is fair 😅

Ready again for review 👍
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
komamitsu
komamitsu approved these changes Feb 11, 2025
View reviewed changes

@komamitsu komamitsu left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@komamitsu
komamitsu merged commit 1cbd05f into msgpack:main Feb 11, 2025
@komamitsu komamitsu assigned brenbar Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@komamitsu komamitsu komamitsu approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@brenbar brenbar

Labels

feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@brenbar @komamitsu
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.