GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — [SYSTEMDS-3421] Add serialization support for UDF encoders by zohrehasadi00 · Pull Request #2397 · apache/systemds

Conversation

zohrehasadi00 commented Jan 13, 2026

Copy link

Copy Markdown

Contributor

Adds UDF encoder type to ColumnEncoder enum and EncoderFactory -> UDF encoders can be created/serialized.
Implements readExternal/writeExternal for ColumnEncoderUDF to persist function name and domain size.
Adds a regression test.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

zohrehasadi00 added 2 commits

January 13, 2026 10:04


          Support serialization for UDF encoders

343b0dd

Signed-off-by: Zohreh <[email protected]>


          Add serialization test for UDF encoder

4edce13

Signed-off-by: Zohreh <[email protected]>

github-project-automation Bot added this to SystemDS PR Queue

Jan 13, 2026

github-project-automation Bot moved this to In Progress in SystemDS PR Queue

Jan 13, 2026

zohrehasadi00 commented Feb 7, 2026

Copy link

Copy Markdown

Contributor Author

@janniklinde This is my first time contributing to an open-source project voluntarily. Should I do anything from my side, or will someone review the PR automatically through the queue or sth? tnx.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

janniklinde commented Feb 16, 2026

Copy link

Copy Markdown

Contributor

Thanks for the PR @zohrehasadi00. I'm not really familiar with this issue but I don't see how this fully addresses issue 3421? @Baunsgaard do you have any comments on this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Baunsgaard reviewed

Feb 16, 2026

View reviewed changes

Baunsgaard left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, i think it looks fine.

However, I think the PR currently is missing some tests for null handling of strings, because the federated site has some special handling of UDF functions that sometimes serialize null values. Could we add some more tests for different serialization scenarios?

There is some history on this issue, where I made a followup PR to merge another:

See:
#1716
and:
#1681

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

zohrehasadi00 reacted with thumbs up emoji

All reactions

👍 1 reaction

Comment thread

src/main/java/org/apache/sysds/runtime/transform/encode/EncoderFactory.java

Comment on lines -279 to +295

		return new ColumnEncoderDummycode();
		case FeatureHash:
		return new ColumnEncoderFeatureHash();
		case PassThrough:
		return new ColumnEncoderPassThrough();
		case Recode:
		return new ColumnEncoderRecode();
		case WordEmbedding:
		return new ColumnEncoderWordEmbedding();
		case BagOfWords:
		return new ColumnEncoderBagOfWords();
		default:
		throw new DMLRuntimeException("Unsupported encoder type: " + etype);
		return new ColumnEncoderDummycode();
		case FeatureHash:
		return new ColumnEncoderFeatureHash();
		case PassThrough:
		return new ColumnEncoderPassThrough();
		case Recode:
		return new ColumnEncoderRecode();
		case WordEmbedding:
		return new ColumnEncoderWordEmbedding();
		case BagOfWords:
		return new ColumnEncoderBagOfWords();
		case UDF:
		return new ColumnEncoderUDF();
		default:
		throw new DMLRuntimeException("Unsupported encoder type: " + etype);

Baunsgaard Feb 16, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the indentation is somehow changed. Should be fixed to highlight the new case you added.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

This branch has not been deployed

No deployments

Reviewers

Baunsgaard

Baunsgaard left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

SystemDS PR Queue

Status: In Progress

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.