GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Skip uninitialized typed properties when serializing objects by nicolas-grekas · Pull Request #5396 · php/php-src

Conversation

@nicolas-grekas

@nicolas-grekas nicolas-grekas commented Apr 15, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Partially reverts 846b647: instead of throwing, this skips uninitialized typed properties when serializing objects.

This preserves the unserialize(serialize($obj)) == $obj identity.

Fixes bug https://bugs.php.net/79447

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nicolas-grekas nicolas-grekas mentioned this pull request Apr 16, 2020
[Serializer] Catch \Throwable in getCacheKey() symfony/symfony#36336
Closed
@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

cc @TysonAndre

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

As before, I'm not really sure this is the right behavior, but I'm willing to accept it in the interest of pragmatism.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas

nicolas-grekas commented Apr 16, 2020

Copy link
Copy Markdown
Contributor Author

I'm willing to accept it in the interest of pragmatism.

that's all I need :)

I was wondering: can/should we narrow this case even further and restrict to properties that have no default value only? If yes, how do we achieve this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas

nicolas-grekas commented Apr 16, 2020

Copy link
Copy Markdown
Contributor Author

I added a second commit to serialize uninitialized nullable typed properties with a default value to null. This means that such uninitialized properties will unserialize to null instead of the corresponding default value. This is a better approximation of the original state. WDYT?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TysonAndre
TysonAndre reviewed Apr 16, 2020
View reviewed changes
Comment thread ext/standard/var.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

I added a second commit to serialize uninitialized nullable typed properties with a default value to null. This means that such uninitialized properties will unserialize to null instead of the corresponding default value. This is a better approximation of the original state. WDYT?

This would be inconsistent with normal serialization behavior though. I also don't like making the behavior dependent on whether the type is nullable or not.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TysonAndre
TysonAndre reviewed Apr 16, 2020
View reviewed changes
Comment thread ext/standard/var.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

TysonAndre
TysonAndre reviewed Apr 16, 2020
View reviewed changes
Comment thread ext/standard/var.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

TysonAndre
TysonAndre reviewed Apr 16, 2020
View reviewed changes
Comment thread ext/standard/var.c Outdated

@TysonAndre TysonAndre Apr 16, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is going to return success, it might make some sense to add a placeholder such as IS_UNDEF zend_hash_add(ht, name, undef_zval) when building this, then check for IS_UNDEF when using the resulting table

Otherwise, __sleep will inconsistently fail to warn about return ['prop', 'prop']; when prop is an uninitialized typed property

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@nicolas-grekas nicolas-grekas Apr 16, 2020

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would that not break on unserialize(), since that's the point of this PR?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@nikic nikic Apr 20, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think trying to preserve that warning is worth the complications (using IS_UNDEF for this is not possible -- IS_UNDEF in hashtable means the element doesn't exist).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nicolas-grekas

nicolas-grekas commented Apr 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

This would be inconsistent with normal serialization behavior though.

How that? The normal serialization behavior is to serialize a null when __sleep() returns the name of a missing property. That's consistent to me, no?

I also don't like making the behavior dependent on whether the type is nullable or not.

Isn't this the best we can do with the current serialization format?
The alternative (the 1st commit) is to skip these properties, which means unserializing them to they default value (when the originally serialized property was uninitialized). null is closer to the original to me, thus this proposal.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TysonAndre
TysonAndre reviewed Apr 16, 2020
View reviewed changes
Comment thread ext/standard/tests/serialize/sleep_uninitialized_typed_prop.phpt Outdated

@TysonAndre TysonAndre Apr 16, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it will do something I wouldn't expect for an unset public array $notNullable = ['someValue'];

class Example {
    public array $notNullable = ['someValue'];
    public function __sleep() { return ['notNullable']; }  // also ['notNullable', 'notNullable'] should emit a notice.
}
$x = new Example();
unset($x->notNullable);
serialize($x);  // I'd expect this to throw because unserialize(serialize($x)) != $x, but this code looks like it won't
			if (Z_ISUNDEF_P(prop) || !ZEND_TYPE_ALLOW_NULL(prop_info->type)) {
				return SUCCESS;

Also, further behaviors would probably be more readable in individual test files.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
TysonAndre
TysonAndre reviewed Apr 16, 2020
View reviewed changes
Comment thread ext/standard/var.c Outdated

@TysonAndre TysonAndre Apr 16, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably document that in the case that ZEND_TYPE_ALLOW_NULL(prop_info->type), the serializer will later convert IS_UNDEF to null in the serialized value as the closest available equivalent.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

This would be inconsistent with normal serialization behavior though.

How that? The normal serialization behavior is to serialize a null when __sleep() returns the name of a missing property. That's consistent to me, no?

I mean the normal, non-sleep behavior. The premise of this change, to me, is to align the behavior of sleep and non-sleep serialization, in that both would completely ignore uninitialized properties.

I also don't like making the behavior dependent on whether the type is nullable or not.

Isn't this the best we can do with the current serialization format?
The alternative (the 1st commit) is to skip these properties, which means unserializing them to they default value (when the originally serialized property was uninitialized). null is closer to the original to me, thus this proposal.

I don't think it's really closer. I guess the angle you're going for here is that "null" also makes "isset" return false. On the other hand, the default is what you get if you don't initialize the property, one could reasonably argue that is closest to the "uninitialized" state.

If you take into account that you can only run into this special case if a) the property is nullable b) has a specified default value and c) has been explicitly unset, the special case does not seem worthwhile.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas nicolas-grekas closed this Apr 16, 2020
@nicolas-grekas
nicolas-grekas force-pushed the ser-sleep branch from 0a960d5 to 0a2fd0d Compare April 16, 2020 13:45
@nicolas-grekas

nicolas-grekas commented Apr 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Oops, bad push, reopening.

I don't think it's really closer. I guess the angle you're going for here is that "null" also makes "isset" return false. On the other hand, the default is what you get if you don't initialize the property, one could reasonably argue that is closest to the "uninitialized" state.

If you take into account that you can only run into this special case if a) the property is nullable b) has a specified default value and c) has been explicitly unset, the special case does not seem worthwhile.

OK, works for me, I removed the second commit, back to simpler behavior that doesn't depend on defaults nor nullable types.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas nicolas-grekas reopened this Apr 16, 2020
@TysonAndre

TysonAndre commented Apr 16, 2020

Copy link
Copy Markdown
Contributor

As an example of what I was referring to for non-nullable properties - this change makes sense for some use cases, but not others.

The intent of the earlier change was to make sure that unserialize(serialize()) continued to be as close as possible to the original data (and start throwing when the data would be unrepresentable)

<?php
class A {
    public int $x;
    public function __sleep() {
        return ['x'];
    }
}
class B {
    public int $x = 2;
    public function __sleep() {
        return ['x'];
    }
}
$a = new A();
$a->x = 2;
unset($a->x);
serialize($a);  // should not throw

$b = new B();
unset($b->x);
serialize($b);  // should throw instead of serializing a representation of the properties that's different from the actual properties when unserializing
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas

nicolas-grekas commented Apr 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

I see what you mean @TysonAndre. I'm going to let @nikic arbitrate this one.
Please, just one request if we go with throwing in some cases: that Error should be turned into an Exception. See my previous comment on the topic.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@TysonAndre

TysonAndre commented Apr 16, 2020

Copy link
Copy Markdown
Contributor

My personal preference would be to only do this in the case where the property default is IS_UNDEF (i.e. public int $x;, public ?stdClass $y;, etc).

That was definitely an oversight in my previous change.

For the other change (for everything in __sleep), I'd understand it, but I think this might introduce an inconsistency.
I haven't run the code, and I'll get back to this tonight, but:

  • public ?int $x = 1; would be serialized as NULL if there was no __sleep(), and with the latest revision of this change, would not be serialized at all if it was mentioned in __sleep(), which is inconsistent in a different way?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@TysonAndre

TysonAndre commented Apr 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

that Error should be turned into an Exception. See my previous comment on the topic.

I'm fine with that
Any specific exception subclass in mind? https://www.php.net/manual/en/spl.exceptions.php

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas

nicolas-grekas commented Apr 16, 2020

Copy link
Copy Markdown
Contributor Author

Not any specific in mind. RuntimeException maybe?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

Please, just one request if we go with throwing in some cases: that Error should be turned into an Exception. See my previous comment on the topic.

I think Error is already the correct exception type here: It's the same as you get when accessing an uninitialized property directly.

The issue is that you're trying to fix a Doctrine bug ... inside Symfony. If we decide that this should continue throwing, then this issue really needs to be addressed directly inside Doctrine, by fixing their __sleep implementation. If they have a throwing __sleep, that affects more than just Symfony.


The rest of this discussion seems to have circled back around to all the issue we already discussed in #5027 :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

The intent of the earlier change was to make sure that unserialize(serialize()) continued to be as close as possible to the original data (and start throwing when the data would be unrepresentable)

I think the core problem here is that this is already not true for plain serialization, no __sleep involved:

<?php
class A {
    public int $x = 2;
}
$a = new A;
unset($a->x);
var_dump(unserialize(serialize($a)));

This will return A { $a = 2 } rather than A { $a = undef }. As we discussed before, the only way to fix this problem is to actually change the serialization format to introduce a new U; type.

Given that this is already the behavior we have for non-sleep serialization, I feel like it's not overly awful to also adopt the same behavior when it comes to __sleep.

nicolas-grekas reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas

nicolas-grekas commented Apr 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

I think Error is already the correct exception type here: It's the same as you get when accessing an uninitialized property directly.

Accessing an uninitialized property directly is quite different from accessing it indirectly via unserialize(). To me that's not an Error anymore, but a failure to serialize. The function could just return false actually, that'd be fine for what it does.

The place where we serialize is barely related to Doctrine: that's a quite common code that computes a hash for an arbitrary value. If we want to write this logic in a failsafe and generic way, we should not have to skip any throwables...

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikic

nikic commented Apr 16, 2020

Copy link
Copy Markdown
Member

Accessing an uninitialized property directly is quite different from accessing it indirectly via unserialize(). To me that's not an Error anymore, but a failure to serialize. The function could just return false actually, that'd be fine for what it does.

The place where we serialize is barely related to Doctrine: that's a quite common code that computes a hash for an arbitrary value. If we want to write this logic in a failsafe and generic way, we should not have to skip any throwables...

Okay, I can see both sides of the argument here. It depends on whether one wants to view this as "your __sleep implementation is broken, go fix it" or as "this object does not support serialization at this time".

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@TysonAndre

TysonAndre commented Apr 17, 2020

Copy link
Copy Markdown
Contributor

As we discussed before, the only way to fix this problem is to actually change the serialization format to introduce a new U; type.

Yeah, I was thinking about that as serialize($data, ['serialize_undefined' => true]) (config setting name to be determined)

TysonAndre#14 starts working on the bare minimum to serialize and unserialize U; as a property just to see if it was reasonably doable, and is extremely broken and incomplete. I probably won't get around to working on it this month.

An alternative would be serialize($data, ['minimum_php_version_id' => '8.0']), but

  • Some developers might expect apps to work exactly the same way as before for serialize(unserialize()). If an option such as ['binary_format' => true] gets added to serialize(), they might want to use that without changing the way undeclared properties are serialized.
    (Not necessarily a great argument, but still an argument)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@cmb69 cmb69 added the Bug label Apr 17, 2020
nikic
nikic reviewed Apr 20, 2020
View reviewed changes

@nikic nikic left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing this in conjunction with #5413, I think we should go forward with this change.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread ext/standard/var.c Outdated

@nikic nikic Apr 20, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think trying to preserve that warning is worth the complications (using IS_UNDEF for this is not possible -- IS_UNDEF in hashtable means the element doesn't exist).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread ext/standard/var.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@nicolas-grekas
Skip uninitialized typed properties when serializing objects
c53accd
Partially reverts 846b647: instead of
throwing, this skips uninitialized typed properties when serializing objects.

This preserves the `unserialize(serialize($obj)) == $obj` identity.

Fixes bug https://bugs.php.net/79447
@nicolas-grekas
nicolas-grekas force-pushed the ser-sleep branch from a7f71aa to c53accd Compare April 20, 2020 16:51
nikic
nikic approved these changes Apr 20, 2020
View reviewed changes

@nikic nikic left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG from my side.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@php-pulls php-pulls closed this in 73d02c3 Apr 23, 2020
@nicolas-grekas nicolas-grekas mentioned this pull request Apr 23, 2020
Proxy generating __sleep with uninitialized properties (PHP7.4.2+) doctrine/common#886
Closed
@TysonAndre TysonAndre mentioned this pull request Apr 24, 2020
Skip uninitialized/unset typed properties when serializing objects in php 7.4+ igbinary/igbinary#272
Open
@nicolas-grekas
nicolas-grekas deleted the ser-sleep branch April 30, 2020 11:46
@carusogabriel carusogabriel modified the milestone: PHP 8.0 May 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TysonAndre TysonAndre TysonAndre left review comments
+1 more reviewer
@nikic nikic nikic approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

Bug

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@nicolas-grekas @nikic @TysonAndre @cmb69 @carusogabriel
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.