GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-145876: Do not mask AttributeErrors raised during dictionary unpacking by serhiy-storchaka · Pull Request #145906 · python/cpython Conversation
|
| "'%.200s' object is not a mapping", | ||
| Py_TYPE(update_o)->tp_name); | ||
| PyObject *exc = _PyErr_GetRaisedException(tstate); | ||
| int has_keys = PyObject_HasAttrWithError(update_o, &_Py_ID(keys)); |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I suppose a user could be absolutely insane and delete the .keys() method of their object in the middle of throwing their error. Then this check here wouldn't really detect the original cause of the error. Ideally this check was "atomic" with our actual attempt at accessing keys. But this seems like such an edge case that the more performant check-details-only-after-error seems like the right balance of perf vs correctness.
If this API existed, I think it would be better to inspect the exc PyObject to see if it was
update_oOr, the other more heavy-handed solution would be to adjust the PyDict_Update(dict_o, update_o); line above to be a more explicit API with a result object such as
UpdateResult update_result;
PyDict_Update(dict_o, update_o, &update_result);
if (update_result.error) ...
if (update_result.error_cause = NO_KEYS) ...
but not sure about the perf, maintenance, etc costs of doing this.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Inspecting the exception object is not enough, because the user code can raise an exception which look exactly the same as when accessing the key attribute.. And we will get the original issue. We would need a special variants of PyDict_Merge() and PyMapping_Keys(). This has too much cost.
Note that dict.update() tests the existence of keys prior to calling PyMapping_Keys(), so it looks up keys twice. This adds some overhead, but this is necessary. In dictionary unpacking we can avoid such overhead.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| Py_DECREF(exc); | ||
| } | ||
| else { | ||
| _PyErr_ChainExceptions1(exc); |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I'm assuming that this C API is the thing responsible for the ...during the above exception...another exception was raised... message. Should the test cases above check that the user sees this error chain? Currently it looks like we only check for the root cause of the error.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This only happens when the second lookup of the keys attribute raises an exception different from AttributeError, while the first lookup was successful or raised an AttributeError. This is very uncommon case and in needs a special complex code for testing. I do not think that it is is worth to do.
The main effect is that if the second lookup of the keys fails, we do not swallow the exception (it can be KeyboardInterrupt). Swallowing exception is always bad.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| int has_keys = PyObject_HasAttrWithError(update_o, &_Py_ID(keys)); | ||
| if (has_keys == 0) { | ||
| _PyErr_Format(tstate, PyExc_TypeError, | ||
| "'%T' object is not a mapping", |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Flagging that this is changing the formatting string from '%.200s' to '%T', but I'm assuming that is intentional.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| "Value after ** must be a mapping, not %.200s", | ||
| Py_TYPE(kwargs)->tp_name); | ||
| PyObject *exc = _PyErr_GetRaisedException(tstate); | ||
| int has_keys = PyObject_HasAttrWithError(kwargs, &_Py_ID(keys)); |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Similar to above, I would love a more watertight/atomic way of checking the real cause of the error.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks for the responses to my review @serhiy-storchaka. It looks like you already considered all the potential concerns I raised, and you know better than me the pros/cons of the solutions. So this looks good to me! Is there anything I can do to help move this forward? Would a benchmark be useful to verify this doesn't hurt performance too much? Should I do that by adding a test case to the pyperformance repo, similar to this PR from 2 months ago? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Nothing. I only waited if there are any suggestions or objections. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Thank you! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
… unpacking (pythonGH-145906) AttributeErrors raised in keys() or __getitem__() during dictionary unpacking ({**mymapping} or func(**mymapping)) are no longer masked by TypeError.
… unpacking (pythonGH-145906) AttributeErrors raised in keys() or __getitem__() during dictionary unpacking ({**mymapping} or func(**mymapping)) are no longer masked by TypeError.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.