GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-112433: Add optional _align_ attribute to ctypes.Structure by monkeyman192 · Pull Request #113790 · python/cpython Conversation
|
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It is an interesting idea.
But what should be the size and the alignment of B in the following example?
class A(Structure): x: c_uint32 class B(A): _align_ = 8 y: c_uint32
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Thanks for the review! My intuition would say "8 and 8". |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
!buildbot s390x |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 2329fa3 🤖 The command will test the builders whose names match following regular expression: The builders matched are:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
https://buildbot.python.org/all/#/builders/390/builds/1350/steps/5/logs/stdio |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
In the process of making the tests run with both big and little endian versions of the from ctypes import * class AlignedUnion(BigEndianUnion): _fields_ = [ ("a", c_uint32), ("b", c_int32), ] class Main(BigEndianStructure): _fields_ = [ ("first", c_uint32), ("union", AlignedUnion), ] m = Main() If I run this on my machine (little endian), I get the following error: This seems like a bug and is the reason the last few tests look the way they do... |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
It was fixed not long time ago in #105102. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Ok great! I know that as per the requirements for PR's it's required to make each commit separate and to not rebase, but is it fine to rebase the whole branch onto the current head so that I get the fix for this? I haven't contributed before and just want to make sure I'm following the right procedure... Edit: Oh I see you already merged the code into the branch, thanks! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Don't rebase, just merge. I just merged |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Some questions and suggestions to tests. Please add more checks for offsets, alignments and sizes.
Interesting, how does setting _align_ in a Union subclass affect it?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| self.assertEqual(main.first, 7) | ||
| self.assertEqual(main.string.value, b'hello world!') | ||
| self.assertEqual( | ||
| bytes(main.string.__buffer__(inspect.BufferFlags.SIMPLE)), |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Maybe simply bytes(main.string)?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| self.assertEqual(main.string.value, b'hello world!') | ||
| self.assertEqual( | ||
| bytes(main.string.__buffer__(inspect.BufferFlags.SIMPLE)), | ||
| b'\x68\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64\x21\x00\x00\x00\x00' |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
b'hello world!\0\0\0\0' could be more readable.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
| if ubase == LittleEndianUnion: | ||
| self.assertEqual(bytes(main.union.b), b"\3\0\0\0\4\0\0\0") | ||
| else: | ||
| self.assertEqual(bytes(main.union.b), b"\0\0\0\3\0\0\0\4") |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It is simply data[8:].
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| class TestAlignedStructures(unittest.TestCase): | ||
| def test_aligned_string(self): | ||
| for base, data in ( | ||
| (LittleEndianStructure, bytearray( |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
You can also use struct.pack() to generate data. It may be more compact and more readable.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks for the thorough review! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your response @monkeyman192. I have yet few suggestions for polishing test.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| ) | ||
| self.assertEqual(bytes(main.string), b'hello world!\0\0\0\0') | ||
| self.assertEqual(Main.string.offset, 16) | ||
| self.assertEqual(Main.string.size, 16) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It is not affected by the _align_ setting, because its size was already 16. Maybe make the value field type c_char * 12? Then the _align_ setting will affect also the size of Aligned.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| ("y", c_uint32), | ||
| ("x", c_ubyte), | ||
| ("y", SomeBools), | ||
| ("z", c_uint32), |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
What if make z a byte and make SomeBools containing only 2 fields? Then without the _align_ setting Main can be packed in just 4 bytes. The _align_ setting will made larger difference.
Or just set _align_ to 8. This would cover also the next test.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| ("y", SomeBoolsTooBig), | ||
| ("z", c_uint32), | ||
| ] | ||
| with self.assertRaises(ValueError) as ctx: |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It is still not very informative test. If you want to test that the _align_ setting affects the size of Main, you can do this more directly by checking sizeof(Main). If you want to check also content, use the data of size 12.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I guess with this test I was just wanting to ensure that the appropriate error is raised when you have a Structure/Union with an alignment which will cause the size it expects to be bigger than what it is given. Rather than checking anything about the content itself.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| self.assertEqual(main.unsigned, 0xD6) | ||
| self.assertEqual(main.signed, -42) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Oh, it is difficult to test unions with different byte order. But your test will also pass if badding bytes are added before unsigned (i.e. if Main.unsigned.offset is 7). I would use all different bytes in the input data, and tested:
self.assertEqual(main.unsigned, data[0]) self.assertEqual(main.signed, data[0] - 256)
Please add also checks for alignments, offsets and sizes.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| self.assertEqual(bytes(main.string), b'hello world!\0\0\0\0') | ||
| self.assertEqual(Main.string.offset, 16) | ||
| self.assertEqual(Main.string.size, 16) | ||
| self.assertEqual(alignment(main.string), 16) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Does alignment(Main.string) work?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It does not since Main.string is a ctypes.CField which has no alignment info.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I realised that I have neither documented nor tested the case of having both |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Sorry for the ping @serhiy-storchaka just checking to see if there was anything else I needed to address with this. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
!buildbot s390x Debian |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit a7bc0fe 🤖 The command will test the builders whose names match following regular expression: The builders matched are:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Ah, I just realised that this was already in the |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.