GeezerVison

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All commit authors signed the Contributor License Agreement.

Good morning @serhiy-storchaka,
thank you very much for the review and the nice compliment 😃 .

Easy changes

I have pushed changes to address the documentation and dbm.dumb comment remarks.

Organize not being fail-safe

On the topic of reorganize() inherently being not fail-safe, I agree with you. It is my understanding that dbm.dumb does very little to be robust in general. Most notably, it relies on its __del__ method to write back, which may not always be called in case of trouble. This is why I chose this less-resource-intensive implementation over a more robust one. If we want to go for more robustness, any of these two approaches should work, although none have a 100% robustness guarantee:

• After every block is moved in the binary file, rewrite the index
  • Advantage: keep disk space usage down
  • Disadvantage: many small writes can occur.
  • Disadvantage: If crash / KeyboardInterrupt happens between moment block is moved and index is rewritten, database is corrupt
• Reorganize binary file in a temporary file and only once completed rename it to overwrite old version and rewrite index
  • Advantage: minimizes window of time during which db would be corrupt in case of crash
  • Disadvantage: uses more disk space.
  • Disadvantage: If crash / KeyboardInterrupt happens between when we replace old binary file with new one and moment when we replace old index file with new one, database is corrupt

Let me know how you think we should proceed. SQLITE and GDBM are designed to be much more robust and use the second technique to ensure failed reorganizations don't corrupt the database. Crucially, however, they handle a single binary file with both index and data, hence either everything is renamed atomically or nothing is.

Concurrency

My apologies, but I would disagree dbm.dumb supports any concurrency at all... I totally understand what you mean but for me if the readers can access deleted keys and if they aren't notified when a value is updated for an existing key the behaviour during concurrency is undefined, not limitedly supported.

Examples:

• I want to update a key with a new value. In this circumstance, __setitem__ will first check if the blocks used by the previous value are more or equal than those required by the new value. If so, it makes the change in-place. Let's say the old value occupied twice as much blocks as the new value. __setitem__ will write the new value in-place, but keep the second-half of the blocks untouched. When a reader with the old index will read, he will read as many bytes as required for the old value. So he will read the new value, but also some gibblerish left out by __setitem__. This would crash modules like shelve that expect the value to be valid pickles.
• In a similar fashion if I am updating a key with a new value larger than the old one, readers will keep reading the old value.
• We already touched upon the deletion.

Of course this problem is well-known to any developer who looked at the source code for dbm and dbm.dumb. However, I found the documentation was a bit lackluster on these problems, so I have also updated it to warn users of these limitations (see the warning under dbm.dumb.open).

One thing that was already well explained in the documentation, however:

The shelve module does not support concurrent read/write access to shelved objects. (Multiple simultaneous read accesses are safe.) When a program has a shelf open for writing, no other program should have it open for reading or writing.

Please let me know how you would like to proceed on this issue as well.

In the meantime, have a great day! 😃

Thank you @serhiy-storchaka 😄 . Just FYI, since I had merge conflicts due to the whatsnew I have merged with main between you marking as approved and me reading the message.

This is great; thanks for the proposal and implementation, @Andrea-Oliveri; thanks for managing this, @serhiy-storchaka.