GeezerVison is showing github.com — dumbdbm should not commit if in read mode · Issue #73033 · python/cpython
dumbdbm should not commit if in read mode #73033Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-12-07.09:12:22.519>
created_at = <Date 2016-12-01.03:50:06.052>
labels = ['3.7', 'type-feature', 'library']
title = 'dumbdbm should not commit if in read mode'
updated_at = <Date 2017-03-31.16:36:22.839>
user = 'https://github.com/joanthanng' bugs.python.org fields:
activity = <Date 2017-03-31.16:36:22.839>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-12-07.09:12:22.519>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2016-12-01.03:50:06.052>
creator = 'Jonathan Ng'
dependencies = []
files = ['45722']
hgrepos = []
issue_num = 28847
keywords = ['patch']
message_count = 13.0
messages = ['282133', '282158', '282166', '282167', '282170', '282218', '282219', '282264', '282265', '282266', '282269', '282271', '282602']
nosy_count = 4.0
nosy_names = ['python-dev', 'martin.panter', 'serhiy.storchaka', 'Jonathan Ng']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28847'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7'] Activityjoanthanng commented on Dec 1, 2016 MannequinAuthor More actions Or at the very least, if there is an OSError in _update, an error should be raised instead of ignoring this error.
In the current state of the code, if there was an OSError while reading the dirfile, the dirfile would be overwritten with a blank file when it is closed. joanthanng changed the title [-]dumbdbm should not commit if[/-] [+]dumbdbm should not commit if in read mode[/+] on Dec 1, 2016serhiy-storchaka commented on Dec 1, 2016 Do you have concrete example when not ignoring an OSError in _update causes an issue? This is needed for writing tests. joanthanng commented on Dec 1, 2016 MannequinAuthor More actions I'm not sure how to create an OSError.
But perhaps something like this:
'''
from dbm import dumb
import os
db = dumb.open('temp', flag='n')
db['foo'] = 'bar'
db.close()
db = dumb.open('temp', flag='r')
print(len(db.keys()))
db.close
os.rename('temp.dir', 'temp_.dir') # simulates OSError
db = dumb.open('temp', flag='r')
os.rename('temp_.dir', 'temp.dir')
db.close()
db = dumb.open('temp', flag='r')
assert len(db.keys()) > 0
'''serhiy-storchaka commented on Dec 1, 2016 This example is too artificial.
But there is a real issue: opening read-only files in read mode. Currently this causes a PermissionError on closing.
For backward compatibility flags 'r' and 'w' are ignored. I.e. opening with 'r' and 'w' creates a file if it is not existing, and opening with 'r' allows modifying the database. Since 3.6 this emits deprecation warnings (bpo-21708). In future versions this will be an error.
Proposed patch makes two changes:
-
The index file no longer written if the database was not modified. This increases performance and adds a support of read-only files.
-
A deprecation warning is raised when the index file is absent in 'r' and 'w' modes. In future versions this will be an error.
May be the first change can be backported. joanthanng commented on Dec 1, 2016 MannequinAuthor More actions #1 makes sense to be backported.
On Thu, Dec 1, 2016 at 8:41 PM, Serhiy Storchaka <[email protected]>
wrote:
Serhiy Storchaka added the comment:
This example is too artificial.
But there is a real issue: opening read-only files in read mode. Currently
this causes a PermissionError on closing.
For backward compatibility flags 'r' and 'w' are ignored. I.e. opening
with 'r' and 'w' creates a file if it is not existing, and opening with 'r'
allows modifying the database. Since 3.6 this emits deprecation warnings
(bpo-21708). In future versions this will be an error.
Proposed patch makes two changes:
-
The index file no longer written if the database was not modified. This
increases performance and adds a support of read-only files.
-
A deprecation warning is raised when the index file is absent in 'r'
and 'w' modes. In future versions this will be an error.
May be the first change can be backported.
----------
keywords: +patch
stage: -> patch review
type: behavior -> enhancement
versions: +Python 3.7
Added file: http://bugs.python.org/file45722/dbm_dumb_readonly.patch
Python tracker <[email protected]>
<http://bugs.python.org/issue28847\>
python-dev commented on Dec 2, 2016 serhiy-storchaka commented on Dec 2, 2016 The first part is committed in 2.7. I'll commit it in 3.5-3.7 after releasing 3.6.0. vadmium commented on Dec 3, 2016 python-dev commented on Dec 3, 2016 vadmium commented on Dec 3, 2016 http://buildbot.python.org/all/builders/AMD64%20Windows8%202.7/builds/9/steps/test/logs/stdio
======================================================================
ERROR: test_readonly_files (test.test_dumbdbm.DumbDBMTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_dumbdbm.py", line 190, in test_readonly_files
test_support.rmtree(dir)
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_support.py", line 289, in rmtree
_rmtree(path)
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_support.py", line 245, in _rmtree
_waitfor(_rmtree_inner, path, waitall=True)
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_support.py", line 199, in _waitfor
func(pathname)
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_support.py", line 244, in _rmtree_inner
_force_run(path, os.unlink, fullname)
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_support.py", line 194, in _force_run
return func(*args)
WindowsError: [Error 5] Access is denied: '@test_3796_tmp\\db.dat'
----------------------------------------------------------------------
'test_dumbdbm' left behind directory '@test_3796_tmp'
. . .
test test_dumbdbm crashed -- <type 'exceptions.WindowsError'>: [Error 5] Access is denied: '@test_3796_tmp\\db.dat'
Traceback (most recent call last):
File "D:\buildarea\2.7.bolen-windows8\build\PCbuild\..\Lib\test\regrtest.py", line 948, in runtest_inner
test_time = time.time() - start_time
File "D:\buildarea\2.7.bolen-windows8\build\PCbuild\..\Lib\test\regrtest.py", line 899, in __exit__
restore(original)
File "D:\buildarea\2.7.bolen-windows8\build\PCbuild\..\Lib\test\regrtest.py", line 876, in restore_files
test_support.rmtree(fn)
. . .
'test_dumbdbm' left behind directory '@test_3796_tmp' and it couldn't be removed: [Error 5] Access is denied: '@test_3796_tmp\\db.dat'
Traceback (most recent call last):
File "D:\buildarea\2.7.bolen-windows8\build\PCbuild\..\Lib\test\regrtest.py", line 1679, in <module>
main()
File "D:\buildarea\2.7.bolen-windows8\build\lib\contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "D:\buildarea\2.7.bolen-windows8\build\lib\test\test_support.py", line 765, in temp_cwd
rmtree(name)python-dev commented on Dec 3, 2016 serhiy-storchaka commented on Dec 3, 2016 Thanks Martin for pointing on this. Now buildbots are green. python-dev commented on Dec 7, 2016
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|