GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Upgrade dbm, implement sqlite3 autocommit by youknowone · Pull Request #6616 · RustPython/RustPython

Upgrade dbm, implement sqlite3 autocommit #6616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 3 commits into RustPython:main from youknowone:dbm
Jan 1, 2026

Uh oh!

There was an error while loading. Please reload this page.
Merged

Upgrade dbm, implement sqlite3 autocommit #6616

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8102e89
sqlite autocommit
youknowone Jan 1, 2026
16666bd
Update dbm
youknowone Dec 31, 2025
8d70096
fix
youknowone Jan 1, 2026
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view

There are no files selected for viewing

21 changes: 13 additions & 8 deletions 21 Lib/dbm/__init__.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import dbm
d = dbm.open(file, 'w', 0o666)

The returned object is a dbm.gnu, dbm.ndbm or dbm.dumb object, dependent on the
The returned object is a dbm.sqlite3, dbm.gnu, dbm.ndbm or dbm.dumb database object, dependent on the
type of database being opened (determined by the whichdb function) in the case
of an existing dbm. If the dbm does not exist and the create or new flag ('c'
or 'n') was specified, the dbm type will be determined by the availability of
Expand Down Expand Up @@ -38,7 +38,7 @@
class error(Exception):
pass

_names = ['dbm.gnu', 'dbm.ndbm', 'dbm.dumb']
_names = ['dbm.sqlite3', 'dbm.gnu', 'dbm.ndbm', 'dbm.dumb']
_defaultmod = None
_modules = {}

Expand Down Expand Up @@ -109,17 +109,18 @@ def whichdb(filename):
"""

# Check for ndbm first -- this has a .pag and a .dir file
filename = os.fsencode(filename)
try:
f = io.open(filename + ".pag", "rb")
f = io.open(filename + b".pag", "rb")
f.close()
f = io.open(filename + ".dir", "rb")
f = io.open(filename + b".dir", "rb")
f.close()
return "dbm.ndbm"
except OSError:
# some dbm emulations based on Berkeley DB generate a .db file
# some do not, but they should be caught by the bsd checks
try:
f = io.open(filename + ".db", "rb")
f = io.open(filename + b".db", "rb")
f.close()
# guarantee we can actually open the file using dbm
# kind of overkill, but since we are dealing with emulations
Expand All @@ -134,12 +135,12 @@ def whichdb(filename):
# Check for dumbdbm next -- this has a .dir and a .dat file
try:
# First check for presence of files
os.stat(filename + ".dat")
size = os.stat(filename + ".dir").st_size
os.stat(filename + b".dat")
size = os.stat(filename + b".dir").st_size
# dumbdbm files with no keys are empty
if size == 0:
return "dbm.dumb"
f = io.open(filename + ".dir", "rb")
f = io.open(filename + b".dir", "rb")
try:
if f.read(1) in (b"'", b'"'):
return "dbm.dumb"
Expand All @@ -163,6 +164,10 @@ def whichdb(filename):
if len(s) != 4:
return ""

# Check for SQLite3 header string.
if s16 == b"SQLite format 3\0":
return "dbm.sqlite3"

# Convert to 4-byte int in native byte order -- return "" if impossible
try:
(magic,) = struct.unpack("=l", s)
Expand Down
11 changes: 7 additions & 4 deletions 11 Lib/dbm/dumb.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ class _Database(collections.abc.MutableMapping):
_io = _io # for _commit()

def __init__(self, filebasename, mode, flag='c'):
filebasename = self._os.fsencode(filebasename)
self._mode = mode
self._readonly = (flag == 'r')

Expand All @@ -54,14 +55,14 @@ def __init__(self, filebasename, mode, flag='c'):
# where key is the string key, pos is the offset into the dat
# file of the associated value's first byte, and siz is the number
# of bytes in the associated value.
self._dirfile = filebasename + '.dir'
self._dirfile = filebasename + b'.dir'

# The data file is a binary file pointed into by the directory
# file, and holds the values associated with keys. Each value
# begins at a _BLOCKSIZE-aligned byte offset, and is a raw
# binary 8-bit string value.
self._datfile = filebasename + '.dat'
self._bakfile = filebasename + '.bak'
self._datfile = filebasename + b'.dat'
self._bakfile = filebasename + b'.bak'

# The index is an in-memory dict, mirroring the directory file.
self._index = None # maps keys to (pos, siz) pairs
Expand Down Expand Up @@ -97,7 +98,8 @@ def _update(self, flag):
except OSError:
if flag not in ('c', 'n'):
raise
self._modified = True
with self._io.open(self._dirfile, 'w', encoding="Latin-1") as f:
self._chmod(self._dirfile)
else:
with f:
for line in f:
Expand Down Expand Up @@ -133,6 +135,7 @@ def _commit(self):
# position; UTF-8, though, does care sometimes.
entry = "%r, %r\n" % (key.decode('Latin-1'), pos_and_siz_pair)
f.write(entry)
self._modified = False

sync = _commit

Expand Down
3 changes: 3 additions & 0 deletions 3 Lib/dbm/gnu.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"""Provide the _gdbm module as a dbm submodule."""

from _gdbm import *
3 changes: 3 additions & 0 deletions 3 Lib/dbm/ndbm.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"""Provide the _dbm module as a dbm submodule."""

from _dbm import *
144 changes: 144 additions & 0 deletions 144 Lib/dbm/sqlite3.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
import os
import sqlite3
from pathlib import Path
from contextlib import suppress, closing
from collections.abc import MutableMapping

BUILD_TABLE = """
CREATE TABLE IF NOT EXISTS Dict (
key BLOB UNIQUE NOT NULL,
value BLOB NOT NULL
)
"""
GET_SIZE = "SELECT COUNT (key) FROM Dict"
LOOKUP_KEY = "SELECT value FROM Dict WHERE key = CAST(? AS BLOB)"
STORE_KV = "REPLACE INTO Dict (key, value) VALUES (CAST(? AS BLOB), CAST(? AS BLOB))"
DELETE_KEY = "DELETE FROM Dict WHERE key = CAST(? AS BLOB)"
ITER_KEYS = "SELECT key FROM Dict"


class error(OSError):
pass


_ERR_CLOSED = "DBM object has already been closed"
_ERR_REINIT = "DBM object does not support reinitialization"


def _normalize_uri(path):
path = Path(path)
uri = path.absolute().as_uri()
while "//" in uri:
uri = uri.replace("//", "/")
return uri


class _Database(MutableMapping):

def __init__(self, path, /, *, flag, mode):
if hasattr(self, "_cx"):
raise error(_ERR_REINIT)

path = os.fsdecode(path)
match flag:
case "r":
flag = "ro"
case "w":
flag = "rw"
case "c":
flag = "rwc"
Path(path).touch(mode=mode, exist_ok=True)
case "n":
flag = "rwc"
Path(path).unlink(missing_ok=True)
Path(path).touch(mode=mode)
case _:
raise ValueError("Flag must be one of 'r', 'w', 'c', or 'n', "
f"not {flag!r}")

# We use the URI format when opening the database.
uri = _normalize_uri(path)
uri = f"{uri}?mode={flag}"
if flag == "ro":
# Add immutable=1 to allow read-only SQLite access even if wal/shm missing
uri += "&immutable=1"

try:
self._cx = sqlite3.connect(uri, autocommit=True, uri=True)
except sqlite3.Error as exc:
raise error(str(exc))

if flag != "ro":
# This is an optimization only; it's ok if it fails.
with suppress(sqlite3.OperationalError):
self._cx.execute("PRAGMA journal_mode = wal")

if flag == "rwc":
self._execute(BUILD_TABLE)

def _execute(self, *args, **kwargs):
if not self._cx:
raise error(_ERR_CLOSED)
try:
return closing(self._cx.execute(*args, **kwargs))
except sqlite3.Error as exc:
raise error(str(exc))

def __len__(self):
with self._execute(GET_SIZE) as cu:
row = cu.fetchone()
return row[0]

def __getitem__(self, key):
with self._execute(LOOKUP_KEY, (key,)) as cu:
row = cu.fetchone()
if not row:
raise KeyError(key)
return row[0]

def __setitem__(self, key, value):
self._execute(STORE_KV, (key, value))

def __delitem__(self, key):
with self._execute(DELETE_KEY, (key,)) as cu:
if not cu.rowcount:
raise KeyError(key)

def __iter__(self):
try:
with self._execute(ITER_KEYS) as cu:
for row in cu:
yield row[0]
except sqlite3.Error as exc:
raise error(str(exc))

def close(self):
if self._cx:
self._cx.close()
self._cx = None

def keys(self):
return list(super().keys())

def __enter__(self):
return self

def __exit__(self, *args):
self.close()


def open(filename, /, flag="r", mode=0o666):
"""Open a dbm.sqlite3 database and return the dbm object.

The 'filename' parameter is the name of the database file.

The optional 'flag' parameter can be one of ...:
'r' (default): open an existing database for read only access
'w': open an existing database for read/write access
'c': create a database if it does not exist; open for read/write access
'n': always create a new, empty database; open for read/write access

The optional 'mode' parameter is the Unix file access mode of the database;
only used when creating a new database. Default: 0o666.
"""
return _Database(filename, flag=flag, mode=mode)
Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.