GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-160699: Prevent concurrent reuse of cached SQLite statements by mangod12 · Pull Request #160706 · python/cpython

Conversation

@mangod12

@mangod12 mangod12 commented Oct 26, 2026

Copy link
Copy Markdown

Summary

Concurrent calls to execute() on a shared SQLite connection can obtain the same cached sqlite3_stmt before SQLite marks it busy. The cursor releases the GIL around SQLite operations, leaving a race where concurrent cursors can reset or use the same statement and crash the process.

Reserve cached statements before releasing the GIL. If a cached statement is already reserved, create an uncached statement for the concurrent execution. Release the reservation when the statement is reset.

Add a regression test that runs concurrent parameterized queries on a shared connection and verifies the returned BLOB contents.

Fixes gh-160699.

Verification

  • Windows debug build succeeded.
  • PCbuild/amd64/python_d.exe -m unittest test.test_sqlite3.test_dbapi: 211 passed, 2 skipped.
  • Tools/patchcheck/patchcheck.py: passed; NEWS fragment recognized.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mangod12
Prevent concurrent reuse of SQLite statements
4172afa
@mangod12
mangod12 requested a review from erlend-aasland as a code owner October 26, 2026 13:40
@bedevere-app bedevere-app Bot added the awaiting review label Oct 26, 2026
@python-cla-bot

python-cla-bot Bot commented Oct 26, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot mentioned this pull request Oct 26, 2026
sqlite3: concurrent execute() on a shared Connection crashes the process with a native access violation (threadsafety == 3) #160699
Open
@bedevere-app

bedevere-app Bot commented Oct 26, 2026

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mangod12 added 2 commits October 26, 2026 19:11
@mangod12
Add a NEWS entry for the sqlite3 threading fix
d3450fa
@mangod12
pythongh-160699: use serialized SQLite connections
cf13208
@mangod12

mangod12 commented Oct 26, 2026

Copy link
Copy Markdown
Author

CI for PR #160706 fails on all three macOS jobs in the new test_concurrent_queries_on_shared_connection: it segfaults inside sqlite3_prepare_v2 while threads share the connection. I pushed cf13208 to explicitly request SQLITE_OPEN_FULLMUTEX whenever the linked SQLite build has mutex support, while keeping the statement reservation fix. SQLite documents that flag as selecting serialized mode and notes sqlite3_threadsafe() alone does not report runtime mode: https://sqlite.org/threadsafe.html

The new CI run will tell us whether that addresses the macOS failure. I could not build CPython locally here because the Windows C toolchain is unavailable.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mangod12
pythongh-160699: retry documentation preview
76eb2d3
@mangod12

mangod12 commented Oct 26, 2026

Copy link
Copy Markdown
Author

@erlend-aasland The latest CI run for 76eb2d3 failed on macOS Intel with a hosted-runner communication error; GitHub’s annotation reports that the runner lost communication with the server, and the detailed job log is unavailable. Could you rerun that job when convenient? The other macOS jobs passed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mangod12 ChatGPT Codex Connector

mangod12 commented Oct 27, 2026

Copy link
Copy Markdown
Author

@erlend-aasland A separate blocker surfaced in the Read the Docs preview for build 35421294 (commit 76eb2d3): it fetched refs/pull/160706/head, then failed before the docs build with fatal: reference is not a tree when checking out that commit. Could you check the preview’s commit/ref resolution or trigger a fresh preview? The failure appears to happen before Sphinx runs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StanFromIreland StanFromIreland added the AI-needs-explanation AI-assisted work that needs further explanation and follow up from the contributor label Oct 28, 2026
@bedevere-app

bedevere-app Bot commented Oct 28, 2026

Copy link
Copy Markdown

Thank you for your interest in contributing to Python. This is a community-driven project, and we want it to be a welcoming place to learn and collaborate.

Contributors are responsible for reviewing and understanding what they submit, checking its claims, and being able to explain changes in their own words. This helps reviewers focus on the contribution and leaves more time to support other contributors.

This contribution appears to have been authored with AI assistance. Please explain your contribution in your own words and describe how you checked its correctness.

Please read our guidelines for using AI tools.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mangod12

mangod12 commented Oct 29, 2026

Copy link
Copy Markdown
Author

Yes it was done with help of ai. And thanks for the reply.

The issue as i understood was that concurrent execution calls on a shared connection can pick the same cached SQLite statement before sqlite3 mark it busy. Both threads can then try to use that same statement.

This fix reserves that statement with a flag ( in_use) so other cursor gets an uncached one.
Also used fullmutex so that sql lite processes on the shared connection are in order.

For regression i ran concurrent queries from 4 threads sharing connection and returned blob data were checked also that subprocesses end cleanly.

For testing i had issues initially then i installed c toolchain and ran it with help of ai.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Oct 31, 2026

Copy link
Copy Markdown
Member

Is the AI the one who suggested the fix? or would you have been able to make the fix by yourself without it?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mangod12

mangod12 commented Nov 3, 2026

Copy link
Copy Markdown
Author

Would have been able to do it myself also. Much more time and effort though. Ai did help me in the research a bit i was more easily able to get to the required approach but it was not blind ai use if you mean that just doing yes on whatever ai said.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

AI-needs-explanation AI-assisted work that needs further explanation and follow up from the contributor awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

sqlite3: concurrent execute() on a shared Connection crashes the process with a native access violation (threadsafety == 3)

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@mangod12 @picnixz @StanFromIreland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.