GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-42972: Fully implement GC protocol for sqlite3 heap types by erlend-aasland · Pull Request #26104 · python/cpython

Conversation

@erlend-aasland

@erlend-aasland erlend-aasland commented May 13, 2021 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue42972

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Erlend E. Aasland added 6 commits May 13, 2021 19:14
Erlend E. Aasland
sqlite.Connection type now implements GC protocol
6ece632
Erlend E. Aasland
sqlite.Cursor type now implements GC protocol
8660609
Erlend E. Aasland
sqlite.Row type now implements GC protocol
13bc519
Erlend E. Aasland
sqlite.PrepareProtocol type now implements GC protocol
4519017
Erlend E. Aasland
sqlite.Statement type now implements GC protocol
20da088
Erlend E. Aasland
sqlite3.Cache and sqlite3.Node types now implements GC protocol
ca870ac
@erlend-aasland
erlend-aasland requested a review from berkerpeksag as a code owner May 13, 2021 17:28
@bedevere-bot bedevere-bot added the awaiting review label May 13, 2021
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label May 13, 2021
@erlend-aasland
erlend-aasland requested a review from pablogsal May 13, 2021 17:29
@erlend-aasland

erlend-aasland commented May 13, 2021

Copy link
Copy Markdown
Contributor Author

@pablogsal May I add skip news for this PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented May 13, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

@pablogsal:

  • I've added tp_clear functions to (almost) all types, since it is recommended by both the doc's and the dev guide. (It also improves readability, IMO.)
  • In the tp_clear functions, I've converted Py_(X)DECREF's to Py_CLEAR. It that a problem for objects where we know multiple objects have strong refs. to them?
  • Can I normalise the naming for the dealloc functions (=> drop the pysqlite_ prefix)?
  • I've added Py_VISIT's for all containers and objects that are passed to us from "Py space"
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Erlend E. Aasland added 2 commits May 13, 2021 22:13
Erlend E. Aasland
Remove unneeded self->db = NULL
f5d9873
Erlend E. Aasland
Get rid of unneeded cast in cursor_dealloc
34928ed
@erlend-aasland erlend-aasland changed the title bpo-42972: implement full GC support for sqlite3 heap types bpo-42972: Fully implement GC protocol for sqlite3 heap types May 14, 2021
@erlend-aasland erlend-aasland added the needs backport to 3.10 label May 14, 2021
@pablogsal

pablogsal commented May 17, 2021

Copy link
Copy Markdown
Member

Will review this as soon as possible, but ping me if I haven't done this week :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented May 17, 2021

Copy link
Copy Markdown
Contributor Author

Will review this as soon as possible, but ping me if I haven't done this week :)

No stress; I'm trying to figure out a way to reliably test this :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented May 19, 2021

Copy link
Copy Markdown
Member
  • In the tp_clear functions, I've converted Py_(X)DECREF's to Py_CLEAR. It that a problem for objects where we know multiple objects have strong refs. to them?

Not sure I understand the question correctly, but in general Py_CLEAR is always better. It is the recommended and safe way to clear data attributes of arbitrary types while decrementing their reference counts. If you were to call Py_XDECREF() instead on the attribute before setting it to NULL, there is a possibility that the attribute’s destructor would call back into code that reads the attribute again (especially if there is a reference cycle).

  • Can I normalise the naming for the dealloc functions (=> drop the pysqlite_ prefix)?

Go ahead!

  • I've added Py_VISIT's for all containers and objects that are passed to us from "Py space"

👍

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

shihai1991
shihai1991 reviewed May 23, 2021
View reviewed changes
Comment thread Modules/_sqlite/cache.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_sqlite/connection.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@shihai1991 shihai1991 added the skip news label May 23, 2021
@erlend-aasland

erlend-aasland commented May 23, 2021

Copy link
Copy Markdown
Contributor Author

Not sure I understand the question correctly, but in general Py_CLEAR is always better.

No worries, I was mixing up some ref. count concepts. You can disregard that question :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Erlend E. Aasland added 2 commits May 23, 2021 17:42
Erlend E. Aasland
Address review: visit node->key, node->data, and con->isolation_level
590ff8e
Erlend E. Aasland
Harden cache clear
b080ff7
@erlend-aasland
erlend-aasland force-pushed the bpo-42972/sqlite branch from 65d5b9e to b080ff7 Compare May 24, 2021 07:08
Erlend E. Aasland
Normalize connection_dealloc naming
2bcfa29
@erlend-aasland

erlend-aasland commented May 24, 2021

Copy link
Copy Markdown
Contributor Author

@pablogsal Would you mind reviewing the changes? I cannot come up with a way to reliably test this; I guess that would have been easier if sqlite3 had a module state and used multi-phase init (long way to get there).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland erlend-aasland mentioned this pull request May 24, 2021
bpo-42862: Use functools.lru_cache iso. _sqlite.Cache in sqlite3 module #24203
Merged
pablogsal
pablogsal reviewed May 24, 2021
View reviewed changes
Comment thread Modules/_sqlite/prepare_protocol.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

pablogsal
pablogsal reviewed May 24, 2021
View reviewed changes
Comment thread Modules/_sqlite/row.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

pablogsal
pablogsal reviewed May 24, 2021
View reviewed changes
Comment thread Modules/_sqlite/statement.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented May 24, 2021

Copy link
Copy Markdown
Member

Can you also make a refleak run?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented May 25, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Can you also make a refleak run?

Yep! That's a normal buildbot run, right, or did you have something else in mind? I've already done a test -R : run locally. @pablogsal

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Erlend E. Aasland added 3 commits May 25, 2021 08:36
Erlend E. Aasland
Address review: visit row PyObjects for consistency
f8eb733
Erlend E. Aasland
Normalize statement dealloc naming
ab4c6e6
Erlend E. Aasland
Clear sqlite3 stmt resources in stmt_clear()
926787d
@pablogsal

pablogsal commented May 25, 2021

Copy link
Copy Markdown
Member

Thanks a lot for the patience @erlend-aasland, this is very close to be ready for lending. The GC changes are always tricky so we need to be sure to get them right :)

erlend-aasland reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented May 25, 2021

Copy link
Copy Markdown
Contributor Author

Thanks a lot for the patience @erlend-aasland, this is very close to be ready for lending. The GC changes are always tricky so we need to be sure to get them right :)

No problem, I fully agree :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 25, 2021
@bedevere-bot

bedevere-bot commented May 25, 2021

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 926787d 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 25, 2021
pablogsal
pablogsal approved these changes May 25, 2021
View reviewed changes

@pablogsal pablogsal left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Great job!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with hooray emoji
All reactions
  • 🎉 1 reaction
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels May 25, 2021
@pablogsal
pablogsal merged commit d3c277a into python:main May 25, 2021
@miss-islington

miss-islington commented May 25, 2021

Copy link
Copy Markdown
Contributor

Thanks @erlend-aasland for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label May 25, 2021
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 25, 2021
Erlend Egeberg Aasland @miss-islington
bpo-42972: Fully implement GC protocol for sqlite3 heap types (python…
da7a8d0
…GH-26104)

(cherry picked from commit d3c277a)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
@bedevere-bot

bedevere-bot commented May 25, 2021

Copy link
Copy Markdown

GH-26361 is a backport of this pull request to the 3.10 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 label May 25, 2021
@erlend-aasland

erlend-aasland commented May 25, 2021

Copy link
Copy Markdown
Contributor Author

Great job!

Fantastic, thank you so much for helping out!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland
erlend-aasland deleted the bpo-42972/sqlite branch May 25, 2021 17:46
miss-islington added a commit that referenced this pull request May 25, 2021
@miss-islington Erlend Egeberg Aasland
bpo-42972: Fully implement GC protocol for sqlite3 heap types (GH-26104)
e8d9df0
(cherry picked from commit d3c277a)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
@erlend-aasland erlend-aasland mentioned this pull request May 29, 2021
bpo-42972: Fix sqlite3 traverse/clear #26452
Merged
@pablogsal pablogsal mentioned this pull request Jul 18, 2021
[3.10] Correct the order of regen-abidump #27228
Closed
@erlend-aasland erlend-aasland mentioned this pull request Aug 20, 2021
bpo-24905: Support BLOB incremental I/O in sqlite module #271
Closed
@vstinner vstinner mentioned this pull request Jun 9, 2022
[C API] Heap types (PyType_FromSpec) must fully implement the GC protocol #87138
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shihai1991 shihai1991 shihai1991 left review comments
@pablogsal pablogsal pablogsal approved these changes
@berkerpeksag berkerpeksag Awaiting requested review from berkerpeksag

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@erlend-aasland @pablogsal @bedevere-bot @miss-islington @shihai1991 @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.