Description

erlend-aasland

opened on Sep 8, 2021

Issue body actions

BPO	45138
Nosy	@JelleZijlstra, @miss-islington, @erlend-aasland
PRs	bpo-45138: Expand traced SQL statements in sqlite3 trace callback #28240 bpo-45138: Fix regression in GH-28240 #31783 bpo-45138: Revert GH-28240: Expand traced SQL statements #31788

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/erlend-aasland'
closed_at = None
created_at = <Date 2021-09-08.14:05:14.995>
labels = ['extension-modules', 'type-feature', '3.11']
title = '[sqlite3] expand bound values in traced statements when possible'
updated_at = <Date 2022-03-09.17:40:07.234>
user = 'https://github.com/erlend-aasland'

bugs.python.org fields:

activity = <Date 2022-03-09.17:40:07.234>
actor = 'miss-islington'
assignee = 'erlendaasland'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2021-09-08.14:05:14.995>
creator = 'erlendaasland'
dependencies = []
files = []
hgrepos = []
issue_num = 45138
keywords = ['patch']
message_count = 6.0
messages = ['401383', '414784', '414785', '414787', '414793', '414806']
nosy_count = 3.0
nosy_names = ['JelleZijlstra', 'miss-islington', 'erlendaasland']
pr_nums = ['28240', '31783', '31788']
priority = 'low'
resolution = 'fixed'
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45138'
versions = ['Python 3.11']

Activity

1. erlend-aasland commented on Sep 8, 2021

   

   erlend-aasland

   on Sep 8, 2021

   ContributorAuthor

   More actions

   For SQLite 3.14.0 and newer, we're using the v2 trace API. This means that the trace callback receives a pointer to the sqlite3_stmt object. We can use the sqlite3_stmt pointer to retrieve expanded SQL string.

   The following statement...:
   cur.executemany("insert into t values(?)", ((v,) for v in range(3)))

   ...will produce the following traces:
   insert into t values(0)
   insert into t values(1)
   insert into t values(2)

   ...instead of:
   insert into t values(?)
   insert into t values(?)
   insert into t values(?)
2. erlend-aasland

   added

   3.11only security fixesonly security fixes

   on Sep 8, 2021
3. erlend-aasland

   self-assigned this

   on Sep 8, 2021
4. erlend-aasland

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   type-featureA feature request or enhancementA feature request or enhancement

   3.11only security fixesonly security fixes

   on Sep 8, 2021
5. erlend-aasland

   self-assigned this

   on Sep 8, 2021
6. erlend-aasland

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   type-featureA feature request or enhancementA feature request or enhancement

   on Sep 8, 2021
7. erlend-aasland

   changed the title [-][sqlite3] expand bound values in traced statements if possible[/-] [+][sqlite3] expand bound values in traced statements when possible[/+] on Sep 8, 2021
8. erlend-aasland

   changed the title [-][sqlite3] expand bound values in traced statements if possible[/-] [+][sqlite3] expand bound values in traced statements when possible[/+] on Sep 8, 2021
9. JelleZijlstra commented on Mar 9, 2022

   

   JelleZijlstra

   on Mar 9, 2022

   Member

   More actions

   New changeset d177751 by Erlend Egeberg Aasland in branch 'main':
   bpo-45138: Expand traced SQL statements in sqlite3 trace callback (GH-28240)
   d177751
10. JelleZijlstra commented on Mar 9, 2022

    

    JelleZijlstra

    on Mar 9, 2022

    Member

    More actions

    Thanks for the contribution!
11. JelleZijlstra

    closed this as completedon Mar 9, 2022
12. JelleZijlstra

    closed this as completedon Mar 9, 2022
13. 2 remaining items
14. erlend-aasland

    reopened this on Mar 9, 2022
15. erlend-aasland commented on Mar 9, 2022

    

    erlend-aasland

    on Mar 9, 2022

    ContributorAuthor

    More actions

    Ah, one of my very first contributions, gh-84498, comes back to haunt me:

    In gh-84498, we migrated from the old SQLite trace API (sqlite3_trace) to SQLite trace v2 API (sqlite3_trace_v2). #19581, which introduced this change, introduced a bug: the old trace API _implicitly_ expanded bound parameters; the new trace API does not. However, there was no tests for this behaviour, so the regression was unnoticed¹.

    So, this bpo is actually a bug fix; not a feature. It should be backported to 3.10, which contains the regression.

    I'm preparing a fix for #28240, and I'll prepare a 3.10 backport including both #19581 and its upcoming fix.

    Footnotes

    1. There has been no bug reports regarding this change in behaviour, so it seems to have gone under most people's radar. ↩
16. miss-islington commented on Mar 9, 2022

    

    miss-islington

    on Mar 9, 2022

    Contributor

    More actions

    New changeset e801e88 by Erlend Egeberg Aasland in branch 'main':
    bpo-45138: Revert #72427: Expand traced SQL statements (GH-31788)
    e801e88
17. ezio-melotti

    transferred this issue fromon Apr 10, 2022
18. erlend-aasland

    added

    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    3.10 (EOL)end of lifeend of life

    and removed

    type-featureA feature request or enhancementA feature request or enhancement

    on Apr 10, 2022
19. erlend-aasland

    changed the title [-][sqlite3] expand bound values in traced statements when possible[/-] [+]sqlite3 regression: bound values are not expanded in traced statements[/+] on Apr 29, 2022
20. erlend-aasland

    added a commit that references this issue on Apr 29, 2022

    pythongh-89301: Fix regression with bound values in traced statements

    ab35f4b
21. JelleZijlstra

    closed this as completedin #92053on May 2, 2022
22. JelleZijlstra

    added a commit that references this issue on May 2, 2022

    gh-89301: Fix regression with bound values in traced SQLite statements (

    721aa96
23. JelleZijlstra

    added a commit that references this issue on May 2, 2022

    [3.10] gh-89301: Fix regression with bound values in traced SQLite st…

    178d79a
24. erlend-aasland

    added

    topic-sqlite3

    on May 16, 2022

Issue actions