GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-34080: Fix a memory leak in the compiler. by serhiy-storchaka · Pull Request #8222 · python/cpython

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Jul 10, 2018 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

https://bugs.python.org/issue34080

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@serhiy-storchaka
bpo-34080: Fix a memory leak in the compiler.
e628d8b
@serhiy-storchaka serhiy-storchaka added type-bug An unexpected behavior, bug, or error needs backport to 3.6 labels Jul 10, 2018
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jul 10, 2018
@bedevere-bot bedevere-bot added the awaiting merge label Jul 10, 2018
@zhangyangyu

zhangyangyu commented Jul 10, 2018

Copy link
Copy Markdown
Member

In parsetok.c, err_ret->text is assigned a not malloced string, with Barry as BDFL ..., does it matter?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@serhiy-storchaka

serhiy-storchaka commented Jul 10, 2018

Copy link
Copy Markdown
Member Author

Interesting. This looks as a bug, but it is not related to this PR, because it sets err_ret->error = E_SYNTAX, while this PR fixes a case for err_ret->error = E_ERROR. Please open a separate issue for that bug.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
zhangyangyu
zhangyangyu approved these changes Jul 10, 2018
View reviewed changes
vstinner
vstinner requested changes Jul 10, 2018
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IHMO err_input() is fine, it shouldn't clear err->msg if err->error == E_ERROR. I would suggest to fix err_clear() instead:

diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index bcd1ca931d..1377be3cd5 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -1324,6 +1324,10 @@ static void
 err_free(perrdetail *err)
 {
     Py_CLEAR(err->filename);
+    if (err->text != NULL) {
+        PyObject_FREE(err->text);
+        err->text = NULL;
+    }
 }
 
 /* Set the error appropriate to the given input error code (see errcode.h) */

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot removed the awaiting merge label Jul 10, 2018
@bedevere-bot

bedevere-bot commented Jul 10, 2018

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot added the awaiting changes label Jul 10, 2018
@vstinner

vstinner commented Jul 10, 2018

Copy link
Copy Markdown
Member

I would suggest to fix err_clear() instead: (...)

FYI "./python -m test -j0 -R 3:3 test_compile test_grammar" pass with this change, but then the following issue should be addressed as well?

"In parsetok.c, err_ret->text is assigned a not malloced string, with Barry as BDFL ..., does it matter?"
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@zhangyangyu

zhangyangyu commented Jul 11, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

err_input is only called when the parsing operation fails. err_free is always called no matter the operation succeeds or fails. err->text has value only when error occurs while err->filename is always initialized. That may be why the cleaning logic is put in err_input. Anyway, no matter where it is put, I think we can remove the NULL test since PyObject_FREE can handle it itself.

@vstinner It's issue34084. Add you to the nosy list.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Jul 11, 2018

Copy link
Copy Markdown
Member

I wrote PR #8242 to implement my proposed fix and fix also the issue spotted by @zhangyangyu.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@serhiy-storchaka serhiy-storchaka mentioned this pull request Jul 11, 2018
bpo-34080: Fix memory leak on parsing error #8242
Closed
@vstinner
vstinner merged commit 993030a into python:master Jul 11, 2018
@bedevere-bot bedevere-bot removed the awaiting changes label Jul 11, 2018
@miss-islington

miss-islington commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

Thanks @serhiy-storchaka for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 2.7, 3.6, 3.7.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Jul 11, 2018

Copy link
Copy Markdown

GH-8256 is a backport of this pull request to the 3.7 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed needs backport to 3.7 labels Jul 11, 2018
@bedevere-bot

bedevere-bot commented Jul 11, 2018

Copy link
Copy Markdown

GH-8257 is a backport of this pull request to the 3.6 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 11, 2018
@serhiy-storchaka @miss-islington
bpo-34080: Fix a memory leak in the compiler. (pythonGH-8222)
9ccaf60 
(cherry picked from commit 993030a)

Co-authored-by: Serhiy Storchaka <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 11, 2018
@serhiy-storchaka @miss-islington
bpo-34080: Fix a memory leak in the compiler. (pythonGH-8222)
a283613 
(cherry picked from commit 993030a)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 2.7 label Jul 11, 2018
@bedevere-bot

bedevere-bot commented Jul 11, 2018

Copy link
Copy Markdown

GH-8258 is a backport of this pull request to the 2.7 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 11, 2018
@serhiy-storchaka @miss-islington
bpo-34080: Fix a memory leak in the compiler. (pythonGH-8222)
25ec40e 
(cherry picked from commit 993030a)

Co-authored-by: Serhiy Storchaka <[email protected]>
miss-islington added a commit that referenced this pull request Jul 11, 2018
@miss-islington @serhiy-storchaka
bpo-34080: Fix a memory leak in the compiler. (GH-8222)
a45fa39 
(cherry picked from commit 993030a)

Co-authored-by: Serhiy Storchaka <[email protected]>
miss-islington added a commit that referenced this pull request Jul 11, 2018
@miss-islington @serhiy-storchaka
bpo-34080: Fix a memory leak in the compiler. (GH-8222)
9beed0c 
(cherry picked from commit 993030a)

Co-authored-by: Serhiy Storchaka <[email protected]>
vstinner pushed a commit that referenced this pull request Jul 11, 2018
@miss-islington @serhiy-storchaka @vstinner
bpo-34080: Fix a memory leak in the compiler. (GH-8222) (GH-8257)
3b06285 
(cherry picked from commit 993030a)

Co-authored-by: Serhiy Storchaka <[email protected]>
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Jul 14, 2018
@CuriousLearner
Merge branch 'master' into bitwise_docs
11c1706 
* master: (2633 commits)
  bpo-34087: Fix buffer overflow in int(s) and similar functions (pythonGH-8274)
  bpo-34108: Fix double carriage return in 2to3 on Windows (python#8271)
  bpo-4260: Document that ctypes.xFUNCTYPE are decorators (pythonGH-7924)
  bpo-33723: Fix test_time.test_thread_time() (pythonGH-8267)
  bpo-33967: Remove use of deprecated assertRaisesRegexp() (pythonGH-8261)
  bpo-34080: Fix a memory leak in the compiler. (pythonGH-8222)
  Enable GUI testing on Travis Linux builds via Xvfb (pythonGH-7887)
  bpo-23927: Make getargs.c skipitem() skipping 'w*'. (pythonGH-8192)
  bpo-33648: Remove PY_WARN_ON_C_LOCALE (pythonGH-7114)
  bpo-34092, test_logging: increase SMTPHandlerTest timeout (pythonGH-8245)
  Simplify __all__ in multiprocessing (pythonGH-6856)
  bpo-34083: Update dict order in Functional HOWTO (pythonGH-8230)
  Doc: Point to Simple statements section instead of PEP (pythonGH-8238)
  bpo-29442: Replace optparse with argparse in setup.py (pythonGH-139)
  bpo-33597: Add What's New for PyGC_Head (pythonGH-8236)
  Dataclasses: Fix example on 30.6.8, add method should receive a list rather than an integer. (pythonGH-8038)
  Fix documentation for input and output tutorial (pythonGH-8231)
  bpo-34009: Expand on platform support changes (pythonGH-8022)
  Factor-out two substantially identical code blocks. (pythonGH-8219)
  bpo-34031: fix incorrect usage of self.fail in two tests (pythonGH-8091)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Jul 15, 2018
@CuriousLearner
Merge branch 'master' into fix-issue8525
fdc3a3d 
* master: (1159 commits)
  bpo-34087: Fix buffer overflow in int(s) and similar functions (pythonGH-8274)
  bpo-34108: Fix double carriage return in 2to3 on Windows (python#8271)
  bpo-4260: Document that ctypes.xFUNCTYPE are decorators (pythonGH-7924)
  bpo-33723: Fix test_time.test_thread_time() (pythonGH-8267)
  bpo-33967: Remove use of deprecated assertRaisesRegexp() (pythonGH-8261)
  bpo-34080: Fix a memory leak in the compiler. (pythonGH-8222)
  Enable GUI testing on Travis Linux builds via Xvfb (pythonGH-7887)
  bpo-23927: Make getargs.c skipitem() skipping 'w*'. (pythonGH-8192)
  bpo-33648: Remove PY_WARN_ON_C_LOCALE (pythonGH-7114)
  bpo-34092, test_logging: increase SMTPHandlerTest timeout (pythonGH-8245)
  Simplify __all__ in multiprocessing (pythonGH-6856)
  bpo-34083: Update dict order in Functional HOWTO (pythonGH-8230)
  Doc: Point to Simple statements section instead of PEP (pythonGH-8238)
  bpo-29442: Replace optparse with argparse in setup.py (pythonGH-139)
  bpo-33597: Add What's New for PyGC_Head (pythonGH-8236)
  Dataclasses: Fix example on 30.6.8, add method should receive a list rather than an integer. (pythonGH-8038)
  Fix documentation for input and output tutorial (pythonGH-8231)
  bpo-34009: Expand on platform support changes (pythonGH-8022)
  Factor-out two substantially identical code blocks. (pythonGH-8219)
  bpo-34031: fix incorrect usage of self.fail in two tests (pythonGH-8091)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Jul 21, 2018
@CuriousLearner
Merge branch 'master' of github.com:CuriousLearner/cpython into fix-i…
7b039a0 
…ssue-33014

* 'master' of github.com:CuriousLearner/cpython: (722 commits)
  bpo-34087: Fix buffer overflow in int(s) and similar functions (pythonGH-8274)
  bpo-34108: Fix double carriage return in 2to3 on Windows (python#8271)
  bpo-4260: Document that ctypes.xFUNCTYPE are decorators (pythonGH-7924)
  bpo-33723: Fix test_time.test_thread_time() (pythonGH-8267)
  bpo-33967: Remove use of deprecated assertRaisesRegexp() (pythonGH-8261)
  bpo-34080: Fix a memory leak in the compiler. (pythonGH-8222)
  Enable GUI testing on Travis Linux builds via Xvfb (pythonGH-7887)
  bpo-23927: Make getargs.c skipitem() skipping 'w*'. (pythonGH-8192)
  bpo-33648: Remove PY_WARN_ON_C_LOCALE (pythonGH-7114)
  bpo-34092, test_logging: increase SMTPHandlerTest timeout (pythonGH-8245)
  Simplify __all__ in multiprocessing (pythonGH-6856)
  bpo-34083: Update dict order in Functional HOWTO (pythonGH-8230)
  Doc: Point to Simple statements section instead of PEP (pythonGH-8238)
  bpo-29442: Replace optparse with argparse in setup.py (pythonGH-139)
  bpo-33597: Add What's New for PyGC_Head (pythonGH-8236)
  Dataclasses: Fix example on 30.6.8, add method should receive a list rather than an integer. (pythonGH-8038)
  Fix documentation for input and output tutorial (pythonGH-8231)
  bpo-34009: Expand on platform support changes (pythonGH-8022)
  Factor-out two substantially identical code blocks. (pythonGH-8219)
  bpo-34031: fix incorrect usage of self.fail in two tests (pythonGH-8091)
  ...
@serhiy-storchaka
serhiy-storchaka deleted the tokenizer-error-memory-leak branch November 6, 2018 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner requested changes
@zhangyangyu zhangyangyu zhangyangyu approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

type-bug An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@serhiy-storchaka @zhangyangyu @bedevere-bot @vstinner @miss-islington @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.