GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-35214: Initial clang MemorySanitizer support by gpshead · Pull Request #10479 · python/cpython

Conversation

@gpshead

@gpshead gpshead commented Nov 12, 2018 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Adds a convenient configure flags for --with-memory-sanitizer and --with-undefined-behavior-sanitizer.

Fixes up the CPython code base to be mostly memory sanitizer clean. These changes come from patches we've been using at Google.

https://bugs.python.org/issue35214

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
gpshead added 3 commits November 11, 2018 23:13
@gpshead
Add configure flags for msan and ubsan.
f0184ab
This makes it easier to enable clang's memory and undefined behavior
sanitizers on a build without manually setting CFLAGS and LDFLAGS.

This also encodes the detail that address sanitizer and memory sanitizer
should disable pymalloc.
@gpshead
Define MEMORY_SANITIZER when appropriate.
dcf57d4
This allows code to easily take conditional action when in an msan
build.  Usually used for telling msan things it can't otherwise
figure out such as syscall, asm behavior, or about libc functions
it isn't yet aware of.
@gpshead
MemorySanitizer build fixes.
ffaeb17
These teach MSan about syscalls, asm behavior, or otherwise address
clang memory sanitizer flagged issues.
@gpshead gpshead added needs backport to 3.6 labels Nov 12, 2018
@gpshead gpshead self-assigned this Nov 12, 2018
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Nov 12, 2018
@bedevere-bot bedevere-bot added the awaiting merge label Nov 12, 2018
@gpshead
gpshead requested a review from Yhg1s November 12, 2018 07:30
@gpshead

gpshead commented Nov 12, 2018

Copy link
Copy Markdown
Member Author

A news entry will come. I put the PR up to see what the CI thinks of this on various platforms.

I run the current undefined behavior sanitizer buildbot and am setting up a memory sanitizer buildbot. This is a prerequisite for that to be useful.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead
NEWS entry
7430763
@gpshead
gpshead merged commit 1584a00 into python:master Nov 12, 2018
@miss-islington

miss-islington commented Nov 12, 2018

Copy link
Copy Markdown
Contributor

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Nov 12, 2018
@miss-islington

miss-islington commented Nov 12, 2018

Copy link
Copy Markdown
Contributor

Sorry, @gpshead, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 1584a0081500d35dc93ff88e5836df35faf3e3e2 3.7

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Nov 12, 2018

Copy link
Copy Markdown
Contributor

Sorry, @gpshead, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 1584a0081500d35dc93ff88e5836df35faf3e3e2 3.6

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead added a commit to gpshead/cpython that referenced this pull request Nov 12, 2018
@gpshead
bpo-35214: Initial clang MemorySanitizer support (pythonGH-10479)
1d1d852
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.

(cherry picked from commit 1584a00)
gpshead added a commit to gpshead/cpython that referenced this pull request Nov 12, 2018
@gpshead
[3.7] bpo-35214: Initial clang MemorySanitizer support (pythonGH-10479)
ecd00fb
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this..
(cherry picked from commit 1584a00)

Co-authored-by: Gregory P. Smith <[email protected]>
@bedevere-bot

bedevere-bot commented Nov 12, 2018

Copy link
Copy Markdown

GH-10492 is a backport of this pull request to the 3.7 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.7 label Nov 12, 2018
gpshead added a commit that referenced this pull request Nov 12, 2018
@gpshead
[3.7] bpo-35214: Initial clang MemorySanitizer support (GH-10479) (GH…
5f4d05d
…-10492)

Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.

(cherry picked from commit 1584a00)

Co-authored-by: Gregory P. Smith <[email protected]> [Google LLC]
gpshead added a commit to gpshead/cpython that referenced this pull request Nov 12, 2018
@gpshead
bpo-35214: Initial clang MemorySanitizer support (pythonGH-10479)
a0b6493
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.

(cherry picked from commit 1584a00)

Contributed by Gregory P. Smith [Google LLC]
gpshead added a commit to gpshead/cpython that referenced this pull request Nov 12, 2018
@gpshead
[3.6] bpo-35214: Initial clang MemorySanitizer support (pythonGH-10479)
7a68340
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this..
(cherry picked from commit 1584a00)

Co-authored-by: Gregory P. Smith <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 3.6 label Nov 12, 2018
@bedevere-bot

bedevere-bot commented Nov 12, 2018

Copy link
Copy Markdown

GH-10493 is a backport of this pull request to the 3.6 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead
gpshead deleted the initial-msan-fixes branch November 12, 2018 22:11
gpshead added a commit that referenced this pull request Nov 13, 2018
@gpshead
[3.6] bpo-35214: Initial clang MemorySanitizer support (GH-10479) (GH…
3b5b1c0
…-10493)

Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.

(cherry picked from commit 1584a00)

Contributed by Gregory P. Smith [Google LLC]

Also includes a whitespace fix from make patchcheck to _posixsubprocess.c - unrelated to the main change that makes the CI happy so I'm just doing it now rather than creating a separate PR.
@gpshead gpshead mentioned this pull request Aug 22, 2023
Get the test suite passing with clang Memory Sanitizer enabled #79395
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Yhg1s Yhg1s Awaiting requested review from Yhg1s

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@gpshead gpshead

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@gpshead @miss-islington @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.