GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-112301: Add fortify source level 3 to default compiler options by nohlson · Pull Request #121520 · python/cpython

Conversation

@nohlson

@nohlson nohlson commented Jul 8, 2024

Copy link
Copy Markdown
Contributor

gh-112301: Added -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 to default compiler options for all builds

This option adds runtime protections for glibc to abort execution when unsafe behavior is encountered. Here are the GNU docs on the option

This is a very brief writeup that I found useful from Red Hat explaining some benefits

Also the OpenSSF compiler hardening guidance gives a very good description of the option

This is an option that theoretically affects the runtime so pyperformance benchmarks were run. The benchmark for this branch shows little overall impact but does impact some benchmark types:

NOTE: I would recommend looking into the details of the benchmarks in the links

Tag Geometric Mean
apps 1.00x slower
asyncio Not Significant
math 1.01x faster
regex 1.03x slower
serialize 1.01x faster
startup 1.00x faster
template 1.00x slower
overall 1.00x faster

A benchmark was run a few weeks ago with this option that was slightly different:

Tag Geometric Mean
apps 1.03x slower
asyncio 1.01x slower
math 1.00x slower
regex 1.02x faster
serialize 1.01x slower
startup 1.01x slower
template 1.01x slower
overall 1.01x slower

The benchmarks show that overall there is little memory impact and overall very small performance impact but within benchmark types there is some movement . Many compilers use -D_FORTIFY_SOURCE=2 by default. Level 3 adds additional bounds checking. My recommendation and the recommendation of the OpenSSF guidance is to raise to level 3 for this protection but would like to discuss further.

Attn: @mdboom

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nohlson
nohlson requested review from corona10 and erlend-aasland as code owners July 8, 2024 23:38
@bedevere-app bedevere-app Bot added the awaiting review label Jul 8, 2024
@nohlson nohlson changed the title Add fortify source level 3 to default compiler options gh-112301: Add fortify source level 3 to default compiler options Jul 8, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Jul 8, 2024
Consider applying flags for warnings about potential security issues #112301
Closed
@nohlson

nohlson commented Jul 8, 2024

Copy link
Copy Markdown
Contributor Author

@corona10 can we test with buildbots?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10 corona10 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 9, 2024
@bedevere-bot

bedevere-bot commented Jul 9, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @corona10 for commit 453e34f 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 9, 2024
erlend-aasland
erlend-aasland reviewed Jul 9, 2024
View reviewed changes
Comment thread Misc/NEWS.d/next/Security/2024-07-08-23-39-04.gh-issue-112301.TD8G01.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@ghost

ghost commented Jul 9, 2024 •
edited by ghost
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.
CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10

corona10 commented Jul 11, 2024

Copy link
Copy Markdown
Member

@nohlson I will take a look at this PR by this weekend, sorry, I am too busy these days.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10 corona10 self-assigned this Jul 11, 2024
nohlson and others added 3 commits July 14, 2024 17:28
@nohlson @corona10
Add fortify source level 3 to default compiler options
e2a4d2f
@blurb-it @corona10
📜🤖 Added by blurb_it.
0a1eba9
@nohlson @erlend-aasland @corona10
Update news to better reflect impact of the compiler option
7a595e7
Co-authored-by: Erlend E. Aasland <[email protected]>
@corona10
corona10 force-pushed the enable-fortify-source-level-three branch from 1d61e81 to 7a595e7 Compare July 14, 2024 08:28
corona10
corona10 reviewed Jul 14, 2024
View reviewed changes

@corona10 corona10 left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nohlson
I have 2 questions.

  1. Is this flag the subset of the overall policy that we want to measure?
    #112301 (comment)
  2. Do you have a plan to provide some kind of ./configure --disable-openssf-guide?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nohlson

nohlson commented Jul 17, 2024

Copy link
Copy Markdown
Contributor Author

@nohlson I have 2 questions.

  1. Is this flag the subset of the overall policy that we want to measure?
    Consider applying flags for warnings about potential security issues #112301 (comment)
  2. Do you have a plan to provide some kind of ./configure --disable-openssf-guide?
  1. This is a subset of the overall policy based on the OpenSSF guidance.
  2. I do not intend to add an argument to configure to turn off these options, however I would be interested in maybe including the ability to disable the new options that do impact performance benchmarks. This is theoretically one of those options however we didn't see a measurable impact. I could see there being a configure argument that disables the performance impacting options if we were to introduce one that has a measurable impact.

I would prefer the options we introduce for this effort to be always on, and put a lot of consideration into benchmark impacting options we do include, and possibly not enabling them if we deem the impact too much rather than enabling everything and making a new configure argument.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10

corona10 commented Jul 18, 2024

Copy link
Copy Markdown
Member

@nohlson

I do not intend to add an argument to configure to turn off these options, however I would be interested in maybe including the ability to disable the new options that do impact performance benchmarks.

I am pretty sure that there will be users who want to disable options that we added for the OpenSSF guide with a single configuration command :)
Adding the optional argument will be helpful to them(even if enabling OpenSSF options is more recommended), I will submit the PR in this week.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

corona10
corona10 approved these changes Jul 18, 2024
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Jul 18, 2024
@corona10
corona10 merged commit bdab67e into python:main Jul 18, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Jul 18, 2024
@vstinner

vstinner commented Jul 18, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This change broke RHEL8 buildbot, test_cext and test_cppext fail with:

  /usr/include/features.h:393:5: error: #warning _FORTIFY_SOURCE > 2 is treated like 2 on this platform [-Werror=cpp]
   #   warning _FORTIFY_SOURCE > 2 is treated like 2 on this platform
       ^~~~~~~
  cc1plus: all warnings being treated as errors
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10

corona10 commented Jul 19, 2024

Copy link
Copy Markdown
Member

@nohlson Would you like to check?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jul 19, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

I suggest checking for the FORTIFY flag using -Werror in configure.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom mdboom mentioned this pull request Jul 19, 2024
Introduce configure --disable-openssf-guide or --enable-openssf-guide #121996
Closed
@erlend-aasland

erlend-aasland commented Jul 19, 2024

Copy link
Copy Markdown
Contributor

I suggest checking for the FORTIFY flag using -Werror in configure.

We already do.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jul 20, 2024

Copy link
Copy Markdown
Member

We already do.

For other flags yes, but apparently, not for _FORTIFY_SOURCE=3:

# Enable flags that warn and protect for potential security vulnerabilities.
# These flags should be enabled by default for all builds.
AX_CHECK_COMPILE_FLAG([-fstack-protector-strong], [BASECFLAGS="$BASECFLAGS -fstack-protector-strong"], [AC_MSG_WARN([-fstack-protector-strong not supported])], [-Werror])
AX_CHECK_COMPILE_FLAG([-Wtrampolines], [BASECFLAGS="$BASECFLAGS -Wtrampolines"], [AC_MSG_WARN([-Wtrampolines not supported])], [-Werror])
AX_CHECK_COMPILE_FLAG([-D_FORTIFY_SOURCE=3], [BASECFLAGS="$BASECFLAGS -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3"], [AC_MSG_WARN([-D_FORTIFY_SOURCE=3 not supported])])
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou added a commit to encukou/cpython that referenced this pull request Jul 22, 2024
@encukou
pythongh-112301: Revert "Add fortify source level 3 to default compil…
24d901a
…er options (pythongh-121520)"

Adding the flag broke buildbots.

This reverts commit bdab67e.
@nohlson nohlson mentioned this pull request Jul 22, 2024
gh-112301: Make fortify source option check -Werror #122141
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erlend-aasland erlend-aasland erlend-aasland left review comments
@corona10 corona10 corona10 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@corona10 corona10

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@nohlson @bedevere-bot @corona10 @vstinner @erlend-aasland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.