GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Attempt 2 - Fix Missing Git Executable Causing ClusterFuzz Crash by DaveLak · Pull Request #1909 · gitpython-developers/GitPython

Conversation

@DaveLak

@DaveLak DaveLak commented Apr 26, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This is a second attempt at #1906 and should resolve:

  • Fuzz Tests Are Crashing at Start-up on ClusterFuzz #1905
  • gitpython: Failing on every run google/oss-fuzz#10600

PR #1906 had the right idea but wrong implementation. The differences between the ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image where it was tested led to the issue not being fully resolved.

The root cause of the issue is the same: A Git executable is not globally available in the ClusterFuzz container environment where OSS-Fuzz executes fuzz tests.

#1906 attempted to fix the issue by bundling the Git binary and using
GitPython's git.refresh(<full-path-to-git-executable>) method to set it inside the TestOneInput function of the test harness.

However, GitPython attempts to set the binary at import time via its __init__ hook, and crashes the test if no executable is found during the import.

This issue is fixed here by setting the environment variable that GitPython looks in before importing it, so it's available for the import. This was tested by setting the $PATH to an empty string inside the test files, which reproduced the crash, then adding the changes introduced here with $PATH still empty, which avoided the crash indicating that the bundled Git executable is working as expected.


Note: I didn't test this in a full ClusterFuzz environment, so it's worth noting that while it may be possible for this to continue failing because of the difference between environments that the Git executable was produced in vs. where it's run: I don't believe that will be an issue in practice because the guidance for projects (including Git itself) in OSS-Fuzz is to compile in build.sh with the libraries necessary to create the final binary that is executed in ClusterFuzz. The changes here assume that the apt repo version of Git has everything statically built. If, for some reason, it turns out that a static binary isn't what is installed, the alternative will be to build git from source or mock it. But based on my local testing thus far, I think (and hope) that won't be necessary, and this changeset should be sufficient.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@DaveLak
Attempt 2 - Fix Missing Git Executable Causing ClusterFuzz Crash
dac3535
This is a second attempt at gitpython-developers#1906 and should resolve:
- gitpython-developers#1905
- google/oss-fuzz#10600

PR gitpython-developers#1906 had the right idea but wrong implementation, and the differences between
the ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image where
the fix was tested led to the issue not being fully resolved.

The root cause of the issue is the same: A Git executable is not globally
available in the ClusterFuzz container environment where OSS-Fuzz executes
fuzz tests.

 gitpython-developers#1906 attempted to fix the issue by bundling the Git binary and using
GitPython's `git.refresh(<full-path-to-git-executable>)` method to set it
inside the `TestOneInput` function of the test harness.

However, GitPython attempts to set the binary at import time via its `__init__`
hook, and crashes the test if no executable is found during the import.

This issue is fixed here by setting the environment variable that GitPython
looks in before importing it, so it's available for the import. This was tested
by setting the `$PATH` to an empty string inside the test files, which
reproduced the crash, then adding the changes introduced here with `$PATH` still
empty, which avoided the crash indicating that the bundled Git executable is
working as expected.
@DaveLak DaveLak mentioned this pull request Apr 26, 2024
Fuzz Tests Are Crashing at Start-up on ClusterFuzz #1905
Closed
1 task done
Byron
Byron approved these changes Apr 27, 2024
View reviewed changes

@Byron Byron left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for taking charge of this! No worries about taking more steps either :).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DaveLak reacted with rocket emoji
All reactions
  • 🚀 1 reaction
@Byron
Byron merged commit 797009d into gitpython-developers:main Apr 27, 2024
@DaveLak
DaveLak deleted the attempt-two-fuzzing-fix-missing-git-in-clusterfuzz branch April 27, 2024 14:48
@DaveLak

DaveLak commented Apr 28, 2024

Copy link
Copy Markdown
Contributor Author

It worked! I'm seeing logs from successful fuzz target runs over the last two hours or so!

Byron reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@DaveLak DaveLak mentioned this pull request Apr 28, 2024
gitpython: Failing on every run google/oss-fuzz#10600
Open
DaveLak added a commit to DaveLak/GitPython that referenced this pull request Jun 4, 2024
@DaveLak
Fix Improper Import Order Breaking fuzz_submodule Fuzzer
491e134
ClusterFuzz runs of the `fuzz_submodule` target have been failing
because the `git` import was placed before the condition that sets the
Git executable path.

The order in which `git` is imported matters because it attempts to find
a Git executable as the import is loaded (via `refresh()` in
`git/__init__.py`.) As per gitpython-developers#1909, we configure the ClusterFuzz
environment to use a bundled Git executable via the env variable
condition in all fuzz targets.
@DaveLak DaveLak mentioned this pull request Jun 4, 2024
Fix Improper Import Order Breaking fuzz_submodule Fuzzer #1926
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Byron Byron Byron approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@DaveLak @Byron
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.