GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — avoid self._errpipe_write double close by graingert · Pull Request #466 · MagicStack/uvloop

Conversation

@graingert

@graingert graingert commented May 11, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Fixes #414

the I couldn't get the original MRE from the issue https://gist.github.com/decorator-factory/6b5276ab5fb87f4c009f1d22dc4b20a9 to always fail - it seems to be intermittent. However the following always fails for me without this patch:

import os
import asyncio
import subprocess

import uvloop
uvloop.install()


async def main():
    done = False
    def preexec_fn():
        pass

    async def fork():
        nonlocal done
        try:
            for i in range(1000):
                await asyncio.sleep(0)
                try:
                    proc = await asyncio.create_subprocess_exec("true", preexec_fn=preexec_fn)
                    await proc.communicate()
                except subprocess.SubprocessError:
                    print("done")
        finally:
            done = True

    def write_file():
        while not done:
            with open("demo", "w") as f:
                f.write("hello")
                print(f.fileno())

    await asyncio.gather(
        fork(),
        asyncio.to_thread(write_file)
    )

asyncio.run(main())

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
decorator-factory and graingert reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
graingert
graingert commented May 11, 2022
View reviewed changes
Comment thread uvloop/handles/process.pyx
self._finish_init()

os_close(self._errpipe_write)
self._errpipe_write = -1

@graingert graingert May 11, 2022

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the write_pipe can be closed again sometimes:

uvloop/uvloop/handles/process.pyx

Lines 118 to 122 in 3e71ddc

try:
os_close(self._errpipe_write)
except OSError:
# Might be already closed
pass

if another thread allocates a file descriptor in this time then instead of raising OSError here the fd will be closed incorrectly

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@graingert graingert mentioned this pull request May 11, 2022
Pathlib throws "Bad file descriptor", but only with uvloop. #414
Closed
cbay added a commit to cbay/uvloop that referenced this pull request May 16, 2022
@cbay
https://github.com/MagicStack/uvloop/pull/466
bc028f8
@graingert
avoid self._errpipe_write double close
b6d0e79 
Fixes MagicStack#414
@graingert
graingert force-pushed the patch-1 branch from 5d6fca1 to b6d0e79 Compare July 16, 2022 21:47
@graingert

graingert commented Jul 16, 2022

Copy link
Copy Markdown
Contributor Author

@fantix can I get a review of this please?
fantix reacted with rocket emoji
All reactions
  • 🚀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@graingert
Merge branch 'master' into patch-1
efb39bd
fantix
fantix approved these changes Jul 17, 2022
View reviewed changes

@fantix fantix left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good; I was trying to write a test but it's going to take some time, let's merge this first.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@graingert
Merge branch 'master' into patch-1
2f31146
@fantix
fantix merged commit 72140d7 into MagicStack:master Jul 17, 2022
@graingert
graingert deleted the patch-1 branch July 17, 2022 15:23
@fantix fantix mentioned this pull request Jul 17, 2022
Add test for preexec_fn fd double close issue #479
Closed
fantix added a commit that referenced this pull request Sep 13, 2022
@fantix
uvloop v0.17.0
c4c3f9f 
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.

Changes
=======

* Expose uv_loop_t pointer for integration with other C-extensions (#310)
  (by @pranavtbhat in b332eb8 for #310)

* Support python 3.11+ (#473)
  (by @zeroday0619 in 8e42921 for #473)

* Expose libuv uv_fs_event functionality (#474)
  (by @jensbjorgensen @fantix in 74d381e for #474)

* Activate debug mode when `-X dev` is used
  (by @jack1142 in 637a77a)

* Expose uv_version() for libuv API compatibility (#491)
  (by @fantix in 089f6cb for #491)

* Fix loop.getaddrinfo() and tests (#495)
  (by @fantix in 598b16f for #495)

* Bump to libuv 1.43.0
  (by @fantix in 94e5e53)

Fixes
=====

* _TransProtPair is no longer defined in asyncio.events
  (by @jensbjorgensen in fae5f7f)

* use a TypeVar for asyncio.BaseProtocol (#478)
  (by @graingert in 3aacb35 for #478)

* Fix segfault in TimerHandle.when() after cleared
  (by @jensbjorgensen in c39afff for #469)

* Avoid self._errpipe_write double close (#466)
  (by @graingert in 72140d7 for #466)

* Fix typo in test (#456)
  (by @kianmeng in 033d52d for #456)

* Fix potential infinite loop (#446)
  (by @kfur in ada43c0 for #446)

* use a stack of self._fds_to_close to prevent double closes (#481)
  (by @graingert in 3214cf6 for #481)

* Fix incorrect main thread id value forking from a thread  (#453)
  (by @horpto @fantix in e7934c8 for #453)

* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
  (by @byllyfish in e04637e for #439)

* Queue write only after processing all buffers (#445)
  (by @jakirkham @fantix in 9c6ecb6 for #445)

* Drop Python 3.6 support for thread ident
  (by @fantix in 9c37930)

* bugfix: write to another transport in resume_writing() fails (#498)
  (by @fantix in d2deffe for #498)

Build
=====

* Upgrade GitHub Actions (#477) (#480)
  (by @cclauss in fcbf422 for #477, 1008694 for #480)

* typo `same as same`
  (by @YoSTEALTH in fedba80)

* setup.py: allow to override extra_compile_args (#443)
  (by @giuliobenetti in a130375 for #443)

* Drop hack in setup.py in finalize_options (492)
  (by @fantix in 2f1bc83 for #492)

* Fix tests invocation on release CI worklow (#489)
  (by @ben9923 in d6a2b59 for #489)

Documentation
=============

* use asyncio.Runner loop_factory on 3.11+ (#472)
  (by @graingert in 31ba48c for #472)

* Fix CI badge in docs, remove remaining Travis CI references from docs
  (by @Nothing4You in c6901a7)

* Fix typo in README
  (by @monosans in 73d7253)
@fantix fantix mentioned this pull request Sep 13, 2022
uvloop v0.17.0 #501
Merged
fantix added a commit that referenced this pull request Sep 14, 2022
@fantix
uvloop v0.17.0
afb3268 
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.

Changes
=======

* Expose uv_loop_t pointer for integration with other C-extensions (#310)
  (by @pranavtbhat in b332eb8 for #310)

* Support python 3.11+ (#473)
  (by @zeroday0619 in 8e42921 for #473)

* Expose libuv uv_fs_event functionality (#474)
  (by @jensbjorgensen @fantix in 74d381e for #474)

* Activate debug mode when `-X dev` is used
  (by @jack1142 in 637a77a)

* Expose uv_version() for libuv API compatibility (#491)
  (by @fantix in 089f6cb for #491)

* Fix loop.getaddrinfo() and tests (#495)
  (by @fantix in 598b16f for #495)

* Bump to libuv 1.43.0
  (by @fantix in 94e5e53)

Fixes
=====

* _TransProtPair is no longer defined in asyncio.events
  (by @jensbjorgensen in fae5f7f)

* use a TypeVar for asyncio.BaseProtocol (#478)
  (by @graingert in 3aacb35 for #478)

* Fix segfault in TimerHandle.when() after cleared
  (by @jensbjorgensen in c39afff for #469)

* Avoid self._errpipe_write double close (#466)
  (by @graingert in 72140d7 for #466)

* Fix typo in test (#456)
  (by @kianmeng in 033d52d for #456)

* Fix potential infinite loop (#446)
  (by @kfur in ada43c0 for #446)

* use a stack of self._fds_to_close to prevent double closes (#481)
  (by @graingert in 3214cf6 for #481)

* Fix incorrect main thread id value forking from a thread  (#453)
  (by @horpto @fantix in e7934c8 for #453)

* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
  (by @byllyfish in e04637e for #439)

* Queue write only after processing all buffers (#445)
  (by @jakirkham @fantix in 9c6ecb6 for #445)

* Drop Python 3.6 support for thread ident
  (by @fantix in 9c37930)

* bugfix: write to another transport in resume_writing() fails (#498)
  (by @fantix in d2deffe for #498)

Build
=====

* Upgrade GitHub Actions (#477) (#480)
  (by @cclauss in fcbf422 for #477, 1008694 for #480)

* typo `same as same`
  (by @YoSTEALTH in fedba80)

* setup.py: allow to override extra_compile_args (#443)
  (by @giuliobenetti in a130375 for #443)

* Drop hack in setup.py in finalize_options (492)
  (by @fantix in 2f1bc83 for #492)

* Fix tests invocation on release CI worklow (#489)
  (by @ben9923 in d6a2b59 for #489)

Documentation
=============

* use asyncio.Runner loop_factory on 3.11+ (#472)
  (by @graingert in 31ba48c for #472)

* Fix CI badge in docs, remove remaining Travis CI references from docs
  (by @Nothing4You in c6901a7)

* Fix typo in README
  (by @monosans in 73d7253)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fantix fantix fantix approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Pathlib throws "Bad file descriptor", but only with uvloop.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@graingert @fantix
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.