GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src: remap invalid file descriptors using `dup2` by obiwac · Pull Request #44461 · nodejs/node

Conversation

@obiwac

@obiwac obiwac commented Aug 30, 2022

Copy link
Copy Markdown
Contributor

When checking for the validity of the stdio file descriptors in PlatformInit(), this was the code previously used (as in #875):

// Make sure file descriptors 0-2 are valid before we start logging anything.
for (int fd = STDIN_FILENO; fd <= STDERR_FILENO; fd += 1) {
  struct stat ignored;
  if (fstat(fd, &ignored) == 0)
    continue;
  // Anything but EBADF means something is seriously wrong.  We don't
  // have to special-case EINTR, fstat() is not interruptible.
  if (errno != EBADF)
    abort();
  if (fd != open("/dev/null", O_RDWR))
    abort();
}

As I understand it, the intention behind this is to map file descriptors to /dev/null if they don't exist, by hoping the next file descriptor to be given will be the one which doesn't exist. This is imperfect as it is very platform dependent and breaks when /dev/null has already been given a different file descriptor in the same process, but e.g. stdin has disappeared in the meantime (as is the case with this issue on FreeBSD with the daemon tool when not passing -f: nodejs/help#2411).

Instead, this patch uses the dup2(2) syscall to properly remap the missing stdio file descriptor to what /dev/null's file descriptor actually is.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@nodejs-github-bot

nodejs-github-bot commented Aug 30, 2022

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/startup
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Aug 30, 2022
@obiwac
obiwac force-pushed the main branch 6 times, most recently from 69293a1 to b3c392e Compare August 31, 2022 11:02
addaleax
addaleax reviewed Aug 31, 2022
View reviewed changes
Comment thread src/node.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

@addaleax

addaleax commented Aug 31, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

by hoping the next file descriptor to be given will be the one which doesn't exist

“hoping” is doing a lot of work here 🙂 This was actually spec’d behavior in previous POSIX versions, and I was quite surprised to learn (through this PR) that the spec actually changed in this regard.

I’d leave a comment in the source code about this, since others might be surprised by this as well.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@obiwac

obiwac commented Aug 31, 2022

Copy link
Copy Markdown
Contributor Author

I’d leave a comment in the source code about this, since others might be surprised by this as well.

Got you. I'll do that in a little bit ✌️

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@obiwac
obiwac force-pushed the main branch from b3c392e to c905070 Compare August 31, 2022 14:15
@obiwac
obiwac requested a review from addaleax August 31, 2022 14:16
addaleax
addaleax approved these changes Aug 31, 2022
View reviewed changes
@theanarkh theanarkh added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Aug 31, 2022
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Aug 31, 2022
@nodejs-github-bot

nodejs-github-bot commented Aug 31, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/46354/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

joyeecheung
joyeecheung approved these changes Sep 1, 2022
View reviewed changes
bnoordhuis
bnoordhuis reviewed Sep 1, 2022
View reviewed changes

@bnoordhuis bnoordhuis left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, this change adds more failure paths and I don't think it's actually necessary.

The "open() returns the lowest available file descriptor" convention is so deeply codified in the UNIX ecosystem that it's never going away, no matter what POSIX says. There's simply too much software built around that assumption.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/node.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/node.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

@obiwac

obiwac commented Sep 1, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

The "open() returns the lowest available file descriptor" convention is so deeply codified in the UNIX ecosystem that it's never going away, no matter what POSIX says.

While I do agree with this, there still are specific cases where "available" means something different to what's expected. Case in point:

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

int main(void) {
	system("echo && fstat | grep a.out");
	int fd = open("/dev/null", O_RDWR);
	printf("\n/dev/null: %d\n", fd);

	return 0;
}

Running this on FreeBSD in a separate session with daemon -o log.log -r /tmp/a.out and then closing said session (thus making stdin invalid):

root     a.out      89214 text [restricted]    519 -rwxr-xr-x   15424  r
root     a.out      89214   wd /        107129 drwxr-xr-x      16  r
root     a.out      89214 root /        107106 drwxr-xr-x      24  r
root     a.out      89214 jail /        107106 drwxr-xr-x      24  r
root     a.out      89214    0 -         -         bad    -
root     a.out      89214    1* pipe fffff8002e0e7d00 <-> fffff8002e0e7ba0      0 rw
root     a.out      89214    2* pipe fffff8002e0e7d00 <-> fffff8002e0e7ba0      0 rw
root     a.out      89214    3 [restricted]     24 crw-rw-rw-    null rw

/dev/null: 3

(This is actually the issue nodejs/help#2411 had.)

Here, 0 should be available, yet /dev/null is still opened as 3. Something else that works is closing 0 before opening /dev/null, but IMHO at that point it's kinda relying on flaky behaviour.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@obiwac
obiwac force-pushed the main branch 2 times, most recently from da101fe to 326a490 Compare September 1, 2022 11:21
@obiwac
src: remap invalid file descriptors using dup2
326a490
When checking for the validity of the stdio file descriptors
(nodejs#875), ones which don't exist are intended to be remapped to
/dev/null (and, if that doesn't work, we abort).

This however doesn't work on all platforms and in all cases, and is not
anymore required by POSIX; instead, use the `dup2` syscall as a more
robust solution (conforms to POSIX.1).

Fixes: nodejs/help#2411
Refs: nodejs#875
@github-actions github-actions Bot mentioned this pull request Sep 2, 2022
CI Reliability 2022-09-02 nodejs/reliability#365
Open
24 tasks
@bnoordhuis

bnoordhuis commented Sep 2, 2022

Copy link
Copy Markdown
Member

Ah okay, fair enough. Fd 0 points to a kind of zombie file description in your example, i.e., it's technically still open but not actually usable?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bnoordhuis
bnoordhuis approved these changes Sep 2, 2022
View reviewed changes
This was referenced Sep 3, 2022
CI Reliability 2022-09-03 nodejs/reliability#366
Open
CI Reliability 2022-09-04 nodejs/reliability#367
Open
@obiwac

obiwac commented Sep 4, 2022

Copy link
Copy Markdown
Contributor Author

Fd 0 points to a kind of zombie file description in your example, i.e., it's technically still open but not actually usable?

AFAIU the file descriptor is never actually closed, so it isn't ever marked as unused (fdunused()), which is why it's never removed from the file descriptor table.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request Sep 5, 2022
CI Reliability 2022-09-05 nodejs/reliability#368
Open
17 tasks
@bnoordhuis

bnoordhuis commented Sep 5, 2022

Copy link
Copy Markdown
Member

Wait, but fstat() still fails with EBADF? How does that work?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request Sep 6, 2022
CI Reliability 2022-09-06 nodejs/reliability#369
Open
17 tasks
@obiwac

obiwac commented Sep 6, 2022

Copy link
Copy Markdown
Contributor Author

I don't know, there are plenty of places along the line where fstat(2) can fail with EBADF, not just because a file descriptor is not found in the file descriptor table (you can check out kern_fstat(), my quick guess is that fp->f_ops is set to &badfileops at some point before being unused).

I could investigate further if you'd like an exact answer but I don't see how it's too relevant outside of the FreeBSD kernel :P

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bnoordhuis

bnoordhuis commented Sep 8, 2022

Copy link
Copy Markdown
Member

"Zombie file descriptor" is good enough for me. :-)

obiwac reacted with laugh emoji
All reactions
  • 😄 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bnoordhuis bnoordhuis added request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. and removed needs-ci PRs that need a full CI run. labels Sep 8, 2022
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 8, 2022
@nodejs-github-bot

nodejs-github-bot commented Sep 8, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/46479/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@obiwac

obiwac commented Oct 25, 2022

Copy link
Copy Markdown
Contributor Author

Is there anything left for me to do here?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax addaleax added author ready PRs with CI started, the required approvals, and no outstanding review comments. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. labels Oct 25, 2022
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 25, 2022
@nodejs-github-bot

nodejs-github-bot commented Oct 25, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47466/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax addaleax added the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 26, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 26, 2022
@nodejs-github-bot
nodejs-github-bot merged commit 0c8b141 into nodejs:main Oct 26, 2022
@nodejs-github-bot

nodejs-github-bot commented Oct 26, 2022

Copy link
Copy Markdown
Collaborator

Landed in 0c8b141

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

RafaelGSS pushed a commit that referenced this pull request Nov 1, 2022
@obiwac @RafaelGSS
src: remap invalid file descriptors using dup2
49e8001
When checking for the validity of the stdio file descriptors
(#875), ones which don't exist are intended to be remapped to
/dev/null (and, if that doesn't work, we abort).

This however doesn't work on all platforms and in all cases, and is not
anymore required by POSIX; instead, use the `dup2` syscall as a more
robust solution (conforms to POSIX.1).

Fixes: nodejs/help#2411
Refs: #875
PR-URL: #44461
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
@RafaelGSS RafaelGSS mentioned this pull request Nov 1, 2022
v19.1.0 proposal #45269
Merged
RafaelGSS pushed a commit that referenced this pull request Nov 10, 2022
@obiwac @RafaelGSS
src: remap invalid file descriptors using dup2
8d56822
When checking for the validity of the stdio file descriptors
(#875), ones which don't exist are intended to be remapped to
/dev/null (and, if that doesn't work, we abort).

This however doesn't work on all platforms and in all cases, and is not
anymore required by POSIX; instead, use the `dup2` syscall as a more
robust solution (conforms to POSIX.1).

Fixes: nodejs/help#2411
Refs: #875
PR-URL: #44461
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
@danielleadams danielleadams mentioned this pull request Dec 30, 2022
v18.13.0 release proposal #46025
Merged
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
@obiwac @danielleadams
src: remap invalid file descriptors using dup2
802cefb
When checking for the validity of the stdio file descriptors
(#875), ones which don't exist are intended to be remapped to
/dev/null (and, if that doesn't work, we abort).

This however doesn't work on all platforms and in all cases, and is not
anymore required by POSIX; instead, use the `dup2` syscall as a more
robust solution (conforms to POSIX.1).

Fixes: nodejs/help#2411
Refs: #875
PR-URL: #44461
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
@obiwac @danielleadams
src: remap invalid file descriptors using dup2
325254c
When checking for the validity of the stdio file descriptors
(#875), ones which don't exist are intended to be remapped to
/dev/null (and, if that doesn't work, we abort).

This however doesn't work on all platforms and in all cases, and is not
anymore required by POSIX; instead, use the `dup2` syscall as a more
robust solution (conforms to POSIX.1).

Fixes: nodejs/help#2411
Refs: #875
PR-URL: #44461
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
@JonnyBurger JonnyBurger mentioned this pull request Jun 29, 2023
Command failed with EBADF remotion-dev/remotion#2452
Closed
@robobun robobun mentioned this pull request Oct 3, 2026
Startup: reopen fd 0-2 as /dev/null only when the fd is closed oven-sh/bun#41749
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bnoordhuis bnoordhuis bnoordhuis approved these changes
@addaleax addaleax addaleax approved these changes
@joyeecheung joyeecheung joyeecheung approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. c++ Issues and PRs that require attention from people who are familiar with C++.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@obiwac @nodejs-github-bot @addaleax @bnoordhuis @joyeecheung @theanarkh
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.