GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src,worker: runtime error on pthread creation failure by HarshithaKP · Pull Request #32344 · nodejs/node

Conversation

@HarshithaKP

@HarshithaKP HarshithaKP commented Mar 18, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

with large number of worker threads, uv_pthread_create_ex was failing,
that leads to assertion failure. Convert it into a runtime error.

Fixes: #32319

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. worker Issues and PRs related to the worker_threads module and Worker API. labels Mar 18, 2020
@HarshithaKP

HarshithaKP commented Mar 18, 2020

Copy link
Copy Markdown
Member Author
events.js:292
      throw er; // Unhandled 'error' event
      ^
Error [ERR_WORKER_INIT_FAILED]: Worker initialization failure: EAGAIN
    at Worker.[kOnExit] (internal/worker.js:216:26)
    at Worker.<computed>.onexit (internal/worker.js:162:20)
    at Worker.startThread (<anonymous>)
    at new Worker (internal/worker.js:205:19)
    at Object.<anonymous> (/foo/wm/foo.js:6:19)
    at Module._compile (internal/modules/cjs/loader.js:1202:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
    at Module.load (internal/modules/cjs/loader.js:1051:32)
    at Function.Module._load (internal/modules/cjs/loader.js:947:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
Emitted 'error' event on Worker instance at:
    at Worker.[kOnExit] (internal/worker.js:216:12)
    at Worker.<computed>.onexit (internal/worker.js:162:20)
    [... lines matching original stack trace ...]
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12) {
  code: 'ERR_WORKER_INIT_FAILED'
}

Instead of assertion failure, now we get a runtime error like this.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jasnell

jasnell commented Mar 18, 2020

Copy link
Copy Markdown
Member

Change looks good. Ideally this would come with a test but given the nature of the failure that's going to be difficult I'd imagine.

HarshithaKP reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@HarshithaKP

HarshithaKP commented Mar 20, 2020

Copy link
Copy Markdown
Member Author

when I was doing testing with this, I found this out: the error was not actually caught by the error handler, instead thrown as unahndled exception.
debugging further, I found out that the location of failure (uv_pthread_create) is too early in the worker construction (compared to the previous case where it was failing in the worker thread itself), and when the emit happens the callback was not yet registered.
not sure what to do here:

  • delay emitting till the next tick?
  • throw error, so the caller can catch?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

addaleax
addaleax reviewed Mar 20, 2020
View reviewed changes

@addaleax addaleax left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

throw error, so the caller can catch?

Yes, I’d suggest doing that.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/node_worker.cc Outdated

@addaleax addaleax Mar 20, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please return here if the result of Set() is empty instead of using .Check().

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
bnoordhuis
bnoordhuis reviewed Mar 20, 2020
View reviewed changes
Comment thread src/node_worker.cc Outdated
Comment on lines 661 to 663

@bnoordhuis bnoordhuis Mar 20, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
w->object()->Set(w->env()->context(),
w->env()->message_port_string(),
Undefined(w->env()->isolate())).Check();
w->object()->Set(w->env()->context(),
w->env()->message_port_string(),
Undefined(w->env()->isolate())).Check();

(arguments should line up - but like Anna said, don't use Check().)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/node_worker.cc Outdated
Comment on lines 664 to 673

@bnoordhuis bnoordhuis Mar 20, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this ternary-heave style very hard to read. Can I suggest this?

Isolate* isolate = w->env()->isolate();
HandleScope handle_scope(isolate);
// ...
Local<Value> exit_code_v = Integer::New(isolate, w->exit_code_);
Local<Value> null_v = Null(isolate).As<Value>();
Local<Value> argv[3] = {exit_code_v, null_v, null_v};

if (w->custom_error_ != nullptr)
  args[1] = OneByteString(isolate, w->custom_error_);

if (!w->custom_error_str_.empty())
  args[2] = OneByteString(isolate, w->custom_error_str_.c_str());

Or alternatively:

Local<Value> exit_code_v = Integer::New(isolate, w->exit_code_);
Local<Value> null_v = Null(isolate).As<Value>();
Local<Value> custom_error_v = null_v;
Local<Value> custom_error_str_v = null_v;

if (w->custom_error_ != nullptr)
  custom_error_v = OneByteString(isolate, w->custom_error_);

if (!w->custom_error_str_.empty())
  custom_error_str_v = OneByteString(isolate, w->custom_error_str_.c_str());

Local<Value> argv[] = {exit_code_v, custom_error_v, custom_error_str_v};

I've left out the .As<Value>() casts because I don't think they're necessary here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Mar 20, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw, we don’t need the ternaries here anyway because we already know that custom_error_ and custom_error_str_ are set.

But either way, that should go away if this is converted to throwing an exception.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
HarshithaKP added 2 commits March 21, 2020 02:48
@HarshithaKP
src,worker: runtime error on pthread creation
9fcd394
with large number of worker threads pthread
fails with hard assertion.
Instead of hard assertion throw a runtime error.

Refs: nodejs#32319
@HarshithaKP
fixup: convert emit to throw
c064d2d
@HarshithaKP
HarshithaKP force-pushed the pthread_create_failure branch from 831cffc to c064d2d Compare March 21, 2020 06:53
@HarshithaKP

HarshithaKP commented Mar 21, 2020

Copy link
Copy Markdown
Member Author

@addaleax and @bnoordhuis, PTAL.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bnoordhuis
bnoordhuis approved these changes Mar 21, 2020
View reviewed changes
Comment thread src/node_worker.cc Outdated
HandleScope handle_scope(isolate);
THROW_ERR_WORKER_INIT_FAILED(isolate, err_buf);
}
w->MakeWeak();

@bnoordhuis bnoordhuis Mar 21, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
w->MakeWeak();
w->MakeWeak();

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@HarshithaKP HarshithaKP Mar 23, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bnoordhuis, thanks. Fixed it.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@HarshithaKP
fixup: address review comment
2f5a48d
devsnek
devsnek previously requested changes Mar 23, 2020
View reviewed changes

@devsnek devsnek left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert the file permissions back to 0644

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@HarshithaKP
fixup: address review comment
9a5b658
@HarshithaKP

HarshithaKP commented Mar 23, 2020

Copy link
Copy Markdown
Member Author

@devsnek, thanks. reverted the file permissions to 644. PTAL.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax
addaleax dismissed devsnek’s stale review March 29, 2020 19:34

addressed

addaleax
addaleax approved these changes Mar 29, 2020
View reviewed changes
@addaleax addaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Mar 29, 2020
@nodejs-github-bot

nodejs-github-bot commented Mar 29, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/30195/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jasnell
jasnell approved these changes Mar 29, 2020
View reviewed changes
@nodejs-github-bot

nodejs-github-bot commented Mar 29, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/30222/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax

addaleax commented Mar 30, 2020

Copy link
Copy Markdown
Member

Landed in b6459ec

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax addaleax closed this Mar 30, 2020
addaleax pushed a commit that referenced this pull request Mar 30, 2020
@HarshithaKP @addaleax
worker: runtime error on pthread creation
b6459ec
With large number of worker threads pthread
fails with hard assertion.
Instead of hard assertion throw a runtime error.

PR-URL: #32344
Fixes: #32319
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
addaleax added a commit to addaleax/node that referenced this pull request Mar 30, 2020
@addaleax
src: clean up worker thread creation code
09d3469
Instead of setting and then in the case of error un-setting properties,
only set them when no error occurs.

Refs: nodejs#32344
@addaleax addaleax mentioned this pull request Mar 30, 2020
src: minor worker code cleanups #32562
Closed
2 tasks done
addaleax pushed a commit that referenced this pull request Mar 30, 2020
@HarshithaKP @addaleax
worker: runtime error on pthread creation
833d78a
With large number of worker threads pthread
fails with hard assertion.
Instead of hard assertion throw a runtime error.

PR-URL: #32344
Fixes: #32319
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
addaleax added a commit that referenced this pull request Apr 2, 2020
@addaleax
src: clean up worker thread creation code
be87651
Instead of setting and then in the case of error un-setting properties,
only set them when no error occurs.

Refs: #32344

PR-URL: #32562
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
addaleax added a commit that referenced this pull request Apr 2, 2020
@addaleax
src: remove loop_init_failed_ from Worker class
f558a76
There’s no reason for this to not be stored alongside the loop
data structure itself.

PR-URL: #32562
Refs: #32344
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 7, 2020
@addaleax @BethGriggs
src: clean up worker thread creation code
cf73be1
Instead of setting and then in the case of error un-setting properties,
only set them when no error occurs.

Refs: #32344

PR-URL: #32562
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 7, 2020
@addaleax @BethGriggs
src: remove loop_init_failed_ from Worker class
387c63a
There’s no reason for this to not be stored alongside the loop
data structure itself.

PR-URL: #32562
Refs: #32344
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Apr 12, 2020
@addaleax @targos
src: clean up worker thread creation code
0faaa7c
Instead of setting and then in the case of error un-setting properties,
only set them when no error occurs.

Refs: #32344

PR-URL: #32562
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Apr 12, 2020
@addaleax @targos
src: remove loop_init_failed_ from Worker class
4df3ac2
There’s no reason for this to not be stored alongside the loop
data structure itself.

PR-URL: #32562
Refs: #32344
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@targos targos mentioned this pull request Apr 13, 2020
v13.13.0 release proposal #32813
Merged
@targos targos added backport-blocked-v12.x and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. backport-blocked-v12.x labels Apr 22, 2020
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
@HarshithaKP @targos
worker: runtime error on pthread creation
903b543
With large number of worker threads pthread
fails with hard assertion.
Instead of hard assertion throw a runtime error.

PR-URL: nodejs#32344
Fixes: nodejs#32319
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
@addaleax @targos
src: clean up worker thread creation code
fa41f32
Instead of setting and then in the case of error un-setting properties,
only set them when no error occurs.

Refs: nodejs#32344

PR-URL: nodejs#32562
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
@addaleax @targos
src: remove loop_init_failed_ from Worker class
2847ab2
There’s no reason for this to not be stored alongside the loop
data structure itself.

PR-URL: nodejs#32562
Refs: nodejs#32344
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Apr 28, 2020
@HarshithaKP @targos
worker: runtime error on pthread creation
ade4ec6
With large number of worker threads pthread
fails with hard assertion.
Instead of hard assertion throw a runtime error.

PR-URL: #32344
Fixes: #32319
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Apr 28, 2020
@addaleax @targos
src: clean up worker thread creation code
9f6ed72
Instead of setting and then in the case of error un-setting properties,
only set them when no error occurs.

Refs: #32344

PR-URL: #32562
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Apr 28, 2020
@addaleax @targos
src: remove loop_init_failed_ from Worker class
464ff85
There’s no reason for this to not be stored alongside the loop
data structure itself.

PR-URL: #32562
Refs: #32344
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@targos targos mentioned this pull request May 2, 2020
v12.17.0 release proposal #33197
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bnoordhuis bnoordhuis bnoordhuis approved these changes
@jasnell jasnell jasnell approved these changes
@addaleax addaleax addaleax approved these changes
+1 more reviewer
@devsnek devsnek devsnek left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. worker Issues and PRs related to the worker_threads module and Worker API.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

assertion failure in ../src/node_worker.cc:647 with 4K worker threads

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@HarshithaKP @jasnell @nodejs-github-bot @addaleax @bnoordhuis @devsnek @targos
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.