GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src: trace threadpool event by theanarkh · Pull Request #44458 · nodejs/node

Conversation

@theanarkh

@theanarkh theanarkh commented Aug 30, 2022

Copy link
Copy Markdown
Contributor

trace threadpool events, such as APIs of Blob, zlib, crypto and node_api.

  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Aug 30, 2022
@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch from 6f35dbf to 245ef10 Compare August 30, 2022 16:35
legendecas
legendecas reviewed Aug 30, 2022
View reviewed changes
Comment thread src/threadpoolwork-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch 2 times, most recently from b87e591 to 39bd5b5 Compare August 30, 2022 17:44
VoltrexKeyva
VoltrexKeyva reviewed Aug 31, 2022
View reviewed changes
Comment thread src/node_internals.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch from 39bd5b5 to 39f400b Compare August 31, 2022 02:26
legendecas
legendecas reviewed Aug 31, 2022
View reviewed changes
Comment thread src/node_internals.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch from 39f400b to ef94baa Compare August 31, 2022 02:40
legendecas
legendecas reviewed Aug 31, 2022
View reviewed changes
Comment thread src/threadpoolwork-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/tracing.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/tracing.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch 2 times, most recently from 40c49ab to 672d9ad Compare August 31, 2022 13:24
addaleax
addaleax reviewed Aug 31, 2022
View reviewed changes
Comment thread src/node_internals.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch 2 times, most recently from e077266 to 24d32ce Compare September 1, 2022 06:28
RaisinTen
RaisinTen reviewed Sep 2, 2022
View reviewed changes
Comment thread src/threadpoolwork-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/threadpoolwork-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/threadpoolwork-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/threadpoolwork-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh

theanarkh commented Sep 9, 2022

Copy link
Copy Markdown
Contributor Author

@legendecas @RaisinTen Hi, can we just trace the total time of work. the code is as follows.

void ThreadPoolWork::ScheduleWork() {
  env_->IncreaseWaitingRequestCounter();

  TRACE_EVENT_NESTABLE_ASYNC_BEGIN0(...);

  int status = uv_queue_work(
      env_->event_loop(),
      &work_req_,
      [](uv_work_t* req) {
        ThreadPoolWork* self = ContainerOf(&ThreadPoolWork::work_req_, req);
        self->DoThreadPoolWork();
      },
      [](uv_work_t* req, int status) {
        ThreadPoolWork* self = ContainerOf(&ThreadPoolWork::work_req_, req);
        self->env_->DecreaseWaitingRequestCounter();

        TRACE_EVENT_NESTABLE_ASYNC_END1(...);

        self->AfterThreadPoolWork(status);
      });
  CHECK_EQ(status, 0);
}

Or trace all the time by two different event type.

void ThreadPoolWork::ScheduleWork() {
  env_->IncreaseWaitingRequestCounter();

  TRACE_EVENT_NESTABLE_ASYNC_BEGIN0(...);

  int status = uv_queue_work(
      env_->event_loop(),
      &work_req_,
      [](uv_work_t* req) {
        ThreadPoolWork* self = ContainerOf(&ThreadPoolWork::work_req_, req);
        TRACE_EVENT_BEGIN(...);
        self->DoThreadPoolWork();
        TRACE_EVENT_END(...);
      },
      [](uv_work_t* req, int status) {
        ThreadPoolWork* self = ContainerOf(&ThreadPoolWork::work_req_, req);
        self->env_->DecreaseWaitingRequestCounter();

        TRACE_EVENT_NESTABLE_ASYNC_END1(...);

        self->AfterThreadPoolWork(status);
      });
  CHECK_EQ(status, 0);
}
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@legendecas

legendecas commented Sep 14, 2022

Copy link
Copy Markdown
Member

I'd still find that tracing the synchronous event is important. So the latter alternative is LGTM.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch 2 times, most recently from 1703fd4 to 05cbb9c Compare September 14, 2022 16:26
@theanarkh

theanarkh commented Sep 14, 2022

Copy link
Copy Markdown
Contributor Author

It seems we can not use TRACE_EVENT_BEGIN0 and TRACE_EVENT_END0 in another thread, i will take a look later.
https://github.com/nodejs/node/actions/runs/3054446637/jobs/4926355622.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch 3 times, most recently from a8b5f8f to b1128a1 Compare October 26, 2022 00:38
@theanarkh

theanarkh commented Oct 26, 2022

Copy link
Copy Markdown
Contributor Author

@legendecas Hi, it seems the TRACE_EVENT_xx is not thread-safe, so i record the time of synchronous API call in args field of async event. what do you think ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh
theanarkh force-pushed the trace_threadpoolwork_event branch from b1128a1 to 1f402fc Compare October 26, 2022 01:30
@legendecas

legendecas commented Oct 26, 2022

Copy link
Copy Markdown
Member

@theanarkh The macros are declared to be thread-safe. If there is any problem related to thread safety, it should be fixed.

I'd find the problem here is related to the incomplete shutdown -- e.g. the global state of the tracing controller is not reset after the platform is shut down https://github.com/nodejs/node/blob/main/src/tracing/trace_event.cc#L8. This global state is used in the macro expansion: https://github.com/nodejs/node/blob/main/src/tracing/trace_event.h#L474. We should properly shut down the platform with the tracing global state.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@theanarkh

theanarkh commented Oct 26, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Do you mean this test case https://github.com/nodejs/node/actions/runs/3054446637/jobs/4926355622 ? I remember when i change common.platformTimeout(10) to common.platformTimeout(100), the test exits normally🤔.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@legendecas

legendecas commented Oct 26, 2022

Copy link
Copy Markdown
Member

@theanarkh yeah, IIUC that test case is exactly testing a premature exit while the async work is still in progress.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Oct 31, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47605/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request Nov 1, 2022
CI Reliability 2022-11-01 nodejs/reliability#417
Open
17 tasks
@theanarkh theanarkh added the commit-queue PRs queued for automated landing through the Commit Queue. label Nov 1, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Nov 1, 2022
@nodejs-github-bot
nodejs-github-bot merged commit 8fb282e into nodejs:main Nov 1, 2022
@nodejs-github-bot

nodejs-github-bot commented Nov 1, 2022

Copy link
Copy Markdown
Collaborator

Landed in 8fb282e

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

RafaelGSS pushed a commit that referenced this pull request Nov 1, 2022
@theanarkh @RafaelGSS
src: call uv_library_shutdown before DisposePlatform
4a31d2a
When the process exits, there may be tasks in the thread pool
that need to access data in the platform, such as trace agent.
So make sure the thread pool exits first.
see #44458

PR-URL: #45226
Refs: #44458
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Santiago Gimeno <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Nov 1, 2022
@theanarkh @RafaelGSS
src: trace threadpool event
7ccabbf
PR-URL: #44458
Reviewed-By: Chengzhong Wu <[email protected]>
@theanarkh theanarkh mentioned this pull request Nov 1, 2022
trace_events: add new categories #45266
Merged
4 tasks done
@RafaelGSS RafaelGSS mentioned this pull request Nov 1, 2022
v19.1.0 proposal #45269
Merged
This was referenced Nov 2, 2022
CI Reliability 2022-11-02 nodejs/reliability#418
Open
CI Reliability 2022-11-05 nodejs/reliability#419
Open
RafaelGSS pushed a commit that referenced this pull request Nov 10, 2022
@theanarkh @RafaelGSS
src: call uv_library_shutdown before DisposePlatform
42db849
When the process exits, there may be tasks in the thread pool
that need to access data in the platform, such as trace agent.
So make sure the thread pool exits first.
see #44458

PR-URL: #45226
Refs: #44458
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Santiago Gimeno <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Nov 10, 2022
@theanarkh @RafaelGSS
src: trace threadpool event
3e46ebd
PR-URL: #44458
Reviewed-By: Chengzhong Wu <[email protected]>
nodejs-github-bot pushed a commit that referenced this pull request Nov 19, 2022
@theanarkh
trace_events: add new categories
bcc704f
PR-URL: #45266
Refs: #45092
Refs: #44458
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
ruyadorno pushed a commit that referenced this pull request Nov 21, 2022
@theanarkh @ruyadorno
trace_events: add new categories
f441b04
PR-URL: #45266
Refs: #45092
Refs: #44458
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@danielleadams danielleadams mentioned this pull request Dec 30, 2022
v18.13.0 release proposal #46025
Merged
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
@theanarkh @danielleadams
src: call uv_library_shutdown before DisposePlatform
1a437c3
When the process exits, there may be tasks in the thread pool
that need to access data in the platform, such as trace agent.
So make sure the thread pool exits first.
see #44458

PR-URL: #45226
Refs: #44458
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Santiago Gimeno <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
@theanarkh @danielleadams
src: trace threadpool event
4ee33a4
PR-URL: #44458
Reviewed-By: Chengzhong Wu <[email protected]>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
@theanarkh @danielleadams
trace_events: add new categories
a2256ba
PR-URL: #45266
Refs: #45092
Refs: #44458
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
@theanarkh @danielleadams
src: call uv_library_shutdown before DisposePlatform
8c4ac6d
When the process exits, there may be tasks in the thread pool
that need to access data in the platform, such as trace agent.
So make sure the thread pool exits first.
see #44458

PR-URL: #45226
Refs: #44458
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Santiago Gimeno <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
@theanarkh @danielleadams
src: trace threadpool event
4634aa9
PR-URL: #44458
Reviewed-By: Chengzhong Wu <[email protected]>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
@theanarkh @danielleadams
trace_events: add new categories
210cbcb
PR-URL: #45266
Refs: #45092
Refs: #44458
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
danielleadams pushed a commit that referenced this pull request Jan 5, 2023
@theanarkh @danielleadams
trace_events: add new categories
31d2ce8
PR-URL: #45266
Refs: #45092
Refs: #44458
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell left review comments
@addaleax addaleax addaleax left review comments
@RaisinTen RaisinTen RaisinTen left review comments
@legendecas legendecas legendecas approved these changes
+1 more reviewer
@VoltrexKeyva VoltrexKeyva VoltrexKeyva left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@theanarkh @legendecas @nodejs-github-bot @jasnell @addaleax @RaisinTen @VoltrexKeyva
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.