GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src: use STL containers instead of v8 values for static module data by joyeecheung · Pull Request #24384 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Nov 15, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::unordered_maps. This has the following benefits:

  • It's slightly lighter in weight compared to storing things on the
    v8 heap. Also it may be slightly faster since the preivous v8::Object is
    already in dictionary mode - though the difference is very small
    given the number of native modules is limited.
  • The source and code cache generation templates are now much simpler
    since they just initialize static arrays and manipulate STL
    constructs. They are also easier to debug from the C++'s side,
    especially early in the bootstrap process when no inspector
    can be attached.
  • The static native module data can be accessed independent of any
    Environment or Isolate, and it's easy to look them up from the
    C++'s side.
  • It's now impossible to mutate the source code used to compile
    native modules from the JS land since it's completely separate
    from the v8 heap. We can still get the constant strings from
    process.binding('natives') but that's all.

A few drive-by fixes:

  • Remove DecorateErrorStack in LookupAndCompile - We don't need to
    capture the exception to decorate when we encounter
    errors during native module compilation, as those errors should be
    syntax errors and v8 is able to decorate them well. We use
    CompileFunctionInContext so there is no need to worry about
    wrappers either.
  • The code cache could be rejected when node is started with v8 flags.
    Instead of aborting in that case, simply keep a record in the
    native_module_without_cache set.
  • Refactor js2c.py a bit, reduce code duplication and inline Render()
    to make the one-byte/two-byte special treatment easier to read.
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nodejs-github-bot

nodejs-github-bot commented Nov 15, 2018

Copy link
Copy Markdown
Collaborator

@joyeecheung build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1626/pipeline

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. labels Nov 15, 2018
@joyeecheung

joyeecheung commented Nov 15, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

CI: https://ci.nodejs.org/job/node-test-pull-request/18661/
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/275/

cc @addaleax @devsnek @refack

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Nov 15, 2018

Copy link
Copy Markdown
Member Author

Also @hashseed as I understand there is no way to use the cache if node is started with v8 flags which leads to a mismatch of the flag hash in the serialized code cache data, is that correct?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

addaleax
addaleax approved these changes Nov 15, 2018
View reviewed changes
Comment thread src/node_native_module.cc Outdated

@addaleax addaleax Nov 15, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, maybe this one could better be implemented as an ToV8Value() overload from util.h? It fits in with the others pretty well, I think (the downside being that we always use UTF8 there unconditionally…)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@joyeecheung joyeecheung Nov 16, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax The current implementation is only capable of converting std::set<const std::string> to an array...not exactly what we want here, but I can leave a TODO

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/node_native_module.h Outdated

@addaleax addaleax Nov 15, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, did you consider using std::unordered_map and std::unordered_set over std::map/std::set? I don’t know if we need to worry a lot about the performance tradeoffs here, but I personally tend to use the unordered versions when I can

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@joyeecheung joyeecheung Nov 16, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is a reason not to use the unordered version...good idea, I'll change that

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
refack
refack reviewed Nov 16, 2018
View reviewed changes
Comment thread src/node_native_module.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

refack
refack reviewed Nov 16, 2018
View reviewed changes
Comment thread tools/generate_code_cache.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

refack
refack reviewed Nov 16, 2018
View reviewed changes

@refack refack left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow! 😮 Looks great.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
fhinkel
fhinkel approved these changes Nov 16, 2018
View reviewed changes
@joyeecheung
joyeecheung force-pushed the map branch from d26e166 to 9b60fd6 Compare November 16, 2018 14:05
@refack

refack commented Nov 16, 2018

Copy link
Copy Markdown
Contributor

@joyeecheung could you gist the new node_javascript.cc and node_code_cache.cc? I'm curious how did they turn out looking like.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Nov 16, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@refack If I remove the static char data, they look like this: https://gist.github.com/joyeecheung/9b41e0b9309d9492270ba23a92e2733d

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Nov 16, 2018

Copy link
Copy Markdown
Member Author

Thanks for the reviews, updated.

CI: https://ci.nodejs.org/job/node-test-pull-request/18673/
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/276/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@refack

refack commented Nov 16, 2018

Copy link
Copy Markdown
Contributor

@refack If I remove the static char data, they look like this: gist.github.com/joyeecheung/9b41e0b9309d9492270ba23a92e2733d

IMO they look much better than before 😄

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the map branch from 9b60fd6 to 5a81e0d Compare November 16, 2018 15:22
@joyeecheung

joyeecheung commented Nov 16, 2018

Copy link
Copy Markdown
Member Author

Rebased: https://ci.nodejs.org/job/node-test-pull-request/18675/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Nov 16, 2018

Copy link
Copy Markdown
Member Author

So...interesting enough, this uncovers a circular dependency in internal/streams/lazy_transform.js on the pis - since now the order of keys in internalBinding('native_module').source depends on the implementation of unordered_map (previously it depends on the order by which js2c inserts the strings into the v8::Object), now on the pis somehow lazy_transform.js can be required before crypto.js is required which triggers the dependency issue.

I opened #24396 to fix this.

refack reacted with laugh emoji refack reacted with heart emoji
All reactions
  • 😄 1 reaction
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung joyeecheung added the blocked PRs that are blocked by other issues or PRs. label Nov 16, 2018
@joyeecheung joyeecheung mentioned this pull request Nov 16, 2018
stream: do not use crypto.DEFAULT_ENCODING in lazy_transform.js #24396
Closed
3 tasks done
TimothyGu
TimothyGu reviewed Nov 16, 2018
View reviewed changes
Comment thread src/node_native_module.cc Outdated

@TimothyGu TimothyGu Nov 16, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the return value cached / should it be cached? Right now it seems like every time internalBinding('native_module').source is accessed a new object is created.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@joyeecheung joyeecheung Nov 16, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not cached, but that does not really matter since this is only used by tools/generate_code_cache.js, the native module loader in the binary only gets to call into a C++ function that can access the std::unordered_map directly.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TimothyGu reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@joyeecheung

joyeecheung commented Nov 16, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

So...interesting: std::unordered_map is slower than std::map in this case

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/275

                                                                                   confidence improvement accuracy (*)   (**)  (***)
 misc/startup.js mode='process' script='benchmark/fixtures/require-cachable' dur=1                -0.95 %       ±1.05% ±1.40% ±1.82%
 misc/startup.js mode='process' script='test/fixtures/semicolon' dur=1                      *      1.46 %       ±1.23% ±1.64% ±2.14%
 misc/startup.js mode='worker' script='benchmark/fixtures/require-cachable' dur=1                 -0.35 %       ±0.85% ±1.13% ±1.47%
 misc/startup.js mode='worker' script='test/fixtures/semicolon' dur=1                             -0.09 %       ±1.06% ±1.41% ±1.83%

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/276

                                                                                   confidence improvement accuracy (*)   (**)  (***)
 misc/startup.js mode='process' script='benchmark/fixtures/require-cachable' dur=1         **     -1.58 %       ±1.18% ±1.57% ±2.04%
 misc/startup.js mode='process' script='test/fixtures/semicolon' dur=1                             0.13 %       ±1.37% ±1.82% ±2.37%
 misc/startup.js mode='worker' script='benchmark/fixtures/require-cachable' dur=1         ***     -2.43 %       ±0.86% ±1.14% ±1.49%
 misc/startup.js mode='worker' script='test/fixtures/semicolon' dur=1                              0.12 %       ±0.87% ±1.16% ±1.51%

I tried to google around and turns out that's not quite surprising, given that we only have < 200 items, and we only insert items and don't touch them once the map is initialized, hashing on the strings could be slower than searching in the tree. e.g. See https://stackoverflow.com/questions/4846798/why-would-map-be-much-faster-than-unordered-map (this almost tempted me to try implementing a trie..nah)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the map branch from 470d91d to e1c5eb8 Compare November 17, 2018 02:13
@Trott

Trott commented Nov 18, 2018

Copy link
Copy Markdown
Member

Removing blocked label because 413fcad landed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Trott Trott removed the blocked PRs that are blocked by other issues or PRs. label Nov 18, 2018
@refack

refack commented Nov 18, 2018

Copy link
Copy Markdown
Contributor

CI (rebased on master): https://ci.nodejs.org/job/node-test-pull-request/18724/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TimothyGu
TimothyGu approved these changes Nov 18, 2018
View reviewed changes

@TimothyGu TimothyGu left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than some fairly superficial comments.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread tools/generate_code_cache.js Outdated

@TimothyGu TimothyGu Nov 18, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use isUint8Array from util.types?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

joyeecheung reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread tools/generate_code_cache.js Outdated

@TimothyGu TimothyGu Nov 18, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change should be unneeded right?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@joyeecheung joyeecheung Nov 19, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TimothyGu Yeah, just so that it's clearer since this length is only used to calculate a string displayed in KB/MB etc. human readable format.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@joyeecheung joyeecheung Nov 19, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll switch this to const size = cachedData.byteLength - I think that makes this a bit more self-explanatory and less ambiguous

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/node.cc Outdated

@TimothyGu TimothyGu Nov 18, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: bootstrap

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

joyeecheung reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread src/node_native_module.cc Outdated

@TimothyGu TimothyGu Nov 18, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d make this a method, as it signals that getting the value is a relatively expensive operation (and that the returned object is different every time this is called).

Ditto for GetCacheUsage().

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

joyeecheung reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
jasnell
jasnell approved these changes Nov 19, 2018
View reviewed changes
joyeecheung added 2 commits November 19, 2018 19:46
@joyeecheung
src: use STL containers instead of v8 values for static module data
52608c8
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:

- It's slightly lighter in weight compared to storing things on the
  v8 heap. Also it may be slightly faster since the preivous v8::Object is
  already in dictionary mode - though the difference is very small
  given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
  since they just initialize static arrays and manipulate STL
  constructs. They are also easier to debug from the C++'s side,
  especially early in the bootstrap process when no inspector
  can be attached.
- The static native module data can be accessed independent of any
  Environment or Isolate, and it's easy to look them up from the
  C++'s side.
- It's now impossible to mutate the source code used to compile
  native modules from the JS land since it's completely separate
  from the v8 heap. We can still get the constant strings from
  process.binding('natives') but that's all.

A few drive-by fixes:

- Remove DecorateErrorStack in LookupAndCompile - We don't need to
  capture the exception to decorate when we encounter
  errors during native module compilation, as those errors should be
  syntax errors and v8 is able to decorate them well. We use
  CompileFunctionInContext so there is no need to worry about
  wrappers either.
- The code cache could be rejected when node is started with v8 flags.
  Instead of aborting in that case, simply keep a record in the
  native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
  to make the one-byte/two-byte special treatment easier to read.
@joyeecheung
fixup! src: use STL containers instead of v8 values for static module…
161c9f8
… data
@joyeecheung

joyeecheung commented Nov 19, 2018

Copy link
Copy Markdown
Member Author

Fixed a few more nits & rebased & squashed the fixup into one. CI: https://ci.nodejs.org/job/node-test-pull-request/18758/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Nov 19, 2018

Copy link
Copy Markdown
Member Author

Landed in 7778c03, thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung joyeecheung closed this Nov 19, 2018
joyeecheung added a commit that referenced this pull request Nov 19, 2018
@joyeecheung
src: use STL containers instead of v8 values for static module data
7778c03
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:

- It's slightly lighter in weight compared to storing things on the
  v8 heap. Also it may be slightly faster since the preivous v8::Object
  is already in dictionary mode - though the difference is very small
  given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
  since they just initialize static arrays and manipulate STL
  constructs.
- The static native module data can be accessed independently of any
  Environment or Isolate, and it's easy to look them up from the
  C++'s side.
- It's now impossible to mutate the source code used to compile
  native modules from the JS land since it's completely separate
  from the v8 heap. We can still get the constant strings from
  process.binding('natives') but that's all.

A few drive-by fixes:

- Remove DecorateErrorStack in LookupAndCompile - We don't need to
  capture the exception to decorate when we encounter
  errors during native module compilation, as those errors should be
  syntax errors and v8 is able to decorate them well. We use
  CompileFunctionInContext so there is no need to worry about
  wrappers either.
- The code cache could be rejected when node is started with v8 flags.
  Instead of aborting in that case, simply keep a record in the
  native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
  to make the one-byte/two-byte special treatment easier to read.

PR-URL: #24384
Fixes: https://github.com/Remove
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@joyeecheung joyeecheung added the backport-requested-v10.x label Nov 19, 2018
@joyeecheung

joyeecheung commented Nov 19, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

This should be backported together with #24382 and #24382 (for 10.x at least, I am not sure if we need to backport this to 8.x? Mostly because I don't know if at this point 8.x patches are backported on an as-needed basis or we are still trying to backport everything possible onto 8.x)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

targos pushed a commit that referenced this pull request Nov 19, 2018
@joyeecheung @targos
src: use STL containers instead of v8 values for static module data
3eb94a2
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:

- It's slightly lighter in weight compared to storing things on the
  v8 heap. Also it may be slightly faster since the preivous v8::Object
  is already in dictionary mode - though the difference is very small
  given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
  since they just initialize static arrays and manipulate STL
  constructs.
- The static native module data can be accessed independently of any
  Environment or Isolate, and it's easy to look them up from the
  C++'s side.
- It's now impossible to mutate the source code used to compile
  native modules from the JS land since it's completely separate
  from the v8 heap. We can still get the constant strings from
  process.binding('natives') but that's all.

A few drive-by fixes:

- Remove DecorateErrorStack in LookupAndCompile - We don't need to
  capture the exception to decorate when we encounter
  errors during native module compilation, as those errors should be
  syntax errors and v8 is able to decorate them well. We use
  CompileFunctionInContext so there is no need to worry about
  wrappers either.
- The code cache could be rejected when node is started with v8 flags.
  Instead of aborting in that case, simply keep a record in the
  native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
  to make the one-byte/two-byte special treatment easier to read.

PR-URL: #24384
Fixes: https://github.com/Remove
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
rvagg pushed a commit that referenced this pull request Nov 28, 2018
@joyeecheung @rvagg
src: use STL containers instead of v8 values for static module data
21e9aa2
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:

- It's slightly lighter in weight compared to storing things on the
  v8 heap. Also it may be slightly faster since the preivous v8::Object
  is already in dictionary mode - though the difference is very small
  given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
  since they just initialize static arrays and manipulate STL
  constructs.
- The static native module data can be accessed independently of any
  Environment or Isolate, and it's easy to look them up from the
  C++'s side.
- It's now impossible to mutate the source code used to compile
  native modules from the JS land since it's completely separate
  from the v8 heap. We can still get the constant strings from
  process.binding('natives') but that's all.

A few drive-by fixes:

- Remove DecorateErrorStack in LookupAndCompile - We don't need to
  capture the exception to decorate when we encounter
  errors during native module compilation, as those errors should be
  syntax errors and v8 is able to decorate them well. We use
  CompileFunctionInContext so there is no need to worry about
  wrappers either.
- The code cache could be rejected when node is started with v8 flags.
  Instead of aborting in that case, simply keep a record in the
  native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
  to make the one-byte/two-byte special treatment easier to read.

PR-URL: #24384
Fixes: https://github.com/Remove
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@BridgeAR BridgeAR mentioned this pull request Dec 5, 2018
v11.4.0 proposal #24854
Merged
4 tasks done
This was referenced Dec 7, 2018
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#261
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#262
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#263
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#264
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#265
Closed
deepak1556 added a commit to electron/electron that referenced this pull request Dec 19, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
20b2270
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 19, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
268e8f6
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 19, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
4fa0196
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 20, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
ae725c6
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 24, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
e2bb541
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 24, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
d4c8b97
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 25, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
34c3f01
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Dec 26, 2018
@deepak1556
fix: Use per process native module loader for compiled JS source
6bd8872
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Jan 3, 2019
@deepak1556
fix: Use per process native module loader for compiled JS source
05c1ca2
nodejs/node#24384
nornagon pushed a commit to electron/electron that referenced this pull request Jan 3, 2019
@deepak1556 @nornagon
fix: Use per process native module loader for compiled JS source
72bccaa
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Jan 4, 2019
@deepak1556
fix: Use per process native module loader for compiled JS source
d05ad20
nodejs/node#24384
deepak1556 added a commit to deepak1556/atom-shell that referenced this pull request Jan 8, 2019
@deepak1556
fix: Use per process native module loader for compiled JS source
65a1776
nodejs/node#24384
deepak1556 added a commit to electron/electron that referenced this pull request Jan 10, 2019
@deepak1556
fix: Use per process native module loader for compiled JS source
b652958
nodejs/node#24384
nornagon pushed a commit to electron/electron that referenced this pull request Jan 11, 2019
@deepak1556 @nornagon
fix: Use per process native module loader for compiled JS source
3a3075b
nodejs/node#24384
nornagon pushed a commit to electron/electron that referenced this pull request Jan 11, 2019
@deepak1556 @nornagon
fix: Use per process native module loader for compiled JS source
e3be342
nodejs/node#24384
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
@joyeecheung @refack
src: use STL containers instead of v8 values for static module data
4db3b26
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:

- It's slightly lighter in weight compared to storing things on the
  v8 heap. Also it may be slightly faster since the preivous v8::Object
  is already in dictionary mode - though the difference is very small
  given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
  since they just initialize static arrays and manipulate STL
  constructs.
- The static native module data can be accessed independently of any
  Environment or Isolate, and it's easy to look them up from the
  C++'s side.
- It's now impossible to mutate the source code used to compile
  native modules from the JS land since it's completely separate
  from the v8 heap. We can still get the constant strings from
  process.binding('natives') but that's all.

A few drive-by fixes:

- Remove DecorateErrorStack in LookupAndCompile - We don't need to
  capture the exception to decorate when we encounter
  errors during native module compilation, as those errors should be
  syntax errors and v8 is able to decorate them well. We use
  CompileFunctionInContext so there is no need to worry about
  wrappers either.
- The code cache could be rejected when node is started with v8 flags.
  Instead of aborting in that case, simply keep a record in the
  native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
  to make the one-byte/two-byte special treatment easier to read.

PR-URL: nodejs#24384
Fixes: https://github.com/Remove
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell approved these changes
@addaleax addaleax addaleax approved these changes
+3 more reviewers
@refack refack refack left review comments
@fhinkel fhinkel fhinkel approved these changes
@TimothyGu TimothyGu TimothyGu approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@joyeecheung @nodejs-github-bot @refack @Trott @fhinkel @jasnell @addaleax @TimothyGu
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.