GeezerVison is showing github.com — [WIP] Persist resolutions and program information and reuse it when building program incrementally or during editor by sheetalkamat · Pull Request #41004 · microsoft/TypeScript
Conversation
This PR adds option persistResolutions which perists the program and resolutions information into the tsbuildinfo file. This option is meant to also mean that if module is resolved to file, dont watch any of the failed lookup locations for those, and always use that file as resolved file unless that file is deleted. The unresolved modules are still resolved again to see if it can be resolved at time of program creation.
This also adds option cleanPersistedProgram to remove the persisted program information (without loosing incremental build information) which helps in creating program as if this option was not enabled. So if user changes location of where the resolution needs to be, they can use this option.
The main changes are:
- Program now takes
oldProgram as ProgramFromBuildInfo which is built from program information saved in tsbuildinfo. The stored information is partial information needed to make decisions about whether program structure can be completely reused or only module resolutions can be safely used or cannot reuse at all.
- Missing file paths are calculated on demand instead of saving them in program all the time
- Source files store resolved modules and resolved type reference directives with their failed lookup locations
- During watch, we watch failed lookup locations only if module resolution failed. Successful resolution is not watched.
- Builder change to store the program information needed to make decisions on reuse of structure when
persistResolutions is true. Need to be stored with and without --out option.
- Editor reads the tsbuildinfo program and uses that as old program to generate new program so it can also benefit from saved resolutions
- As part of this we also need to make sure that tsserver uses hash of source file text as version as otherwise we wont be able to tell if tsbuildinfo source file is same as version its referring to.
- Rest of the changes are just some refactoring and making sure we are using versions, or partial program/sourceFile info instead of checking if they are strictly equal
- Store if file was from project reference and program and
oldProgram differ in useSourceOfProjectReferenceRedirect, then structure cannot be reused and in case of persistentResolution it means module resolution can be reused.
- tsserver logs are generated as baseline to verify details instead of having to check projects programmatically so that its easier to update them
Questions to answer :
- Is name
persistResolutions ok. Given we do need to store more information than just resolutions but resolutions are the main things that determine the behavior of if they should be retained or not.
- Is
cleanPersistedProgram now useful. It was in the start because when i was experimenting , module name resolution whether failed or succeeded would be re-used so if you did npm install, you would want to run this command to clean up. Now that we still resolve failed lookups or if resolved file is deleted, things should flow more smoothly and just deleting tsbuildinfo file should be ok in that rare case.. The new behaviour works better with editing experience.
- To check how saving resolution when resolved to .js file for eg. is handled when new .d.ts or .ts file is created. (Because without allowJs the resolution will find the js file but wont be part of the program so do we need to do something special for these kind of scenarios) or just let user use the
cleanPersistedProgram option. Will it be intuitive?
- Is
cleanPersistedProgram option to be run and then restart the tsserver to handle the the project updates?
- What kind of reuse we are looking at if we only store module resolution
- Reuse type reference directive resolution from old program? We just reuse module resolution currently, should it also include type reference resolutions.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
DanielRosenwasser and stagefright5 reacted with eyes emoji
|
This is a really interesting idea. What sorts of speed-ups/memory reduction savings have you seen on larger projects?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sheetal Nandi (sheetalkamat)
changed the title
Persist resolutions and program information and reuse it when building program incrementally or during editor
[WIP] Persist resolutions and program information and reuse it when building program incrementally or during editor
May 26, 2021
…d to a file
This ensure --watch and editor behaviour matches with what happens without watch and there is no confusion as well as we are not watching unnecessary things
… enabling persistResolutions)
…ect reference files in the program so editor can use correct program
|
Found this PR again after seeing it on the TypeScript 5.0 roadmap.
This is a really interesting idea. What sorts of speed-ups/memory reduction savings have you seen on larger projects?
I think it'll be extremely significant. Back when I was investigating this for a large monorepo, I saw a 72.21% improvement in one of our team's packages. Details in #40964
Looking forward to some version of this in 5.0. Thanks!
|
justsml and iway1 reacted with eyes emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
Copilot will automatically review once the pull request is marked ready for review
Uh oh!
There was an error while loading. Please reload this page.
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|