GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [WIP] Persist resolutions and program information and reuse it when building program incrementally or during editor by sheetalkamat · Pull Request #41004 · microsoft/TypeScript

Conversation

@sheetalkamat

@sheetalkamat Sheetal Nandi (sheetalkamat) commented Oct 8, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This PR adds option persistResolutions which perists the program and resolutions information into the tsbuildinfo file. This option is meant to also mean that if module is resolved to file, dont watch any of the failed lookup locations for those, and always use that file as resolved file unless that file is deleted. The unresolved modules are still resolved again to see if it can be resolved at time of program creation.
This also adds option cleanPersistedProgram to remove the persisted program information (without loosing incremental build information) which helps in creating program as if this option was not enabled. So if user changes location of where the resolution needs to be, they can use this option.

The main changes are:

  • Program now takes oldProgram as ProgramFromBuildInfo which is built from program information saved in tsbuildinfo. The stored information is partial information needed to make decisions about whether program structure can be completely reused or only module resolutions can be safely used or cannot reuse at all.
  • Missing file paths are calculated on demand instead of saving them in program all the time
  • Source files store resolved modules and resolved type reference directives with their failed lookup locations
  • During watch, we watch failed lookup locations only if module resolution failed. Successful resolution is not watched.
  • Builder change to store the program information needed to make decisions on reuse of structure when persistResolutions is true. Need to be stored with and without --out option.
  • Editor reads the tsbuildinfo program and uses that as old program to generate new program so it can also benefit from saved resolutions
  • As part of this we also need to make sure that tsserver uses hash of source file text as version as otherwise we wont be able to tell if tsbuildinfo source file is same as version its referring to.
  • Rest of the changes are just some refactoring and making sure we are using versions, or partial program/sourceFile info instead of checking if they are strictly equal
  • Store if file was from project reference and program and oldProgram differ in useSourceOfProjectReferenceRedirect, then structure cannot be reused and in case of persistentResolution it means module resolution can be reused.
  • tsserver logs are generated as baseline to verify details instead of having to check projects programmatically so that its easier to update them

Questions to answer :

  1. Is name persistResolutions ok. Given we do need to store more information than just resolutions but resolutions are the main things that determine the behavior of if they should be retained or not.
  2. Is cleanPersistedProgram now useful. It was in the start because when i was experimenting , module name resolution whether failed or succeeded would be re-used so if you did npm install, you would want to run this command to clean up. Now that we still resolve failed lookups or if resolved file is deleted, things should flow more smoothly and just deleting tsbuildinfo file should be ok in that rare case.. The new behaviour works better with editing experience.
  3. To check how saving resolution when resolved to .js file for eg. is handled when new .d.ts or .ts file is created. (Because without allowJs the resolution will find the js file but wont be part of the program so do we need to do something special for these kind of scenarios) or just let user use the cleanPersistedProgram option. Will it be intuitive?
  4. Is cleanPersistedProgram option to be run and then restart the tsserver to handle the the project updates?
  5. What kind of reuse we are looking at if we only store module resolution
  6. Reuse type reference directive resolution from old program? We just reuse module resolution currently, should it also include type reference resolutions.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DanielRosenwasser and stagefright5 reacted with eyes emoji
All reactions
  • 👀 2 reactions
@typescript-bot TypeScript Bot (typescript-bot) assigned Sheetal Nandi (sheetalkamat) Oct 8, 2020
@typescript-bot TypeScript Bot (typescript-bot) added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 8, 2020
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch 2 times, most recently from cb79ae2 to 36d89be Compare October 12, 2020 18:28
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch from 2f726c1 to 18035fa Compare October 21, 2020 19:55
@sheetalkamat Sheetal Nandi (sheetalkamat) mentioned this pull request Dec 12, 2020
Use hash of source file text as version for the source file in tsserver #41938
Closed
@sheetalkamat Sheetal Nandi (sheetalkamat) mentioned this pull request Jan 6, 2021
Reuse Module Resolutions Within Project References During Initial Program Creation #40964
Closed
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch from 18035fa to d05151a Compare March 4, 2021 01:40
@sheetalkamat Sheetal Nandi (sheetalkamat) mentioned this pull request Mar 4, 2021
Ensure that when new file affecting global scope is added, the signatures are updated #43084
Merged
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch 3 times, most recently from aa893c3 to 7e761cd Compare March 24, 2021 00:38
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch from 073ed19 to fb609c0 Compare March 30, 2021 00:34
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch 4 times, most recently from 12ca63c to abe3b73 Compare May 18, 2021 23:13
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch from a0e33af to 7c202bd Compare May 26, 2021 00:02
@sheetalkamat Sheetal Nandi (sheetalkamat) changed the title [WIP] Persistent resolutions Persist resolutions and program information and reuse it when building program incrementally or during editor May 26, 2021
@sheetalkamat
Sheetal Nandi (sheetalkamat) requested review from Daniel Rosenwasser (DanielRosenwasser), Ryan Cavanaugh (RyanCavanaugh), Andrew Casey (amcasey), Andrew Branch (andrewbranch) and Nathan Shively-Sanders (sandersn) May 26, 2021 00:44
@sheetalkamat
Sheetal Nandi (sheetalkamat) marked this pull request as ready for review May 26, 2021 00:45
@DanielRosenwasser

Daniel Rosenwasser (DanielRosenwasser) commented May 26, 2021

Copy link
Copy Markdown
Member

This is a really interesting idea. What sorts of speed-ups/memory reduction savings have you seen on larger projects?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sheetalkamat Sheetal Nandi (sheetalkamat) changed the title Persist resolutions and program information and reuse it when building program incrementally or during editor [WIP] Persist resolutions and program information and reuse it when building program incrementally or during editor May 26, 2021
@sheetalkamat
Sheetal Nandi (sheetalkamat) marked this pull request as draft May 26, 2021 16:58
@andrewbranch Andrew Branch (andrewbranch) mentioned this pull request May 26, 2021
Ensure AutoImportProviderProject can share source files with main program #44274
Merged
Sheetal Nandi (sheetalkamat) added 19 commits June 2, 2021 14:59
@sheetalkamat
Handle typeref, reference directives and lib directives
d847b00
@sheetalkamat
Baseline readable file include reason
03ee004
@sheetalkamat
Map resolutions
00005be
@sheetalkamat
Refactor
a3e7aaa
@sheetalkamat
More tests
9d7cdad
@sheetalkamat
MissingFilePaths are on demand and dont need to be stored
0080cb5
@sheetalkamat
Tests for tsserver scenario
f0be476
@sheetalkamat
Actually make the tsserver tests with persistResolutions
0fc318e
@sheetalkamat
Refactoring
4dbd53f
@sheetalkamat
Persist resolutions in services
86a3b17
@sheetalkamat
Add more tests
8741321
@sheetalkamat
persistResolutions will reuse resolutions only if it is resolved
841b9ae
@sheetalkamat
Some more tests
688b635
@sheetalkamat
Do not watch failed lookup locations in persistResolutions if resolve…
8ec6dab
…d to a file

This ensure --watch and editor behaviour matches with what happens without watch and there is no confusion as well as we are not watching unnecessary things
@sheetalkamat
Add tests for auto type reference directives
76ec5cd
@sheetalkamat
Save names of auto type directive names so that we can update program…
4118283
… correctly
@sheetalkamat
Tests for project references and persistResolutions (without actually…
d604d19
… enabling persistResolutions)
@sheetalkamat
Tests with actual persistResolutions
5d658c6
@sheetalkamat
Handle the difference is useSourceOfProjectReferenceRedirect and proj…
f0db440
…ect reference files in the program so editor can use correct program
@sheetalkamat
Sheetal Nandi (sheetalkamat) force-pushed the persistentResolutions branch from 95eeb32 to f0db440 Compare June 2, 2021 23:41
@sheetalkamat
Merge branch 'main' into persistentResolutions
dafa644
@sheetalkamat Sheetal Nandi (sheetalkamat) mentioned this pull request Jun 8, 2021
[WIP] Instead of persisting whole program persist only resolutions #44500
Closed
@sheetalkamat Sheetal Nandi (sheetalkamat) added the Experiment A fork with an experimental idea which might not make it into master label Aug 11, 2022
@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) mentioned this pull request Aug 25, 2022
TypeScript 4.9 Iteration Plan #50457
Closed
@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) added this to the TypeScript 5.0.0 milestone Oct 31, 2022
@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) mentioned this pull request Oct 31, 2022
TypeScript 5.0 Iteration Plan #51362
Closed
@sheetalkamat Sheetal Nandi (sheetalkamat) closed this Nov 9, 2022
@sheetalkamat
Sheetal Nandi (sheetalkamat) deleted the persistentResolutions branch November 9, 2022 20:02
@gluxon

Brandon Cheng (gluxon) commented Nov 28, 2022

Copy link
Copy Markdown
Contributor

Found this PR again after seeing it on the TypeScript 5.0 roadmap.

This is a really interesting idea. What sorts of speed-ups/memory reduction savings have you seen on larger projects?

I think it'll be extremely significant. Back when I was investigating this for a large monorepo, I saw a 72.21% improvement in one of our team's packages. Details in #40964

Looking forward to some version of this in 5.0. Thanks!

justsml and iway1 reacted with eyes emoji
All reactions
  • 👀 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@microsoft Microsoft (microsoft) locked as resolved and limited conversation to collaborators Oct 21, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) Awaiting requested review from DanielRosenwasser
@amcasey Andrew Casey (amcasey) Awaiting requested review from amcasey
@andrewbranch Andrew Branch (andrewbranch) Awaiting requested review from andrewbranch
@RyanCavanaugh Ryan Cavanaugh (RyanCavanaugh) Awaiting requested review from RyanCavanaugh
@sandersn Nathan Shively-Sanders (sandersn) Awaiting requested review from sandersn
Copilot code review Copilot
Copilot will automatically review once the pull request is marked ready for review

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@sheetalkamat Sheetal Nandi (sheetalkamat)

Labels

Author: Team Experiment A fork with an experimental idea which might not make it into master For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

None yet

Milestone

TypeScript 5.0.0

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@sheetalkamat @DanielRosenwasser @gluxon @typescript-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.