GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Parallelize the windows CI to enable us to run all tests all the time by TravisEz13 · Pull Request #8868 · PowerShell/PowerShell

Conversation

@TravisEz13

@TravisEz13 Travis Plunk (TravisEz13) commented Feb 12, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

PR Summary

Split Windows CI into the following Jobs

  1. Combine Build and xUnit, which published build artifact
  2. Elevated CI tests
  3. Unelevated CI test
  4. Elevated everything else tests
  5. Unelevated everything else tests
  6. Package build and test
  7. Verify xUnit result

PR Context

Before this test:

  • A CI run would take ~14 minutes
  • A Feature run would take ~30 minutes

After this change, it always runs all tests and takes ~20 minutes.
Job Unelevated everything else tests is the long poll, taking ~4 minutes longer than the next shortest job. We should be able to get this back down to ~15 if we split that test job somehow.

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Change is not breaking
  • Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
  • User-facing changes
    • Not Applicable
    • OR
    • Documentation needed
      • Issue filed:
  • Testing - New and feature
    • N/A or can only be tested interactively
    • OR
    • Make sure you've added a new test if existing tests do not effectively test the code changed
      • Add [feature] to your commit messages if the change is significant or affects feature tests

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Travis Plunk (TravisEz13) added 8 commits February 11, 2019 15:10
@TravisEz13
remove duplicate code
392714f
@TravisEz13
add tagset feature - ci.psm1
3699e90
@TravisEz13
refactor xUnit into build - ci.psm1
9064c2b
@TravisEz13
save and publish build
02ad617
@TravisEz13
add build template
de8dce9
@TravisEz13
add test template
ff0c88c
@TravisEz13
add packaging template
4edb6e7
@TravisEz13
use templates
a43a89f
@TravisEz13
Travis Plunk (TravisEz13) requested review from Paul Higinbotham (PaulHigin), Aditya Patwardhan (adityapatwardhan), Andrew (anmenaga) and Dongbo Wang (daxian-dbw) as code owners February 12, 2019 01:32
@TravisEz13
fix codefactor issues
03806c4
@TravisEz13
Travis Plunk (TravisEz13) force-pushed the split_package_CI branch from 03df89d to 03806c4 Compare February 12, 2019 01:43
@TravisEz13 Travis Plunk (TravisEz13) changed the title Split package ci Parallelize the windows CI to enable us to run all tests all the time Feb 12, 2019
@iSazonov

Ilya (iSazonov) commented Feb 12, 2019

Copy link
Copy Markdown
Collaborator

We should be able to get this back down to ~15 if we split that test job somehow.

By projects?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@TravisEz13

Travis Plunk (TravisEz13) commented Feb 12, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Ilya (@iSazonov) I don't want to design the solution right now. By project, would be a possibility, but optimally, we should try to find a way that makes sense and divides the time close to evenly.

I think our first goal would be to do a similar refactoring of the non-windows CI system.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@iSazonov

Ilya (iSazonov) commented Feb 13, 2019 •
edited by TravisEz13
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

Travis Plunk (@TravisEz13) Why we do not select Build in a separate job to use the result for the rest jobs?

Also perhaps we could use strategy-matrix to unify process.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@TravisEz13

Travis Plunk (TravisEz13) commented Feb 13, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Ilya (@iSazonov) Review the change. We do the build as a separate job and use the results (artifacts) for the rest of the jobs.

Matrix is less flexible that designing the pipeline yourself. We would not be able to do the build is a separate job and use the artifacts for the rest of the jobs.

Using templates gets you all the advantages of a matrix (not having to restate the job multiple times, only declare how many times you want to run it), with the added ability to depend on previous jobs.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
adityapatwardhan
Aditya Patwardhan (adityapatwardhan) approved these changes Feb 13, 2019
View reviewed changes
Comment thread tools/ci.psm1
$SkipFailing
)
$env:CoreOutput = Split-Path -Parent (Get-PSOutput -Options (Get-PSOptions))
if(!(Test-Path "$env:CoreOutput\pwsh.exe"))

@adityapatwardhan Aditya Patwardhan (adityapatwardhan) Feb 13, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to work on Linux and macOS

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@iSazonov Ilya (iSazonov) Feb 14, 2019

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the PR xUnit tests run only on Windows. Previously we run the tests on Unix-s too.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@TravisEz13 Travis Plunk (TravisEz13) Feb 15, 2019

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR doesn't change Linux or macOS. I'll deal with this issue when refactoring macOS and Linux

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@iSazonov

Ilya (iSazonov) commented Feb 14, 2019

Copy link
Copy Markdown
Collaborator

Travis Plunk (@TravisEz13) Thanks for clarify! I see now how this works.

For reference Setting up parallel jobs, Slicing the test suite
https://docs.microsoft.com/en-us/azure/devops/pipelines/test/parallel-testing-any-test-runner?view=azure-devops#setting-up-parallel-jobs
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
PaulHigin
Paul Higinbotham (PaulHigin) approved these changes Feb 14, 2019
View reviewed changes
@TravisEz13
Travis Plunk (TravisEz13) merged commit a7f7985 into PowerShell:master Feb 15, 2019
@TravisEz13
Travis Plunk (TravisEz13) deleted the split_package_CI branch February 15, 2019 00:33
@iSazonov Ilya (iSazonov) assigned Travis Plunk (TravisEz13) Feb 15, 2019
@TravisEz13 Travis Plunk (TravisEz13) added the CL-Test Indicates that a PR should be marked as a test change in the Change Log label Feb 27, 2019
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
@TravisEz13
Parallelize the windows CI to enable us to run all tests all the time (…
88dd423 
…PowerShell#8868)

Split Windows CI into the following Jobs

1. Combine Build and xUnit, which published build artifact
1. Elevated CI tests
1. Unelevated CI test
1. Elevated everything else tests
1. Unelevated everything else tests
1. Package build and test
1. Verify xUnit result

## PR Context  

Before this test:

- A CI run would take ~14 minutes
- A Feature run would take ~30 minutes

After this change, it always runs all tests and takes ~20 minutes.
Job `Unelevated everything else tests` is the long poll, taking ~4 minutes longer than the next shortest job.  We should be able to get this back down to ~15 if we split that test job somehow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iSazonov Ilya (iSazonov) iSazonov left review comments
@adityapatwardhan Aditya Patwardhan (adityapatwardhan) adityapatwardhan approved these changes
@anmenaga Andrew (anmenaga) Awaiting requested review from anmenaga
@daxian-dbw Dongbo Wang (daxian-dbw) Awaiting requested review from daxian-dbw
+1 more reviewer
@PaulHigin Paul Higinbotham (PaulHigin) PaulHigin approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@TravisEz13 Travis Plunk (TravisEz13)

Labels

CL-Test Indicates that a PR should be marked as a test change in the Change Log

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@TravisEz13 @iSazonov @adityapatwardhan @PaulHigin
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.