GeezerVison is showing github.com — Random order of test execution · Issue #422 · utPLSQL/utPLSQL
Random order of test execution #422DescriptionDefine suite level annotation to shuffle tests on execution Activityjgebal commented on Jul 26, 2017 It should be a runtime paramerer I think.
This way you can easily check for test independency.
Also, do we want to have random or simply reverse order? mathewbutler commented on Jul 26, 2017 On of the principles of xunit test approaches is that tests should be independent and so relying on ordering of tests (outside of before or beforeall annotation) is considered "a bad thing".
Maintaining independent tests means that it is easier to diagnose failed test since the programmer only needs to look at a single test scope to understand everything needed to diagnose.
Enabling this capability feels like a change in direction of ethos of a test suite.
Just my 0.2c 😀 Reacted by Roxane Sancho y Fran jgebal commented on Aug 31, 2017 That is why it should be a runtime parameter, not an annotation I think.
Order (random or not) should not be part of test-suite definition.
I love the idea of random or even simply reverse order of execution in suite.
The ordering should not affect the setup/cleanup order though. jgebal commented on Oct 24, 2017
Enabling this capability feels like a change in direction of ethos of a test suite
@mathewbutler - It would definitely be a positive change. mathewbutler commented on Oct 25, 2017 @jgebal I’m just saying that we shouldn’t introduce anything that enables a repeatable order of execution of tests. Random is by definition not reliably repeatable, so fine. Shoelace commented on Oct 9, 2018 if a 'random' option is available at runtime (or annotation) then maybe a seed number should be output somewhere and allowed to be passed in should you need to recreate a "random run" to diagnose the issue. lwasylow commented on Feb 26, 2019 jgebal commented on Feb 26, 2019 Yes. And this feature should be quite easy to implement now (hopefully) since the suote is build by running SQL on cache.
The only thing to confirm is that it will not have negative impact on clients (SQLDeveloper) PhilippSalvisberg commented on Feb 26, 2019 The SQL Developer extension does not directly executes the tests. It just calls ut.run or ut_runner.run. Everything is fine as long as it can rely on the sort order provided by the reporters. added a commit that references this issue on Mar 26, 2019This issue will close once commit 23da3d7 is merged into the 'develop' branch.23da3d7 added a commit that references this issue on Jun 18, 2019This issue will close once commit 1a5c525 is merged into the 'develop' branch.1a5c525
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|