GeezerVison is showing github.com — Exclude Objects from code coverage by wildcard · Issue #1053 · utPLSQL/utPLSQL
Exclude Objects from code coverage by wildcard #1053DescriptionIs your feature request related to a problem? Please describe.
We have a database schema which includes temporary objects - this means objects which are required only for a short time like procedure for upgrades or objects which are still work in progress - and legacy test code. Temporary objects are always prefixed with the username of the author like:
windowsusr_my_table
Test Procedures are prefixed with test_ and cannot be moved to utPLSQL because they do an integration test rather than a unit test as they access and read data from remote systems and therefore need manual checks of the data.
Describe the solution you'd like
We would like to exclude objects using a wildcard like:
windowsusr_*
test_*
Describe alternatives you've considered
I tried to list every object which currently exists and which should be excluded but gave up after about 15 names.
Additional context
We use Oracle 12.1 with Sqldeveloper and installed the utPLSQL plugin. Activityjgebal commented on Mar 12, 2020 We would need to make few decisions here:
- should we use % or * as wildcard aka Oracle vs. OS style
- should it be passed using the same parameters or should we create new ones
- should we use simple wildcard or REGEXP?
For No. 1 - I don't have an strong opinion, but whatever we choose should be taken as standard. No other input parameters accept wildcards so far so this would lay foundations for future enhancements in other areas.
For No. 2 - I would lean towards reuse of existing parameters. They would simply become more flexible.
I didn't check the code yet but as far as I remember it shouldn't be too hard to implement. wolframhaussig commented on Mar 12, 2020 Here are my 2 cents:
- I think we should use % as this is a framework for Oracle users so we should use the Oracle style
- I think reusing the parameters would be fine
- I think regular expressions are overpowered in this case although this would be helpful for us as the Windows-Usernames have a predefined structure and we could catch all usernames with a single RegEx instead of one wildcard entry for each developer.
jgebal commented on Mar 13, 2020 I was thinking a bit.
The currently accepted syntax makes it impossible to convert to regex support without breaking backward compatibility.
This is because you can pass schema.object or object and both are valid.
The . in current parameter syntax is the problem.
Given the above and the fact that filtering objects by using regexp_like can bring extra penalty on performance I think we would need to stick with regular wildcard filtering.
Wildcard filtering using like expression has also one pitfall. The underscore character ( _ ) is the default and most commonly used separator in object names. It is also a single character substitution in the like operator.
With that, the only way to introduce full wildcards in a non-breaking way would be to use OS-style substitution characters:
* - match zero or more characters? - match exactly one character
Another option would be to use Oracle wildcard matching but only to support % and not _ (match exactly one character). This however seems like a big limitation of functionality.
A mixed option would be to allow for use of * and % interchangeably but only allow for ? as a single-char match.
The more I think about it the more I lean towards using OS-style substitution characters.
Examples:
begin
ut.run(
'unit_test_schema', ut_coverage_html_reporter(),
a_include_objects => ut_varchar2_list(
'user_???.bonus*',
'???_user.*_process_*'
),
a_exclude_objects => ut_varchar2_list(
'user_???.*award*',
'???_user.*calc*'
)
);
end;@pesse @lwasylow @PhilippSalvisberg - any inputs from your side? Reacted by Philipp Salvisberg jgebal commented on Mar 13, 2020 Current documentation can be found here for reference jgebal commented on Mar 13, 2020 PhilippSalvisberg commented on Mar 13, 2020 @jgebal I agree with your proposal since it simplifies the handling of unquoted identifiers. However, it will make it more difficult in the future to support quoted identifiers (e.g. "My _strange% ., *Package? Name"). Quoted identifiers are probably not relevant for test code. But code under test is another story (might be not in control of the developer). So I wonder if it wouldn't be easier or cleaner to separate schema and object and provide dedicated parameters for include and exclude that accept a case-insensitive regular expression (I do not think that case-sensitive handling is ever needed, even if quoted names are case sensitive). Something like this
begin
ut.run(
a_path => 'unit_test_schema',
a_reporter => ut_coverage_html_reporter(),
a_include_schema_expr => '(user_...)|(..._user)',
a_include_object_expr => '(bonus.*)|(.*_process_.*)',
a_exclude_schema_expr => '(user_t..)|(t.._user)',
a_exclude_object_expr => '(.*award.*)|(.*calc.*)'
);
end;
Reacted by Lukasz Wasylow and Satya Padala lwasylow commented on Mar 13, 2020 Like idea of @PhilippSalvisberg. Nice granularity level enough to cover most requirements. I think simple like of oracle will be sufficient. Unfortunately any new input parameters will explode into load number of ut run overloads so maybe its time to introduce a new subtype that will go into include objects exclude objects that is a tab of varchar2 etc.? That way we keep old functionality not break and reduce new specs jgebal commented on Jun 28, 2020 @lwasylow
I think we could go along with suggestion from @PhilippSalvisberg without exploding the parameters.
We would add those parameters to all currently supported parameter combinations.
We would need to enforce precedence of parameters when both a_include_objects and a_include_object_expr are provided by
user.
This can be done in few ways:
- document that
a_include_object_expr overrides the a_include_objects
- raise exception when both are provided. note that the current pattern is not to raise exceptions on bad parameters but rather ignore them.
- accept both lists and use both for inclusion with an OR operator. For exclusions we would assume that we accept both as exclusion filters
That way it should be relatively easy to implement lwasylow commented on Feb 9, 2022 I have drafted a first checking of approach to the problem I as initial start decided to use logic as follow:
a_include_object_expr and a_include_schema_expr silently overwrites a_include_object_expr
I don't like the idea of raising exception as currently a_include_object_expr silently overwrites a_coverage_schemas anyway.
The approach of OR operator is tempting however I feel like a_include_object_expr is more specialized to extract info regarding one or two packages where a_include_object/schema_expr is more to filter results.
@PhilippSalvisberg @jgebal
What are your thoughts on this ? PhilippSalvisberg commented on Feb 9, 2022
The approach of OR operator is tempting however I feel like a_include_object_expr is more specialized to extract info regarding one or two packages where a_include_object/schema_expr is more to filter results.
What do you mean with the OR operator? The example I've drafted in this comment is using a single regular expression. lwasylow commented on Feb 9, 2022 @PhilippSalvisberg I was referring to proposal that @jgebal said an earlier
This can be done in few ways:
document that a_include_object_expr overrides the a_include_objects
raise exception when both are provided. note that the current pattern is not to raise exceptions on bad parameters but rather ignore them.
accept both lists and use both for inclusion with an OR operator. For exclusions we would assume that we accept both as exclusion filters
As for implementation I think your idea is good and implemented this that way. Only question remains about existing attributes / filters how they would behave with a new ones. PhilippSalvisberg commented on Feb 10, 2022
As for implementation I think your idea is good and implemented this that way. Only question remains about existing attributes / filters how they would behave with a new ones.
Personally I like errors better. It happens when someone uses the new feature. It's a precise feedback. It tells you that want you defined does not work. As a result you simply change the call. Done. Otherwise it "works" but maybe does not produce the result the expected result. I have to dig deeper to find out that something has has been ignored. Furthermore I think the implementation will become simpler when you clearly state what is allowed and want not. lwasylow commented on Feb 26, 2022 @jgebal, so currently I follow the principles as follows
- include regex silently overwrite include objects but dont interfere with exclude objects
- exclude regex silently overwrite exclude objects but wont interfere with include objects.
I prefer silent approach to be consistent with others, l but if you think its best to warn users abiut ignoring we could maybe see if we can push warning? Dont think error is good.
Second matter is precedence, do we want to fully overwrite include exclude when regex is supplied?
To clarify what I mean is when supply include regex will not interfere with exclude objects and similar for exclude version as per above.But we can make it so that regex fully take control of filter and include regex will ignore including and excluding objects ans use only regex.
Personally i think that is excessive and limit flexibility? jgebal commented on Feb 26, 2022 I understand where Philipp is coming from with the exceptions.
It is making it harder to understand what is going on when you don't get expected results (something doesn't work) and you don't know why.
Change of the appear from silently ignoring incompatible parameters to raising exceptions would however mean one of two things:
- we are inconsistent, if we only do this for those parameters
- we introduce a breaking change
Both are not acceptable for me at this time.
With that I suggest silent override.
We could provide warning on ignored parameters as a separate feature/enhancement.
Reacted by Philipp Salvisberg jgebal commented on Feb 26, 2022 Also, I need to read the entire coverage documentation to see if it is still consistent and clear. malugrz commented on Mar 7, 2023 I'm using utPLSQL-cli to run the tests on a build server. I found the parameter "-exclude" to exclude objects. It works with full names. However, I want to use patterns to exclude several objects with similar names. What would be the correct syntax for wildcards? I cannot find it in the documentation. lwasylow commented on Mar 7, 2023 Which version of utplsql your are using? We have introduced a new feature in latest release. Have you checked this? malugrz commented on Mar 8, 2023 I'm using v3.1.12.3589.
In the release notes there's only one feature mentioned: Add ability to run tests by part of a name.
Not sure if this is related to excluding objects for the coverage report.
- Do I need to upgrade?
- What would be the correct syntax for wildcards to exclude objects for the coverage report?
lwasylow commented on Mar 8, 2023 Ok I understand now your question.
So there is section in documentation
https://www.utplsql.org/utPLSQL/latest/userguide/coverage.html#filtering-objects-in-coverage-reports
I think what you looking for is
- a_exclude_schema_expr ( varchar(4000) ) - string of regex expression of schemas to be excluded from the coverage report. Case-insensitive.
- a_exclude_object_expr ( varchar(4000) ) - string of regex expression of objects ( without schema name ) to be excluded from the coverage report. Case-insensitive._
malugrz commented on Mar 9, 2023 Thank you for the link. However, it referes to the manual execution of tests with coverage. For the automatic execution I can only use the utPLSQL-cli. If you look at the code of the class RunPicocliCommand (https://github.com/utPLSQL/utPLSQL-cli/blob/develop/src/main/java/org/utplsql/cli/RunPicocliCommand.java) you can see there is no such parameter "a_exclude_schema_expr" or "a_exclude_object_expr".
However, there's the parameter "-exclude" wich works fine, if you pass the full name of the object. So far I've been not successful to pass regexp in this parameter nor do I know the correct format of an accepted regular expression.
Maybe utPLSQL-cli is not adapted yet to support exclusions à la a_exclude_object_expr? jgebal commented on Mar 9, 2023 utplsql-cli is not supporting the new parameters yet.
We need to extend cli as well as maven plugin to be up to date with utplsql api enchancements.
When it can be done is a question probably more for @pesse jgebal commented on Mar 9, 2023 @malugrz
Could you open a new issue on utplsql-cli for that? This issue can be linked for reference
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|