GeezerVison

When building reusable expectation procedures, the failure message doesn't provide sufficient in formation for identifying the lines where expectations have failures.

So if I have few tests using the same procedure and each test has more than one expectation, it's really hard or maybe impossible to track which one has failed.

Example:

create or replace package check_stack is
  --%suite

  --%test
  procedure test_1;
  
  --%test
  procedure test_2;
  
end;
/

create or replace package body check_stack is

  procedure do_check( p_actual varchar2, p_matcher ut_matcher ) is 
  begin
    ut.expect(p_actual).to_( p_matcher );
  end;
  
  function get_actual return varchar2 is
  begin
    return chr(ascii('a') + trunc(dbms_random.value(0,3)));
  end;

  procedure test_1 is
  begin
    do_check( get_actual(), equal('b'));
    do_check( get_actual(), equal('b'));
    do_check( get_actual(), equal('b'));
  end;  

  procedure test_2 is
  begin
    do_check('ab'||get_actual()||'ef', be_like('%c%'));
    do_check('ab'||get_actual()||'ef', be_like('%c%'));
    do_check('ab'||get_actual()||'ef', be_like('%c%'));
  end;

end;
/

exec ut.run('check_stack');

Can produce outputs as below:

check_stack
  test_1 [.005 sec] (FAILED - 1)
  test_2 [.008 sec] (FAILED - 2)
 
Failures:
 
  1) test_1
      Actual: 'c' (varchar2) was expected to equal: 'b' (varchar2)
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      
       
  2) test_2
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      
       
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      
       
Finished in .017699 seconds
2 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)

Proposed solution

On failures, show full stack of calls within a test so that the outputs from below failures could look like this:

check_stack
  test_1 [.005 sec] (FAILED - 1)
  test_2 [.008 sec] (FAILED - 2)
 
Failures:
 
  1) test_1
      Actual: 'c' (varchar2) was expected to equal: 'b' (varchar2)
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      at "UT3.CHECK_STACK.DO_CHECK", line 16
      
       
  2) test_2
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      at "UT3.CHECK_STACK.DO_CHECK", line 22
      
       
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      at "UT3.CHECK_STACK.DO_CHECK", line 24
      
       
Finished in .017699 seconds
2 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)

With such addition, it will be much easier to navigate to the rig place in test code and identify the root cause of failure.

@PhilippSalvisberg - would that impact SQLDeveloper plugin work you're doing? Would there be any additional rework needed on your end?

Activity

jgebal
added
enhancement
on Jul 4, 2019
jgebal
self-assigned this
on Jul 4, 2019
jgebal
closed this as completedin #963 on Jul 13, 2019

Metadata

Assignees

jgebal

Labels

enhancement

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.