GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Feature/json object by lwasylow · Pull Request #928 · utPLSQL/utPLSQL

Conversation

@lwasylow

@lwasylow lwasylow commented May 24, 2019

Copy link
Copy Markdown
Member

Adding a JSON object comparison.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
lwasylow added 8 commits May 18, 2019 00:39
@lwasylow
Initial checkin.
ef1976a
General skelet of code
@lwasylow
Adding json tree parsing.
9e7b6c4
Enabling 12.2
@lwasylow
Update tests.
ba1392a
Update sonar rules.
@lwasylow
small fixes
021e48d
@lwasylow
PHASE 2:
86aa49d
Moving diff into temp table and init data compare during compare implementation to avoid finding same differences twice
Adding have count and negated have count matcher.l
@lwasylow
Added more complex tests.
8e224c0
Added test when extract a piece from two different jsons.
Small fixes to code on same hierarchy parent arrays
@lwasylow
Update mddocs
1a17e7a
Fix to json traverse array when object is element of array, the element name is content of object which can be different order but same semantic content.
@lwasylow
Fixing sonar
c16de97
@lwasylow

lwasylow commented May 30, 2019

Copy link
Copy Markdown
Member Author

The functionality is working, now next step is to use conditional compilation within packages to install on 12.1 and less. This will require create a dummy type json_element_t in these versions as conditional compilation doesn't work in type specs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

lwasylow added 2 commits May 31, 2019 07:11
@lwasylow
Switch to use supertype
45e91e1
@lwasylow
PHASE3 : Adding a functionality for install in pre 12.2
ac8ab96
@lwasylow lwasylow changed the title [WIP] Feature/json object Feature/json object May 31, 2019
@lwasylow
PHASE3 : Adding a functionality for install in pre 12.2
587da95
@lwasylow lwasylow added enhancement idea labels May 31, 2019
LUKASZ104 and others added 11 commits May 31, 2019 14:04
LUKASZ104 LUKASZ104
PHASE3 : Adding a functionality for install in pre 12.2
7192a51
LUKASZ104 LUKASZ104
PHASE3 : Adding a functionality for install in pre 12.2
f475258
LUKASZ104 LUKASZ104
PHASE3 : Adding a functionality for install in pre 12.2
d48bdc2
@lwasylow
PHASE3 : Adding a functionality for install in pre 12.2
0aa94d3
@lwasylow
Merge branch 'develop' of https://github.com/utPLSQL/utPLSQL into fea…
b775475
…ture/json_object
@lwasylow
update tests
a5c7ca0
@lwasylow
Update set define off
f6c95e5
@lwasylow
Code cleanup
e87aa6b
@lwasylow
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/jso…
78394ab
…n_object
@lwasylow
fix slash
273b21a
@lwasylow
Creating a dummy specs and removing conditional compilation for code …
9f8cc71
…cleanup
@lwasylow lwasylow added this to the v3.1.7 milestone Jun 8, 2019
@lwasylow
lwasylow requested a review from jgebal June 8, 2019 22:10
jgebal
jgebal reviewed Jun 10, 2019
View reviewed changes
Comment thread docs/userguide/advanced_data_comparison.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread docs/userguide/advanced_data_comparison.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread docs/userguide/advanced_data_comparison.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread docs/userguide/expectations.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread docs/userguide/expectations.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread docs/userguide/expectations.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread source/core/ut_utils.pkb Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread source/expectations/data_values/ut_json_data_diff_tmp.sql Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread source/expectations/matchers/ut_equal.tpb Resolved

Uh oh!

There was an error while loading. Please reload this page.

@lwasylow
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/jso…
6f827ae
…n_object
lwasylow added 3 commits June 13, 2019 07:21
@lwasylow
PR fixes.
9031856
@lwasylow
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/jso…
7452983
…n_object
@lwasylow
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/jso…
6dd2026
…n_object
jgebal
jgebal reviewed Jun 15, 2019
View reviewed changes
Comment thread development/cleanup.sh Resolved

Uh oh!

There was an error while loading. Please reload this page.

@jgebal
Reverting development script changes.
ee73b69
@jgebal

jgebal commented Jun 16, 2019

Copy link
Copy Markdown
Member

For some reasons coverage os very low on new code.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jgebal

jgebal commented Jun 16, 2019

Copy link
Copy Markdown
Member

Even coveralls.io reports low coverage numbers - https://coveralls.io/builds/24018499
Do you thing it is something related to conditional compilation?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lwasylow

lwasylow commented Jun 16, 2019

Copy link
Copy Markdown
Member Author

Will check it later, but if coverage was executed on 12.1 or less then that would explain as a none of new json expectation code would be executed or the dummy type json element and its subtype. Does coverage runs on all builds?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lwasylow

lwasylow commented Jun 16, 2019

Copy link
Copy Markdown
Member Author

So code will be there in all versions but pre 12.2 is dormant

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jgebal

jgebal commented Jun 16, 2019

Copy link
Copy Markdown
Member

I've just kicked off 18c build on the PR - will see if it ges better.
Could be sonar issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lwasylow

lwasylow commented Jun 16, 2019

Copy link
Copy Markdown
Member Author

Looking that fixed the thing

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jgebal added 2 commits June 17, 2019 16:49
@jgebal
Refactored JSON code.
d6a6801
Changed formatting of JSON-diff output.
Changed date formats.
@jgebal
Fixed install on Oracle < 12.2
6586342
jgebal
jgebal approved these changes Jun 17, 2019
View reviewed changes
@jgebal
Merge branch 'develop' into feature/json_object
6e9dc12
@lwasylow
lwasylow merged commit ba2e765 into develop Jun 17, 2019
@jgebal
jgebal deleted the feature/json_object branch June 17, 2019 23:16
lwasylow added a commit that referenced this pull request Jun 18, 2019
@lwasylow
Release/v3.1.7 (#942)
1a5c525
* Test to verify there's a problem with cursor-comparison on 12.1

when there are several numbers with different precision and we make use of package-level nested table types

* Typo

* Initial checkin

* Adding handling of xml error

* Adding new API user.

* Updated `install.md`

* Updated project version after build [skip ci]

* Separating tests.

* Update properties

* Change user that executes tests as helper that is super user.

* Updated version to `3.1.7`

* Updated project version after build [skip ci]

* Adding next set of tests migrated.
expectations/binary/test_equal
expectations/binary/test_expect_to_be_less_than
expectations/test_expectation_anydata
expectations/test_expectation_processor

* Adding next set of tests migrated.
expectations/binary/test_be_less_or_equal

changing name of function to more meaningful.

* Fixing test package

* Added set of tests:
expectations/binary/test_be_greater_or_equal
expectations/binary/test_be_greater_than
expectations/unary/test_expect_not_to_be_null

Passing username implicit into ut_test_runner.

* Adding set of tests :
expectations/unary/test_expect_to_be_empty
expectations/unary/test_expect_to_be_null
expectations/unary/test_expect_to_be_true_false
expectations/unary/test_expect_to_have_count

Adding dummy type for testing.

* Adding schema prefix to failing tests from 12.1

* Fixing development scripts

* Fixed bug for deployments with uninstall included.
Added new user to `template.env.sh`

* Updated project version after build [skip ci]

* FIX : removal of non existing types from uninstall script

* INSTALL:Is it too many monkeys ?:)

* Added support for random order of test execution.
Resolves #422

* Adding set of tests:
api/test_ut_run

* Adding set of tests:
api/test_ut_run

* Fixing one bad test.
Workaround for current issue with `unordered()` comparison of blank lines.

* Fixing issue with null vs null being returned as false.
Fixing change in Oracle 12.2+ where XML is stored as binary XML and not preserving insignificant white space. This require store data as clob.

* Revert test

* Fixed bad test.
Added `dbms_output.enable` in `core` test package

* Adding set of tests:
api/test_ut_run

* Adding set of tests:
core/annotations/test_annotation_parser
core/annotations/test_annot_throws_exception
core/annotations/test_annotation_manager

* Adding set of tests:
core/test_ut_test
core/test_ut_utils

* Adding set of tests:
core/test_ut_executable
core/test_ut_suite

* Adding set of tests:
core/test_file_mapper
core/test_output_buffer
core/test_suite_manager
core/test_ut_executable

* Adding tests
core/test_suite_builder

* Adding tests:
reporters
reporters/test_junit_reporter
reporters/test_sonar_test_reporter
reporters/test_teamcity_reporter
reporters/test_tfs_junit_reporter

* Fixed storage for clob in output bugger table - make it inline.
Changed how delete from output buffer table is handled - use ROWID.
Increased fetch size from 100 to 3000 rows for output buffer.

* Improved performance for reading data from output buffer.
Decreased read limit back to 1000 as it should not have significant performance impact anymore.

* Increased read limit to 5000.
Fixed bug with increasing `max_message_id` even when nothing was read.

* Adding set of tests:
reporters/test_coverage
reporters/test_debug_reporter
reporters/test_documentation_reporter
reporters/test_extended_coverage
reporters/test_realtime_reporter

Fixing some helper methods.

* Split output buffers into two.
Now we have separate buffer for CLOB data and VARCHAR2 text data.

* Adding extra tests:
reporters/test_coverage/test_cov_cobertura_reporter
reporters/test_coverage/test_coverage_sonar_reporter
reporters/test_coverage/test_coveralls_reporter
reporters/test_coverage/test_html_extended_reporter

* Updated documentation for random order of test execution

* Adding tests:
reporters/test_coverage
reporters/test_coverage/test_html_proftab_reporter
reporters/test_proftab_coverage

* Updated project version after build [skip ci]

* Fixing broken test.
Adding test:
reporters/test_coverage/test_html_extended_reporter
reporters/test_extended_coverage

* Adding test:
unary/test_expect_to_be_not_null

* Adding global setup

* document how to avoid Oracle Bug 14402514 (see #891)

* Fixes #891 - use to_match instead of to_be_like to avoid Oracle bug 14402514 on some 11.2.0.4 instances

* Updated project version after build [skip ci]

* Update cleanup script after branching from random order branch

* Fixed invalid suitepath in `core` package.
Updated cli version to 3.1.6
Added top package `utplsql` with dbms_output.enable for both test users.
Renamed suitepath to match username (ut3_tester)

* Updated project version after build [skip ci]

* Cursor error handling

* Update tests for handling usr friendly error

* Applying fixes as per PR review.
Removal of unused procedure from expactation_processor
Switch to use constant in error message creation
Update tests to match full message.

* Cleanup whitespaces

* fixing ORA-00907 when comparing ref cursors with BINARY_DOUBLE/BINRAY_FLOAT columns

fixing ORA-00907: missing right parenthesis when comparing ref cursors with BINARY_DOUBLE/BINARY_FLOAT columns, because these column types do not support type length

* fix/ORA-00907: adding simple testcase

* fix/ORA-00907: shortening test case name

* fix/ORA-00907: wrong schema name in test case check

* Updated project version after build [skip ci]

* Call stack differs based on version

* Fix typo

* Update error lines

* Adding ROWID to list of types that do not provide a length.

* cleaning up tests

* Update tests

* testing wildcard, there is some unexpected white space that is not appearing on local tests or other similar for expected.

* Fix : No length datatypes mapping. This is to address issue when Oracle dont expect a length value against data type but cursor description return a max_len not being null.

* Update test

* Mysterious whitespace appears from time to time.
TODO

* Updated project version after build [skip ci]

* Fixing issue with a special column names non XML compliant (#903)

* Fixing issue with a special column names that do not adhere to XML rules.

* removing unused global constant.

* Modification after PR comments.
Inline call to function.
Remove constants to be private.
Expose only one function rest to be private

* Updated project version after build [skip ci]

* Adding trigger solution for annotation parsing.

* Fixed some issues with trigger
Some tests need to be fixed to disable trigger before testing
Some tests need to be added to prove trigger is working

* Fixed uninstall script and constant.
Disabled `-scc` flag in tests

* Removed commented code.

* Added tags for tests and tags for suite.

* Switch to use VARCHAR2 instead of UT_VARCHAR2_ROWS()

Signed-off-by: lwasylow <[email protected]>

* Switch to use VARCHAR2 instead of UT_VARCHAR2_ROWS()

Signed-off-by: lwasylow <[email protected]>

* Fix/oracle bug with xmlspaces (#895)

* Address issue with whitespace in XML.
Cursor describe is ignoring whitespace so max len of column with value ' t ' is showing as 1.
XMLTABLE when extracting values from tag with whitespaces its ignoring them. It works properly if there is any character there e.g ' t '.

* Update tests

* Updated project version after build [skip ci]

* Adding Tests to tags #suitebuilder

* Change annotation to --%tags

* Fixing empty tags to use xmlgen

* Adding tests for ut.run
Change query to use regexp as instr was giving false positives.

* Added tests for ut.run procedure

* Added extra tests for ut run function

* Fixing error message function

* Update docs

* Update docs and tests

* Fixes to PR comments

* Fixes to PR comments

* Updates to PR

* Separated the install for trigger.

* Phase 2 : switch to varchar_rows()

* Number precision from dbms_sql not correct (#908)

* Fixing issue with number from dual and precision being incorrectly read.

* Updated tests

* Updated project version after build [skip ci]

* add tests on 19c DB version

* Phase 2 : switch to varchar_rows()

* Remove a 4k tests

* Added constant

* Updated project version after build [skip ci]

* Added-back rigger installation to CI/CD process.

* Moved test to new package structure and improved test details.

* Fixed test to run only on 12 and above.

* Updated project version after build [skip ci]

* Removed garbage after PR merge.

* Updates to PR.
Switch to common add tags to suite item

* Move query to template to simplify view

* Updated project version after build [skip ci]

* Fix to issue when column was removed from the middle of cursor using a exclude or include.
this left a gap in orders and cause an columns mismatch on position for remainder of columns.

* Fix typo

* Make sure that order of the columns is kept when rewriting an order and nothing was removed.

* Partition should not be across column name but a full table set.

* Removal of row_number as its excessive we dont partition by any

* fix message_id increment

* the fix

* shorten test name

* fix type

* Updated project version after build [skip ci]

* Updated project version after build [skip ci]

* Fixed issue with missing newlines in cobertura coverage for non-executed packages.
Resolves #917

* Updated project version after build [skip ci]

* Initial checkin

* Failure details in sonar test reporter are now CDATA

* Update ut_junit_reporter.tpb

* Added handling of CDATA in failure messages for XML reporters.

* Fixing PR comments.

* Update docs with sample

* TAG: Phase2
Adding a new attribute filterpath that is used for filtering cursor in anydata / refcursor.
This value will be different from cursor in anydata as we skip root element.

* TAG: Phase2
Adding a new attribute filterpath that is used for filtering cursor in anydata / refcursor.
This value will be different from cursor in anydata as we skip root element.

* PHASE 2

* Fixing unit test for Oracle 11.2 & 12.1

* Fixing code duplication

* Updated project version after build [skip ci]

* Fixes #921 - Assert input params in dynamic SQL

Used dbms_assert.qualified_sql_name whenever possible.
This is much less invasive than dbms_assert.enquote_name.
Code converting case is left untouched.

* Updated project version after build [skip ci]

* Fixing sonar violation on duplicated code.

* Updated project version after build [skip ci]

* update tests to be less complicated

* Fixing encoding issue

* Updated project version after build [skip ci]

* Renamed `gc_tag` to `gc_tags`
Added debug logging of suite cache query.

* Updated project version after build [skip ci]

* Fixed install of release until feature is merged back.

* Enabled-back dbms_preprocessor

* Fixed bug in parsing annotations from DDL trigger.

* Fixed issues with parsing code through trigger.
Added disabling trigger on annotation parser tests.

* Fixed re-install of trigger.

* Updated readme for installation with trigger.
Added additional tests for annotation management with / without trigger.
Fixed code for situations when trigger gets added on pre-existing tests.

* Addressed review comments.
Refactored install scripts to avoid duplication.
Added check for `oracle_maintained` schema when firing DDL trigger.

* Fixed issue with style check.
Reorganized code to avoid problems with conditional compilation on 12.2 for triggers.

* Updated project version after build [skip ci]

* Fixing sonar exclusions for analysis and coverage.

* Updated project version after build [skip ci]

* Fixed output_buffer purging error.
Resolves #934

* Updated project version after build [skip ci]

* add test cases for warning texts in tests, suites

* implement new test cases and adapt existing ones

Structure of test cases is affected since new PL/SQL package is created which is also used in the whole run.

* emit warning texts on test and suite level

* complete cleanup after run

* Updated project version after build [skip ci]

* Removed dependency on `dbms_utility.name_resolve`.

Fixed issue with utPLSQL failing to run when schema contains package named the same as schema owning it.
It is now possible to have utPLSQL test package like this:
```
create or replace package some_schema.some_schema as
  --%suite

  --%test
  procedure sample_test;
end;
/

create or replace package body some_schema.some_schema as
   procedure sample_test is begin ut.expect(1).to_equal(1); end;
end;
/
```

The tests from the package can be invoked by calling:
`exec ut.run('some_schema.some_schema');`

If only schema is provided, all test packages in schema are executed.
`exec ut.run('some_schema');`

Resolves #569
Resolves #885

* Fixed issue with parsing by trigger when source contains leading comments.

* Fixed typo.

* Improved time needed for clob-to-table conversion.
Timing before change (with dbms_lob.read):
`clob_to_table - Took:+000000000 00:00:00.012957000`
After change:
`clob_to_table - Took:+000000000 00:00:00.001236000`
Speed improved by order of magnitude.

* Resolved issues with regex for DDL trigger processing.

* Adressed review comment.

* Fixed typo.

* Updated project version after build [skip ci]

* Updated project version after build [skip ci]

* Feature/json object (#928)

* Initial checkin.
General skelet of code

* Adding json tree parsing.
Enabling 12.2

* Update tests.
Update sonar rules.

* small fixes

* PHASE 2:
Moving diff into temp table and init data compare during compare implementation to avoid finding same differences twice
Adding have count and negated have count matcher.l

* Added more complex tests.
Added test when extract a piece from two different jsons.
Small fixes to code on same hierarchy parent arrays

* Update mddocs
Fix to json traverse array when object is element of array, the element name is content of object which can be different order but same semantic content.

* Fixing sonar

* Switch to use supertype

* PHASE3 : Adding a functionality for install in pre 12.2

* PHASE3 : Adding a functionality for install in pre 12.2

* PHASE3 : Adding a functionality for install in pre 12.2

* PHASE3 : Adding a functionality for install in pre 12.2

* PHASE3 : Adding a functionality for install in pre 12.2

* PHASE3 : Adding a functionality for install in pre 12.2

* update tests

* Update set define off

* Code cleanup

* fix slash

* Creating a dummy specs and removing conditional compilation for code cleanup

* PR fixes.

* Reverting development script changes.

* Refactored JSON code.
Changed formatting of JSON-diff output.
Changed date formats.

* Fixed install on Oracle < 12.2

* Updated project version after build [skip ci]

* Update readme.md

* Updated copyright year

* Updated project version after build [skip ci]
@lwasylow lwasylow mentioned this pull request Jun 18, 2019
Release/v3.1.7 (#942) #943
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jgebal jgebal jgebal approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

enhancement idea

Projects

None yet

Milestone

v3.1.7

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@lwasylow @jgebal
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.