GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||
GeezerVison is showing github.com — Read INT128-backed NUMERIC/DECIMAL values without overflowing or rounding by fdcastel · Pull Request #1305 · FirebirdSQL/NETProvider Conversation
Member
Fixes #1301. SummaryINT128-backed Changes
Behavior changes (breaking)
All other values decode exactly as before. I compared master's decoding with this one on 2,000,000 random values (1 to 38 digits, scale 0 to 38, seeded):
The same comparison measured decoding time (Stopwatch, boxed like the row decoder; one machine):
Tests
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactions…ding NUMERIC/DECIMAL with precision 19-38 were converted to System.Decimal while the row was decoded: - Values beyond System.Decimal (e.g. 38 digits) threw OverflowException in the middle of decoding the row. On the managed (wire) client this left the rest of the packet unread and desynchronized the connection: every later command on it failed with "ArgumentOutOfRangeException: operation=...". - Values with large scale but small magnitude overflowed too, e.g. 1.000...0 as NUMERIC(38,30). - Values with more than 28 fractional digits were rounded silently, e.g. 1E-30 as NUMERIC(38,30) was read as 0. - The native client returned BigInteger for NUMERIC(38,0)/DECIMAL(38,0) although GetFieldType() reports System.Decimal (managed returned decimal). Now the value is converted exactly: values that fit System.Decimal are returned exactly as before (smallest scale, e.g. 1234.5), on both clients. Values that do not fit are kept exactly (internal BigDecimal), so the row is always read and the connection stays usable; for those values: - GetValue/GetDecimal/GetFieldValue<decimal> throw OverflowException (only when that column is accessed), - GetString returns the exact value, - GetDouble/GetFloat return the nearest value, - GetFieldValue<BigInteger> returns integral values exactly (also for values that fit decimal; fractional values throw OverflowException). As GetFieldValue<BigInteger> and INT128 parameters share DbValue.GetInt128, an integral decimal parameter value can now be passed to an INT128 parameter (it threw InvalidCastException before).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ReviewersNo reviewsUh oh!
There was an error while loading. Please reload this page. AssigneesNo one assignedLabels
None yet
ProjectsMilestoneNo milestoneDevelopmentSuccessfully merging this pull request may close these issues. NUMERIC(19..38) values beyond System.Decimal break the row (and the connection), small values are rounded silentlyUh oh!
There was an error while loading. Please reload this page. Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later. | ||||||||||||||||||||||||
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |