GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [sqllib] Fix overflow bugs by mihaibudiu · Pull Request #7691 · feldera/feldera

[sqllib] Fix overflow bugs #7691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mihaibudiu merged 5 commits into main from sqllib-overflow-bugs
Oct 29, 2026

Uh oh!

There was an error while loading. Please reload this page.
Merged

[sqllib] Fix overflow bugs #7691

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9e7b2f3
[sqllib] Fix DynamicDecimal arithmetic, years of an interval, and DEC…
mihaibudiu Oct 28, 2026
5e9ced5
[SQL] Report overflow for ABS and unary minus of the minimum value
mihaibudiu Oct 28, 2026
2f9da40
[sqllib] Fix ROUND and TRUNCATE of floating point values with many di…
mihaibudiu Oct 28, 2026
4f7d6ab
[SQL] Do not warn when a Calcite optimization step fails
mihaibudiu Oct 28, 2026
bf0201e
[fxp] Add DynamicDecimal values whose exponents differ by more than 38
mihaibudiu Oct 29, 2026
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view

There are no files selected for viewing

207 changes: 174 additions & 33 deletions 207 crates/fxp/src/dynamic.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use crate::{
parse_decimal, pow10, u256::I256,
};

use num_traits::CheckedDiv;
use num_traits::{CheckedDiv, CheckedMul};
use rand::Rng;
use rand::distributions::uniform::{UniformInt, UniformSampler};

Expand Down Expand Up @@ -163,6 +163,55 @@ impl DynamicDecimal {
Self::new(self.sig.abs(), self.exponent)
}

/// Returns `sig * 10**-exponent`. Drops the fractional digits past the
/// largest exponent.
fn from_wide_exponent(sig: i128, exponent: usize) -> Self {
let excess = exponent.saturating_sub(u8::MAX.into());
let sig = if excess > 38 { 0 } else { sig / pow10(excess) };
Self::new(sig, (exponent - excess) as u8)
}

/// Returns `value * 10**-exponent`, or `None` if the integer part does not
/// fit in the significand. Drops the trailing digits of `value` that do
/// not fit in the significand, and the fractional digits past the largest
/// exponent.
fn checked_from_i256(value: I256, exponent: usize) -> Option<Self> {
let (sig, dropped) = value.reduce_to_i128();
let exponent = exponent.checked_sub(dropped)?;
Some(Self::from_wide_exponent(sig, exponent))
}

/// Returns `coarse + fine`, truncated toward zero, or `None` if the integer
/// part does not fit in the significand. `fine` has more than 38
/// fractional digits more than `coarse`.
fn add_distant_exponents(coarse: Self, fine: Self) -> Option<Self> {
if coarse.sig == 0 {
return Some(fine);
}
// Keep the digits of `fine` down to 37 places past the last digit of
// `coarse`, and replace the digits past them by one digit with their
// sign. The exact sum and this sum lie between the same two multiples
// of 10 in units of that digit, so both truncate to the same value once
// that digit is dropped.
let dropped = (fine.exponent - coarse.exponent) as usize - 37;
let (kept, rest) = if dropped > 38 {
(0, fine.sig)
} else {
(fine.sig / pow10(dropped), fine.sig % pow10(dropped))
};
let fine_sig = kept * 10 + rest.signum();
let sum = I256::from_product(coarse.sig, pow10(38)) + I256::from(fine_sig);
let (sig, dropped) = sum.reduce_to_i128();
// The last digit of `sum` stands for the dropped digits of `fine`
let (sig, dropped) = if dropped == 0 {
(sig / 10, 1)
} else {
(sig, dropped)
};
let exponent = (coarse.exponent as usize + 38).checked_sub(dropped)?;
Some(Self::from_wide_exponent(sig, exponent))
}

/// Deserializes [DynamicDecimal] into `Fixed`. If successful, returns the
/// original value from before serialization. If `S < value.exponent`, then
/// some trailing decimals are lost, by rounding toward even. Returns an
Expand Down Expand Up @@ -196,35 +245,27 @@ impl Add<DynamicDecimal> for DynamicDecimal {
type Output = Self;

fn add(self, other: Self) -> Self {
match self.exponent.cmp(&other.exponent) {
Ordering::Less => {
let factor = pow10((other.exponent - self.exponent) as usize);
if self.sig <= i128::MAX / factor / 10 {
DynamicDecimal::new(
other.sig.checked_add(self.sig * factor).unwrap(),
other.exponent,
)
} else {
let result = (I256::from_product(self.sig, factor) + I256::from(other.sig))
.reduce_to_i128();
DynamicDecimal::new(result.0, result.1 as u8)
}
}
Ordering::Equal => {
DynamicDecimal::new(self.sig.checked_add(other.sig).unwrap(), self.exponent)
}
Ordering::Greater => {
let factor = pow10((self.exponent - other.exponent) as usize);
if other.sig <= i128::MAX / factor / 10 {
DynamicDecimal::new(
self.sig.checked_add(other.sig * factor).unwrap(),
self.exponent,
)
} else {
let result = (I256::from_product(other.sig, factor) + I256::from(self.sig))
.reduce_to_i128();
DynamicDecimal::new(result.0, result.1 as u8)
}
// `coarse` has fewer fractional digits, so it is scaled to the exponent of `fine`
let (coarse, fine) = if self.exponent <= other.exponent {
(self, other)
} else {
(other, self)
};
if fine.exponent - coarse.exponent > 38 {
return DynamicDecimal::add_distant_exponents(coarse, fine)
.unwrap_or_else(|| panic!("Overflow adding {} + {}", self, other));
}
let factor = pow10((fine.exponent - coarse.exponent) as usize);
let sum = coarse
.sig
.checked_mul(factor)
.and_then(|scaled| scaled.checked_add(fine.sig));
match sum {
Some(sum) => DynamicDecimal::new(sum, fine.exponent),
None => {
let sum = I256::from_product(coarse.sig, factor) + I256::from(fine.sig);
DynamicDecimal::checked_from_i256(sum, fine.exponent.into())
.unwrap_or_else(|| panic!("Overflow adding {} + {}", self, other))
}
}
}
Expand All @@ -242,8 +283,17 @@ impl Mul<DynamicDecimal> for DynamicDecimal {
type Output = Self;

fn mul(self, other: Self) -> Self {
let result = I256::from_product(self.sig, other.sig).reduce_to_i128();
DynamicDecimal::new(result.0, result.1 as u8 + self.exponent + other.exponent)
self.checked_mul(&other)
.unwrap_or_else(|| panic!("Overflow multiplying {} * {}", self, other))
}
}

impl CheckedMul for DynamicDecimal {
fn checked_mul(&self, other: &Self) -> Option<Self> {
let product = I256::from_product(self.sig, other.sig);
// The exponent of a product is the sum of the exponents.
let exponent = self.exponent as usize + other.exponent as usize;
DynamicDecimal::checked_from_i256(product, exponent)
}
}

Expand Down Expand Up @@ -445,7 +495,11 @@ try_to_unsigned_int!(usize);

impl From<DynamicDecimal> for f64 {
fn from(value: DynamicDecimal) -> Self {
value.significand() as f64 / pow10(value.exponent() as usize) as f64
match checked_pow10(value.exponent().into()) {
Some(divisor) => value.significand() as f64 / divisor as f64,
// The divisor does not fit in an `i128`, so parse the decimal digits.
None => value.to_string().parse().unwrap(),
}
}
}

Expand Down Expand Up @@ -546,6 +600,7 @@ impl UniformDecimal {
#[cfg(test)]
mod test {
use crate::{DynamicDecimal, OutOfRange, ParseDecimalError};
use num_traits::CheckedMul;

#[test]
fn eq() {
Expand Down Expand Up @@ -763,6 +818,92 @@ mod test {
}
}

#[test]
fn to_f64() {
assert_eq!(f64::from(DynamicDecimal::new(15, 1)), 1.5);
// 10**100 and 10**44 do not fit in an `i128`.
assert_eq!(f64::from(DynamicDecimal::new(1, 100)), 1e-100);
assert_eq!(f64::from(DynamicDecimal::new(-12345, 44)), -1.2345e-40);
}

#[test]
fn mul_wide() {
// The products of these significands do not fit in an `i128`, so `mul`
// drops their trailing digits.
let x = DynamicDecimal::new(10i128.pow(20) + 1, 10);
assert_eq!(x * x, DynamicDecimal::new(10i128.pow(20) + 2, 0));
let fraction: DynamicDecimal = "0.12345678901234567890123456789012345".parse().unwrap();
let billion = DynamicDecimal::new(1_000_000_000, 0);
let expected: DynamicDecimal = "123456789.01234567890123456789012345".parse().unwrap();
assert_eq!(fraction * billion, expected);
assert_eq!(billion * fraction, expected);
}

#[test]
#[should_panic(expected = "Overflow multiplying")]
fn mul_overflow() {
let x = DynamicDecimal::new(10i128.pow(20), 0);
let _ = x * x;
}

#[test]
fn checked_mul() {
let x = DynamicDecimal::new(10i128.pow(20), 0);
assert_eq!(x.checked_mul(&x), None);
let y = DynamicDecimal::new(3, 1);
assert_eq!(y.checked_mul(&y), Some(DynamicDecimal::new(9, 2)));
}

#[test]
fn mul_many_fractional_digits() {
// A product with more than 255 fractional digits drops the digits past the 255th.
let tiny = DynamicDecimal::new(1, 200);
assert_eq!(tiny * tiny, DynamicDecimal::ZERO);
let x = DynamicDecimal::new(123456789, 200);
let y = DynamicDecimal::new(1, 60);
assert_eq!(x * y, DynamicDecimal::new(1234, 255));
}

#[test]
fn add_wide() {
// `add` computes these sums with 256 bits.
let big = DynamicDecimal::new(2 * 10i128.pow(35), 0);
let quarter = DynamicDecimal::new(25, 2);
let expected: DynamicDecimal = format!("{}.25", 2 * 10i128.pow(35)).parse().unwrap();
assert_eq!(big + quarter, expected);
assert_eq!(quarter + big, expected);
// The digits of 10**37 + 0.001 after the decimal point do not fit.
let large = DynamicDecimal::new(10i128.pow(37), 0);
let thousandth = DynamicDecimal::new(1, 3);
assert_eq!(large + thousandth, large);
assert_eq!(thousandth + large, large);
// -10**37 * 100 does not fit in an `i128`; -10**37 + 0.25 keeps one fractional digit.
let negative: DynamicDecimal = format!("-{}.7", 10i128.pow(37) - 1).parse().unwrap();
assert_eq!(-large + quarter, negative);
assert_eq!(quarter + -large, negative);
// The sum of two significands with the same exponent does not fit in an `i128`.
let half_max = DynamicDecimal::new(i128::MAX, 1);
assert_eq!(half_max + half_max, DynamicDecimal::new(i128::MAX / 5, 0));
}

#[test]
fn add_distant_exponents() {
// The exponents differ by more than 38.
let one = DynamicDecimal::ONE;
let tiny = DynamicDecimal::new(1, 200);
assert_eq!(one + tiny, one);
assert_eq!(tiny + one, one);
assert_eq!(DynamicDecimal::ZERO + tiny, tiny);
// 1 - 10**-200 truncates to 37 nines after the decimal point.
let nines = DynamicDecimal::new(10i128.pow(37) - 1, 37);
assert_eq!(one - tiny, nines);
assert_eq!(-one + tiny, -nines);
// The leading digits of 1.23 * 10**-37 are kept: 1 + 10**-37 and 1 - 2 * 10**-37.
let small = DynamicDecimal::new(123, 39);
assert_eq!(one + small, DynamicDecimal::new(10i128.pow(37) + 1, 37));
assert_eq!(one - small, DynamicDecimal::new(10i128.pow(37) - 2, 37));
}

#[test]
fn sub() {
fn f(n: f64) -> DynamicDecimal {
Expand Down
19 changes: 19 additions & 0 deletions 19 crates/fxp/src/fixed.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2032,6 +2032,25 @@ mod test {
test(-99_999_999.6, None);
}

#[test]
fn round_far_left() {
// Rounding more than 38 digits away from the last digit of the scale
// gives 0.
fn test<const P: usize, const S: usize>(value: Fixed<P, S>, digits: i32) {
assert_eq!(value.checked_round(digits), Some(Fixed::ZERO));
assert_eq!(value.checked_round_ties_even(digits), Some(Fixed::ZERO));
assert_eq!((-value).checked_round(digits), Some(Fixed::ZERO));
}
let five = Fixed::<10, 0>::try_from(5).unwrap();
for digits in [-38, -39, -40, i32::MIN] {
test(five, digits);
}
let half = Fixed::<38, 38>::new(5, 1).unwrap();
for digits in [-1, -2, i32::MIN] {
test(half, digits);
}
}

#[test]
fn trunc_digits() {
let x = Fixed::<10, 4>(245368746);
Expand Down
4 changes: 3 additions & 1 deletion 4 crates/fxp/src/lib.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,9 @@ fn round_inner(value: i128, scale: i32, n: i32, halfway: Halfway) -> Option<i128
let position = scale.saturating_sub(n);
if position <= 0 {
Some(value)
} else if value.abs() < 5 * pow10(position as usize - 1) {
} else if position > 38 || value.abs() < 5 * pow10(position as usize - 1) {
// `|value| <= i128::MAX < 5 * 10**38`, so `value` rounds to 0 at any
// position over 38.
Some(0)
} else {
let divisor = pow10(position as usize);
Expand Down
18 changes: 6 additions & 12 deletions 18 crates/sqllib/src/casts.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,11 @@ use crate::{
use chrono::{NaiveDate, NaiveDateTime, NaiveTime};
use dbsp::algebra::{F32, F64, HasOne, HasZero};
use num::{One, Zero};
use num_traits::{CheckedDiv, cast::NumCast};
use num_traits::{CheckedDiv, CheckedMul, cast::NumCast};
use regex::{Captures, Regex};
use smallstr::SmallString;
use std::{
error::Error,
fmt::Display,
fmt::Write,
iter::once,
marker::PhantomData,
ops::{Div, Mul},
error::Error, fmt::Display, fmt::Write, iter::once, marker::PhantomData, ops::Div,
sync::LazyLock,
};

Expand Down Expand Up @@ -1935,13 +1930,12 @@ fn convert_to_ShortInterval_SqlDecimal<const P: usize, const S: usize>(
) -> SqlResult<ShortInterval> {
let dd = DynamicDecimal::from(value);
let mul = DynamicDecimal::from(multiplier);
let val = dd.mul(mul);
i64::try_from(val)
dd.checked_mul(&mul)
.and_then(|val| i64::try_from(val).ok())
.map(ShortInterval::from_microseconds)
.map_err(|e| {
.ok_or_else(|| {
SqlRuntimeError::from_string(format!(
"Overflow during conversion of {value} to INTERVAL {}: {}",
kind, e
"Overflow during conversion of {value} to INTERVAL {kind}"
))
})
}
Expand Down
Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.