Description

stonebig

opened on Aug 27, 2023

Issue body actions

with duckdb-0.8.0

pydef py_sin(s:int) ->float:
    "sinus function : example loading module, handling input/output as strings"
    import math as py_math
    return ( py_math.sin(s*1));
WITH RECURSIVE
  cnt(x) AS (
     SELECT 1
     UNION ALL
     SELECT x+1 FROM cnt
      where x < 300000
  )

select sum(py_sin(x))  from cnt 

takes 130 seconds, while 0.4 seconds with SQLite

WITH RECURSIVE
  cnt(x) AS (
     SELECT 1
     UNION ALL
     SELECT x+1 FROM cnt
      where x < 300000 
  )

select sum(sin(x))  from cnt 

takes also 110 seconds, while 0.1 second with SQLite integrated math

Activity

1. stonebig commented on Aug 27, 2023

   

   stonebig

   on Aug 27, 2023

   ContributorAuthor

   More actions

   this takes 1 second (or less) on duckdB:
   select sum(sin(x)) from range(1 ,300001)cnt(x)

   so 10 seconds are due to not liking "recursive" CTE in a 300 000 loop (30 000 loop /second limit)

   Then there is a big slowdown in using UDF, versus SQLite.
   ( 3 000 loop/second limit, versus 300 000 loop/second for SQLite)

   not using "recursive" CTE and not using UDF (in this test) makes it quick like SQLite
2. stonebig commented on Sep 3, 2023

   

   stonebig

   on Sep 3, 2023

   ContributorAuthor

   More actions

   when duckdb gets less bad at UDF, it will be an interesting change
   becoming fluent with massive datas makes tcl/tk a problem.
3. stonebig commented on Sep 30, 2023

   

   stonebig

   on Sep 30, 2023

   ContributorAuthor

   More actions

   This takes 10 seconds to duckdb

   pydef py_sin(s:int) ->float:
       "sinus function : example loading module, handling input/output as strings"
       import math as py_math
       return ( py_math.sin(s*1));
   
   WITH  
     cnt(x) AS (
        SELECT * from generate_series (1, 30000 )
     )
   
   select sum(py_sin(x))  from cnt 
   `````
   
   
   This takes 0.1 second to duckdb
   `````
   WITH cnt(x)  AS (select * from generate_series(1, 3000000)  )
   
   select sum(sin(x))  from cnt ;
   ````
   
   
4. stonebig

   added this to the 2025-05 buildable in the clouds ? milestone on May 4, 2025
5. stonebig

   modified the milestones: 2025-05 better cloud build + 3.14 free-threading, 2026-01 pandas-3 on Dec 22, 2025

Issue actions