GeezerVison is showing github.com — Snowflake support does not include list features · Issue #2280 · feast-dev/feast
Snowflake support does not include list features #2280Activityadchia commented on Feb 3, 2022 CollaboratorAuthor More actions sfc-gh-madkins commented on Feb 19, 2022 @adchia @woop here is the docs related to semi-structured data in Snowflake: https://docs.snowflake.com/en/sql-reference/data-types-semistructured.html#array
Arrays for list feature types are possible in Snowflake, but there is no enforcement of data type within the array, its stored as type VARIANT, where VARIANT, can store values of any other type, including OBJECT and ARRAY, up to a maximum size of 16 MB. Snowflake does not currently support fixed-size arrays or arrays of elements of a specific non-VARIANT type.
Im curious to your initial thoughts on this. We can certainly test an array if it complies with a certain data type that is supported or throw an error. If it does then execellent, if not we throw an error to remove the feature stale commented on Jun 19, 2022 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. achals commented on Jul 13, 2022 sfc-gh-madkins commented on Dec 16, 2022 stale commented on Apr 26, 2023 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. sfc-gh-madkins commented on Apr 26, 2023 JohnLemmonMedely commented on Sep 5, 2023 Looks like we'll need this for our implementation so I'll be taking a look at implementing this in our sprint starting tomorrow. sfc-gh-madkins commented on Sep 5, 2023 @JohnLemmonMedely I am afraid snowflake does not support structured types on the server side. All semistructured types are currently returns as strings. We are working to support this in the next couple of quarters. JohnLemmonMedely commented on Sep 5, 2023 @sfc-gh-madkins I was planning on using the method you suggested earlier and attempt to cast them to what they need to be and throwing errors when we can't. It'll be better to guarantee data types via Snowflake but until that's possible I can enforce those guarantees myself via the data pipeline.
We can certainly test an array if it complies with a certain data type that is supported or throw an error. If it does then execellent, if not we throw an error to remove the feature
sfc-gh-madkins commented on Sep 5, 2023 will look forward to the PR :)
…
On Tue, Sep 5, 2023 at 4:51 PM John Lemmon ***@***.***> wrote:
@sfc-gh-madkins < https://github.com/sfc-gh-madkins> I was planning on
using the method you suggested earlier and attempt to cast them to what
they need to be and throwing errors when we can't. It'll be better to
guarantee data types via Snowflake but until that's possible I can enforce
those guarantees myself via the data pipeline.
We can certainly test an array if it complies with a certain data type
that is supported or throw an error. If it does then execellent, if not we
throw an error to remove the feature
—
Reply to this email directly, view it on GitHub
< #2280 (comment)>,
or unsubscribe
< https://github.com/notifications/unsubscribe-auth/ATSRCU3WGHHQXHAVBAE467LXY6NH7ANCNFSM5NPKRUQQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
JohnLemmonMedely commented on Dec 8, 2023 @sfc-gh-madkins I had to pause my work on the PR but we were able to use my fork in production. I'm trying to see if it makes sense at this point to get the PR ready to merge or if I should just wait on Snowflake adding structured types instead. Is that still on the Roadmap to add early next year? sfc-gh-madkins commented on Dec 8, 2023 let me ping to see where this is at
…JohnLemmonMedely commented on Dec 15, 2023 @sfc-gh-madkins just following up on if Snowflake still plans on supporting structured types soon. I just need to know if work will start on it over the next quarter or not. sfc-gh-madkins commented on Dec 16, 2023 negative. this is continuing to be pushed out due to lack of customer
demand. however, we do now have vector support capabilities for both fixed
and floating point 32 bit numbers, however, the client arent ready yet
…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|