GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: Feature/upgrade 3.11 and remove bytewax by lokeshrangineni · Pull Request #4046 · feast-dev/feast

Conversation

@lokeshrangineni

@lokeshrangineni lokeshrangineni commented Mar 27, 2024

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
lokeshrangineni added 3 commits March 27, 2024 00:56
@lokeshrangineni
Upgrading the python version to 3.11 and also removing the bytewax de…
408f400 
…pendencies.

Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Removing missing bytewax dependencies from the previous commit.
1214f71 
Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Merge branch 'feast-dev:master' into feature/upgrade-3.11-remove-bytewax
2dd2051
@lokeshrangineni lokeshrangineni changed the title Feature/upgrade 3.11 remove bytewax Feature/upgrade 3.11 and remove bytewax Mar 27, 2024
lokeshrangineni added 4 commits March 27, 2024 13:51
@lokeshrangineni
Merge branch 'feast-dev:master' into feature/upgrade-3.11-remove-bytewax
228e3b4
@lokeshrangineni
Removing duplicate dependency ibis which was added by mistake. this s…
f0971a8 
…hould fix the failure of python unit test cases.

Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Removing the bytewax materialization engine.
1cfa4d5 
Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Removing the bytewax documentation.
822cd67 
Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni lokeshrangineni changed the title Feature/upgrade 3.11 and remove bytewax feat: Feature/upgrade 3.11 and remove bytewax Apr 1, 2024
lokeshrangineni added 3 commits April 1, 2024 15:51
@lokeshrangineni
Merge branch 'feast-dev:master' into feature/upgrade-3.11-remove-bytewax
43ef03b
@lokeshrangineni
Adding missed duckdb dependencies.
10ddd01 
Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Merge branch 'feature/origin-master' into feature/upgrade-3.11-remove…
68da3f0 
…-bytewax

# Conflicts:
#	sdk/python/feast/infra/materialization/contrib/bytewax/bytewax_materialization_engine.py
@lokeshrangineni
lokeshrangineni marked this pull request as ready for review April 2, 2024 17:12
@lokeshrangineni
lokeshrangineni requested a review from whoahbot as a code owner April 2, 2024 17:12
@jeremyary jeremyary added the ok-to-test label Apr 2, 2024
@lokeshrangineni lokeshrangineni changed the title feat: Feature/upgrade 3.11 and remove bytewax HOLD - feat: Feature/upgrade 3.11 and remove bytewax Apr 3, 2024
@lokeshrangineni

lokeshrangineni commented Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

I am looking at the PR again to modify the the logic without actually removing the bytewax engine. If this approach works then probably i will update the PR otherwise we can go ahead with this approach.
tokoko reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
This was referenced Apr 9, 2024
feat: Kubernetes materialization engine written based on bytewax #4087
Merged
chore: Omit bytewax deps and materialization engine #4098
Merged
@jeremyary jeremyary changed the title HOLD - feat: Feature/upgrade 3.11 and remove bytewax feat: Feature/upgrade 3.11 and remove bytewax Apr 23, 2024
lokeshrangineni and others added 10 commits April 25, 2024 11:51
@lokeshrangineni
Merge remote-tracking branch 'origin/master' into feature/upgrade-3.1…
6a66539 
…1-remove-bytewax

# Conflicts:
#	.github/workflows/build_wheels.yml
#	.github/workflows/nightly-ci.yml
#	.github/workflows/unit_tests.yml
#	sdk/python/feast/infra/materialization/kubernetes/Dockerfile
#	sdk/python/feast/infra/materialization/kubernetes/k8s_materialization_engine.py
#	sdk/python/feast/infra/materialization/kubernetes/k8s_materialization_job.py
#	sdk/python/feast/infra/materialization/kubernetes/k8s_materialization_task.py
#	sdk/python/feast/infra/materialization/kubernetes/main.py
#	setup.py
@lokeshrangineni
updating the requirements.txt files as per the latest code.
b8fdfbd 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
updating the requirements.txt files as per the latest code.
4bec6b0 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Revert "updating the requirements.txt files as per the latest code."
fb6d0e5 
Signed-off-by: Lokesh Rangineni <[email protected]>

This reverts commit 4bec6b0.
@lokeshrangineni @tokoko
@jeremyary @feast-ci-bot @pushkarmoi @tmihalac @TomSteenbergen
fix: Changed the code the way mysql container is initialized. (feast-…
d089b0b 
…dev#4140)

* chore: Move feast install to docker build in java it tests (feast-dev#4126)

* chore: Move feast install to docker build in java it tests

Signed-off-by: tokoko <[email protected]>

* remove commented out lines in compose file

Signed-off-by: tokoko <[email protected]>

* make local compose mode default

Signed-off-by: tokoko <[email protected]>

* limit COPY contents

Signed-off-by: tokoko <[email protected]>

* remove requirements.txt from java tests docker image

Signed-off-by: tokoko <[email protected]>

* include pyproject.toml in dockerfile

Signed-off-by: tokoko <[email protected]>

* change links to depends_on

Signed-off-by: tokoko <[email protected]>

* try updating setup-python to v5

Signed-off-by: tokoko <[email protected]>

* pin macos image to macos-12

Signed-off-by: tokoko <[email protected]>

* force rerun

Signed-off-by: tokoko <[email protected]>

---------

Signed-off-by: tokoko <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* chore: Incorporate release 0.37.1 to master (feast-dev#4127)

* chore(release): release 0.37.1

## [0.37.1](feast-dev/feast@v0.37.0...v0.37.1) (2024-04-17)

### Bug Fixes

* Pgvector patch ([feast-dev#4108](feast-dev#4108)) ([1a1f0b1](feast-dev@1a1f0b1))

### Reverts

* Reverts "fix: Using version args to install the correct feast version" ([feast-dev#4112](feast-dev#4112)) ([d5ded69](feast-dev@d5ded69)), closes [feast-dev#3953](feast-dev#3953)

* chore: Move feast install to docker build in java it tests (feast-dev#4126)

* chore: Move feast install to docker build in java it tests

Signed-off-by: tokoko <[email protected]>

* remove commented out lines in compose file

Signed-off-by: tokoko <[email protected]>

* make local compose mode default

Signed-off-by: tokoko <[email protected]>

* limit COPY contents

Signed-off-by: tokoko <[email protected]>

* remove requirements.txt from java tests docker image

Signed-off-by: tokoko <[email protected]>

* include pyproject.toml in dockerfile

Signed-off-by: tokoko <[email protected]>

* change links to depends_on

Signed-off-by: tokoko <[email protected]>

* try updating setup-python to v5

Signed-off-by: tokoko <[email protected]>

* pin macos image to macos-12

Signed-off-by: tokoko <[email protected]>

* force rerun

Signed-off-by: tokoko <[email protected]>

---------

Signed-off-by: tokoko <[email protected]>

---------

Signed-off-by: tokoko <[email protected]>
Co-authored-by: feast-ci-bot <[email protected]>
Co-authored-by: Tornike Gurgenidze <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* chore: Install python dependencies with uv in workflows (feast-dev#4086)

* install dependencies in unit-tests with uv

Signed-off-by: tokoko <[email protected]>

* install dependencies in unit-tests with uv

Signed-off-by: tokoko <[email protected]>

* enable caching, change linter job

Signed-off-by: tokoko <[email protected]>

* change local integration tests to uv

Signed-off-by: tokoko <[email protected]>

* change all installs to uv

Signed-off-by: tokoko <[email protected]>

* try adding uv cache

Signed-off-by: tokoko <[email protected]>

* fix lambda cache step name

Signed-off-by: tokoko <[email protected]>

* reenable caches for uv

Signed-off-by: tokoko <[email protected]>

* remove dangling cache step

Signed-off-by: tokoko <[email protected]>

---------

Signed-off-by: tokoko <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* feat: Make arrow primary interchange for online ODFV execution (feast-dev#4143)

* rewrite online flow to use transform_arrow

Signed-off-by: tokoko <[email protected]>

* fix transformation server

Signed-off-by: tokoko <[email protected]>

---------

Signed-off-by: tokoko <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* feat: Feast/IKV online store documentation (feast-dev#4146)

* feat: Feast/IKV online store documentation

Signed-off-by: Pushkar Gupta <[email protected]>

* functionality matric

Signed-off-by: Pushkar Gupta <[email protected]>

* more changes

Signed-off-by: Pushkar Gupta <[email protected]>

* mount dir

Signed-off-by: Pushkar Gupta <[email protected]>

---------

Signed-off-by: Pushkar Gupta <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* fix: Default value is not set in Redis connection string using environment variable (feast-dev#4136)

Removed documentation of Redis connection string supporting default values when using environment variables as it isn't supported

Fixes feast-dev#3669

Signed-off-by: Theodor Mihalache <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* fix: Make sure schema is used when calling `get_table_query_string` method for Snowflake datasource (feast-dev#4131)

* Fix get_table_query_string method for Snowflake datasource

Signed-off-by: TomSteenbergen <[email protected]>

* Add quotes to table string

Signed-off-by: TomSteenbergen <[email protected]>

---------

Signed-off-by: TomSteenbergen <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* fix: Change checkout action back to v3 from v5 which isn't released yet (feast-dev#4147)

Signed-off-by: Jeremy Ary <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>

* changed the code the way mysql container is initialized. Trying to fix the issue - feast-dev#4128
Also going to check if this change will be resolved in the github actions as well.

Signed-off-by: Lokesh Rangineni <[email protected]>

* reformatted the file to resolve lint error.

Signed-off-by: Lokesh Rangineni <[email protected]>

* reformatted the file to resolve lint error.

Signed-off-by: Lokesh Rangineni <[email protected]>

---------

Signed-off-by: tokoko <[email protected]>
Signed-off-by: Lokesh Rangineni <[email protected]>
Signed-off-by: Pushkar Gupta <[email protected]>
Signed-off-by: Theodor Mihalache <[email protected]>
Signed-off-by: TomSteenbergen <[email protected]>
Signed-off-by: Jeremy Ary <[email protected]>
Co-authored-by: Tornike Gurgenidze <[email protected]>
Co-authored-by: Jeremy Ary <[email protected]>
Co-authored-by: feast-ci-bot <[email protected]>
Co-authored-by: Pushkar Gupta <[email protected]>
Co-authored-by: Theodor Mihalache <[email protected]>
Co-authored-by: Tom Steenbergen <[email protected]>
@lokeshrangineni
just checking if this resolves the issue with dask incompatibility.
d9d59af 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
updating 3.9 dependencies to make it work with 3.11 as well.
89ba1d8 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
updating 3.10 dependencies to make it work with 3.11 as well.
2bfedcf 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Revert "updating 3.10 dependencies to make it work with 3.11 as well."
29585f2 
Signed-off-by: Lokesh Rangineni <[email protected]>

This reverts commit 8d77135.
@lokeshrangineni
updating dask version so that it can work on python 3.11
bada729 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
lokeshrangineni force-pushed the feature/upgrade-3.11-remove-bytewax branch from 279b043 to bada729 Compare April 25, 2024 21:07
jeremyary
jeremyary reviewed Apr 26, 2024
View reviewed changes
Comment thread .github/workflows/unit_tests.yml Outdated
python-version: [ "3.9", "3.10" ]
os: [ ubuntu-latest, macos-12 ]
python-version: [ "3.9", "3.10", "3.11"]
os: [ ubuntu-latest, macOS-12 ]

@jeremyary jeremyary Apr 26, 2024

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUPER nitpicky of me, but casing here is fine in lowercase which is how GitHub references them too

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread sdk/python/requirements/py3.9-requirements.txt
sniffio==1.3.1
# via anyio
sqlalchemy[mypy]==2.0.29
sqlalchemy[mypy]==1.4.52

@jeremyary jeremyary Apr 26, 2024

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not saying it's wrong, more curious, can you please share what's causing the downgrade of sqlalchemy?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@lokeshrangineni lokeshrangineni Apr 26, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just re-ran the command to regenerate this file but not sure why this was modified. I have not modified the file manually. I will pay attention to it.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread setup.py
"uvicorn[standard]>=0.14.0,<1",
"gunicorn; platform_system != 'Windows'",
"dask[dataframe]>=2021.1.0",
"dask[dataframe]>=2024.4.2",

@jeremyary jeremyary Apr 26, 2024

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like I saw an answer in slack somewhere that I can't find at the moment, but this is a pretty big version jump, was Dask having issues with new Python version?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@lokeshrangineni lokeshrangineni Apr 26, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had a issue with Dask with new python version. Build is failing so had to upgrade but i have not tried the immediate version so let me give a shot with the version released in 2022.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
jeremyary reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@jeremyary jeremyary Apr 26, 2024

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're good! I was just looking for reason - since we need to upgrade we might as well catch up

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread setup.py
"googleapis-common-protos>=1.52.0,<2",
"google-cloud-bigquery[pandas]>=2,<3.13.0",
"google-cloud-bigquery-storage >= 2.0.0,<3",
"google-cloud-bigquery-storage >= 2.23.0,<3",

@jeremyary jeremyary Apr 26, 2024

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we encounter an issue w/BigQuery dependency too?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@lokeshrangineni lokeshrangineni Apr 26, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing it out. I had modified this version in between to resolve an issue with dask library. probably we don't need this change. i will remove this upgrade and see what happens.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
jeremyary reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@jeremyary

jeremyary commented Apr 26, 2024

Copy link
Copy Markdown
Collaborator

@lokeshrangineni looking really good, appreciate the diligence, I know you've been sticking with this one for a while!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
haoxu0 and others added 5 commits April 26, 2024 15:16
@haoxu0 @lokeshrangineni
fix: Update doc (feast-dev#4153)
95355c4 
update doc

Signed-off-by: cmuhao <[email protected]>
@lokeshrangineni
Upgrading the python version to 3.11 and also removing the bytewax de…
e7dddd3 
…pendencies.

Signed-off-by: Lokesh Rangineni <[email protected]>

Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
updating 3.10 dependencies to make it work with 3.11 as well.
5d4c322 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
Revert "updating 3.10 dependencies to make it work with 3.11 as well."
e7b9c13 
Signed-off-by: Lokesh Rangineni <[email protected]>

This reverts commit 8d77135.
@lokeshrangineni
updating dask version so that it can work on python 3.11
3d005b0 
Signed-off-by: Lokesh Rangineni <[email protected]>
@lokeshrangineni
lokeshrangineni changed the base branch from master to woop-patch-2 April 29, 2024 16:08
@lokeshrangineni
lokeshrangineni requested review from a team, DvirDukhan, kevjumba, niklasvm, sfc-gh-madkins, sudohainguyen and younggyu-oh as code owners April 29, 2024 16:08
@lokeshrangineni
lokeshrangineni changed the base branch from woop-patch-2 to master April 29, 2024 16:08
@lokeshrangineni
Merge remote-tracking branch 'refs/remotes/origin/master' into featur…
61d3319 
…e/upgrade-3.11-remove-bytewax
@jeremyary

jeremyary commented Apr 30, 2024

Copy link
Copy Markdown
Collaborator

work continuing here due to conflict complexity
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@jeremyary jeremyary closed this Apr 30, 2024
@lokeshrangineni
lokeshrangineni deleted the feature/upgrade-3.11-remove-bytewax branch June 20, 2024 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeremyary jeremyary jeremyary left review comments
@whoahbot whoahbot Awaiting requested review from whoahbot
@sfc-gh-madkins sfc-gh-madkins Awaiting requested review from sfc-gh-madkins
@DvirDukhan DvirDukhan Awaiting requested review from DvirDukhan
@sudohainguyen sudohainguyen Awaiting requested review from sudohainguyen
@niklasvm niklasvm Awaiting requested review from niklasvm
@kevjumba kevjumba Awaiting requested review from kevjumba
@younggyu-oh younggyu-oh Awaiting requested review from younggyu-oh

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@lokeshrangineni @jeremyary @haoxu0
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.