GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: support of pyspark-client by SemyonSinchenko · Pull Request #863 · graphframes/graphframes

Conversation

SemyonSinchenko commented Jun 16, 2026

Copy link

Copy Markdown

Collaborator

What changes were proposed in this pull request?

Small adjustments to support pyspark-client and testing flow.

Why are the changes needed?

Close #861

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

psavalle reacted with hooray emoji

All reactions

🎉 1 reaction


          feat: support of pyspark-client

e46ed2e

SemyonSinchenko self-assigned this

Jun 16, 2026

SemyonSinchenko added the pyspark-connect GraphFrames on PySpark Connect label

Jun 16, 2026


          fix: all tests passed with pyspark_client

2c00c77

SemyonSinchenko requested a review from james-willis

June 16, 2026 12:34

james-willis approved these changes

Jun 16, 2026

View reviewed changes

Comment thread

.github/workflows/pyspark-client-ci.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

.github/workflows/pyspark-client-ci.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/dev/run_connect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

python/pyproject.toml Resolved

Uh oh!

There was an error while loading. Please reload this page.

james-willis self-requested a review

June 16, 2026 17:31

james-willis requested changes

Jun 16, 2026

View reviewed changes

james-willis left a comment

Copy link

Copy Markdown

Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly some typos and other small things

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

SemyonSinchenko and others added 3 commits

June 16, 2026 19:32


          Apply suggestions from code review

dec1497

Co-authored-by: James Willis <[email protected]>


          fix: typo and regenerate lock

694f437


          fix: typo

26455bd

james-willis approved these changes

Jun 16, 2026

View reviewed changes

james-willis commented Jun 16, 2026

Copy link

Copy Markdown

Collaborator

aside from the CI failing I approve.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          fix: explicitly mark it as remote for spark

18ac883

codecov-commenter commented Jun 16, 2026

Copy link

Copy Markdown

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.26%. Comparing base (e705223) to head (18ac883).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #863      +/-   ##
==========================================
- Coverage   81.63%   79.26%   -2.37%     
==========================================
  Files          81       81              
  Lines        4682     4712      +30     
  Branches      574      554      -20     
==========================================
- Hits         3822     3735      -87     
- Misses        860      977     +117

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

SemyonSinchenko and others added 2 commits

June 16, 2026 21:21


          feat: ??

1ae0141


          fix: thin-client CI dev-group collision + gate approx on server version

b14ec87

The pyspark-client CI job ran 'poetry install --with=connect', which also
pulled the non-optional dev group and its full pyspark dependency. Full
pyspark and pyspark-client both own the pyspark/ namespace, so version.py
was clobbered and __version__ no longer reflected the thin client. Exclude
the dev group so only pyspark-client is installed.

Also gate the approx triangleCount algorithm on the connected server
version (spark.version) instead of the client library __version__, since
the approximate algorithm is a server-side capability and client/server
versions can differ under Spark Connect.

SemyonSinchenko merged commit f4b2654 into graphframes:main

Jun 16, 2026

10 checks passed

Uh oh!

There was an error while loading. Please reload this page.

james-willis added a commit to james-willis/graphframes that referenced this pull request

Jun 16, 2026


          ci: format dev connect scripts to satisfy extended linting

d3ec103

run_connect.py and stop_connect.py were added in graphframes#863 before the linting scope was extended to dev/. Apply black/isort and add a noqa for an unavoidable long config string so python-ci passes.

james-willis added a commit that referenced this pull request

Jun 16, 2026


          ci: add codespell spell check and extend python linting to dev scripts (

28d181c

#864)

* ci: add codespell spell check and extend python linting to dev scripts

- Add codespell (config in .codespellrc) covering yml, python and scala
  via a new Spell Check workflow and a pre-commit hook; fix the typos it
  surfaced across sources and docs. graphx/ is excluded to stay aligned
  with upstream Apache Spark.
- Extend black/flake8/isort to the dev/ and python/dev/ scripts in both
  the Python CI code-style step and pre-commit.

* ci: pass explicit config to black/isort so root dev/ does not break discovery

When ../dev is included, black/isort resolve their config from the common
parent of all paths (the repo root, which has no pyproject.toml) and fall
back to defaults. Pass --config/--settings-path explicitly.

* ci: format dev connect scripts to satisfy extended linting

run_connect.py and stop_connect.py were added in #863 before the linting scope was extended to dev/. Apply black/isort and add a noqa for an unavoidable long config string so python-ci passes.

Reviewers

james-willis

james-willis approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

SemyonSinchenko

Labels

pyspark-connect

GraphFrames on PySpark Connect

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat: support pyspark-client for Spark Connect

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.