GeezerVison

Connected components correctness fixes

• Python connectedComponents(algorithm="graphx") no longer stops after 31 iterations: the max_iter default was written 2 ^ 31 - 2 (XOR, not power), so graphs needing more than 31 supersteps silently returned split components
• randomized_contraction now maps its output back to non-integral (e.g. string) vertex IDs when useLabelsAsComponents = false, instead of failing with CAST_INVALID_INPUT under ANSI or returning wrong results
• randomized_contraction cleans up its checkpoint directory when it fails, not only on success
• KCore and randomized_contraction register their temp functions with qualified identifiers, fixing Function identifier must be fully qualified on Spark 4.2

What's Changed

• fix: point the CC benchmark row at randomized_contraction by @james-willis in #891
• fix: map randomized_contraction output back to non-integral vertex IDs by @james-willis in #893
• fix: clean randomized contraction checkpoints on failure by @SauronShepherd in #895
• ci: run python lint on tests/ and docs/, as its own job by @james-willis in #896
• doc: Pregel Tutorial by @rjurney in #809
• fix: use qualified function identifiers for temp functions by @ledanieln in #900
• fix: connectedComponents max_iter default evaluated to 31 due to ^ precedence by @simplegaurav in #902
• docs: sync connected components docs with the code by @simplegaurav in #903
• docs: randomized_contraction always needs a checkpoint directory by @yuchaoran2011 in #909

Full Changelog: v0.12.2...v0.12.3

Contributors

Connected components performance & Spark Connect fixes

• The randomized_contraction connected-components algorithm now benchmarks better than two_phase with AQE and uses roughly half the memory
• pageRank no longer raises an AttributeError on Spark Connect
• Tutorial dependencies are published as PEP 621 extras, installable with pip install graphframes-py[tutorials]
• click moved to the main dependencies so the graphframes CLI works without the tutorials extra

What's Changed

• chore(deps-dev): bump setuptools from 80.9.0 to 83.0.0 in /python by @dependabot[bot] in #875
• Fix: move click to main dependencies and lazy-import tutorials in console.py by @rjurney with @Copilot in #881
• Fix: restore the broken star history chart by @PingouinFerreux in #883
• fix: publish tutorials dependencies as PEP 621 extras by @james-willis in #885
• feat: GQL-1 -- update py7zr by @SemyonSinchenko in #867
• chore: add test for 873 by @SemyonSinchenko in #886
• fix: resolve AttributeError in pageRank on Spark Connect (issue #889) by @MCBoarder289 in #890
• perf: better wcc by @SemyonSinchenko in #888

New Contributors

• @rjurney with @Copilot made their first contribution in #881
• @PingouinFerreux made their first contribution in #883
• @MCBoarder289 made their first contribution in #890

Full Changelog: v0.12.1...v0.12.2

Contributors

PySpark Client fixes & compatibility

• py4j imports were relocated
• small fixes across API

What's Changed

• fix: doc build by @SemyonSinchenko in #860
• feat: support of pyspark-client by @SemyonSinchenko in #863
• ci: add codespell spell check and extend python linting to dev scripts by @james-willis in #864

Full Changelog: v0.12.0...v0.12.1

Contributors

What's Changed

• fix: revert upload-pages-artifact to v4 in docs workflow by @james-willis in #820
• doc: sync docs with code by @SemyonSinchenko in #822
• chore: fix docs build by @SemyonSinchenko in #821
• docs: add Wherobots CTA to 0.11.0 release post by @james-willis in #823
• add reversed implementation to Pregel API by @slavlotski in #813
• chore(deps-dev): bump pytest from 8.4.1 to 9.0.3 in /python by @dependabot[bot] in #826
• Potential fix for code scanning alert no. 1: Workflow does not contain permissions by @SemyonSinchenko in #832
• Potential fix for code scanning alert no. 2: Workflow does not contain permissions by @SemyonSinchenko in #831
• Potential fix for code scanning alert no. 3: Workflow does not contain permissions by @SemyonSinchenko in #830
• chore(deps-dev): bump urllib3 from 2.6.3 to 2.7.0 in /python by @dependabot[bot] in #837
• chore(deps-dev): bump idna from 3.10 to 3.15 in /python by @dependabot[bot] in #838
• chore(deps-dev): bump pyarrow from 20.0.0 to 23.0.1 in /python by @dependabot[bot] in #844
• Add Spark CBO config tips for boosting motif finding performance by @WeichenXu123 in #845
• Optimization: Add pruning for connected component 2 phase algorithm by @WeichenXu123 in #846
• chore: README update by @SemyonSinchenko in #850
• chore: sitemap generation by @SemyonSinchenko in #852
• chore: sbt-eclipse plugin by @SemyonSinchenko in #848
• feat: add NeighborhoodAwareCDLP community detection algorithm by @SemyonSinchenko in #825
• feat: all paths between two nodes by @SemyonSinchenko in #828
• fix: use git modification time as docs mtime by @SemyonSinchenko in #855
• feat: pregel memory optimizations by @SemyonSinchenko in #847
• feat: HyperANF implementation by @SemyonSinchenko in #841
• feat: 012 release blogpost by @SemyonSinchenko in #857

New Contributors

• @slavlotski made their first contribution in #813

Full Changelog: v0.11.0...v0.12.0

Contributors

What's Changed

• doc: add Maven Central coordinates section to installation guide by @SemyonSinchenko in #774
• fix: Support string IDs in GraphFrame powerIterationClustering by @SemyonSinchenko in #773
• fix: fix links in docs by @SemyonSinchenko in #777
• feat: more tests for fromEdges by @SemyonSinchenko in #778
• fix: allow chaining with fixed-length patterns by @goungoun in #771
• feat: update SVDPP docs and code by @SemyonSinchenko in #779
• feat: Randomized Contraction CC by @SemyonSinchenko in #776
• fix: allow undirected fixed length pattern by @goungoun in #781
• fix: anonymous vertex suppot for variable length pattern by @goungoun in #783
• feat: run tests agains spark 4.1 by @SemyonSinchenko in #787
• feat: remove numpy and nose by @SemyonSinchenko in #788
• feat: add approximate triangle counting algorithm using DataSketches by @SemyonSinchenko in #789
• Update README.md by @Yorko in #793
• chore(deps-dev): bump protobuf from 5.29.5 to 5.29.6 in /python by @dependabot[bot] in #794
• chore: switch to LDBC parquets for benchmarks + refactoring of them by @SemyonSinchenko in #799
• feat: pyspark property graphs by @SemyonSinchenko in #798
• docs: add AGENTS.md for AI agent development guidance by @james-willis in #800
• fix(kcore): swap sendMsgToSrc/sendMsgToDst column references by @DavidGi in #802
• feat: random walks and embeddings by @SemyonSinchenko in #752
• feat: refactor ConnectedComponents API by @SemyonSinchenko in #803
• feat(pregel): automatically skip second join when dst columns not needed by @james-willis in #795
• chore: Fix typos in doc/05-blog/998-graphframes-010-release by @juanli16 in #811
• chore(deps-dev): bump requests from 2.32.4 to 2.33.0 in /python by @dependabot[bot] in #812
• feat: aggregate nbrs api by @SemyonSinchenko in #792
• chore(deps-dev): bump pygments from 2.19.2 to 2.20.0 in /python by @dependabot[bot] in #815
• fix: use trim instead f strip by @SemyonSinchenko in #816
• fix: Java 8 compatibility for KMinSampling and benchmarks by @james-willis in #817
• docs: add 0.11.0 release blog post by @james-willis in #814
• chore: bump versions for publishing to latest by @SemyonSinchenko in #818
• docs: add future work section to 0.11.0 release post by @james-willis in #819

New Contributors

• @Yorko made their first contribution in #793
• @DavidGi made their first contribution in #802
• @juanli16 made their first contribution in #811

Full Changelog: v0.10.1...v0.11.0

Contributors

What's Changed

• chore(deps-dev): bump brotli from 1.1.0 to 1.2.0 in /python by @dependabot[bot] in #751
• chore(deps-dev): bump urllib3 from 2.5.0 to 2.6.0 in /python by @dependabot[bot] in #756
• add more LDBC benchmark datasets by @james-willis in #764
• feat(pregel): add requiredSrcColumns/requiredDstColumns API for triplet memory optimization by @joelrobin18 in #763
• fix: mixing directed and undirected edges in motif throws parse error by @goungoun in #754
• Fix: Throw IllegalArgumentException for null IDs/Src/Dst in toGraphX (Fixes #765) by @JeganVG in #766
• chore(deps-dev): bump urllib3 from 2.6.0 to 2.6.3 in /python by @dependabot[bot] in #767
• Update README.md to fix broken network motif tutorial link by @russelljurney-upside in #770
• revert ConnectedComponents back to 0.9.3 version to fix perf regression by @james-willis in #772

New Contributors

• @JeganVG made their first contribution in #766
• @russelljurney-upside made their first contribution in #770

Full Changelog: v0.10.0...v0.10.1

Contributors

What's Changed

• feat: initial commit with a copy of GraphX code by @SemyonSinchenko in #680
• chore: update image paths in README files to new directory location by @SemyonSinchenko in #683
• feat: performance improvement in GraphX CDLP by @SemyonSinchenko in #681
• feat: fix memory leaks in GraphX by @SemyonSinchenko in #687
• feat: improvements in GF-based algorithms performance by @SemyonSinchenko in #690
• feat: add LDBC Connected Components example by @SemyonSinchenko in #692
• hotfix: java 8 did not have Path.of by @SemyonSinchenko in #694
• feat: variable length pattern matching by @goungoun in #685
• doc: update docs by @SemyonSinchenko in #699
• feat: performance and API sync by @SemyonSinchenko in #698
• feat: add validate method by @SemyonSinchenko in #695
• feat: add Rocha–Thatte cycle detection algorithm by @SemyonSinchenko in #700
• feat: aggregatemessages multiple columns by @SemyonSinchenko in #701
• chore: update Dockerfile by @SemyonSinchenko in #702
• feat: change default branch to "main" by @SemyonSinchenko in #703
• feat: add a test for pagerank by @SemyonSinchenko in #707
• feat: method to transform graph to undirected by @SemyonSinchenko in #705
• feat: simplified implementation of triplets by @SemyonSinchenko in #708
• feat: better TriangleCount by @SemyonSinchenko in #704
• doc: Documentation on setting up the environment by @joelrobin18 in #719
• chore: Unpersist cached data from the test suite by @joelrobin18 in #720
• feat: refactoring of python && docs && sync pyspark with core by @SemyonSinchenko in #711
• feat: add bidirectional motif pattern by @goungoun in #717
• feat: add undirected edge motif pattern by @goungoun in #712
• Improve result formatting for variable length pattern and undirected pattern by @goungoun in #728
• bug: prevent negated bidirectional edge by @goungoun in #739
• Add Graphframe Type Degree Methods by @joelrobin18 in #742
• fix: make py4j pyspark api robust by @SemyonSinchenko in #743
• feat: K-Core by @SemyonSinchenko in #740
• feat: reduce memory consumption of cycles detection by @SemyonSinchenko in #731
• feat: handle edges directions in SP by @SemyonSinchenko in #737
• feat: maximal independent set by @SemyonSinchenko in #744
• feat: prepare the release 0.10.0 by @SemyonSinchenko in #725

New Contributors

• @goungoun made their first contribution in #685
• @joelrobin18 made their first contribution in #719

Full Changelog: v0.9.3...v0.10.0

Contributors

What's Changed

• bug: Don't publish root project by @Kimahriman in #661
• feat: add tests for the case from #375 by @SemyonSinchenko in #663
• feat: localCheckpoints + docs by @SemyonSinchenko in #662
• feat: replace UDF by built-in function in skewedJoin by @SemyonSinchenko in #666
• feat: initial top-level structure of PropertyGraph by @SemyonSinchenko in #613
• fix: update javadocs to fix the build by @SemyonSinchenko in #670
• feat: S-sized benchmarks by @SemyonSinchenko in #671
• feat: updates in SBT and new site engine by @SemyonSinchenko in #673
• hotfix: fix broken links in docs by @SemyonSinchenko in #675
• feat: prepare 0.9.3 release by @SemyonSinchenko in #677

Full Changelog: v0.9.2...v0.9.3

Contributors

Overview

• fix publishing issues
• fix docs publishing issues

What's Changed

• fix: publish connect sources and javadocs by @SemyonSinchenko in #654
• feat: fix docs path by @SemyonSinchenko in #656
• feat: 0.9.2 release by @SemyonSinchenko in #658

Full Changelog: v0.9.1...v0.9.2

Contributors

Overview

Mostly this release is about publishing fixes (version string, README, PySpark Classic dependencies, etc.). There are no changes in core or API.

What's Changed

• Fix versioning by @Kimahriman in #641
• bug: Let classic python version work without grpc deps by @Kimahriman in #647
• fix: resolve gf-connect is not published by @SemyonSinchenko in #645
• feat: prepare the 0.9.1 release by @SemyonSinchenko in #650

Full Changelog: v0.9.0...v0.9.1

Contributors