GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add 128-bit vector support by The-Alchemist · Pull Request #80 · simdjson/simdjson-java

Conversation

@The-Alchemist

@The-Alchemist The-Alchemist commented Sep 17, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Summary

  • Add org.simdjson.species=128 and accept 128-bit preferred vectors so parsing works on Apple Silicon and similar hardware (tested on M4)
  • Implement StructuralIndexer.index128() using four 16-byte lanes per 64-byte block (same layout as simdjson ARM64 simd8x64 stage-1).
  • Add test128 to tests

Feedback welcome!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@The-Alchemist The-Alchemist changed the title Add 128-bit vector support for stage-1 indexing Add 128-bit vector support Sep 17, 2026
@The-Alchemist
The-Alchemist marked this pull request as ready for review September 17, 2026 21:50
@The-Alchemist
Add 128-bit vector support for stage-1 structural indexing
afee2c3 
Platforms such as Apple Silicon use 128-bit ByteVector as the preferred
species; rejecting that width broke ./gradlew test and made forced 256-bit
runs very slow. Implement index128 with four 16-byte lanes per 64-byte block,
aligned with simdjson ARM64 stage-1 (simd8x64).

Add test128 to CI and check, and wire species-specific test tasks to the test
classpath so test128/test256/test512 run the suite with the correct JVM flag.

Signed-off-by: The-Alchemist <[email protected]>
@The-Alchemist
The-Alchemist force-pushed the feature/128bit-support branch from 42ec31d to afee2c3 Compare September 17, 2026 21:51
@piotrrzysko

piotrrzysko commented Sep 20, 2026

Copy link
Copy Markdown
Member

Thanks! Have you benchmarked this? There have been attempts to add 128-bit support, but the performance wasn’t great:

  • http://github.com/simdjson/simdjson-java/pull/41
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@The-Alchemist

The-Alchemist commented Sep 20, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

i didn't know about #41 , thanks for pointing it out, @piotrrzysko .

All this is on a Apple M4 Max. I can try on AWS Graviton too.

Benchmark Results (GraalVM, Apple M4 Max)

  • openjdk version "25.0.4.1" 2026-08-18
    • OpenJDK Runtime Environment GraalVM CE 25.3.4.1-dev+1.1 (build 25.0.4.1+1-jvmci-25.3-b22)
    • OpenJDK 64-Bit Server VM GraalVM CE 25.3.4.1-dev+1.1 (build 25.0.4.1+1-jvmci-25.3-b22, mixed mode, sharing)
Benchmark                                                                              Mode  Cnt     Score     Error  Units
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_fastjson        thrpt    5  2565.538 ±  75.186  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jackson         thrpt    5  1571.695 ± 120.597  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jsoniter_scala  thrpt    5  3272.079 ±  41.738  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjson        thrpt    5  3977.643 ± 617.107  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjsonPadded  thrpt    5  4511.874 ± 256.134  ops/s

Benchmark Results (OpenJDK25, Apple M4 Max)

  • openjdk version "25.0.4.1" 2026-08-18
    • OpenJDK Runtime Environment GraalVM CE 25.3.4.1-dev+1.1 (build 25.0.4.1+1-jvmci-25.3-b22)
    • OpenJDK 64-Bit Server VM GraalVM CE 25.3.4.1-dev+1.1 (build 25.0.4.1+1-jvmci-25.3-b22, mixed mode, sharing)
Benchmark                                                                              Mode  Cnt     Score     Error  Units
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_fastjson        thrpt    5  2240.564 ±  56.163  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jackson         thrpt    5  1696.094 ±  21.949  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jsoniter_scala  thrpt    5  3211.085 ±  25.532  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjson        thrpt    5  2996.914 ± 221.540  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjsonPadded  thrpt    5  3138.239 ±  34.415  ops/s

Is it worth adding ARM64 in the CI build matrix? I can submit a separate PR.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@The-Alchemist

The-Alchemist commented Sep 21, 2026

Copy link
Copy Markdown
Author

got some numbers on AWS Graviton: t4g.large, ARM Neoverse-N1 (2 vCPUs, 1 socket, 2 cores)

  • openjdk version "25.0.4.1" 2026-08-18 LTS
    • OpenJDK Runtime Environment Corretto-25.0.4.8.1 (build 25.0.4.1+8-LTS)
    • OpenJDK 64-Bit Server VM Corretto-25.0.4.8.1 (build 25.0.4.1+8-LTS, mixed mode, sharing)
Benchmark                                                                              Mode  Cnt    Score    Error  Units
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_fastjson        thrpt    5  606.570 ± 24.441  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jackson         thrpt    5  462.214 ± 53.461  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jsoniter_scala  thrpt    5  862.968 ± 29.667  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjson        thrpt    5  732.215 ± 34.920  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjsonPadded  thrpt    5  752.215 ± 25.292  ops/s
  • openjdk version "25.0.4.1" 2026-08-18
    • OpenJDK Runtime Environment GraalVM CE 25.3.4.1+1.1 (build 25.0.4.1+1-jvmci-25.3-b22)
    • OpenJDK 64-Bit Server VM GraalVM CE 25.3.4.1+1.1 (build 25.0.4.1+1-jvmci-25.3-b22, mixed mode, sharing)
Benchmark                                                                              Mode  Cnt    Score     Error  Units
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_fastjson        thrpt    5  571.661 ± 465.161  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jackson         thrpt    5  393.593 ±  24.094  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jsoniter_scala  thrpt    5  782.638 ±  46.143  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjson        thrpt    5  888.635 ±  63.897  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjsonPadded  thrpt    5  916.457 ± 115.962  ops/s

I'll try Graviton3 next. Some quick LLM analysis says our solution is not optimal for Graviton's particular SIMD implementation.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@The-Alchemist

The-Alchemist commented Sep 21, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

Graviton 4. m8g.large

  • openjdk version "25.0.4.1" 2026-08-18
    • OpenJDK Runtime Environment GraalVM CE 25.3.4.1+1.1 (build 25.0.4.1+1-jvmci-25.3-b22)
    • OpenJDK 64-Bit Server VM GraalVM CE 25.3.4.1+1.1 (build 25.0.4.1+1-jvmci-25.3-b22, mixed mode, sharing)
Benchmark                                                                              Mode  Cnt     Score     Error  Units
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_fastjson        thrpt    5  1544.730 ±  14.999  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jackson         thrpt    5   862.859 ±  12.850  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_jsoniter_scala  thrpt    5  1712.707 ±  30.936  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjson        thrpt    5  1960.092 ± 272.553  ops/s
SchemaBasedParseAndSelectBenchmark.countUniqueUsersWithDefaultProfile_simdjsonPadded  thrpt    5  2002.614 ± 218.537  ops/s
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@The-Alchemist @piotrrzysko
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.