GeezerVison is showing github.com — Fix TwitterBenchmark which only measured a validation error by Meemaw · Pull Request #4499 · graphql-java/graphql-java
Conversation
Meemaw
commented
Oct 17, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Problem
TwitterBenchmark builds a query that is 150 levels deep. Since e4c287c ("Add query depth and field count limits to Validator") validation enforces QueryComplexityLimits with a default maxDepth of 100, so every benchmark execution fails with:
MaxQueryDepthExceeded: Query depth 101 exceeds maximum allowed depth 100
and returns data == null. Because the benchmark uses a persisted query cache, it has been measuring the return of a cached validation error (~8M ops/s) rather than execution, and those numbers flow into the performance results history.
Fix
- Pass a
QueryComplexityLimits with maxDepth(DEPTH + 1) through the request's GraphQLContext (QueryComplexityLimits.KEY) rather than changing the global default, so other benchmarks in the same JVM are unaffected.
- Execute the query once during class initialisation and fail fast if it returns errors, so the benchmark cannot silently degrade into measuring an error path again.
Verification
- Before: query returns
MaxQueryDepthExceeded, data == null; benchmark reports ~8,090,000 ops/s.
- After: query returns no errors and fully nested data (150 levels, 150 leaves at the bottom); benchmark reports 359 ± 12 ops/s (~2.8 ms per ~22.6k field query, Apple Silicon, JDK 21).
- Temporarily lowering the limit to
DEPTH - 1 makes class initialisation throw ExceptionInInitializerError, confirming the fail-fast check works.
graphql.archunit.* tests pass (JMH fork rule).
Note: the historical TwitterBenchmark numbers recorded since the limits were introduced are not comparable with results after this change.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
The benchmark query is 150 levels deep, but since validation started
enforcing QueryComplexityLimits (default maxDepth=100) every execution
failed with MaxQueryDepthExceeded and returned no data. The benchmark
was therefore measuring a cached validation error (~8M ops/s) rather
than execution.
Raise the max depth for the benchmark requests via the GraphQLContext
and fail fast at class initialisation if the query does not execute
cleanly, so the benchmark cannot silently regress like this again.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Test Report
Test Results
| Java Version |
Total |
Passed |
Failed |
Errors |
Skipped |
| Java 11 |
5962 (-20 🔴) |
5906 (-20 🔴) |
0 (±0) |
0 (±0) |
56 (±0) |
| Java 17 |
5962 (-20 🔴) |
5905 (-20 🔴) |
0 (±0) |
0 (±0) |
57 (±0) |
| Java 21 |
5962 (-20 🔴) |
5905 (-20 🔴) |
0 (±0) |
0 (±0) |
57 (±0) |
| Java 25 |
5962 (-20 🔴) |
5905 (-20 🔴) |
0 (±0) |
0 (±0) |
57 (±0) |
| jcstress |
32 (±0) |
32 (±0) |
0 (±0) |
0 (±0) |
0 (±0) |
| Total |
23880 (-80 🔴) |
23653 (-80 🔴) |
0 (±0) |
0 (±0) |
227 (±0) |
Code Coverage (Java 25)
| Metric |
Covered |
Missed |
Coverage |
vs Master |
| Lines |
30140 |
3118 |
90.6% |
±0.0% |
| Branches |
8838 |
1520 |
85.3% |
±0.0% |
| Methods |
8048 |
1207 |
87.0% |
±0.0% |
Changed Class Coverage (2 classes)
| Class |
Line |
Branch |
Method |
| g.e.ExecutionStrategy |
±0.0% |
-0.8% 🔴 |
±0.0% |
| g.s.i.TypeDefinitionRegistry |
-0.3% 🔴 |
±0.0% |
-1.3% 🔴 |
ExecutionStrategy — method details
| Method |
Line |
Branch |
| lambda$buildFieldValueMap$0 |
100.0% |
90.0% (-10.0% 🔴) |
TypeDefinitionRegistry — method details
| Method |
Line |
Branch |
| getType |
0.0% (-100.0% 🔴) |
— |
Full HTML report: build artifact jacoco-html-report
Updated: 2026-10-19 07:19:35 UTC
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This branch has not been deployed
No deployments
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.