GeezerVison is showing github.com — Include a caching document provider implementation by default · Issue #4107 · graphql-java/graphql-java
Include a caching document provider implementation by default #4107DescriptionCaching the parsing and validation of a document can save significant latency.
For very large operations, and tools like Relay produce very large operations, the parse and validate latency can be upwards of 15-20ms (p99)
By caching this step you can amortize the cost over many requests and hence cut nearly all of this latency out.
The graphql.execution.preparsed.PreparsedDocumentProvider is the interface that allows you to build a caching document mechanism but out of the box graphql-java does not provide a working implementation
I propose that we
- Create a
public class CachingDocumentProvider implements PreparsedDocumentProvider
- That is used Caffeine as the cache
- Provide a GraphQLUnusualConfiguration mechanism to to tune the cache
- Ship with this TURN on by default (in 2 versions)
- Shade in https://github.com/ben-manes/caffeine as the cache implementation
The implication here is that https://github.com/ben-manes/caffeine would become a dependency (shaded) and we have mostly aimed for low to zero dependencies but I think the juice is worth the squeeze here. All operations will run faster with a cache in place.
I dont know right now of the implications for library frameworks such as Spring GraphQL. Do they have one themselves, is it always on? Hence this issue to stand as a discussion mechanism Activitybbakerman commented on Sep 12, 2025 com.github.ben-manes.caffeine:caffeine:3.1.8
com.google.errorprone:error_prone_annotations:2.21.1
org.checkerframework:checker-qual:3.37.0
Caffeine has the following dependencies should we choose to include it as a dep andimarek commented on Sep 12, 2025 What is it specifically from caffeine that we would leverage?
Could we not have a default implementation that ist just a hashmap and people could adapt caffeine to it themself?
Other alternatives:
-
Release it as extra maven artifact
-
multi module jar (no idea how/if that would work)
-
have a runtime reflection detection like spring that leverages caffeine if in the classpath
bclozel commented on Sep 12, 2025 What are the typical challenges for implementing such an interface?
I noticed one implementation in DGS and it doesn't look too extensive. I'm wondering if shading (extra jar size, extra maintenance work for the team) is the best solution, since applications might want to use their custom version of caffeine, with custom options and maybe share the allocated memory with other bits of the application?
Maybe a sample implementation in the reference docs and more guidance around persisted queries would be more valuable to the community. bbakerman commented on Sep 14, 2025
What is it specifically from caffeine that we would leverage?
Could we not have a default implementation that ist just a hashmap and people could adapt caffeine to it themself?
The reason for Caffeine is that we can then release graphql-java (eventually) with this caching turned on. This will make ALL graphql-java implementations faster - a lot laster - 5 / 10/ 20 ms faster
We cant ship with a HashMap because of its unbounded nature. There must be a LRU type cache in place so we dont by default blow memory limits. And this is what leads us to Caffeine. Its a world class cache, it can efficiently evict entries and hence means we can have this on by default but be sensible in memory usage
Other alternatives:
Release it as extra maven artifact
Yes considered that but then its can be on by default. Hence the 5 / 10/ 20 ms faster savings are not on by default.
multi module jar (no idea how/if that would work)
No idea either
have a runtime reflection detection like spring that leverages caffeine if in the classpath
This is not crazy however I would like to put the configuration of the cache into the graphql.GraphQLUnusualConfiguration and hence expose direct API level for configuring the cache
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
.maximumSize(10_000)
.expireAfterWrite(Duration.ofMinutes(5))
.refreshAfterWrite(Duration.ofMinutes(1))
.build(key -> createExpensiveGraph(key));
but perhaps we can look more info this. eg becomes a NO op if there if Caffeine is not present. bbakerman commented on Sep 21, 2025 Benchmark (queryCount) (querySize) Mode Cnt Score Error Units
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 10 50 avgt 6 0.922 ± 0.024 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 10 500 avgt 6 8.414 ± 0.525 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 10 5000 avgt 6 31.682 ± 0.284 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 50 50 avgt 6 4.662 ± 0.357 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 50 500 avgt 6 41.018 ± 0.626 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 50 5000 avgt 6 154.708 ± 6.785 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 500 50 avgt 6 47.500 ± 3.699 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 500 500 avgt 6 414.916 ± 9.643 ms/op
CachingDocumentBenchmark.benchMarkCachingOffAvgTime 500 5000 avgt 6 1623.578 ± 113.649 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 10 50 avgt 6 0.090 ± 0.005 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 10 500 avgt 6 0.832 ± 0.060 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 10 5000 avgt 6 0.206 ± 0.001 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 50 50 avgt 6 0.451 ± 0.021 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 50 500 avgt 6 4.075 ± 0.038 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 50 5000 avgt 6 0.390 ± 0.008 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 500 50 avgt 6 4.420 ± 0.021 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 500 500 avgt 6 41.472 ± 2.652 ms/op
CachingDocumentBenchmark.benchMarkCachingOnAvgTime 500 5000 avgt 6 2.505 ± 0.200 ms/op
This is some early benchmarking from having the caching in place. As you can see the larger the query the great the payback. This is expected since the time for parsing text is directly correlated with the size of the text.
So caching query parsing and validation pays off as queries become larger and with tools like Relay "computer generating queries" they can get very large easily. paulbakker commented on Oct 6, 2025 Hey folks, I think frameworks (e.g., Spring/DGS) should provide this instead.
It's easy to provide for a framework, and that way graphql-java doesn't need to pull in a new external dependency.
One of the strengths of GraphQL-Java is that it's a foundational library without many dependencies, configurations, or opinions. Adding caching by default gets into a gray area without much benefit because frameworks can already do this anyway.
That being said, I agree on the benefits of having a cache; we're seeing this in practice as well. github-actions commented on Dec 6, 2025 Hello, this issue has been inactive for 60 days, so we're marking it as stale. If you would like to continue this discussion, please comment within the next 30 days or we'll close the issue.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|
