Gradle Under the Hood: ClassLoader Harvesting vs. Network CI Crashes
In the first article of our _Gradle Under the Hood_ series, we explored how we decoupled incremental build tracking from volatile compiler outputs to achieve bulletproof caching.
Today, we dive into a critical continuous integration (CI) dilemma that almost every plugin author faces: **How do you execute complex, isolated task workloads on headless CI nodes without crashing due to missing network dependencies or restricted local caches?**
Let’s unpack how we engineered the **JPersist Suite** verification engines to harvest their own running binary classpaths on-the-fly, how that architectural assumptions shattered across older Gradle versions, and how we leveraged native classpath streaming to achieve bulletproof reliability in our latest `v1.8.1` patch.
### The Headless CI Dilemma
To protect the Gradle build daemon from memory leaks and classpath contamination, advanced automation tasks must run inside a sandbox. Gradle provides the magnificent **Worker API** , allowing you to execute tasks via isolated classloaders (`classLoaderIsolation`):
workerExecutor.classLoaderIsolation { workerSpec ->
workerSpec.classpath.from(toolsetFiles)
}.submit(MyValidationWorker) { params -> ... }
**Where it breaks on CI runners:**
To populate `toolsetFiles`, a plugin typically declares an internal detached configuration and injects remote text coordinates (GAV format):
project.dependencies.add(detachedConfigName, "io.github.jpersist.tools:persistence-tools-api:3.0.0")
When running in a standard developer workspace, this works fine. But in a highly isolated, enterprise continuous integration runner (or mono-repository workspaces where individual component segments are built independently), this text-based lookup introduces massive fragility:
1. If the required companion library artifacts have not been officially merged and pushed to Maven Central yet, the resolution crashes with a `ModuleNotFoundException`.
2. If your CI pipelines restrict or completely disable access to `mavenLocal()`, sibling projects cannot share local intermediate build caches.
3. Forcing remote dependency resolution loops over internal libraries during a task execution step slows down your CI velocity.
### The First Iteration: ClassLoader Harvesting (v1.8.0)
While building **JPersist v1.8.0** , we asked ourselves: _Why are we forcing Gradle to perform a slow, network-bound dependency lookup against Maven Central for library classes that the plugin itself is already running on?_
Since your primary plugin (e.g., `hibernate-persistence-gradle-plugin`) already declares a dependency on the core tools API module to compile, those physical classes **are already loaded into memory by the Plugin’s ClassLoader**.
Instead of using text coordinates, we introduced **ClassLoader Harvesting**. The plugin programmatically queried its own running runtime environment to discover the exact physical `.jar` file path on disk hosting the core contracts, and fed those file handles straight to the isolated Worker via protection domain lookups:
// Our initial v1.8.0 lookups implementation
Class<?> providerClass = Class.forName(markerClassName, false, classLoader)
def codeSourceLocation = providerClass.protectionDomain.codeSource?.location
This successfully generated a clean local file collection array, completely insulating the task action from network overhead.
### The Plot Twist: Cross-Version Version Alignment
We rolled out this design, and our local test suites under modern Gradle 9.x were glowing green. However, real-world integration testing against older long-term support (LTS) streams—specifically **Gradle 7.6.6 and Gradle 8.7** —unveiled subtle API signature and dependency lifecycle discrepancies that we hadn't anticipated.
1. **Lazy vs. Eager Extensions:** Under Gradle 9.x, the `Configuration.extendsFrom()` method natively unwraps lazy `NamedDomainObjectProvider` targets. In Gradle 8.x, passing a provider directly threw a `MissingMethodExecutionFailed` exception because the legacy eager API strictly expected a realized `Configuration` instance.
2. **The Transitive Sandbox Leak:** In mixed multi-project environments (where one project sub-module runs a legacy Hibernate 5 stack and another runs a modern Hibernate 6 stack), Gradle 7.x's older configuration engine aggressively elevated the dependency graph inside detached file scopes. Hibernate 6 classes accidentally leaked into our Hibernate 5 worker sandbox, stripping away legacy methods and breaking tasks with a critical `NoSuchMethodError`.
### The v1.8.1 Evolution: Native Classpath Streaming
To guarantee 100% universal stability across Gradle 7.6.6, 8.7, and 9.x without hardcoding user-space definitions, we evolved our **ClassLoader Harvesting** implementation into a more robust two-tier pattern.
First, we isolated the version lookup itself. Instead of scraping runtime class paths on the fly, the plugin now references a lean `persistence-plugin-info.properties` file generated incrementally at build time. This reads the exact published coordinate version out of a detached configuration, resolving smoothly without leaking transitive scopes:
static ConfigurableFileCollection resolvePersistenceToolsFiles(Project project, Provider<String> jpaVersionProvider) {
def version = jpaVersionProvider.get()
boolean isLegacy = isLegacyJpaVersion(version)
String providerArtifact = isLegacy ? "persistence-tools-legacy" : "persistence-tools-jakarta"
String toolsVersion = "3.0.0" // Baseline safe fallback
def props = new Properties()
def resourceStream = JakartaPersistencePlugin.class.classLoader.getResourceAsStream("persistence-plugin-info.properties")
if (resourceStream != null) {
resourceStream.withCloseable { props.load(it) }
toolsVersion = props.getProperty("persistence-tools.version").trim()
}
def detachedConfig = project.configurations.detachedConfiguration(
project.dependencies.create("io.github.jpersist.tools:providerArtifact:{toolsVersion}")
)
return project.files(detachedConfig)
}
Second, we paired this with **Native Classpath Streaming**. We bypassed raw, unchecked file collections entirely and mapped the task's execution graph straight to the project's native compilation classpath provider.
By feeding `sourceSet.compileClasspath` directly into the task's configuration container, Gradle's native resolution engine handles conflicts, dependency trees, and caching boundaries naturally at execution time while remaining completely backward compatible:
// Inside HibernateEnhancementPlugin.groovy (v1.8.1)
def enhanceConfigName = sourceSet.name == 'main' ? "hibernateEnhanceClasspath" : "\${sourceSet.name}HibernateEnhanceClasspath"
def enhanceConfigProvider = project.configurations.register(enhanceConfigName) { config ->
config.canBeConsumed = false
config.canBeResolved = true
// Eagerly unpack via .get() for Gradle 8.x signature compatibility
config.extendsFrom(project.configurations.named(sourceSet.compileClasspathConfigurationName).get())
}
// Wire the lazy file collections straight into the task configuration graph
enhanceTask.enhanceClasspath.from(enhanceConfigProvider)
enhanceTask.enhanceClasspath.from(JakartaPersistencePlugin.resolvePersistenceToolsFiles(project, activeVersionProvider))
### The Engineering Win
By combining localized classloader resource harvesting with native compilation classpath streaming, we secured three major structural improvements for our users:
1. **100% Offline Independence:** The isolated worker tasks (`validatePersistenceSchema`, `lintNamedQueries`, and `hibernateEnhance`) can run completely offline on headless CI nodes. They don't care about `mavenLocal()` locks or staging delays on remote repositories.
2. **Absolute Lifecycle Parity:** Whether your enterprise application is running on Gradle 7.6.6, Gradle 8.7, or cutting-edge Gradle 9.x matrices, the configuration-time signatures match perfectly without crashing.
3. **Pristine Daemon State:** Tool binaries are contained strictly within the worker's parent context, keeping the global Gradle daemon's main classpath clean and unpolluted.
In **Part 3** of our _Gradle Under the Hood_ series, we will look at **Thin Archives** and discuss why narrowing dynamic dependency propagation to exclusive `compileOnly` scopes is crucial to prevent runtime classpath pollution in enterprise Jakarta EE application servers.
_What are your experiences dealing with classloader leakage or configuration API breaks between different Gradle lifecycles? Let's discuss in the comments below!_
🔗 Check out the JPersist open-source architecture: https://github.com/jpersist/persistence