Why Build gRPC Directly on Reactor Netty
I recently started building grpc-reactor: an experimental gRPC implementation built directly on Reactor Netty HTTP/2. The project uses Protobuf but has zero runtime dependency on grpc-java transport, `ClientCall`, `ServerCall`, or `StreamObserver`.
This is not about proving grpc-java is bad. grpc-java is mature, stable, and covers load balancing, NameResolver, retries, and rich observability. This project answers a different question: if your programming model is already Reactor, can `Mono` and `Flux` flow all the way from the generated API down to the HTTP/2 stream, without adapting between two async abstractions?
This post is based on JDK 25, Gradle 9.2.1, Reactor 3.8.6, Reactor Netty 1.3.6, and Netty 4.2.15.Final. The `main` branch has completed Stage 0 through Stage 9: beyond protocol, four RPC cardinalities, production transport, DNS, codegen, and standard services, Stage 9 adds client/server interceptors, non-blocking call observers, optional bounded binary logging, canonical interop smoke tests, deterministic parser fuzz, and connection/cancellation churn regression.
## The API Goal Is Not Wrapping StreamObserver
Protobuf methods have four cardinalities. The target API maps them directly:
Protobuf Method | Reactor Signature
---|---
unary | `Mono<Resp> method(Mono<Req>)`
server streaming | `Flux<Resp> method(Mono<Req>)`
client streaming | `Mono<Resp> method(Flux<Req>)`
bidirectional streaming | `Flux<Resp> method(Flux<Req>)`
Why does gRPC define four instead of just unary? It's essentially a 2x2 combination where request and response each independently choose "single value or stream":
| Single Response | Streaming Response
---|---|---
**Single Request** | unary | server streaming
**Streaming Request** | client streaming | bidirectional
They solve different scenarios: unary covers classic request-response; server streaming handles server push (event subscriptions, large paginated pulls); client streaming handles bulk uploads (file chunks, batch writes); bidirectional streaming handles real-time two-way communication (chat, collaborative editing). These aren't invented patterns — HTTP/2 streams are inherently full-duplex. A single connection can multiplex hundreds of concurrent streams, with both request and response sending multiple data frames independently. gRPC elevates this transport capability to first-class API semantics: not a variant of chunked transfer encoding, but compiler-level type checking that callers and implementers match.
The low-level transport retains a single unified model:
Flux<Req> -> HTTP/2 stream -> Flux<Resp>
Generated code applies cardinality checks like `single()` at the boundary. This avoids implementing four separate network logic paths for four RPC types, while explicitly rejecting empty requests, duplicate requests, or duplicate responses in unary calls.
## Compatibility Targets the Wire Protocol
The project's compatibility target is the public gRPC over HTTP/2 protocol spec, not grpc-java internal APIs:
generated Reactor API
|
call dispatcher / service registry
|
marshaller + message framer/deframer
|
metadata + status + deadline
|
Reactor Netty HTTP/2 stream
Each RPC corresponds to one HTTP/2 stream. Requests start with a HEADERS frame containing at minimum:
:method POST
:path /<package.Service>/<Method>
content-type application/grpc+proto
te trailers
grpc-timeout <relative timeout, e.g. 100m for 100ms>
Message bodies use Length-Prefixed-Message format encapsulated in DATA frames — each Protobuf message is preceded by a five-byte envelope (1 byte compression flag + 4 bytes big-endian length). A single DATA frame may contain multiple gRPC messages, and a large message may span multiple DATA frames.
**Why does gRPC require HTTP trailers?** This is one of the protocol's most counterintuitive designs. HTTP status codes are nearly useless for gRPC — the protocol requires the HTTP layer to always return 200, with the actual call result (`grpc-status` and `grpc-message`) placed in trailers. The reason: in streaming scenarios, the server may have already sent thousands of messages, and whether it ultimately succeeded or failed can only be determined after processing the last piece of data. HTTP headers are sent before the body and cannot carry this posterior result. Trailers are the only mechanism in HTTP/2 that can append metadata after the body.
The protocol also defines **trailers-only** mode: when the server can determine failure before reading the body (e.g., path not found, authentication failed), `grpc-status` is returned directly in response headers without sending a body, saving one round-trip. Clients must check both headers and trailers to correctly extract the final status.
## Modules Split Along Protocol Boundaries
The project currently has eight modules:
grpc-reactor-protocol → Transport-independent protocol primitives
grpc-reactor-transport → Reactor Netty HTTP/2 mapping
grpc-reactor-codegen → Protoc plugin, generates Reactor stubs
grpc-reactor-gradle-plugin → Gradle integration
grpc-reactor-maven-plugin → Maven generate-sources integration
grpc-reactor-services → Optional Health, Reflection & Channelz standard services
grpc-reactor-binlog → Optional, bounded canonical binary logging
grpc-reactor-interop-test → grpc-java compatibility tests
`protocol` handles only transport-independent values and codecs: message framing, metadata, status, timeout, compression, and protobuf marshallers. It depends on Reactor Core and Netty Buffer but not Reactor Netty — meaning the protocol layer can be tested independently without real HTTP/2 connections.
`transport` maps the protocol onto Reactor Netty HTTP/2, handling client, server, service registry, and per-call context.
`codegen` consumes Protobuf `CodeGeneratorRequest` and generates type-safe Reactor client and service binders.
`services` uses the same codegen to generate canonical gRPC service bindings, implementing Health v1, Reflection v1, and Channelz v1 on top of transport's immutable descriptor/diagnostics snapshots. This module requires explicit registration — adding the dependency alone won't expose management endpoints.
`binlog` is also an explicitly-enabled standalone module. It captures canonical binary-log v1 events via a transport interceptor, controlling information exposure and memory limits through metadata/message truncation, sensitive key redaction, and fixed-capacity sinks.
`interop-test` introduces grpc-java in test scope. grpc-java serves as the compatibility oracle here, never entering the project runtime.
## Dependency Selection and Version Pinning
The runtime dependency chain is intentionally kept short:
Dependency | Version | Purpose
---|---|---
Reactor Core | 3.8.6 | Mono/Flux programming model
Reactor Netty | 1.3.6 | HTTP/2 client/server
Netty | 4.2.15 | ByteBuf, HTTP/2 codec
Protobuf-java | 4.35.1 | Message serialization
The project does not introduce Spring, Micrometer, or any DI framework. Tests use JUnit 6.1.2 and Reactor Test's `StepVerifier`. grpc-java 1.82.2 appears only in `interop-test`'s test classpath with zero runtime intrusion.
Versions are centrally managed via `gradle/libs.versions.toml` with Gradle dependency locking generating lock files, ensuring fully reproducible builds across machines.
## Why the Protocol Layer Must Come First
It's tempting to start with "spin up an HTTP/2 Server" — you quickly get an echo demo, but it pushes the truly difficult problems to later:
* DATA may split in the middle of the five-byte gRPC header;
* A single DATA buffer may contain multiple messages;
* Metadata allows duplicate keys, and binary values require Base64;
* Timeout wire values are at most eight digits with a unit suffix;
* Final status comes from trailers;
* ByteBuf must be released on success, failure, and cancellation paths;
* Reactive Streams demand counts messages, HTTP/2 flow control counts bytes.
Therefore the project progresses by Stage: first lock down the build and interop fixtures, then complete the protocol layer, then implement unary transport, streaming, production features, and codegen. Each Stage has executable exit criteria — "classes have been created" is not a completion standard.
## Stage 0: Build Baseline and Interop Fixture
Before writing any protocol code, Stage 0 solves "how to prove code is correct":
**JDK 25 compilation** — The project uses `-Xlint:all -parameters -encoding UTF-8` for strict compilation. All warnings are compilation errors; silent suppression is not allowed. JDK 25 was chosen to validate Netty and Protobuf compatibility on the latest JVM early.
**Spotless formatting** — Unified Eclipse formatter config plus ktlint. `spotlessCheck` is the first gate in CI. This eliminates all code review discussions about formatting.
**interop.proto test fixture** — Defines a test service covering all four RPC cardinalities:
service InteropTestService {
rpc Unary (TestRequest) returns (TestResponse);
rpc ServerStreaming (TestRequest) returns (stream TestResponse);
rpc ClientStreaming (stream TestRequest) returns (TestResponse);
rpc BidirectionalStreaming (stream TestRequest) returns (stream TestResponse);
}
**GrpcJavaFixture** — In the `interop-test` module, a test utility class starts both a grpc-java server and client, providing `start()` / `close()` lifecycle. Through it, bidirectional verification is possible: Reactor client calls grpc-java server, and grpc-java client calls Reactor server. Both use the same `.proto` generated code, ensuring wire compatibility.
**Utilities** :
* `FreePorts`: Allocates independent ports for each test, avoiding parallel test conflicts;
* `TlsTestCertificates`: Pre-generates self-signed certificates for subsequent TLS tests;
* `LeakDetection`: Integrates Netty's `ResourceLeakDetector`, ensuring ByteBuf leaks are immediately exposed in tests.
**Stage 0 exit criteria** : `./gradlew clean test` passes from fresh checkout, CI is green on Linux + Java 25, protoc generation is deterministically reproducible, grpc-java fixture communicates bidirectionally.
## Staged Verification Strategy
The project progresses through 12 Stages, each with clear goals, executable exit criteria, and regression coverage:
Stage | Goal | Key Deliverable
---|---|---
0 | Build baseline | CI, formatting, interop fixture
1 | Protocol foundation | Frame codec, metadata, status, timeout, compression
2 | Unary transport | End-to-end h2c unary call
3 | Server streaming | Multi-message response stream
4 | Full cardinality | Client streaming + bidirectional
5 | Production transport | TLS, gzip, deadline, GOAWAY, connection pool, keepalive
6 | Name resolution | DNS, subchannel, pick_first, round_robin
7 | Codegen & build integration | Protoc plugin, descriptor registry, Gradle/Maven plugins
8 | Standard services | Health v1, Reflection v1, Channelz v1
9 | Operations & hardening | Interceptor, observer, binlog, canonical smoke, fuzz/churn
10+ | Optional protocol extensions | Load balancing/reporting protocols & Channelz v2
Each Stage completion requires: `./gradlew clean spotlessCheck test --no-daemon` passes in full, and all prior Stage tests continue running as regression. This means when Stage 4 completes, Stage 2's unary tests are still green.
## Current Verification Results
The project uses this unified gate that simultaneously checks formatting, compilation, protocol tests, transport tests, and grpc-java interop tests:
./gradlew clean spotlessCheck test --no-daemon
This command passes as of this writing. The Stage 9 gate currently includes 122 actionable tasks; the dependency baseline is Protobuf 4.35.1, grpc-java 1.82.2, and JUnit 6.1.2. Under JDK 25, you'll still see Protobuf `Unsafe`, Netty/Gradle native access, and Gradle deprecated feature warnings — they don't affect test results but need ongoing tracking through future JDK and Gradle upgrades.
Subsequent posts will continue discussing the five-byte gRPC message envelope, ByteBuf ownership, four RPC cardinalities, production transport, DNS, code generation, standard management services, and how to integrate interceptors, observation, and binary logging without breaking Reactive Streams semantics.
