[1/x] feat: add proto/gRPC plugin for structured inter-service communication#211
Open
[1/x] feat: add proto/gRPC plugin for structured inter-service communication#211
Conversation
Add a new first-class appkit plugin that brings protobuf message definitions, gRPC transport, and binary serialization to the SDK. This enables typed data contracts between plugins, external services, and Databricks Jobs pipelines. Toolchain: @bufbuild/protobuf v2 + @connectrpc/connect - Native ESM, tree-shakeable, ~25KB combined runtime - Connect protocol shares Express HTTP port (no envoy proxy) - Optional standalone HTTP/2 for native gRPC clients (Python/Go) What's included: Proto definitions (proto/appkit/v1/): - common.proto: Error, Pagination, Value, Metadata - pipeline.proto: JobResult, DataBatch, DataRow, StageSchema - services.proto: JobDataService, PipelineService RPCs Plugin (packages/appkit/src/plugins/proto/): - ProtoPlugin with full lifecycle integration (interceptors, telemetry, retry, timeout, user context) - GrpcServer: shared mode (Express middleware) + standalone HTTP/2 - GrpcClientFactory: typed client creation with transport selection - ProtoSerializer: binary/JSON ser/de + UC Volume read/write - Build-integrated codegen: buf generate → shared/src/proto/ Build pipeline: - pnpm proto:generate runs buf codegen (Turbo-cached) - Generated TS types in packages/shared/src/proto/ (committed) - biome.json excludes generated files Tests: - Plugin lifecycle, exports surface, route registration - Server service registration, dedup, router mount - Client factory transport selection - Serialization round-trip correctness Signed-off-by: Evgenii Kniazev <[email protected]>
keugenek
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a new first-class appkit plugin that brings protobuf message definitions, gRPC transport, and binary serialization to the SDK. This enables typed data contracts between plugins, external services, and Databricks Jobs pipelines.
Toolchain: @bufbuild/protobuf v2 + @connectrpc/connect
What's included:
Proto definitions (proto/appkit/v1/):
Plugin (packages/appkit/src/plugins/proto/):
Build pipeline:
Tests: